diff options
| author | Jani Nikula <jani.nikula@intel.com> | 2018-11-02 09:00:55 +0200 |
|---|---|---|
| committer | Jani Nikula <jani.nikula@intel.com> | 2018-11-02 09:01:28 +0200 |
| commit | 7a085c3aad94cce7e11031c6800e41668418ae4c (patch) | |
| tree | b3474f8bd622f87ad39227057f9de60959c8bd95 | |
| parent | 18cde299df33ff73908f7e5dcebf321fb79ed36c (diff) | |
| parent | f9885ef875e9160454392f85159163674159c51f (diff) | |
| download | linux-stable-7a085c3aad94cce7e11031c6800e41668418ae4c.tar.gz linux-stable-7a085c3aad94cce7e11031c6800e41668418ae4c.tar.bz2 linux-stable-7a085c3aad94cce7e11031c6800e41668418ae4c.zip | |
Merge drm/drm-next into drm-intel-next-queued
Although there's nothing crucial missing, it's been a long time since
the last backmerge. Catch up with drm-next.
Signed-off-by: Jani Nikula <jani.nikula@intel.com>
1395 files changed, 21309 insertions, 13632 deletions
diff --git a/Documentation/ABI/stable/sysfs-devices-system-xen_memory b/Documentation/ABI/stable/sysfs-devices-system-xen_memory index caa311d59ac1..6d83f95a8a8e 100644 --- a/Documentation/ABI/stable/sysfs-devices-system-xen_memory +++ b/Documentation/ABI/stable/sysfs-devices-system-xen_memory @@ -75,3 +75,12 @@ Contact: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Description: Amount (in KiB) of low (or normal) memory in the balloon. + +What: /sys/devices/system/xen_memory/xen_memory0/scrub_pages +Date: September 2018 +KernelVersion: 4.20 +Contact: xen-devel@lists.xenproject.org +Description: + Control scrubbing pages before returning them to Xen for others domains + use. Can be set with xen_scrub_pages cmdline + parameter. Default value controlled with CONFIG_XEN_SCRUB_PAGES_DEFAULT. diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 9871e649ffef..92eb1f42240d 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -3523,6 +3523,12 @@ ramdisk_size= [RAM] Sizes of RAM disks in kilobytes See Documentation/blockdev/ramdisk.txt. + random.trust_cpu={on,off} + [KNL] Enable or disable trusting the use of the + CPU's random number generator (if available) to + fully seed the kernel's CRNG. Default is controlled + by CONFIG_RANDOM_TRUST_CPU. + ras=option[,option,...] [KNL] RAS-specific options cec_disable [X86] @@ -4994,6 +5000,12 @@ Disables the PV optimizations forcing the HVM guest to run as generic HVM guest with no PV drivers. + xen_scrub_pages= [XEN] + Boolean option to control scrubbing pages before giving them back + to Xen, for use by other domains. Can be also changed at runtime + with /sys/devices/system/xen_memory/xen_memory0/scrub_pages. + Default value controlled with CONFIG_XEN_SCRUB_PAGES_DEFAULT. + xirc2ps_cs= [NET,PCMCIA] Format: <irq>,<irq_mask>,<io>,<full_duplex>,<do_sound>,<lockup_hack>[,<irq2>[,<irq3>[,<irq4>]]] diff --git a/Documentation/device-mapper/dm-raid.txt b/Documentation/device-mapper/dm-raid.txt index 390c145f01d7..52a719b49afd 100644 --- a/Documentation/device-mapper/dm-raid.txt +++ b/Documentation/device-mapper/dm-raid.txt @@ -348,3 +348,7 @@ Version History 1.13.1 Fix deadlock caused by early md_stop_writes(). Also fix size an state races. 1.13.2 Fix raid redundancy validation and avoid keeping raid set frozen +1.14.0 Fix reshape race on small devices. Fix stripe adding reshape + deadlock/potential data corruption. Update superblock when + specific devices are requested via rebuild. Fix RAID leg + rebuild errors. diff --git a/Documentation/devicetree/bindings/display/bridge/renesas,lvds.txt b/Documentation/devicetree/bindings/display/bridge/renesas,lvds.txt index 5a4e379bb414..3aeb0ec06fd0 100644 --- a/Documentation/devicetree/bindings/display/bridge/renesas,lvds.txt +++ b/Documentation/devicetree/bindings/display/bridge/renesas,lvds.txt @@ -15,10 +15,21 @@ Required properties: - "renesas,r8a7796-lvds" for R8A7796 (R-Car M3-W) compatible LVDS encoders - "renesas,r8a77970-lvds" for R8A77970 (R-Car V3M) compatible LVDS encoders - "renesas,r8a77980-lvds" for R8A77980 (R-Car V3H) compatible LVDS encoders + - "renesas,r8a77990-lvds" for R8A77990 (R-Car E3) compatible LVDS encoders - "renesas,r8a77995-lvds" for R8A77995 (R-Car D3) compatible LVDS encoders - reg: Base address and length for the memory-mapped registers -- clocks: A phandle + clock-specifier pair for the functional clock +- clocks: A list of phandles + clock-specifier pairs, one for each entry in + the clock-names property. +- clock-names: Name of the clocks. This property is model-dependent. + - The functional clock, which mandatory for all models, shall be listed + first, and shall be named "fck". + - On R8A77990 and R8A77995, the LVDS encoder can use the EXTAL or + DU_DOTCLKINx clocks. Those clocks are optional. When supplied they must be + named "extal" and "dclkin.x" respectively, with "x" being the DU_DOTCLKIN + numerical index. + - When the clocks property only contains the functional clock, the + clock-names property may be omitted. - resets: A phandle + reset specifier for the module reset Required nodes: diff --git a/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt b/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt index 2fff8b406f4c..be377786e8cd 100644 --- a/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt +++ b/Documentation/devicetree/bindings/display/exynos/exynos_dsim.txt @@ -21,6 +21,9 @@ Required properties: - samsung,pll-clock-frequency: specifies frequency of the oscillator clock - #address-cells, #size-cells: should be set respectively to <1> and <0> according to DSI host bindings (see MIPI DSI bindings [1]) + - samsung,burst-clock-frequency: specifies DSI frequency in high-speed burst + mode + - samsung,esc-clock-frequency: specifies DSI frequency in escape mode Optional properties: - power-domains: a phandle to DSIM power domain node @@ -29,25 +32,9 @@ Child nodes: Should contain DSI peripheral nodes (see MIPI DSI bindings [1]). Video interfaces: - Device node can contain video interface port nodes according to [2]. - The following are properties specific to those nodes: - - port node inbound: - - reg: (required) must be 0. - port node outbound: - - reg: (required) must be 1. - - endpoint node connected from mic node (reg = 0): - - remote-endpoint: specifies the endpoint in mic node. This node is required - for Exynos5433 mipi dsi. So mic can access to panel node - throughout this dsi node. - endpoint node connected to panel node (reg = 1): - - remote-endpoint: specifies the endpoint in panel node. This node is - required in all kinds of exynos mipi dsi to represent - the connection between mipi dsi and panel. - - samsung,burst-clock-frequency: specifies DSI frequency in high-speed burst - mode - - samsung,esc-clock-frequency: specifies DSI frequency in escape mode + Device node can contain following video interface port nodes according to [2]: + 0: RGB input, + 1: DSI output [1]: Documentation/devicetree/bindings/display/mipi-dsi-bus.txt [2]: Documentation/devicetree/bindings/media/video-interfaces.txt diff --git a/Documentation/devicetree/bindings/display/panel/innolux,tv123wam.txt b/Documentation/devicetree/bindings/display/panel/innolux,p120zdg-bf1.txt index a9b35265fa13..513f03466aba 100644 --- a/Documentation/devicetree/bindings/display/panel/innolux,tv123wam.txt +++ b/Documentation/devicetree/bindings/display/panel/innolux,p120zdg-bf1.txt @@ -1,20 +1,22 @@ -Innolux TV123WAM 12.3 inch eDP 2K display panel +Innolux P120ZDG-BF1 12.02 inch eDP 2K display panel This binding is compatible with the simple-panel binding, which is specified in simple-panel.txt in this directory. Required properties: -- compatible: should be "innolux,tv123wam" +- compatible: should be "innolux,p120zdg-bf1" - power-supply: regulator to provide the supply voltage Optional properties: - enable-gpios: GPIO pin to enable or disable the panel - backlight: phandle of the backlight device attached to the panel +- no-hpd: If HPD isn't hooked up; add this property. Example: panel_edp: panel-edp { - compatible = "innolux,tv123wam"; + compatible = "innolux,p120zdg-bf1"; enable-gpios = <&msmgpio 31 GPIO_ACTIVE_LOW>; power-supply = <&pm8916_l2>; backlight = <&backlight>; + no-hpd; }; diff --git a/Documentation/devicetree/bindings/display/panel/simple-panel.txt b/Documentation/devicetree/bindings/display/panel/simple-panel.txt index 45a457ad38f0..b2b872c710f2 100644 --- a/Documentation/devicetree/bindings/display/panel/simple-panel.txt +++ b/Documentation/devicetree/bindings/display/panel/simple-panel.txt @@ -11,6 +11,9 @@ Optional properties: - ddc-i2c-bus: phandle of an I2C controller used for DDC EDID probing - enable-gpios: GPIO pin to enable or disable the panel - backlight: phandle of the backlight device attached to the panel +- no-hpd: This panel is supposed to communicate that it's ready via HPD + (hot plug detect) signal, but the signal isn't hooked up so we should + hardcode the max delay from the panel spec when powering up the panel. Example: diff --git a/Documentation/devicetree/bindings/display/renesas,du.txt b/Documentation/devicetree/bindings/display/renesas,du.txt index caae2348a292..9de67be632d1 100644 --- a/Documentation/devicetree/bindings/display/renesas,du.txt +++ b/Documentation/devicetree/bindings/display/renesas,du.txt @@ -16,6 +16,7 @@ Required Properties: - "renesas,du-r8a77965" for R8A77965 (R-Car M3-N) compatible DU - "renesas,du-r8a77970" for R8A77970 (R-Car V3M) compatible DU - "renesas,du-r8a77980" for R8A77980 (R-Car V3H) compatible DU + - "renesas,du-r8a77990" for R8A77990 (R-Car E3) compatible DU - "renesas,du-r8a77995" for R8A77995 (R-Car D3) compatible DU - reg: the memory-mapped I/O registers base address and length @@ -63,6 +64,7 @@ corresponding to each DU output. R8A77965 (R-Car M3-N) DPAD 0 HDMI 0 LVDS 0 - R8A77970 (R-Car V3M) DPAD 0 LVDS 0 - - R8A77980 (R-Car V3H) DPAD 0 LVDS 0 - - + R8A77990 (R-Car E3) DPAD 0 LVDS 0 LVDS 1 - R8A77995 (R-Car D3) DPAD 0 LVDS 0 LVDS 1 - diff --git a/Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt b/Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt index 22d6dda587c5..7854fff4fc16 100644 --- a/Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt +++ b/Documentation/devicetree/bindings/display/sunxi/sun4i-drm.txt @@ -376,6 +376,8 @@ Required properties: * allwinner,sun8i-a83t-de2-mixer-0 * allwinner,sun8i-a83t-de2-mixer-1 * allwinner,sun8i-h3-de2-mixer-0 + * allwinner,sun8i-r40-de2-mixer-0 + * allwinner,sun8i-r40-de2-mixer-1 * allwinner,sun8i-v3s-de2-mixer * allwinner,sun50i-a64-de2-mixer-0 * allwinner,sun50i-a64-de2-mixer-1 diff --git a/Documentation/devicetree/bindings/i2c/i2c-imx-lpi2c.txt b/Documentation/devicetree/bindings/i2c/i2c-imx-lpi2c.txt index 00e4365d7206..091c8dfd3229 100644 --- a/Documentation/devicetree/bindings/i2c/i2c-imx-lpi2c.txt +++ b/Documentation/devicetree/bindings/i2c/i2c-imx-lpi2c.txt @@ -3,7 +3,6 @@ Required properties: - compatible : - "fsl,imx7ulp-lpi2c" for LPI2C compatible with the one integrated on i.MX7ULP soc - - "fsl,imx8dv-lpi2c" for LPI2C compatible with the one integrated on i.MX8DV soc - reg : address and length of the lpi2c master registers - interrupts : lpi2c interrupt - clocks : lpi2c clock specifier @@ -11,7 +10,7 @@ Required properties: Examples: lpi2c7: lpi2c7@40a50000 { - compatible = "fsl,imx8dv-lpi2c"; + compatible = "fsl,imx7ulp-lpi2c"; reg = <0x40A50000 0x10000>; interrupt-parent = <&intc>; interrupts = <GIC_SPI 37 IRQ_TYPE_LEVEL_HIGH>; diff --git a/Documentation/devicetree/bindings/input/gpio-keys.txt b/Documentation/devicetree/bindings/input/gpio-keys.txt index 996ce84352cb..7cccc49b6bea 100644 --- a/Documentation/devicetree/bindings/input/gpio-keys.txt +++ b/Documentation/devicetree/bindings/input/gpio-keys.txt @@ -1,4 +1,4 @@ -Device-Tree bindings for input/gpio_keys.c keyboard driver +Device-Tree bindings for input/keyboard/gpio_keys.c keyboard driver Required properties: - compatible = "gpio-keys"; diff --git a/Documentation/devicetree/bindings/net/cpsw.txt b/Documentation/devicetree/bindings/net/cpsw.txt index 41089369f891..b3acebe08eb0 100644 --- a/Documentation/devicetree/bindings/net/cpsw.txt +++ b/Documentation/devicetree/bindings/net/cpsw.txt @@ -19,6 +19,10 @@ Required properties: - slaves : Specifies number for slaves - active_slave : Specifies the slave to use for time stamping, ethtool and SIOCGMIIPHY +- cpsw-phy-sel : Specifies the phandle to the CPSW phy mode selection + device. See also cpsw-phy-sel.txt for it's binding. + Note that in legacy cases cpsw-phy-sel may be + a child device instead of a phandle. Optional properties: - ti,hwmods : Must be "cpgmac0" @@ -75,6 +79,7 @@ Examples: cpts_clock_mult = <0x80000000>; cpts_clock_shift = <29>; syscon = <&cm>; + cpsw-phy-sel = <&phy_sel>; cpsw_emac0: slave@0 { phy_id = <&davinci_mdio>, <0>; phy-mode = "rgmii-txid"; @@ -103,6 +108,7 @@ Examples: cpts_clock_mult = <0x80000000>; cpts_clock_shift = <29>; syscon = <&cm>; + cpsw-phy-sel = <&phy_sel>; cpsw_emac0: slave@0 { phy_id = <&davinci_mdio>, <0>; phy-mode = "rgmii-txid"; diff --git a/Documentation/devicetree/bindings/net/macb.txt b/Documentation/devicetree/bindings/net/macb.txt index 457d5ae16f23..3e17ac1d5d58 100644 --- a/Documentation/devicetree/bindings/net/macb.txt +++ b/Documentation/devicetree/bindings/net/macb.txt @@ -10,6 +10,7 @@ Required properties: Use "cdns,pc302-gem" for Picochip picoXcell pc302 and later devices based on the Cadence GEM, or the generic form: "cdns,gem". Use "atmel,sama5d2-gem" for the GEM IP (10/100) available on Atmel sama5d2 SoCs. + Use "atmel,sama5d3-macb" for the 10/100Mbit IP available on Atmel sama5d3 SoCs. Use "atmel,sama5d3-gem" for the Gigabit IP available on Atmel sama5d3 SoCs. Use "atmel,sama5d4-gem" for the GEM IP (10/100) available on Atmel sama5d4 SoCs. Use "cdns,zynq-gem" Xilinx Zynq-7xxx SoC. diff --git a/Documentation/devicetree/bindings/net/sh_eth.txt b/Documentation/devicetree/bindings/net/sh_eth.txt index 76db9f13ad96..abc36274227c 100644 --- a/Documentation/devicetree/bindings/net/sh_eth.txt +++ b/Documentation/devicetree/bindings/net/sh_eth.txt @@ -16,6 +16,7 @@ Required properties: "renesas,ether-r8a7794" if the device is a part of R8A7794 SoC. "renesas,gether-r8a77980" if the device is a part of R8A77980 SoC. "renesas,ether-r7s72100" if the device is a part of R7S72100 SoC. + "renesas,ether-r7s9210" if the device is a part of R7S9210 SoC. "renesas,rcar-gen1-ether" for a generic R-Car Gen1 device. "renesas,rcar-gen2-ether" for a generic R-Car Gen2 or RZ/G1 device. diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt index 4b2084d0f1fb..a6c6a8af48a2 100644 --- a/Documentation/filesystems/vfs.txt +++ b/Documentation/filesystems/vfs.txt @@ -848,7 +848,7 @@ struct file_operations ---------------------- This describes how the VFS can manipulate an open file. As of kernel -4.1, the following members are defined: +4.18, the following members are defined: struct file_operations { struct module *owner; @@ -858,11 +858,11 @@ struct file_operations { ssize_t (*read_iter) (struct kiocb *, struct iov_iter *); ssize_t (*write_iter) (struct kiocb *, struct iov_iter *); int (*iterate) (struct file *, struct dir_context *); + int (*iterate_shared) (struct file *, struct dir_context *); __poll_t (*poll) (struct file *, struct poll_table_struct *); long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long); long (*compat_ioctl) (struct file *, unsigned int, unsigned long); int (*mmap) (struct file *, struct vm_area_struct *); - int (*mremap)(struct file *, struct vm_area_struct *); int (*open) (struct inode *, struct file *); int (*flush) (struct file *, fl_owner_t id); int (*release) (struct inode *, struct file *); @@ -882,6 +882,10 @@ struct file_operations { #ifndef CONFIG_MMU unsigned (*mmap_capabilities)(struct file *); #endif + ssize_t (*copy_file_range)(struct file *, loff_t, struct file *, loff_t, size_t, unsigned int); + int (*clone_file_range)(struct file *, loff_t, struct file *, loff_t, u64); + int (*dedupe_file_range)(struct file *, loff_t, struct file *, loff_t, u64); + int (*fadvise)(struct file *, loff_t, loff_t, int); }; Again, all methods are called without any locks being held, unless @@ -899,6 +903,9 @@ otherwise noted. iterate: called when the VFS needs to read the directory contents + iterate_shared: called when the VFS needs to read the directory contents + when filesystem supports concurrent dir iterators + poll: called by the VFS when a process wants to check if there is activity on this file and (optionally) go to sleep until there is activity. Called by the select(2) and poll(2) system calls @@ -951,6 +958,16 @@ otherwise noted. fallocate: called by the VFS to preallocate blocks or punch a hole. + copy_file_range: called by the copy_file_range(2) system call. + + clone_file_range: called by the ioctl(2) system call for FICLONERANGE and + FICLONE commands. + + dedupe_file_range: called by the ioctl(2) system call for FIDEDUPERANGE + command. + + fadvise: possibly called by the fadvise64() system call. + Note that the file operations are implemented by the specific filesystem in which the inode resides. When opening a device node (character or block special) most filesystems will call special diff --git a/Documentation/media/uapi/dvb/video_function_calls.rst b/Documentation/media/uapi/dvb/video_function_calls.rst index 3f4f6c9ffad7..a4222b6cd2d3 100644 --- a/Documentation/media/uapi/dvb/video_function_calls.rst +++ b/Documentation/media/uapi/dvb/video_function_calls.rst @@ -33,4 +33,3 @@ Video Function Calls video-clear-buffer video-set-streamtype video-set-format - video-set-attributes diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst index 61f918b10a0c..d1bf143b446f 100644 --- a/Documentation/process/changes.rst +++ b/Documentation/process/changes.rst @@ -86,7 +86,7 @@ pkg-config The build system, as of 4.18, requires pkg-config to check for installed kconfig tools and to determine flags settings for use in -'make {menu,n,g,x}config'. Previously pkg-config was being used but not +'make {g,x}config'. Previously pkg-config was being used but not verified or documented. Flex diff --git a/Documentation/process/code-of-conduct.rst b/Documentation/process/code-of-conduct.rst new file mode 100644 index 000000000000..ab7c24b5478c --- /dev/null +++ b/Documentation/process/code-of-conduct.rst @@ -0,0 +1,81 @@ +Contributor Covenant Code of Conduct +++++++++++++++++++++++++++++++++++++ + +Our Pledge +========== + +In the interest of fostering an open and welcoming environment, we as +contributors and maintainers pledge to making participation in our project and +our community a harassment-free experience for everyone, regardless of age, body +size, disability, ethnicity, sex characteristics, gender identity and +expression, level of experience, education, socio-economic status, nationality, +personal appearance, race, religion, or sexual identity and orientation. + +Our Standards +============= + +Examples of behavior that contributes to creating a positive environment +include: + +* Using welcoming and inclusive language +* Being respectful of differing viewpoints and experiences +* Gracefully accepting constructive criticism +* Focusing on what is best for the community +* Showing empathy towards other community members + + +Examples of unacceptable behavior by participants include: + +* The use of sexualized language or imagery and unwelcome sexual attention or + advances +* Trolling, insulting/derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others’ private information, such as a physical or electronic + address, without explicit permission +* Other conduct which could reasonably be considered inappropriate in a + professional setting + + +Our Responsibilities +==================== + +Maintainers are responsible for clarifying the standards of acceptable behavior +and are expected to take appropriate and fair corrective action in response to +any instances of unacceptable behavior. + +Maintainers have the right and responsibility to remove, edit, or reject +comments, commits, code, wiki edits, issues, and other contributions that are +not aligned to this Code of Conduct, or to ban temporarily or permanently any +contributor for other behaviors that they deem inappropriate, threatening, +offensive, or harmful. + +Scope +===== + +This Code of Conduct applies both within project spaces and in public spaces +when an individual is representing the project or its community. Examples of +representing a project or community include using an official project e-mail +address, posting via an official social media account, or acting as an appointed +representative at an online or offline event. Representation of a project may be +further defined and clarified by project maintainers. + +Enforcement +=========== + +Instances of abusive, harassing, or otherwise unacceptable behavior may be +reported by contacting the Technical Advisory Board (TAB) at +<tab@lists.linux-foundation.org>. All complaints will be reviewed and +investigated and will result in a response that is deemed necessary and +appropriate to the circumstances. The TAB is obligated to maintain +confidentiality with regard to the reporter of an incident. Further details of +specific enforcement policies may be posted separately. + +Maintainers who do not follow or enforce the Code of Conduct in good faith may +face temporary or permanent repercussions as determined by other members of the +project’s leadership. + +Attribution +=========== + +This Code of Conduct is adapted from the Contributor Covenant, version 1.4, +available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html diff --git a/Documentation/process/code-of-conflict.rst b/Documentation/process/code-of-conflict.rst deleted file mode 100644 index 47b6de763203..000000000000 --- a/Documentation/process/code-of-conflict.rst +++ /dev/null @@ -1,28 +0,0 @@ -Code of Conflict ----------------- - -The Linux kernel development effort is a very personal process compared -to "traditional" ways of developing software. Your code and ideas -behind it will be carefully reviewed, often resulting in critique and -criticism. The review will almost always require improvements to the -code before it can be included in the kernel. Know that this happens -because everyone involved wants to see the best possible solution for -the overall success of Linux. This development process has been proven -to create the most robust operating system kernel ever, and we do not -want to do anything to cause the quality of submission and eventual -result to ever decrease. - -If however, anyone feels personally abused, threatened, or otherwise -uncomfortable due to this process, that is not acceptable. If so, -please contact the Linux Foundation's Technical Advisory Board at -<tab@lists.linux-foundation.org>, or the individual members, and they -will work to resolve the issue to the best of their ability. For more -information on who is on the Technical Advisory Board and what their -role is, please see: - - - http://www.linuxfoundation.org/projects/linux/tab - -As a reviewer of code, please strive to keep things civil and focused on -the technical issues involved. We are all humans, and frustrations can -be high on both sides of the process. Try to keep in mind the immortal -words of Bill and Ted, "Be excellent to each other." diff --git a/Documentation/process/index.rst b/Documentation/process/index.rst index 37bd0628b6ee..9ae3e317bddf 100644 --- a/Documentation/process/index.rst +++ b/Documentation/process/index.rst @@ -20,7 +20,7 @@ Below are the essential guides that every developer should read. :maxdepth: 1 howto - code-of-conflict + code-of-conduct development-process submitting-patches coding-style diff --git a/Documentation/scsi/scsi-parameters.txt b/Documentation/scsi/scsi-parameters.txt index 25a4b4cf04a6..92999d4e0cb8 100644 --- a/Documentation/scsi/scsi-parameters.txt +++ b/Documentation/scsi/scsi-parameters.txt @@ -97,6 +97,11 @@ parameters may be changed at runtime by the command allowing boot to proceed. none ignores them, expecting user space to do the scan. + scsi_mod.use_blk_mq= + [SCSI] use blk-mq I/O path by default + See SCSI_MQ_DEFAULT in drivers/scsi/Kconfig. + Format: <y/n> + sim710= [SCSI,HW] See header of drivers/scsi/sim710.c. diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt index c664064f76fb..647f94128a85 100644 --- a/Documentation/virtual/kvm/api.txt +++ b/Documentation/virtual/kvm/api.txt @@ -4510,7 +4510,8 @@ Do not enable KVM_FEATURE_PV_UNHALT if you disable HLT exits. Architectures: s390 Parameters: none Returns: 0 on success, -EINVAL if hpage module parameter was not set - or cmma is enabled + or cmma is enabled, or the VM has the KVM_VM_S390_UCONTROL + flag set With this capability the KVM support for memory backing with 1m pages through hugetlbfs can be enabled for a VM. After the capability is @@ -4521,6 +4522,15 @@ hpage module parameter is not set to 1, -EINVAL is returned. While it is generally possible to create a huge page backed VM without this capability, the VM will not be able to run. +7.14 KVM_CAP_MSR_PLATFORM_INFO + +Architectures: x86 +Parameters: args[0] whether feature should be enabled or not + +With this capability, a guest may read the MSR_PLATFORM_INFO MSR. Otherwise, +a #GP would be raised when the guest tries to access. Currently, this +capability does not enable write permissions of this MSR for the guest. + 8. Other capabilities. ---------------------- diff --git a/Documentation/x86/earlyprintk.txt b/Documentation/x86/earlyprintk.txt index 688e3eeed21d..46933e06c972 100644 --- a/Documentation/x86/earlyprintk.txt +++ b/Documentation/x86/earlyprintk.txt @@ -35,25 +35,25 @@ and two USB cables, connected like this: ( If your system does not list a debug port capability then you probably won't be able to use the USB debug key. ) - b.) You also need a Netchip USB debug cable/key: + b.) You also need a NetChip USB debug cable/key: http://www.plxtech.com/products/NET2000/NET20DC/default.asp - This is a small blue plastic connector with two USB connections, + This is a small blue plastic connector with two USB connections; it draws power from its USB connections. c.) You need a second client/console system with a high speed USB 2.0 port. - d.) The Netchip device must be plugged directly into the physical + d.) The NetChip device must be plugged directly into the physical debug port on the "host/target" system. You cannot use a USB hub in between the physical debug port and the "host/target" system. The EHCI debug controller is bound to a specific physical USB - port and the Netchip device will only work as an early printk + port and the NetChip device will only work as an early printk device in this port. The EHCI host controllers are electrically wired such that the EHCI debug controller is hooked up to the - first physical and there is no way to change this via software. + first physical port and there is no way to change this via software. You can find the physical port through experimentation by trying each physical port on the system and rebooting. Or you can try and use lsusb or look at the kernel info messages emitted by the @@ -65,9 +65,9 @@ and two USB cables, connected like this: to the hardware vendor, because there is no reason not to wire this port into one of the physically accessible ports. - e.) It is also important to note, that many versions of the Netchip + e.) It is also important to note, that many versions of the NetChip device require the "client/console" system to be plugged into the - right and side of the device (with the product logo facing up and + right hand side of the device (with the product logo facing up and readable left to right). The reason being is that the 5 volt power supply is taken from only one side of the device and it must be the side that does not get rebooted. @@ -81,13 +81,18 @@ and two USB cables, connected like this: CONFIG_EARLY_PRINTK_DBGP=y And you need to add the boot command line: "earlyprintk=dbgp". + (If you are using Grub, append it to the 'kernel' line in - /etc/grub.conf) + /etc/grub.conf. If you are using Grub2 on a BIOS firmware system, + append it to the 'linux' line in /boot/grub2/grub.cfg. If you are + using Grub2 on an EFI firmware system, append it to the 'linux' + or 'linuxefi' line in /boot/grub2/grub.cfg or + /boot/efi/EFI/<distro>/grub.cfg.) On systems with more than one EHCI debug controller you must specify the correct EHCI debug controller number. The ordering comes from the PCI bus enumeration of the EHCI controllers. The - default with no number argument is "0" the first EHCI debug + default with no number argument is "0" or the first EHCI debug controller. To use the second EHCI debug controller, you would use the command line: "earlyprintk=dbgp1" @@ -111,7 +116,7 @@ and two USB cables, connected like this: see the raw output. c.) On Nvidia Southbridge based systems: the kernel will try to probe - and find out which port has debug device connected. + and find out which port has a debug device connected. 3. Testing that it works fine: diff --git a/MAINTAINERS b/MAINTAINERS index 567e03759af9..69373eb328d4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2311,6 +2311,7 @@ F: drivers/clocksource/cadence_ttc_timer.c F: drivers/i2c/busses/i2c-cadence.c F: drivers/mmc/host/sdhci-of-arasan.c F: drivers/edac/synopsys_edac.c +F: drivers/i2c/busses/i2c-xiic.c ARM64 PORT (AARCH64 ARCHITECTURE) M: Catalin Marinas <catalin.marinas@arm.com> @@ -4720,8 +4721,11 @@ F: drivers/gpu/drm/tdfx/ DRM DRIVER FOR USB DISPLAYLINK VIDEO ADAPTERS M: Dave Airlie <airlied@redhat.com> +R: Sean Paul <sean@poorly.run> +L: dri-devel@lists.freedesktop.org S: Odd Fixes F: drivers/gpu/drm/udl/ +T: git git://anongit.freedesktop.org/drm/drm-misc DRM DRIVER FOR VMWARE VIRTUAL GPU M: "VMware Graphics" <linux-graphics-maintainer@vmware.com> @@ -4751,8 +4755,8 @@ F: include/uapi/drm/ F: include/linux/vga* DRM DRIVERS AND MISC GPU PATCHES -M: Gustavo Padovan <gustavo@padovan.org> M: Maarten Lankhorst <maarten.lankhorst@linux.intel.com> +M: Maxime Ripard <maxime.ripard@bootlin.com> M: Sean Paul <sean@poorly.run> W: https://01.org/linuxgraphics/gfx-docs/maintainer-tools/drm-misc.html S: Maintained @@ -4821,6 +4825,7 @@ F: drivers/gpu/drm/fsl-dcu/ F: Documentation/devicetree/bindings/display/fsl,dcu.txt F: Documentation/devicetree/bindings/display/fsl,tcon.txt F: Documentation/devicetree/bindings/display/panel/nec,nl4827hc19-05b.txt +T: git git://anongit.freedesktop.org/drm/drm-misc DRM DRIVERS FOR FREESCALE IMX M: Philipp Zabel <p.zabel@pengutronix.de> @@ -5625,6 +5630,8 @@ F: lib/fault-inject.c FBTFT Framebuffer drivers M: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> +L: dri-devel@lists.freedesktop.org +L: linux-fbdev@vger.kernel.org S: Maintained F: drivers/staging/fbtft/ @@ -6060,7 +6067,7 @@ F: Documentation/gcc-plugins.txt GASKET DRIVER FRAMEWORK M: Rob Springer <rspringer@google.com> -M: John Joseph <jnjoseph@google.com> +M: Todd Poynor <toddpoynor@google.com> M: Ben Chan <benchan@chromium.org> S: Maintained F: drivers/staging/gasket/ @@ -7016,6 +7023,20 @@ F: drivers/crypto/vmx/aes* F: drivers/crypto/vmx/ghash* F: drivers/crypto/vmx/ppc-xlate.pl +IBM Power PCI Hotplug Driver for RPA-compliant PPC64 platform +M: Tyrel Datwyler <tyreld@linux.vnet.ibm.com> +L: linux-pci@vger.kernel.org +L: linuxppc-dev@lists.ozlabs.org +S: Supported +F: drivers/pci/hotplug/rpaphp* + +IBM Power IO DLPAR Driver for RPA-compliant PPC64 platform +M: Tyrel Datwyler <tyreld@linux.vnet.ibm.com> +L: linux-pci@vger.kernel.org +L: linuxppc-dev@lists.ozlabs.org +S: Supported +F: drivers/pci/hotplug/rpadlpar* + IBM ServeRAID RAID DRIVER S: Orphan F: drivers/scsi/ips.* @@ -8300,7 +8321,7 @@ F: include/linux/libata.h F: Documentation/devicetree/bindings/ata/ LIBLOCKDEP -M: Sasha Levin <alexander.levin@verizon.com> +M: Sasha Levin <alexander.levin@microsoft.com> S: Maintained F: tools/lib/lockdep/ @@ -9700,13 +9721,6 @@ Q: http://patchwork.linuxtv.org/project/linux-media/list/ S: Maintained F: drivers/media/dvb-frontends/mn88473* -PCI DRIVER FOR MOBIVEIL PCIE IP -M: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in> -L: linux-pci@vger.kernel.org -S: Supported -F: Documentation/devicetree/bindings/pci/mobiveil-pcie.txt -F: drivers/pci/controller/pcie-mobiveil.c - MODULE SUPPORT M: Jessica Yu <jeyu@kernel.org> T: git git://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux.git modules-next @@ -9889,9 +9903,12 @@ F: drivers/media/tuners/mxl5007t.* MXSFB DRM DRIVER M: Marek Vasut <marex@denx.de> +M: Stefan Agner <stefan@agner.ch> +L: dri-devel@lists.freedesktop.org S: Supported F: drivers/gpu/drm/mxsfb/ F: Documentation/devicetree/bindings/display/mxsfb.txt +T: git git://anongit.freedesktop.org/drm/drm-misc MYRICOM MYRI-10G 10GbE DRIVER (MYRI10GE) M: Chris Lee <christopher.lee@cspi.com> @@ -10933,7 +10950,7 @@ M: Willy Tarreau <willy@haproxy.com> M: Ksenija Stanojevic <ksenija.stanojevic@gmail.com> S: Odd Fixes F: Documentation/auxdisplay/lcd-panel-cgram.txt -F: drivers/misc/panel.c +F: drivers/auxdisplay/panel.c PARALLEL PORT SUBSYSTEM M: Sudip Mukherjee <sudipm.mukherjee@gmail.com> @@ -11121,6 +11138,13 @@ F: include/uapi/linux/switchtec_ioctl.h F: include/linux/switchtec.h F: drivers/ntb/hw/mscc/ +PCI DRIVER FOR MOBIVEIL PCIE IP +M: Subrahmanya Lingappa <l.subrahmanya@mobiveil.co.in> +L: linux-pci@vger.kernel.org +S: Supported +F: Documentation/devicetree/bindings/pci/mobiveil-pcie.txt +F: drivers/pci/controller/pcie-mobiveil.c + PCI DRIVER FOR MVEBU (Marvell Armada 370 and Armada XP SOC support) M: Thomas Petazzoni <thomas.petazzoni@free-electrons.com> M: Jason Cooper <jason@lakedaemon.net> @@ -11154,7 +11178,7 @@ F: drivers/pci/controller/dwc/pci-exynos.c PCI DRIVER FOR SYNOPSYS DESIGNWARE M: Jingoo Han <jingoohan1@gmail.com> -M: Joao Pinto <Joao.Pinto@synopsys.com> +M: Gustavo Pimentel <gustavo.pimentel@synopsys.com> L: linux-pci@vger.kernel.org S: Maintained F: Documentation/devicetree/bindings/pci/designware-pcie.txt @@ -11187,8 +11211,14 @@ F: tools/pci/ PCI ENHANCED ERROR HANDLING (EEH) FOR POWERPC M: Russell Currey <ruscur@russell.cc> +M: Sam Bobroff <sbobroff@linux.ibm.com> +M: Oliver O'Halloran <oohall@gmail.com> L: linuxppc-dev@lists.ozlabs.org S: Supported +F: Documentation/PCI/pci-error-recovery.txt +F: drivers/pci/pcie/aer.c +F: drivers/pci/pcie/dpc.c +F: drivers/pci/pcie/err.c F: Documentation/powerpc/eeh-pci-error-recovery.txt F: arch/powerpc/kernel/eeh*.c F: arch/powerpc/platforms/*/eeh*.c @@ -11346,10 +11376,10 @@ S: Maintained F: drivers/platform/x86/peaq-wmi.c PER-CPU MEMORY ALLOCATOR +M: Dennis Zhou <dennis@kernel.org> M: Tejun Heo <tj@kernel.org> M: Christoph Lameter <cl@linux.com> -M: Dennis Zhou <dennisszhou@gmail.com> -T: git git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu.git +T: git git://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu.git S: Maintained F: include/linux/percpu*.h F: mm/percpu*.c @@ -12244,6 +12274,7 @@ F: Documentation/networking/rds.txt RDT - RESOURCE ALLOCATION M: Fenghua Yu <fenghua.yu@intel.com> +M: Reinette Chatre <reinette.chatre@intel.com> L: linux-kernel@vger.kernel.org S: Supported F: arch/x86/kernel/cpu/intel_rdt* @@ -13433,9 +13464,8 @@ F: drivers/i2c/busses/i2c-synquacer.c F: Documentation/devicetree/bindings/i2c/i2c-synquacer.txt SOCIONEXT UNIPHIER SOUND DRIVER -M: Katsuhiro Suzuki <suzuki.katsuhiro@socionext.com> L: alsa-devel@alsa-project.org (moderated for non-subscribers) -S: Maintained +S: Orphan F: sound/soc/uniphier/ SOEKRIS NET48XX LED SUPPORT @@ -15905,6 +15935,7 @@ F: net/x25/ X86 ARCHITECTURE (32-BIT AND 64-BIT) M: Thomas Gleixner <tglx@linutronix.de> M: Ingo Molnar <mingo@redhat.com> +M: Borislav Petkov <bp@alien8.de> R: "H. Peter Anvin" <hpa@zytor.com> M: x86@kernel.org L: linux-kernel@vger.kernel.org @@ -15933,6 +15964,15 @@ M: Borislav Petkov <bp@alien8.de> S: Maintained F: arch/x86/kernel/cpu/microcode/* +X86 MM +M: Dave Hansen <dave.hansen@linux.intel.com> +M: Andy Lutomirski <luto@kernel.org> +M: Peter Zijlstra <peterz@infradead.org> +L: linux-kernel@vger.kernel.org +T: git git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git x86/mm +S: Maintained +F: arch/x86/mm/ + X86 PLATFORM DRIVERS M: Darren Hart <dvhart@infradead.org> M: Andy Shevchenko <andy@infradead.org> @@ -2,7 +2,7 @@ VERSION = 4 PATCHLEVEL = 19 SUBLEVEL = 0 -EXTRAVERSION = -rc2 +EXTRAVERSION = -rc6 NAME = Merciless Moray # *DOCUMENTATION* @@ -299,19 +299,7 @@ KERNELRELEASE = $(shell cat include/config/kernel.release 2> /dev/null) KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION) export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION -# SUBARCH tells the usermode build what the underlying arch is. That is set -# first, and if a usermode build is happening, the "ARCH=um" on the command -# line overrides the setting of ARCH below. If a native build is happening, -# then ARCH is assigned, getting whatever value it gets normally, and -# SUBARCH is subsequently ignored. - -SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \ - -e s/sun4u/sparc64/ \ - -e s/arm.*/arm/ -e s/sa110/arm/ \ - -e s/s390x/s390/ -e s/parisc64/parisc/ \ - -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \ - -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \ - -e s/riscv.*/riscv/) +include scripts/subarch.include # Cross compiling and selecting different set of gcc/bin-utils # --------------------------------------------------------------------------- @@ -616,6 +604,11 @@ CFLAGS_GCOV := -fprofile-arcs -ftest-coverage \ $(call cc-disable-warning,maybe-uninitialized,) export CFLAGS_GCOV +# The arch Makefiles can override CC_FLAGS_FTRACE. We may also append it later. +ifdef CONFIG_FUNCTION_TRACER + CC_FLAGS_FTRACE := -pg +endif + # The arch Makefile can set ARCH_{CPP,A,C}FLAGS to override the default # values of the respective KBUILD_* variables ARCH_CPPFLAGS := @@ -755,9 +748,6 @@ KBUILD_CFLAGS += $(call cc-option, -femit-struct-debug-baseonly) \ endif ifdef CONFIG_FUNCTION_TRACER -ifndef CC_FLAGS_FTRACE -CC_FLAGS_FTRACE := -pg -endif ifdef CONFIG_FTRACE_MCOUNT_RECORD # gcc 5 supports generating the mcount tables directly ifeq ($(call cc-option-yn,-mrecord-mcount),y) diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig index 6d5eb8267e42..b4441b0764d7 100644 --- a/arch/arc/Kconfig +++ b/arch/arc/Kconfig @@ -9,6 +9,7 @@ config ARC def_bool y select ARC_TIMERS + select ARCH_HAS_PTE_SPECIAL select ARCH_HAS_SYNC_DMA_FOR_CPU select ARCH_HAS_SYNC_DMA_FOR_DEVICE select ARCH_HAS_SG_CHAIN @@ -28,8 +29,12 @@ config ARC select GENERIC_SMP_IDLE_THREAD select HAVE_ARCH_KGDB select HAVE_ARCH_TRACEHOOK + select HAVE_DEBUG_STACKOVERFLOW select HAVE_FUTEX_CMPXCHG if FUTEX + select HAVE_GENERIC_DMA_COHERENT select HAVE_IOREMAP_PROT + select HAVE_KERNEL_GZIP + select HAVE_KERNEL_LZMA select HAVE_KPROBES select HAVE_KRETPROBES select HAVE_MEMBLOCK @@ -44,11 +49,6 @@ config ARC select OF_EARLY_FLATTREE select OF_RESERVED_MEM select PERF_USE_VMALLOC if ARC_CACHE_VIPT_ALIASING - select HAVE_DEBUG_STACKOVERFLOW - select HAVE_GENERIC_DMA_COHERENT - select HAVE_KERNEL_GZIP - select HAVE_KERNEL_LZMA - select ARCH_HAS_PTE_SPECIAL config ARCH_HAS_CACHE_LINE_SIZE def_bool y diff --git a/arch/arc/Makefile b/arch/arc/Makefile index fb026196aaab..99cce77ab98f 100644 --- a/arch/arc/Makefile +++ b/arch/arc/Makefile @@ -43,10 +43,7 @@ ifdef CONFIG_ARC_CURR_IN_REG LINUXINCLUDE += -include ${src}/arch/arc/include/asm/current.h endif -upto_gcc44 := $(call cc-ifversion, -le, 0404, y) -atleast_gcc44 := $(call cc-ifversion, -ge, 0404, y) - -cflags-$(atleast_gcc44) += -fsection-anchors +cflags-y += -fsection-anchors cflags-$(CONFIG_ARC_HAS_LLSC) += -mlock cflags-$(CONFIG_ARC_HAS_SWAPE) += -mswape @@ -82,11 +79,6 @@ cflags-$(disable_small_data) += -mno-sdata -fcall-used-gp cflags-$(CONFIG_CPU_BIG_ENDIAN) += -mbig-endian ldflags-$(CONFIG_CPU_BIG_ENDIAN) += -EB -# STAR 9000518362: (fixed with binutils shipping with gcc 4.8) -# arc-linux-uclibc-ld (buildroot) or arceb-elf32-ld (EZChip) don't accept -# --build-id w/o "-marclinux". Default arc-elf32-ld is OK -ldflags-$(upto_gcc44) += -marclinux - LIBGCC := $(shell $(CC) $(cflags-y) --print-libgcc-file-name) # Modules with short calls might break for calls into builtin-kernel diff --git a/arch/arc/boot/dts/axc003.dtsi b/arch/arc/boot/dts/axc003.dtsi index dc91c663bcc0..d75d65ddf8e3 100644 --- a/arch/arc/boot/dts/axc003.dtsi +++ b/arch/arc/boot/dts/axc003.dtsi @@ -94,6 +94,32 @@ }; /* + * Mark DMA peripherals connected via IOC port as dma-coherent. We do + * it via overlay because peripherals defined in axs10x_mb.dtsi are + * used for both AXS101 and AXS103 boards and only AXS103 has IOC (so + * only AXS103 board has HW-coherent DMA peripherals) + * We don't need to mark pgu@17000 as dma-coherent because it uses + * external DMA buffer located outside of IOC aperture. + */ + axs10x_mb { + ethernet@0x18000 { + dma-coherent; + }; + + ehci@0x40000 { + dma-coherent; + }; + + ohci@0x60000 { + dma-coherent; + }; + + mmc@0x15000 { + dma-coherent; + }; + }; + + /* * The DW APB ICTL intc on MB is connected to CPU intc via a * DT "invisible" DW APB GPIO block, configured to simply pass thru * interrupts - setup accordinly in platform init (plat-axs10x/ax10x.c) diff --git a/arch/arc/boot/dts/axc003_idu.dtsi b/arch/arc/boot/dts/axc003_idu.dtsi index 69ff4895f2ba..a05bb737ea63 100644 --- a/arch/arc/boot/dts/axc003_idu.dtsi +++ b/arch/arc/boot/dts/axc003_idu.dtsi @@ -101,6 +101,32 @@ }; /* + * Mark DMA peripherals connected via IOC port as dma-coherent. We do + * it via overlay because peripherals defined in axs10x_mb.dtsi are + * used for both AXS101 and AXS103 boards and only AXS103 has IOC (so + * only AXS103 board has HW-coherent DMA peripherals) + * We don't need to mark pgu@17000 as dma-coherent because it uses + * external DMA buffer located outside of IOC aperture. + */ + axs10x_mb { + ethernet@0x18000 { + dma-coherent; + }; + + ehci@0x40000 { + dma-coherent; + }; + + ohci@0x60000 { + dma-coherent; + }; + + mmc@0x15000 { + dma-coherent; + }; + }; + + /* * This INTC is actually connected to DW APB GPIO * which acts as a wire between MB INTC and CPU INTC. * GPIO INTC is configured in platform init code diff --git a/arch/arc/boot/dts/axs10x_mb.dtsi b/arch/arc/boot/dts/axs10x_mb.dtsi index 47b74fbc403c..37bafd44e36d 100644 --- a/arch/arc/boot/dts/axs10x_mb.dtsi +++ b/arch/arc/boot/dts/axs10x_mb.dtsi @@ -9,6 +9,10 @@ */ / { + aliases { + ethernet = &gmac; + }; + axs10x_mb { compatible = "simple-bus"; #address-cells = <1>; @@ -68,7 +72,7 @@ }; }; - ethernet@0x18000 { + gmac: ethernet@0x18000 { #interrupt-cells = <1>; compatible = "snps,dwmac"; reg = < 0x18000 0x2000 >; @@ -81,6 +85,7 @@ max-speed = <100>; resets = <&creg_rst 5>; reset-names = "stmmaceth"; + mac-address = [00 00 00 00 00 00]; /* Filled in by U-Boot */ }; ehci@0x40000 { diff --git a/arch/arc/boot/dts/hsdk.dts b/arch/arc/boot/dts/hsdk.dts index 006aa3de5348..ef149f59929a 100644 --- a/arch/arc/boot/dts/hsdk.dts +++ b/arch/arc/boot/dts/hsdk.dts @@ -25,6 +25,10 @@ bootargs = "earlycon=uart8250,mmio32,0xf0005000,115200n8 console=ttyS0,115200n8 debug print-fatal-signals=1"; }; + aliases { + ethernet = &gmac; + }; + cpus { #address-cells = <1>; #size-cells = <0>; @@ -163,7 +167,7 @@ #clock-cells = <0>; }; - ethernet@8000 { + gmac: ethernet@8000 { #interrupt-cells = <1>; compatible = "snps,dwmac"; reg = <0x8000 0x2000>; @@ -176,6 +180,8 @@ phy-handle = <&phy0>; resets = <&cgu_rst HSDK_ETH_RESET>; reset-names = "stmmaceth"; + mac-address = [00 00 00 00 00 00]; /* Filled in by U-Boot */ + dma-coherent; mdio { #address-cells = <1>; @@ -194,12 +200,14 @@ compatible = "snps,hsdk-v1.0-ohci", "generic-ohci"; reg = <0x60000 0x100>; interrupts = <15>; + dma-coherent; }; ehci@40000 { compatible = "snps,hsdk-v1.0-ehci", "generic-ehci"; reg = <0x40000 0x100>; interrupts = <15>; + dma-coherent; }; mmc@a000 { @@ -212,6 +220,7 @@ clock-names = "biu", "ciu"; interrupts = <12>; bus-width = <4>; + dma-coherent; }; }; diff --git a/arch/arc/configs/axs101_defconfig b/arch/arc/configs/axs101_defconfig index a635ea972304..41bc08be6a3b 100644 --- a/arch/arc/configs/axs101_defconfig +++ b/arch/arc/configs/axs101_defconfig @@ -1,5 +1,3 @@ -CONFIG_DEFAULT_HOSTNAME="ARCLinux" -# CONFIG_SWAP is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y # CONFIG_CROSS_MEMORY_ATTACH is not set @@ -63,7 +61,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y CONFIG_MOUSE_SERIAL=y CONFIG_MOUSE_SYNAPTICS_USB=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_DW=y diff --git a/arch/arc/configs/axs103_defconfig b/arch/arc/configs/axs103_defconfig index aa507e423075..1e1c4a8011b5 100644 --- a/arch/arc/configs/axs103_defconfig +++ b/arch/arc/configs/axs103_defconfig @@ -1,5 +1,3 @@ -CONFIG_DEFAULT_HOSTNAME="ARCLinux" -# CONFIG_SWAP is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y # CONFIG_CROSS_MEMORY_ATTACH is not set @@ -64,7 +62,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y CONFIG_MOUSE_SERIAL=y CONFIG_MOUSE_SYNAPTICS_USB=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_DW=y diff --git a/arch/arc/configs/axs103_smp_defconfig b/arch/arc/configs/axs103_smp_defconfig index eba07f468654..6b0c0cfd5c30 100644 --- a/arch/arc/configs/axs103_smp_defconfig +++ b/arch/arc/configs/axs103_smp_defconfig @@ -1,5 +1,3 @@ -CONFIG_DEFAULT_HOSTNAME="ARCLinux" -# CONFIG_SWAP is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y # CONFIG_CROSS_MEMORY_ATTACH is not set @@ -65,7 +63,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y CONFIG_MOUSE_SERIAL=y CONFIG_MOUSE_SYNAPTICS_USB=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_DW=y diff --git a/arch/arc/configs/haps_hs_defconfig b/arch/arc/configs/haps_hs_defconfig index 098b19fbaa51..240dd2cd5148 100644 --- a/arch/arc/configs/haps_hs_defconfig +++ b/arch/arc/configs/haps_hs_defconfig @@ -1,4 +1,3 @@ -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y @@ -57,7 +56,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y # CONFIG_SERIO_SERPORT is not set CONFIG_SERIO_ARC_PS2=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=1 diff --git a/arch/arc/configs/haps_hs_smp_defconfig b/arch/arc/configs/haps_hs_smp_defconfig index 0104c404d897..14ae7e5acc7c 100644 --- a/arch/arc/configs/haps_hs_smp_defconfig +++ b/arch/arc/configs/haps_hs_smp_defconfig @@ -1,4 +1,3 @@ -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y @@ -60,7 +59,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y # CONFIG_SERIO_SERPORT is not set CONFIG_SERIO_ARC_PS2=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=1 diff --git a/arch/arc/configs/hsdk_defconfig b/arch/arc/configs/hsdk_defconfig index 6491be0ddbc9..1dec2b4bc5e6 100644 --- a/arch/arc/configs/hsdk_defconfig +++ b/arch/arc/configs/hsdk_defconfig @@ -1,4 +1,3 @@ -CONFIG_DEFAULT_HOSTNAME="ARCLinux" CONFIG_SYSVIPC=y # CONFIG_CROSS_MEMORY_ATTACH is not set CONFIG_NO_HZ_IDLE=y diff --git a/arch/arc/configs/nps_defconfig b/arch/arc/configs/nps_defconfig index 7c9c706ae7f6..31ba224bbfb4 100644 --- a/arch/arc/configs/nps_defconfig +++ b/arch/arc/configs/nps_defconfig @@ -59,7 +59,6 @@ CONFIG_NETCONSOLE=y # CONFIG_INPUT_MOUSE is not set # CONFIG_SERIO is not set # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=1 diff --git a/arch/arc/configs/nsim_700_defconfig b/arch/arc/configs/nsim_700_defconfig index 99e05cf63fca..8e0b8b134cd9 100644 --- a/arch/arc/configs/nsim_700_defconfig +++ b/arch/arc/configs/nsim_700_defconfig @@ -1,5 +1,4 @@ # CONFIG_LOCALVERSION_AUTO is not set -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y @@ -44,7 +43,6 @@ CONFIG_LXT_PHY=y # CONFIG_INPUT_MOUSE is not set # CONFIG_SERIO is not set # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_ARC=y CONFIG_SERIAL_ARC_CONSOLE=y # CONFIG_HW_RANDOM is not set diff --git a/arch/arc/configs/nsim_hs_defconfig b/arch/arc/configs/nsim_hs_defconfig index 0dc4f9b737e7..739b90e5e893 100644 --- a/arch/arc/configs/nsim_hs_defconfig +++ b/arch/arc/configs/nsim_hs_defconfig @@ -1,5 +1,4 @@ # CONFIG_LOCALVERSION_AUTO is not set -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set CONFIG_SYSVIPC=y CONFIG_POSIX_MQUEUE=y @@ -45,7 +44,6 @@ CONFIG_DEVTMPFS=y # CONFIG_INPUT_MOUSE is not set # CONFIG_SERIO is not set # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_ARC=y CONFIG_SERIAL_ARC_CONSOLE=y # CONFIG_HW_RANDOM is not set diff --git a/arch/arc/configs/nsim_hs_smp_defconfig b/arch/arc/configs/nsim_hs_smp_defconfig index be3c30a15e54..b5895bdf3a93 100644 --- a/arch/arc/configs/nsim_hs_smp_defconfig +++ b/arch/arc/configs/nsim_hs_smp_defconfig @@ -1,5 +1,4 @@ # CONFIG_LOCALVERSION_AUTO is not set -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set # CONFIG_CROSS_MEMORY_ATTACH is not set CONFIG_HIGH_RES_TIMERS=y @@ -44,7 +43,6 @@ CONFIG_DEVTMPFS=y # CONFIG_INPUT_MOUSE is not set # CONFIG_SERIO is not set # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_ARC=y CONFIG_SERIAL_ARC_CONSOLE=y # CONFIG_HW_RANDOM is not set diff --git a/arch/arc/configs/nsimosci_defconfig b/arch/arc/configs/nsimosci_defconfig index 3a74b9b21772..f14eeff7d308 100644 --- a/arch/arc/configs/nsimosci_defconfig +++ b/arch/arc/configs/nsimosci_defconfig @@ -1,5 +1,4 @@ # CONFIG_LOCALVERSION_AUTO is not set -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set CONFIG_SYSVIPC=y # CONFIG_CROSS_MEMORY_ATTACH is not set @@ -48,7 +47,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y # CONFIG_SERIO_SERPORT is not set CONFIG_SERIO_ARC_PS2=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=1 diff --git a/arch/arc/configs/nsimosci_hs_defconfig b/arch/arc/configs/nsimosci_hs_defconfig index ea2834b4dc1d..025298a48305 100644 --- a/arch/arc/configs/nsimosci_hs_defconfig +++ b/arch/arc/configs/nsimosci_hs_defconfig @@ -1,5 +1,4 @@ # CONFIG_LOCALVERSION_AUTO is not set -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set CONFIG_SYSVIPC=y # CONFIG_CROSS_MEMORY_ATTACH is not set @@ -47,7 +46,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y # CONFIG_SERIO_SERPORT is not set CONFIG_SERIO_ARC_PS2=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=1 diff --git a/arch/arc/configs/nsimosci_hs_smp_defconfig b/arch/arc/configs/nsimosci_hs_smp_defconfig index 80a5a1b4924b..df7b77b13b82 100644 --- a/arch/arc/configs/nsimosci_hs_smp_defconfig +++ b/arch/arc/configs/nsimosci_hs_smp_defconfig @@ -1,4 +1,3 @@ -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_SWAP is not set CONFIG_SYSVIPC=y # CONFIG_CROSS_MEMORY_ATTACH is not set @@ -58,7 +57,6 @@ CONFIG_MOUSE_PS2_TOUCHKIT=y # CONFIG_SERIO_SERPORT is not set CONFIG_SERIO_ARC_PS2=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=1 diff --git a/arch/arc/configs/tb10x_defconfig b/arch/arc/configs/tb10x_defconfig index 2cc87f909747..a7f65313f84a 100644 --- a/arch/arc/configs/tb10x_defconfig +++ b/arch/arc/configs/tb10x_defconfig @@ -57,7 +57,6 @@ CONFIG_STMMAC_ETH=y # CONFIG_SERIO is not set # CONFIG_VT is not set # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_NR_UARTS=1 diff --git a/arch/arc/configs/vdk_hs38_defconfig b/arch/arc/configs/vdk_hs38_defconfig index f629493929ea..db47c3541f15 100644 --- a/arch/arc/configs/vdk_hs38_defconfig +++ b/arch/arc/configs/vdk_hs38_defconfig @@ -1,5 +1,4 @@ # CONFIG_LOCALVERSION_AUTO is not set -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_CROSS_MEMORY_ATTACH is not set CONFIG_HIGH_RES_TIMERS=y CONFIG_IKCONFIG=y @@ -53,7 +52,6 @@ CONFIG_NATIONAL_PHY=y CONFIG_MOUSE_PS2_TOUCHKIT=y CONFIG_SERIO_ARC_PS2=y # CONFIG_LEGACY_PTYS is not set -# CONFIG_DEVKMEM is not set CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_DW=y diff --git a/arch/arc/configs/vdk_hs38_smp_defconfig b/arch/arc/configs/vdk_hs38_smp_defconfig index 21f0ca26a05d..a8ac5e917d9a 100644 --- a/arch/arc/configs/vdk_hs38_smp_defconfig +++ b/arch/arc/configs/vdk_hs38_smp_defconfig @@ -1,5 +1,4 @@ # CONFIG_LOCALVERSION_AUTO is not set -CONFIG_DEFAULT_HOSTNAME="ARCLinux" # CONFIG_CROSS_MEMORY_ATTACH is not set CONFIG_HIGH_RES_TIMERS=y CONFIG_IKCONFIG=y diff --git a/arch/arc/include/asm/atomic.h b/arch/arc/include/asm/atomic.h index 4e0072730241..158af079838d 100644 --- a/arch/arc/include/asm/atomic.h +++ b/arch/arc/include/asm/atomic.h @@ -84,7 +84,7 @@ static inline int atomic_fetch_##op(int i, atomic_t *v) \ "1: llock %[orig], [%[ctr]] \n" \ " " #asm_op " %[val], %[orig], %[i] \n" \ " scond %[val], [%[ctr]] \n" \ - " \n" \ + " bnz 1b \n" \ : [val] "=&r" (val), \ [orig] "=&r" (orig) \ : [ctr] "r" (&v->counter), \ diff --git a/arch/arc/include/asm/dma-mapping.h b/arch/arc/include/asm/dma-mapping.h new file mode 100644 index 000000000000..c946c0a83e76 --- /dev/null +++ b/arch/arc/include/asm/dma-mapping.h @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: GPL-2.0 +// (C) 2018 Synopsys, Inc. (www.synopsys.com) + +#ifndef ASM_ARC_DMA_MAPPING_H +#define ASM_ARC_DMA_MAPPING_H + +#include <asm-generic/dma-mapping.h> + +void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, + const struct iommu_ops *iommu, bool coherent); +#define arch_setup_dma_ops arch_setup_dma_ops + +#endif diff --git a/arch/arc/kernel/troubleshoot.c b/arch/arc/kernel/troubleshoot.c index 783b20354f8b..e8d9fb452346 100644 --- a/arch/arc/kernel/troubleshoot.c +++ b/arch/arc/kernel/troubleshoot.c @@ -83,9 +83,6 @@ done: static void show_faulting_vma(unsigned long address, char *buf) { struct vm_area_struct *vma; - struct inode *inode; - unsigned long ino = 0; - dev_t dev = 0; char *nm = buf; struct mm_struct *active_mm = current->active_mm; @@ -99,12 +96,10 @@ static void show_faulting_vma(unsigned long address, char *buf) * if the container VMA is not found */ if (vma && (vma->vm_start <= address)) { - struct file *file = vma->vm_file; - if (file) { - nm = file_path(file, buf, PAGE_SIZE - 1); - inode = file_inode(vma->vm_file); - dev = inode->i_sb->s_dev; - ino = inode->i_ino; + if (vma->vm_file) { + nm = file_path(vma->vm_file, buf, PAGE_SIZE - 1); + if (IS_ERR(nm)) + nm = "?"; } pr_info(" @off 0x%lx in [%s]\n" " VMA: 0x%08lx to 0x%08lx\n", diff --git a/arch/arc/mm/cache.c b/arch/arc/mm/cache.c index 25c631942500..f2701c13a66b 100644 --- a/arch/arc/mm/cache.c +++ b/arch/arc/mm/cache.c @@ -65,7 +65,7 @@ char *arc_cache_mumbojumbo(int c, char *buf, int len) n += scnprintf(buf + n, len - n, "Peripherals\t: %#lx%s%s\n", perip_base, - IS_AVAIL3(ioc_exists, ioc_enable, ", IO-Coherency ")); + IS_AVAIL3(ioc_exists, ioc_enable, ", IO-Coherency (per-device) ")); return buf; } @@ -897,15 +897,6 @@ static void __dma_cache_wback_slc(phys_addr_t start, unsigned long sz) } /* - * DMA ops for systems with IOC - * IOC hardware snoops all DMA traffic keeping the caches consistent with - * memory - eliding need for any explicit cache maintenance of DMA buffers - */ -static void __dma_cache_wback_inv_ioc(phys_addr_t start, unsigned long sz) {} -static void __dma_cache_inv_ioc(phys_addr_t start, unsigned long sz) {} -static void __dma_cache_wback_ioc(phys_addr_t start, unsigned long sz) {} - -/* * Exported DMA API */ void dma_cache_wback_inv(phys_addr_t start, unsigned long sz) @@ -1153,6 +1144,19 @@ noinline void __init arc_ioc_setup(void) { unsigned int ioc_base, mem_sz; + /* + * As for today we don't support both IOC and ZONE_HIGHMEM enabled + * simultaneously. This happens because as of today IOC aperture covers + * only ZONE_NORMAL (low mem) and any dma transactions outside this + * region won't be HW coherent. + * If we want to use both IOC and ZONE_HIGHMEM we can use + * bounce_buffer to handle dma transactions to HIGHMEM. + * Also it is possible to modify dma_direct cache ops or increase IOC + * aperture size if we are planning to use HIGHMEM without PAE. + */ + if (IS_ENABLED(CONFIG_HIGHMEM)) + panic("IOC and HIGHMEM can't be used simultaneously"); + /* Flush + invalidate + disable L1 dcache */ __dc_disable(); @@ -1264,11 +1268,7 @@ void __init arc_cache_init_master(void) if (is_isa_arcv2() && ioc_enable) arc_ioc_setup(); - if (is_isa_arcv2() && ioc_enable) { - __dma_cache_wback_inv = __dma_cache_wback_inv_ioc; - __dma_cache_inv = __dma_cache_inv_ioc; - __dma_cache_wback = __dma_cache_wback_ioc; - } else if (is_isa_arcv2() && l2_line_sz && slc_enable) { + if (is_isa_arcv2() && l2_line_sz && slc_enable) { __dma_cache_wback_inv = __dma_cache_wback_inv_slc; __dma_cache_inv = __dma_cache_inv_slc; __dma_cache_wback = __dma_cache_wback_slc; @@ -1277,6 +1277,12 @@ void __init arc_cache_init_master(void) __dma_cache_inv = __dma_cache_inv_l1; __dma_cache_wback = __dma_cache_wback_l1; } + /* + * In case of IOC (say IOC+SLC case), pointers above could still be set + * but end up not being relevant as the first function in chain is not + * called at all for @dma_direct_ops + * arch_sync_dma_for_cpu() -> dma_cache_*() -> __dma_cache_*() + */ } void __ref arc_cache_init(void) diff --git a/arch/arc/mm/dma.c b/arch/arc/mm/dma.c index ec47e6079f5d..c75d5c3470e3 100644 --- a/arch/arc/mm/dma.c +++ b/arch/arc/mm/dma.c @@ -6,20 +6,17 @@ * published by the Free Software Foundation. */ -/* - * DMA Coherent API Notes - * - * I/O is inherently non-coherent on ARC. So a coherent DMA buffer is - * implemented by accessing it using a kernel virtual address, with - * Cache bit off in the TLB entry. - * - * The default DMA address == Phy address which is 0x8000_0000 based. - */ - #include <linux/dma-noncoherent.h> #include <asm/cache.h> #include <asm/cacheflush.h> +/* + * ARCH specific callbacks for generic noncoherent DMA ops (dma/noncoherent.c) + * - hardware IOC not available (or "dma-coherent" not set for device in DT) + * - But still handle both coherent and non-coherent requests from caller + * + * For DMA coherent hardware (IOC) generic code suffices + */ void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, gfp_t gfp, unsigned long attrs) { @@ -27,42 +24,29 @@ void *arch_dma_alloc(struct device *dev, size_t size, dma_addr_t *dma_handle, struct page *page; phys_addr_t paddr; void *kvaddr; - int need_coh = 1, need_kvaddr = 0; - - page = alloc_pages(gfp, order); - if (!page) - return NULL; + bool need_coh = !(attrs & DMA_ATTR_NON_CONSISTENT); /* - * IOC relies on all data (even coherent DMA data) being in cache - * Thus allocate normal cached memory - * - * The gains with IOC are two pronged: - * -For streaming data, elides need for cache maintenance, saving - * cycles in flush code, and bus bandwidth as all the lines of a - * buffer need to be flushed out to memory - * -For coherent data, Read/Write to buffers terminate early in cache - * (vs. always going to memory - thus are faster) + * __GFP_HIGHMEM flag is cleared by upper layer functions + * (in include/linux/dma-mapping.h) so we should never get a + * __GFP_HIGHMEM here. */ - if ((is_isa_arcv2() && ioc_enable) || - (attrs & DMA_ATTR_NON_CONSISTENT)) - need_coh = 0; + BUG_ON(gfp & __GFP_HIGHMEM); - /* - * - A coherent buffer needs MMU mapping to enforce non-cachability - * - A highmem page needs a virtual handle (hence MMU mapping) - * independent of cachability - */ - if (PageHighMem(page) || need_coh) - need_kvaddr = 1; + page = alloc_pages(gfp, order); + if (!page) + return NULL; /* This is linear addr (0x8000_0000 based) */ paddr = page_to_phys(page); *dma_handle = paddr; - /* This is kernel Virtual address (0x7000_0000 based) */ - if (need_kvaddr) { + /* + * A coherent buffer needs MMU mapping to enforce non-cachability. + * kvaddr is kernel Virtual address (0x7000_0000 based). + */ + if (need_coh) { kvaddr = ioremap_nocache(paddr, size); if (kvaddr == NULL) { __free_pages(page, order); @@ -93,12 +77,8 @@ void arch_dma_free(struct device *dev, size_t size, void *vaddr, { phys_addr_t paddr = dma_handle; struct page *page = virt_to_page(paddr); - int is_non_coh = 1; - - is_non_coh = (attrs & DMA_ATTR_NON_CONSISTENT) || - (is_isa_arcv2() && ioc_enable); - if (PageHighMem(page) || !is_non_coh) + if (!(attrs & DMA_ATTR_NON_CONSISTENT)) iounmap((void __force __iomem *)vaddr); __free_pages(page, get_order(size)); @@ -185,3 +165,23 @@ void arch_sync_dma_for_cpu(struct device *dev, phys_addr_t paddr, break; } } + +/* + * Plug in coherent or noncoherent dma ops + */ +void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size, + const struct iommu_ops *iommu, bool coherent) +{ + /* + * IOC hardware snoops all DMA traffic keeping the caches consistent + * with memory - eliding need for any explicit cache maintenance of + * DMA buffers - so we can use dma_direct cache ops. + */ + if (is_isa_arcv2() && ioc_enable && coherent) { + set_dma_ops(dev, &dma_direct_ops); + dev_info(dev, "use dma_direct_ops cache ops\n"); + } else { + set_dma_ops(dev, &dma_noncoherent_ops); + dev_info(dev, "use dma_noncoherent_ops cache ops\n"); + } +} diff --git a/arch/arm/boot/dts/sama5d3_emac.dtsi b/arch/arm/boot/dts/sama5d3_emac.dtsi index 7cb235ef0fb6..6e9e1c2f9def 100644 --- a/arch/arm/boot/dts/sama5d3_emac.dtsi +++ b/arch/arm/boot/dts/sama5d3_emac.dtsi @@ -41,7 +41,7 @@ }; macb1: ethernet@f802c000 { - compatible = "cdns,at91sam9260-macb", "cdns,macb"; + compatible = "atmel,sama5d3-macb", "cdns,at91sam9260-macb", "cdns,macb"; reg = <0xf802c000 0x100>; interrupts = <35 IRQ_TYPE_LEVEL_HIGH 3>; pinctrl-names = "default"; diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h index 79906cecb091..3ad482d2f1eb 100644 --- a/arch/arm/include/asm/kvm_host.h +++ b/arch/arm/include/asm/kvm_host.h @@ -223,7 +223,6 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, struct kvm_vcpu_events *events); #define KVM_ARCH_WANT_MMU_NOTIFIER -int kvm_unmap_hva(struct kvm *kvm, unsigned long hva); int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end); void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte); diff --git a/arch/arm64/boot/dts/allwinner/sun50i-h6-pine-h64.dts b/arch/arm64/boot/dts/allwinner/sun50i-h6-pine-h64.dts index ceffc40810ee..48daec7f78ba 100644 --- a/arch/arm64/boot/dts/allwinner/sun50i-h6-pine-h64.dts +++ b/arch/arm64/boot/dts/allwinner/sun50i-h6-pine-h64.dts @@ -46,6 +46,7 @@ pinctrl-0 = <&mmc0_pins>; vmmc-supply = <®_cldo1>; cd-gpios = <&pio 5 6 GPIO_ACTIVE_LOW>; + bus-width = <4>; status = "okay"; }; @@ -56,6 +57,7 @@ vqmmc-supply = <®_bldo2>; non-removable; cap-mmc-hw-reset; + bus-width = <8>; status = "okay"; }; diff --git a/arch/arm64/include/asm/jump_label.h b/arch/arm64/include/asm/jump_label.h index 1b5e0e843c3a..7e2b3e360086 100644 --- a/arch/arm64/include/asm/jump_label.h +++ b/arch/arm64/include/asm/jump_label.h @@ -28,7 +28,7 @@ static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { - asm goto("1: nop\n\t" + asm_volatile_goto("1: nop\n\t" ".pushsection __jump_table, \"aw\"\n\t" ".align 3\n\t" ".quad 1b, %l[l_yes], %c0\n\t" @@ -42,7 +42,7 @@ l_yes: static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) { - asm goto("1: b %l[l_yes]\n\t" + asm_volatile_goto("1: b %l[l_yes]\n\t" ".pushsection __jump_table, \"aw\"\n\t" ".align 3\n\t" ".quad 1b, %l[l_yes], %c0\n\t" diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index f26055f2306e..3d6d7336f871 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -61,8 +61,7 @@ struct kvm_arch { u64 vmid_gen; u32 vmid; - /* 1-level 2nd stage table and lock */ - spinlock_t pgd_lock; + /* 1-level 2nd stage table, protected by kvm->mmu_lock */ pgd_t *pgd; /* VTTBR value associated with above pgd and vmid */ @@ -357,7 +356,6 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, struct kvm_vcpu_events *events); #define KVM_ARCH_WANT_MMU_NOTIFIER -int kvm_unmap_hva(struct kvm *kvm, unsigned long hva); int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end); void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte); diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 95ac7374d723..4c8b13bede80 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -54,6 +54,7 @@ arm64-obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o \ arm64-obj-$(CONFIG_ARM64_RELOC_TEST) += arm64-reloc-test.o arm64-reloc-test-y := reloc_test_core.o reloc_test_syms.o arm64-obj-$(CONFIG_CRASH_DUMP) += crash_dump.o +arm64-obj-$(CONFIG_CRASH_CORE) += crash_core.o arm64-obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o arm64-obj-$(CONFIG_ARM64_SSBD) += ssbd.o diff --git a/arch/arm64/kernel/crash_core.c b/arch/arm64/kernel/crash_core.c new file mode 100644 index 000000000000..ca4c3e12d8c5 --- /dev/null +++ b/arch/arm64/kernel/crash_core.c @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (C) Linaro. + * Copyright (C) Huawei Futurewei Technologies. + */ + +#include <linux/crash_core.h> +#include <asm/memory.h> + +void arch_crash_save_vmcoreinfo(void) +{ + VMCOREINFO_NUMBER(VA_BITS); + /* Please note VMCOREINFO_NUMBER() uses "%d", not "%x" */ + vmcoreinfo_append_str("NUMBER(kimage_voffset)=0x%llx\n", + kimage_voffset); + vmcoreinfo_append_str("NUMBER(PHYS_OFFSET)=0x%llx\n", + PHYS_OFFSET); + vmcoreinfo_append_str("KERNELOFFSET=%lx\n", kaslr_offset()); +} diff --git a/arch/arm64/kernel/machine_kexec.c b/arch/arm64/kernel/machine_kexec.c index f6a5c6bc1434..922add8adb74 100644 --- a/arch/arm64/kernel/machine_kexec.c +++ b/arch/arm64/kernel/machine_kexec.c @@ -358,14 +358,3 @@ void crash_free_reserved_phys_range(unsigned long begin, unsigned long end) } } #endif /* CONFIG_HIBERNATION */ - -void arch_crash_save_vmcoreinfo(void) -{ - VMCOREINFO_NUMBER(VA_BITS); - /* Please note VMCOREINFO_NUMBER() uses "%d", not "%x" */ - vmcoreinfo_append_str("NUMBER(kimage_voffset)=0x%llx\n", - kimage_voffset); - vmcoreinfo_append_str("NUMBER(PHYS_OFFSET)=0x%llx\n", - PHYS_OFFSET); - vmcoreinfo_append_str("KERNELOFFSET=%lx\n", kaslr_offset()); -} diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index d496ef579859..ca46153d7915 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -98,8 +98,10 @@ static void activate_traps_vhe(struct kvm_vcpu *vcpu) val = read_sysreg(cpacr_el1); val |= CPACR_EL1_TTA; val &= ~CPACR_EL1_ZEN; - if (!update_fp_enabled(vcpu)) + if (!update_fp_enabled(vcpu)) { val &= ~CPACR_EL1_FPEN; + __activate_traps_fpsimd32(vcpu); + } write_sysreg(val, cpacr_el1); @@ -114,8 +116,10 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu) val = CPTR_EL2_DEFAULT; val |= CPTR_EL2_TTA | CPTR_EL2_TZ; - if (!update_fp_enabled(vcpu)) + if (!update_fp_enabled(vcpu)) { val |= CPTR_EL2_TFP; + __activate_traps_fpsimd32(vcpu); + } write_sysreg(val, cptr_el2); } @@ -129,7 +133,6 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu) if (cpus_have_const_cap(ARM64_HAS_RAS_EXTN) && (hcr & HCR_VSE)) write_sysreg_s(vcpu->arch.vsesr_el2, SYS_VSESR_EL2); - __activate_traps_fpsimd32(vcpu); if (has_vhe()) activate_traps_vhe(vcpu); else diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 65f86271f02b..8080c9f489c3 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -985,8 +985,9 @@ int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) pmd = READ_ONCE(*pmdp); - /* No-op for empty entry and WARN_ON for valid entry */ - if (!pmd_present(pmd) || !pmd_table(pmd)) { + if (!pmd_present(pmd)) + return 1; + if (!pmd_table(pmd)) { VM_WARN_ON(!pmd_table(pmd)); return 1; } @@ -1007,8 +1008,9 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pud = READ_ONCE(*pudp); - /* No-op for empty entry and WARN_ON for valid entry */ - if (!pud_present(pud) || !pud_table(pud)) { + if (!pud_present(pud)) + return 1; + if (!pud_table(pud)) { VM_WARN_ON(!pud_table(pud)); return 1; } diff --git a/arch/hexagon/include/asm/bitops.h b/arch/hexagon/include/asm/bitops.h index 5e4a59b3ec1b..2691a1857d20 100644 --- a/arch/hexagon/include/asm/bitops.h +++ b/arch/hexagon/include/asm/bitops.h @@ -211,7 +211,7 @@ static inline long ffz(int x) * This is defined the same way as ffs. * Note fls(0) = 0, fls(1) = 1, fls(0x80000000) = 32. */ -static inline long fls(int x) +static inline int fls(int x) { int r; @@ -232,7 +232,7 @@ static inline long fls(int x) * the libc and compiler builtin ffs routines, therefore * differs in spirit from the above ffz (man ffs). */ -static inline long ffs(int x) +static inline int ffs(int x) { int r; diff --git a/arch/hexagon/kernel/dma.c b/arch/hexagon/kernel/dma.c index 77459df34e2e..7ebe7ad19d15 100644 --- a/arch/hexagon/kernel/dma.c +++ b/arch/hexagon/kernel/dma.c @@ -60,7 +60,7 @@ static void *hexagon_dma_alloc_coherent(struct device *dev, size_t size, panic("Can't create %s() memory pool!", __func__); else gen_pool_add(coherent_pool, - pfn_to_virt(max_low_pfn), + (unsigned long)pfn_to_virt(max_low_pfn), hexagon_coherent_pool_size, -1); } diff --git a/arch/m68k/mm/mcfmmu.c b/arch/m68k/mm/mcfmmu.c index 70dde040779b..f5453d944ff5 100644 --- a/arch/m68k/mm/mcfmmu.c +++ b/arch/m68k/mm/mcfmmu.c @@ -172,7 +172,7 @@ void __init cf_bootmem_alloc(void) high_memory = (void *)_ramend; /* Reserve kernel text/data/bss */ - memblock_reserve(memstart, memstart - _rambase); + memblock_reserve(_rambase, memstart - _rambase); m68k_virt_to_node_shift = fls(_ramend - 1) - 6; module_fixup(NULL, __start_fixup, __stop_fixup); diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h index a9af1d2dcd69..2c1c53d12179 100644 --- a/arch/mips/include/asm/kvm_host.h +++ b/arch/mips/include/asm/kvm_host.h @@ -931,7 +931,6 @@ enum kvm_mips_fault_result kvm_trap_emul_gva_fault(struct kvm_vcpu *vcpu, bool write); #define KVM_ARCH_WANT_MMU_NOTIFIER -int kvm_unmap_hva(struct kvm *kvm, unsigned long hva); int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end); void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte); diff --git a/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h b/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h index 4901833498f7..8441b2698e64 100644 --- a/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h +++ b/arch/mips/include/asm/mach-lantiq/xway/xway_dma.h @@ -40,6 +40,7 @@ struct ltq_dma_channel { int desc; /* the current descriptor */ struct ltq_dma_desc *desc_base; /* the descriptor base */ int phys; /* physical addr */ + struct device *dev; }; enum { diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c index 019035d7225c..8f845f6e5f42 100644 --- a/arch/mips/kernel/vdso.c +++ b/arch/mips/kernel/vdso.c @@ -13,6 +13,7 @@ #include <linux/err.h> #include <linux/init.h> #include <linux/ioport.h> +#include <linux/kernel.h> #include <linux/mm.h> #include <linux/sched.h> #include <linux/slab.h> @@ -20,6 +21,7 @@ #include <asm/abi.h> #include <asm/mips-cps.h> +#include <asm/page.h> #include <asm/vdso.h> /* Kernel-provided data used by the VDSO. */ @@ -128,12 +130,30 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp) vvar_size = gic_size + PAGE_SIZE; size = vvar_size + image->size; + /* + * Find a region that's large enough for us to perform the + * colour-matching alignment below. + */ + if (cpu_has_dc_aliases) + size += shm_align_mask + 1; + base = get_unmapped_area(NULL, 0, size, 0, 0); if (IS_ERR_VALUE(base)) { ret = base; goto out; } + /* + * If we suffer from dcache aliasing, ensure that the VDSO data page + * mapping is coloured the same as the kernel's mapping of that memory. + * This ensures that when the kernel updates the VDSO data userland + * will observe it without requiring cache invalidations. + */ + if (cpu_has_dc_aliases) { + base = __ALIGN_MASK(base, shm_align_mask); + base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask; + } + data_addr = base + gic_size; vdso_addr = data_addr + PAGE_SIZE; diff --git a/arch/mips/kvm/mmu.c b/arch/mips/kvm/mmu.c index ee64db032793..d8dcdb350405 100644 --- a/arch/mips/kvm/mmu.c +++ b/arch/mips/kvm/mmu.c @@ -512,16 +512,6 @@ static int kvm_unmap_hva_handler(struct kvm *kvm, gfn_t gfn, gfn_t gfn_end, return 1; } -int kvm_unmap_hva(struct kvm *kvm, unsigned long hva) -{ - unsigned long end = hva + PAGE_SIZE; - - handle_hva_to_gpa(kvm, hva, end, &kvm_unmap_hva_handler, NULL); - - kvm_mips_callbacks->flush_shadow_all(kvm); - return 0; -} - int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end) { handle_hva_to_gpa(kvm, start, end, &kvm_unmap_hva_handler, NULL); diff --git a/arch/mips/lantiq/xway/dma.c b/arch/mips/lantiq/xway/dma.c index 4b9fbb6744ad..664f2f7f55c1 100644 --- a/arch/mips/lantiq/xway/dma.c +++ b/arch/mips/lantiq/xway/dma.c @@ -130,7 +130,7 @@ ltq_dma_alloc(struct ltq_dma_channel *ch) unsigned long flags; ch->desc = 0; - ch->desc_base = dma_zalloc_coherent(NULL, + ch->desc_base = dma_zalloc_coherent(ch->dev, LTQ_DESC_NUM * LTQ_DESC_SIZE, &ch->phys, GFP_ATOMIC); @@ -182,7 +182,7 @@ ltq_dma_free(struct ltq_dma_channel *ch) if (!ch->desc_base) return; ltq_dma_close(ch); - dma_free_coherent(NULL, LTQ_DESC_NUM * LTQ_DESC_SIZE, + dma_free_coherent(ch->dev, LTQ_DESC_NUM * LTQ_DESC_SIZE, ch->desc_base, ch->phys); } EXPORT_SYMBOL_GPL(ltq_dma_free); diff --git a/arch/nds32/Kconfig b/arch/nds32/Kconfig index 1d4248fa55e9..7068f341133d 100644 --- a/arch/nds32/Kconfig +++ b/arch/nds32/Kconfig @@ -40,6 +40,10 @@ config NDS32 select NO_IOPORT_MAP select RTC_LIB select THREAD_INFO_IN_TASK + select HAVE_FUNCTION_TRACER + select HAVE_FUNCTION_GRAPH_TRACER + select HAVE_FTRACE_MCOUNT_RECORD + select HAVE_DYNAMIC_FTRACE help Andes(nds32) Linux support. diff --git a/arch/nds32/Makefile b/arch/nds32/Makefile index 63f4f173e5f4..3509fac10491 100644 --- a/arch/nds32/Makefile +++ b/arch/nds32/Makefile @@ -5,6 +5,10 @@ KBUILD_DEFCONFIG := defconfig comma = , +ifdef CONFIG_FUNCTION_TRACER +arch-y += -malways-save-lp -mno-relax +endif + KBUILD_CFLAGS += $(call cc-option, -mno-sched-prolog-epilog) KBUILD_CFLAGS += -mcmodel=large diff --git a/arch/nds32/include/asm/elf.h b/arch/nds32/include/asm/elf.h index 56c479058802..f5f9cf7e0544 100644 --- a/arch/nds32/include/asm/elf.h +++ b/arch/nds32/include/asm/elf.h @@ -121,9 +121,9 @@ struct elf32_hdr; */ #define ELF_CLASS ELFCLASS32 #ifdef __NDS32_EB__ -#define ELF_DATA ELFDATA2MSB; +#define ELF_DATA ELFDATA2MSB #else -#define ELF_DATA ELFDATA2LSB; +#define ELF_DATA ELFDATA2LSB #endif #define ELF_ARCH EM_NDS32 #define USE_ELF_CORE_DUMP diff --git a/arch/nds32/include/asm/ftrace.h b/arch/nds32/include/asm/ftrace.h new file mode 100644 index 000000000000..2f96cc96aa35 --- /dev/null +++ b/arch/nds32/include/asm/ftrace.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_NDS32_FTRACE_H +#define __ASM_NDS32_FTRACE_H + +#ifdef CONFIG_FUNCTION_TRACER + +#define HAVE_FUNCTION_GRAPH_FP_TEST + +#define MCOUNT_ADDR ((unsigned long)(_mcount)) +/* mcount call is composed of three instructions: + * sethi + ori + jral + */ +#define MCOUNT_INSN_SIZE 12 + +extern void _mcount(unsigned long parent_ip); + +#ifdef CONFIG_DYNAMIC_FTRACE + +#define FTRACE_ADDR ((unsigned long)_ftrace_caller) + +#ifdef __NDS32_EL__ +#define INSN_NOP 0x09000040 +#define INSN_SIZE(insn) (((insn & 0x00000080) == 0) ? 4 : 2) +#define IS_SETHI(insn) ((insn & 0x000000fe) == 0x00000046) +#define ENDIAN_CONVERT(insn) be32_to_cpu(insn) +#else /* __NDS32_EB__ */ +#define INSN_NOP 0x40000009 +#define INSN_SIZE(insn) (((insn & 0x80000000) == 0) ? 4 : 2) +#define IS_SETHI(insn) ((insn & 0xfe000000) == 0x46000000) +#define ENDIAN_CONVERT(insn) (insn) +#endif + +extern void _ftrace_caller(unsigned long parent_ip); +static inline unsigned long ftrace_call_adjust(unsigned long addr) +{ + return addr; +} +struct dyn_arch_ftrace { +}; + +#endif /* CONFIG_DYNAMIC_FTRACE */ + +#endif /* CONFIG_FUNCTION_TRACER */ + +#endif /* __ASM_NDS32_FTRACE_H */ diff --git a/arch/nds32/include/asm/nds32.h b/arch/nds32/include/asm/nds32.h index 19b19394a936..68c38151c3e4 100644 --- a/arch/nds32/include/asm/nds32.h +++ b/arch/nds32/include/asm/nds32.h @@ -17,6 +17,7 @@ #else #define FP_OFFSET (-2) #endif +#define LP_OFFSET (-1) extern void __init early_trap_init(void); static inline void GIE_ENABLE(void) diff --git a/arch/nds32/include/asm/uaccess.h b/arch/nds32/include/asm/uaccess.h index 18a009f3804d..362a32d9bd16 100644 --- a/arch/nds32/include/asm/uaccess.h +++ b/arch/nds32/include/asm/uaccess.h @@ -38,7 +38,7 @@ struct exception_table_entry { extern int fixup_exception(struct pt_regs *regs); #define KERNEL_DS ((mm_segment_t) { ~0UL }) -#define USER_DS ((mm_segment_t) {TASK_SIZE - 1}) +#define USER_DS ((mm_segment_t) {TASK_SIZE - 1}) #define get_ds() (KERNEL_DS) #define get_fs() (current_thread_info()->addr_limit) @@ -49,11 +49,11 @@ static inline void set_fs(mm_segment_t fs) current_thread_info()->addr_limit = fs; } -#define segment_eq(a, b) ((a) == (b)) +#define segment_eq(a, b) ((a) == (b)) #define __range_ok(addr, size) (size <= get_fs() && addr <= (get_fs() -size)) -#define access_ok(type, addr, size) \ +#define access_ok(type, addr, size) \ __range_ok((unsigned long)addr, (unsigned long)size) /* * Single-value transfer routines. They automatically use the right @@ -75,70 +75,73 @@ static inline void set_fs(mm_segment_t fs) * versions are void (ie, don't return a value as such). */ -#define get_user(x,p) \ -({ \ - long __e = -EFAULT; \ - if(likely(access_ok(VERIFY_READ, p, sizeof(*p)))) { \ - __e = __get_user(x,p); \ - } else \ - x = 0; \ - __e; \ -}) -#define __get_user(x,ptr) \ +#define get_user __get_user \ + +#define __get_user(x, ptr) \ ({ \ long __gu_err = 0; \ - __get_user_err((x),(ptr),__gu_err); \ + __get_user_check((x), (ptr), __gu_err); \ __gu_err; \ }) -#define __get_user_error(x,ptr,err) \ +#define __get_user_error(x, ptr, err) \ ({ \ - __get_user_err((x),(ptr),err); \ - (void) 0; \ + __get_user_check((x), (ptr), (err)); \ + (void)0; \ }) -#define __get_user_err(x,ptr,err) \ +#define __get_user_check(x, ptr, err) \ +({ \ + const __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_READ, __p, sizeof(*__p))) { \ + __get_user_err((x), __p, (err)); \ + } else { \ + (x) = 0; (err) = -EFAULT; \ + } \ +}) + +#define __get_user_err(x, ptr, err) \ do { \ - unsigned long __gu_addr = (unsigned long)(ptr); \ unsigned long __gu_val; \ __chk_user_ptr(ptr); \ switch (sizeof(*(ptr))) { \ case 1: \ - __get_user_asm("lbi",__gu_val,__gu_addr,err); \ + __get_user_asm("lbi", __gu_val, (ptr), (err)); \ break; \ case 2: \ - __get_user_asm("lhi",__gu_val,__gu_addr,err); \ + __get_user_asm("lhi", __gu_val, (ptr), (err)); \ break; \ case 4: \ - __get_user_asm("lwi",__gu_val,__gu_addr,err); \ + __get_user_asm("lwi", __gu_val, (ptr), (err)); \ break; \ case 8: \ - __get_user_asm_dword(__gu_val,__gu_addr,err); \ + __get_user_asm_dword(__gu_val, (ptr), (err)); \ break; \ default: \ BUILD_BUG(); \ break; \ } \ - (x) = (__typeof__(*(ptr)))__gu_val; \ + (x) = (__force __typeof__(*(ptr)))__gu_val; \ } while (0) -#define __get_user_asm(inst,x,addr,err) \ - asm volatile( \ - "1: "inst" %1,[%2]\n" \ - "2:\n" \ - " .section .fixup,\"ax\"\n" \ - " .align 2\n" \ - "3: move %0, %3\n" \ - " move %1, #0\n" \ - " b 2b\n" \ - " .previous\n" \ - " .section __ex_table,\"a\"\n" \ - " .align 3\n" \ - " .long 1b, 3b\n" \ - " .previous" \ - : "+r" (err), "=&r" (x) \ - : "r" (addr), "i" (-EFAULT) \ - : "cc") +#define __get_user_asm(inst, x, addr, err) \ + __asm__ __volatile__ ( \ + "1: "inst" %1,[%2]\n" \ + "2:\n" \ + " .section .fixup,\"ax\"\n" \ + " .align 2\n" \ + "3: move %0, %3\n" \ + " move %1, #0\n" \ + " b 2b\n" \ + " .previous\n" \ + " .section __ex_table,\"a\"\n" \ + " .align 3\n" \ + " .long 1b, 3b\n" \ + " .previous" \ + : "+r" (err), "=&r" (x) \ + : "r" (addr), "i" (-EFAULT) \ + : "cc") #ifdef __NDS32_EB__ #define __gu_reg_oper0 "%H1" @@ -149,61 +152,66 @@ do { \ #endif #define __get_user_asm_dword(x, addr, err) \ - asm volatile( \ - "\n1:\tlwi " __gu_reg_oper0 ",[%2]\n" \ - "\n2:\tlwi " __gu_reg_oper1 ",[%2+4]\n" \ - "3:\n" \ - " .section .fixup,\"ax\"\n" \ - " .align 2\n" \ - "4: move %0, %3\n" \ - " b 3b\n" \ - " .previous\n" \ - " .section __ex_table,\"a\"\n" \ - " .align 3\n" \ - " .long 1b, 4b\n" \ - " .long 2b, 4b\n" \ - " .previous" \ - : "+r"(err), "=&r"(x) \ - : "r"(addr), "i"(-EFAULT) \ - : "cc") -#define put_user(x,p) \ -({ \ - long __e = -EFAULT; \ - if(likely(access_ok(VERIFY_WRITE, p, sizeof(*p)))) { \ - __e = __put_user(x,p); \ - } \ - __e; \ -}) -#define __put_user(x,ptr) \ + __asm__ __volatile__ ( \ + "\n1:\tlwi " __gu_reg_oper0 ",[%2]\n" \ + "\n2:\tlwi " __gu_reg_oper1 ",[%2+4]\n" \ + "3:\n" \ + " .section .fixup,\"ax\"\n" \ + " .align 2\n" \ + "4: move %0, %3\n" \ + " b 3b\n" \ + " .previous\n" \ + " .section __ex_table,\"a\"\n" \ + " .align 3\n" \ + " .long 1b, 4b\n" \ + " .long 2b, 4b\n" \ + " .previous" \ + : "+r"(err), "=&r"(x) \ + : "r"(addr), "i"(-EFAULT) \ + : "cc") + +#define put_user __put_user \ + +#define __put_user(x, ptr) \ ({ \ long __pu_err = 0; \ - __put_user_err((x),(ptr),__pu_err); \ + __put_user_err((x), (ptr), __pu_err); \ __pu_err; \ }) -#define __put_user_error(x,ptr,err) \ +#define __put_user_error(x, ptr, err) \ +({ \ + __put_user_err((x), (ptr), (err)); \ + (void)0; \ +}) + +#define __put_user_check(x, ptr, err) \ ({ \ - __put_user_err((x),(ptr),err); \ - (void) 0; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_WRITE, __p, sizeof(*__p))) { \ + __put_user_err((x), __p, (err)); \ + } else { \ + (err) = -EFAULT; \ + } \ }) -#define __put_user_err(x,ptr,err) \ +#define __put_user_err(x, ptr, err) \ do { \ - unsigned long __pu_addr = (unsigned long)(ptr); \ __typeof__(*(ptr)) __pu_val = (x); \ __chk_user_ptr(ptr); \ switch (sizeof(*(ptr))) { \ case 1: \ - __put_user_asm("sbi",__pu_val,__pu_addr,err); \ + __put_user_asm("sbi", __pu_val, (ptr), (err)); \ break; \ case 2: \ - __put_user_asm("shi",__pu_val,__pu_addr,err); \ + __put_user_asm("shi", __pu_val, (ptr), (err)); \ break; \ case 4: \ - __put_user_asm("swi",__pu_val,__pu_addr,err); \ + __put_user_asm("swi", __pu_val, (ptr), (err)); \ break; \ case 8: \ - __put_user_asm_dword(__pu_val,__pu_addr,err); \ + __put_user_asm_dword(__pu_val, (ptr), (err)); \ break; \ default: \ BUILD_BUG(); \ @@ -211,22 +219,22 @@ do { \ } \ } while (0) -#define __put_user_asm(inst,x,addr,err) \ - asm volatile( \ - "1: "inst" %1,[%2]\n" \ - "2:\n" \ - " .section .fixup,\"ax\"\n" \ - " .align 2\n" \ - "3: move %0, %3\n" \ - " b 2b\n" \ - " .previous\n" \ - " .section __ex_table,\"a\"\n" \ - " .align 3\n" \ - " .long 1b, 3b\n" \ - " .previous" \ - : "+r" (err) \ - : "r" (x), "r" (addr), "i" (-EFAULT) \ - : "cc") +#define __put_user_asm(inst, x, addr, err) \ + __asm__ __volatile__ ( \ + "1: "inst" %1,[%2]\n" \ + "2:\n" \ + " .section .fixup,\"ax\"\n" \ + " .align 2\n" \ + "3: move %0, %3\n" \ + " b 2b\n" \ + " .previous\n" \ + " .section __ex_table,\"a\"\n" \ + " .align 3\n" \ + " .long 1b, 3b\n" \ + " .previous" \ + : "+r" (err) \ + : "r" (x), "r" (addr), "i" (-EFAULT) \ + : "cc") #ifdef __NDS32_EB__ #define __pu_reg_oper0 "%H2" @@ -237,23 +245,24 @@ do { \ #endif #define __put_user_asm_dword(x, addr, err) \ - asm volatile( \ - "\n1:\tswi " __pu_reg_oper0 ",[%1]\n" \ - "\n2:\tswi " __pu_reg_oper1 ",[%1+4]\n" \ - "3:\n" \ - " .section .fixup,\"ax\"\n" \ - " .align 2\n" \ - "4: move %0, %3\n" \ - " b 3b\n" \ - " .previous\n" \ - " .section __ex_table,\"a\"\n" \ - " .align 3\n" \ - " .long 1b, 4b\n" \ - " .long 2b, 4b\n" \ - " .previous" \ - : "+r"(err) \ - : "r"(addr), "r"(x), "i"(-EFAULT) \ - : "cc") + __asm__ __volatile__ ( \ + "\n1:\tswi " __pu_reg_oper0 ",[%1]\n" \ + "\n2:\tswi " __pu_reg_oper1 ",[%1+4]\n" \ + "3:\n" \ + " .section .fixup,\"ax\"\n" \ + " .align 2\n" \ + "4: move %0, %3\n" \ + " b 3b\n" \ + " .previous\n" \ + " .section __ex_table,\"a\"\n" \ + " .align 3\n" \ + " .long 1b, 4b\n" \ + " .long 2b, 4b\n" \ + " .previous" \ + : "+r"(err) \ + : "r"(addr), "r"(x), "i"(-EFAULT) \ + : "cc") + extern unsigned long __arch_clear_user(void __user * addr, unsigned long n); extern long strncpy_from_user(char *dest, const char __user * src, long count); extern __must_check long strlen_user(const char __user * str); diff --git a/arch/nds32/kernel/Makefile b/arch/nds32/kernel/Makefile index 42792743e8b9..27cded39fa66 100644 --- a/arch/nds32/kernel/Makefile +++ b/arch/nds32/kernel/Makefile @@ -21,3 +21,9 @@ extra-y := head.o vmlinux.lds obj-y += vdso/ + +obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o + +ifdef CONFIG_FUNCTION_TRACER +CFLAGS_REMOVE_ftrace.o = $(CC_FLAGS_FTRACE) +endif diff --git a/arch/nds32/kernel/atl2c.c b/arch/nds32/kernel/atl2c.c index 0c6d031a1c4a..0c5386e72098 100644 --- a/arch/nds32/kernel/atl2c.c +++ b/arch/nds32/kernel/atl2c.c @@ -9,7 +9,8 @@ void __iomem *atl2c_base; static const struct of_device_id atl2c_ids[] __initconst = { - {.compatible = "andestech,atl2c",} + {.compatible = "andestech,atl2c",}, + {} }; static int __init atl2c_of_init(void) diff --git a/arch/nds32/kernel/ex-entry.S b/arch/nds32/kernel/ex-entry.S index b8ae4e9a6b93..21a144071566 100644 --- a/arch/nds32/kernel/ex-entry.S +++ b/arch/nds32/kernel/ex-entry.S @@ -118,7 +118,7 @@ common_exception_handler: /* interrupt */ 2: #ifdef CONFIG_TRACE_IRQFLAGS - jal trace_hardirqs_off + jal __trace_hardirqs_off #endif move $r0, $sp sethi $lp, hi20(ret_from_intr) diff --git a/arch/nds32/kernel/ex-exit.S b/arch/nds32/kernel/ex-exit.S index 03e4f7788a18..f00af92f7e22 100644 --- a/arch/nds32/kernel/ex-exit.S +++ b/arch/nds32/kernel/ex-exit.S @@ -138,8 +138,8 @@ no_work_pending: #ifdef CONFIG_TRACE_IRQFLAGS lwi $p0, [$sp+(#IPSW_OFFSET)] andi $p0, $p0, #0x1 - la $r10, trace_hardirqs_off - la $r9, trace_hardirqs_on + la $r10, __trace_hardirqs_off + la $r9, __trace_hardirqs_on cmovz $r9, $p0, $r10 jral $r9 #endif diff --git a/arch/nds32/kernel/ftrace.c b/arch/nds32/kernel/ftrace.c new file mode 100644 index 000000000000..a0a9679ad5de --- /dev/null +++ b/arch/nds32/kernel/ftrace.c @@ -0,0 +1,309 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/ftrace.h> +#include <linux/uaccess.h> +#include <asm/cacheflush.h> + +#ifndef CONFIG_DYNAMIC_FTRACE +extern void (*ftrace_trace_function)(unsigned long, unsigned long, + struct ftrace_ops*, struct pt_regs*); +extern int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace); +extern void ftrace_graph_caller(void); + +noinline void __naked ftrace_stub(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct pt_regs *regs) +{ + __asm__ (""); /* avoid to optimize as pure function */ +} + +noinline void _mcount(unsigned long parent_ip) +{ + /* save all state by the compiler prologue */ + + unsigned long ip = (unsigned long)__builtin_return_address(0); + + if (ftrace_trace_function != ftrace_stub) + ftrace_trace_function(ip - MCOUNT_INSN_SIZE, parent_ip, + NULL, NULL); + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + if (ftrace_graph_return != (trace_func_graph_ret_t)ftrace_stub + || ftrace_graph_entry != ftrace_graph_entry_stub) + ftrace_graph_caller(); +#endif + + /* restore all state by the compiler epilogue */ +} +EXPORT_SYMBOL(_mcount); + +#else /* CONFIG_DYNAMIC_FTRACE */ + +noinline void __naked ftrace_stub(unsigned long ip, unsigned long parent_ip, + struct ftrace_ops *op, struct pt_regs *regs) +{ + __asm__ (""); /* avoid to optimize as pure function */ +} + +noinline void __naked _mcount(unsigned long parent_ip) +{ + __asm__ (""); /* avoid to optimize as pure function */ +} +EXPORT_SYMBOL(_mcount); + +#define XSTR(s) STR(s) +#define STR(s) #s +void _ftrace_caller(unsigned long parent_ip) +{ + /* save all state needed by the compiler prologue */ + + /* + * prepare arguments for real tracing function + * first arg : __builtin_return_address(0) - MCOUNT_INSN_SIZE + * second arg : parent_ip + */ + __asm__ __volatile__ ( + "move $r1, %0 \n\t" + "addi $r0, %1, #-" XSTR(MCOUNT_INSN_SIZE) "\n\t" + : + : "r" (parent_ip), "r" (__builtin_return_address(0))); + + /* a placeholder for the call to a real tracing function */ + __asm__ __volatile__ ( + "ftrace_call: \n\t" + "nop \n\t" + "nop \n\t" + "nop \n\t"); + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER + /* a placeholder for the call to ftrace_graph_caller */ + __asm__ __volatile__ ( + "ftrace_graph_call: \n\t" + "nop \n\t" + "nop \n\t" + "nop \n\t"); +#endif + /* restore all state needed by the compiler epilogue */ +} + +int __init ftrace_dyn_arch_init(void) +{ + return 0; +} + +int ftrace_arch_code_modify_prepare(void) +{ + set_all_modules_text_rw(); + return 0; +} + +int ftrace_arch_code_modify_post_process(void) +{ + set_all_modules_text_ro(); + return 0; +} + +static unsigned long gen_sethi_insn(unsigned long addr) +{ + unsigned long opcode = 0x46000000; + unsigned long imm = addr >> 12; + unsigned long rt_num = 0xf << 20; + + return ENDIAN_CONVERT(opcode | rt_num | imm); +} + +static unsigned long gen_ori_insn(unsigned long addr) +{ + unsigned long opcode = 0x58000000; + unsigned long imm = addr & 0x0000fff; + unsigned long rt_num = 0xf << 20; + unsigned long ra_num = 0xf << 15; + + return ENDIAN_CONVERT(opcode | rt_num | ra_num | imm); +} + +static unsigned long gen_jral_insn(unsigned long addr) +{ + unsigned long opcode = 0x4a000001; + unsigned long rt_num = 0x1e << 20; + unsigned long rb_num = 0xf << 10; + + return ENDIAN_CONVERT(opcode | rt_num | rb_num); +} + +static void ftrace_gen_call_insn(unsigned long *call_insns, + unsigned long addr) +{ + call_insns[0] = gen_sethi_insn(addr); /* sethi $r15, imm20u */ + call_insns[1] = gen_ori_insn(addr); /* ori $r15, $r15, imm15u */ + call_insns[2] = gen_jral_insn(addr); /* jral $lp, $r15 */ +} + +static int __ftrace_modify_code(unsigned long pc, unsigned long *old_insn, + unsigned long *new_insn, bool validate) +{ + unsigned long orig_insn[3]; + + if (validate) { + if (probe_kernel_read(orig_insn, (void *)pc, MCOUNT_INSN_SIZE)) + return -EFAULT; + if (memcmp(orig_insn, old_insn, MCOUNT_INSN_SIZE)) + return -EINVAL; + } + + if (probe_kernel_write((void *)pc, new_insn, MCOUNT_INSN_SIZE)) + return -EPERM; + + return 0; +} + +static int ftrace_modify_code(unsigned long pc, unsigned long *old_insn, + unsigned long *new_insn, bool validate) +{ + int ret; + + ret = __ftrace_modify_code(pc, old_insn, new_insn, validate); + if (ret) + return ret; + + flush_icache_range(pc, pc + MCOUNT_INSN_SIZE); + + return ret; +} + +int ftrace_update_ftrace_func(ftrace_func_t func) +{ + unsigned long pc = (unsigned long)&ftrace_call; + unsigned long old_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + unsigned long new_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + + if (func != ftrace_stub) + ftrace_gen_call_insn(new_insn, (unsigned long)func); + + return ftrace_modify_code(pc, old_insn, new_insn, false); +} + +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) +{ + unsigned long pc = rec->ip; + unsigned long nop_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + unsigned long call_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + + ftrace_gen_call_insn(call_insn, addr); + + return ftrace_modify_code(pc, nop_insn, call_insn, true); +} + +int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, + unsigned long addr) +{ + unsigned long pc = rec->ip; + unsigned long nop_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + unsigned long call_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + + ftrace_gen_call_insn(call_insn, addr); + + return ftrace_modify_code(pc, call_insn, nop_insn, true); +} +#endif /* CONFIG_DYNAMIC_FTRACE */ + +#ifdef CONFIG_FUNCTION_GRAPH_TRACER +void prepare_ftrace_return(unsigned long *parent, unsigned long self_addr, + unsigned long frame_pointer) +{ + unsigned long return_hooker = (unsigned long)&return_to_handler; + struct ftrace_graph_ent trace; + unsigned long old; + int err; + + if (unlikely(atomic_read(¤t->tracing_graph_pause))) + return; + + old = *parent; + + trace.func = self_addr; + trace.depth = current->curr_ret_stack + 1; + + /* Only trace if the calling function expects to */ + if (!ftrace_graph_entry(&trace)) + return; + + err = ftrace_push_return_trace(old, self_addr, &trace.depth, + frame_pointer, NULL); + + if (err == -EBUSY) + return; + + *parent = return_hooker; +} + +noinline void ftrace_graph_caller(void) +{ + unsigned long *parent_ip = + (unsigned long *)(__builtin_frame_address(2) - 4); + + unsigned long selfpc = + (unsigned long)(__builtin_return_address(1) - MCOUNT_INSN_SIZE); + + unsigned long frame_pointer = + (unsigned long)__builtin_frame_address(3); + + prepare_ftrace_return(parent_ip, selfpc, frame_pointer); +} + +extern unsigned long ftrace_return_to_handler(unsigned long frame_pointer); +void __naked return_to_handler(void) +{ + __asm__ __volatile__ ( + /* save state needed by the ABI */ + "smw.adm $r0,[$sp],$r1,#0x0 \n\t" + + /* get original return address */ + "move $r0, $fp \n\t" + "bal ftrace_return_to_handler\n\t" + "move $lp, $r0 \n\t" + + /* restore state nedded by the ABI */ + "lmw.bim $r0,[$sp],$r1,#0x0 \n\t"); +} + +#ifdef CONFIG_DYNAMIC_FTRACE +extern unsigned long ftrace_graph_call; + +static int ftrace_modify_graph_caller(bool enable) +{ + unsigned long pc = (unsigned long)&ftrace_graph_call; + unsigned long nop_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + unsigned long call_insn[3] = {INSN_NOP, INSN_NOP, INSN_NOP}; + + ftrace_gen_call_insn(call_insn, (unsigned long)ftrace_graph_caller); + + if (enable) + return ftrace_modify_code(pc, nop_insn, call_insn, true); + else + return ftrace_modify_code(pc, call_insn, nop_insn, true); +} + +int ftrace_enable_ftrace_graph_caller(void) +{ + return ftrace_modify_graph_caller(true); +} + +int ftrace_disable_ftrace_graph_caller(void) +{ + return ftrace_modify_graph_caller(false); +} +#endif /* CONFIG_DYNAMIC_FTRACE */ + +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */ + + +#ifdef CONFIG_TRACE_IRQFLAGS +noinline void __trace_hardirqs_off(void) +{ + trace_hardirqs_off(); +} +noinline void __trace_hardirqs_on(void) +{ + trace_hardirqs_on(); +} +#endif /* CONFIG_TRACE_IRQFLAGS */ diff --git a/arch/nds32/kernel/module.c b/arch/nds32/kernel/module.c index 4167283d8293..1e31829cbc2a 100644 --- a/arch/nds32/kernel/module.c +++ b/arch/nds32/kernel/module.c @@ -40,7 +40,7 @@ void do_reloc16(unsigned int val, unsigned int *loc, unsigned int val_mask, tmp2 = tmp & loc_mask; if (partial_in_place) { - tmp &= (!loc_mask); + tmp &= (~loc_mask); tmp = tmp2 | ((tmp + ((val & val_mask) >> val_shift)) & val_mask); } else { @@ -70,7 +70,7 @@ void do_reloc32(unsigned int val, unsigned int *loc, unsigned int val_mask, tmp2 = tmp & loc_mask; if (partial_in_place) { - tmp &= (!loc_mask); + tmp &= (~loc_mask); tmp = tmp2 | ((tmp + ((val & val_mask) >> val_shift)) & val_mask); } else { diff --git a/arch/nds32/kernel/stacktrace.c b/arch/nds32/kernel/stacktrace.c index 8b231e910ea6..d974c0c1c65f 100644 --- a/arch/nds32/kernel/stacktrace.c +++ b/arch/nds32/kernel/stacktrace.c @@ -4,6 +4,7 @@ #include <linux/sched/debug.h> #include <linux/sched/task_stack.h> #include <linux/stacktrace.h> +#include <linux/ftrace.h> void save_stack_trace(struct stack_trace *trace) { @@ -16,6 +17,7 @@ void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace) unsigned long *fpn; int skip = trace->skip; int savesched; + int graph_idx = 0; if (tsk == current) { __asm__ __volatile__("\tori\t%0, $fp, #0\n":"=r"(fpn)); @@ -29,10 +31,12 @@ void save_stack_trace_tsk(struct task_struct *tsk, struct stack_trace *trace) && (fpn >= (unsigned long *)TASK_SIZE)) { unsigned long lpp, fpp; - lpp = fpn[-1]; + lpp = fpn[LP_OFFSET]; fpp = fpn[FP_OFFSET]; if (!__kernel_text_address(lpp)) break; + else + lpp = ftrace_graph_ret_addr(tsk, &graph_idx, lpp, NULL); if (savesched || !in_sched_functions(lpp)) { if (skip) { diff --git a/arch/nds32/kernel/traps.c b/arch/nds32/kernel/traps.c index a6205fd4db52..1496aab48998 100644 --- a/arch/nds32/kernel/traps.c +++ b/arch/nds32/kernel/traps.c @@ -8,6 +8,7 @@ #include <linux/kdebug.h> #include <linux/sched/task_stack.h> #include <linux/uaccess.h> +#include <linux/ftrace.h> #include <asm/proc-fns.h> #include <asm/unistd.h> @@ -94,28 +95,6 @@ static void dump_instr(struct pt_regs *regs) set_fs(fs); } -#ifdef CONFIG_FUNCTION_GRAPH_TRACER -#include <linux/ftrace.h> -static void -get_real_ret_addr(unsigned long *addr, struct task_struct *tsk, int *graph) -{ - if (*addr == (unsigned long)return_to_handler) { - int index = tsk->curr_ret_stack; - - if (tsk->ret_stack && index >= *graph) { - index -= *graph; - *addr = tsk->ret_stack[index].ret; - (*graph)++; - } - } -} -#else -static inline void -get_real_ret_addr(unsigned long *addr, struct task_struct *tsk, int *graph) -{ -} -#endif - #define LOOP_TIMES (100) static void __dump(struct task_struct *tsk, unsigned long *base_reg) { @@ -126,7 +105,8 @@ static void __dump(struct task_struct *tsk, unsigned long *base_reg) while (!kstack_end(base_reg)) { ret_addr = *base_reg++; if (__kernel_text_address(ret_addr)) { - get_real_ret_addr(&ret_addr, tsk, &graph); + ret_addr = ftrace_graph_ret_addr( + tsk, &graph, ret_addr, NULL); print_ip_sym(ret_addr); } if (--cnt < 0) @@ -137,15 +117,12 @@ static void __dump(struct task_struct *tsk, unsigned long *base_reg) !((unsigned long)base_reg & 0x3) && ((unsigned long)base_reg >= TASK_SIZE)) { unsigned long next_fp; -#if !defined(NDS32_ABI_2) - ret_addr = base_reg[0]; - next_fp = base_reg[1]; -#else - ret_addr = base_reg[-1]; + ret_addr = base_reg[LP_OFFSET]; next_fp = base_reg[FP_OFFSET]; -#endif if (__kernel_text_address(ret_addr)) { - get_real_ret_addr(&ret_addr, tsk, &graph); + + ret_addr = ftrace_graph_ret_addr( + tsk, &graph, ret_addr, NULL); print_ip_sym(ret_addr); } if (--cnt < 0) @@ -196,11 +173,10 @@ void die(const char *str, struct pt_regs *regs, int err) pr_emerg("CPU: %i\n", smp_processor_id()); show_regs(regs); pr_emerg("Process %s (pid: %d, stack limit = 0x%p)\n", - tsk->comm, tsk->pid, task_thread_info(tsk) + 1); + tsk->comm, tsk->pid, end_of_stack(tsk)); if (!user_mode(regs) || in_interrupt()) { - dump_mem("Stack: ", regs->sp, - THREAD_SIZE + (unsigned long)task_thread_info(tsk)); + dump_mem("Stack: ", regs->sp, (regs->sp + PAGE_SIZE) & PAGE_MASK); dump_instr(regs); dump_stack(); } diff --git a/arch/nds32/kernel/vmlinux.lds.S b/arch/nds32/kernel/vmlinux.lds.S index 288313b886ef..9e90f30a181d 100644 --- a/arch/nds32/kernel/vmlinux.lds.S +++ b/arch/nds32/kernel/vmlinux.lds.S @@ -13,14 +13,26 @@ OUTPUT_ARCH(nds32) ENTRY(_stext_lma) jiffies = jiffies_64; +#if defined(CONFIG_GCOV_KERNEL) +#define NDS32_EXIT_KEEP(x) x +#else +#define NDS32_EXIT_KEEP(x) +#endif + SECTIONS { _stext_lma = TEXTADDR - LOAD_OFFSET; . = TEXTADDR; __init_begin = .; HEAD_TEXT_SECTION + .exit.text : { + NDS32_EXIT_KEEP(EXIT_TEXT) + } INIT_TEXT_SECTION(PAGE_SIZE) INIT_DATA_SECTION(16) + .exit.data : { + NDS32_EXIT_KEEP(EXIT_DATA) + } PERCPU_SECTION(L1_CACHE_BYTES) __init_end = .; diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h index 13a688fc8cd0..2fdc865ca374 100644 --- a/arch/powerpc/include/asm/book3s/64/pgtable.h +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h @@ -1051,7 +1051,6 @@ static inline void vmemmap_remove_mapping(unsigned long start, return hash__vmemmap_remove_mapping(start, page_size); } #endif -struct page *realmode_pfn_to_page(unsigned long pfn); static inline pte_t pmd_pte(pmd_t pmd) { diff --git a/arch/powerpc/include/asm/iommu.h b/arch/powerpc/include/asm/iommu.h index ab3a4fba38e3..3d4b88cb8599 100644 --- a/arch/powerpc/include/asm/iommu.h +++ b/arch/powerpc/include/asm/iommu.h @@ -220,8 +220,6 @@ extern void iommu_del_device(struct device *dev); extern int __init tce_iommu_bus_notifier_init(void); extern long iommu_tce_xchg(struct iommu_table *tbl, unsigned long entry, unsigned long *hpa, enum dma_data_direction *direction); -extern long iommu_tce_xchg_rm(struct iommu_table *tbl, unsigned long entry, - unsigned long *hpa, enum dma_data_direction *direction); #else static inline void iommu_register_group(struct iommu_table_group *table_group, int pci_domain_number, diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h index b2f89b621b15..b694d6af1150 100644 --- a/arch/powerpc/include/asm/mmu_context.h +++ b/arch/powerpc/include/asm/mmu_context.h @@ -38,6 +38,7 @@ extern long mm_iommu_ua_to_hpa(struct mm_iommu_table_group_mem_t *mem, unsigned long ua, unsigned int pageshift, unsigned long *hpa); extern long mm_iommu_ua_to_hpa_rm(struct mm_iommu_table_group_mem_t *mem, unsigned long ua, unsigned int pageshift, unsigned long *hpa); +extern void mm_iommu_ua_mark_dirty_rm(struct mm_struct *mm, unsigned long ua); extern long mm_iommu_mapped_inc(struct mm_iommu_table_group_mem_t *mem); extern void mm_iommu_mapped_dec(struct mm_iommu_table_group_mem_t *mem); #endif diff --git a/arch/powerpc/include/asm/setup.h b/arch/powerpc/include/asm/setup.h index 1a951b00465d..1fffbba8d6a5 100644 --- a/arch/powerpc/include/asm/setup.h +++ b/arch/powerpc/include/asm/setup.h @@ -9,6 +9,7 @@ extern void ppc_printk_progress(char *s, unsigned short hex); extern unsigned int rtas_data; extern unsigned long long memory_limit; +extern bool init_mem_is_free; extern unsigned long klimit; extern void *zalloc_maybe_bootmem(size_t size, gfp_t mask); diff --git a/arch/powerpc/kernel/exceptions-64s.S b/arch/powerpc/kernel/exceptions-64s.S index ea04dfb8c092..2d8fc8c9da7a 100644 --- a/arch/powerpc/kernel/exceptions-64s.S +++ b/arch/powerpc/kernel/exceptions-64s.S @@ -1314,9 +1314,7 @@ EXC_REAL_BEGIN(denorm_exception_hv, 0x1500, 0x100) #ifdef CONFIG_PPC_DENORMALISATION mfspr r10,SPRN_HSRR1 - mfspr r11,SPRN_HSRR0 /* save HSRR0 */ andis. r10,r10,(HSRR1_DENORM)@h /* denorm? */ - addi r11,r11,-4 /* HSRR0 is next instruction */ bne+ denorm_assist #endif @@ -1382,6 +1380,8 @@ END_FTR_SECTION_IFCLR(CPU_FTR_ARCH_207S) */ XVCPSGNDP32(32) denorm_done: + mfspr r11,SPRN_HSRR0 + subi r11,r11,4 mtspr SPRN_HSRR0,r11 mtcrf 0x80,r9 ld r9,PACA_EXGEN+EX_R9(r13) diff --git a/arch/powerpc/kernel/iommu.c b/arch/powerpc/kernel/iommu.c index af7a20dc6e09..19b4c628f3be 100644 --- a/arch/powerpc/kernel/iommu.c +++ b/arch/powerpc/kernel/iommu.c @@ -1013,31 +1013,6 @@ long iommu_tce_xchg(struct iommu_table *tbl, unsigned long entry, } EXPORT_SYMBOL_GPL(iommu_tce_xchg); -#ifdef CONFIG_PPC_BOOK3S_64 -long iommu_tce_xchg_rm(struct iommu_table *tbl, unsigned long entry, - unsigned long *hpa, enum dma_data_direction *direction) -{ - long ret; - - ret = tbl->it_ops->exchange_rm(tbl, entry, hpa, direction); - - if (!ret && ((*direction == DMA_FROM_DEVICE) || - (*direction == DMA_BIDIRECTIONAL))) { - struct page *pg = realmode_pfn_to_page(*hpa >> PAGE_SHIFT); - - if (likely(pg)) { - SetPageDirty(pg); - } else { - tbl->it_ops->exchange_rm(tbl, entry, hpa, direction); - ret = -EFAULT; - } - } - - return ret; -} -EXPORT_SYMBOL_GPL(iommu_tce_xchg_rm); -#endif - int iommu_take_ownership(struct iommu_table *tbl) { unsigned long flags, i, sz = (tbl->it_size + 7) >> 3; diff --git a/arch/powerpc/kernel/tm.S b/arch/powerpc/kernel/tm.S index 6bffbc5affe7..7716374786bd 100644 --- a/arch/powerpc/kernel/tm.S +++ b/arch/powerpc/kernel/tm.S @@ -176,13 +176,27 @@ _GLOBAL(tm_reclaim) std r1, PACATMSCRATCH(r13) ld r1, PACAR1(r13) - /* Store the PPR in r11 and reset to decent value */ std r11, GPR11(r1) /* Temporary stash */ + /* + * Move the saved user r1 to the kernel stack in case PACATMSCRATCH is + * clobbered by an exception once we turn on MSR_RI below. + */ + ld r11, PACATMSCRATCH(r13) + std r11, GPR1(r1) + + /* + * Store r13 away so we can free up the scratch SPR for the SLB fault + * handler (needed once we start accessing the thread_struct). + */ + GET_SCRATCH0(r11) + std r11, GPR13(r1) + /* Reset MSR RI so we can take SLB faults again */ li r11, MSR_RI mtmsrd r11, 1 + /* Store the PPR in r11 and reset to decent value */ mfspr r11, SPRN_PPR HMT_MEDIUM @@ -207,11 +221,11 @@ _GLOBAL(tm_reclaim) SAVE_GPR(8, r7) /* user r8 */ SAVE_GPR(9, r7) /* user r9 */ SAVE_GPR(10, r7) /* user r10 */ - ld r3, PACATMSCRATCH(r13) /* user r1 */ + ld r3, GPR1(r1) /* user r1 */ ld r4, GPR7(r1) /* user r7 */ ld r5, GPR11(r1) /* user r11 */ ld r6, GPR12(r1) /* user r12 */ - GET_SCRATCH0(8) /* user r13 */ + ld r8, GPR13(r1) /* user r13 */ std r3, GPR1(r7) std r4, GPR7(r7) std r5, GPR11(r7) diff --git a/arch/powerpc/kvm/book3s_64_mmu_hv.c b/arch/powerpc/kvm/book3s_64_mmu_hv.c index 3c0e8fb2b773..68e14afecac8 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_hv.c +++ b/arch/powerpc/kvm/book3s_64_mmu_hv.c @@ -358,7 +358,7 @@ static int kvmppc_mmu_book3s_64_hv_xlate(struct kvm_vcpu *vcpu, gva_t eaddr, unsigned long pp, key; unsigned long v, orig_v, gr; __be64 *hptep; - int index; + long int index; int virtmode = vcpu->arch.shregs.msr & (data ? MSR_DR : MSR_IR); if (kvm_is_radix(vcpu->kvm)) diff --git a/arch/powerpc/kvm/book3s_64_mmu_radix.c b/arch/powerpc/kvm/book3s_64_mmu_radix.c index 0af1c0aea1fe..933c574e1cf7 100644 --- a/arch/powerpc/kvm/book3s_64_mmu_radix.c +++ b/arch/powerpc/kvm/book3s_64_mmu_radix.c @@ -525,8 +525,8 @@ int kvmppc_book3s_radix_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, unsigned long ea, unsigned long dsisr) { struct kvm *kvm = vcpu->kvm; - unsigned long mmu_seq, pte_size; - unsigned long gpa, gfn, hva, pfn; + unsigned long mmu_seq; + unsigned long gpa, gfn, hva; struct kvm_memory_slot *memslot; struct page *page = NULL; long ret; @@ -623,9 +623,10 @@ int kvmppc_book3s_radix_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, */ hva = gfn_to_hva_memslot(memslot, gfn); if (upgrade_p && __get_user_pages_fast(hva, 1, 1, &page) == 1) { - pfn = page_to_pfn(page); upgrade_write = true; } else { + unsigned long pfn; + /* Call KVM generic code to do the slow-path check */ pfn = __gfn_to_pfn_memslot(memslot, gfn, false, NULL, writing, upgrade_p); @@ -639,63 +640,45 @@ int kvmppc_book3s_radix_page_fault(struct kvm_run *run, struct kvm_vcpu *vcpu, } } - /* See if we can insert a 1GB or 2MB large PTE here */ - level = 0; - if (page && PageCompound(page)) { - pte_size = PAGE_SIZE << compound_order(compound_head(page)); - if (pte_size >= PUD_SIZE && - (gpa & (PUD_SIZE - PAGE_SIZE)) == - (hva & (PUD_SIZE - PAGE_SIZE))) { - level = 2; - pfn &= ~((PUD_SIZE >> PAGE_SHIFT) - 1); - } else if (pte_size >= PMD_SIZE && - (gpa & (PMD_SIZE - PAGE_SIZE)) == - (hva & (PMD_SIZE - PAGE_SIZE))) { - level = 1; - pfn &= ~((PMD_SIZE >> PAGE_SHIFT) - 1); - } - } - /* - * Compute the PTE value that we need to insert. + * Read the PTE from the process' radix tree and use that + * so we get the shift and attribute bits. */ - if (page) { - pgflags = _PAGE_READ | _PAGE_EXEC | _PAGE_PRESENT | _PAGE_PTE | - _PAGE_ACCESSED; - if (writing || upgrade_write) - pgflags |= _PAGE_WRITE | _PAGE_DIRTY; - pte = pfn_pte(pfn, __pgprot(pgflags)); + local_irq_disable(); + ptep = __find_linux_pte(vcpu->arch.pgdir, hva, NULL, &shift); + pte = *ptep; + local_irq_enable(); + + /* Get pte level from shift/size */ + if (shift == PUD_SHIFT && + (gpa & (PUD_SIZE - PAGE_SIZE)) == + (hva & (PUD_SIZE - PAGE_SIZE))) { + level = 2; + } else if (shift == PMD_SHIFT && + (gpa & (PMD_SIZE - PAGE_SIZE)) == + (hva & (PMD_SIZE - PAGE_SIZE))) { + level = 1; } else { - /* - * Read the PTE from the process' radix tree and use that - * so we get the attribute bits. - */ - local_irq_disable(); - ptep = __find_linux_pte(vcpu->arch.pgdir, hva, NULL, &shift); - pte = *ptep; - local_irq_enable(); - if (shift == PUD_SHIFT && - (gpa & (PUD_SIZE - PAGE_SIZE)) == - (hva & (PUD_SIZE - PAGE_SIZE))) { - level = 2; - } else if (shift == PMD_SHIFT && - (gpa & (PMD_SIZE - PAGE_SIZE)) == - (hva & (PMD_SIZE - PAGE_SIZE))) { - level = 1; - } else if (shift && shift != PAGE_SHIFT) { - /* Adjust PFN */ - unsigned long mask = (1ul << shift) - PAGE_SIZE; - pte = __pte(pte_val(pte) | (hva & mask)); - } - pte = __pte(pte_val(pte) | _PAGE_EXEC | _PAGE_ACCESSED); - if (writing || upgrade_write) { - if (pte_val(pte) & _PAGE_WRITE) - pte = __pte(pte_val(pte) | _PAGE_DIRTY); - } else { - pte = __pte(pte_val(pte) & ~(_PAGE_WRITE | _PAGE_DIRTY)); + level = 0; + if (shift > PAGE_SHIFT) { + /* + * If the pte maps more than one page, bring over + * bits from the virtual address to get the real + * address of the specific single page we want. + */ + unsigned long rpnmask = (1ul << shift) - PAGE_SIZE; + pte = __pte(pte_val(pte) | (hva & rpnmask)); } } + pte = __pte(pte_val(pte) | _PAGE_EXEC | _PAGE_ACCESSED); + if (writing || upgrade_write) { + if (pte_val(pte) & _PAGE_WRITE) + pte = __pte(pte_val(pte) | _PAGE_DIRTY); + } else { + pte = __pte(pte_val(pte) & ~(_PAGE_WRITE | _PAGE_DIRTY)); + } + /* Allocate space in the tree and write the PTE */ ret = kvmppc_create_pte(kvm, pte, gpa, level, mmu_seq); @@ -725,10 +708,10 @@ int kvm_unmap_radix(struct kvm *kvm, struct kvm_memory_slot *memslot, gpa, shift); kvmppc_radix_tlbie_page(kvm, gpa, shift); if ((old & _PAGE_DIRTY) && memslot->dirty_bitmap) { - unsigned long npages = 1; + unsigned long psize = PAGE_SIZE; if (shift) - npages = 1ul << (shift - PAGE_SHIFT); - kvmppc_update_dirty_map(memslot, gfn, npages); + psize = 1ul << shift; + kvmppc_update_dirty_map(memslot, gfn, psize); } } return 0; diff --git a/arch/powerpc/kvm/book3s_64_vio_hv.c b/arch/powerpc/kvm/book3s_64_vio_hv.c index 506a4d400458..6821ead4b4eb 100644 --- a/arch/powerpc/kvm/book3s_64_vio_hv.c +++ b/arch/powerpc/kvm/book3s_64_vio_hv.c @@ -187,12 +187,35 @@ long kvmppc_gpa_to_ua(struct kvm *kvm, unsigned long gpa, EXPORT_SYMBOL_GPL(kvmppc_gpa_to_ua); #ifdef CONFIG_KVM_BOOK3S_HV_POSSIBLE -static void kvmppc_rm_clear_tce(struct iommu_table *tbl, unsigned long entry) +static long iommu_tce_xchg_rm(struct mm_struct *mm, struct iommu_table *tbl, + unsigned long entry, unsigned long *hpa, + enum dma_data_direction *direction) +{ + long ret; + + ret = tbl->it_ops->exchange_rm(tbl, entry, hpa, direction); + + if (!ret && ((*direction == DMA_FROM_DEVICE) || + (*direction == DMA_BIDIRECTIONAL))) { + __be64 *pua = IOMMU_TABLE_USERSPACE_ENTRY_RM(tbl, entry); + /* + * kvmppc_rm_tce_iommu_do_map() updates the UA cache after + * calling this so we still get here a valid UA. + */ + if (pua && *pua) + mm_iommu_ua_mark_dirty_rm(mm, be64_to_cpu(*pua)); + } + + return ret; +} + +static void kvmppc_rm_clear_tce(struct kvm *kvm, struct iommu_table *tbl, + unsigned long entry) { unsigned long hpa = 0; enum dma_data_direction dir = DMA_NONE; - iommu_tce_xchg_rm(tbl, entry, &hpa, &dir); + iommu_tce_xchg_rm(kvm->mm, tbl, entry, &hpa, &dir); } static long kvmppc_rm_tce_iommu_mapped_dec(struct kvm *kvm, @@ -224,7 +247,7 @@ static long kvmppc_rm_tce_iommu_do_unmap(struct kvm *kvm, unsigned long hpa = 0; long ret; - if (iommu_tce_xchg_rm(tbl, entry, &hpa, &dir)) + if (iommu_tce_xchg_rm(kvm->mm, tbl, entry, &hpa, &dir)) /* * real mode xchg can fail if struct page crosses * a page boundary @@ -236,7 +259,7 @@ static long kvmppc_rm_tce_iommu_do_unmap(struct kvm *kvm, ret = kvmppc_rm_tce_iommu_mapped_dec(kvm, tbl, entry); if (ret) - iommu_tce_xchg_rm(tbl, entry, &hpa, &dir); + iommu_tce_xchg_rm(kvm->mm, tbl, entry, &hpa, &dir); return ret; } @@ -282,7 +305,7 @@ static long kvmppc_rm_tce_iommu_do_map(struct kvm *kvm, struct iommu_table *tbl, if (WARN_ON_ONCE_RM(mm_iommu_mapped_inc(mem))) return H_CLOSED; - ret = iommu_tce_xchg_rm(tbl, entry, &hpa, &dir); + ret = iommu_tce_xchg_rm(kvm->mm, tbl, entry, &hpa, &dir); if (ret) { mm_iommu_mapped_dec(mem); /* @@ -371,7 +394,7 @@ long kvmppc_rm_h_put_tce(struct kvm_vcpu *vcpu, unsigned long liobn, return ret; WARN_ON_ONCE_RM(1); - kvmppc_rm_clear_tce(stit->tbl, entry); + kvmppc_rm_clear_tce(vcpu->kvm, stit->tbl, entry); } kvmppc_tce_put(stt, entry, tce); @@ -520,7 +543,7 @@ long kvmppc_rm_h_put_tce_indirect(struct kvm_vcpu *vcpu, goto unlock_exit; WARN_ON_ONCE_RM(1); - kvmppc_rm_clear_tce(stit->tbl, entry); + kvmppc_rm_clear_tce(vcpu->kvm, stit->tbl, entry); } kvmppc_tce_put(stt, entry + i, tce); @@ -571,7 +594,7 @@ long kvmppc_rm_h_stuff_tce(struct kvm_vcpu *vcpu, return ret; WARN_ON_ONCE_RM(1); - kvmppc_rm_clear_tce(stit->tbl, entry); + kvmppc_rm_clear_tce(vcpu->kvm, stit->tbl, entry); } } diff --git a/arch/powerpc/lib/checksum_64.S b/arch/powerpc/lib/checksum_64.S index 886ed94b9c13..d05c8af4ac51 100644 --- a/arch/powerpc/lib/checksum_64.S +++ b/arch/powerpc/lib/checksum_64.S @@ -443,6 +443,9 @@ _GLOBAL(csum_ipv6_magic) addc r0, r8, r9 ld r10, 0(r4) ld r11, 8(r4) +#ifdef CONFIG_CPU_LITTLE_ENDIAN + rotldi r5, r5, 8 +#endif adde r0, r0, r10 add r5, r5, r7 adde r0, r0, r11 diff --git a/arch/powerpc/lib/code-patching.c b/arch/powerpc/lib/code-patching.c index 850f3b8f4da5..6ae2777c220d 100644 --- a/arch/powerpc/lib/code-patching.c +++ b/arch/powerpc/lib/code-patching.c @@ -28,6 +28,12 @@ static int __patch_instruction(unsigned int *exec_addr, unsigned int instr, { int err; + /* Make sure we aren't patching a freed init section */ + if (init_mem_is_free && init_section_contains(exec_addr, 4)) { + pr_debug("Skipping init section patching addr: 0x%px\n", exec_addr); + return 0; + } + __put_user_size(instr, patch_addr, 4, err); if (err) return err; diff --git a/arch/powerpc/mm/init_64.c b/arch/powerpc/mm/init_64.c index 51ce091914f9..7a9886f98b0c 100644 --- a/arch/powerpc/mm/init_64.c +++ b/arch/powerpc/mm/init_64.c @@ -308,55 +308,6 @@ void register_page_bootmem_memmap(unsigned long section_nr, { } -/* - * We do not have access to the sparsemem vmemmap, so we fallback to - * walking the list of sparsemem blocks which we already maintain for - * the sake of crashdump. In the long run, we might want to maintain - * a tree if performance of that linear walk becomes a problem. - * - * realmode_pfn_to_page functions can fail due to: - * 1) As real sparsemem blocks do not lay in RAM continously (they - * are in virtual address space which is not available in the real mode), - * the requested page struct can be split between blocks so get_page/put_page - * may fail. - * 2) When huge pages are used, the get_page/put_page API will fail - * in real mode as the linked addresses in the page struct are virtual - * too. - */ -struct page *realmode_pfn_to_page(unsigned long pfn) -{ - struct vmemmap_backing *vmem_back; - struct page *page; - unsigned long page_size = 1 << mmu_psize_defs[mmu_vmemmap_psize].shift; - unsigned long pg_va = (unsigned long) pfn_to_page(pfn); - - for (vmem_back = vmemmap_list; vmem_back; vmem_back = vmem_back->list) { - if (pg_va < vmem_back->virt_addr) - continue; - - /* After vmemmap_list entry free is possible, need check all */ - if ((pg_va + sizeof(struct page)) <= - (vmem_back->virt_addr + page_size)) { - page = (struct page *) (vmem_back->phys + pg_va - - vmem_back->virt_addr); - return page; - } - } - - /* Probably that page struct is split between real pages */ - return NULL; -} -EXPORT_SYMBOL_GPL(realmode_pfn_to_page); - -#else - -struct page *realmode_pfn_to_page(unsigned long pfn) -{ - struct page *page = pfn_to_page(pfn); - return page; -} -EXPORT_SYMBOL_GPL(realmode_pfn_to_page); - #endif /* CONFIG_SPARSEMEM_VMEMMAP */ #ifdef CONFIG_PPC_BOOK3S_64 diff --git a/arch/powerpc/mm/mem.c b/arch/powerpc/mm/mem.c index 5c8530d0c611..04ccb274a620 100644 --- a/arch/powerpc/mm/mem.c +++ b/arch/powerpc/mm/mem.c @@ -63,6 +63,7 @@ #endif unsigned long long memory_limit; +bool init_mem_is_free; #ifdef CONFIG_HIGHMEM pte_t *kmap_pte; @@ -396,6 +397,7 @@ void free_initmem(void) { ppc_md.progress = ppc_printk_progress; mark_initmem_nx(); + init_mem_is_free = true; free_initmem_default(POISON_FREE_INITMEM); } diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c index c9ee9e23845f..56c2234cc6ae 100644 --- a/arch/powerpc/mm/mmu_context_iommu.c +++ b/arch/powerpc/mm/mmu_context_iommu.c @@ -18,11 +18,15 @@ #include <linux/migrate.h> #include <linux/hugetlb.h> #include <linux/swap.h> +#include <linux/sizes.h> #include <asm/mmu_context.h> #include <asm/pte-walk.h> static DEFINE_MUTEX(mem_list_mutex); +#define MM_IOMMU_TABLE_GROUP_PAGE_DIRTY 0x1 +#define MM_IOMMU_TABLE_GROUP_PAGE_MASK ~(SZ_4K - 1) + struct mm_iommu_table_group_mem_t { struct list_head next; struct rcu_head rcu; @@ -263,6 +267,9 @@ static void mm_iommu_unpin(struct mm_iommu_table_group_mem_t *mem) if (!page) continue; + if (mem->hpas[i] & MM_IOMMU_TABLE_GROUP_PAGE_DIRTY) + SetPageDirty(page); + put_page(page); mem->hpas[i] = 0; } @@ -360,7 +367,6 @@ struct mm_iommu_table_group_mem_t *mm_iommu_lookup_rm(struct mm_struct *mm, return ret; } -EXPORT_SYMBOL_GPL(mm_iommu_lookup_rm); struct mm_iommu_table_group_mem_t *mm_iommu_find(struct mm_struct *mm, unsigned long ua, unsigned long entries) @@ -390,7 +396,7 @@ long mm_iommu_ua_to_hpa(struct mm_iommu_table_group_mem_t *mem, if (pageshift > mem->pageshift) return -EFAULT; - *hpa = *va | (ua & ~PAGE_MASK); + *hpa = (*va & MM_IOMMU_TABLE_GROUP_PAGE_MASK) | (ua & ~PAGE_MASK); return 0; } @@ -413,11 +419,31 @@ long mm_iommu_ua_to_hpa_rm(struct mm_iommu_table_group_mem_t *mem, if (!pa) return -EFAULT; - *hpa = *pa | (ua & ~PAGE_MASK); + *hpa = (*pa & MM_IOMMU_TABLE_GROUP_PAGE_MASK) | (ua & ~PAGE_MASK); return 0; } -EXPORT_SYMBOL_GPL(mm_iommu_ua_to_hpa_rm); + +extern void mm_iommu_ua_mark_dirty_rm(struct mm_struct *mm, unsigned long ua) +{ + struct mm_iommu_table_group_mem_t *mem; + long entry; + void *va; + unsigned long *pa; + + mem = mm_iommu_lookup_rm(mm, ua, PAGE_SIZE); + if (!mem) + return; + + entry = (ua - mem->ua) >> PAGE_SHIFT; + va = &mem->hpas[entry]; + + pa = (void *) vmalloc_to_phys(va); + if (!pa) + return; + + *pa |= MM_IOMMU_TABLE_GROUP_PAGE_DIRTY; +} long mm_iommu_mapped_inc(struct mm_iommu_table_group_mem_t *mem) { diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index 35ac5422903a..59d07bd5374a 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -1204,7 +1204,9 @@ int find_and_online_cpu_nid(int cpu) int new_nid; /* Use associativity from first thread for all siblings */ - vphn_get_associativity(cpu, associativity); + if (vphn_get_associativity(cpu, associativity)) + return cpu_to_node(cpu); + new_nid = associativity_to_nid(associativity); if (new_nid < 0 || !node_possible(new_nid)) new_nid = first_online_node; @@ -1452,7 +1454,8 @@ static struct timer_list topology_timer; static void reset_topology_timer(void) { - mod_timer(&topology_timer, jiffies + topology_timer_secs * HZ); + if (vphn_enabled) + mod_timer(&topology_timer, jiffies + topology_timer_secs * HZ); } #ifdef CONFIG_SMP diff --git a/arch/powerpc/mm/pkeys.c b/arch/powerpc/mm/pkeys.c index 333b1f80c435..b271b283c785 100644 --- a/arch/powerpc/mm/pkeys.c +++ b/arch/powerpc/mm/pkeys.c @@ -45,7 +45,7 @@ static void scan_pkey_feature(void) * Since any pkey can be used for data or execute, we will just treat * all keys as equal and track them as one entity. */ - pkeys_total = be32_to_cpu(vals[0]); + pkeys_total = vals[0]; pkeys_devtree_defined = true; } diff --git a/arch/powerpc/platforms/powernv/pci-ioda-tce.c b/arch/powerpc/platforms/powernv/pci-ioda-tce.c index 6c5db1acbe8d..fe9691040f54 100644 --- a/arch/powerpc/platforms/powernv/pci-ioda-tce.c +++ b/arch/powerpc/platforms/powernv/pci-ioda-tce.c @@ -276,7 +276,7 @@ long pnv_pci_ioda2_table_alloc_pages(int nid, __u64 bus_offset, level_shift = entries_shift + 3; level_shift = max_t(unsigned int, level_shift, PAGE_SHIFT); - if ((level_shift - 3) * levels + page_shift >= 60) + if ((level_shift - 3) * levels + page_shift >= 55) return -EINVAL; /* Allocate TCE table */ diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h new file mode 100644 index 000000000000..c9fecd120d18 --- /dev/null +++ b/arch/riscv/include/asm/asm-prototypes.h @@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_PROTOTYPES_H + +#include <linux/ftrace.h> +#include <asm-generic/asm-prototypes.h> + +#endif /* _ASM_RISCV_PROTOTYPES_H */ diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c index db20dc630e7e..aee603123030 100644 --- a/arch/riscv/kernel/setup.c +++ b/arch/riscv/kernel/setup.c @@ -85,15 +85,8 @@ atomic_t hart_lottery; #ifdef CONFIG_BLK_DEV_INITRD static void __init setup_initrd(void) { - extern char __initramfs_start[]; - extern unsigned long __initramfs_size; unsigned long size; - if (__initramfs_size > 0) { - initrd_start = (unsigned long)(&__initramfs_start); - initrd_end = initrd_start + __initramfs_size; - } - if (initrd_start >= initrd_end) { printk(KERN_INFO "initrd not found or empty"); goto disable; diff --git a/arch/s390/crypto/paes_s390.c b/arch/s390/crypto/paes_s390.c index 80b27294c1de..ab9a0ebecc19 100644 --- a/arch/s390/crypto/paes_s390.c +++ b/arch/s390/crypto/paes_s390.c @@ -208,7 +208,7 @@ static int cbc_paes_crypt(struct blkcipher_desc *desc, unsigned long modifier, walk->dst.virt.addr, walk->src.virt.addr, n); if (k) ret = blkcipher_walk_done(desc, walk, nbytes - k); - if (n < k) { + if (k < n) { if (__cbc_paes_set_key(ctx) != 0) return blkcipher_walk_done(desc, walk, -EIO); memcpy(param.key, ctx->pk.protkey, MAXPROTKEYSIZE); diff --git a/arch/s390/include/asm/mmu.h b/arch/s390/include/asm/mmu.h index f31a15044c24..a8418e1379eb 100644 --- a/arch/s390/include/asm/mmu.h +++ b/arch/s390/include/asm/mmu.h @@ -16,7 +16,13 @@ typedef struct { unsigned long asce; unsigned long asce_limit; unsigned long vdso_base; - /* The mmu context allocates 4K page tables. */ + /* + * The following bitfields need a down_write on the mm + * semaphore when they are written to. As they are only + * written once, they can be read without a lock. + * + * The mmu context allocates 4K page tables. + */ unsigned int alloc_pgste:1; /* The mmu context uses extended page tables. */ unsigned int has_pgste:1; diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 91ad4a9425c0..ac5da6b0b862 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -481,7 +481,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) break; case KVM_CAP_S390_HPAGE_1M: r = 0; - if (hpage) + if (hpage && !kvm_is_ucontrol(kvm)) r = 1; break; case KVM_CAP_S390_MEM_OP: @@ -691,11 +691,13 @@ static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap) mutex_lock(&kvm->lock); if (kvm->created_vcpus) r = -EBUSY; - else if (!hpage || kvm->arch.use_cmma) + else if (!hpage || kvm->arch.use_cmma || kvm_is_ucontrol(kvm)) r = -EINVAL; else { r = 0; + down_write(&kvm->mm->mmap_sem); kvm->mm->context.allow_gmap_hpage_1m = 1; + up_write(&kvm->mm->mmap_sem); /* * We might have to create fake 4k page * tables. To avoid that the hardware works on diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c index d68f10441a16..8679bd74d337 100644 --- a/arch/s390/kvm/priv.c +++ b/arch/s390/kvm/priv.c @@ -280,9 +280,11 @@ retry: goto retry; } } - if (rc) - return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING); up_read(¤t->mm->mmap_sem); + if (rc == -EFAULT) + return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING); + if (rc < 0) + return rc; vcpu->run->s.regs.gprs[reg1] &= ~0xff; vcpu->run->s.regs.gprs[reg1] |= key; return 0; @@ -324,9 +326,11 @@ retry: goto retry; } } - if (rc < 0) - return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING); up_read(¤t->mm->mmap_sem); + if (rc == -EFAULT) + return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING); + if (rc < 0) + return rc; kvm_s390_set_psw_cc(vcpu, rc); return 0; } @@ -390,12 +394,12 @@ static int handle_sske(struct kvm_vcpu *vcpu) FAULT_FLAG_WRITE, &unlocked); rc = !rc ? -EAGAIN : rc; } + up_read(¤t->mm->mmap_sem); if (rc == -EFAULT) return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING); - - up_read(¤t->mm->mmap_sem); - if (rc >= 0) - start += PAGE_SIZE; + if (rc < 0) + return rc; + start += PAGE_SIZE; } if (m3 & (SSKE_MC | SSKE_MR)) { @@ -1002,13 +1006,15 @@ static int handle_pfmf(struct kvm_vcpu *vcpu) FAULT_FLAG_WRITE, &unlocked); rc = !rc ? -EAGAIN : rc; } + up_read(¤t->mm->mmap_sem); if (rc == -EFAULT) return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING); - - up_read(¤t->mm->mmap_sem); - if (rc >= 0) - start += PAGE_SIZE; + if (rc == -EAGAIN) + continue; + if (rc < 0) + return rc; } + start += PAGE_SIZE; } if (vcpu->run->s.regs.gprs[reg1] & PFMF_FSC) { if (psw_bits(vcpu->arch.sie_block->gpsw).eaba == PSW_BITS_AMODE_64BIT) { diff --git a/arch/s390/kvm/vsie.c b/arch/s390/kvm/vsie.c index 63844b95c22c..a2b28cd1e3fe 100644 --- a/arch/s390/kvm/vsie.c +++ b/arch/s390/kvm/vsie.c @@ -173,7 +173,8 @@ static int shadow_crycb(struct kvm_vcpu *vcpu, struct vsie_page *vsie_page) return set_validity_icpt(scb_s, 0x0039U); /* copy only the wrapping keys */ - if (read_guest_real(vcpu, crycb_addr + 72, &vsie_page->crycb, 56)) + if (read_guest_real(vcpu, crycb_addr + 72, + vsie_page->crycb.dea_wrapping_key_mask, 56)) return set_validity_icpt(scb_s, 0x0035U); scb_s->ecb3 |= ecb3_flags; diff --git a/arch/s390/mm/gmap.c b/arch/s390/mm/gmap.c index bb44990c8212..911c7ded35f1 100644 --- a/arch/s390/mm/gmap.c +++ b/arch/s390/mm/gmap.c @@ -708,11 +708,13 @@ void gmap_discard(struct gmap *gmap, unsigned long from, unsigned long to) vmaddr |= gaddr & ~PMD_MASK; /* Find vma in the parent mm */ vma = find_vma(gmap->mm, vmaddr); + if (!vma) + continue; /* * We do not discard pages that are backed by * hugetlbfs, so we don't have to refault them. */ - if (vma && is_vm_hugetlb_page(vma)) + if (is_vm_hugetlb_page(vma)) continue; size = min(to - gaddr, PMD_SIZE - (gaddr & ~PMD_MASK)); zap_page_range(vma, vmaddr, size); diff --git a/arch/sparc/kernel/of_device_32.c b/arch/sparc/kernel/of_device_32.c index 3641a294ed54..e4abe9b8f97a 100644 --- a/arch/sparc/kernel/of_device_32.c +++ b/arch/sparc/kernel/of_device_32.c @@ -9,6 +9,7 @@ #include <linux/irq.h> #include <linux/of_device.h> #include <linux/of_platform.h> +#include <linux/dma-mapping.h> #include <asm/leon.h> #include <asm/leon_amba.h> @@ -381,6 +382,9 @@ static struct platform_device * __init scan_one_device(struct device_node *dp, else dev_set_name(&op->dev, "%08x", dp->phandle); + op->dev.coherent_dma_mask = DMA_BIT_MASK(32); + op->dev.dma_mask = &op->dev.coherent_dma_mask; + if (of_device_register(op)) { printk("%s: Could not register of device.\n", dp->full_name); diff --git a/arch/sparc/kernel/of_device_64.c b/arch/sparc/kernel/of_device_64.c index 44e4d4435bed..6df6086968c6 100644 --- a/arch/sparc/kernel/of_device_64.c +++ b/arch/sparc/kernel/of_device_64.c @@ -2,6 +2,7 @@ #include <linux/string.h> #include <linux/kernel.h> #include <linux/of.h> +#include <linux/dma-mapping.h> #include <linux/init.h> #include <linux/export.h> #include <linux/mod_devicetable.h> @@ -675,6 +676,8 @@ static struct platform_device * __init scan_one_device(struct device_node *dp, dev_set_name(&op->dev, "root"); else dev_set_name(&op->dev, "%08x", dp->phandle); + op->dev.coherent_dma_mask = DMA_BIT_MASK(32); + op->dev.dma_mask = &op->dev.coherent_dma_mask; if (of_device_register(op)) { printk("%s: Could not register of device.\n", diff --git a/arch/x86/boot/compressed/mem_encrypt.S b/arch/x86/boot/compressed/mem_encrypt.S index eaa843a52907..a480356e0ed8 100644 --- a/arch/x86/boot/compressed/mem_encrypt.S +++ b/arch/x86/boot/compressed/mem_encrypt.S @@ -25,20 +25,6 @@ ENTRY(get_sev_encryption_bit) push %ebx push %ecx push %edx - push %edi - - /* - * RIP-relative addressing is needed to access the encryption bit - * variable. Since we are running in 32-bit mode we need this call/pop - * sequence to get the proper relative addressing. - */ - call 1f -1: popl %edi - subl $1b, %edi - - movl enc_bit(%edi), %eax - cmpl $0, %eax - jge .Lsev_exit /* Check if running under a hypervisor */ movl $1, %eax @@ -69,15 +55,12 @@ ENTRY(get_sev_encryption_bit) movl %ebx, %eax andl $0x3f, %eax /* Return the encryption bit location */ - movl %eax, enc_bit(%edi) jmp .Lsev_exit .Lno_sev: xor %eax, %eax - movl %eax, enc_bit(%edi) .Lsev_exit: - pop %edi pop %edx pop %ecx pop %ebx @@ -113,8 +96,6 @@ ENTRY(set_sev_encryption_mask) ENDPROC(set_sev_encryption_mask) .data -enc_bit: - .int 0xffffffff #ifdef CONFIG_AMD_MEM_ENCRYPT .balign 8 diff --git a/arch/x86/crypto/aegis128-aesni-glue.c b/arch/x86/crypto/aegis128-aesni-glue.c index acd11b3bf639..2a356b948720 100644 --- a/arch/x86/crypto/aegis128-aesni-glue.c +++ b/arch/x86/crypto/aegis128-aesni-glue.c @@ -379,7 +379,6 @@ static int __init crypto_aegis128_aesni_module_init(void) { if (!boot_cpu_has(X86_FEATURE_XMM2) || !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE) || !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL)) return -ENODEV; diff --git a/arch/x86/crypto/aegis128l-aesni-glue.c b/arch/x86/crypto/aegis128l-aesni-glue.c index 2071c3d1ae07..dbe8bb980da1 100644 --- a/arch/x86/crypto/aegis128l-aesni-glue.c +++ b/arch/x86/crypto/aegis128l-aesni-glue.c @@ -379,7 +379,6 @@ static int __init crypto_aegis128l_aesni_module_init(void) { if (!boot_cpu_has(X86_FEATURE_XMM2) || !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE) || !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL)) return -ENODEV; diff --git a/arch/x86/crypto/aegis256-aesni-glue.c b/arch/x86/crypto/aegis256-aesni-glue.c index b5f2a8fd5a71..8bebda2de92f 100644 --- a/arch/x86/crypto/aegis256-aesni-glue.c +++ b/arch/x86/crypto/aegis256-aesni-glue.c @@ -379,7 +379,6 @@ static int __init crypto_aegis256_aesni_module_init(void) { if (!boot_cpu_has(X86_FEATURE_XMM2) || !boot_cpu_has(X86_FEATURE_AES) || - !boot_cpu_has(X86_FEATURE_OSXSAVE) || !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL)) return -ENODEV; diff --git a/arch/x86/crypto/morus1280-sse2-glue.c b/arch/x86/crypto/morus1280-sse2-glue.c index 95cf857d2cbb..f40244eaf14d 100644 --- a/arch/x86/crypto/morus1280-sse2-glue.c +++ b/arch/x86/crypto/morus1280-sse2-glue.c @@ -40,7 +40,6 @@ MORUS1280_DECLARE_ALGS(sse2, "morus1280-sse2", 350); static int __init crypto_morus1280_sse2_module_init(void) { if (!boot_cpu_has(X86_FEATURE_XMM2) || - !boot_cpu_has(X86_FEATURE_OSXSAVE) || !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL)) return -ENODEV; diff --git a/arch/x86/crypto/morus640-sse2-glue.c b/arch/x86/crypto/morus640-sse2-glue.c index 615fb7bc9a32..9afaf8f8565a 100644 --- a/arch/x86/crypto/morus640-sse2-glue.c +++ b/arch/x86/crypto/morus640-sse2-glue.c @@ -40,7 +40,6 @@ MORUS640_DECLARE_ALGS(sse2, "morus640-sse2", 400); static int __init crypto_morus640_sse2_module_init(void) { if (!boot_cpu_has(X86_FEATURE_XMM2) || - !boot_cpu_has(X86_FEATURE_OSXSAVE) || !cpu_has_xfeatures(XFEATURE_MASK_SSE, NULL)) return -ENODEV; diff --git a/arch/x86/events/intel/lbr.c b/arch/x86/events/intel/lbr.c index f3e006bed9a7..c88ed39582a1 100644 --- a/arch/x86/events/intel/lbr.c +++ b/arch/x86/events/intel/lbr.c @@ -1272,4 +1272,8 @@ void intel_pmu_lbr_init_knl(void) x86_pmu.lbr_sel_mask = LBR_SEL_MASK; x86_pmu.lbr_sel_map = snb_lbr_sel_map; + + /* Knights Landing does have MISPREDICT bit */ + if (x86_pmu.intel_cap.lbr_format == LBR_FORMAT_LIP) + x86_pmu.intel_cap.lbr_format = LBR_FORMAT_EIP_FLAGS; } diff --git a/arch/x86/hyperv/hv_apic.c b/arch/x86/hyperv/hv_apic.c index 5b0f613428c2..2c43e3055948 100644 --- a/arch/x86/hyperv/hv_apic.c +++ b/arch/x86/hyperv/hv_apic.c @@ -95,8 +95,8 @@ static void hv_apic_eoi_write(u32 reg, u32 val) */ static bool __send_ipi_mask_ex(const struct cpumask *mask, int vector) { - struct ipi_arg_ex **arg; - struct ipi_arg_ex *ipi_arg; + struct hv_send_ipi_ex **arg; + struct hv_send_ipi_ex *ipi_arg; unsigned long flags; int nr_bank = 0; int ret = 1; @@ -105,7 +105,7 @@ static bool __send_ipi_mask_ex(const struct cpumask *mask, int vector) return false; local_irq_save(flags); - arg = (struct ipi_arg_ex **)this_cpu_ptr(hyperv_pcpu_input_arg); + arg = (struct hv_send_ipi_ex **)this_cpu_ptr(hyperv_pcpu_input_arg); ipi_arg = *arg; if (unlikely(!ipi_arg)) @@ -135,7 +135,7 @@ ipi_mask_ex_done: static bool __send_ipi_mask(const struct cpumask *mask, int vector) { int cur_cpu, vcpu; - struct ipi_arg_non_ex ipi_arg; + struct hv_send_ipi ipi_arg; int ret = 1; trace_hyperv_send_ipi_mask(mask, vector); diff --git a/arch/x86/include/asm/atomic.h b/arch/x86/include/asm/atomic.h index b143717b92b3..ce84388e540c 100644 --- a/arch/x86/include/asm/atomic.h +++ b/arch/x86/include/asm/atomic.h @@ -80,11 +80,11 @@ static __always_inline void arch_atomic_sub(int i, atomic_t *v) * true if the result is zero, or false for all * other cases. */ -#define arch_atomic_sub_and_test arch_atomic_sub_and_test static __always_inline bool arch_atomic_sub_and_test(int i, atomic_t *v) { GEN_BINARY_RMWcc(LOCK_PREFIX "subl", v->counter, "er", i, "%0", e); } +#define arch_atomic_sub_and_test arch_atomic_sub_and_test /** * arch_atomic_inc - increment atomic variable @@ -92,12 +92,12 @@ static __always_inline bool arch_atomic_sub_and_test(int i, atomic_t *v) * * Atomically increments @v by 1. */ -#define arch_atomic_inc arch_atomic_inc static __always_inline void arch_atomic_inc(atomic_t *v) { asm volatile(LOCK_PREFIX "incl %0" : "+m" (v->counter)); } +#define arch_atomic_inc arch_atomic_inc /** * arch_atomic_dec - decrement atomic variable @@ -105,12 +105,12 @@ static __always_inline void arch_atomic_inc(atomic_t *v) * * Atomically decrements @v by 1. */ -#define arch_atomic_dec arch_atomic_dec static __always_inline void arch_atomic_dec(atomic_t *v) { asm volatile(LOCK_PREFIX "decl %0" : "+m" (v->counter)); } +#define arch_atomic_dec arch_atomic_dec /** * arch_atomic_dec_and_test - decrement and test @@ -120,11 +120,11 @@ static __always_inline void arch_atomic_dec(atomic_t *v) * returns true if the result is 0, or false for all other * cases. */ -#define arch_atomic_dec_and_test arch_atomic_dec_and_test static __always_inline bool arch_atomic_dec_and_test(atomic_t *v) { GEN_UNARY_RMWcc(LOCK_PREFIX "decl", v->counter, "%0", e); } +#define arch_atomic_dec_and_test arch_atomic_dec_and_test /** * arch_atomic_inc_and_test - increment and test @@ -134,11 +134,11 @@ static __always_inline bool arch_atomic_dec_and_test(atomic_t *v) * and returns true if the result is zero, or false for all * other cases. */ -#define arch_atomic_inc_and_test arch_atomic_inc_and_test static __always_inline bool arch_atomic_inc_and_test(atomic_t *v) { GEN_UNARY_RMWcc(LOCK_PREFIX "incl", v->counter, "%0", e); } +#define arch_atomic_inc_and_test arch_atomic_inc_and_test /** * arch_atomic_add_negative - add and test if negative @@ -149,11 +149,11 @@ static __always_inline bool arch_atomic_inc_and_test(atomic_t *v) * if the result is negative, or false when * result is greater than or equal to zero. */ -#define arch_atomic_add_negative arch_atomic_add_negative static __always_inline bool arch_atomic_add_negative(int i, atomic_t *v) { GEN_BINARY_RMWcc(LOCK_PREFIX "addl", v->counter, "er", i, "%0", s); } +#define arch_atomic_add_negative arch_atomic_add_negative /** * arch_atomic_add_return - add integer and return diff --git a/arch/x86/include/asm/atomic64_32.h b/arch/x86/include/asm/atomic64_32.h index ef959f02d070..6a5b0ec460da 100644 --- a/arch/x86/include/asm/atomic64_32.h +++ b/arch/x86/include/asm/atomic64_32.h @@ -205,12 +205,12 @@ static inline long long arch_atomic64_sub(long long i, atomic64_t *v) * * Atomically increments @v by 1. */ -#define arch_atomic64_inc arch_atomic64_inc static inline void arch_atomic64_inc(atomic64_t *v) { __alternative_atomic64(inc, inc_return, /* no output */, "S" (v) : "memory", "eax", "ecx", "edx"); } +#define arch_atomic64_inc arch_atomic64_inc /** * arch_atomic64_dec - decrement atomic64 variable @@ -218,12 +218,12 @@ static inline void arch_atomic64_inc(atomic64_t *v) * * Atomically decrements @v by 1. */ -#define arch_atomic64_dec arch_atomic64_dec static inline void arch_atomic64_dec(atomic64_t *v) { __alternative_atomic64(dec, dec_return, /* no output */, "S" (v) : "memory", "eax", "ecx", "edx"); } +#define arch_atomic64_dec arch_atomic64_dec /** * arch_atomic64_add_unless - add unless the number is a given value @@ -245,7 +245,6 @@ static inline int arch_atomic64_add_unless(atomic64_t *v, long long a, return (int)a; } -#define arch_atomic64_inc_not_zero arch_atomic64_inc_not_zero static inline int arch_atomic64_inc_not_zero(atomic64_t *v) { int r; @@ -253,8 +252,8 @@ static inline int arch_atomic64_inc_not_zero(atomic64_t *v) "S" (v) : "ecx", "edx", "memory"); return r; } +#define arch_atomic64_inc_not_zero arch_atomic64_inc_not_zero -#define arch_atomic64_dec_if_positive arch_atomic64_dec_if_positive static inline long long arch_atomic64_dec_if_positive(atomic64_t *v) { long long r; @@ -262,6 +261,7 @@ static inline long long arch_atomic64_dec_if_positive(atomic64_t *v) "S" (v) : "ecx", "memory"); return r; } +#define arch_atomic64_dec_if_positive arch_atomic64_dec_if_positive #undef alternative_atomic64 #undef __alternative_atomic64 diff --git a/arch/x86/include/asm/atomic64_64.h b/arch/x86/include/asm/atomic64_64.h index 4343d9b4f30e..5f851d92eecd 100644 --- a/arch/x86/include/asm/atomic64_64.h +++ b/arch/x86/include/asm/atomic64_64.h @@ -71,11 +71,11 @@ static inline void arch_atomic64_sub(long i, atomic64_t *v) * true if the result is zero, or false for all * other cases. */ -#define arch_atomic64_sub_and_test arch_atomic64_sub_and_test static inline bool arch_atomic64_sub_and_test(long i, atomic64_t *v) { GEN_BINARY_RMWcc(LOCK_PREFIX "subq", v->counter, "er", i, "%0", e); } +#define arch_atomic64_sub_and_test arch_atomic64_sub_and_test /** * arch_atomic64_inc - increment atomic64 variable @@ -83,13 +83,13 @@ static inline bool arch_atomic64_sub_and_test(long i, atomic64_t *v) * * Atomically increments @v by 1. */ -#define arch_atomic64_inc arch_atomic64_inc static __always_inline void arch_atomic64_inc(atomic64_t *v) { asm volatile(LOCK_PREFIX "incq %0" : "=m" (v->counter) : "m" (v->counter)); } +#define arch_atomic64_inc arch_atomic64_inc /** * arch_atomic64_dec - decrement atomic64 variable @@ -97,13 +97,13 @@ static __always_inline void arch_atomic64_inc(atomic64_t *v) * * Atomically decrements @v by 1. */ -#define arch_atomic64_dec arch_atomic64_dec static __always_inline void arch_atomic64_dec(atomic64_t *v) { asm volatile(LOCK_PREFIX "decq %0" : "=m" (v->counter) : "m" (v->counter)); } +#define arch_atomic64_dec arch_atomic64_dec /** * arch_atomic64_dec_and_test - decrement and test @@ -113,11 +113,11 @@ static __always_inline void arch_atomic64_dec(atomic64_t *v) * returns true if the result is 0, or false for all other * cases. */ -#define arch_atomic64_dec_and_test arch_atomic64_dec_and_test static inline bool arch_atomic64_dec_and_test(atomic64_t *v) { GEN_UNARY_RMWcc(LOCK_PREFIX "decq", v->counter, "%0", e); } +#define arch_atomic64_dec_and_test arch_atomic64_dec_and_test /** * arch_atomic64_inc_and_test - increment and test @@ -127,11 +127,11 @@ static inline bool arch_atomic64_dec_and_test(atomic64_t *v) * and returns true if the result is zero, or false for all * other cases. */ -#define arch_atomic64_inc_and_test arch_atomic64_inc_and_test static inline bool arch_atomic64_inc_and_test(atomic64_t *v) { GEN_UNARY_RMWcc(LOCK_PREFIX "incq", v->counter, "%0", e); } +#define arch_atomic64_inc_and_test arch_atomic64_inc_and_test /** * arch_atomic64_add_negative - add and test if negative @@ -142,11 +142,11 @@ static inline bool arch_atomic64_inc_and_test(atomic64_t *v) * if the result is negative, or false when * result is greater than or equal to zero. */ -#define arch_atomic64_add_negative arch_atomic64_add_negative static inline bool arch_atomic64_add_negative(long i, atomic64_t *v) { GEN_BINARY_RMWcc(LOCK_PREFIX "addq", v->counter, "er", i, "%0", s); } +#define arch_atomic64_add_negative arch_atomic64_add_negative /** * arch_atomic64_add_return - add and return diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h index e203169931c7..6390bd8c141b 100644 --- a/arch/x86/include/asm/fixmap.h +++ b/arch/x86/include/asm/fixmap.h @@ -14,6 +14,16 @@ #ifndef _ASM_X86_FIXMAP_H #define _ASM_X86_FIXMAP_H +/* + * Exposed to assembly code for setting up initial page tables. Cannot be + * calculated in assembly code (fixmap entries are an enum), but is sanity + * checked in the actual fixmap C code to make sure that the fixmap is + * covered fully. + */ +#define FIXMAP_PMD_NUM 2 +/* fixmap starts downwards from the 507th entry in level2_fixmap_pgt */ +#define FIXMAP_PMD_TOP 507 + #ifndef __ASSEMBLY__ #include <linux/kernel.h> #include <asm/acpi.h> diff --git a/arch/x86/include/asm/hyperv-tlfs.h b/arch/x86/include/asm/hyperv-tlfs.h index e977b6b3a538..00e01d215f74 100644 --- a/arch/x86/include/asm/hyperv-tlfs.h +++ b/arch/x86/include/asm/hyperv-tlfs.h @@ -726,19 +726,21 @@ struct hv_enlightened_vmcs { #define HV_STIMER_AUTOENABLE (1ULL << 3) #define HV_STIMER_SINT(config) (__u8)(((config) >> 16) & 0x0F) -struct ipi_arg_non_ex { - u32 vector; - u32 reserved; - u64 cpu_mask; -}; - struct hv_vpset { u64 format; u64 valid_bank_mask; u64 bank_contents[]; }; -struct ipi_arg_ex { +/* HvCallSendSyntheticClusterIpi hypercall */ +struct hv_send_ipi { + u32 vector; + u32 reserved; + u64 cpu_mask; +}; + +/* HvCallSendSyntheticClusterIpiEx hypercall */ +struct hv_send_ipi_ex { u32 vector; u32 reserved; struct hv_vpset vp_set; diff --git a/arch/x86/include/asm/kdebug.h b/arch/x86/include/asm/kdebug.h index 395c9631e000..75f1e35e7c15 100644 --- a/arch/x86/include/asm/kdebug.h +++ b/arch/x86/include/asm/kdebug.h @@ -22,10 +22,20 @@ enum die_val { DIE_NMIUNKNOWN, }; +enum show_regs_mode { + SHOW_REGS_SHORT, + /* + * For when userspace crashed, but we don't think it's our fault, and + * therefore don't print kernel registers. + */ + SHOW_REGS_USER, + SHOW_REGS_ALL +}; + extern void die(const char *, struct pt_regs *,long); extern int __must_check __die(const char *, struct pt_regs *, long); extern void show_stack_regs(struct pt_regs *regs); -extern void __show_regs(struct pt_regs *regs, int all); +extern void __show_regs(struct pt_regs *regs, enum show_regs_mode); extern void show_iret_regs(struct pt_regs *regs); extern unsigned long oops_begin(void); extern void oops_end(unsigned long, struct pt_regs *, int signr); diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 00ddb0c9e612..09b2e3e2cf1b 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -869,6 +869,8 @@ struct kvm_arch { bool x2apic_format; bool x2apic_broadcast_quirk_disabled; + + bool guest_can_read_msr_platform_info; }; struct kvm_vm_stat { @@ -1022,6 +1024,7 @@ struct kvm_x86_ops { void (*refresh_apicv_exec_ctrl)(struct kvm_vcpu *vcpu); void (*hwapic_irr_update)(struct kvm_vcpu *vcpu, int max_irr); void (*hwapic_isr_update)(struct kvm_vcpu *vcpu, int isr); + bool (*guest_apic_has_interrupt)(struct kvm_vcpu *vcpu); void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); void (*set_virtual_apic_mode)(struct kvm_vcpu *vcpu); void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu, hpa_t hpa); @@ -1055,6 +1058,7 @@ struct kvm_x86_ops { bool (*umip_emulated)(void); int (*check_nested_events)(struct kvm_vcpu *vcpu, bool external_intr); + void (*request_immediate_exit)(struct kvm_vcpu *vcpu); void (*sched_in)(struct kvm_vcpu *kvm, int cpu); @@ -1237,19 +1241,12 @@ enum emulation_result { #define EMULTYPE_NO_DECODE (1 << 0) #define EMULTYPE_TRAP_UD (1 << 1) #define EMULTYPE_SKIP (1 << 2) -#define EMULTYPE_RETRY (1 << 3) -#define EMULTYPE_NO_REEXECUTE (1 << 4) -#define EMULTYPE_NO_UD_ON_FAIL (1 << 5) -#define EMULTYPE_VMWARE (1 << 6) -int x86_emulate_instruction(struct kvm_vcpu *vcpu, unsigned long cr2, - int emulation_type, void *insn, int insn_len); - -static inline int emulate_instruction(struct kvm_vcpu *vcpu, - int emulation_type) -{ - return x86_emulate_instruction(vcpu, 0, - emulation_type | EMULTYPE_NO_REEXECUTE, NULL, 0); -} +#define EMULTYPE_ALLOW_RETRY (1 << 3) +#define EMULTYPE_NO_UD_ON_FAIL (1 << 4) +#define EMULTYPE_VMWARE (1 << 5) +int kvm_emulate_instruction(struct kvm_vcpu *vcpu, int emulation_type); +int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu, + void *insn, int insn_len); void kvm_enable_efer_bits(u64); bool kvm_valid_efer(struct kvm_vcpu *vcpu, u64 efer); @@ -1450,7 +1447,6 @@ asmlinkage void kvm_spurious_fault(void); ____kvm_handle_fault_on_reboot(insn, "") #define KVM_ARCH_WANT_MMU_NOTIFIER -int kvm_unmap_hva(struct kvm *kvm, unsigned long hva); int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end); int kvm_test_age_hva(struct kvm *kvm, unsigned long hva); @@ -1463,7 +1459,7 @@ void kvm_vcpu_reset(struct kvm_vcpu *vcpu, bool init_event); void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu); int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low, - unsigned long ipi_bitmap_high, int min, + unsigned long ipi_bitmap_high, u32 min, unsigned long icr, int op_64_bit); u64 kvm_get_arch_capabilities(void); @@ -1490,6 +1486,7 @@ extern bool kvm_find_async_pf_gfn(struct kvm_vcpu *vcpu, gfn_t gfn); int kvm_skip_emulated_instruction(struct kvm_vcpu *vcpu); int kvm_complete_insn_gp(struct kvm_vcpu *vcpu, int err); +void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu); int kvm_is_in_guest(void); diff --git a/arch/x86/include/asm/mem_encrypt.h b/arch/x86/include/asm/mem_encrypt.h index c0643831706e..616f8e637bc3 100644 --- a/arch/x86/include/asm/mem_encrypt.h +++ b/arch/x86/include/asm/mem_encrypt.h @@ -48,10 +48,13 @@ int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size); /* Architecture __weak replacement functions */ void __init mem_encrypt_init(void); +void __init mem_encrypt_free_decrypted_mem(void); bool sme_active(void); bool sev_active(void); +#define __bss_decrypted __attribute__((__section__(".bss..decrypted"))) + #else /* !CONFIG_AMD_MEM_ENCRYPT */ #define sme_me_mask 0ULL @@ -77,6 +80,8 @@ early_set_memory_decrypted(unsigned long vaddr, unsigned long size) { return 0; static inline int __init early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 0; } +#define __bss_decrypted + #endif /* CONFIG_AMD_MEM_ENCRYPT */ /* @@ -88,6 +93,8 @@ early_set_memory_encrypted(unsigned long vaddr, unsigned long size) { return 0; #define __sme_pa(x) (__pa(x) | sme_me_mask) #define __sme_pa_nodebug(x) (__pa_nodebug(x) | sme_me_mask) +extern char __start_bss_decrypted[], __end_bss_decrypted[], __start_bss_decrypted_unused[]; + #endif /* __ASSEMBLY__ */ #endif /* __X86_MEM_ENCRYPT_H__ */ diff --git a/arch/x86/include/asm/pgtable-2level.h b/arch/x86/include/asm/pgtable-2level.h index 24c6cf5f16b7..60d0f9015317 100644 --- a/arch/x86/include/asm/pgtable-2level.h +++ b/arch/x86/include/asm/pgtable-2level.h @@ -19,9 +19,6 @@ static inline void native_set_pte(pte_t *ptep , pte_t pte) static inline void native_set_pmd(pmd_t *pmdp, pmd_t pmd) { -#ifdef CONFIG_PAGE_TABLE_ISOLATION - pmd.pud.p4d.pgd = pti_set_user_pgtbl(&pmdp->pud.p4d.pgd, pmd.pud.p4d.pgd); -#endif *pmdp = pmd; } @@ -61,9 +58,6 @@ static inline pte_t native_ptep_get_and_clear(pte_t *xp) #ifdef CONFIG_SMP static inline pmd_t native_pmdp_get_and_clear(pmd_t *xp) { -#ifdef CONFIG_PAGE_TABLE_ISOLATION - pti_set_user_pgtbl(&xp->pud.p4d.pgd, __pgd(0)); -#endif return __pmd(xchg((pmdval_t *)xp, 0)); } #else @@ -73,9 +67,6 @@ static inline pmd_t native_pmdp_get_and_clear(pmd_t *xp) #ifdef CONFIG_SMP static inline pud_t native_pudp_get_and_clear(pud_t *xp) { -#ifdef CONFIG_PAGE_TABLE_ISOLATION - pti_set_user_pgtbl(&xp->p4d.pgd, __pgd(0)); -#endif return __pud(xchg((pudval_t *)xp, 0)); } #else diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index e4ffa565a69f..690c0307afed 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1195,7 +1195,7 @@ static inline pmd_t pmdp_establish(struct vm_area_struct *vma, return xchg(pmdp, pmd); } else { pmd_t old = *pmdp; - *pmdp = pmd; + WRITE_ONCE(*pmdp, pmd); return old; } } diff --git a/arch/x86/include/asm/pgtable_64.h b/arch/x86/include/asm/pgtable_64.h index f773d5e6c8cc..9c85b54bf03c 100644 --- a/arch/x86/include/asm/pgtable_64.h +++ b/arch/x86/include/asm/pgtable_64.h @@ -14,6 +14,7 @@ #include <asm/processor.h> #include <linux/bitops.h> #include <linux/threads.h> +#include <asm/fixmap.h> extern p4d_t level4_kernel_pgt[512]; extern p4d_t level4_ident_pgt[512]; @@ -22,7 +23,7 @@ extern pud_t level3_ident_pgt[512]; extern pmd_t level2_kernel_pgt[512]; extern pmd_t level2_fixmap_pgt[512]; extern pmd_t level2_ident_pgt[512]; -extern pte_t level1_fixmap_pgt[512]; +extern pte_t level1_fixmap_pgt[512 * FIXMAP_PMD_NUM]; extern pgd_t init_top_pgt[]; #define swapper_pg_dir init_top_pgt @@ -55,15 +56,15 @@ struct mm_struct; void set_pte_vaddr_p4d(p4d_t *p4d_page, unsigned long vaddr, pte_t new_pte); void set_pte_vaddr_pud(pud_t *pud_page, unsigned long vaddr, pte_t new_pte); -static inline void native_pte_clear(struct mm_struct *mm, unsigned long addr, - pte_t *ptep) +static inline void native_set_pte(pte_t *ptep, pte_t pte) { - *ptep = native_make_pte(0); + WRITE_ONCE(*ptep, pte); } -static inline void native_set_pte(pte_t *ptep, pte_t pte) +static inline void native_pte_clear(struct mm_struct *mm, unsigned long addr, + pte_t *ptep) { - *ptep = pte; + native_set_pte(ptep, native_make_pte(0)); } static inline void native_set_pte_atomic(pte_t *ptep, pte_t pte) @@ -73,7 +74,7 @@ static inline void native_set_pte_atomic(pte_t *ptep, pte_t pte) static inline void native_set_pmd(pmd_t *pmdp, pmd_t pmd) { - *pmdp = pmd; + WRITE_ONCE(*pmdp, pmd); } static inline void native_pmd_clear(pmd_t *pmd) @@ -109,7 +110,7 @@ static inline pmd_t native_pmdp_get_and_clear(pmd_t *xp) static inline void native_set_pud(pud_t *pudp, pud_t pud) { - *pudp = pud; + WRITE_ONCE(*pudp, pud); } static inline void native_pud_clear(pud_t *pud) @@ -137,13 +138,13 @@ static inline void native_set_p4d(p4d_t *p4dp, p4d_t p4d) pgd_t pgd; if (pgtable_l5_enabled() || !IS_ENABLED(CONFIG_PAGE_TABLE_ISOLATION)) { - *p4dp = p4d; + WRITE_ONCE(*p4dp, p4d); return; } pgd = native_make_pgd(native_p4d_val(p4d)); pgd = pti_set_user_pgtbl((pgd_t *)p4dp, pgd); - *p4dp = native_make_p4d(native_pgd_val(pgd)); + WRITE_ONCE(*p4dp, native_make_p4d(native_pgd_val(pgd))); } static inline void native_p4d_clear(p4d_t *p4d) @@ -153,7 +154,7 @@ static inline void native_p4d_clear(p4d_t *p4d) static inline void native_set_pgd(pgd_t *pgdp, pgd_t pgd) { - *pgdp = pti_set_user_pgtbl(pgdp, pgd); + WRITE_ONCE(*pgdp, pti_set_user_pgtbl(pgdp, pgd)); } static inline void native_pgd_clear(pgd_t *pgd) diff --git a/arch/x86/include/uapi/asm/kvm.h b/arch/x86/include/uapi/asm/kvm.h index 86299efa804a..fd23d5778ea1 100644 --- a/arch/x86/include/uapi/asm/kvm.h +++ b/arch/x86/include/uapi/asm/kvm.h @@ -377,6 +377,7 @@ struct kvm_sync_regs { #define KVM_X86_QUIRK_LINT0_REENABLED (1 << 0) #define KVM_X86_QUIRK_CD_NW_CLEARED (1 << 1) +#define KVM_X86_QUIRK_LAPIC_MMIO_HOLE (1 << 2) #define KVM_STATE_NESTED_GUEST_MODE 0x00000001 #define KVM_STATE_NESTED_RUN_PENDING 0x00000002 diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c index 9f148e3d45b4..7654febd5102 100644 --- a/arch/x86/kernel/apic/vector.c +++ b/arch/x86/kernel/apic/vector.c @@ -413,7 +413,7 @@ static int activate_managed(struct irq_data *irqd) if (WARN_ON_ONCE(cpumask_empty(vector_searchmask))) { /* Something in the core code broke! Survive gracefully */ pr_err("Managed startup for irq %u, but no CPU\n", irqd->irq); - return EINVAL; + return -EINVAL; } ret = assign_managed_vector(irqd, vector_searchmask); diff --git a/arch/x86/kernel/apm_32.c b/arch/x86/kernel/apm_32.c index ec00d1ff5098..f7151cd03cb0 100644 --- a/arch/x86/kernel/apm_32.c +++ b/arch/x86/kernel/apm_32.c @@ -1640,6 +1640,7 @@ static int do_open(struct inode *inode, struct file *filp) return 0; } +#ifdef CONFIG_PROC_FS static int proc_apm_show(struct seq_file *m, void *v) { unsigned short bx; @@ -1719,6 +1720,7 @@ static int proc_apm_show(struct seq_file *m, void *v) units); return 0; } +#endif static int apm(void *unused) { diff --git a/arch/x86/kernel/cpu/intel_rdt.h b/arch/x86/kernel/cpu/intel_rdt.h index 4e588f36228f..285eb3ec4200 100644 --- a/arch/x86/kernel/cpu/intel_rdt.h +++ b/arch/x86/kernel/cpu/intel_rdt.h @@ -382,6 +382,11 @@ static inline bool is_mbm_event(int e) e <= QOS_L3_MBM_LOCAL_EVENT_ID); } +struct rdt_parse_data { + struct rdtgroup *rdtgrp; + char *buf; +}; + /** * struct rdt_resource - attributes of an RDT resource * @rid: The index of the resource @@ -423,16 +428,19 @@ struct rdt_resource { struct rdt_cache cache; struct rdt_membw membw; const char *format_str; - int (*parse_ctrlval) (void *data, struct rdt_resource *r, - struct rdt_domain *d); + int (*parse_ctrlval)(struct rdt_parse_data *data, + struct rdt_resource *r, + struct rdt_domain *d); struct list_head evt_list; int num_rmid; unsigned int mon_scale; unsigned long fflags; }; -int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d); -int parse_bw(void *_buf, struct rdt_resource *r, struct rdt_domain *d); +int parse_cbm(struct rdt_parse_data *data, struct rdt_resource *r, + struct rdt_domain *d); +int parse_bw(struct rdt_parse_data *data, struct rdt_resource *r, + struct rdt_domain *d); extern struct mutex rdtgroup_mutex; @@ -536,6 +544,7 @@ int rdtgroup_pseudo_lock_create(struct rdtgroup *rdtgrp); void rdtgroup_pseudo_lock_remove(struct rdtgroup *rdtgrp); struct rdt_domain *get_domain_from_cpu(int cpu, struct rdt_resource *r); int update_domains(struct rdt_resource *r, int closid); +int closids_supported(void); void closid_free(int closid); int alloc_rmid(void); void free_rmid(u32 rmid); diff --git a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c index af358ca05160..0f53049719cd 100644 --- a/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c +++ b/arch/x86/kernel/cpu/intel_rdt_ctrlmondata.c @@ -64,19 +64,19 @@ static bool bw_validate(char *buf, unsigned long *data, struct rdt_resource *r) return true; } -int parse_bw(void *_buf, struct rdt_resource *r, struct rdt_domain *d) +int parse_bw(struct rdt_parse_data *data, struct rdt_resource *r, + struct rdt_domain *d) { - unsigned long data; - char *buf = _buf; + unsigned long bw_val; if (d->have_new_ctrl) { rdt_last_cmd_printf("duplicate domain %d\n", d->id); return -EINVAL; } - if (!bw_validate(buf, &data, r)) + if (!bw_validate(data->buf, &bw_val, r)) return -EINVAL; - d->new_ctrl = data; + d->new_ctrl = bw_val; d->have_new_ctrl = true; return 0; @@ -123,18 +123,13 @@ static bool cbm_validate(char *buf, u32 *data, struct rdt_resource *r) return true; } -struct rdt_cbm_parse_data { - struct rdtgroup *rdtgrp; - char *buf; -}; - /* * Read one cache bit mask (hex). Check that it is valid for the current * resource type. */ -int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d) +int parse_cbm(struct rdt_parse_data *data, struct rdt_resource *r, + struct rdt_domain *d) { - struct rdt_cbm_parse_data *data = _data; struct rdtgroup *rdtgrp = data->rdtgrp; u32 cbm_val; @@ -195,11 +190,17 @@ int parse_cbm(void *_data, struct rdt_resource *r, struct rdt_domain *d) static int parse_line(char *line, struct rdt_resource *r, struct rdtgroup *rdtgrp) { - struct rdt_cbm_parse_data data; + struct rdt_parse_data data; char *dom = NULL, *id; struct rdt_domain *d; unsigned long dom_id; + if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP && + r->rid == RDT_RESOURCE_MBA) { + rdt_last_cmd_puts("Cannot pseudo-lock MBA resource\n"); + return -EINVAL; + } + next: if (!line || line[0] == '\0') return 0; diff --git a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c index b799c00bef09..1b8e86a5d5e1 100644 --- a/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c +++ b/arch/x86/kernel/cpu/intel_rdt_rdtgroup.c @@ -97,6 +97,12 @@ void rdt_last_cmd_printf(const char *fmt, ...) * limited as the number of resources grows. */ static int closid_free_map; +static int closid_free_map_len; + +int closids_supported(void) +{ + return closid_free_map_len; +} static void closid_init(void) { @@ -111,6 +117,7 @@ static void closid_init(void) /* CLOSID 0 is always reserved for the default group */ closid_free_map &= ~1; + closid_free_map_len = rdt_min_closid; } static int closid_alloc(void) @@ -802,7 +809,7 @@ static int rdt_bit_usage_show(struct kernfs_open_file *of, sw_shareable = 0; exclusive = 0; seq_printf(seq, "%d=", dom->id); - for (i = 0; i < r->num_closid; i++, ctrl++) { + for (i = 0; i < closids_supported(); i++, ctrl++) { if (!closid_allocated(i)) continue; mode = rdtgroup_mode_by_closid(i); @@ -989,7 +996,7 @@ bool rdtgroup_cbm_overlaps(struct rdt_resource *r, struct rdt_domain *d, /* Check for overlap with other resource groups */ ctrl = d->ctrl_val; - for (i = 0; i < r->num_closid; i++, ctrl++) { + for (i = 0; i < closids_supported(); i++, ctrl++) { ctrl_b = (unsigned long *)ctrl; mode = rdtgroup_mode_by_closid(i); if (closid_allocated(i) && i != closid && @@ -1024,16 +1031,27 @@ static bool rdtgroup_mode_test_exclusive(struct rdtgroup *rdtgrp) { int closid = rdtgrp->closid; struct rdt_resource *r; + bool has_cache = false; struct rdt_domain *d; for_each_alloc_enabled_rdt_resource(r) { + if (r->rid == RDT_RESOURCE_MBA) + continue; + has_cache = true; list_for_each_entry(d, &r->domains, list) { if (rdtgroup_cbm_overlaps(r, d, d->ctrl_val[closid], - rdtgrp->closid, false)) + rdtgrp->closid, false)) { + rdt_last_cmd_puts("schemata overlaps\n"); return false; + } } } + if (!has_cache) { + rdt_last_cmd_puts("cannot be exclusive without CAT/CDP\n"); + return false; + } + return true; } @@ -1085,7 +1103,6 @@ static ssize_t rdtgroup_mode_write(struct kernfs_open_file *of, rdtgrp->mode = RDT_MODE_SHAREABLE; } else if (!strcmp(buf, "exclusive")) { if (!rdtgroup_mode_test_exclusive(rdtgrp)) { - rdt_last_cmd_printf("schemata overlaps\n"); ret = -EINVAL; goto out; } @@ -1155,8 +1172,8 @@ static int rdtgroup_size_show(struct kernfs_open_file *of, struct rdt_resource *r; struct rdt_domain *d; unsigned int size; - bool sep = false; - u32 cbm; + bool sep; + u32 ctrl; rdtgrp = rdtgroup_kn_lock_live(of->kn); if (!rdtgrp) { @@ -1174,6 +1191,7 @@ static int rdtgroup_size_show(struct kernfs_open_file *of, } for_each_alloc_enabled_rdt_resource(r) { + sep = false; seq_printf(s, "%*s:", max_name_width, r->name); list_for_each_entry(d, &r->domains, list) { if (sep) @@ -1181,8 +1199,13 @@ static int rdtgroup_size_show(struct kernfs_open_file *of, if (rdtgrp->mode == RDT_MODE_PSEUDO_LOCKSETUP) { size = 0; } else { - cbm = d->ctrl_val[rdtgrp->closid]; - size = rdtgroup_cbm_to_size(r, d, cbm); + ctrl = (!is_mba_sc(r) ? + d->ctrl_val[rdtgrp->closid] : + d->mbps_val[rdtgrp->closid]); + if (r->rid == RDT_RESOURCE_MBA) + size = ctrl; + else + size = rdtgroup_cbm_to_size(r, d, ctrl); } seq_printf(s, "%d=%u", d->id, size); sep = true; @@ -2336,12 +2359,18 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp) u32 *ctrl; for_each_alloc_enabled_rdt_resource(r) { + /* + * Only initialize default allocations for CBM cache + * resources + */ + if (r->rid == RDT_RESOURCE_MBA) + continue; list_for_each_entry(d, &r->domains, list) { d->have_new_ctrl = false; d->new_ctrl = r->cache.shareable_bits; used_b = r->cache.shareable_bits; ctrl = d->ctrl_val; - for (i = 0; i < r->num_closid; i++, ctrl++) { + for (i = 0; i < closids_supported(); i++, ctrl++) { if (closid_allocated(i) && i != closid) { mode = rdtgroup_mode_by_closid(i); if (mode == RDT_MODE_PSEUDO_LOCKSETUP) @@ -2373,6 +2402,12 @@ static int rdtgroup_init_alloc(struct rdtgroup *rdtgrp) } for_each_alloc_enabled_rdt_resource(r) { + /* + * Only initialize default allocations for CBM cache + * resources + */ + if (r->rid == RDT_RESOURCE_MBA) + continue; ret = update_domains(r, rdtgrp->closid); if (ret < 0) { rdt_last_cmd_puts("failed to initialize allocations\n"); diff --git a/arch/x86/kernel/cpu/microcode/amd.c b/arch/x86/kernel/cpu/microcode/amd.c index 0624957aa068..07b5fc00b188 100644 --- a/arch/x86/kernel/cpu/microcode/amd.c +++ b/arch/x86/kernel/cpu/microcode/amd.c @@ -504,6 +504,7 @@ static enum ucode_state apply_microcode_amd(int cpu) struct microcode_amd *mc_amd; struct ucode_cpu_info *uci; struct ucode_patch *p; + enum ucode_state ret; u32 rev, dummy; BUG_ON(raw_smp_processor_id() != cpu); @@ -521,9 +522,8 @@ static enum ucode_state apply_microcode_amd(int cpu) /* need to apply patch? */ if (rev >= mc_amd->hdr.patch_id) { - c->microcode = rev; - uci->cpu_sig.rev = rev; - return UCODE_OK; + ret = UCODE_OK; + goto out; } if (__apply_microcode_amd(mc_amd)) { @@ -531,13 +531,21 @@ static enum ucode_state apply_microcode_amd(int cpu) cpu, mc_amd->hdr.patch_id); return UCODE_ERROR; } - pr_info("CPU%d: new patch_level=0x%08x\n", cpu, - mc_amd->hdr.patch_id); - uci->cpu_sig.rev = mc_amd->hdr.patch_id; - c->microcode = mc_amd->hdr.patch_id; + rev = mc_amd->hdr.patch_id; + ret = UCODE_UPDATED; + + pr_info("CPU%d: new patch_level=0x%08x\n", cpu, rev); - return UCODE_UPDATED; +out: + uci->cpu_sig.rev = rev; + c->microcode = rev; + + /* Update boot_cpu_data's revision too, if we're on the BSP: */ + if (c->cpu_index == boot_cpu_data.cpu_index) + boot_cpu_data.microcode = rev; + + return ret; } static int install_equiv_cpu_table(const u8 *buf) diff --git a/arch/x86/kernel/cpu/microcode/intel.c b/arch/x86/kernel/cpu/microcode/intel.c index 97ccf4c3b45b..16936a24795c 100644 --- a/arch/x86/kernel/cpu/microcode/intel.c +++ b/arch/x86/kernel/cpu/microcode/intel.c @@ -795,6 +795,7 @@ static enum ucode_state apply_microcode_intel(int cpu) struct ucode_cpu_info *uci = ucode_cpu_info + cpu; struct cpuinfo_x86 *c = &cpu_data(cpu); struct microcode_intel *mc; + enum ucode_state ret; static int prev_rev; u32 rev; @@ -817,9 +818,8 @@ static enum ucode_state apply_microcode_intel(int cpu) */ rev = intel_get_microcode_revision(); if (rev >= mc->hdr.rev) { - uci->cpu_sig.rev = rev; - c->microcode = rev; - return UCODE_OK; + ret = UCODE_OK; + goto out; } /* @@ -848,10 +848,17 @@ static enum ucode_state apply_microcode_intel(int cpu) prev_rev = rev; } + ret = UCODE_UPDATED; + +out: uci->cpu_sig.rev = rev; - c->microcode = rev; + c->microcode = rev; + + /* Update boot_cpu_data's revision too, if we're on the BSP: */ + if (c->cpu_index == boot_cpu_data.cpu_index) + boot_cpu_data.microcode = rev; - return UCODE_UPDATED; + return ret; } static enum ucode_state generic_load_microcode(int cpu, void *data, size_t size, diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c index f56895106ccf..2b5886401e5f 100644 --- a/arch/x86/kernel/dumpstack.c +++ b/arch/x86/kernel/dumpstack.c @@ -146,7 +146,7 @@ static void show_regs_if_on_stack(struct stack_info *info, struct pt_regs *regs, * they can be printed in the right context. */ if (!partial && on_stack(info, regs, sizeof(*regs))) { - __show_regs(regs, 0); + __show_regs(regs, SHOW_REGS_SHORT); } else if (partial && on_stack(info, (void *)regs + IRET_FRAME_OFFSET, IRET_FRAME_SIZE)) { @@ -344,7 +344,7 @@ void oops_end(unsigned long flags, struct pt_regs *regs, int signr) oops_exit(); /* Executive summary in case the oops scrolled away */ - __show_regs(&exec_summary_regs, true); + __show_regs(&exec_summary_regs, SHOW_REGS_ALL); if (!signr) return; @@ -407,14 +407,9 @@ void die(const char *str, struct pt_regs *regs, long err) void show_regs(struct pt_regs *regs) { - bool all = true; - show_regs_print_info(KERN_DEFAULT); - if (IS_ENABLED(CONFIG_X86_32)) - all = !user_mode(regs); - - __show_regs(regs, all); + __show_regs(regs, user_mode(regs) ? SHOW_REGS_USER : SHOW_REGS_ALL); /* * When in-kernel, we also print out the stack at the time of the fault.. diff --git a/arch/x86/kernel/eisa.c b/arch/x86/kernel/eisa.c index f260e452e4f8..e8c8c5d78dbd 100644 --- a/arch/x86/kernel/eisa.c +++ b/arch/x86/kernel/eisa.c @@ -7,11 +7,17 @@ #include <linux/eisa.h> #include <linux/io.h> +#include <xen/xen.h> + static __init int eisa_bus_probe(void) { - void __iomem *p = ioremap(0x0FFFD9, 4); + void __iomem *p; + + if (xen_pv_domain() && !xen_initial_domain()) + return 0; - if (readl(p) == 'E' + ('I'<<8) + ('S'<<16) + ('A'<<24)) + p = ioremap(0x0FFFD9, 4); + if (p && readl(p) == 'E' + ('I' << 8) + ('S' << 16) + ('A' << 24)) EISA_bus = 1; iounmap(p); return 0; diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index 8047379e575a..ddee1f0870c4 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -35,6 +35,7 @@ #include <asm/bootparam_utils.h> #include <asm/microcode.h> #include <asm/kasan.h> +#include <asm/fixmap.h> /* * Manage page tables very early on. @@ -112,6 +113,7 @@ static bool __head check_la57_support(unsigned long physaddr) unsigned long __head __startup_64(unsigned long physaddr, struct boot_params *bp) { + unsigned long vaddr, vaddr_end; unsigned long load_delta, *p; unsigned long pgtable_flags; pgdval_t *pgd; @@ -165,7 +167,8 @@ unsigned long __head __startup_64(unsigned long physaddr, pud[511] += load_delta; pmd = fixup_pointer(level2_fixmap_pgt, physaddr); - pmd[506] += load_delta; + for (i = FIXMAP_PMD_TOP; i > FIXMAP_PMD_TOP - FIXMAP_PMD_NUM; i--) + pmd[i] += load_delta; /* * Set up the identity mapping for the switchover. These @@ -235,6 +238,21 @@ unsigned long __head __startup_64(unsigned long physaddr, sme_encrypt_kernel(bp); /* + * Clear the memory encryption mask from the .bss..decrypted section. + * The bss section will be memset to zero later in the initialization so + * there is no need to zero it after changing the memory encryption + * attribute. + */ + if (mem_encrypt_active()) { + vaddr = (unsigned long)__start_bss_decrypted; + vaddr_end = (unsigned long)__end_bss_decrypted; + for (; vaddr < vaddr_end; vaddr += PMD_SIZE) { + i = pmd_index(vaddr); + pmd[i] -= sme_get_me_mask(); + } + } + + /* * Return the SME encryption mask (if SME is active) to be used as a * modifier for the initial pgdir entry programmed into CR3. */ diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 15ebc2fc166e..a3618cf04cf6 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -24,6 +24,7 @@ #include "../entry/calling.h" #include <asm/export.h> #include <asm/nospec-branch.h> +#include <asm/fixmap.h> #ifdef CONFIG_PARAVIRT #include <asm/asm-offsets.h> @@ -445,13 +446,20 @@ NEXT_PAGE(level2_kernel_pgt) KERNEL_IMAGE_SIZE/PMD_SIZE) NEXT_PAGE(level2_fixmap_pgt) - .fill 506,8,0 - .quad level1_fixmap_pgt - __START_KERNEL_map + _PAGE_TABLE_NOENC - /* 8MB reserved for vsyscalls + a 2MB hole = 4 + 1 entries */ - .fill 5,8,0 + .fill (512 - 4 - FIXMAP_PMD_NUM),8,0 + pgtno = 0 + .rept (FIXMAP_PMD_NUM) + .quad level1_fixmap_pgt + (pgtno << PAGE_SHIFT) - __START_KERNEL_map \ + + _PAGE_TABLE_NOENC; + pgtno = pgtno + 1 + .endr + /* 6 MB reserved space + a 2MB hole */ + .fill 4,8,0 NEXT_PAGE(level1_fixmap_pgt) + .rept (FIXMAP_PMD_NUM) .fill 512,8,0 + .endr #undef PMDS diff --git a/arch/x86/kernel/kvmclock.c b/arch/x86/kernel/kvmclock.c index 1e6764648af3..013fe3d21dbb 100644 --- a/arch/x86/kernel/kvmclock.c +++ b/arch/x86/kernel/kvmclock.c @@ -28,6 +28,7 @@ #include <linux/sched/clock.h> #include <linux/mm.h> #include <linux/slab.h> +#include <linux/set_memory.h> #include <asm/hypervisor.h> #include <asm/mem_encrypt.h> @@ -61,9 +62,10 @@ early_param("no-kvmclock-vsyscall", parse_no_kvmclock_vsyscall); (PAGE_SIZE / sizeof(struct pvclock_vsyscall_time_info)) static struct pvclock_vsyscall_time_info - hv_clock_boot[HVC_BOOT_ARRAY_SIZE] __aligned(PAGE_SIZE); -static struct pvclock_wall_clock wall_clock; + hv_clock_boot[HVC_BOOT_ARRAY_SIZE] __bss_decrypted __aligned(PAGE_SIZE); +static struct pvclock_wall_clock wall_clock __bss_decrypted; static DEFINE_PER_CPU(struct pvclock_vsyscall_time_info *, hv_clock_per_cpu); +static struct pvclock_vsyscall_time_info *hvclock_mem; static inline struct pvclock_vcpu_time_info *this_cpu_pvti(void) { @@ -236,6 +238,45 @@ static void kvm_shutdown(void) native_machine_shutdown(); } +static void __init kvmclock_init_mem(void) +{ + unsigned long ncpus; + unsigned int order; + struct page *p; + int r; + + if (HVC_BOOT_ARRAY_SIZE >= num_possible_cpus()) + return; + + ncpus = num_possible_cpus() - HVC_BOOT_ARRAY_SIZE; + order = get_order(ncpus * sizeof(*hvclock_mem)); + + p = alloc_pages(GFP_KERNEL, order); + if (!p) { + pr_warn("%s: failed to alloc %d pages", __func__, (1U << order)); + return; + } + + hvclock_mem = page_address(p); + + /* + * hvclock is shared between the guest and the hypervisor, must + * be mapped decrypted. + */ + if (sev_active()) { + r = set_memory_decrypted((unsigned long) hvclock_mem, + 1UL << order); + if (r) { + __free_pages(p, order); + hvclock_mem = NULL; + pr_warn("kvmclock: set_memory_decrypted() failed. Disabling\n"); + return; + } + } + + memset(hvclock_mem, 0, PAGE_SIZE << order); +} + static int __init kvm_setup_vsyscall_timeinfo(void) { #ifdef CONFIG_X86_64 @@ -250,6 +291,9 @@ static int __init kvm_setup_vsyscall_timeinfo(void) kvm_clock.archdata.vclock_mode = VCLOCK_PVCLOCK; #endif + + kvmclock_init_mem(); + return 0; } early_initcall(kvm_setup_vsyscall_timeinfo); @@ -269,8 +313,10 @@ static int kvmclock_setup_percpu(unsigned int cpu) /* Use the static page for the first CPUs, allocate otherwise */ if (cpu < HVC_BOOT_ARRAY_SIZE) p = &hv_clock_boot[cpu]; + else if (hvclock_mem) + p = hvclock_mem + cpu - HVC_BOOT_ARRAY_SIZE; else - p = kzalloc(sizeof(*p), GFP_KERNEL); + return -ENOMEM; per_cpu(hv_clock_per_cpu, cpu) = p; return p ? 0 : -ENOMEM; diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index afdb303285f8..8dc69d82567e 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -91,7 +91,7 @@ unsigned paravirt_patch_call(void *insnbuf, if (len < 5) { #ifdef CONFIG_RETPOLINE - WARN_ONCE("Failing to patch indirect CALL in %ps\n", (void *)addr); + WARN_ONCE(1, "Failing to patch indirect CALL in %ps\n", (void *)addr); #endif return len; /* call too long for patch site */ } @@ -111,7 +111,7 @@ unsigned paravirt_patch_jmp(void *insnbuf, const void *target, if (len < 5) { #ifdef CONFIG_RETPOLINE - WARN_ONCE("Failing to patch indirect JMP in %ps\n", (void *)addr); + WARN_ONCE(1, "Failing to patch indirect JMP in %ps\n", (void *)addr); #endif return len; /* call too long for patch site */ } diff --git a/arch/x86/kernel/process_32.c b/arch/x86/kernel/process_32.c index 2924fd447e61..5046a3c9dec2 100644 --- a/arch/x86/kernel/process_32.c +++ b/arch/x86/kernel/process_32.c @@ -59,7 +59,7 @@ #include <asm/intel_rdt_sched.h> #include <asm/proto.h> -void __show_regs(struct pt_regs *regs, int all) +void __show_regs(struct pt_regs *regs, enum show_regs_mode mode) { unsigned long cr0 = 0L, cr2 = 0L, cr3 = 0L, cr4 = 0L; unsigned long d0, d1, d2, d3, d6, d7; @@ -85,7 +85,7 @@ void __show_regs(struct pt_regs *regs, int all) printk(KERN_DEFAULT "DS: %04x ES: %04x FS: %04x GS: %04x SS: %04x EFLAGS: %08lx\n", (u16)regs->ds, (u16)regs->es, (u16)regs->fs, gs, ss, regs->flags); - if (!all) + if (mode != SHOW_REGS_ALL) return; cr0 = read_cr0(); diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c index a451bc374b9b..ea5ea850348d 100644 --- a/arch/x86/kernel/process_64.c +++ b/arch/x86/kernel/process_64.c @@ -62,7 +62,7 @@ __visible DEFINE_PER_CPU(unsigned long, rsp_scratch); /* Prints also some state that isn't saved in the pt_regs */ -void __show_regs(struct pt_regs *regs, int all) +void __show_regs(struct pt_regs *regs, enum show_regs_mode mode) { unsigned long cr0 = 0L, cr2 = 0L, cr3 = 0L, cr4 = 0L, fs, gs, shadowgs; unsigned long d0, d1, d2, d3, d6, d7; @@ -87,9 +87,17 @@ void __show_regs(struct pt_regs *regs, int all) printk(KERN_DEFAULT "R13: %016lx R14: %016lx R15: %016lx\n", regs->r13, regs->r14, regs->r15); - if (!all) + if (mode == SHOW_REGS_SHORT) return; + if (mode == SHOW_REGS_USER) { + rdmsrl(MSR_FS_BASE, fs); + rdmsrl(MSR_KERNEL_GS_BASE, shadowgs); + printk(KERN_DEFAULT "FS: %016lx GS: %016lx\n", + fs, shadowgs); + return; + } + asm("movl %%ds,%0" : "=r" (ds)); asm("movl %%cs,%0" : "=r" (cs)); asm("movl %%es,%0" : "=r" (es)); diff --git a/arch/x86/kernel/topology.c b/arch/x86/kernel/topology.c index 12cbe2b88c0f..738bf42b0218 100644 --- a/arch/x86/kernel/topology.c +++ b/arch/x86/kernel/topology.c @@ -111,8 +111,10 @@ int arch_register_cpu(int num) /* * Currently CPU0 is only hotpluggable on Intel platforms. Other * vendors can add hotplug support later. + * Xen PV guests don't support CPU0 hotplug at all. */ - if (c->x86_vendor != X86_VENDOR_INTEL) + if (c->x86_vendor != X86_VENDOR_INTEL || + boot_cpu_has(X86_FEATURE_XENPV)) cpu0_hotpluggable = 0; /* diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c index 1463468ba9a0..6490f618e096 100644 --- a/arch/x86/kernel/tsc.c +++ b/arch/x86/kernel/tsc.c @@ -1415,7 +1415,7 @@ static bool __init determine_cpu_tsc_frequencies(bool early) static unsigned long __init get_loops_per_jiffy(void) { - unsigned long lpj = tsc_khz * KHZ; + u64 lpj = (u64)tsc_khz * KHZ; do_div(lpj, HZ); return lpj; diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index 8bde0a419f86..5dd3317d761f 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -65,6 +65,23 @@ jiffies_64 = jiffies; #define ALIGN_ENTRY_TEXT_BEGIN . = ALIGN(PMD_SIZE); #define ALIGN_ENTRY_TEXT_END . = ALIGN(PMD_SIZE); +/* + * This section contains data which will be mapped as decrypted. Memory + * encryption operates on a page basis. Make this section PMD-aligned + * to avoid splitting the pages while mapping the section early. + * + * Note: We use a separate section so that only this section gets + * decrypted to avoid exposing more than we wish. + */ +#define BSS_DECRYPTED \ + . = ALIGN(PMD_SIZE); \ + __start_bss_decrypted = .; \ + *(.bss..decrypted); \ + . = ALIGN(PAGE_SIZE); \ + __start_bss_decrypted_unused = .; \ + . = ALIGN(PMD_SIZE); \ + __end_bss_decrypted = .; \ + #else #define X86_ALIGN_RODATA_BEGIN @@ -74,6 +91,7 @@ jiffies_64 = jiffies; #define ALIGN_ENTRY_TEXT_BEGIN #define ALIGN_ENTRY_TEXT_END +#define BSS_DECRYPTED #endif @@ -355,6 +373,7 @@ SECTIONS __bss_start = .; *(.bss..page_aligned) *(.bss) + BSS_DECRYPTED . = ALIGN(PAGE_SIZE); __bss_stop = .; } diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c index 0cefba28c864..fbb0e6df121b 100644 --- a/arch/x86/kvm/lapic.c +++ b/arch/x86/kvm/lapic.c @@ -548,7 +548,7 @@ int kvm_apic_set_irq(struct kvm_vcpu *vcpu, struct kvm_lapic_irq *irq, } int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low, - unsigned long ipi_bitmap_high, int min, + unsigned long ipi_bitmap_high, u32 min, unsigned long icr, int op_64_bit) { int i; @@ -571,18 +571,31 @@ int kvm_pv_send_ipi(struct kvm *kvm, unsigned long ipi_bitmap_low, rcu_read_lock(); map = rcu_dereference(kvm->arch.apic_map); + if (min > map->max_apic_id) + goto out; /* Bits above cluster_size are masked in the caller. */ - for_each_set_bit(i, &ipi_bitmap_low, BITS_PER_LONG) { - vcpu = map->phys_map[min + i]->vcpu; - count += kvm_apic_set_irq(vcpu, &irq, NULL); + for_each_set_bit(i, &ipi_bitmap_low, + min((u32)BITS_PER_LONG, (map->max_apic_id - min + 1))) { + if (map->phys_map[min + i]) { + vcpu = map->phys_map[min + i]->vcpu; + count += kvm_apic_set_irq(vcpu, &irq, NULL); + } } min += cluster_size; - for_each_set_bit(i, &ipi_bitmap_high, BITS_PER_LONG) { - vcpu = map->phys_map[min + i]->vcpu; - count += kvm_apic_set_irq(vcpu, &irq, NULL); + + if (min > map->max_apic_id) + goto out; + + for_each_set_bit(i, &ipi_bitmap_high, + min((u32)BITS_PER_LONG, (map->max_apic_id - min + 1))) { + if (map->phys_map[min + i]) { + vcpu = map->phys_map[min + i]->vcpu; + count += kvm_apic_set_irq(vcpu, &irq, NULL); + } } +out: rcu_read_unlock(); return count; } @@ -1331,9 +1344,8 @@ EXPORT_SYMBOL_GPL(kvm_lapic_reg_read); static int apic_mmio_in_range(struct kvm_lapic *apic, gpa_t addr) { - return kvm_apic_hw_enabled(apic) && - addr >= apic->base_address && - addr < apic->base_address + LAPIC_MMIO_LENGTH; + return addr >= apic->base_address && + addr < apic->base_address + LAPIC_MMIO_LENGTH; } static int apic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this, @@ -1345,6 +1357,15 @@ static int apic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this, if (!apic_mmio_in_range(apic, address)) return -EOPNOTSUPP; + if (!kvm_apic_hw_enabled(apic) || apic_x2apic_mode(apic)) { + if (!kvm_check_has_quirk(vcpu->kvm, + KVM_X86_QUIRK_LAPIC_MMIO_HOLE)) + return -EOPNOTSUPP; + + memset(data, 0xff, len); + return 0; + } + kvm_lapic_reg_read(apic, offset, len, data); return 0; @@ -1904,6 +1925,14 @@ static int apic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this, if (!apic_mmio_in_range(apic, address)) return -EOPNOTSUPP; + if (!kvm_apic_hw_enabled(apic) || apic_x2apic_mode(apic)) { + if (!kvm_check_has_quirk(vcpu->kvm, + KVM_X86_QUIRK_LAPIC_MMIO_HOLE)) + return -EOPNOTSUPP; + + return 0; + } + /* * APIC register must be aligned on 128-bits boundary. * 32/64/128 bits registers must be accessed thru 32 bits. diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c index a282321329b5..d7e9bce6ff61 100644 --- a/arch/x86/kvm/mmu.c +++ b/arch/x86/kvm/mmu.c @@ -899,7 +899,7 @@ static void walk_shadow_page_lockless_end(struct kvm_vcpu *vcpu) { /* * Make sure the write to vcpu->mode is not reordered in front of - * reads to sptes. If it does, kvm_commit_zap_page() can see us + * reads to sptes. If it does, kvm_mmu_commit_zap_page() can see us * OUTSIDE_GUEST_MODE and proceed to free the shadow page table. */ smp_store_release(&vcpu->mode, OUTSIDE_GUEST_MODE); @@ -1853,11 +1853,6 @@ static int kvm_handle_hva(struct kvm *kvm, unsigned long hva, return kvm_handle_hva_range(kvm, hva, hva + 1, data, handler); } -int kvm_unmap_hva(struct kvm *kvm, unsigned long hva) -{ - return kvm_handle_hva(kvm, hva, 0, kvm_unmap_rmapp); -} - int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end) { return kvm_handle_hva_range(kvm, start, end, 0, kvm_unmap_rmapp); @@ -5217,7 +5212,7 @@ static int make_mmu_pages_available(struct kvm_vcpu *vcpu) int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, void *insn, int insn_len) { - int r, emulation_type = EMULTYPE_RETRY; + int r, emulation_type = 0; enum emulation_result er; bool direct = vcpu->arch.mmu.direct_map; @@ -5230,10 +5225,8 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, r = RET_PF_INVALID; if (unlikely(error_code & PFERR_RSVD_MASK)) { r = handle_mmio_page_fault(vcpu, cr2, direct); - if (r == RET_PF_EMULATE) { - emulation_type = 0; + if (r == RET_PF_EMULATE) goto emulate; - } } if (r == RET_PF_INVALID) { @@ -5260,8 +5253,19 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t cr2, u64 error_code, return 1; } - if (mmio_info_in_cache(vcpu, cr2, direct)) - emulation_type = 0; + /* + * vcpu->arch.mmu.page_fault returned RET_PF_EMULATE, but we can still + * optimistically try to just unprotect the page and let the processor + * re-execute the instruction that caused the page fault. Do not allow + * retrying MMIO emulation, as it's not only pointless but could also + * cause us to enter an infinite loop because the processor will keep + * faulting on the non-existent MMIO address. Retrying an instruction + * from a nested guest is also pointless and dangerous as we are only + * explicitly shadowing L1's page tables, i.e. unprotecting something + * for L1 isn't going to magically fix whatever issue cause L2 to fail. + */ + if (!mmio_info_in_cache(vcpu, cr2, direct) && !is_guest_mode(vcpu)) + emulation_type = EMULTYPE_ALLOW_RETRY; emulate: /* * On AMD platforms, under certain conditions insn_len may be zero on #NPF. @@ -5413,7 +5417,12 @@ void kvm_mmu_setup(struct kvm_vcpu *vcpu) { MMU_WARN_ON(VALID_PAGE(vcpu->arch.mmu.root_hpa)); - kvm_init_mmu(vcpu, true); + /* + * kvm_mmu_setup() is called only on vCPU initialization. + * Therefore, no need to reset mmu roots as they are not yet + * initialized. + */ + kvm_init_mmu(vcpu, false); } static void kvm_mmu_invalidate_zap_pages_in_memslot(struct kvm *kvm, diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c index 6276140044d0..d96092b35936 100644 --- a/arch/x86/kvm/svm.c +++ b/arch/x86/kvm/svm.c @@ -776,7 +776,7 @@ static void skip_emulated_instruction(struct kvm_vcpu *vcpu) } if (!svm->next_rip) { - if (emulate_instruction(vcpu, EMULTYPE_SKIP) != + if (kvm_emulate_instruction(vcpu, EMULTYPE_SKIP) != EMULATE_DONE) printk(KERN_DEBUG "%s: NOP\n", __func__); return; @@ -1226,8 +1226,7 @@ static __init int sev_hardware_setup(void) min_sev_asid = cpuid_edx(0x8000001F); /* Initialize SEV ASID bitmap */ - sev_asid_bitmap = kcalloc(BITS_TO_LONGS(max_sev_asid), - sizeof(unsigned long), GFP_KERNEL); + sev_asid_bitmap = bitmap_zalloc(max_sev_asid, GFP_KERNEL); if (!sev_asid_bitmap) return 1; @@ -1405,7 +1404,7 @@ static __exit void svm_hardware_unsetup(void) int cpu; if (svm_sev_enabled()) - kfree(sev_asid_bitmap); + bitmap_free(sev_asid_bitmap); for_each_possible_cpu(cpu) svm_cpu_uninit(cpu); @@ -2715,7 +2714,7 @@ static int gp_interception(struct vcpu_svm *svm) WARN_ON_ONCE(!enable_vmware_backdoor); - er = emulate_instruction(vcpu, + er = kvm_emulate_instruction(vcpu, EMULTYPE_VMWARE | EMULTYPE_NO_UD_ON_FAIL); if (er == EMULATE_USER_EXIT) return 0; @@ -2819,7 +2818,7 @@ static int io_interception(struct vcpu_svm *svm) string = (io_info & SVM_IOIO_STR_MASK) != 0; in = (io_info & SVM_IOIO_TYPE_MASK) != 0; if (string) - return emulate_instruction(vcpu, 0) == EMULATE_DONE; + return kvm_emulate_instruction(vcpu, 0) == EMULATE_DONE; port = io_info >> 16; size = (io_info & SVM_IOIO_SIZE_MASK) >> SVM_IOIO_SIZE_SHIFT; @@ -3861,7 +3860,7 @@ static int iret_interception(struct vcpu_svm *svm) static int invlpg_interception(struct vcpu_svm *svm) { if (!static_cpu_has(X86_FEATURE_DECODEASSISTS)) - return emulate_instruction(&svm->vcpu, 0) == EMULATE_DONE; + return kvm_emulate_instruction(&svm->vcpu, 0) == EMULATE_DONE; kvm_mmu_invlpg(&svm->vcpu, svm->vmcb->control.exit_info_1); return kvm_skip_emulated_instruction(&svm->vcpu); @@ -3869,13 +3868,13 @@ static int invlpg_interception(struct vcpu_svm *svm) static int emulate_on_interception(struct vcpu_svm *svm) { - return emulate_instruction(&svm->vcpu, 0) == EMULATE_DONE; + return kvm_emulate_instruction(&svm->vcpu, 0) == EMULATE_DONE; } static int rsm_interception(struct vcpu_svm *svm) { - return x86_emulate_instruction(&svm->vcpu, 0, 0, - rsm_ins_bytes, 2) == EMULATE_DONE; + return kvm_emulate_instruction_from_buffer(&svm->vcpu, + rsm_ins_bytes, 2) == EMULATE_DONE; } static int rdpmc_interception(struct vcpu_svm *svm) @@ -4700,7 +4699,7 @@ static int avic_unaccelerated_access_interception(struct vcpu_svm *svm) ret = avic_unaccel_trap_write(svm); } else { /* Handling Fault */ - ret = (emulate_instruction(&svm->vcpu, 0) == EMULATE_DONE); + ret = (kvm_emulate_instruction(&svm->vcpu, 0) == EMULATE_DONE); } return ret; @@ -6747,7 +6746,7 @@ e_free: static int sev_dbg_crypt(struct kvm *kvm, struct kvm_sev_cmd *argp, bool dec) { unsigned long vaddr, vaddr_end, next_vaddr; - unsigned long dst_vaddr, dst_vaddr_end; + unsigned long dst_vaddr; struct page **src_p, **dst_p; struct kvm_sev_dbg debug; unsigned long n; @@ -6763,7 +6762,6 @@ static int sev_dbg_crypt(struct kvm *kvm, struct kvm_sev_cmd *argp, bool dec) size = debug.len; vaddr_end = vaddr + size; dst_vaddr = debug.dst_uaddr; - dst_vaddr_end = dst_vaddr + size; for (; vaddr < vaddr_end; vaddr = next_vaddr) { int len, s_off, d_off; @@ -7150,6 +7148,8 @@ static struct kvm_x86_ops svm_x86_ops __ro_after_init = { .check_intercept = svm_check_intercept, .handle_external_intr = svm_handle_external_intr, + .request_immediate_exit = __kvm_request_immediate_exit, + .sched_in = svm_sched_in, .pmu_ops = &amd_pmu_ops, diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c index 1d26f3c4985b..06412ba46aa3 100644 --- a/arch/x86/kvm/vmx.c +++ b/arch/x86/kvm/vmx.c @@ -397,6 +397,7 @@ struct loaded_vmcs { int cpu; bool launched; bool nmi_known_unmasked; + bool hv_timer_armed; /* Support for vnmi-less CPUs */ int soft_vnmi_blocked; ktime_t entry_time; @@ -1019,6 +1020,8 @@ struct vcpu_vmx { int ple_window; bool ple_window_dirty; + bool req_immediate_exit; + /* Support for PML */ #define PML_ENTITY_NUM 512 struct page *pml_pg; @@ -2864,6 +2867,8 @@ static void vmx_prepare_switch_to_guest(struct kvm_vcpu *vcpu) u16 fs_sel, gs_sel; int i; + vmx->req_immediate_exit = false; + if (vmx->loaded_cpu_state) return; @@ -5393,9 +5398,10 @@ static int vmx_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) * To use VMXON (and later other VMX instructions), a guest * must first be able to turn on cr4.VMXE (see handle_vmon()). * So basically the check on whether to allow nested VMX - * is here. + * is here. We operate under the default treatment of SMM, + * so VMX cannot be enabled under SMM. */ - if (!nested_vmx_allowed(vcpu)) + if (!nested_vmx_allowed(vcpu) || is_smm(vcpu)) return 1; } @@ -6183,6 +6189,27 @@ static void vmx_complete_nested_posted_interrupt(struct kvm_vcpu *vcpu) nested_mark_vmcs12_pages_dirty(vcpu); } +static bool vmx_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) +{ + struct vcpu_vmx *vmx = to_vmx(vcpu); + void *vapic_page; + u32 vppr; + int rvi; + + if (WARN_ON_ONCE(!is_guest_mode(vcpu)) || + !nested_cpu_has_vid(get_vmcs12(vcpu)) || + WARN_ON_ONCE(!vmx->nested.virtual_apic_page)) + return false; + + rvi = vmcs_read16(GUEST_INTR_STATUS) & 0xff; + + vapic_page = kmap(vmx->nested.virtual_apic_page); + vppr = *((u32 *)(vapic_page + APIC_PROCPRI)); + kunmap(vmx->nested.virtual_apic_page); + + return ((rvi & 0xf0) > (vppr & 0xf0)); +} + static inline bool kvm_vcpu_trigger_posted_interrupt(struct kvm_vcpu *vcpu, bool nested) { @@ -6983,7 +7010,7 @@ static int handle_rmode_exception(struct kvm_vcpu *vcpu, * Cause the #SS fault with 0 error code in VM86 mode. */ if (((vec == GP_VECTOR) || (vec == SS_VECTOR)) && err_code == 0) { - if (emulate_instruction(vcpu, 0) == EMULATE_DONE) { + if (kvm_emulate_instruction(vcpu, 0) == EMULATE_DONE) { if (vcpu->arch.halt_request) { vcpu->arch.halt_request = 0; return kvm_vcpu_halt(vcpu); @@ -7054,7 +7081,7 @@ static int handle_exception(struct kvm_vcpu *vcpu) if (!vmx->rmode.vm86_active && is_gp_fault(intr_info)) { WARN_ON_ONCE(!enable_vmware_backdoor); - er = emulate_instruction(vcpu, + er = kvm_emulate_instruction(vcpu, EMULTYPE_VMWARE | EMULTYPE_NO_UD_ON_FAIL); if (er == EMULATE_USER_EXIT) return 0; @@ -7157,7 +7184,7 @@ static int handle_io(struct kvm_vcpu *vcpu) ++vcpu->stat.io_exits; if (string) - return emulate_instruction(vcpu, 0) == EMULATE_DONE; + return kvm_emulate_instruction(vcpu, 0) == EMULATE_DONE; port = exit_qualification >> 16; size = (exit_qualification & 7) + 1; @@ -7231,7 +7258,7 @@ static int handle_set_cr4(struct kvm_vcpu *vcpu, unsigned long val) static int handle_desc(struct kvm_vcpu *vcpu) { WARN_ON(!(vcpu->arch.cr4 & X86_CR4_UMIP)); - return emulate_instruction(vcpu, 0) == EMULATE_DONE; + return kvm_emulate_instruction(vcpu, 0) == EMULATE_DONE; } static int handle_cr(struct kvm_vcpu *vcpu) @@ -7480,7 +7507,7 @@ static int handle_vmcall(struct kvm_vcpu *vcpu) static int handle_invd(struct kvm_vcpu *vcpu) { - return emulate_instruction(vcpu, 0) == EMULATE_DONE; + return kvm_emulate_instruction(vcpu, 0) == EMULATE_DONE; } static int handle_invlpg(struct kvm_vcpu *vcpu) @@ -7547,7 +7574,7 @@ static int handle_apic_access(struct kvm_vcpu *vcpu) return kvm_skip_emulated_instruction(vcpu); } } - return emulate_instruction(vcpu, 0) == EMULATE_DONE; + return kvm_emulate_instruction(vcpu, 0) == EMULATE_DONE; } static int handle_apic_eoi_induced(struct kvm_vcpu *vcpu) @@ -7704,8 +7731,8 @@ static int handle_ept_misconfig(struct kvm_vcpu *vcpu) if (!static_cpu_has(X86_FEATURE_HYPERVISOR)) return kvm_skip_emulated_instruction(vcpu); else - return x86_emulate_instruction(vcpu, gpa, EMULTYPE_SKIP, - NULL, 0) == EMULATE_DONE; + return kvm_emulate_instruction(vcpu, EMULTYPE_SKIP) == + EMULATE_DONE; } return kvm_mmu_page_fault(vcpu, gpa, PFERR_RSVD_MASK, NULL, 0); @@ -7748,7 +7775,7 @@ static int handle_invalid_guest_state(struct kvm_vcpu *vcpu) if (kvm_test_request(KVM_REQ_EVENT, vcpu)) return 1; - err = emulate_instruction(vcpu, 0); + err = kvm_emulate_instruction(vcpu, 0); if (err == EMULATE_USER_EXIT) { ++vcpu->stat.mmio_exits; @@ -7966,6 +7993,9 @@ static __init int hardware_setup(void) kvm_x86_ops->enable_log_dirty_pt_masked = NULL; } + if (!cpu_has_vmx_preemption_timer()) + kvm_x86_ops->request_immediate_exit = __kvm_request_immediate_exit; + if (cpu_has_vmx_preemption_timer() && enable_preemption_timer) { u64 vmx_msr; @@ -9208,7 +9238,8 @@ static int handle_pml_full(struct kvm_vcpu *vcpu) static int handle_preemption_timer(struct kvm_vcpu *vcpu) { - kvm_lapic_expired_hv_timer(vcpu); + if (!to_vmx(vcpu)->req_immediate_exit) + kvm_lapic_expired_hv_timer(vcpu); return 1; } @@ -10595,24 +10626,43 @@ static void atomic_switch_perf_msrs(struct vcpu_vmx *vmx) msrs[i].host, false); } -static void vmx_arm_hv_timer(struct kvm_vcpu *vcpu) +static void vmx_arm_hv_timer(struct vcpu_vmx *vmx, u32 val) +{ + vmcs_write32(VMX_PREEMPTION_TIMER_VALUE, val); + if (!vmx->loaded_vmcs->hv_timer_armed) + vmcs_set_bits(PIN_BASED_VM_EXEC_CONTROL, + PIN_BASED_VMX_PREEMPTION_TIMER); + vmx->loaded_vmcs->hv_timer_armed = true; +} + +static void vmx_update_hv_timer(struct kvm_vcpu *vcpu) { struct vcpu_vmx *vmx = to_vmx(vcpu); u64 tscl; u32 delta_tsc; - if (vmx->hv_deadline_tsc == -1) + if (vmx->req_immediate_exit) { + vmx_arm_hv_timer(vmx, 0); return; + } - tscl = rdtsc(); - if (vmx->hv_deadline_tsc > tscl) - /* sure to be 32 bit only because checked on set_hv_timer */ - delta_tsc = (u32)((vmx->hv_deadline_tsc - tscl) >> - cpu_preemption_timer_multi); - else - delta_tsc = 0; + if (vmx->hv_deadline_tsc != -1) { + tscl = rdtsc(); + if (vmx->hv_deadline_tsc > tscl) + /* set_hv_timer ensures the delta fits in 32-bits */ + delta_tsc = (u32)((vmx->hv_deadline_tsc - tscl) >> + cpu_preemption_timer_multi); + else + delta_tsc = 0; + + vmx_arm_hv_timer(vmx, delta_tsc); + return; + } - vmcs_write32(VMX_PREEMPTION_TIMER_VALUE, delta_tsc); + if (vmx->loaded_vmcs->hv_timer_armed) + vmcs_clear_bits(PIN_BASED_VM_EXEC_CONTROL, + PIN_BASED_VMX_PREEMPTION_TIMER); + vmx->loaded_vmcs->hv_timer_armed = false; } static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) @@ -10672,7 +10722,7 @@ static void __noclone vmx_vcpu_run(struct kvm_vcpu *vcpu) atomic_switch_perf_msrs(vmx); - vmx_arm_hv_timer(vcpu); + vmx_update_hv_timer(vcpu); /* * If this vCPU has touched SPEC_CTRL, restore the guest's value if @@ -11427,16 +11477,18 @@ static void vmx_start_preemption_timer(struct kvm_vcpu *vcpu) u64 preemption_timeout = get_vmcs12(vcpu)->vmx_preemption_timer_value; struct vcpu_vmx *vmx = to_vmx(vcpu); - if (vcpu->arch.virtual_tsc_khz == 0) - return; - - /* Make sure short timeouts reliably trigger an immediate vmexit. - * hrtimer_start does not guarantee this. */ - if (preemption_timeout <= 1) { + /* + * A timer value of zero is architecturally guaranteed to cause + * a VMExit prior to executing any instructions in the guest. + */ + if (preemption_timeout == 0) { vmx_preemption_timer_fn(&vmx->nested.preemption_timer); return; } + if (vcpu->arch.virtual_tsc_khz == 0) + return; + preemption_timeout <<= VMX_MISC_EMULATED_PREEMPTION_TIMER_RATE; preemption_timeout *= 1000000; do_div(preemption_timeout, vcpu->arch.virtual_tsc_khz); @@ -11646,11 +11698,15 @@ static int nested_vmx_check_apicv_controls(struct kvm_vcpu *vcpu, * bits 15:8 should be zero in posted_intr_nv, * the descriptor address has been already checked * in nested_get_vmcs12_pages. + * + * bits 5:0 of posted_intr_desc_addr should be zero. */ if (nested_cpu_has_posted_intr(vmcs12) && (!nested_cpu_has_vid(vmcs12) || !nested_exit_intr_ack_set(vcpu) || - vmcs12->posted_intr_nv & 0xff00)) + (vmcs12->posted_intr_nv & 0xff00) || + (vmcs12->posted_intr_desc_addr & 0x3f) || + (!page_address_valid(vcpu, vmcs12->posted_intr_desc_addr)))) return -EINVAL; /* tpr shadow is needed by all apicv features. */ @@ -12076,11 +12132,10 @@ static int prepare_vmcs02(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12, exec_control = vmcs12->pin_based_vm_exec_control; - /* Preemption timer setting is only taken from vmcs01. */ - exec_control &= ~PIN_BASED_VMX_PREEMPTION_TIMER; + /* Preemption timer setting is computed directly in vmx_vcpu_run. */ exec_control |= vmcs_config.pin_based_exec_ctrl; - if (vmx->hv_deadline_tsc == -1) - exec_control &= ~PIN_BASED_VMX_PREEMPTION_TIMER; + exec_control &= ~PIN_BASED_VMX_PREEMPTION_TIMER; + vmx->loaded_vmcs->hv_timer_armed = false; /* Posted interrupts setting is only taken from vmcs12. */ if (nested_cpu_has_posted_intr(vmcs12)) { @@ -12318,6 +12373,9 @@ static int check_vmentry_prereqs(struct kvm_vcpu *vcpu, struct vmcs12 *vmcs12) vmcs12->guest_activity_state != GUEST_ACTIVITY_HLT) return VMXERR_ENTRY_INVALID_CONTROL_FIELD; + if (nested_cpu_has_vpid(vmcs12) && !vmcs12->virtual_processor_id) + return VMXERR_ENTRY_INVALID_CONTROL_FIELD; + if (nested_vmx_check_io_bitmap_controls(vcpu, vmcs12)) return VMXERR_ENTRY_INVALID_CONTROL_FIELD; @@ -12537,8 +12595,11 @@ static int enter_vmx_non_root_mode(struct kvm_vcpu *vcpu, u32 *exit_qual) struct vmcs12 *vmcs12 = get_vmcs12(vcpu); bool from_vmentry = !!exit_qual; u32 dummy_exit_qual; + u32 vmcs01_cpu_exec_ctrl; int r = 0; + vmcs01_cpu_exec_ctrl = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL); + enter_guest_mode(vcpu); if (!(vmcs12->vm_entry_controls & VM_ENTRY_LOAD_DEBUG_CONTROLS)) @@ -12575,6 +12636,25 @@ static int enter_vmx_non_root_mode(struct kvm_vcpu *vcpu, u32 *exit_qual) } /* + * If L1 had a pending IRQ/NMI until it executed + * VMLAUNCH/VMRESUME which wasn't delivered because it was + * disallowed (e.g. interrupts disabled), L0 needs to + * evaluate if this pending event should cause an exit from L2 + * to L1 or delivered directly to L2 (e.g. In case L1 don't + * intercept EXTERNAL_INTERRUPT). + * + * Usually this would be handled by L0 requesting a + * IRQ/NMI window by setting VMCS accordingly. However, + * this setting was done on VMCS01 and now VMCS02 is active + * instead. Thus, we force L0 to perform pending event + * evaluation by requesting a KVM_REQ_EVENT. + */ + if (vmcs01_cpu_exec_ctrl & + (CPU_BASED_VIRTUAL_INTR_PENDING | CPU_BASED_VIRTUAL_NMI_PENDING)) { + kvm_make_request(KVM_REQ_EVENT, vcpu); + } + + /* * Note no nested_vmx_succeed or nested_vmx_fail here. At this point * we are no longer running L1, and VMLAUNCH/VMRESUME has not yet * returned as far as L1 is concerned. It will only return (and set @@ -12841,6 +12921,11 @@ static int vmx_check_nested_events(struct kvm_vcpu *vcpu, bool external_intr) return 0; } +static void vmx_request_immediate_exit(struct kvm_vcpu *vcpu) +{ + to_vmx(vcpu)->req_immediate_exit = true; +} + static u32 vmx_get_preemption_timer_value(struct kvm_vcpu *vcpu) { ktime_t remaining = @@ -13231,12 +13316,7 @@ static void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason, vmcs_write32(VM_EXIT_MSR_LOAD_COUNT, vmx->msr_autoload.host.nr); vmcs_write32(VM_ENTRY_MSR_LOAD_COUNT, vmx->msr_autoload.guest.nr); vmcs_write64(TSC_OFFSET, vcpu->arch.tsc_offset); - if (vmx->hv_deadline_tsc == -1) - vmcs_clear_bits(PIN_BASED_VM_EXEC_CONTROL, - PIN_BASED_VMX_PREEMPTION_TIMER); - else - vmcs_set_bits(PIN_BASED_VM_EXEC_CONTROL, - PIN_BASED_VMX_PREEMPTION_TIMER); + if (kvm_has_tsc_control) decache_tsc_multiplier(vmx); @@ -13440,18 +13520,12 @@ static int vmx_set_hv_timer(struct kvm_vcpu *vcpu, u64 guest_deadline_tsc) return -ERANGE; vmx->hv_deadline_tsc = tscl + delta_tsc; - vmcs_set_bits(PIN_BASED_VM_EXEC_CONTROL, - PIN_BASED_VMX_PREEMPTION_TIMER); - return delta_tsc == 0; } static void vmx_cancel_hv_timer(struct kvm_vcpu *vcpu) { - struct vcpu_vmx *vmx = to_vmx(vcpu); - vmx->hv_deadline_tsc = -1; - vmcs_clear_bits(PIN_BASED_VM_EXEC_CONTROL, - PIN_BASED_VMX_PREEMPTION_TIMER); + to_vmx(vcpu)->hv_deadline_tsc = -1; } #endif @@ -13932,6 +14006,14 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu, ~(KVM_STATE_NESTED_SMM_GUEST_MODE | KVM_STATE_NESTED_SMM_VMXON)) return -EINVAL; + /* + * SMM temporarily disables VMX, so we cannot be in guest mode, + * nor can VMLAUNCH/VMRESUME be pending. Outside SMM, SMM flags + * must be zero. + */ + if (is_smm(vcpu) ? kvm_state->flags : kvm_state->vmx.smm.flags) + return -EINVAL; + if ((kvm_state->vmx.smm.flags & KVM_STATE_NESTED_SMM_GUEST_MODE) && !(kvm_state->vmx.smm.flags & KVM_STATE_NESTED_SMM_VMXON)) return -EINVAL; @@ -13988,9 +14070,6 @@ static int vmx_set_nested_state(struct kvm_vcpu *vcpu, check_vmentry_postreqs(vcpu, vmcs12, &exit_qual)) return -EINVAL; - if (kvm_state->flags & KVM_STATE_NESTED_RUN_PENDING) - vmx->nested.nested_run_pending = 1; - vmx->nested.dirty_vmcs12 = true; ret = enter_vmx_non_root_mode(vcpu, NULL); if (ret) @@ -14078,6 +14157,7 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .apicv_post_state_restore = vmx_apicv_post_state_restore, .hwapic_irr_update = vmx_hwapic_irr_update, .hwapic_isr_update = vmx_hwapic_isr_update, + .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, .sync_pir_to_irr = vmx_sync_pir_to_irr, .deliver_posted_interrupt = vmx_deliver_posted_interrupt, @@ -14111,6 +14191,7 @@ static struct kvm_x86_ops vmx_x86_ops __ro_after_init = { .umip_emulated = vmx_umip_emulated, .check_nested_events = vmx_check_nested_events, + .request_immediate_exit = vmx_request_immediate_exit, .sched_in = vmx_sched_in, diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 506bd2b4b8bb..edbf00ec56b3 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -628,7 +628,7 @@ bool pdptrs_changed(struct kvm_vcpu *vcpu) gfn_t gfn; int r; - if (is_long_mode(vcpu) || !is_pae(vcpu)) + if (is_long_mode(vcpu) || !is_pae(vcpu) || !is_paging(vcpu)) return false; if (!test_bit(VCPU_EXREG_PDPTR, @@ -2537,7 +2537,6 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) break; case MSR_PLATFORM_INFO: if (!msr_info->host_initiated || - data & ~MSR_PLATFORM_INFO_CPUID_FAULT || (!(data & MSR_PLATFORM_INFO_CPUID_FAULT) && cpuid_fault_enabled(vcpu))) return 1; @@ -2780,6 +2779,9 @@ int kvm_get_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) msr_info->data = vcpu->arch.osvw.status; break; case MSR_PLATFORM_INFO: + if (!msr_info->host_initiated && + !vcpu->kvm->arch.guest_can_read_msr_platform_info) + return 1; msr_info->data = vcpu->arch.msr_platform_info; break; case MSR_MISC_FEATURES_ENABLES: @@ -2927,6 +2929,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_SPLIT_IRQCHIP: case KVM_CAP_IMMEDIATE_EXIT: case KVM_CAP_GET_MSR_FEATURES: + case KVM_CAP_MSR_PLATFORM_INFO: r = 1; break; case KVM_CAP_SYNC_REGS: @@ -4007,19 +4010,23 @@ long kvm_arch_vcpu_ioctl(struct file *filp, break; BUILD_BUG_ON(sizeof(user_data_size) != sizeof(user_kvm_nested_state->size)); + r = -EFAULT; if (get_user(user_data_size, &user_kvm_nested_state->size)) - return -EFAULT; + break; r = kvm_x86_ops->get_nested_state(vcpu, user_kvm_nested_state, user_data_size); if (r < 0) - return r; + break; if (r > user_data_size) { if (put_user(r, &user_kvm_nested_state->size)) - return -EFAULT; - return -E2BIG; + r = -EFAULT; + else + r = -E2BIG; + break; } + r = 0; break; } @@ -4031,19 +4038,21 @@ long kvm_arch_vcpu_ioctl(struct file *filp, if (!kvm_x86_ops->set_nested_state) break; + r = -EFAULT; if (copy_from_user(&kvm_state, user_kvm_nested_state, sizeof(kvm_state))) - return -EFAULT; + break; + r = -EINVAL; if (kvm_state.size < sizeof(kvm_state)) - return -EINVAL; + break; if (kvm_state.flags & ~(KVM_STATE_NESTED_RUN_PENDING | KVM_STATE_NESTED_GUEST_MODE)) - return -EINVAL; + break; /* nested_run_pending implies guest_mode. */ if (kvm_state.flags == KVM_STATE_NESTED_RUN_PENDING) - return -EINVAL; + break; r = kvm_x86_ops->set_nested_state(vcpu, user_kvm_nested_state, &kvm_state); break; @@ -4350,6 +4359,10 @@ split_irqchip_unlock: kvm->arch.pause_in_guest = true; r = 0; break; + case KVM_CAP_MSR_PLATFORM_INFO: + kvm->arch.guest_can_read_msr_platform_info = cap->args[0]; + r = 0; + break; default: r = -EINVAL; break; @@ -4987,7 +5000,7 @@ int handle_ud(struct kvm_vcpu *vcpu) emul_type = 0; } - er = emulate_instruction(vcpu, emul_type); + er = kvm_emulate_instruction(vcpu, emul_type); if (er == EMULATE_USER_EXIT) return 0; if (er != EMULATE_DONE) @@ -5870,7 +5883,10 @@ static bool reexecute_instruction(struct kvm_vcpu *vcpu, gva_t cr2, gpa_t gpa = cr2; kvm_pfn_t pfn; - if (emulation_type & EMULTYPE_NO_REEXECUTE) + if (!(emulation_type & EMULTYPE_ALLOW_RETRY)) + return false; + + if (WARN_ON_ONCE(is_guest_mode(vcpu))) return false; if (!vcpu->arch.mmu.direct_map) { @@ -5958,7 +5974,10 @@ static bool retry_instruction(struct x86_emulate_ctxt *ctxt, */ vcpu->arch.last_retry_eip = vcpu->arch.last_retry_addr = 0; - if (!(emulation_type & EMULTYPE_RETRY)) + if (!(emulation_type & EMULTYPE_ALLOW_RETRY)) + return false; + + if (WARN_ON_ONCE(is_guest_mode(vcpu))) return false; if (x86_page_table_writing_insn(ctxt)) @@ -6276,7 +6295,19 @@ restart: return r; } -EXPORT_SYMBOL_GPL(x86_emulate_instruction); + +int kvm_emulate_instruction(struct kvm_vcpu *vcpu, int emulation_type) +{ + return x86_emulate_instruction(vcpu, 0, emulation_type, NULL, 0); +} +EXPORT_SYMBOL_GPL(kvm_emulate_instruction); + +int kvm_emulate_instruction_from_buffer(struct kvm_vcpu *vcpu, + void *insn, int insn_len) +{ + return x86_emulate_instruction(vcpu, 0, 0, insn, insn_len); +} +EXPORT_SYMBOL_GPL(kvm_emulate_instruction_from_buffer); static int kvm_fast_pio_out(struct kvm_vcpu *vcpu, int size, unsigned short port) @@ -7343,6 +7374,12 @@ void kvm_vcpu_reload_apic_access_page(struct kvm_vcpu *vcpu) } EXPORT_SYMBOL_GPL(kvm_vcpu_reload_apic_access_page); +void __kvm_request_immediate_exit(struct kvm_vcpu *vcpu) +{ + smp_send_reschedule(vcpu->cpu); +} +EXPORT_SYMBOL_GPL(__kvm_request_immediate_exit); + /* * Returns 1 to let vcpu_run() continue the guest execution loop without * exiting to the userspace. Otherwise, the value will be returned to the @@ -7547,7 +7584,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) if (req_immediate_exit) { kvm_make_request(KVM_REQ_EVENT, vcpu); - smp_send_reschedule(vcpu->cpu); + kvm_x86_ops->request_immediate_exit(vcpu); } trace_kvm_entry(vcpu->vcpu_id); @@ -7734,7 +7771,7 @@ static inline int complete_emulated_io(struct kvm_vcpu *vcpu) { int r; vcpu->srcu_idx = srcu_read_lock(&vcpu->kvm->srcu); - r = emulate_instruction(vcpu, EMULTYPE_NO_DECODE); + r = kvm_emulate_instruction(vcpu, EMULTYPE_NO_DECODE); srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx); if (r != EMULATE_DONE) return 0; @@ -7811,6 +7848,29 @@ static int complete_emulated_mmio(struct kvm_vcpu *vcpu) return 0; } +/* Swap (qemu) user FPU context for the guest FPU context. */ +static void kvm_load_guest_fpu(struct kvm_vcpu *vcpu) +{ + preempt_disable(); + copy_fpregs_to_fpstate(&vcpu->arch.user_fpu); + /* PKRU is separately restored in kvm_x86_ops->run. */ + __copy_kernel_to_fpregs(&vcpu->arch.guest_fpu.state, + ~XFEATURE_MASK_PKRU); + preempt_enable(); + trace_kvm_fpu(1); +} + +/* When vcpu_run ends, restore user space FPU context. */ +static void kvm_put_guest_fpu(struct kvm_vcpu *vcpu) +{ + preempt_disable(); + copy_fpregs_to_fpstate(&vcpu->arch.guest_fpu); + copy_kernel_to_fpregs(&vcpu->arch.user_fpu.state); + preempt_enable(); + ++vcpu->stat.fpu_reload; + trace_kvm_fpu(0); +} + int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run) { int r; @@ -8159,7 +8219,7 @@ static int __set_sregs(struct kvm_vcpu *vcpu, struct kvm_sregs *sregs) kvm_update_cpuid(vcpu); idx = srcu_read_lock(&vcpu->kvm->srcu); - if (!is_long_mode(vcpu) && is_pae(vcpu)) { + if (!is_long_mode(vcpu) && is_pae(vcpu) && is_paging(vcpu)) { load_pdptrs(vcpu, vcpu->arch.walk_mmu, kvm_read_cr3(vcpu)); mmu_reset_needed = 1; } @@ -8388,29 +8448,6 @@ static void fx_init(struct kvm_vcpu *vcpu) vcpu->arch.cr0 |= X86_CR0_ET; } -/* Swap (qemu) user FPU context for the guest FPU context. */ -void kvm_load_guest_fpu(struct kvm_vcpu *vcpu) -{ - preempt_disable(); - copy_fpregs_to_fpstate(&vcpu->arch.user_fpu); - /* PKRU is separately restored in kvm_x86_ops->run. */ - __copy_kernel_to_fpregs(&vcpu->arch.guest_fpu.state, - ~XFEATURE_MASK_PKRU); - preempt_enable(); - trace_kvm_fpu(1); -} - -/* When vcpu_run ends, restore user space FPU context. */ -void kvm_put_guest_fpu(struct kvm_vcpu *vcpu) -{ - preempt_disable(); - copy_fpregs_to_fpstate(&vcpu->arch.guest_fpu); - copy_kernel_to_fpregs(&vcpu->arch.user_fpu.state); - preempt_enable(); - ++vcpu->stat.fpu_reload; - trace_kvm_fpu(0); -} - void kvm_arch_vcpu_free(struct kvm_vcpu *vcpu) { void *wbinvd_dirty_mask = vcpu->arch.wbinvd_dirty_mask; @@ -8834,6 +8871,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) kvm->arch.kvmclock_offset = -ktime_get_boot_ns(); pvclock_update_vm_gtod_copy(kvm); + kvm->arch.guest_can_read_msr_platform_info = true; + INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn); INIT_DELAYED_WORK(&kvm->arch.kvmclock_sync_work, kvmclock_sync_fn); @@ -9182,6 +9221,13 @@ void kvm_arch_flush_shadow_memslot(struct kvm *kvm, kvm_page_track_flush_slot(kvm, slot); } +static inline bool kvm_guest_apic_has_interrupt(struct kvm_vcpu *vcpu) +{ + return (is_guest_mode(vcpu) && + kvm_x86_ops->guest_apic_has_interrupt && + kvm_x86_ops->guest_apic_has_interrupt(vcpu)); +} + static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) { if (!list_empty_careful(&vcpu->async_pf.done)) @@ -9206,7 +9252,8 @@ static inline bool kvm_vcpu_has_events(struct kvm_vcpu *vcpu) return true; if (kvm_arch_interrupt_allowed(vcpu) && - kvm_cpu_has_interrupt(vcpu)) + (kvm_cpu_has_interrupt(vcpu) || + kvm_guest_apic_has_interrupt(vcpu))) return true; if (kvm_hv_has_stimer_pending(vcpu)) diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 257f27620bc2..67b9568613f3 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -274,6 +274,8 @@ int kvm_mtrr_get_msr(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata); bool kvm_mtrr_check_gfn_range_consistency(struct kvm_vcpu *vcpu, gfn_t gfn, int page_num); bool kvm_vector_hashing_enabled(void); +int x86_emulate_instruction(struct kvm_vcpu *vcpu, unsigned long cr2, + int emulation_type, void *insn, int insn_len); #define KVM_SUPPORTED_XCR0 (XFEATURE_MASK_FP | XFEATURE_MASK_SSE \ | XFEATURE_MASK_YMM | XFEATURE_MASK_BNDREGS \ diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 7a8fc26c1115..faca978ebf9d 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -815,10 +815,14 @@ void free_kernel_image_pages(void *begin, void *end) set_memory_np_noalias(begin_ul, len_pages); } +void __weak mem_encrypt_free_decrypted_mem(void) { } + void __ref free_initmem(void) { e820__reallocate_tables(); + mem_encrypt_free_decrypted_mem(); + free_kernel_image_pages(&__init_begin, &__init_end); } diff --git a/arch/x86/mm/mem_encrypt.c b/arch/x86/mm/mem_encrypt.c index b2de398d1fd3..006f373f54ab 100644 --- a/arch/x86/mm/mem_encrypt.c +++ b/arch/x86/mm/mem_encrypt.c @@ -348,6 +348,30 @@ bool sev_active(void) EXPORT_SYMBOL(sev_active); /* Architecture __weak replacement functions */ +void __init mem_encrypt_free_decrypted_mem(void) +{ + unsigned long vaddr, vaddr_end, npages; + int r; + + vaddr = (unsigned long)__start_bss_decrypted_unused; + vaddr_end = (unsigned long)__end_bss_decrypted; + npages = (vaddr_end - vaddr) >> PAGE_SHIFT; + + /* + * The unused memory range was mapped decrypted, change the encryption + * attribute from decrypted to encrypted before freeing it. + */ + if (mem_encrypt_active()) { + r = set_memory_encrypted(vaddr, npages); + if (r) { + pr_warn("failed to free unused decrypted pages\n"); + return; + } + } + + free_init_pages("unused decrypted", vaddr, vaddr_end); +} + void __init mem_encrypt_init(void) { if (!sme_me_mask) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index e848a4811785..089e78c4effd 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -269,7 +269,7 @@ static void mop_up_one_pmd(struct mm_struct *mm, pgd_t *pgdp) if (pgd_val(pgd) != 0) { pmd_t *pmd = (pmd_t *)pgd_page_vaddr(pgd); - *pgdp = native_make_pgd(0); + pgd_clear(pgdp); paravirt_release_pmd(pgd_val(pgd) >> PAGE_SHIFT); pmd_free(mm, pmd); @@ -494,7 +494,7 @@ int ptep_set_access_flags(struct vm_area_struct *vma, int changed = !pte_same(*ptep, entry); if (changed && dirty) - *ptep = entry; + set_pte(ptep, entry); return changed; } @@ -509,7 +509,7 @@ int pmdp_set_access_flags(struct vm_area_struct *vma, VM_BUG_ON(address & ~HPAGE_PMD_MASK); if (changed && dirty) { - *pmdp = entry; + set_pmd(pmdp, entry); /* * We had a write-protection fault here and changed the pmd * to to more permissive. No need to flush the TLB for that, @@ -529,7 +529,7 @@ int pudp_set_access_flags(struct vm_area_struct *vma, unsigned long address, VM_BUG_ON(address & ~HPAGE_PUD_MASK); if (changed && dirty) { - *pudp = entry; + set_pud(pudp, entry); /* * We had a write-protection fault here and changed the pud * to to more permissive. No need to flush the TLB for that, @@ -637,6 +637,15 @@ void __native_set_fixmap(enum fixed_addresses idx, pte_t pte) { unsigned long address = __fix_to_virt(idx); +#ifdef CONFIG_X86_64 + /* + * Ensure that the static initial page tables are covering the + * fixmap completely. + */ + BUILD_BUG_ON(__end_of_permanent_fixed_addresses > + (FIXMAP_PMD_NUM * PTRS_PER_PTE)); +#endif + if (idx >= __end_of_fixed_addresses) { BUG(); return; diff --git a/arch/x86/platform/efi/efi_32.c b/arch/x86/platform/efi/efi_32.c index 05ca14222463..9959657127f4 100644 --- a/arch/x86/platform/efi/efi_32.c +++ b/arch/x86/platform/efi/efi_32.c @@ -85,10 +85,9 @@ pgd_t * __init efi_call_phys_prolog(void) void __init efi_call_phys_epilog(pgd_t *save_pgd) { + load_fixmap_gdt(0); load_cr3(save_pgd); __flush_tlb_all(); - - load_fixmap_gdt(0); } void __init efi_runtime_update_mappings(void) diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c index 2fe5c9b1816b..dd461c0167ef 100644 --- a/arch/x86/xen/mmu_pv.c +++ b/arch/x86/xen/mmu_pv.c @@ -1907,7 +1907,7 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) /* L3_k[511] -> level2_fixmap_pgt */ convert_pfn_mfn(level3_kernel_pgt); - /* L3_k[511][506] -> level1_fixmap_pgt */ + /* L3_k[511][508-FIXMAP_PMD_NUM ... 507] -> level1_fixmap_pgt */ convert_pfn_mfn(level2_fixmap_pgt); /* We get [511][511] and have Xen's version of level2_kernel_pgt */ @@ -1952,7 +1952,11 @@ void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn) set_page_prot(level2_ident_pgt, PAGE_KERNEL_RO); set_page_prot(level2_kernel_pgt, PAGE_KERNEL_RO); set_page_prot(level2_fixmap_pgt, PAGE_KERNEL_RO); - set_page_prot(level1_fixmap_pgt, PAGE_KERNEL_RO); + + for (i = 0; i < FIXMAP_PMD_NUM; i++) { + set_page_prot(level1_fixmap_pgt + i * PTRS_PER_PTE, + PAGE_KERNEL_RO); + } /* Pin down new L4 */ pin_pagetable_pfn(MMUEXT_PIN_L4_TABLE, diff --git a/arch/x86/xen/pmu.c b/arch/x86/xen/pmu.c index 7d00d4ad44d4..95997e6c0696 100644 --- a/arch/x86/xen/pmu.c +++ b/arch/x86/xen/pmu.c @@ -478,7 +478,7 @@ static void xen_convert_regs(const struct xen_pmu_regs *xen_regs, irqreturn_t xen_pmu_irq_handler(int irq, void *dev_id) { int err, ret = IRQ_NONE; - struct pt_regs regs; + struct pt_regs regs = {0}; const struct xen_pmu_data *xenpmu_data = get_xenpmu_data(); uint8_t xenpmu_flags = get_xenpmu_flags(); diff --git a/arch/xtensa/Kconfig b/arch/xtensa/Kconfig index 04d038f3b6fa..b9ad83a0ee5d 100644 --- a/arch/xtensa/Kconfig +++ b/arch/xtensa/Kconfig @@ -4,6 +4,7 @@ config ZONE_DMA config XTENSA def_bool y + select ARCH_HAS_SG_CHAIN select ARCH_HAS_SYNC_DMA_FOR_CPU select ARCH_HAS_SYNC_DMA_FOR_DEVICE select ARCH_NO_COHERENT_DMA_MMAP if !MMU diff --git a/arch/xtensa/Makefile b/arch/xtensa/Makefile index 295c120ed099..d67e30faff9c 100644 --- a/arch/xtensa/Makefile +++ b/arch/xtensa/Makefile @@ -64,11 +64,7 @@ endif vardirs := $(patsubst %,arch/xtensa/variants/%/,$(variant-y)) plfdirs := $(patsubst %,arch/xtensa/platforms/%/,$(platform-y)) -ifeq ($(KBUILD_SRC),) -KBUILD_CPPFLAGS += $(patsubst %,-I%include,$(vardirs) $(plfdirs)) -else KBUILD_CPPFLAGS += $(patsubst %,-I$(srctree)/%include,$(vardirs) $(plfdirs)) -endif KBUILD_DEFCONFIG := iss_defconfig diff --git a/arch/xtensa/platforms/iss/setup.c b/arch/xtensa/platforms/iss/setup.c index f4bbb28026f8..58709e89a8ed 100644 --- a/arch/xtensa/platforms/iss/setup.c +++ b/arch/xtensa/platforms/iss/setup.c @@ -78,23 +78,28 @@ static struct notifier_block iss_panic_block = { void __init platform_setup(char **p_cmdline) { + static void *argv[COMMAND_LINE_SIZE / sizeof(void *)] __initdata; + static char cmdline[COMMAND_LINE_SIZE] __initdata; int argc = simc_argc(); int argv_size = simc_argv_size(); if (argc > 1) { - void **argv = alloc_bootmem(argv_size); - char *cmdline = alloc_bootmem(argv_size); - int i; + if (argv_size > sizeof(argv)) { + pr_err("%s: command line too long: argv_size = %d\n", + __func__, argv_size); + } else { + int i; - cmdline[0] = 0; - simc_argv((void *)argv); + cmdline[0] = 0; + simc_argv((void *)argv); - for (i = 1; i < argc; ++i) { - if (i > 1) - strcat(cmdline, " "); - strcat(cmdline, argv[i]); + for (i = 1; i < argc; ++i) { + if (i > 1) + strcat(cmdline, " "); + strcat(cmdline, argv[i]); + } + *p_cmdline = cmdline; } - *p_cmdline = cmdline; } atomic_notifier_chain_register(&panic_notifier_list, &iss_panic_block); diff --git a/block/bfq-cgroup.c b/block/bfq-cgroup.c index 58c6efa9f9a9..9fe5952d117d 100644 --- a/block/bfq-cgroup.c +++ b/block/bfq-cgroup.c @@ -275,9 +275,9 @@ static void bfqg_and_blkg_get(struct bfq_group *bfqg) void bfqg_and_blkg_put(struct bfq_group *bfqg) { - bfqg_put(bfqg); - blkg_put(bfqg_to_blkg(bfqg)); + + bfqg_put(bfqg); } /* @stats = 0 */ diff --git a/block/bio.c b/block/bio.c index b12966e415d3..0093bed81c0e 100644 --- a/block/bio.c +++ b/block/bio.c @@ -1684,7 +1684,7 @@ void generic_end_io_acct(struct request_queue *q, int req_op, const int sgrp = op_stat_group(req_op); int cpu = part_stat_lock(); - part_stat_add(cpu, part, ticks[sgrp], duration); + part_stat_add(cpu, part, nsecs[sgrp], jiffies_to_nsecs(duration)); part_round_stats(q, cpu, part); part_dec_in_flight(q, part, op_is_write(req_op)); @@ -2015,7 +2015,8 @@ int bio_associate_blkg(struct bio *bio, struct blkcg_gq *blkg) { if (unlikely(bio->bi_blkg)) return -EBUSY; - blkg_get(blkg); + if (!blkg_try_get(blkg)) + return -ENODEV; bio->bi_blkg = blkg; return 0; } diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c index 694595b29b8f..c630e02836a8 100644 --- a/block/blk-cgroup.c +++ b/block/blk-cgroup.c @@ -310,28 +310,11 @@ struct blkcg_gq *blkg_lookup_create(struct blkcg *blkcg, } } -static void blkg_pd_offline(struct blkcg_gq *blkg) -{ - int i; - - lockdep_assert_held(blkg->q->queue_lock); - lockdep_assert_held(&blkg->blkcg->lock); - - for (i = 0; i < BLKCG_MAX_POLS; i++) { - struct blkcg_policy *pol = blkcg_policy[i]; - - if (blkg->pd[i] && !blkg->pd[i]->offline && - pol->pd_offline_fn) { - pol->pd_offline_fn(blkg->pd[i]); - blkg->pd[i]->offline = true; - } - } -} - static void blkg_destroy(struct blkcg_gq *blkg) { struct blkcg *blkcg = blkg->blkcg; struct blkcg_gq *parent = blkg->parent; + int i; lockdep_assert_held(blkg->q->queue_lock); lockdep_assert_held(&blkcg->lock); @@ -340,6 +323,13 @@ static void blkg_destroy(struct blkcg_gq *blkg) WARN_ON_ONCE(list_empty(&blkg->q_node)); WARN_ON_ONCE(hlist_unhashed(&blkg->blkcg_node)); + for (i = 0; i < BLKCG_MAX_POLS; i++) { + struct blkcg_policy *pol = blkcg_policy[i]; + + if (blkg->pd[i] && pol->pd_offline_fn) + pol->pd_offline_fn(blkg->pd[i]); + } + if (parent) { blkg_rwstat_add_aux(&parent->stat_bytes, &blkg->stat_bytes); blkg_rwstat_add_aux(&parent->stat_ios, &blkg->stat_ios); @@ -382,7 +372,6 @@ static void blkg_destroy_all(struct request_queue *q) struct blkcg *blkcg = blkg->blkcg; spin_lock(&blkcg->lock); - blkg_pd_offline(blkg); blkg_destroy(blkg); spin_unlock(&blkcg->lock); } @@ -1053,59 +1042,64 @@ static struct cftype blkcg_legacy_files[] = { { } /* terminate */ }; +/* + * blkcg destruction is a three-stage process. + * + * 1. Destruction starts. The blkcg_css_offline() callback is invoked + * which offlines writeback. Here we tie the next stage of blkg destruction + * to the completion of writeback associated with the blkcg. This lets us + * avoid punting potentially large amounts of outstanding writeback to root + * while maintaining any ongoing policies. The next stage is triggered when + * the nr_cgwbs count goes to zero. + * + * 2. When the nr_cgwbs count goes to zero, blkcg_destroy_blkgs() is called + * and handles the destruction of blkgs. Here the css reference held by + * the blkg is put back eventually allowing blkcg_css_free() to be called. + * This work may occur in cgwb_release_workfn() on the cgwb_release + * workqueue. Any submitted ios that fail to get the blkg ref will be + * punted to the root_blkg. + * + * 3. Once the blkcg ref count goes to zero, blkcg_css_free() is called. + * This finally frees the blkcg. + */ + /** * blkcg_css_offline - cgroup css_offline callback * @css: css of interest * - * This function is called when @css is about to go away and responsible - * for offlining all blkgs pd and killing all wbs associated with @css. - * blkgs pd offline should be done while holding both q and blkcg locks. - * As blkcg lock is nested inside q lock, this function performs reverse - * double lock dancing. - * - * This is the blkcg counterpart of ioc_release_fn(). + * This function is called when @css is about to go away. Here the cgwbs are + * offlined first and only once writeback associated with the blkcg has + * finished do we start step 2 (see above). */ static void blkcg_css_offline(struct cgroup_subsys_state *css) { struct blkcg *blkcg = css_to_blkcg(css); - struct blkcg_gq *blkg; - - spin_lock_irq(&blkcg->lock); - - hlist_for_each_entry(blkg, &blkcg->blkg_list, blkcg_node) { - struct request_queue *q = blkg->q; - - if (spin_trylock(q->queue_lock)) { - blkg_pd_offline(blkg); - spin_unlock(q->queue_lock); - } else { - spin_unlock_irq(&blkcg->lock); - cpu_relax(); - spin_lock_irq(&blkcg->lock); - } - } - - spin_unlock_irq(&blkcg->lock); + /* this prevents anyone from attaching or migrating to this blkcg */ wb_blkcg_offline(blkcg); + + /* put the base cgwb reference allowing step 2 to be triggered */ + blkcg_cgwb_put(blkcg); } /** - * blkcg_destroy_all_blkgs - destroy all blkgs associated with a blkcg + * blkcg_destroy_blkgs - responsible for shooting down blkgs * @blkcg: blkcg of interest * - * This function is called when blkcg css is about to free and responsible for - * destroying all blkgs associated with @blkcg. - * blkgs should be removed while holding both q and blkcg locks. As blkcg lock + * blkgs should be removed while holding both q and blkcg locks. As blkcg lock * is nested inside q lock, this function performs reverse double lock dancing. + * Destroying the blkgs releases the reference held on the blkcg's css allowing + * blkcg_css_free to eventually be called. + * + * This is the blkcg counterpart of ioc_release_fn(). */ -static void blkcg_destroy_all_blkgs(struct blkcg *blkcg) +void blkcg_destroy_blkgs(struct blkcg *blkcg) { spin_lock_irq(&blkcg->lock); + while (!hlist_empty(&blkcg->blkg_list)) { struct blkcg_gq *blkg = hlist_entry(blkcg->blkg_list.first, - struct blkcg_gq, - blkcg_node); + struct blkcg_gq, blkcg_node); struct request_queue *q = blkg->q; if (spin_trylock(q->queue_lock)) { @@ -1117,6 +1111,7 @@ static void blkcg_destroy_all_blkgs(struct blkcg *blkcg) spin_lock_irq(&blkcg->lock); } } + spin_unlock_irq(&blkcg->lock); } @@ -1125,8 +1120,6 @@ static void blkcg_css_free(struct cgroup_subsys_state *css) struct blkcg *blkcg = css_to_blkcg(css); int i; - blkcg_destroy_all_blkgs(blkcg); - mutex_lock(&blkcg_pol_mutex); list_del(&blkcg->all_blkcgs_node); @@ -1189,6 +1182,7 @@ blkcg_css_alloc(struct cgroup_subsys_state *parent_css) INIT_HLIST_HEAD(&blkcg->blkg_list); #ifdef CONFIG_CGROUP_WRITEBACK INIT_LIST_HEAD(&blkcg->cgwb_list); + refcount_set(&blkcg->cgwb_refcnt, 1); #endif list_add_tail(&blkcg->all_blkcgs_node, &all_blkcgs); @@ -1480,11 +1474,8 @@ void blkcg_deactivate_policy(struct request_queue *q, list_for_each_entry(blkg, &q->blkg_list, q_node) { if (blkg->pd[pol->plid]) { - if (!blkg->pd[pol->plid]->offline && - pol->pd_offline_fn) { + if (pol->pd_offline_fn) pol->pd_offline_fn(blkg->pd[pol->plid]); - blkg->pd[pol->plid]->offline = true; - } pol->pd_free_fn(blkg->pd[pol->plid]); blkg->pd[pol->plid] = NULL; } @@ -1519,8 +1510,10 @@ int blkcg_policy_register(struct blkcg_policy *pol) for (i = 0; i < BLKCG_MAX_POLS; i++) if (!blkcg_policy[i]) break; - if (i >= BLKCG_MAX_POLS) + if (i >= BLKCG_MAX_POLS) { + pr_warn("blkcg_policy_register: BLKCG_MAX_POLS too small\n"); goto err_unlock; + } /* Make sure cpd/pd_alloc_fn and cpd/pd_free_fn in pairs */ if ((!pol->cpd_alloc_fn ^ !pol->cpd_free_fn) || diff --git a/block/blk-core.c b/block/blk-core.c index dee56c282efb..cff0a60ee200 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -2163,9 +2163,12 @@ static inline bool bio_check_ro(struct bio *bio, struct hd_struct *part) { const int op = bio_op(bio); - if (part->policy && (op_is_write(op) && !op_is_flush(op))) { + if (part->policy && op_is_write(op)) { char b[BDEVNAME_SIZE]; + if (op_is_flush(bio->bi_opf) && !bio_sectors(bio)) + return false; + WARN_ONCE(1, "generic_make_request: Trying to write " "to read-only block-device %s (partno %d)\n", @@ -2730,17 +2733,15 @@ void blk_account_io_done(struct request *req, u64 now) * containing request is enough. */ if (blk_do_io_stat(req) && !(req->rq_flags & RQF_FLUSH_SEQ)) { - unsigned long duration; const int sgrp = op_stat_group(req_op(req)); struct hd_struct *part; int cpu; - duration = nsecs_to_jiffies(now - req->start_time_ns); cpu = part_stat_lock(); part = req->part; part_stat_inc(cpu, part, ios[sgrp]); - part_stat_add(cpu, part, ticks[sgrp], duration); + part_stat_add(cpu, part, nsecs[sgrp], now - req->start_time_ns); part_round_stats(req->q, cpu, part); part_dec_in_flight(req->q, part, rq_data_dir(req)); diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index 94e1ed667b6e..41317c50a446 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -322,16 +322,11 @@ void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, /* * __blk_mq_update_nr_hw_queues will update the nr_hw_queues and - * queue_hw_ctx after freeze the queue. So we could use q_usage_counter - * to avoid race with it. __blk_mq_update_nr_hw_queues will users - * synchronize_rcu to ensure all of the users go out of the critical - * section below and see zeroed q_usage_counter. + * queue_hw_ctx after freeze the queue, so we use q_usage_counter + * to avoid race with it. */ - rcu_read_lock(); - if (percpu_ref_is_zero(&q->q_usage_counter)) { - rcu_read_unlock(); + if (!percpu_ref_tryget(&q->q_usage_counter)) return; - } queue_for_each_hw_ctx(q, hctx, i) { struct blk_mq_tags *tags = hctx->tags; @@ -347,7 +342,7 @@ void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, bt_for_each(hctx, &tags->breserved_tags, fn, priv, true); bt_for_each(hctx, &tags->bitmap_tags, fn, priv, false); } - rcu_read_unlock(); + blk_queue_exit(q); } static int bt_alloc(struct sbitmap_queue *bt, unsigned int depth, diff --git a/block/blk-mq.c b/block/blk-mq.c index 85a1c1a59c72..e3c39ea8e17b 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1628,7 +1628,7 @@ void blk_mq_flush_plug_list(struct blk_plug *plug, bool from_schedule) BUG_ON(!rq->q); if (rq->mq_ctx != this_ctx) { if (this_ctx) { - trace_block_unplug(this_q, depth, from_schedule); + trace_block_unplug(this_q, depth, !from_schedule); blk_mq_sched_insert_requests(this_q, this_ctx, &ctx_list, from_schedule); @@ -1648,7 +1648,7 @@ void blk_mq_flush_plug_list(struct blk_plug *plug, bool from_schedule) * on 'ctx_list'. Do those. */ if (this_ctx) { - trace_block_unplug(this_q, depth, from_schedule); + trace_block_unplug(this_q, depth, !from_schedule); blk_mq_sched_insert_requests(this_q, this_ctx, &ctx_list, from_schedule); } diff --git a/block/blk-throttle.c b/block/blk-throttle.c index a3eede00d302..01d0620a4e4a 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2129,8 +2129,9 @@ static inline void throtl_update_latency_buckets(struct throtl_data *td) static void blk_throtl_assoc_bio(struct throtl_grp *tg, struct bio *bio) { #ifdef CONFIG_BLK_DEV_THROTTLING_LOW - if (bio->bi_css) - bio_associate_blkg(bio, tg_to_blkg(tg)); + /* fallback to root_blkg if we fail to get a blkg ref */ + if (bio->bi_css && (bio_associate_blkg(bio, tg_to_blkg(tg)) == -ENODEV)) + bio_associate_blkg(bio, bio->bi_disk->queue->root_blkg); bio_issue_init(&bio->bi_issue, bio_sectors(bio)); #endif } diff --git a/block/elevator.c b/block/elevator.c index 6a06b5d040e5..fae58b2f906f 100644 --- a/block/elevator.c +++ b/block/elevator.c @@ -609,7 +609,7 @@ void elv_drain_elevator(struct request_queue *q) while (e->type->ops.sq.elevator_dispatch_fn(q, 1)) ; - if (q->nr_sorted && printed++ < 10) { + if (q->nr_sorted && !blk_queue_is_zoned(q) && printed++ < 10 ) { printk(KERN_ERR "%s: forced dispatching is broken " "(nr_sorted=%u), please report this\n", q->elevator->type->elevator_name, q->nr_sorted); diff --git a/block/genhd.c b/block/genhd.c index 8cc719a37b32..be5bab20b2ab 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -1343,18 +1343,18 @@ static int diskstats_show(struct seq_file *seqf, void *v) part_stat_read(hd, ios[STAT_READ]), part_stat_read(hd, merges[STAT_READ]), part_stat_read(hd, sectors[STAT_READ]), - jiffies_to_msecs(part_stat_read(hd, ticks[STAT_READ])), + (unsigned int)part_stat_read_msecs(hd, STAT_READ), part_stat_read(hd, ios[STAT_WRITE]), part_stat_read(hd, merges[STAT_WRITE]), part_stat_read(hd, sectors[STAT_WRITE]), - jiffies_to_msecs(part_stat_read(hd, ticks[STAT_WRITE])), + (unsigned int)part_stat_read_msecs(hd, STAT_WRITE), inflight[0], jiffies_to_msecs(part_stat_read(hd, io_ticks)), jiffies_to_msecs(part_stat_read(hd, time_in_queue)), part_stat_read(hd, ios[STAT_DISCARD]), part_stat_read(hd, merges[STAT_DISCARD]), part_stat_read(hd, sectors[STAT_DISCARD]), - jiffies_to_msecs(part_stat_read(hd, ticks[STAT_DISCARD])) + (unsigned int)part_stat_read_msecs(hd, STAT_DISCARD) ); } disk_part_iter_exit(&piter); diff --git a/block/partition-generic.c b/block/partition-generic.c index 5a8975a1201c..d3d14e81fb12 100644 --- a/block/partition-generic.c +++ b/block/partition-generic.c @@ -136,18 +136,18 @@ ssize_t part_stat_show(struct device *dev, part_stat_read(p, ios[STAT_READ]), part_stat_read(p, merges[STAT_READ]), (unsigned long long)part_stat_read(p, sectors[STAT_READ]), - jiffies_to_msecs(part_stat_read(p, ticks[STAT_READ])), + (unsigned int)part_stat_read_msecs(p, STAT_READ), part_stat_read(p, ios[STAT_WRITE]), part_stat_read(p, merges[STAT_WRITE]), (unsigned long long)part_stat_read(p, sectors[STAT_WRITE]), - jiffies_to_msecs(part_stat_read(p, ticks[STAT_WRITE])), + (unsigned int)part_stat_read_msecs(p, STAT_WRITE), inflight[0], jiffies_to_msecs(part_stat_read(p, io_ticks)), jiffies_to_msecs(part_stat_read(p, time_in_queue)), part_stat_read(p, ios[STAT_DISCARD]), part_stat_read(p, merges[STAT_DISCARD]), (unsigned long long)part_stat_read(p, sectors[STAT_DISCARD]), - jiffies_to_msecs(part_stat_read(p, ticks[STAT_DISCARD]))); + (unsigned int)part_stat_read_msecs(p, STAT_DISCARD)); } ssize_t part_inflight_show(struct device *dev, struct device_attribute *attr, diff --git a/drivers/acpi/acpi_lpss.c b/drivers/acpi/acpi_lpss.c index 9706613eecf9..bf64cfa30feb 100644 --- a/drivers/acpi/acpi_lpss.c +++ b/drivers/acpi/acpi_lpss.c @@ -879,7 +879,7 @@ static void acpi_lpss_dismiss(struct device *dev) #define LPSS_GPIODEF0_DMA_LLP BIT(13) static DEFINE_MUTEX(lpss_iosf_mutex); -static bool lpss_iosf_d3_entered; +static bool lpss_iosf_d3_entered = true; static void lpss_iosf_enter_d3_state(void) { diff --git a/drivers/acpi/bus.c b/drivers/acpi/bus.c index 292088fcc624..d2e29a19890d 100644 --- a/drivers/acpi/bus.c +++ b/drivers/acpi/bus.c @@ -35,11 +35,11 @@ #include <linux/delay.h> #ifdef CONFIG_X86 #include <asm/mpspec.h> +#include <linux/dmi.h> #endif #include <linux/acpi_iort.h> #include <linux/pci.h> #include <acpi/apei.h> -#include <linux/dmi.h> #include <linux/suspend.h> #include "internal.h" @@ -82,10 +82,6 @@ static const struct dmi_system_id dsdt_dmi_table[] __initconst = { }, {} }; -#else -static const struct dmi_system_id dsdt_dmi_table[] __initconst = { - {} -}; #endif /* -------------------------------------------------------------------------- @@ -1033,11 +1029,16 @@ void __init acpi_early_init(void) acpi_permanent_mmap = true; +#ifdef CONFIG_X86 /* * If the machine falls into the DMI check table, - * DSDT will be copied to memory + * DSDT will be copied to memory. + * Note that calling dmi_check_system() here on other architectures + * would not be OK because only x86 initializes dmi early enough. + * Thankfully only x86 systems need such quirks for now. */ dmi_check_system(dsdt_dmi_table); +#endif status = acpi_reallocate_root_table(); if (ACPI_FAILURE(status)) { diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c index 3f3b7b253445..64fd96eada31 100644 --- a/drivers/android/binder_alloc.c +++ b/drivers/android/binder_alloc.c @@ -332,6 +332,35 @@ err_no_vma: return vma ? -ENOMEM : -ESRCH; } + +static inline void binder_alloc_set_vma(struct binder_alloc *alloc, + struct vm_area_struct *vma) +{ + if (vma) + alloc->vma_vm_mm = vma->vm_mm; + /* + * If we see alloc->vma is not NULL, buffer data structures set up + * completely. Look at smp_rmb side binder_alloc_get_vma. + * We also want to guarantee new alloc->vma_vm_mm is always visible + * if alloc->vma is set. + */ + smp_wmb(); + alloc->vma = vma; +} + +static inline struct vm_area_struct *binder_alloc_get_vma( + struct binder_alloc *alloc) +{ + struct vm_area_struct *vma = NULL; + + if (alloc->vma) { + /* Look at description in binder_alloc_set_vma */ + smp_rmb(); + vma = alloc->vma; + } + return vma; +} + static struct binder_buffer *binder_alloc_new_buf_locked( struct binder_alloc *alloc, size_t data_size, @@ -348,7 +377,7 @@ static struct binder_buffer *binder_alloc_new_buf_locked( size_t size, data_offsets_size; int ret; - if (alloc->vma == NULL) { + if (!binder_alloc_get_vma(alloc)) { binder_alloc_debug(BINDER_DEBUG_USER_ERROR, "%d: binder_alloc_buf, no vma\n", alloc->pid); @@ -723,9 +752,7 @@ int binder_alloc_mmap_handler(struct binder_alloc *alloc, buffer->free = 1; binder_insert_free_buffer(alloc, buffer); alloc->free_async_space = alloc->buffer_size / 2; - barrier(); - alloc->vma = vma; - alloc->vma_vm_mm = vma->vm_mm; + binder_alloc_set_vma(alloc, vma); mmgrab(alloc->vma_vm_mm); return 0; @@ -754,10 +781,10 @@ void binder_alloc_deferred_release(struct binder_alloc *alloc) int buffers, page_count; struct binder_buffer *buffer; - BUG_ON(alloc->vma); - buffers = 0; mutex_lock(&alloc->mutex); + BUG_ON(alloc->vma); + while ((n = rb_first(&alloc->allocated_buffers))) { buffer = rb_entry(n, struct binder_buffer, rb_node); @@ -900,7 +927,7 @@ int binder_alloc_get_allocated_count(struct binder_alloc *alloc) */ void binder_alloc_vma_close(struct binder_alloc *alloc) { - WRITE_ONCE(alloc->vma, NULL); + binder_alloc_set_vma(alloc, NULL); } /** @@ -935,7 +962,7 @@ enum lru_status binder_alloc_free_page(struct list_head *item, index = page - alloc->pages; page_addr = (uintptr_t)alloc->buffer + index * PAGE_SIZE; - vma = alloc->vma; + vma = binder_alloc_get_vma(alloc); if (vma) { if (!mmget_not_zero(alloc->vma_vm_mm)) goto err_mmget; diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c index 172e32840256..a9dd4ea7467d 100644 --- a/drivers/ata/libata-core.c +++ b/drivers/ata/libata-core.c @@ -5359,10 +5359,20 @@ void ata_qc_complete(struct ata_queued_cmd *qc) */ int ata_qc_complete_multiple(struct ata_port *ap, u64 qc_active) { + u64 done_mask, ap_qc_active = ap->qc_active; int nr_done = 0; - u64 done_mask; - done_mask = ap->qc_active ^ qc_active; + /* + * If the internal tag is set on ap->qc_active, then we care about + * bit0 on the passed in qc_active mask. Move that bit up to match + * the internal tag. + */ + if (ap_qc_active & (1ULL << ATA_TAG_INTERNAL)) { + qc_active |= (qc_active & 0x01) << ATA_TAG_INTERNAL; + qc_active ^= qc_active & 0x01; + } + + done_mask = ap_qc_active ^ qc_active; if (unlikely(done_mask & qc_active)) { ata_port_err(ap, "illegal qc_active transition (%08llx->%08llx)\n", @@ -7394,4 +7404,4 @@ EXPORT_SYMBOL_GPL(ata_cable_unknown); EXPORT_SYMBOL_GPL(ata_cable_ignore); EXPORT_SYMBOL_GPL(ata_cable_sata); EXPORT_SYMBOL_GPL(ata_host_get); -EXPORT_SYMBOL_GPL(ata_host_put);
\ No newline at end of file +EXPORT_SYMBOL_GPL(ata_host_put); diff --git a/drivers/base/firmware_loader/main.c b/drivers/base/firmware_loader/main.c index 0943e7065e0e..b3c0498ee433 100644 --- a/drivers/base/firmware_loader/main.c +++ b/drivers/base/firmware_loader/main.c @@ -209,21 +209,24 @@ static struct fw_priv *__lookup_fw_priv(const char *fw_name) static int alloc_lookup_fw_priv(const char *fw_name, struct firmware_cache *fwc, struct fw_priv **fw_priv, void *dbuf, - size_t size) + size_t size, enum fw_opt opt_flags) { struct fw_priv *tmp; spin_lock(&fwc->lock); - tmp = __lookup_fw_priv(fw_name); - if (tmp) { - kref_get(&tmp->ref); - spin_unlock(&fwc->lock); - *fw_priv = tmp; - pr_debug("batched request - sharing the same struct fw_priv and lookup for multiple requests\n"); - return 1; + if (!(opt_flags & FW_OPT_NOCACHE)) { + tmp = __lookup_fw_priv(fw_name); + if (tmp) { + kref_get(&tmp->ref); + spin_unlock(&fwc->lock); + *fw_priv = tmp; + pr_debug("batched request - sharing the same struct fw_priv and lookup for multiple requests\n"); + return 1; + } } + tmp = __allocate_fw_priv(fw_name, fwc, dbuf, size); - if (tmp) + if (tmp && !(opt_flags & FW_OPT_NOCACHE)) list_add(&tmp->list, &fwc->head); spin_unlock(&fwc->lock); @@ -493,7 +496,8 @@ int assign_fw(struct firmware *fw, struct device *device, */ static int _request_firmware_prepare(struct firmware **firmware_p, const char *name, - struct device *device, void *dbuf, size_t size) + struct device *device, void *dbuf, size_t size, + enum fw_opt opt_flags) { struct firmware *firmware; struct fw_priv *fw_priv; @@ -511,7 +515,8 @@ _request_firmware_prepare(struct firmware **firmware_p, const char *name, return 0; /* assigned */ } - ret = alloc_lookup_fw_priv(name, &fw_cache, &fw_priv, dbuf, size); + ret = alloc_lookup_fw_priv(name, &fw_cache, &fw_priv, dbuf, size, + opt_flags); /* * bind with 'priv' now to avoid warning in failure path @@ -571,7 +576,8 @@ _request_firmware(const struct firmware **firmware_p, const char *name, goto out; } - ret = _request_firmware_prepare(&fw, name, device, buf, size); + ret = _request_firmware_prepare(&fw, name, device, buf, size, + opt_flags); if (ret <= 0) /* error or already assigned */ goto out; diff --git a/drivers/base/memory.c b/drivers/base/memory.c index c8a1cb0b6136..817320c7c4c1 100644 --- a/drivers/base/memory.c +++ b/drivers/base/memory.c @@ -417,25 +417,23 @@ static ssize_t show_valid_zones(struct device *dev, int nid; /* - * The block contains more than one zone can not be offlined. - * This can happen e.g. for ZONE_DMA and ZONE_DMA32 - */ - if (!test_pages_in_a_zone(start_pfn, start_pfn + nr_pages, &valid_start_pfn, &valid_end_pfn)) - return sprintf(buf, "none\n"); - - start_pfn = valid_start_pfn; - nr_pages = valid_end_pfn - start_pfn; - - /* * Check the existing zone. Make sure that we do that only on the * online nodes otherwise the page_zone is not reliable */ if (mem->state == MEM_ONLINE) { + /* + * The block contains more than one zone can not be offlined. + * This can happen e.g. for ZONE_DMA and ZONE_DMA32 + */ + if (!test_pages_in_a_zone(start_pfn, start_pfn + nr_pages, + &valid_start_pfn, &valid_end_pfn)) + return sprintf(buf, "none\n"); + start_pfn = valid_start_pfn; strcat(buf, page_zone(pfn_to_page(start_pfn))->name); goto out; } - nid = pfn_to_nid(start_pfn); + nid = mem->nid; default_zone = zone_for_pfn_range(MMOP_ONLINE_KEEP, nid, start_pfn, nr_pages); strcat(buf, default_zone->name); diff --git a/drivers/block/floppy.c b/drivers/block/floppy.c index 48f622728ce6..f2b6f4da1034 100644 --- a/drivers/block/floppy.c +++ b/drivers/block/floppy.c @@ -3467,6 +3467,9 @@ static int fd_locked_ioctl(struct block_device *bdev, fmode_t mode, unsigned int (struct floppy_struct **)&outparam); if (ret) return ret; + memcpy(&inparam.g, outparam, + offsetof(struct floppy_struct, name)); + outparam = &inparam.g; break; case FDMSGON: UDP->flags |= FTD_MSG; diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 3863c00372bb..14a51254c3db 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -1239,6 +1239,9 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd, case NBD_SET_SOCK: return nbd_add_socket(nbd, arg, false); case NBD_SET_BLKSIZE: + if (!arg || !is_power_of_2(arg) || arg < 512 || + arg > PAGE_SIZE) + return -EINVAL; nbd_size_set(nbd, arg, div_s64(config->bytesize, arg)); return 0; diff --git a/drivers/block/null_blk.h b/drivers/block/null_blk.h index d81781f22dba..34e0030f0592 100644 --- a/drivers/block/null_blk.h +++ b/drivers/block/null_blk.h @@ -87,10 +87,10 @@ struct nullb { #ifdef CONFIG_BLK_DEV_ZONED int null_zone_init(struct nullb_device *dev); void null_zone_exit(struct nullb_device *dev); -blk_status_t null_zone_report(struct nullb *nullb, - struct nullb_cmd *cmd); -void null_zone_write(struct nullb_cmd *cmd); -void null_zone_reset(struct nullb_cmd *cmd); +blk_status_t null_zone_report(struct nullb *nullb, struct bio *bio); +void null_zone_write(struct nullb_cmd *cmd, sector_t sector, + unsigned int nr_sectors); +void null_zone_reset(struct nullb_cmd *cmd, sector_t sector); #else static inline int null_zone_init(struct nullb_device *dev) { @@ -98,11 +98,14 @@ static inline int null_zone_init(struct nullb_device *dev) } static inline void null_zone_exit(struct nullb_device *dev) {} static inline blk_status_t null_zone_report(struct nullb *nullb, - struct nullb_cmd *cmd) + struct bio *bio) { return BLK_STS_NOTSUPP; } -static inline void null_zone_write(struct nullb_cmd *cmd) {} -static inline void null_zone_reset(struct nullb_cmd *cmd) {} +static inline void null_zone_write(struct nullb_cmd *cmd, sector_t sector, + unsigned int nr_sectors) +{ +} +static inline void null_zone_reset(struct nullb_cmd *cmd, sector_t sector) {} #endif /* CONFIG_BLK_DEV_ZONED */ #endif /* __NULL_BLK_H */ diff --git a/drivers/block/null_blk_main.c b/drivers/block/null_blk_main.c index 6127e3ff7b4b..093b614d6524 100644 --- a/drivers/block/null_blk_main.c +++ b/drivers/block/null_blk_main.c @@ -1157,16 +1157,33 @@ static void null_restart_queue_async(struct nullb *nullb) } } +static bool cmd_report_zone(struct nullb *nullb, struct nullb_cmd *cmd) +{ + struct nullb_device *dev = cmd->nq->dev; + + if (dev->queue_mode == NULL_Q_BIO) { + if (bio_op(cmd->bio) == REQ_OP_ZONE_REPORT) { + cmd->error = null_zone_report(nullb, cmd->bio); + return true; + } + } else { + if (req_op(cmd->rq) == REQ_OP_ZONE_REPORT) { + cmd->error = null_zone_report(nullb, cmd->rq->bio); + return true; + } + } + + return false; +} + static blk_status_t null_handle_cmd(struct nullb_cmd *cmd) { struct nullb_device *dev = cmd->nq->dev; struct nullb *nullb = dev->nullb; int err = 0; - if (req_op(cmd->rq) == REQ_OP_ZONE_REPORT) { - cmd->error = null_zone_report(nullb, cmd); + if (cmd_report_zone(nullb, cmd)) goto out; - } if (test_bit(NULLB_DEV_FL_THROTTLED, &dev->flags)) { struct request *rq = cmd->rq; @@ -1234,10 +1251,24 @@ static blk_status_t null_handle_cmd(struct nullb_cmd *cmd) cmd->error = errno_to_blk_status(err); if (!cmd->error && dev->zoned) { - if (req_op(cmd->rq) == REQ_OP_WRITE) - null_zone_write(cmd); - else if (req_op(cmd->rq) == REQ_OP_ZONE_RESET) - null_zone_reset(cmd); + sector_t sector; + unsigned int nr_sectors; + int op; + + if (dev->queue_mode == NULL_Q_BIO) { + op = bio_op(cmd->bio); + sector = cmd->bio->bi_iter.bi_sector; + nr_sectors = cmd->bio->bi_iter.bi_size >> 9; + } else { + op = req_op(cmd->rq); + sector = blk_rq_pos(cmd->rq); + nr_sectors = blk_rq_sectors(cmd->rq); + } + + if (op == REQ_OP_WRITE) + null_zone_write(cmd, sector, nr_sectors); + else if (op == REQ_OP_ZONE_RESET) + null_zone_reset(cmd, sector); } out: /* Complete IO by inline, softirq or timer */ diff --git a/drivers/block/null_blk_zoned.c b/drivers/block/null_blk_zoned.c index a979ca00d7be..7c6b86d98700 100644 --- a/drivers/block/null_blk_zoned.c +++ b/drivers/block/null_blk_zoned.c @@ -48,8 +48,8 @@ void null_zone_exit(struct nullb_device *dev) kvfree(dev->zones); } -static void null_zone_fill_rq(struct nullb_device *dev, struct request *rq, - unsigned int zno, unsigned int nr_zones) +static void null_zone_fill_bio(struct nullb_device *dev, struct bio *bio, + unsigned int zno, unsigned int nr_zones) { struct blk_zone_report_hdr *hdr = NULL; struct bio_vec bvec; @@ -57,7 +57,7 @@ static void null_zone_fill_rq(struct nullb_device *dev, struct request *rq, void *addr; unsigned int zones_to_cpy; - bio_for_each_segment(bvec, rq->bio, iter) { + bio_for_each_segment(bvec, bio, iter) { addr = kmap_atomic(bvec.bv_page); zones_to_cpy = bvec.bv_len / sizeof(struct blk_zone); @@ -84,29 +84,24 @@ static void null_zone_fill_rq(struct nullb_device *dev, struct request *rq, } } -blk_status_t null_zone_report(struct nullb *nullb, - struct nullb_cmd *cmd) +blk_status_t null_zone_report(struct nullb *nullb, struct bio *bio) { struct nullb_device *dev = nullb->dev; - struct request *rq = cmd->rq; - unsigned int zno = null_zone_no(dev, blk_rq_pos(rq)); + unsigned int zno = null_zone_no(dev, bio->bi_iter.bi_sector); unsigned int nr_zones = dev->nr_zones - zno; - unsigned int max_zones = (blk_rq_bytes(rq) / - sizeof(struct blk_zone)) - 1; + unsigned int max_zones; + max_zones = (bio->bi_iter.bi_size / sizeof(struct blk_zone)) - 1; nr_zones = min_t(unsigned int, nr_zones, max_zones); - - null_zone_fill_rq(nullb->dev, rq, zno, nr_zones); + null_zone_fill_bio(nullb->dev, bio, zno, nr_zones); return BLK_STS_OK; } -void null_zone_write(struct nullb_cmd *cmd) +void null_zone_write(struct nullb_cmd *cmd, sector_t sector, + unsigned int nr_sectors) { struct nullb_device *dev = cmd->nq->dev; - struct request *rq = cmd->rq; - sector_t sector = blk_rq_pos(rq); - unsigned int rq_sectors = blk_rq_sectors(rq); unsigned int zno = null_zone_no(dev, sector); struct blk_zone *zone = &dev->zones[zno]; @@ -118,7 +113,7 @@ void null_zone_write(struct nullb_cmd *cmd) case BLK_ZONE_COND_EMPTY: case BLK_ZONE_COND_IMP_OPEN: /* Writes must be at the write pointer position */ - if (blk_rq_pos(rq) != zone->wp) { + if (sector != zone->wp) { cmd->error = BLK_STS_IOERR; break; } @@ -126,7 +121,7 @@ void null_zone_write(struct nullb_cmd *cmd) if (zone->cond == BLK_ZONE_COND_EMPTY) zone->cond = BLK_ZONE_COND_IMP_OPEN; - zone->wp += rq_sectors; + zone->wp += nr_sectors; if (zone->wp == zone->start + zone->len) zone->cond = BLK_ZONE_COND_FULL; break; @@ -137,11 +132,10 @@ void null_zone_write(struct nullb_cmd *cmd) } } -void null_zone_reset(struct nullb_cmd *cmd) +void null_zone_reset(struct nullb_cmd *cmd, sector_t sector) { struct nullb_device *dev = cmd->nq->dev; - struct request *rq = cmd->rq; - unsigned int zno = null_zone_no(dev, blk_rq_pos(rq)); + unsigned int zno = null_zone_no(dev, sector); struct blk_zone *zone = &dev->zones[zno]; zone->cond = BLK_ZONE_COND_EMPTY; diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 7915f3b03736..73ed5f3a862d 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@ -4207,11 +4207,13 @@ static ssize_t rbd_parent_show(struct device *dev, count += sprintf(&buf[count], "%s" "pool_id %llu\npool_name %s\n" + "pool_ns %s\n" "image_id %s\nimage_name %s\n" "snap_id %llu\nsnap_name %s\n" "overlap %llu\n", !count ? "" : "\n", /* first? */ spec->pool_id, spec->pool_name, + spec->pool_ns ?: "", spec->image_id, spec->image_name ?: "(unknown)", spec->snap_id, spec->snap_name, rbd_dev->parent_overlap); @@ -4584,47 +4586,177 @@ static int rbd_dev_v2_features(struct rbd_device *rbd_dev) &rbd_dev->header.features); } +struct parent_image_info { + u64 pool_id; + const char *pool_ns; + const char *image_id; + u64 snap_id; + + bool has_overlap; + u64 overlap; +}; + +/* + * The caller is responsible for @pii. + */ +static int decode_parent_image_spec(void **p, void *end, + struct parent_image_info *pii) +{ + u8 struct_v; + u32 struct_len; + int ret; + + ret = ceph_start_decoding(p, end, 1, "ParentImageSpec", + &struct_v, &struct_len); + if (ret) + return ret; + + ceph_decode_64_safe(p, end, pii->pool_id, e_inval); + pii->pool_ns = ceph_extract_encoded_string(p, end, NULL, GFP_KERNEL); + if (IS_ERR(pii->pool_ns)) { + ret = PTR_ERR(pii->pool_ns); + pii->pool_ns = NULL; + return ret; + } + pii->image_id = ceph_extract_encoded_string(p, end, NULL, GFP_KERNEL); + if (IS_ERR(pii->image_id)) { + ret = PTR_ERR(pii->image_id); + pii->image_id = NULL; + return ret; + } + ceph_decode_64_safe(p, end, pii->snap_id, e_inval); + return 0; + +e_inval: + return -EINVAL; +} + +static int __get_parent_info(struct rbd_device *rbd_dev, + struct page *req_page, + struct page *reply_page, + struct parent_image_info *pii) +{ + struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc; + size_t reply_len = PAGE_SIZE; + void *p, *end; + int ret; + + ret = ceph_osdc_call(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc, + "rbd", "parent_get", CEPH_OSD_FLAG_READ, + req_page, sizeof(u64), reply_page, &reply_len); + if (ret) + return ret == -EOPNOTSUPP ? 1 : ret; + + p = page_address(reply_page); + end = p + reply_len; + ret = decode_parent_image_spec(&p, end, pii); + if (ret) + return ret; + + ret = ceph_osdc_call(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc, + "rbd", "parent_overlap_get", CEPH_OSD_FLAG_READ, + req_page, sizeof(u64), reply_page, &reply_len); + if (ret) + return ret; + + p = page_address(reply_page); + end = p + reply_len; + ceph_decode_8_safe(&p, end, pii->has_overlap, e_inval); + if (pii->has_overlap) + ceph_decode_64_safe(&p, end, pii->overlap, e_inval); + + return 0; + +e_inval: + return -EINVAL; +} + +/* + * The caller is responsible for @pii. + */ +static int __get_parent_info_legacy(struct rbd_device *rbd_dev, + struct page *req_page, + struct page *reply_page, + struct parent_image_info *pii) +{ + struct ceph_osd_client *osdc = &rbd_dev->rbd_client->client->osdc; + size_t reply_len = PAGE_SIZE; + void *p, *end; + int ret; + + ret = ceph_osdc_call(osdc, &rbd_dev->header_oid, &rbd_dev->header_oloc, + "rbd", "get_parent", CEPH_OSD_FLAG_READ, + req_page, sizeof(u64), reply_page, &reply_len); + if (ret) + return ret; + + p = page_address(reply_page); + end = p + reply_len; + ceph_decode_64_safe(&p, end, pii->pool_id, e_inval); + pii->image_id = ceph_extract_encoded_string(&p, end, NULL, GFP_KERNEL); + if (IS_ERR(pii->image_id)) { + ret = PTR_ERR(pii->image_id); + pii->image_id = NULL; + return ret; + } + ceph_decode_64_safe(&p, end, pii->snap_id, e_inval); + pii->has_overlap = true; + ceph_decode_64_safe(&p, end, pii->overlap, e_inval); + + return 0; + +e_inval: + return -EINVAL; +} + +static int get_parent_info(struct rbd_device *rbd_dev, + struct parent_image_info *pii) +{ + struct page *req_page, *reply_page; + void *p; + int ret; + + req_page = alloc_page(GFP_KERNEL); + if (!req_page) + return -ENOMEM; + + reply_page = alloc_page(GFP_KERNEL); + if (!reply_page) { + __free_page(req_page); + return -ENOMEM; + } + + p = page_address(req_page); + ceph_encode_64(&p, rbd_dev->spec->snap_id); + ret = __get_parent_info(rbd_dev, req_page, reply_page, pii); + if (ret > 0) + ret = __get_parent_info_legacy(rbd_dev, req_page, reply_page, + pii); + + __free_page(req_page); + __free_page(reply_page); + return ret; +} + static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) { struct rbd_spec *parent_spec; - size_t size; - void *reply_buf = NULL; - __le64 snapid; - void *p; - void *end; - u64 pool_id; - char *image_id; - u64 snap_id; - u64 overlap; + struct parent_image_info pii = { 0 }; int ret; parent_spec = rbd_spec_alloc(); if (!parent_spec) return -ENOMEM; - size = sizeof (__le64) + /* pool_id */ - sizeof (__le32) + RBD_IMAGE_ID_LEN_MAX + /* image_id */ - sizeof (__le64) + /* snap_id */ - sizeof (__le64); /* overlap */ - reply_buf = kmalloc(size, GFP_KERNEL); - if (!reply_buf) { - ret = -ENOMEM; + ret = get_parent_info(rbd_dev, &pii); + if (ret) goto out_err; - } - snapid = cpu_to_le64(rbd_dev->spec->snap_id); - ret = rbd_obj_method_sync(rbd_dev, &rbd_dev->header_oid, - &rbd_dev->header_oloc, "get_parent", - &snapid, sizeof(snapid), reply_buf, size); - dout("%s: rbd_obj_method_sync returned %d\n", __func__, ret); - if (ret < 0) - goto out_err; + dout("%s pool_id %llu pool_ns %s image_id %s snap_id %llu has_overlap %d overlap %llu\n", + __func__, pii.pool_id, pii.pool_ns, pii.image_id, pii.snap_id, + pii.has_overlap, pii.overlap); - p = reply_buf; - end = reply_buf + ret; - ret = -ERANGE; - ceph_decode_64_safe(&p, end, pool_id, out_err); - if (pool_id == CEPH_NOPOOL) { + if (pii.pool_id == CEPH_NOPOOL || !pii.has_overlap) { /* * Either the parent never existed, or we have * record of it but the image got flattened so it no @@ -4633,6 +4765,10 @@ static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) * overlap to 0. The effect of this is that all new * requests will be treated as if the image had no * parent. + * + * If !pii.has_overlap, the parent image spec is not + * applicable. It's there to avoid duplication in each + * snapshot record. */ if (rbd_dev->parent_overlap) { rbd_dev->parent_overlap = 0; @@ -4647,51 +4783,36 @@ static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) /* The ceph file layout needs to fit pool id in 32 bits */ ret = -EIO; - if (pool_id > (u64)U32_MAX) { + if (pii.pool_id > (u64)U32_MAX) { rbd_warn(NULL, "parent pool id too large (%llu > %u)", - (unsigned long long)pool_id, U32_MAX); + (unsigned long long)pii.pool_id, U32_MAX); goto out_err; } - image_id = ceph_extract_encoded_string(&p, end, NULL, GFP_KERNEL); - if (IS_ERR(image_id)) { - ret = PTR_ERR(image_id); - goto out_err; - } - ceph_decode_64_safe(&p, end, snap_id, out_err); - ceph_decode_64_safe(&p, end, overlap, out_err); - /* * The parent won't change (except when the clone is * flattened, already handled that). So we only need to * record the parent spec we have not already done so. */ if (!rbd_dev->parent_spec) { - parent_spec->pool_id = pool_id; - parent_spec->image_id = image_id; - parent_spec->snap_id = snap_id; - - /* TODO: support cloning across namespaces */ - if (rbd_dev->spec->pool_ns) { - parent_spec->pool_ns = kstrdup(rbd_dev->spec->pool_ns, - GFP_KERNEL); - if (!parent_spec->pool_ns) { - ret = -ENOMEM; - goto out_err; - } + parent_spec->pool_id = pii.pool_id; + if (pii.pool_ns && *pii.pool_ns) { + parent_spec->pool_ns = pii.pool_ns; + pii.pool_ns = NULL; } + parent_spec->image_id = pii.image_id; + pii.image_id = NULL; + parent_spec->snap_id = pii.snap_id; rbd_dev->parent_spec = parent_spec; parent_spec = NULL; /* rbd_dev now owns this */ - } else { - kfree(image_id); } /* * We always update the parent overlap. If it's zero we issue * a warning, as we will proceed as if there was no parent. */ - if (!overlap) { + if (!pii.overlap) { if (parent_spec) { /* refresh, careful to warn just once */ if (rbd_dev->parent_overlap) @@ -4702,14 +4823,14 @@ static int rbd_dev_v2_parent_info(struct rbd_device *rbd_dev) rbd_warn(rbd_dev, "clone is standalone (overlap 0)"); } } - rbd_dev->parent_overlap = overlap; + rbd_dev->parent_overlap = pii.overlap; out: ret = 0; out_err: - kfree(reply_buf); + kfree(pii.pool_ns); + kfree(pii.image_id); rbd_spec_put(parent_spec); - return ret; } diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index a71d817e900d..429d20131c7e 100644 --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -2670,8 +2670,8 @@ static void purge_persistent_grants(struct blkfront_info *info) list_del(&gnt_list_entry->node); gnttab_end_foreign_access(gnt_list_entry->gref, 0, 0UL); rinfo->persistent_gnts_c--; - __free_page(gnt_list_entry->page); - kfree(gnt_list_entry); + gnt_list_entry->gref = GRANT_INVALID_REF; + list_add_tail(&gnt_list_entry->node, &rinfo->grants); } spin_unlock_irqrestore(&rinfo->ring_lock, flags); diff --git a/drivers/bluetooth/hci_ldisc.c b/drivers/bluetooth/hci_ldisc.c index 963bb0309e25..ea6238ed5c0e 100644 --- a/drivers/bluetooth/hci_ldisc.c +++ b/drivers/bluetooth/hci_ldisc.c @@ -543,6 +543,8 @@ static void hci_uart_tty_close(struct tty_struct *tty) } clear_bit(HCI_UART_PROTO_SET, &hu->flags); + percpu_free_rwsem(&hu->proto_lock); + kfree(hu); } diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig index ce277ee0a28a..40728491f37b 100644 --- a/drivers/char/Kconfig +++ b/drivers/char/Kconfig @@ -566,5 +566,5 @@ config RANDOM_TRUST_CPU that CPU manufacturer (perhaps with the insistence or mandate of a Nation State's intelligence or law enforcement agencies) has not installed a hidden back door to compromise the CPU's - random number generation facilities. - + random number generation facilities. This can also be configured + at boot with "random.trust_cpu=on/off". diff --git a/drivers/char/ipmi/ipmi_bt_sm.c b/drivers/char/ipmi/ipmi_bt_sm.c index a3397664f800..97d6856c9c0f 100644 --- a/drivers/char/ipmi/ipmi_bt_sm.c +++ b/drivers/char/ipmi/ipmi_bt_sm.c @@ -59,8 +59,6 @@ enum bt_states { BT_STATE_RESET3, BT_STATE_RESTART, BT_STATE_PRINTME, - BT_STATE_CAPABILITIES_BEGIN, - BT_STATE_CAPABILITIES_END, BT_STATE_LONG_BUSY /* BT doesn't get hosed :-) */ }; @@ -86,7 +84,6 @@ struct si_sm_data { int error_retries; /* end of "common" fields */ int nonzero_status; /* hung BMCs stay all 0 */ enum bt_states complete; /* to divert the state machine */ - int BT_CAP_outreqs; long BT_CAP_req2rsp; int BT_CAP_retries; /* Recommended retries */ }; @@ -137,8 +134,6 @@ static char *state2txt(unsigned char state) case BT_STATE_RESET3: return("RESET3"); case BT_STATE_RESTART: return("RESTART"); case BT_STATE_LONG_BUSY: return("LONG_BUSY"); - case BT_STATE_CAPABILITIES_BEGIN: return("CAP_BEGIN"); - case BT_STATE_CAPABILITIES_END: return("CAP_END"); } return("BAD STATE"); } @@ -185,7 +180,6 @@ static unsigned int bt_init_data(struct si_sm_data *bt, struct si_sm_io *io) bt->complete = BT_STATE_IDLE; /* end here */ bt->BT_CAP_req2rsp = BT_NORMAL_TIMEOUT * USEC_PER_SEC; bt->BT_CAP_retries = BT_NORMAL_RETRY_LIMIT; - /* BT_CAP_outreqs == zero is a flag to read BT Capabilities */ return 3; /* We claim 3 bytes of space; ought to check SPMI table */ } @@ -451,7 +445,7 @@ static enum si_sm_result error_recovery(struct si_sm_data *bt, static enum si_sm_result bt_event(struct si_sm_data *bt, long time) { - unsigned char status, BT_CAP[8]; + unsigned char status; static enum bt_states last_printed = BT_STATE_PRINTME; int i; @@ -504,12 +498,6 @@ static enum si_sm_result bt_event(struct si_sm_data *bt, long time) if (status & BT_H_BUSY) /* clear a leftover H_BUSY */ BT_CONTROL(BT_H_BUSY); - bt->timeout = bt->BT_CAP_req2rsp; - - /* Read BT capabilities if it hasn't been done yet */ - if (!bt->BT_CAP_outreqs) - BT_STATE_CHANGE(BT_STATE_CAPABILITIES_BEGIN, - SI_SM_CALL_WITHOUT_DELAY); BT_SI_SM_RETURN(SI_SM_IDLE); case BT_STATE_XACTION_START: @@ -614,37 +602,6 @@ static enum si_sm_result bt_event(struct si_sm_data *bt, long time) BT_STATE_CHANGE(BT_STATE_XACTION_START, SI_SM_CALL_WITH_DELAY); - /* - * Get BT Capabilities, using timing of upper level state machine. - * Set outreqs to prevent infinite loop on timeout. - */ - case BT_STATE_CAPABILITIES_BEGIN: - bt->BT_CAP_outreqs = 1; - { - unsigned char GetBT_CAP[] = { 0x18, 0x36 }; - bt->state = BT_STATE_IDLE; - bt_start_transaction(bt, GetBT_CAP, sizeof(GetBT_CAP)); - } - bt->complete = BT_STATE_CAPABILITIES_END; - BT_STATE_CHANGE(BT_STATE_XACTION_START, - SI_SM_CALL_WITH_DELAY); - - case BT_STATE_CAPABILITIES_END: - i = bt_get_result(bt, BT_CAP, sizeof(BT_CAP)); - bt_init_data(bt, bt->io); - if ((i == 8) && !BT_CAP[2]) { - bt->BT_CAP_outreqs = BT_CAP[3]; - bt->BT_CAP_req2rsp = BT_CAP[6] * USEC_PER_SEC; - bt->BT_CAP_retries = BT_CAP[7]; - } else - printk(KERN_WARNING "IPMI BT: using default values\n"); - if (!bt->BT_CAP_outreqs) - bt->BT_CAP_outreqs = 1; - printk(KERN_WARNING "IPMI BT: req2rsp=%ld secs retries=%d\n", - bt->BT_CAP_req2rsp / USEC_PER_SEC, bt->BT_CAP_retries); - bt->timeout = bt->BT_CAP_req2rsp; - return SI_SM_CALL_WITHOUT_DELAY; - default: /* should never occur */ return error_recovery(bt, status, @@ -655,6 +612,11 @@ static enum si_sm_result bt_event(struct si_sm_data *bt, long time) static int bt_detect(struct si_sm_data *bt) { + unsigned char GetBT_CAP[] = { 0x18, 0x36 }; + unsigned char BT_CAP[8]; + enum si_sm_result smi_result; + int rv; + /* * It's impossible for the BT status and interrupt registers to be * all 1's, (assuming a properly functioning, self-initialized BMC) @@ -665,6 +627,48 @@ static int bt_detect(struct si_sm_data *bt) if ((BT_STATUS == 0xFF) && (BT_INTMASK_R == 0xFF)) return 1; reset_flags(bt); + + /* + * Try getting the BT capabilities here. + */ + rv = bt_start_transaction(bt, GetBT_CAP, sizeof(GetBT_CAP)); + if (rv) { + dev_warn(bt->io->dev, + "Can't start capabilities transaction: %d\n", rv); + goto out_no_bt_cap; + } + + smi_result = SI_SM_CALL_WITHOUT_DELAY; + for (;;) { + if (smi_result == SI_SM_CALL_WITH_DELAY || + smi_result == SI_SM_CALL_WITH_TICK_DELAY) { + schedule_timeout_uninterruptible(1); + smi_result = bt_event(bt, jiffies_to_usecs(1)); + } else if (smi_result == SI_SM_CALL_WITHOUT_DELAY) { + smi_result = bt_event(bt, 0); + } else + break; + } + + rv = bt_get_result(bt, BT_CAP, sizeof(BT_CAP)); + bt_init_data(bt, bt->io); + if (rv < 8) { + dev_warn(bt->io->dev, "bt cap response too short: %d\n", rv); + goto out_no_bt_cap; + } + + if (BT_CAP[2]) { + dev_warn(bt->io->dev, "Error fetching bt cap: %x\n", BT_CAP[2]); +out_no_bt_cap: + dev_warn(bt->io->dev, "using default values\n"); + } else { + bt->BT_CAP_req2rsp = BT_CAP[6] * USEC_PER_SEC; + bt->BT_CAP_retries = BT_CAP[7]; + } + + dev_info(bt->io->dev, "req2rsp=%ld secs retries=%d\n", + bt->BT_CAP_req2rsp / USEC_PER_SEC, bt->BT_CAP_retries); + return 0; } diff --git a/drivers/char/ipmi/ipmi_msghandler.c b/drivers/char/ipmi/ipmi_msghandler.c index 51832b8a2c62..7fc9612070a1 100644 --- a/drivers/char/ipmi/ipmi_msghandler.c +++ b/drivers/char/ipmi/ipmi_msghandler.c @@ -3381,39 +3381,45 @@ int ipmi_register_smi(const struct ipmi_smi_handlers *handlers, rv = handlers->start_processing(send_info, intf); if (rv) - goto out; + goto out_err; rv = __bmc_get_device_id(intf, NULL, &id, NULL, NULL, i); if (rv) { dev_err(si_dev, "Unable to get the device id: %d\n", rv); - goto out; + goto out_err_started; } mutex_lock(&intf->bmc_reg_mutex); rv = __scan_channels(intf, &id); mutex_unlock(&intf->bmc_reg_mutex); + if (rv) + goto out_err_bmc_reg; - out: - if (rv) { - ipmi_bmc_unregister(intf); - list_del_rcu(&intf->link); - mutex_unlock(&ipmi_interfaces_mutex); - synchronize_srcu(&ipmi_interfaces_srcu); - cleanup_srcu_struct(&intf->users_srcu); - kref_put(&intf->refcount, intf_free); - } else { - /* - * Keep memory order straight for RCU readers. Make - * sure everything else is committed to memory before - * setting intf_num to mark the interface valid. - */ - smp_wmb(); - intf->intf_num = i; - mutex_unlock(&ipmi_interfaces_mutex); + /* + * Keep memory order straight for RCU readers. Make + * sure everything else is committed to memory before + * setting intf_num to mark the interface valid. + */ + smp_wmb(); + intf->intf_num = i; + mutex_unlock(&ipmi_interfaces_mutex); - /* After this point the interface is legal to use. */ - call_smi_watchers(i, intf->si_dev); - } + /* After this point the interface is legal to use. */ + call_smi_watchers(i, intf->si_dev); + + return 0; + + out_err_bmc_reg: + ipmi_bmc_unregister(intf); + out_err_started: + if (intf->handlers->shutdown) + intf->handlers->shutdown(intf->send_info); + out_err: + list_del_rcu(&intf->link); + mutex_unlock(&ipmi_interfaces_mutex); + synchronize_srcu(&ipmi_interfaces_srcu); + cleanup_srcu_struct(&intf->users_srcu); + kref_put(&intf->refcount, intf_free); return rv; } @@ -3504,7 +3510,8 @@ void ipmi_unregister_smi(struct ipmi_smi *intf) } srcu_read_unlock(&intf->users_srcu, index); - intf->handlers->shutdown(intf->send_info); + if (intf->handlers->shutdown) + intf->handlers->shutdown(intf->send_info); cleanup_smi_msgs(intf); diff --git a/drivers/char/ipmi/ipmi_si_intf.c b/drivers/char/ipmi/ipmi_si_intf.c index 90ec010bffbd..5faa917df1b6 100644 --- a/drivers/char/ipmi/ipmi_si_intf.c +++ b/drivers/char/ipmi/ipmi_si_intf.c @@ -2083,18 +2083,9 @@ static int try_smi_init(struct smi_info *new_smi) si_to_str[new_smi->io.si_type]); WARN_ON(new_smi->io.dev->init_name != NULL); - kfree(init_name); - - return 0; - -out_err: - if (new_smi->intf) { - ipmi_unregister_smi(new_smi->intf); - new_smi->intf = NULL; - } + out_err: kfree(init_name); - return rv; } @@ -2227,6 +2218,8 @@ static void shutdown_smi(void *send_info) kfree(smi_info->si_sm); smi_info->si_sm = NULL; + + smi_info->intf = NULL; } /* @@ -2240,10 +2233,8 @@ static void cleanup_one_si(struct smi_info *smi_info) list_del(&smi_info->link); - if (smi_info->intf) { + if (smi_info->intf) ipmi_unregister_smi(smi_info->intf); - smi_info->intf = NULL; - } if (smi_info->pdev) { if (smi_info->pdev_registered) diff --git a/drivers/char/ipmi/ipmi_ssif.c b/drivers/char/ipmi/ipmi_ssif.c index 18e4650c233b..29e67a80fb20 100644 --- a/drivers/char/ipmi/ipmi_ssif.c +++ b/drivers/char/ipmi/ipmi_ssif.c @@ -181,6 +181,8 @@ struct ssif_addr_info { struct device *dev; struct i2c_client *client; + struct i2c_client *added_client; + struct mutex clients_mutex; struct list_head clients; @@ -1214,18 +1216,11 @@ static void shutdown_ssif(void *send_info) complete(&ssif_info->wake_thread); kthread_stop(ssif_info->thread); } - - /* - * No message can be outstanding now, we have removed the - * upper layer and it permitted us to do so. - */ - kfree(ssif_info); } static int ssif_remove(struct i2c_client *client) { struct ssif_info *ssif_info = i2c_get_clientdata(client); - struct ipmi_smi *intf; struct ssif_addr_info *addr_info; if (!ssif_info) @@ -1235,9 +1230,7 @@ static int ssif_remove(struct i2c_client *client) * After this point, we won't deliver anything asychronously * to the message handler. We can unregister ourself. */ - intf = ssif_info->intf; - ssif_info->intf = NULL; - ipmi_unregister_smi(intf); + ipmi_unregister_smi(ssif_info->intf); list_for_each_entry(addr_info, &ssif_infos, link) { if (addr_info->client == client) { @@ -1246,6 +1239,8 @@ static int ssif_remove(struct i2c_client *client) } } + kfree(ssif_info); + return 0; } @@ -1648,15 +1643,9 @@ static int ssif_probe(struct i2c_client *client, const struct i2c_device_id *id) out: if (rv) { - /* - * Note that if addr_info->client is assigned, we - * leave it. The i2c client hangs around even if we - * return a failure here, and the failure here is not - * propagated back to the i2c code. This seems to be - * design intent, strange as it may be. But if we - * don't leave it, ssif_platform_remove will not remove - * the client like it should. - */ + if (addr_info) + addr_info->client = NULL; + dev_err(&client->dev, "Unable to start IPMI SSIF: %d\n", rv); kfree(ssif_info); } @@ -1676,7 +1665,8 @@ static int ssif_adapter_handler(struct device *adev, void *opaque) if (adev->type != &i2c_adapter_type) return 0; - i2c_new_device(to_i2c_adapter(adev), &addr_info->binfo); + addr_info->added_client = i2c_new_device(to_i2c_adapter(adev), + &addr_info->binfo); if (!addr_info->adapter_name) return 1; /* Only try the first I2C adapter by default. */ @@ -1849,7 +1839,7 @@ static int ssif_platform_remove(struct platform_device *dev) return 0; mutex_lock(&ssif_infos_mutex); - i2c_unregister_device(addr_info->client); + i2c_unregister_device(addr_info->added_client); list_del(&addr_info->link); kfree(addr_info); diff --git a/drivers/char/ipmi/kcs_bmc.c b/drivers/char/ipmi/kcs_bmc.c index bb882ab161fe..e6124bd548df 100644 --- a/drivers/char/ipmi/kcs_bmc.c +++ b/drivers/char/ipmi/kcs_bmc.c @@ -16,6 +16,8 @@ #include "kcs_bmc.h" +#define DEVICE_NAME "ipmi-kcs" + #define KCS_MSG_BUFSIZ 1000 #define KCS_ZERO_DATA 0 @@ -429,8 +431,6 @@ struct kcs_bmc *kcs_bmc_alloc(struct device *dev, int sizeof_priv, u32 channel) if (!kcs_bmc) return NULL; - dev_set_name(dev, "ipmi-kcs%u", channel); - spin_lock_init(&kcs_bmc->lock); kcs_bmc->channel = channel; @@ -444,7 +444,8 @@ struct kcs_bmc *kcs_bmc_alloc(struct device *dev, int sizeof_priv, u32 channel) return NULL; kcs_bmc->miscdev.minor = MISC_DYNAMIC_MINOR; - kcs_bmc->miscdev.name = dev_name(dev); + kcs_bmc->miscdev.name = devm_kasprintf(dev, GFP_KERNEL, "%s%u", + DEVICE_NAME, channel); kcs_bmc->miscdev.fops = &kcs_bmc_fops; return kcs_bmc; diff --git a/drivers/char/random.c b/drivers/char/random.c index bf5f99fc36f1..c75b6cdf0053 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -779,6 +779,13 @@ static struct crng_state **crng_node_pool __read_mostly; static void invalidate_batched_entropy(void); +static bool trust_cpu __ro_after_init = IS_ENABLED(CONFIG_RANDOM_TRUST_CPU); +static int __init parse_trust_cpu(char *arg) +{ + return kstrtobool(arg, &trust_cpu); +} +early_param("random.trust_cpu", parse_trust_cpu); + static void crng_initialize(struct crng_state *crng) { int i; @@ -799,12 +806,10 @@ static void crng_initialize(struct crng_state *crng) } crng->state[i] ^= rv; } -#ifdef CONFIG_RANDOM_TRUST_CPU - if (arch_init) { + if (trust_cpu && arch_init) { crng_init = 2; pr_notice("random: crng done (trusting CPU's manufacturer)\n"); } -#endif crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1; } diff --git a/drivers/clk/x86/clk-pmc-atom.c b/drivers/clk/x86/clk-pmc-atom.c index 08ef69945ffb..d977193842df 100644 --- a/drivers/clk/x86/clk-pmc-atom.c +++ b/drivers/clk/x86/clk-pmc-atom.c @@ -55,6 +55,7 @@ struct clk_plt_data { u8 nparents; struct clk_plt *clks[PMC_CLK_NUM]; struct clk_lookup *mclk_lookup; + struct clk_lookup *ether_clk_lookup; }; /* Return an index in parent table */ @@ -186,13 +187,6 @@ static struct clk_plt *plt_clk_register(struct platform_device *pdev, int id, pclk->reg = base + PMC_CLK_CTL_OFFSET + id * PMC_CLK_CTL_SIZE; spin_lock_init(&pclk->lock); - /* - * If the clock was already enabled by the firmware mark it as critical - * to avoid it being gated by the clock framework if no driver owns it. - */ - if (plt_clk_is_enabled(&pclk->hw)) - init.flags |= CLK_IS_CRITICAL; - ret = devm_clk_hw_register(&pdev->dev, &pclk->hw); if (ret) { pclk = ERR_PTR(ret); @@ -351,11 +345,20 @@ static int plt_clk_probe(struct platform_device *pdev) goto err_unreg_clk_plt; } + data->ether_clk_lookup = clkdev_hw_create(&data->clks[4]->hw, + "ether_clk", NULL); + if (!data->ether_clk_lookup) { + err = -ENOMEM; + goto err_drop_mclk; + } + plt_clk_free_parent_names_loop(parent_names, data->nparents); platform_set_drvdata(pdev, data); return 0; +err_drop_mclk: + clkdev_drop(data->mclk_lookup); err_unreg_clk_plt: plt_clk_unregister_loop(data, i); plt_clk_unregister_parents(data); @@ -369,6 +372,7 @@ static int plt_clk_remove(struct platform_device *pdev) data = platform_get_drvdata(pdev); + clkdev_drop(data->ether_clk_lookup); clkdev_drop(data->mclk_lookup); plt_clk_unregister_loop(data, PMC_CLK_NUM); plt_clk_unregister_parents(data); diff --git a/drivers/clocksource/timer-atmel-pit.c b/drivers/clocksource/timer-atmel-pit.c index ec8a4376f74f..2fab18fae4fc 100644 --- a/drivers/clocksource/timer-atmel-pit.c +++ b/drivers/clocksource/timer-atmel-pit.c @@ -180,26 +180,29 @@ static int __init at91sam926x_pit_dt_init(struct device_node *node) data->base = of_iomap(node, 0); if (!data->base) { pr_err("Could not map PIT address\n"); - return -ENXIO; + ret = -ENXIO; + goto exit; } data->mck = of_clk_get(node, 0); if (IS_ERR(data->mck)) { pr_err("Unable to get mck clk\n"); - return PTR_ERR(data->mck); + ret = PTR_ERR(data->mck); + goto exit; } ret = clk_prepare_enable(data->mck); if (ret) { pr_err("Unable to enable mck\n"); - return ret; + goto exit; } /* Get the interrupts property */ data->irq = irq_of_parse_and_map(node, 0); if (!data->irq) { pr_err("Unable to get IRQ from DT\n"); - return -EINVAL; + ret = -EINVAL; + goto exit; } /* @@ -227,7 +230,7 @@ static int __init at91sam926x_pit_dt_init(struct device_node *node) ret = clocksource_register_hz(&data->clksrc, pit_rate); if (ret) { pr_err("Failed to register clocksource\n"); - return ret; + goto exit; } /* Set up irq handler */ @@ -236,7 +239,8 @@ static int __init at91sam926x_pit_dt_init(struct device_node *node) "at91_tick", data); if (ret) { pr_err("Unable to setup IRQ\n"); - return ret; + clocksource_unregister(&data->clksrc); + goto exit; } /* Set up and register clockevents */ @@ -254,6 +258,10 @@ static int __init at91sam926x_pit_dt_init(struct device_node *node) clockevents_register_device(&data->clkevt); return 0; + +exit: + kfree(data); + return ret; } TIMER_OF_DECLARE(at91sam926x_pit, "atmel,at91sam9260-pit", at91sam926x_pit_dt_init); diff --git a/drivers/clocksource/timer-fttmr010.c b/drivers/clocksource/timer-fttmr010.c index c020038ebfab..cf93f6419b51 100644 --- a/drivers/clocksource/timer-fttmr010.c +++ b/drivers/clocksource/timer-fttmr010.c @@ -130,13 +130,17 @@ static int fttmr010_timer_set_next_event(unsigned long cycles, cr &= ~fttmr010->t1_enable_val; writel(cr, fttmr010->base + TIMER_CR); - /* Setup the match register forward/backward in time */ - cr = readl(fttmr010->base + TIMER1_COUNT); - if (fttmr010->count_down) - cr -= cycles; - else - cr += cycles; - writel(cr, fttmr010->base + TIMER1_MATCH1); + if (fttmr010->count_down) { + /* + * ASPEED Timer Controller will load TIMER1_LOAD register + * into TIMER1_COUNT register when the timer is re-enabled. + */ + writel(cycles, fttmr010->base + TIMER1_LOAD); + } else { + /* Setup the match register forward in time */ + cr = readl(fttmr010->base + TIMER1_COUNT); + writel(cr + cycles, fttmr010->base + TIMER1_MATCH1); + } /* Start */ cr = readl(fttmr010->base + TIMER_CR); diff --git a/drivers/clocksource/timer-ti-32k.c b/drivers/clocksource/timer-ti-32k.c index 29e2e1a78a43..6949a9113dbb 100644 --- a/drivers/clocksource/timer-ti-32k.c +++ b/drivers/clocksource/timer-ti-32k.c @@ -97,6 +97,9 @@ static int __init ti_32k_timer_init(struct device_node *np) return -ENXIO; } + if (!of_machine_is_compatible("ti,am43")) + ti_32k_timer.cs.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP; + ti_32k_timer.counter = ti_32k_timer.base; /* diff --git a/drivers/cpufreq/qcom-cpufreq-kryo.c b/drivers/cpufreq/qcom-cpufreq-kryo.c index a1830fa25fc5..2a3675c24032 100644 --- a/drivers/cpufreq/qcom-cpufreq-kryo.c +++ b/drivers/cpufreq/qcom-cpufreq-kryo.c @@ -44,7 +44,7 @@ enum _msm8996_version { struct platform_device *cpufreq_dt_pdev, *kryo_cpufreq_pdev; -static enum _msm8996_version __init qcom_cpufreq_kryo_get_msm_id(void) +static enum _msm8996_version qcom_cpufreq_kryo_get_msm_id(void) { size_t len; u32 *msm_id; @@ -222,7 +222,7 @@ static int __init qcom_cpufreq_kryo_init(void) } module_init(qcom_cpufreq_kryo_init); -static void __init qcom_cpufreq_kryo_exit(void) +static void __exit qcom_cpufreq_kryo_exit(void) { platform_device_unregister(kryo_cpufreq_pdev); platform_driver_unregister(&qcom_cpufreq_kryo_driver); diff --git a/drivers/crypto/ccp/psp-dev.c b/drivers/crypto/ccp/psp-dev.c index 218739b961fe..72790d88236d 100644 --- a/drivers/crypto/ccp/psp-dev.c +++ b/drivers/crypto/ccp/psp-dev.c @@ -38,6 +38,17 @@ static DEFINE_MUTEX(sev_cmd_mutex); static struct sev_misc_dev *misc_dev; static struct psp_device *psp_master; +static int psp_cmd_timeout = 100; +module_param(psp_cmd_timeout, int, 0644); +MODULE_PARM_DESC(psp_cmd_timeout, " default timeout value, in seconds, for PSP commands"); + +static int psp_probe_timeout = 5; +module_param(psp_probe_timeout, int, 0644); +MODULE_PARM_DESC(psp_probe_timeout, " default timeout value, in seconds, during PSP device probe"); + +static bool psp_dead; +static int psp_timeout; + static struct psp_device *psp_alloc_struct(struct sp_device *sp) { struct device *dev = sp->dev; @@ -82,10 +93,19 @@ done: return IRQ_HANDLED; } -static void sev_wait_cmd_ioc(struct psp_device *psp, unsigned int *reg) +static int sev_wait_cmd_ioc(struct psp_device *psp, + unsigned int *reg, unsigned int timeout) { - wait_event(psp->sev_int_queue, psp->sev_int_rcvd); + int ret; + + ret = wait_event_timeout(psp->sev_int_queue, + psp->sev_int_rcvd, timeout * HZ); + if (!ret) + return -ETIMEDOUT; + *reg = ioread32(psp->io_regs + psp->vdata->cmdresp_reg); + + return 0; } static int sev_cmd_buffer_len(int cmd) @@ -133,12 +153,15 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) if (!psp) return -ENODEV; + if (psp_dead) + return -EBUSY; + /* Get the physical address of the command buffer */ phys_lsb = data ? lower_32_bits(__psp_pa(data)) : 0; phys_msb = data ? upper_32_bits(__psp_pa(data)) : 0; - dev_dbg(psp->dev, "sev command id %#x buffer 0x%08x%08x\n", - cmd, phys_msb, phys_lsb); + dev_dbg(psp->dev, "sev command id %#x buffer 0x%08x%08x timeout %us\n", + cmd, phys_msb, phys_lsb, psp_timeout); print_hex_dump_debug("(in): ", DUMP_PREFIX_OFFSET, 16, 2, data, sev_cmd_buffer_len(cmd), false); @@ -154,7 +177,18 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret) iowrite32(reg, psp->io_regs + psp->vdata->cmdresp_reg); /* wait for command completion */ - sev_wait_cmd_ioc(psp, ®); + ret = sev_wait_cmd_ioc(psp, ®, psp_timeout); + if (ret) { + if (psp_ret) + *psp_ret = 0; + + dev_err(psp->dev, "sev command %#x timed out, disabling PSP \n", cmd); + psp_dead = true; + + return ret; + } + + psp_timeout = psp_cmd_timeout; if (psp_ret) *psp_ret = reg & PSP_CMDRESP_ERR_MASK; @@ -888,6 +922,8 @@ void psp_pci_init(void) psp_master = sp->psp_data; + psp_timeout = psp_probe_timeout; + if (sev_get_api_version()) goto err; diff --git a/drivers/dax/device.c b/drivers/dax/device.c index 6fd46083e629..948806e57cee 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -392,7 +392,8 @@ static vm_fault_t dev_dax_huge_fault(struct vm_fault *vmf, { struct file *filp = vmf->vma->vm_file; unsigned long fault_size; - int rc, id; + vm_fault_t rc = VM_FAULT_SIGBUS; + int id; pfn_t pfn; struct dev_dax *dev_dax = filp->private_data; @@ -534,6 +535,11 @@ static unsigned long dax_get_unmapped_area(struct file *filp, return current->mm->get_unmapped_area(filp, addr, len, pgoff, flags); } +static const struct address_space_operations dev_dax_aops = { + .set_page_dirty = noop_set_page_dirty, + .invalidatepage = noop_invalidatepage, +}; + static int dax_open(struct inode *inode, struct file *filp) { struct dax_device *dax_dev = inode_dax(inode); @@ -543,6 +549,7 @@ static int dax_open(struct inode *inode, struct file *filp) dev_dbg(&dev_dax->dev, "trace\n"); inode->i_mapping = __dax_inode->i_mapping; inode->i_mapping->host = __dax_inode; + inode->i_mapping->a_ops = &dev_dax_aops; filp->f_mapping = inode->i_mapping; filp->f_wb_err = filemap_sample_wb_err(filp->f_mapping); filp->private_data = dev_dax; diff --git a/drivers/dma/mic_x100_dma.c b/drivers/dma/mic_x100_dma.c index b76cb17d879c..adfd316db1a8 100644 --- a/drivers/dma/mic_x100_dma.c +++ b/drivers/dma/mic_x100_dma.c @@ -639,7 +639,7 @@ static struct mic_dma_device *mic_dma_dev_reg(struct mbus_device *mbdev, int ret; struct device *dev = &mbdev->dev; - mic_dma_dev = kzalloc(sizeof(*mic_dma_dev), GFP_KERNEL); + mic_dma_dev = devm_kzalloc(dev, sizeof(*mic_dma_dev), GFP_KERNEL); if (!mic_dma_dev) { ret = -ENOMEM; goto alloc_error; @@ -664,7 +664,6 @@ static struct mic_dma_device *mic_dma_dev_reg(struct mbus_device *mbdev, reg_error: mic_dma_uninit(mic_dma_dev); init_error: - kfree(mic_dma_dev); mic_dma_dev = NULL; alloc_error: dev_err(dev, "Error at %s %d ret=%d\n", __func__, __LINE__, ret); @@ -674,7 +673,6 @@ alloc_error: static void mic_dma_dev_unreg(struct mic_dma_device *mic_dma_dev) { mic_dma_uninit(mic_dma_dev); - kfree(mic_dma_dev); } /* DEBUGFS CODE */ diff --git a/drivers/firmware/arm_scmi/perf.c b/drivers/firmware/arm_scmi/perf.c index 721e6c57beae..64342944d917 100644 --- a/drivers/firmware/arm_scmi/perf.c +++ b/drivers/firmware/arm_scmi/perf.c @@ -166,7 +166,13 @@ scmi_perf_domain_attributes_get(const struct scmi_handle *handle, u32 domain, le32_to_cpu(attr->sustained_freq_khz); dom_info->sustained_perf_level = le32_to_cpu(attr->sustained_perf_level); - dom_info->mult_factor = (dom_info->sustained_freq_khz * 1000) / + if (!dom_info->sustained_freq_khz || + !dom_info->sustained_perf_level) + /* CPUFreq converts to kHz, hence default 1000 */ + dom_info->mult_factor = 1000; + else + dom_info->mult_factor = + (dom_info->sustained_freq_khz * 1000) / dom_info->sustained_perf_level; memcpy(dom_info->name, attr->name, SCMI_MAX_STR_SIZE); } diff --git a/drivers/firmware/efi/Kconfig b/drivers/firmware/efi/Kconfig index d8e159feb573..89110dfc7127 100644 --- a/drivers/firmware/efi/Kconfig +++ b/drivers/firmware/efi/Kconfig @@ -90,14 +90,17 @@ config EFI_ARMSTUB config EFI_ARMSTUB_DTB_LOADER bool "Enable the DTB loader" depends on EFI_ARMSTUB + default y help Select this config option to add support for the dtb= command line parameter, allowing a device tree blob to be loaded into memory from the EFI System Partition by the stub. - The device tree is typically provided by the platform or by - the bootloader, so this option is mostly for development - purposes only. + If the device tree is provided by the platform or by + the bootloader this option may not be needed. + But, for various development reasons and to maintain existing + functionality for bootloaders that do not have such support + this option is necessary. config EFI_BOOTLOADER_CONTROL tristate "EFI Bootloader Control" diff --git a/drivers/fpga/dfl-fme-pr.c b/drivers/fpga/dfl-fme-pr.c index fc9fd2d0482f..0b840531ef33 100644 --- a/drivers/fpga/dfl-fme-pr.c +++ b/drivers/fpga/dfl-fme-pr.c @@ -420,7 +420,7 @@ static int pr_mgmt_init(struct platform_device *pdev, /* Create region for each port */ fme_region = dfl_fme_create_region(pdata, mgr, fme_br->br, i); - if (!fme_region) { + if (IS_ERR(fme_region)) { ret = PTR_ERR(fme_region); goto destroy_region; } diff --git a/drivers/gpio/gpio-adp5588.c b/drivers/gpio/gpio-adp5588.c index 3530ccd17e04..da9781a2ef4a 100644 --- a/drivers/gpio/gpio-adp5588.c +++ b/drivers/gpio/gpio-adp5588.c @@ -41,6 +41,8 @@ struct adp5588_gpio { uint8_t int_en[3]; uint8_t irq_mask[3]; uint8_t irq_stat[3]; + uint8_t int_input_en[3]; + uint8_t int_lvl_cached[3]; }; static int adp5588_gpio_read(struct i2c_client *client, u8 reg) @@ -173,12 +175,28 @@ static void adp5588_irq_bus_sync_unlock(struct irq_data *d) struct adp5588_gpio *dev = irq_data_get_irq_chip_data(d); int i; - for (i = 0; i <= ADP5588_BANK(ADP5588_MAXGPIO); i++) + for (i = 0; i <= ADP5588_BANK(ADP5588_MAXGPIO); i++) { + if (dev->int_input_en[i]) { + mutex_lock(&dev->lock); + dev->dir[i] &= ~dev->int_input_en[i]; + dev->int_input_en[i] = 0; + adp5588_gpio_write(dev->client, GPIO_DIR1 + i, + dev->dir[i]); + mutex_unlock(&dev->lock); + } + + if (dev->int_lvl_cached[i] != dev->int_lvl[i]) { + dev->int_lvl_cached[i] = dev->int_lvl[i]; + adp5588_gpio_write(dev->client, GPIO_INT_LVL1 + i, + dev->int_lvl[i]); + } + if (dev->int_en[i] ^ dev->irq_mask[i]) { dev->int_en[i] = dev->irq_mask[i]; adp5588_gpio_write(dev->client, GPIO_INT_EN1 + i, dev->int_en[i]); } + } mutex_unlock(&dev->irq_lock); } @@ -221,9 +239,7 @@ static int adp5588_irq_set_type(struct irq_data *d, unsigned int type) else return -EINVAL; - adp5588_gpio_direction_input(&dev->gpio_chip, gpio); - adp5588_gpio_write(dev->client, GPIO_INT_LVL1 + bank, - dev->int_lvl[bank]); + dev->int_input_en[bank] |= bit; return 0; } diff --git a/drivers/gpio/gpio-dwapb.c b/drivers/gpio/gpio-dwapb.c index 28da700f5f52..044888fd96a1 100644 --- a/drivers/gpio/gpio-dwapb.c +++ b/drivers/gpio/gpio-dwapb.c @@ -728,6 +728,7 @@ static int dwapb_gpio_probe(struct platform_device *pdev) out_unregister: dwapb_gpio_unregister(gpio); dwapb_irq_teardown(gpio); + clk_disable_unprepare(gpio->clk); return err; } diff --git a/drivers/gpio/gpiolib-acpi.c b/drivers/gpio/gpiolib-acpi.c index c48ed9d89ff5..8b9d7e42c600 100644 --- a/drivers/gpio/gpiolib-acpi.c +++ b/drivers/gpio/gpiolib-acpi.c @@ -25,7 +25,6 @@ struct acpi_gpio_event { struct list_head node; - struct list_head initial_sync_list; acpi_handle handle; unsigned int pin; unsigned int irq; @@ -49,10 +48,19 @@ struct acpi_gpio_chip { struct mutex conn_lock; struct gpio_chip *chip; struct list_head events; + struct list_head deferred_req_irqs_list_entry; }; -static LIST_HEAD(acpi_gpio_initial_sync_list); -static DEFINE_MUTEX(acpi_gpio_initial_sync_list_lock); +/* + * For gpiochips which call acpi_gpiochip_request_interrupts() before late_init + * (so builtin drivers) we register the ACPI GpioInt event handlers from a + * late_initcall_sync handler, so that other builtin drivers can register their + * OpRegions before the event handlers can run. This list contains gpiochips + * for which the acpi_gpiochip_request_interrupts() has been deferred. + */ +static DEFINE_MUTEX(acpi_gpio_deferred_req_irqs_lock); +static LIST_HEAD(acpi_gpio_deferred_req_irqs_list); +static bool acpi_gpio_deferred_req_irqs_done; static int acpi_gpiochip_find(struct gpio_chip *gc, void *data) { @@ -89,21 +97,6 @@ static struct gpio_desc *acpi_get_gpiod(char *path, int pin) return gpiochip_get_desc(chip, pin); } -static void acpi_gpio_add_to_initial_sync_list(struct acpi_gpio_event *event) -{ - mutex_lock(&acpi_gpio_initial_sync_list_lock); - list_add(&event->initial_sync_list, &acpi_gpio_initial_sync_list); - mutex_unlock(&acpi_gpio_initial_sync_list_lock); -} - -static void acpi_gpio_del_from_initial_sync_list(struct acpi_gpio_event *event) -{ - mutex_lock(&acpi_gpio_initial_sync_list_lock); - if (!list_empty(&event->initial_sync_list)) - list_del_init(&event->initial_sync_list); - mutex_unlock(&acpi_gpio_initial_sync_list_lock); -} - static irqreturn_t acpi_gpio_irq_handler(int irq, void *data) { struct acpi_gpio_event *event = data; @@ -186,7 +179,7 @@ static acpi_status acpi_gpiochip_request_interrupt(struct acpi_resource *ares, gpiod_direction_input(desc); - value = gpiod_get_value(desc); + value = gpiod_get_value_cansleep(desc); ret = gpiochip_lock_as_irq(chip, pin); if (ret) { @@ -229,7 +222,6 @@ static acpi_status acpi_gpiochip_request_interrupt(struct acpi_resource *ares, event->irq = irq; event->pin = pin; event->desc = desc; - INIT_LIST_HEAD(&event->initial_sync_list); ret = request_threaded_irq(event->irq, NULL, handler, irqflags, "ACPI:Event", event); @@ -251,10 +243,9 @@ static acpi_status acpi_gpiochip_request_interrupt(struct acpi_resource *ares, * may refer to OperationRegions from other (builtin) drivers which * may be probed after us. */ - if (handler == acpi_gpio_irq_handler && - (((irqflags & IRQF_TRIGGER_RISING) && value == 1) || - ((irqflags & IRQF_TRIGGER_FALLING) && value == 0))) - acpi_gpio_add_to_initial_sync_list(event); + if (((irqflags & IRQF_TRIGGER_RISING) && value == 1) || + ((irqflags & IRQF_TRIGGER_FALLING) && value == 0)) + handler(event->irq, event); return AE_OK; @@ -283,6 +274,7 @@ void acpi_gpiochip_request_interrupts(struct gpio_chip *chip) struct acpi_gpio_chip *acpi_gpio; acpi_handle handle; acpi_status status; + bool defer; if (!chip->parent || !chip->to_irq) return; @@ -295,6 +287,16 @@ void acpi_gpiochip_request_interrupts(struct gpio_chip *chip) if (ACPI_FAILURE(status)) return; + mutex_lock(&acpi_gpio_deferred_req_irqs_lock); + defer = !acpi_gpio_deferred_req_irqs_done; + if (defer) + list_add(&acpi_gpio->deferred_req_irqs_list_entry, + &acpi_gpio_deferred_req_irqs_list); + mutex_unlock(&acpi_gpio_deferred_req_irqs_lock); + + if (defer) + return; + acpi_walk_resources(handle, "_AEI", acpi_gpiochip_request_interrupt, acpi_gpio); } @@ -325,11 +327,14 @@ void acpi_gpiochip_free_interrupts(struct gpio_chip *chip) if (ACPI_FAILURE(status)) return; + mutex_lock(&acpi_gpio_deferred_req_irqs_lock); + if (!list_empty(&acpi_gpio->deferred_req_irqs_list_entry)) + list_del_init(&acpi_gpio->deferred_req_irqs_list_entry); + mutex_unlock(&acpi_gpio_deferred_req_irqs_lock); + list_for_each_entry_safe_reverse(event, ep, &acpi_gpio->events, node) { struct gpio_desc *desc; - acpi_gpio_del_from_initial_sync_list(event); - if (irqd_is_wakeup_set(irq_get_irq_data(event->irq))) disable_irq_wake(event->irq); @@ -1052,6 +1057,7 @@ void acpi_gpiochip_add(struct gpio_chip *chip) acpi_gpio->chip = chip; INIT_LIST_HEAD(&acpi_gpio->events); + INIT_LIST_HEAD(&acpi_gpio->deferred_req_irqs_list_entry); status = acpi_attach_data(handle, acpi_gpio_chip_dh, acpi_gpio); if (ACPI_FAILURE(status)) { @@ -1198,20 +1204,28 @@ bool acpi_can_fallback_to_crs(struct acpi_device *adev, const char *con_id) return con_id == NULL; } -/* Sync the initial state of handlers after all builtin drivers have probed */ -static int acpi_gpio_initial_sync(void) +/* Run deferred acpi_gpiochip_request_interrupts() */ +static int acpi_gpio_handle_deferred_request_interrupts(void) { - struct acpi_gpio_event *event, *ep; + struct acpi_gpio_chip *acpi_gpio, *tmp; + + mutex_lock(&acpi_gpio_deferred_req_irqs_lock); + list_for_each_entry_safe(acpi_gpio, tmp, + &acpi_gpio_deferred_req_irqs_list, + deferred_req_irqs_list_entry) { + acpi_handle handle; - mutex_lock(&acpi_gpio_initial_sync_list_lock); - list_for_each_entry_safe(event, ep, &acpi_gpio_initial_sync_list, - initial_sync_list) { - acpi_evaluate_object(event->handle, NULL, NULL, NULL); - list_del_init(&event->initial_sync_list); + handle = ACPI_HANDLE(acpi_gpio->chip->parent); + acpi_walk_resources(handle, "_AEI", + acpi_gpiochip_request_interrupt, acpi_gpio); + + list_del_init(&acpi_gpio->deferred_req_irqs_list_entry); } - mutex_unlock(&acpi_gpio_initial_sync_list_lock); + + acpi_gpio_deferred_req_irqs_done = true; + mutex_unlock(&acpi_gpio_deferred_req_irqs_lock); return 0; } /* We must use _sync so that this runs after the first deferred_probe run */ -late_initcall_sync(acpi_gpio_initial_sync); +late_initcall_sync(acpi_gpio_handle_deferred_request_interrupts); diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c index a4f1157d6aa0..d4e7a09598fa 100644 --- a/drivers/gpio/gpiolib-of.c +++ b/drivers/gpio/gpiolib-of.c @@ -31,6 +31,7 @@ static int of_gpiochip_match_node_and_xlate(struct gpio_chip *chip, void *data) struct of_phandle_args *gpiospec = data; return chip->gpiodev->dev.of_node == gpiospec->np && + chip->of_xlate && chip->of_xlate(chip, gpiospec, NULL) >= 0; } diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig index 736b7e67e4ec..4385f00e1d05 100644 --- a/drivers/gpu/drm/Kconfig +++ b/drivers/gpu/drm/Kconfig @@ -110,6 +110,26 @@ config DRM_FBDEV_OVERALLOC is 100. Typical values for double buffering will be 200, triple buffering 300. +config DRM_FBDEV_LEAK_PHYS_SMEM + bool "Shamelessly allow leaking of fbdev physical address (DANGEROUS)" + depends on DRM_FBDEV_EMULATION && EXPERT + default n + help + In order to keep user-space compatibility, we want in certain + use-cases to keep leaking the fbdev physical address to the + user-space program handling the fbdev buffer. + This affects, not only, Amlogic, Allwinner or Rockchip devices + with ARM Mali GPUs using an userspace Blob. + This option is not supported by upstream developers and should be + removed as soon as possible and be considered as a broken and + legacy behaviour from a modern fbdev device driver. + + Please send any bug reports when using this to your proprietary + software vendor that requires this. + + If in doubt, say "N" or spread the word to your closed source + library vendor. + config DRM_LOAD_EDID_FIRMWARE bool "Allow to specify an EDID data set instead of probing for it" depends on DRM diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 6cb35e3dab30..d0102cfc8efb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -81,6 +81,23 @@ #include "amdgpu_bo_list.h" #include "amdgpu_gem.h" +#define MAX_GPU_INSTANCE 16 + +struct amdgpu_gpu_instance +{ + struct amdgpu_device *adev; + int mgpu_fan_enabled; +}; + +struct amdgpu_mgpu_info +{ + struct amdgpu_gpu_instance gpu_ins[MAX_GPU_INSTANCE]; + struct mutex mutex; + uint32_t num_gpu; + uint32_t num_dgpu; + uint32_t num_apu; +}; + /* * Modules parameters. */ @@ -134,6 +151,7 @@ extern int amdgpu_compute_multipipe; extern int amdgpu_gpu_recovery; extern int amdgpu_emu_mode; extern uint amdgpu_smu_memory_pool_size; +extern struct amdgpu_mgpu_info mgpu_info; #ifdef CONFIG_DRM_AMDGPU_SI extern int amdgpu_si_support; @@ -146,6 +164,7 @@ extern int amdgpu_cik_support; #define AMDGPU_DEFAULT_GTT_SIZE_MB 3072ULL /* 3GB by default */ #define AMDGPU_WAIT_IDLE_TIMEOUT_IN_MS 3000 #define AMDGPU_MAX_USEC_TIMEOUT 100000 /* 100 ms */ +#define AMDGPU_FENCE_JIFFIES_TIMEOUT (HZ / 2) /* AMDGPU_IB_POOL_SIZE must be a power of 2 */ #define AMDGPU_IB_POOL_SIZE 16 #define AMDGPU_DEBUGFS_MAX_COMPONENTS 32 @@ -408,16 +427,25 @@ typedef enum _AMDGPU_DOORBELL64_ASSIGNMENT AMDGPU_DOORBELL64_GFX_RING0 = 0x8b, /* - * Other graphics doorbells can be allocated here: from 0x8c to 0xef + * Other graphics doorbells can be allocated here: from 0x8c to 0xdf * Graphics voltage island aperture 1 - * default non-graphics QWORD index is 0xF0 - 0xFF inclusive + * default non-graphics QWORD index is 0xe0 - 0xFF inclusive */ - /* sDMA engines */ - AMDGPU_DOORBELL64_sDMA_ENGINE0 = 0xF0, - AMDGPU_DOORBELL64_sDMA_HI_PRI_ENGINE0 = 0xF1, - AMDGPU_DOORBELL64_sDMA_ENGINE1 = 0xF2, - AMDGPU_DOORBELL64_sDMA_HI_PRI_ENGINE1 = 0xF3, + /* sDMA engines reserved from 0xe0 -oxef */ + AMDGPU_DOORBELL64_sDMA_ENGINE0 = 0xE0, + AMDGPU_DOORBELL64_sDMA_HI_PRI_ENGINE0 = 0xE1, + AMDGPU_DOORBELL64_sDMA_ENGINE1 = 0xE8, + AMDGPU_DOORBELL64_sDMA_HI_PRI_ENGINE1 = 0xE9, + + /* For vega10 sriov, the sdma doorbell must be fixed as follow + * to keep the same setting with host driver, or it will + * happen conflicts + */ + AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 = 0xF0, + AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE0 = 0xF1, + AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 = 0xF2, + AMDGPU_VEGA10_DOORBELL64_sDMA_HI_PRI_ENGINE1 = 0xF3, /* Interrupt handler */ AMDGPU_DOORBELL64_IH = 0xF4, /* For legacy interrupt ring buffer */ @@ -588,31 +616,6 @@ void amdgpu_benchmark(struct amdgpu_device *adev, int test_number); */ void amdgpu_test_moves(struct amdgpu_device *adev); - -/* - * amdgpu smumgr functions - */ -struct amdgpu_smumgr_funcs { - int (*check_fw_load_finish)(struct amdgpu_device *adev, uint32_t fwtype); - int (*request_smu_load_fw)(struct amdgpu_device *adev); - int (*request_smu_specific_fw)(struct amdgpu_device *adev, uint32_t fwtype); -}; - -/* - * amdgpu smumgr - */ -struct amdgpu_smumgr { - struct amdgpu_bo *toc_buf; - struct amdgpu_bo *smu_buf; - /* asic priv smu data */ - void *priv; - spinlock_t smu_lock; - /* smumgr functions */ - const struct amdgpu_smumgr_funcs *smumgr_funcs; - /* ucode loading complete flag */ - uint32_t fw_flags; -}; - /* * ASIC specific register table accessible by UMD */ @@ -948,9 +951,6 @@ struct amdgpu_device { u32 cg_flags; u32 pg_flags; - /* amdgpu smumgr */ - struct amdgpu_smumgr smu; - /* gfx */ struct amdgpu_gfx gfx; @@ -1015,6 +1015,9 @@ struct amdgpu_device { bool has_hw_reset; u8 reset_magic[AMDGPU_RESET_MAGIC_NUM]; + /* s3/s4 mask */ + bool in_suspend; + /* record last mm index being written through WREG32*/ unsigned long last_mm_index; bool in_gpu_reset; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c index 297a5490ad8c..0a4fba196b84 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c @@ -135,7 +135,8 @@ static int acp_poweroff(struct generic_pm_domain *genpd) * 2. power off the acp tiles * 3. check and enter ulv state */ - if (adev->powerplay.pp_funcs->set_powergating_by_smu) + if (adev->powerplay.pp_funcs && + adev->powerplay.pp_funcs->set_powergating_by_smu) amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_ACP, true); } return 0; @@ -517,7 +518,8 @@ static int acp_set_powergating_state(void *handle, struct amdgpu_device *adev = (struct amdgpu_device *)handle; bool enable = state == AMD_PG_STATE_GATE ? true : false; - if (adev->powerplay.pp_funcs->set_powergating_by_smu) + if (adev->powerplay.pp_funcs && + adev->powerplay.pp_funcs->set_powergating_by_smu) amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_ACP, enable); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c index 6488e90ec948..7f0afc526419 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c @@ -359,7 +359,9 @@ out: * * Checks the acpi event and if it matches an atif event, * handles it. - * Returns NOTIFY code + * + * Returns: + * NOTIFY_BAD or NOTIFY_DONE, depending on the event. */ static int amdgpu_atif_handler(struct amdgpu_device *adev, struct acpi_bus_event *event) @@ -373,11 +375,16 @@ static int amdgpu_atif_handler(struct amdgpu_device *adev, if (strcmp(event->device_class, ACPI_VIDEO_CLASS) != 0) return NOTIFY_DONE; + /* Is this actually our event? */ if (!atif || !atif->notification_cfg.enabled || - event->type != atif->notification_cfg.command_code) - /* Not our event */ - return NOTIFY_DONE; + event->type != atif->notification_cfg.command_code) { + /* These events will generate keypresses otherwise */ + if (event->type == ACPI_VIDEO_NOTIFY_PROBE) + return NOTIFY_BAD; + else + return NOTIFY_DONE; + } if (atif->functions.sbios_requests) { struct atif_sbios_requests req; @@ -386,7 +393,7 @@ static int amdgpu_atif_handler(struct amdgpu_device *adev, count = amdgpu_atif_get_sbios_requests(atif, &req); if (count <= 0) - return NOTIFY_DONE; + return NOTIFY_BAD; DRM_DEBUG_DRIVER("ATIF: %d pending SBIOS requests\n", count); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 5661c343a71d..c31a8849e9f8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -76,6 +76,7 @@ void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev) kfd2kgd = amdgpu_amdkfd_gfx_8_0_get_functions(); break; case CHIP_VEGA10: + case CHIP_VEGA20: case CHIP_RAVEN: kfd2kgd = amdgpu_amdkfd_gfx_9_0_get_functions(); break; @@ -123,7 +124,7 @@ static void amdgpu_doorbell_get_kfd_info(struct amdgpu_device *adev, void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) { - int i; + int i, n; int last_valid_bit; if (adev->kfd) { struct kgd2kfd_shared_resources gpu_resources = { @@ -162,7 +163,15 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) &gpu_resources.doorbell_physical_address, &gpu_resources.doorbell_aperture_size, &gpu_resources.doorbell_start_offset); - if (adev->asic_type >= CHIP_VEGA10) { + + if (adev->asic_type < CHIP_VEGA10) { + kgd2kfd->device_init(adev->kfd, &gpu_resources); + return; + } + + n = (adev->asic_type < CHIP_VEGA20) ? 2 : 8; + + for (i = 0; i < n; i += 2) { /* On SOC15 the BIF is involved in routing * doorbells using the low 12 bits of the * address. Communicate the assignments to @@ -170,20 +179,31 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) * process in case of 64-bit doorbells so we * can use each doorbell assignment twice. */ - gpu_resources.sdma_doorbell[0][0] = - AMDGPU_DOORBELL64_sDMA_ENGINE0; - gpu_resources.sdma_doorbell[0][1] = - AMDGPU_DOORBELL64_sDMA_ENGINE0 + 0x200; - gpu_resources.sdma_doorbell[1][0] = - AMDGPU_DOORBELL64_sDMA_ENGINE1; - gpu_resources.sdma_doorbell[1][1] = - AMDGPU_DOORBELL64_sDMA_ENGINE1 + 0x200; - /* Doorbells 0x0f0-0ff and 0x2f0-2ff are reserved for - * SDMA, IH and VCN. So don't use them for the CP. - */ - gpu_resources.reserved_doorbell_mask = 0x1f0; - gpu_resources.reserved_doorbell_val = 0x0f0; + if (adev->asic_type == CHIP_VEGA10) { + gpu_resources.sdma_doorbell[0][i] = + AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 + (i >> 1); + gpu_resources.sdma_doorbell[0][i+1] = + AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 + 0x200 + (i >> 1); + gpu_resources.sdma_doorbell[1][i] = + AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 + (i >> 1); + gpu_resources.sdma_doorbell[1][i+1] = + AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 + 0x200 + (i >> 1); + } else { + gpu_resources.sdma_doorbell[0][i] = + AMDGPU_DOORBELL64_sDMA_ENGINE0 + (i >> 1); + gpu_resources.sdma_doorbell[0][i+1] = + AMDGPU_DOORBELL64_sDMA_ENGINE0 + 0x200 + (i >> 1); + gpu_resources.sdma_doorbell[1][i] = + AMDGPU_DOORBELL64_sDMA_ENGINE1 + (i >> 1); + gpu_resources.sdma_doorbell[1][i+1] = + AMDGPU_DOORBELL64_sDMA_ENGINE1 + 0x200 + (i >> 1); + } } + /* Doorbells 0x0e0-0ff and 0x2e0-2ff are reserved for + * SDMA, IH and VCN. So don't use them for the CP. + */ + gpu_resources.reserved_doorbell_mask = 0x1e0; + gpu_resources.reserved_doorbell_val = 0x0e0; kgd2kfd->device_init(adev->kfd, &gpu_resources); } @@ -250,7 +270,7 @@ void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd) int alloc_gtt_mem(struct kgd_dev *kgd, size_t size, void **mem_obj, uint64_t *gpu_addr, - void **cpu_ptr) + void **cpu_ptr, bool mqd_gfx9) { struct amdgpu_device *adev = (struct amdgpu_device *)kgd; struct amdgpu_bo *bo = NULL; @@ -265,6 +285,10 @@ int alloc_gtt_mem(struct kgd_dev *kgd, size_t size, bp.flags = AMDGPU_GEM_CREATE_CPU_GTT_USWC; bp.type = ttm_bo_type_kernel; bp.resv = NULL; + + if (mqd_gfx9) + bp.flags |= AMDGPU_GEM_CREATE_MQD_GFX9; + r = amdgpu_bo_create(adev, &bp, &bo); if (r) { dev_err(adev->dev, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 41e7dfc3ced3..8e0d4f7196b4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -136,7 +136,7 @@ void amdgpu_amdkfd_gpu_reset(struct kgd_dev *kgd); /* Shared API */ int alloc_gtt_mem(struct kgd_dev *kgd, size_t size, void **mem_obj, uint64_t *gpu_addr, - void **cpu_ptr); + void **cpu_ptr, bool mqd_gfx9); void free_gtt_mem(struct kgd_dev *kgd, void *mem_obj); void get_local_mem_info(struct kgd_dev *kgd, struct kfd_local_mem_info *mem_info); @@ -174,7 +174,7 @@ void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, struct amdgpu_vm *vm); void amdgpu_amdkfd_gpuvm_destroy_process_vm(struct kgd_dev *kgd, void *vm); void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *vm); -uint32_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *vm); +uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *vm); int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( struct kgd_dev *kgd, uint64_t va, uint64_t size, void *vm, struct kgd_mem **mem, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c index 6f0b5267addb..244d9834a381 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v7.c @@ -142,7 +142,7 @@ static uint16_t get_fw_version(struct kgd_dev *kgd, enum kgd_engine_type type); static void set_scratch_backing_va(struct kgd_dev *kgd, uint64_t va, uint32_t vmid); static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, - uint32_t page_table_base); + uint64_t page_table_base); static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid); static int invalidate_tlbs_vmid(struct kgd_dev *kgd, uint16_t vmid); static uint32_t read_vmid_from_vmfault_reg(struct kgd_dev *kgd); @@ -686,7 +686,7 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, while (true) { temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); - if (temp & SDMA0_STATUS_REG__RB_CMD_IDLE__SHIFT) + if (temp & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) break; if (time_after(jiffies, end_jiffies)) return -ETIME; @@ -874,7 +874,7 @@ static uint16_t get_fw_version(struct kgd_dev *kgd, enum kgd_engine_type type) } static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, - uint32_t page_table_base) + uint64_t page_table_base) { struct amdgpu_device *adev = get_amdgpu_device(kgd); @@ -882,7 +882,8 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, pr_err("trying to set page table base for wrong VMID\n"); return; } - WREG32(mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR + vmid - 8, page_table_base); + WREG32(mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR + vmid - 8, + lower_32_bits(page_table_base)); } static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c index ea7c18ce7754..9f149914ad6c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v8.c @@ -45,8 +45,6 @@ enum hqd_dequeue_request_type { RESET_WAVES }; -struct vi_sdma_mqd; - /* * Register access functions */ @@ -100,7 +98,7 @@ static uint16_t get_fw_version(struct kgd_dev *kgd, enum kgd_engine_type type); static void set_scratch_backing_va(struct kgd_dev *kgd, uint64_t va, uint32_t vmid); static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, - uint32_t page_table_base); + uint64_t page_table_base); static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid); static int invalidate_tlbs_vmid(struct kgd_dev *kgd, uint16_t vmid); @@ -282,7 +280,8 @@ static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) lock_srbm(kgd, mec, pipe, 0, 0); - WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK); + WREG32(mmCPC_INT_CNTL, CP_INT_CNTL_RING0__TIME_STAMP_INT_ENABLE_MASK | + CP_INT_CNTL_RING0__OPCODE_ERROR_INT_ENABLE_MASK); unlock_srbm(kgd); @@ -834,7 +833,7 @@ static uint16_t get_fw_version(struct kgd_dev *kgd, enum kgd_engine_type type) } static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, - uint32_t page_table_base) + uint64_t page_table_base) { struct amdgpu_device *adev = get_amdgpu_device(kgd); @@ -842,7 +841,8 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, pr_err("trying to set page table base for wrong VMID\n"); return; } - WREG32(mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR + vmid - 8, page_table_base); + WREG32(mmVM_CONTEXT8_PAGE_TABLE_BASE_ADDR + vmid - 8, + lower_32_bits(page_table_base)); } static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c index c9176537550b..42cb4c4e0929 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gfx_v9.c @@ -138,7 +138,7 @@ static bool get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, static uint16_t get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, uint8_t vmid); static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, - uint32_t page_table_base); + uint64_t page_table_base); static uint16_t get_fw_version(struct kgd_dev *kgd, enum kgd_engine_type type); static void set_scratch_backing_va(struct kgd_dev *kgd, uint64_t va, uint32_t vmid); @@ -1013,11 +1013,10 @@ static uint16_t get_fw_version(struct kgd_dev *kgd, enum kgd_engine_type type) } static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, - uint32_t page_table_base) + uint64_t page_table_base) { struct amdgpu_device *adev = get_amdgpu_device(kgd); - uint64_t base = (uint64_t)page_table_base << PAGE_SHIFT | - AMDGPU_PTE_VALID; + uint64_t base = page_table_base | AMDGPU_PTE_VALID; if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { pr_err("trying to set page table base for wrong VMID %u\n", diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 6ee9dc476c86..df0a059565f9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -1131,11 +1131,15 @@ void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *vm) amdgpu_vm_release_compute(adev, avm); } -uint32_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *vm) +uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *vm) { struct amdgpu_vm *avm = (struct amdgpu_vm *)vm; + struct amdgpu_bo *pd = avm->root.base.bo; + struct amdgpu_device *adev = amdgpu_ttm_adev(pd->tbo.bdev); - return avm->pd_phys_addr >> AMDGPU_GPU_PAGE_SHIFT; + if (adev->asic_type < CHIP_VEGA10) + return avm->pd_phys_addr >> AMDGPU_GPU_PAGE_SHIFT; + return avm->pd_phys_addr; } int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c index 693ec5ea4950..8816c697b205 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cgs.c @@ -367,12 +367,14 @@ static int amdgpu_cgs_get_firmware_info(struct cgs_device *cgs_device, break; case CHIP_POLARIS10: if (type == CGS_UCODE_ID_SMU) { - if ((adev->pdev->device == 0x67df) && - ((adev->pdev->revision == 0xe0) || - (adev->pdev->revision == 0xe3) || - (adev->pdev->revision == 0xe4) || - (adev->pdev->revision == 0xe5) || - (adev->pdev->revision == 0xe7) || + if (((adev->pdev->device == 0x67df) && + ((adev->pdev->revision == 0xe0) || + (adev->pdev->revision == 0xe3) || + (adev->pdev->revision == 0xe4) || + (adev->pdev->revision == 0xe5) || + (adev->pdev->revision == 0xe7) || + (adev->pdev->revision == 0xef))) || + ((adev->pdev->device == 0x6fdf) && (adev->pdev->revision == 0xef))) { info->is_kicker = true; strcpy(fw_name, "amdgpu/polaris10_k_smc.bin"); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index b7ec2b0e5a9b..663043c8f0f5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -1262,10 +1262,10 @@ static int amdgpu_cs_submit(struct amdgpu_cs_parser *p, error_abort: dma_fence_put(&job->base.s_fence->finished); job->base.s_fence = NULL; + amdgpu_mn_unlock(p->mn); error_unlock: amdgpu_job_free(job); - amdgpu_mn_unlock(p->mn); return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c index f5fb93795a69..dd9a4fb9ce39 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c @@ -826,21 +826,13 @@ int amdgpu_debugfs_regs_init(struct amdgpu_device *adev) { struct drm_minor *minor = adev->ddev->primary; struct dentry *ent, *root = minor->debugfs_root; - unsigned i, j; + unsigned int i; for (i = 0; i < ARRAY_SIZE(debugfs_regs); i++) { ent = debugfs_create_file(debugfs_regs_names[i], S_IFREG | S_IRUGO, root, adev, debugfs_regs[i]); - if (IS_ERR(ent)) { - for (j = 0; j < i; j++) { - debugfs_remove(adev->debugfs_regs[i]); - adev->debugfs_regs[i] = NULL; - } - return PTR_ERR(ent); - } - - if (!i) + if (!i && !IS_ERR_OR_NULL(ent)) i_size_write(ent->d_inode, adev->rmmio_size); adev->debugfs_regs[i] = ent; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index bd79d0a31942..30bc345d6fdf 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -1493,8 +1493,6 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) } adev->powerplay.pp_feature = amdgpu_pp_feature_mask; - if (amdgpu_sriov_vf(adev)) - adev->powerplay.pp_feature &= ~PP_GFXOFF_MASK; for (i = 0; i < adev->num_ip_blocks; i++) { if ((amdgpu_ip_block_mask & (1 << i)) == 0) { @@ -1525,6 +1523,92 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) return 0; } +static int amdgpu_device_ip_hw_init_phase1(struct amdgpu_device *adev) +{ + int i, r; + + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].status.sw) + continue; + if (adev->ip_blocks[i].status.hw) + continue; + if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_COMMON || + adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_IH) { + r = adev->ip_blocks[i].version->funcs->hw_init(adev); + if (r) { + DRM_ERROR("hw_init of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + return r; + } + adev->ip_blocks[i].status.hw = true; + } + } + + return 0; +} + +static int amdgpu_device_ip_hw_init_phase2(struct amdgpu_device *adev) +{ + int i, r; + + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].status.sw) + continue; + if (adev->ip_blocks[i].status.hw) + continue; + r = adev->ip_blocks[i].version->funcs->hw_init(adev); + if (r) { + DRM_ERROR("hw_init of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + return r; + } + adev->ip_blocks[i].status.hw = true; + } + + return 0; +} + +static int amdgpu_device_fw_loading(struct amdgpu_device *adev) +{ + int r = 0; + int i; + + if (adev->asic_type >= CHIP_VEGA10) { + for (i = 0; i < adev->num_ip_blocks; i++) { + if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP) { + if (adev->in_gpu_reset || adev->in_suspend) { + if (amdgpu_sriov_vf(adev) && adev->in_gpu_reset) + break; /* sriov gpu reset, psp need to do hw_init before IH because of hw limit */ + r = adev->ip_blocks[i].version->funcs->resume(adev); + if (r) { + DRM_ERROR("resume of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + return r; + } + } else { + r = adev->ip_blocks[i].version->funcs->hw_init(adev); + if (r) { + DRM_ERROR("hw_init of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + return r; + } + } + adev->ip_blocks[i].status.hw = true; + } + } + } + + if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->load_firmware) { + r = adev->powerplay.pp_funcs->load_firmware(adev->powerplay.pp_handle); + if (r) { + pr_err("firmware loading failed\n"); + return r; + } + } + + return 0; +} + /** * amdgpu_device_ip_init - run init for hardware IPs * @@ -1581,19 +1665,21 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev) } } - for (i = 0; i < adev->num_ip_blocks; i++) { - if (!adev->ip_blocks[i].status.sw) - continue; - if (adev->ip_blocks[i].status.hw) - continue; - r = adev->ip_blocks[i].version->funcs->hw_init((void *)adev); - if (r) { - DRM_ERROR("hw_init of IP block <%s> failed %d\n", - adev->ip_blocks[i].version->funcs->name, r); - return r; - } - adev->ip_blocks[i].status.hw = true; - } + r = amdgpu_ucode_create_bo(adev); /* create ucode bo when sw_init complete*/ + if (r) + return r; + + r = amdgpu_device_ip_hw_init_phase1(adev); + if (r) + return r; + + r = amdgpu_device_fw_loading(adev); + if (r) + return r; + + r = amdgpu_device_ip_hw_init_phase2(adev); + if (r) + return r; amdgpu_xgmi_add_device(adev); amdgpu_amdkfd_device_init(adev); @@ -1656,7 +1742,7 @@ static int amdgpu_device_set_cg_state(struct amdgpu_device *adev, for (j = 0; j < adev->num_ip_blocks; j++) { i = state == AMD_CG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; - if (!adev->ip_blocks[i].status.valid) + if (!adev->ip_blocks[i].status.late_initialized) continue; /* skip CG for VCE/UVD, it's handled specially */ if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_UVD && @@ -1686,7 +1772,7 @@ static int amdgpu_device_set_pg_state(struct amdgpu_device *adev, enum amd_power for (j = 0; j < adev->num_ip_blocks; j++) { i = state == AMD_PG_STATE_GATE ? j : adev->num_ip_blocks - j - 1; - if (!adev->ip_blocks[i].status.valid) + if (!adev->ip_blocks[i].status.late_initialized) continue; /* skip CG for VCE/UVD, it's handled specially */ if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_UVD && @@ -1723,7 +1809,7 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) int i = 0, r; for (i = 0; i < adev->num_ip_blocks; i++) { - if (!adev->ip_blocks[i].status.valid) + if (!adev->ip_blocks[i].status.hw) continue; if (adev->ip_blocks[i].version->funcs->late_init) { r = adev->ip_blocks[i].version->funcs->late_init((void *)adev); @@ -1732,8 +1818,8 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) adev->ip_blocks[i].version->funcs->name, r); return r; } - adev->ip_blocks[i].status.late_initialized = true; } + adev->ip_blocks[i].status.late_initialized = true; } amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE); @@ -1803,6 +1889,7 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) continue; if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC) { + amdgpu_ucode_free_bo(adev); amdgpu_free_static_csa(adev); amdgpu_device_wb_fini(adev); amdgpu_device_vram_scratch_fini(adev); @@ -1833,6 +1920,43 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) return 0; } +static int amdgpu_device_enable_mgpu_fan_boost(void) +{ + struct amdgpu_gpu_instance *gpu_ins; + struct amdgpu_device *adev; + int i, ret = 0; + + mutex_lock(&mgpu_info.mutex); + + /* + * MGPU fan boost feature should be enabled + * only when there are two or more dGPUs in + * the system + */ + if (mgpu_info.num_dgpu < 2) + goto out; + + for (i = 0; i < mgpu_info.num_dgpu; i++) { + gpu_ins = &(mgpu_info.gpu_ins[i]); + adev = gpu_ins->adev; + if (!(adev->flags & AMD_IS_APU) && + !gpu_ins->mgpu_fan_enabled && + adev->powerplay.pp_funcs && + adev->powerplay.pp_funcs->enable_mgpu_fan_boost) { + ret = amdgpu_dpm_enable_mgpu_fan_boost(adev); + if (ret) + break; + + gpu_ins->mgpu_fan_enabled = 1; + } + } + +out: + mutex_unlock(&mgpu_info.mutex); + + return ret; +} + /** * amdgpu_device_ip_late_init_func_handler - work handler for ib test * @@ -1847,6 +1971,10 @@ static void amdgpu_device_ip_late_init_func_handler(struct work_struct *work) r = amdgpu_ib_ring_tests(adev); if (r) DRM_ERROR("ib ring test failed (%d).\n", r); + + r = amdgpu_device_enable_mgpu_fan_boost(); + if (r) + DRM_ERROR("enable mgpu fan boost failed (%d).\n", r); } static void amdgpu_device_delay_enable_gfx_off(struct work_struct *work) @@ -2082,7 +2210,8 @@ static int amdgpu_device_ip_resume_phase2(struct amdgpu_device *adev) continue; if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_COMMON || adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC || - adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_IH) + adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_IH || + adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP) continue; r = adev->ip_blocks[i].version->funcs->resume(adev); if (r) { @@ -2114,6 +2243,11 @@ static int amdgpu_device_ip_resume(struct amdgpu_device *adev) r = amdgpu_device_ip_resume_phase1(adev); if (r) return r; + + r = amdgpu_device_fw_loading(adev); + if (r) + return r; + r = amdgpu_device_ip_resume_phase2(adev); return r; @@ -2608,6 +2742,7 @@ int amdgpu_device_suspend(struct drm_device *dev, bool suspend, bool fbcon) if (dev->switch_power_state == DRM_SWITCH_POWER_OFF) return 0; + adev->in_suspend = true; drm_kms_helper_poll_disable(dev); if (fbcon) @@ -2793,6 +2928,8 @@ int amdgpu_device_resume(struct drm_device *dev, bool resume, bool fbcon) #ifdef CONFIG_PM dev->dev->power.disable_depth--; #endif + adev->in_suspend = false; + return 0; } @@ -3061,6 +3198,10 @@ retry: if (r) goto out; + r = amdgpu_device_fw_loading(adev); + if (r) + return r; + r = amdgpu_device_ip_resume_phase2(adev); if (r) goto out; @@ -3117,6 +3258,10 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, /* we need recover gart prior to run SMC/CP/SDMA resume */ amdgpu_gtt_mgr_recover(&adev->mman.bdev.man[TTM_PL_TT]); + r = amdgpu_device_fw_loading(adev); + if (r) + return r; + /* now we are okay to resume SMC/CP/SDMA */ r = amdgpu_device_ip_reinit_late_sriov(adev); if (r) @@ -3194,7 +3339,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, kthread_park(ring->sched.thread); - if (job && job->base.sched == &ring->sched) + if (job && job->base.sched != &ring->sched) continue; drm_sched_hw_job_reset(&ring->sched, job ? &job->base : NULL); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.h index ff24e1cc5b65..f972cd156795 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dpm.h @@ -278,6 +278,9 @@ enum amdgpu_pcie_gen { #define amdgpu_dpm_get_fan_speed_rpm(adev, s) \ ((adev)->powerplay.pp_funcs->get_fan_speed_rpm)((adev)->powerplay.pp_handle, (s)) +#define amdgpu_dpm_set_fan_speed_rpm(adev, s) \ + ((adev)->powerplay.pp_funcs->set_fan_speed_rpm)((adev)->powerplay.pp_handle, (s)) + #define amdgpu_dpm_get_sclk(adev, l) \ ((adev)->powerplay.pp_funcs->get_sclk((adev)->powerplay.pp_handle, (l))) @@ -357,6 +360,10 @@ enum amdgpu_pcie_gen { ((adev)->powerplay.pp_funcs->odn_edit_dpm_table(\ (adev)->powerplay.pp_handle, type, parameter, size)) +#define amdgpu_dpm_enable_mgpu_fan_boost(adev) \ + ((adev)->powerplay.pp_funcs->enable_mgpu_fan_boost(\ + (adev)->powerplay.pp_handle)) + struct amdgpu_dpm { struct amdgpu_ps *ps; /* number of valid power states */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 53e038963936..943dbf3c5da1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -114,8 +114,8 @@ uint amdgpu_pg_mask = 0xffffffff; uint amdgpu_sdma_phase_quantum = 32; char *amdgpu_disable_cu = NULL; char *amdgpu_virtual_display = NULL; -/* OverDrive(bit 14) disabled by default*/ -uint amdgpu_pp_feature_mask = 0xffffbfff; +/* OverDrive(bit 14),gfxoff(bit 15),stutter mode(bit 17) disabled by default*/ +uint amdgpu_pp_feature_mask = 0xfffd3fff; int amdgpu_ngg = 0; int amdgpu_prim_buf_per_se = 0; int amdgpu_pos_buf_per_se = 0; @@ -127,6 +127,9 @@ int amdgpu_compute_multipipe = -1; int amdgpu_gpu_recovery = -1; /* auto */ int amdgpu_emu_mode = 0; uint amdgpu_smu_memory_pool_size = 0; +struct amdgpu_mgpu_info mgpu_info = { + .mutex = __MUTEX_INITIALIZER(mgpu_info.mutex), +}; /** * DOC: vramlimit (int) @@ -837,6 +840,7 @@ static const struct pci_device_id pciidlist[] = { {0x1002, 0x67CA, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10}, {0x1002, 0x67CC, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10}, {0x1002, 0x67CF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10}, + {0x1002, 0x6FDF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS10}, /* Polaris12 */ {0x1002, 0x6980, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12}, {0x1002, 0x6981, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_POLARIS12}, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index 176f28777f5e..5448cf27654e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -196,6 +196,19 @@ int amdgpu_fence_emit_polling(struct amdgpu_ring *ring, uint32_t *s) } /** + * amdgpu_fence_schedule_fallback - schedule fallback check + * + * @ring: pointer to struct amdgpu_ring + * + * Start a timer as fallback to our interrupts. + */ +static void amdgpu_fence_schedule_fallback(struct amdgpu_ring *ring) +{ + mod_timer(&ring->fence_drv.fallback_timer, + jiffies + AMDGPU_FENCE_JIFFIES_TIMEOUT); +} + +/** * amdgpu_fence_process - check for fence activity * * @ring: pointer to struct amdgpu_ring @@ -203,8 +216,10 @@ int amdgpu_fence_emit_polling(struct amdgpu_ring *ring, uint32_t *s) * Checks the current fence value and calculates the last * signalled fence value. Wakes the fence queue if the * sequence number has increased. + * + * Returns true if fence was processed */ -void amdgpu_fence_process(struct amdgpu_ring *ring) +bool amdgpu_fence_process(struct amdgpu_ring *ring) { struct amdgpu_fence_driver *drv = &ring->fence_drv; uint32_t seq, last_seq; @@ -216,8 +231,12 @@ void amdgpu_fence_process(struct amdgpu_ring *ring) } while (atomic_cmpxchg(&drv->last_seq, last_seq, seq) != last_seq); + if (del_timer(&ring->fence_drv.fallback_timer) && + seq != ring->fence_drv.sync_seq) + amdgpu_fence_schedule_fallback(ring); + if (unlikely(seq == last_seq)) - return; + return false; last_seq &= drv->num_fences_mask; seq &= drv->num_fences_mask; @@ -244,6 +263,24 @@ void amdgpu_fence_process(struct amdgpu_ring *ring) dma_fence_put(fence); } while (last_seq != seq); + + return true; +} + +/** + * amdgpu_fence_fallback - fallback for hardware interrupts + * + * @work: delayed work item + * + * Checks for fence activity. + */ +static void amdgpu_fence_fallback(struct timer_list *t) +{ + struct amdgpu_ring *ring = from_timer(ring, t, + fence_drv.fallback_timer); + + if (amdgpu_fence_process(ring)) + DRM_WARN("Fence fallback timer expired on ring %s\n", ring->name); } /** @@ -393,6 +430,8 @@ int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, atomic_set(&ring->fence_drv.last_seq, 0); ring->fence_drv.initialized = false; + timer_setup(&ring->fence_drv.fallback_timer, amdgpu_fence_fallback, 0); + ring->fence_drv.num_fences_mask = num_hw_submission * 2 - 1; spin_lock_init(&ring->fence_drv.lock); ring->fence_drv.fences = kcalloc(num_hw_submission * 2, sizeof(void *), @@ -468,6 +507,7 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) amdgpu_irq_put(adev, ring->fence_drv.irq_src, ring->fence_drv.irq_type); drm_sched_fini(&ring->sched); + del_timer_sync(&ring->fence_drv.fallback_timer); for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); @@ -561,6 +601,27 @@ static const char *amdgpu_fence_get_timeline_name(struct dma_fence *f) } /** + * amdgpu_fence_enable_signaling - enable signalling on fence + * @fence: fence + * + * This function is called with fence_queue lock held, and adds a callback + * to fence_queue that checks if this fence is signaled, and if so it + * signals the fence and removes itself. + */ +static bool amdgpu_fence_enable_signaling(struct dma_fence *f) +{ + struct amdgpu_fence *fence = to_amdgpu_fence(f); + struct amdgpu_ring *ring = fence->ring; + + if (!timer_pending(&ring->fence_drv.fallback_timer)) + amdgpu_fence_schedule_fallback(ring); + + DMA_FENCE_TRACE(&fence->base, "armed on ring %i!\n", ring->idx); + + return true; +} + +/** * amdgpu_fence_free - free up the fence memory * * @rcu: RCU callback head @@ -590,6 +651,7 @@ static void amdgpu_fence_release(struct dma_fence *f) static const struct dma_fence_ops amdgpu_fence_ops = { .get_driver_name = amdgpu_fence_get_driver_name, .get_timeline_name = amdgpu_fence_get_timeline_name, + .enable_signaling = amdgpu_fence_enable_signaling, .release = amdgpu_fence_release, }; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c index 790fd5408ddf..1a656b8657f7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c @@ -392,7 +392,7 @@ void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable) if (!(adev->powerplay.pp_feature & PP_GFXOFF_MASK)) return; - if (!adev->powerplay.pp_funcs->set_powergating_by_smu) + if (!adev->powerplay.pp_funcs || !adev->powerplay.pp_funcs->set_powergating_by_smu) return; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h index f172e92c463c..b61b5c11aead 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h @@ -297,8 +297,7 @@ struct amdgpu_gfx { /* reset mask */ uint32_t grbm_soft_reset; uint32_t srbm_soft_reset; - /* s3/s4 mask */ - bool in_suspend; + /* NGG */ struct amdgpu_ngg ngg; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index 9a5b252784a1..d73367cab4f3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -146,6 +146,8 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) { const uint64_t four_gb = 0x100000000ULL; u64 size_af, size_bf; + /*To avoid the hole, limit the max mc address to AMDGPU_GMC_HOLE_START*/ + u64 max_mc_address = min(adev->gmc.mc_mask, AMDGPU_GMC_HOLE_START - 1); mc->gart_size += adev->pm.smu_prv_buffer_size; @@ -153,7 +155,7 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) * the GART base on a 4GB boundary as well. */ size_bf = mc->fb_start; - size_af = adev->gmc.mc_mask + 1 - ALIGN(mc->fb_end + 1, four_gb); + size_af = max_mc_address + 1 - ALIGN(mc->fb_end + 1, four_gb); if (mc->gart_size > max(size_bf, size_af)) { dev_warn(adev->dev, "limiting GART\n"); @@ -164,7 +166,7 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) (size_af < mc->gart_size)) mc->gart_start = 0; else - mc->gart_start = mc->mc_mask - mc->gart_size + 1; + mc->gart_start = max_mc_address - mc->gart_size + 1; mc->gart_start &= ~(four_gb - 1); mc->gart_end = mc->gart_start + mc->gart_size - 1; @@ -200,16 +202,13 @@ void amdgpu_gmc_agp_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc) } if (size_bf > size_af) { - mc->agp_start = mc->fb_start > mc->gart_start ? - mc->gart_end + 1 : 0; + mc->agp_start = (mc->fb_start - size_bf) & sixteen_gb_mask; mc->agp_size = size_bf; } else { - mc->agp_start = (mc->fb_start > mc->gart_start ? - mc->fb_end : mc->gart_end) + 1, + mc->agp_start = ALIGN(mc->fb_end + 1, sixteen_gb); mc->agp_size = size_af; } - mc->agp_start = ALIGN(mc->agp_start, sixteen_gb); mc->agp_end = mc->agp_start + mc->agp_size - 1; dev_info(adev->dev, "AGP: %lluM 0x%016llX - 0x%016llX\n", mc->agp_size >> 20, mc->agp_start, mc->agp_end); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c index 47817e00f54f..b8963b725dfa 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c @@ -354,6 +354,14 @@ int amdgpu_ib_ring_tests(struct amdgpu_device *adev) if (!ring || !ring->ready) continue; + /* skip IB tests for KIQ in general for the below reasons: + * 1. We never submit IBs to the KIQ + * 2. KIQ doesn't use the EOP interrupts, + * we use some other CP interrupt. + */ + if (ring->funcs->type == AMDGPU_RING_TYPE_KIQ) + continue; + /* MM engine need more time */ if (ring->funcs->type == AMDGPU_RING_TYPE_UVD || ring->funcs->type == AMDGPU_RING_TYPE_VCE || diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c index 3a072a7a39f0..df9b173c3d0b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c @@ -574,7 +574,7 @@ void amdgpu_vmid_mgr_init(struct amdgpu_device *adev) /* skip over VMID 0, since it is the system VM */ for (j = 1; j < id_mgr->num_ids; ++j) { amdgpu_vmid_reset(adev, i, j); - amdgpu_sync_create(&id_mgr->ids[i].active); + amdgpu_sync_create(&id_mgr->ids[j].active); list_add_tail(&id_mgr->ids[j].list, &id_mgr->ids_lru); } } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c index 4ed86218cef3..8af67f649660 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c @@ -24,46 +24,21 @@ #include <drm/drmP.h> #include "amdgpu.h" #include "amdgpu_ih.h" -#include "amdgpu_amdkfd.h" - -/** - * amdgpu_ih_ring_alloc - allocate memory for the IH ring - * - * @adev: amdgpu_device pointer - * - * Allocate a ring buffer for the interrupt controller. - * Returns 0 for success, errors for failure. - */ -static int amdgpu_ih_ring_alloc(struct amdgpu_device *adev) -{ - int r; - - /* Allocate ring buffer */ - if (adev->irq.ih.ring_obj == NULL) { - r = amdgpu_bo_create_kernel(adev, adev->irq.ih.ring_size, - PAGE_SIZE, AMDGPU_GEM_DOMAIN_GTT, - &adev->irq.ih.ring_obj, - &adev->irq.ih.gpu_addr, - (void **)&adev->irq.ih.ring); - if (r) { - DRM_ERROR("amdgpu: failed to create ih ring buffer (%d).\n", r); - return r; - } - } - return 0; -} /** * amdgpu_ih_ring_init - initialize the IH state * * @adev: amdgpu_device pointer + * @ih: ih ring to initialize + * @ring_size: ring size to allocate + * @use_bus_addr: true when we can use dma_alloc_coherent * * Initializes the IH state and allocates a buffer * for the IH ring buffer. * Returns 0 for success, errors for failure. */ -int amdgpu_ih_ring_init(struct amdgpu_device *adev, unsigned ring_size, - bool use_bus_addr) +int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, + unsigned ring_size, bool use_bus_addr) { u32 rb_bufsz; int r; @@ -71,70 +46,76 @@ int amdgpu_ih_ring_init(struct amdgpu_device *adev, unsigned ring_size, /* Align ring size */ rb_bufsz = order_base_2(ring_size / 4); ring_size = (1 << rb_bufsz) * 4; - adev->irq.ih.ring_size = ring_size; - adev->irq.ih.ptr_mask = adev->irq.ih.ring_size - 1; - adev->irq.ih.rptr = 0; - adev->irq.ih.use_bus_addr = use_bus_addr; - - if (adev->irq.ih.use_bus_addr) { - if (!adev->irq.ih.ring) { - /* add 8 bytes for the rptr/wptr shadows and - * add them to the end of the ring allocation. - */ - adev->irq.ih.ring = pci_alloc_consistent(adev->pdev, - adev->irq.ih.ring_size + 8, - &adev->irq.ih.rb_dma_addr); - if (adev->irq.ih.ring == NULL) - return -ENOMEM; - memset((void *)adev->irq.ih.ring, 0, adev->irq.ih.ring_size + 8); - adev->irq.ih.wptr_offs = (adev->irq.ih.ring_size / 4) + 0; - adev->irq.ih.rptr_offs = (adev->irq.ih.ring_size / 4) + 1; - } - return 0; + ih->ring_size = ring_size; + ih->ptr_mask = ih->ring_size - 1; + ih->rptr = 0; + ih->use_bus_addr = use_bus_addr; + + if (use_bus_addr) { + if (ih->ring) + return 0; + + /* add 8 bytes for the rptr/wptr shadows and + * add them to the end of the ring allocation. + */ + ih->ring = dma_alloc_coherent(adev->dev, ih->ring_size + 8, + &ih->rb_dma_addr, GFP_KERNEL); + if (ih->ring == NULL) + return -ENOMEM; + + memset((void *)ih->ring, 0, ih->ring_size + 8); + ih->wptr_offs = (ih->ring_size / 4) + 0; + ih->rptr_offs = (ih->ring_size / 4) + 1; } else { - r = amdgpu_device_wb_get(adev, &adev->irq.ih.wptr_offs); + r = amdgpu_device_wb_get(adev, &ih->wptr_offs); + if (r) + return r; + + r = amdgpu_device_wb_get(adev, &ih->rptr_offs); if (r) { - dev_err(adev->dev, "(%d) ih wptr_offs wb alloc failed\n", r); + amdgpu_device_wb_free(adev, ih->wptr_offs); return r; } - r = amdgpu_device_wb_get(adev, &adev->irq.ih.rptr_offs); + r = amdgpu_bo_create_kernel(adev, ih->ring_size, PAGE_SIZE, + AMDGPU_GEM_DOMAIN_GTT, + &ih->ring_obj, &ih->gpu_addr, + (void **)&ih->ring); if (r) { - amdgpu_device_wb_free(adev, adev->irq.ih.wptr_offs); - dev_err(adev->dev, "(%d) ih rptr_offs wb alloc failed\n", r); + amdgpu_device_wb_free(adev, ih->rptr_offs); + amdgpu_device_wb_free(adev, ih->wptr_offs); return r; } - - return amdgpu_ih_ring_alloc(adev); } + return 0; } /** * amdgpu_ih_ring_fini - tear down the IH state * * @adev: amdgpu_device pointer + * @ih: ih ring to tear down * * Tears down the IH state and frees buffer * used for the IH ring buffer. */ -void amdgpu_ih_ring_fini(struct amdgpu_device *adev) +void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih) { - if (adev->irq.ih.use_bus_addr) { - if (adev->irq.ih.ring) { - /* add 8 bytes for the rptr/wptr shadows and - * add them to the end of the ring allocation. - */ - pci_free_consistent(adev->pdev, adev->irq.ih.ring_size + 8, - (void *)adev->irq.ih.ring, - adev->irq.ih.rb_dma_addr); - adev->irq.ih.ring = NULL; - } + if (ih->use_bus_addr) { + if (!ih->ring) + return; + + /* add 8 bytes for the rptr/wptr shadows and + * add them to the end of the ring allocation. + */ + dma_free_coherent(adev->dev, ih->ring_size + 8, + (void *)ih->ring, ih->rb_dma_addr); + ih->ring = NULL; } else { - amdgpu_bo_free_kernel(&adev->irq.ih.ring_obj, - &adev->irq.ih.gpu_addr, - (void **)&adev->irq.ih.ring); - amdgpu_device_wb_free(adev, adev->irq.ih.wptr_offs); - amdgpu_device_wb_free(adev, adev->irq.ih.rptr_offs); + amdgpu_bo_free_kernel(&ih->ring_obj, &ih->gpu_addr, + (void **)&ih->ring); + amdgpu_device_wb_free(adev, ih->wptr_offs); + amdgpu_device_wb_free(adev, ih->rptr_offs); } } @@ -142,56 +123,43 @@ void amdgpu_ih_ring_fini(struct amdgpu_device *adev) * amdgpu_ih_process - interrupt handler * * @adev: amdgpu_device pointer + * @ih: ih ring to process * * Interrupt hander (VI), walk the IH ring. * Returns irq process return code. */ -int amdgpu_ih_process(struct amdgpu_device *adev) +int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, + void (*callback)(struct amdgpu_device *adev, + struct amdgpu_ih_ring *ih)) { - struct amdgpu_iv_entry entry; u32 wptr; - if (!adev->irq.ih.enabled || adev->shutdown) + if (!ih->enabled || adev->shutdown) return IRQ_NONE; wptr = amdgpu_ih_get_wptr(adev); restart_ih: /* is somebody else already processing irqs? */ - if (atomic_xchg(&adev->irq.ih.lock, 1)) + if (atomic_xchg(&ih->lock, 1)) return IRQ_NONE; - DRM_DEBUG("%s: rptr %d, wptr %d\n", __func__, adev->irq.ih.rptr, wptr); + DRM_DEBUG("%s: rptr %d, wptr %d\n", __func__, ih->rptr, wptr); /* Order reading of wptr vs. reading of IH ring data */ rmb(); - while (adev->irq.ih.rptr != wptr) { - u32 ring_index = adev->irq.ih.rptr >> 2; - - /* Prescreening of high-frequency interrupts */ - if (!amdgpu_ih_prescreen_iv(adev)) { - adev->irq.ih.rptr &= adev->irq.ih.ptr_mask; - continue; - } - - /* Before dispatching irq to IP blocks, send it to amdkfd */ - amdgpu_amdkfd_interrupt(adev, - (const void *) &adev->irq.ih.ring[ring_index]); - - entry.iv_entry = (const uint32_t *) - &adev->irq.ih.ring[ring_index]; - amdgpu_ih_decode_iv(adev, &entry); - adev->irq.ih.rptr &= adev->irq.ih.ptr_mask; - - amdgpu_irq_dispatch(adev, &entry); + while (ih->rptr != wptr) { + callback(adev, ih); + ih->rptr &= ih->ptr_mask; } + amdgpu_ih_set_rptr(adev); - atomic_set(&adev->irq.ih.lock, 0); + atomic_set(&ih->lock, 0); /* make sure wptr hasn't changed while processing */ wptr = amdgpu_ih_get_wptr(adev); - if (wptr != adev->irq.ih.rptr) + if (wptr != ih->rptr) goto restart_ih; return IRQ_HANDLED; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h index 0d5b3f5201d2..9ce8c93ec19b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h @@ -24,12 +24,8 @@ #ifndef __AMDGPU_IH_H__ #define __AMDGPU_IH_H__ -#include "soc15_ih_clientid.h" - struct amdgpu_device; - -#define AMDGPU_IH_CLIENTID_LEGACY 0 -#define AMDGPU_IH_CLIENTID_MAX SOC15_IH_CLIENTID_MAX +struct amdgpu_iv_entry; /* * R6xx+ IH ring @@ -51,22 +47,6 @@ struct amdgpu_ih_ring { dma_addr_t rb_dma_addr; /* only used when use_bus_addr = true */ }; -#define AMDGPU_IH_SRC_DATA_MAX_SIZE_DW 4 - -struct amdgpu_iv_entry { - unsigned client_id; - unsigned src_id; - unsigned ring_id; - unsigned vmid; - unsigned vmid_src; - uint64_t timestamp; - unsigned timestamp_src; - unsigned pasid; - unsigned pasid_src; - unsigned src_data[AMDGPU_IH_SRC_DATA_MAX_SIZE_DW]; - const uint32_t *iv_entry; -}; - /* provided by the ih block */ struct amdgpu_ih_funcs { /* ring read/write ptr handling, called from interrupt context */ @@ -82,9 +62,11 @@ struct amdgpu_ih_funcs { #define amdgpu_ih_decode_iv(adev, iv) (adev)->irq.ih_funcs->decode_iv((adev), (iv)) #define amdgpu_ih_set_rptr(adev) (adev)->irq.ih_funcs->set_rptr((adev)) -int amdgpu_ih_ring_init(struct amdgpu_device *adev, unsigned ring_size, - bool use_bus_addr); -void amdgpu_ih_ring_fini(struct amdgpu_device *adev); -int amdgpu_ih_process(struct amdgpu_device *adev); +int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, + unsigned ring_size, bool use_bus_addr); +void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih); +int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih, + void (*callback)(struct amdgpu_device *adev, + struct amdgpu_ih_ring *ih)); #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index b927e8798534..52c17f6219a7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -51,6 +51,7 @@ #include "atom.h" #include "amdgpu_connectors.h" #include "amdgpu_trace.h" +#include "amdgpu_amdkfd.h" #include <linux/pm_runtime.h> @@ -123,7 +124,7 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev) int r; spin_lock_irqsave(&adev->irq.lock, irqflags); - for (i = 0; i < AMDGPU_IH_CLIENTID_MAX; ++i) { + for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; @@ -147,6 +148,34 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev) } /** + * amdgpu_irq_callback - callback from the IH ring + * + * @adev: amdgpu device pointer + * @ih: amdgpu ih ring + * + * Callback from IH ring processing to handle the entry at the current position + * and advance the read pointer. + */ +static void amdgpu_irq_callback(struct amdgpu_device *adev, + struct amdgpu_ih_ring *ih) +{ + u32 ring_index = ih->rptr >> 2; + struct amdgpu_iv_entry entry; + + /* Prescreening of high-frequency interrupts */ + if (!amdgpu_ih_prescreen_iv(adev)) + return; + + /* Before dispatching irq to IP blocks, send it to amdkfd */ + amdgpu_amdkfd_interrupt(adev, (const void *) &ih->ring[ring_index]); + + entry.iv_entry = (const uint32_t *)&ih->ring[ring_index]; + amdgpu_ih_decode_iv(adev, &entry); + + amdgpu_irq_dispatch(adev, &entry); +} + +/** * amdgpu_irq_handler - IRQ handler * * @irq: IRQ number (unused) @@ -163,7 +192,7 @@ irqreturn_t amdgpu_irq_handler(int irq, void *arg) struct amdgpu_device *adev = dev->dev_private; irqreturn_t ret; - ret = amdgpu_ih_process(adev); + ret = amdgpu_ih_process(adev, &adev->irq.ih, amdgpu_irq_callback); if (ret == IRQ_HANDLED) pm_runtime_mark_last_busy(dev->dev); return ret; @@ -273,7 +302,7 @@ void amdgpu_irq_fini(struct amdgpu_device *adev) cancel_work_sync(&adev->reset_work); } - for (i = 0; i < AMDGPU_IH_CLIENTID_MAX; ++i) { + for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; @@ -313,7 +342,7 @@ int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source) { - if (client_id >= AMDGPU_IH_CLIENTID_MAX) + if (client_id >= AMDGPU_IRQ_CLIENTID_MAX) return -EINVAL; if (src_id >= AMDGPU_MAX_IRQ_SRC_ID) @@ -367,7 +396,7 @@ void amdgpu_irq_dispatch(struct amdgpu_device *adev, trace_amdgpu_iv(entry); - if (client_id >= AMDGPU_IH_CLIENTID_MAX) { + if (client_id >= AMDGPU_IRQ_CLIENTID_MAX) { DRM_DEBUG("Invalid client_id in IV: %d\n", client_id); return; } @@ -440,7 +469,7 @@ void amdgpu_irq_gpu_reset_resume_helper(struct amdgpu_device *adev) { int i, j, k; - for (i = 0; i < AMDGPU_IH_CLIENTID_MAX; ++i) { + for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index 3375ad778edc..f6ce171cb8aa 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -25,19 +25,38 @@ #define __AMDGPU_IRQ_H__ #include <linux/irqdomain.h> +#include "soc15_ih_clientid.h" #include "amdgpu_ih.h" -#define AMDGPU_MAX_IRQ_SRC_ID 0x100 +#define AMDGPU_MAX_IRQ_SRC_ID 0x100 #define AMDGPU_MAX_IRQ_CLIENT_ID 0x100 +#define AMDGPU_IRQ_CLIENTID_LEGACY 0 +#define AMDGPU_IRQ_CLIENTID_MAX SOC15_IH_CLIENTID_MAX + +#define AMDGPU_IRQ_SRC_DATA_MAX_SIZE_DW 4 + struct amdgpu_device; -struct amdgpu_iv_entry; enum amdgpu_interrupt_state { AMDGPU_IRQ_STATE_DISABLE, AMDGPU_IRQ_STATE_ENABLE, }; +struct amdgpu_iv_entry { + unsigned client_id; + unsigned src_id; + unsigned ring_id; + unsigned vmid; + unsigned vmid_src; + uint64_t timestamp; + unsigned timestamp_src; + unsigned pasid; + unsigned pasid_src; + unsigned src_data[AMDGPU_IRQ_SRC_DATA_MAX_SIZE_DW]; + const uint32_t *iv_entry; +}; + struct amdgpu_irq_src { unsigned num_types; atomic_t *enabled_types; @@ -63,7 +82,7 @@ struct amdgpu_irq { bool installed; spinlock_t lock; /* interrupt sources */ - struct amdgpu_irq_client client[AMDGPU_IH_CLIENTID_MAX]; + struct amdgpu_irq_client client[AMDGPU_IRQ_CLIENTID_MAX]; /* status, etc. */ bool msi_enabled; /* msi enabled */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index a64056dadc58..81732a84c2ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -40,6 +40,30 @@ #include "amdgpu_gem.h" #include "amdgpu_display.h" +static void amdgpu_unregister_gpu_instance(struct amdgpu_device *adev) +{ + struct amdgpu_gpu_instance *gpu_instance; + int i; + + mutex_lock(&mgpu_info.mutex); + + for (i = 0; i < mgpu_info.num_gpu; i++) { + gpu_instance = &(mgpu_info.gpu_ins[i]); + if (gpu_instance->adev == adev) { + mgpu_info.gpu_ins[i] = + mgpu_info.gpu_ins[mgpu_info.num_gpu - 1]; + mgpu_info.num_gpu--; + if (adev->flags & AMD_IS_APU) + mgpu_info.num_apu--; + else + mgpu_info.num_dgpu--; + break; + } + } + + mutex_unlock(&mgpu_info.mutex); +} + /** * amdgpu_driver_unload_kms - Main unload function for KMS. * @@ -55,6 +79,8 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) if (adev == NULL) return; + amdgpu_unregister_gpu_instance(adev); + if (adev->rmmio == NULL) goto done_free; @@ -75,6 +101,31 @@ done_free: dev->dev_private = NULL; } +static void amdgpu_register_gpu_instance(struct amdgpu_device *adev) +{ + struct amdgpu_gpu_instance *gpu_instance; + + mutex_lock(&mgpu_info.mutex); + + if (mgpu_info.num_gpu >= MAX_GPU_INSTANCE) { + DRM_ERROR("Cannot register more gpu instance\n"); + mutex_unlock(&mgpu_info.mutex); + return; + } + + gpu_instance = &(mgpu_info.gpu_ins[mgpu_info.num_gpu]); + gpu_instance->adev = adev; + gpu_instance->mgpu_fan_enabled = 0; + + mgpu_info.num_gpu++; + if (adev->flags & AMD_IS_APU) + mgpu_info.num_apu++; + else + mgpu_info.num_dgpu++; + + mutex_unlock(&mgpu_info.mutex); +} + /** * amdgpu_driver_load_kms - Main load function for KMS. * @@ -169,6 +220,7 @@ int amdgpu_driver_load_kms(struct drm_device *dev, unsigned long flags) pm_runtime_put_autosuspend(dev->dev); } + amdgpu_register_gpu_instance(adev); out: if (r) { /* balance pm_runtime_get_sync in amdgpu_driver_unload_kms */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c index 8c334fc808c2..59cc678de8c1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c @@ -704,7 +704,10 @@ static ssize_t amdgpu_set_pp_dpm_sclk(struct device *dev, return ret; if (adev->powerplay.pp_funcs->force_clock_level) - amdgpu_dpm_force_clock_level(adev, PP_SCLK, mask); + ret = amdgpu_dpm_force_clock_level(adev, PP_SCLK, mask); + + if (ret) + return -EINVAL; return count; } @@ -737,7 +740,10 @@ static ssize_t amdgpu_set_pp_dpm_mclk(struct device *dev, return ret; if (adev->powerplay.pp_funcs->force_clock_level) - amdgpu_dpm_force_clock_level(adev, PP_MCLK, mask); + ret = amdgpu_dpm_force_clock_level(adev, PP_MCLK, mask); + + if (ret) + return -EINVAL; return count; } @@ -770,7 +776,10 @@ static ssize_t amdgpu_set_pp_dpm_pcie(struct device *dev, return ret; if (adev->powerplay.pp_funcs->force_clock_level) - amdgpu_dpm_force_clock_level(adev, PP_PCIE, mask); + ret = amdgpu_dpm_force_clock_level(adev, PP_PCIE, mask); + + if (ret) + return -EINVAL; return count; } @@ -1120,12 +1129,19 @@ static ssize_t amdgpu_hwmon_set_pwm1(struct device *dev, struct amdgpu_device *adev = dev_get_drvdata(dev); int err; u32 value; + u32 pwm_mode; /* Can't adjust fan when the card is off */ if ((adev->flags & AMD_IS_PX) && (adev->ddev->switch_power_state != DRM_SWITCH_POWER_ON)) return -EINVAL; + pwm_mode = amdgpu_dpm_get_fan_control_mode(adev); + if (pwm_mode != AMD_FAN_CTRL_MANUAL) { + pr_info("manual fan speed control should be enabled first\n"); + return -EINVAL; + } + err = kstrtou32(buf, 10, &value); if (err) return err; @@ -1187,6 +1203,148 @@ static ssize_t amdgpu_hwmon_get_fan1_input(struct device *dev, return sprintf(buf, "%i\n", speed); } +static ssize_t amdgpu_hwmon_get_fan1_min(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct amdgpu_device *adev = dev_get_drvdata(dev); + u32 min_rpm = 0; + u32 size = sizeof(min_rpm); + int r; + + if (!adev->powerplay.pp_funcs->read_sensor) + return -EINVAL; + + r = amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_MIN_FAN_RPM, + (void *)&min_rpm, &size); + if (r) + return r; + + return snprintf(buf, PAGE_SIZE, "%d\n", min_rpm); +} + +static ssize_t amdgpu_hwmon_get_fan1_max(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct amdgpu_device *adev = dev_get_drvdata(dev); + u32 max_rpm = 0; + u32 size = sizeof(max_rpm); + int r; + + if (!adev->powerplay.pp_funcs->read_sensor) + return -EINVAL; + + r = amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_MAX_FAN_RPM, + (void *)&max_rpm, &size); + if (r) + return r; + + return snprintf(buf, PAGE_SIZE, "%d\n", max_rpm); +} + +static ssize_t amdgpu_hwmon_get_fan1_target(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct amdgpu_device *adev = dev_get_drvdata(dev); + int err; + u32 rpm = 0; + + /* Can't adjust fan when the card is off */ + if ((adev->flags & AMD_IS_PX) && + (adev->ddev->switch_power_state != DRM_SWITCH_POWER_ON)) + return -EINVAL; + + if (adev->powerplay.pp_funcs->get_fan_speed_rpm) { + err = amdgpu_dpm_get_fan_speed_rpm(adev, &rpm); + if (err) + return err; + } + + return sprintf(buf, "%i\n", rpm); +} + +static ssize_t amdgpu_hwmon_set_fan1_target(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t count) +{ + struct amdgpu_device *adev = dev_get_drvdata(dev); + int err; + u32 value; + u32 pwm_mode; + + pwm_mode = amdgpu_dpm_get_fan_control_mode(adev); + if (pwm_mode != AMD_FAN_CTRL_MANUAL) + return -ENODATA; + + /* Can't adjust fan when the card is off */ + if ((adev->flags & AMD_IS_PX) && + (adev->ddev->switch_power_state != DRM_SWITCH_POWER_ON)) + return -EINVAL; + + err = kstrtou32(buf, 10, &value); + if (err) + return err; + + if (adev->powerplay.pp_funcs->set_fan_speed_rpm) { + err = amdgpu_dpm_set_fan_speed_rpm(adev, value); + if (err) + return err; + } + + return count; +} + +static ssize_t amdgpu_hwmon_get_fan1_enable(struct device *dev, + struct device_attribute *attr, + char *buf) +{ + struct amdgpu_device *adev = dev_get_drvdata(dev); + u32 pwm_mode = 0; + + if (!adev->powerplay.pp_funcs->get_fan_control_mode) + return -EINVAL; + + pwm_mode = amdgpu_dpm_get_fan_control_mode(adev); + + return sprintf(buf, "%i\n", pwm_mode == AMD_FAN_CTRL_AUTO ? 0 : 1); +} + +static ssize_t amdgpu_hwmon_set_fan1_enable(struct device *dev, + struct device_attribute *attr, + const char *buf, + size_t count) +{ + struct amdgpu_device *adev = dev_get_drvdata(dev); + int err; + int value; + u32 pwm_mode; + + /* Can't adjust fan when the card is off */ + if ((adev->flags & AMD_IS_PX) && + (adev->ddev->switch_power_state != DRM_SWITCH_POWER_ON)) + return -EINVAL; + + if (!adev->powerplay.pp_funcs->set_fan_control_mode) + return -EINVAL; + + err = kstrtoint(buf, 10, &value); + if (err) + return err; + + if (value == 0) + pwm_mode = AMD_FAN_CTRL_AUTO; + else if (value == 1) + pwm_mode = AMD_FAN_CTRL_MANUAL; + else + return -EINVAL; + + amdgpu_dpm_set_fan_control_mode(adev, pwm_mode); + + return count; +} + static ssize_t amdgpu_hwmon_show_vddgfx(struct device *dev, struct device_attribute *attr, char *buf) @@ -1406,8 +1564,16 @@ static ssize_t amdgpu_hwmon_set_power_cap(struct device *dev, * * - pwm1_max: pulse width modulation fan control maximum level (255) * + * - fan1_min: an minimum value Unit: revolution/min (RPM) + * + * - fan1_max: an maxmum value Unit: revolution/max (RPM) + * * - fan1_input: fan speed in RPM * + * - fan[1-*]_target: Desired fan speed Unit: revolution/min (RPM) + * + * - fan[1-*]_enable: Enable or disable the sensors.1: Enable 0: Disable + * * You can use hwmon tools like sensors to view this information on your system. * */ @@ -1420,6 +1586,10 @@ static SENSOR_DEVICE_ATTR(pwm1_enable, S_IRUGO | S_IWUSR, amdgpu_hwmon_get_pwm1_ static SENSOR_DEVICE_ATTR(pwm1_min, S_IRUGO, amdgpu_hwmon_get_pwm1_min, NULL, 0); static SENSOR_DEVICE_ATTR(pwm1_max, S_IRUGO, amdgpu_hwmon_get_pwm1_max, NULL, 0); static SENSOR_DEVICE_ATTR(fan1_input, S_IRUGO, amdgpu_hwmon_get_fan1_input, NULL, 0); +static SENSOR_DEVICE_ATTR(fan1_min, S_IRUGO, amdgpu_hwmon_get_fan1_min, NULL, 0); +static SENSOR_DEVICE_ATTR(fan1_max, S_IRUGO, amdgpu_hwmon_get_fan1_max, NULL, 0); +static SENSOR_DEVICE_ATTR(fan1_target, S_IRUGO | S_IWUSR, amdgpu_hwmon_get_fan1_target, amdgpu_hwmon_set_fan1_target, 0); +static SENSOR_DEVICE_ATTR(fan1_enable, S_IRUGO | S_IWUSR, amdgpu_hwmon_get_fan1_enable, amdgpu_hwmon_set_fan1_enable, 0); static SENSOR_DEVICE_ATTR(in0_input, S_IRUGO, amdgpu_hwmon_show_vddgfx, NULL, 0); static SENSOR_DEVICE_ATTR(in0_label, S_IRUGO, amdgpu_hwmon_show_vddgfx_label, NULL, 0); static SENSOR_DEVICE_ATTR(in1_input, S_IRUGO, amdgpu_hwmon_show_vddnb, NULL, 0); @@ -1438,6 +1608,10 @@ static struct attribute *hwmon_attributes[] = { &sensor_dev_attr_pwm1_min.dev_attr.attr, &sensor_dev_attr_pwm1_max.dev_attr.attr, &sensor_dev_attr_fan1_input.dev_attr.attr, + &sensor_dev_attr_fan1_min.dev_attr.attr, + &sensor_dev_attr_fan1_max.dev_attr.attr, + &sensor_dev_attr_fan1_target.dev_attr.attr, + &sensor_dev_attr_fan1_enable.dev_attr.attr, &sensor_dev_attr_in0_input.dev_attr.attr, &sensor_dev_attr_in0_label.dev_attr.attr, &sensor_dev_attr_in1_input.dev_attr.attr, @@ -1456,13 +1630,16 @@ static umode_t hwmon_attributes_visible(struct kobject *kobj, struct amdgpu_device *adev = dev_get_drvdata(dev); umode_t effective_mode = attr->mode; - /* Skip fan attributes if fan is not present */ if (adev->pm.no_fan && (attr == &sensor_dev_attr_pwm1.dev_attr.attr || attr == &sensor_dev_attr_pwm1_enable.dev_attr.attr || attr == &sensor_dev_attr_pwm1_max.dev_attr.attr || attr == &sensor_dev_attr_pwm1_min.dev_attr.attr || - attr == &sensor_dev_attr_fan1_input.dev_attr.attr)) + attr == &sensor_dev_attr_fan1_input.dev_attr.attr || + attr == &sensor_dev_attr_fan1_min.dev_attr.attr || + attr == &sensor_dev_attr_fan1_max.dev_attr.attr || + attr == &sensor_dev_attr_fan1_target.dev_attr.attr || + attr == &sensor_dev_attr_fan1_enable.dev_attr.attr)) return 0; /* Skip limit attributes if DPM is not enabled */ @@ -1472,7 +1649,12 @@ static umode_t hwmon_attributes_visible(struct kobject *kobj, attr == &sensor_dev_attr_pwm1.dev_attr.attr || attr == &sensor_dev_attr_pwm1_enable.dev_attr.attr || attr == &sensor_dev_attr_pwm1_max.dev_attr.attr || - attr == &sensor_dev_attr_pwm1_min.dev_attr.attr)) + attr == &sensor_dev_attr_pwm1_min.dev_attr.attr || + attr == &sensor_dev_attr_fan1_input.dev_attr.attr || + attr == &sensor_dev_attr_fan1_min.dev_attr.attr || + attr == &sensor_dev_attr_fan1_max.dev_attr.attr || + attr == &sensor_dev_attr_fan1_target.dev_attr.attr || + attr == &sensor_dev_attr_fan1_enable.dev_attr.attr)) return 0; /* mask fan attributes if we have no bindings for this asic to expose */ @@ -1497,10 +1679,18 @@ static umode_t hwmon_attributes_visible(struct kobject *kobj, /* hide max/min values if we can't both query and manage the fan */ if ((!adev->powerplay.pp_funcs->set_fan_speed_percent && !adev->powerplay.pp_funcs->get_fan_speed_percent) && + (!adev->powerplay.pp_funcs->set_fan_speed_rpm && + !adev->powerplay.pp_funcs->get_fan_speed_rpm) && (attr == &sensor_dev_attr_pwm1_max.dev_attr.attr || attr == &sensor_dev_attr_pwm1_min.dev_attr.attr)) return 0; + if ((!adev->powerplay.pp_funcs->set_fan_speed_rpm && + !adev->powerplay.pp_funcs->get_fan_speed_rpm) && + (attr == &sensor_dev_attr_fan1_max.dev_attr.attr || + attr == &sensor_dev_attr_fan1_min.dev_attr.attr)) + return 0; + /* only APUs have vddnb */ if (!(adev->flags & AMD_IS_APU) && (attr == &sensor_dev_attr_in1_input.dev_attr.attr || @@ -1976,6 +2166,7 @@ void amdgpu_pm_compute_clocks(struct amdgpu_device *adev) static int amdgpu_debugfs_pm_info_pp(struct seq_file *m, struct amdgpu_device *adev) { uint32_t value; + uint64_t value64; uint32_t query = 0; int size; @@ -2014,6 +2205,10 @@ static int amdgpu_debugfs_pm_info_pp(struct seq_file *m, struct amdgpu_device *a seq_printf(m, "GPU Load: %u %%\n", value); seq_printf(m, "\n"); + /* SMC feature mask */ + if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_ENABLED_SMC_FEATURES_MASK, (void *)&value64, &size)) + seq_printf(m, "SMC Feature Mask: 0x%016llx\n", value64); + /* UVD clocks */ if (!amdgpu_dpm_read_sensor(adev, AMDGPU_PP_SENSOR_UVD_POWER, (void *)&value, &size)) { if (!value) { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index bd397d2916fb..25d2f3e757f1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -452,8 +452,6 @@ static int psp_hw_fini(void *handle) if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) return 0; - amdgpu_ucode_fini_bo(adev); - psp_ring_destroy(psp, PSP_RING_TYPE__KM); amdgpu_bo_free_kernel(&psp->tmr_bo, &psp->tmr_mc_addr, &psp->tmr_buf); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 44fc665e4577..4caa301ce454 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -77,6 +77,7 @@ struct amdgpu_fence_driver { bool initialized; struct amdgpu_irq_src *irq_src; unsigned irq_type; + struct timer_list fallback_timer; unsigned num_fences_mask; spinlock_t lock; struct dma_fence **fences; @@ -96,7 +97,7 @@ void amdgpu_fence_driver_resume(struct amdgpu_device *adev); int amdgpu_fence_emit(struct amdgpu_ring *ring, struct dma_fence **fence, unsigned flags); int amdgpu_fence_emit_polling(struct amdgpu_ring *ring, uint32_t *s); -void amdgpu_fence_process(struct amdgpu_ring *ring); +bool amdgpu_fence_process(struct amdgpu_ring *ring); int amdgpu_fence_wait_empty(struct amdgpu_ring *ring); signed long amdgpu_fence_wait_polling(struct amdgpu_ring *ring, uint32_t wait_seq, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h index d17503f0df8e..500113ec65ca 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h @@ -46,10 +46,6 @@ struct amdgpu_sdma_instance { struct amdgpu_sdma { struct amdgpu_sdma_instance instance[AMDGPU_MAX_SDMA_INSTANCES]; -#ifdef CONFIG_DRM_AMDGPU_SI - //SI DMA has a difference trap irq number for the second engine - struct amdgpu_irq_src trap_irq_1; -#endif struct amdgpu_irq_src trap_irq; struct amdgpu_irq_src illegal_inst_irq; int num_instances; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h index 2e87414422f9..e9bf70e2ac51 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace.h @@ -103,7 +103,7 @@ TRACE_EVENT(amdgpu_iv, __entry->src_data[2] = iv->src_data[2]; __entry->src_data[3] = iv->src_data[3]; ), - TP_printk("client_id:%u src_id:%u ring:%u vmid:%u timestamp: %llu pasid:%u src_data: %08x %08x %08x %08x\n", + TP_printk("client_id:%u src_id:%u ring:%u vmid:%u timestamp: %llu pasid:%u src_data: %08x %08x %08x %08x", __entry->client_id, __entry->src_id, __entry->ring_id, __entry->vmid, __entry->timestamp, __entry->pasid, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace_points.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace_points.c index b160b958e5fe..f212402570a5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_trace_points.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_trace_points.c @@ -1,4 +1,4 @@ -// SPDX-License-Identifier: GPL-2.0 +// SPDX-License-Identifier: MIT /* Copyright Red Hat Inc 2010. * * Permission is hereby granted, free of charge, to any person obtaining a diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c index 1fa8bc337859..7b33867036e7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c @@ -297,10 +297,7 @@ amdgpu_ucode_get_load_type(struct amdgpu_device *adev, int load_type) case CHIP_POLARIS11: case CHIP_POLARIS12: case CHIP_VEGAM: - if (!load_type) - return AMDGPU_FW_LOAD_DIRECT; - else - return AMDGPU_FW_LOAD_SMU; + return AMDGPU_FW_LOAD_SMU; case CHIP_VEGA10: case CHIP_RAVEN: case CHIP_VEGA12: @@ -423,32 +420,41 @@ static int amdgpu_ucode_patch_jt(struct amdgpu_firmware_info *ucode, return 0; } -int amdgpu_ucode_init_bo(struct amdgpu_device *adev) +int amdgpu_ucode_create_bo(struct amdgpu_device *adev) { - uint64_t fw_offset = 0; - int i, err; - struct amdgpu_firmware_info *ucode = NULL; - const struct common_firmware_header *header = NULL; - - if (!adev->firmware.fw_size) { - dev_warn(adev->dev, "No ip firmware need to load\n"); - return 0; - } - - if (!adev->in_gpu_reset) { - err = amdgpu_bo_create_kernel(adev, adev->firmware.fw_size, PAGE_SIZE, - amdgpu_sriov_vf(adev) ? AMDGPU_GEM_DOMAIN_VRAM : AMDGPU_GEM_DOMAIN_GTT, - &adev->firmware.fw_buf, - &adev->firmware.fw_buf_mc, - &adev->firmware.fw_buf_ptr); - if (err) { + if (adev->firmware.load_type != AMDGPU_FW_LOAD_DIRECT) { + amdgpu_bo_create_kernel(adev, adev->firmware.fw_size, PAGE_SIZE, + amdgpu_sriov_vf(adev) ? AMDGPU_GEM_DOMAIN_VRAM : AMDGPU_GEM_DOMAIN_GTT, + &adev->firmware.fw_buf, + &adev->firmware.fw_buf_mc, + &adev->firmware.fw_buf_ptr); + if (!adev->firmware.fw_buf) { dev_err(adev->dev, "failed to create kernel buffer for firmware.fw_buf\n"); - goto failed; + return -ENOMEM; + } else if (amdgpu_sriov_vf(adev)) { + memset(adev->firmware.fw_buf_ptr, 0, adev->firmware.fw_size); } } + return 0; +} + +void amdgpu_ucode_free_bo(struct amdgpu_device *adev) +{ + if (adev->firmware.load_type != AMDGPU_FW_LOAD_DIRECT) + amdgpu_bo_free_kernel(&adev->firmware.fw_buf, + &adev->firmware.fw_buf_mc, + &adev->firmware.fw_buf_ptr); +} - memset(adev->firmware.fw_buf_ptr, 0, adev->firmware.fw_size); +int amdgpu_ucode_init_bo(struct amdgpu_device *adev) +{ + uint64_t fw_offset = 0; + int i; + struct amdgpu_firmware_info *ucode = NULL; + /* for baremetal, the ucode is allocated in gtt, so don't need to fill the bo when reset/suspend */ + if (!amdgpu_sriov_vf(adev) && (adev->in_gpu_reset || adev->in_suspend)) + return 0; /* * if SMU loaded firmware, it needn't add SMC, UVD, and VCE * ucode info here @@ -465,7 +471,6 @@ int amdgpu_ucode_init_bo(struct amdgpu_device *adev) for (i = 0; i < adev->firmware.max_ucodes; i++) { ucode = &adev->firmware.ucode[i]; if (ucode->fw) { - header = (const struct common_firmware_header *)ucode->fw->data; amdgpu_ucode_init_single_fw(adev, ucode, adev->firmware.fw_buf_mc + fw_offset, adev->firmware.fw_buf_ptr + fw_offset); if (i == AMDGPU_UCODE_ID_CP_MEC1 && @@ -480,33 +485,4 @@ int amdgpu_ucode_init_bo(struct amdgpu_device *adev) } } return 0; - -failed: - if (err) - adev->firmware.load_type = AMDGPU_FW_LOAD_DIRECT; - - return err; -} - -int amdgpu_ucode_fini_bo(struct amdgpu_device *adev) -{ - int i; - struct amdgpu_firmware_info *ucode = NULL; - - if (!adev->firmware.fw_size) - return 0; - - for (i = 0; i < adev->firmware.max_ucodes; i++) { - ucode = &adev->firmware.ucode[i]; - if (ucode->fw) { - ucode->mc_addr = 0; - ucode->kaddr = NULL; - } - } - - amdgpu_bo_free_kernel(&adev->firmware.fw_buf, - &adev->firmware.fw_buf_mc, - &adev->firmware.fw_buf_ptr); - - return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h index 8f3f1117728c..aa6641b944a0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h @@ -276,8 +276,10 @@ void amdgpu_ucode_print_gpu_info_hdr(const struct common_firmware_header *hdr); int amdgpu_ucode_validate(const struct firmware *fw); bool amdgpu_ucode_hdr_version(union amdgpu_firmware_header *hdr, uint16_t hdr_major, uint16_t hdr_minor); + int amdgpu_ucode_init_bo(struct amdgpu_device *adev); -int amdgpu_ucode_fini_bo(struct amdgpu_device *adev); +int amdgpu_ucode_create_bo(struct amdgpu_device *adev); +void amdgpu_ucode_free_bo(struct amdgpu_device *adev); enum amdgpu_firmware_load_type amdgpu_ucode_get_load_type(struct amdgpu_device *adev, int load_type); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c index 0cc5190f4f36..5f3f54073818 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vce.c @@ -258,6 +258,8 @@ int amdgpu_vce_suspend(struct amdgpu_device *adev) { int i; + cancel_delayed_work_sync(&adev->vce.idle_work); + if (adev->vce.vcpu_bo == NULL) return 0; @@ -268,7 +270,6 @@ int amdgpu_vce_suspend(struct amdgpu_device *adev) if (i == AMDGPU_MAX_VCE_HANDLES) return 0; - cancel_delayed_work_sync(&adev->vce.idle_work); /* TODO: suspending running encoding sessions isn't supported */ return -EINVAL; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c index a73674f9a0f5..27da13df2f11 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c @@ -36,6 +36,7 @@ #include "soc15_common.h" #include "vcn/vcn_1_0_offset.h" +#include "vcn/vcn_1_0_sh_mask.h" /* 1 second timeout */ #define VCN_IDLE_TIMEOUT msecs_to_jiffies(1000) @@ -120,8 +121,7 @@ int amdgpu_vcn_sw_init(struct amdgpu_device *adev) version_major, version_minor, family_id); } - bo_size = AMDGPU_VCN_STACK_SIZE + AMDGPU_VCN_HEAP_SIZE - + AMDGPU_VCN_SESSION_SIZE * 40; + bo_size = AMDGPU_VCN_STACK_SIZE + AMDGPU_VCN_CONTEXT_SIZE; if (adev->firmware.load_type != AMDGPU_FW_LOAD_PSP) bo_size += AMDGPU_GPU_PAGE_ALIGN(le32_to_cpu(hdr->ucode_size_bytes) + 8); r = amdgpu_bo_create_kernel(adev, bo_size, PAGE_SIZE, @@ -162,11 +162,11 @@ int amdgpu_vcn_suspend(struct amdgpu_device *adev) unsigned size; void *ptr; + cancel_delayed_work_sync(&adev->vcn.idle_work); + if (adev->vcn.vcpu_bo == NULL) return 0; - cancel_delayed_work_sync(&adev->vcn.idle_work); - size = amdgpu_bo_size(adev->vcn.vcpu_bo); ptr = adev->vcn.cpu_addr; @@ -212,18 +212,161 @@ int amdgpu_vcn_resume(struct amdgpu_device *adev) return 0; } +static int amdgpu_vcn_pause_dpg_mode(struct amdgpu_device *adev, + struct dpg_pause_state *new_state) +{ + int ret_code; + uint32_t reg_data = 0; + uint32_t reg_data2 = 0; + struct amdgpu_ring *ring; + + /* pause/unpause if state is changed */ + if (adev->vcn.pause_state.fw_based != new_state->fw_based) { + DRM_DEBUG("dpg pause state changed %d:%d -> %d:%d", + adev->vcn.pause_state.fw_based, adev->vcn.pause_state.jpeg, + new_state->fw_based, new_state->jpeg); + + reg_data = RREG32_SOC15(UVD, 0, mmUVD_DPG_PAUSE) & + (~UVD_DPG_PAUSE__NJ_PAUSE_DPG_ACK_MASK); + + if (new_state->fw_based == VCN_DPG_STATE__PAUSE) { + ret_code = 0; + + if (!(reg_data & UVD_DPG_PAUSE__JPEG_PAUSE_DPG_ACK_MASK)) + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_POWER_STATUS, + UVD_POWER_STATUS__UVD_POWER_STATUS_TILES_OFF, + UVD_POWER_STATUS__UVD_POWER_STATUS_MASK, ret_code); + + if (!ret_code) { + /* pause DPG non-jpeg */ + reg_data |= UVD_DPG_PAUSE__NJ_PAUSE_DPG_REQ_MASK; + WREG32_SOC15(UVD, 0, mmUVD_DPG_PAUSE, reg_data); + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_DPG_PAUSE, + UVD_DPG_PAUSE__NJ_PAUSE_DPG_ACK_MASK, + UVD_DPG_PAUSE__NJ_PAUSE_DPG_ACK_MASK, ret_code); + + /* Restore */ + ring = &adev->vcn.ring_enc[0]; + WREG32_SOC15(UVD, 0, mmUVD_RB_BASE_LO, ring->gpu_addr); + WREG32_SOC15(UVD, 0, mmUVD_RB_BASE_HI, upper_32_bits(ring->gpu_addr)); + WREG32_SOC15(UVD, 0, mmUVD_RB_SIZE, ring->ring_size / 4); + WREG32_SOC15(UVD, 0, mmUVD_RB_RPTR, lower_32_bits(ring->wptr)); + WREG32_SOC15(UVD, 0, mmUVD_RB_WPTR, lower_32_bits(ring->wptr)); + + ring = &adev->vcn.ring_enc[1]; + WREG32_SOC15(UVD, 0, mmUVD_RB_BASE_LO2, ring->gpu_addr); + WREG32_SOC15(UVD, 0, mmUVD_RB_BASE_HI2, upper_32_bits(ring->gpu_addr)); + WREG32_SOC15(UVD, 0, mmUVD_RB_SIZE2, ring->ring_size / 4); + WREG32_SOC15(UVD, 0, mmUVD_RB_RPTR2, lower_32_bits(ring->wptr)); + WREG32_SOC15(UVD, 0, mmUVD_RB_WPTR2, lower_32_bits(ring->wptr)); + + ring = &adev->vcn.ring_dec; + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_WPTR, + RREG32_SOC15(UVD, 0, mmUVD_SCRATCH2)); + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_POWER_STATUS, + UVD_PGFSM_CONFIG__UVDM_UVDU_PWR_ON, + UVD_POWER_STATUS__UVD_POWER_STATUS_MASK, ret_code); + } + } else { + /* unpause dpg non-jpeg, no need to wait */ + reg_data &= ~UVD_DPG_PAUSE__NJ_PAUSE_DPG_REQ_MASK; + WREG32_SOC15(UVD, 0, mmUVD_DPG_PAUSE, reg_data); + } + adev->vcn.pause_state.fw_based = new_state->fw_based; + } + + /* pause/unpause if state is changed */ + if (adev->vcn.pause_state.jpeg != new_state->jpeg) { + DRM_DEBUG("dpg pause state changed %d:%d -> %d:%d", + adev->vcn.pause_state.fw_based, adev->vcn.pause_state.jpeg, + new_state->fw_based, new_state->jpeg); + + reg_data = RREG32_SOC15(UVD, 0, mmUVD_DPG_PAUSE) & + (~UVD_DPG_PAUSE__JPEG_PAUSE_DPG_ACK_MASK); + + if (new_state->jpeg == VCN_DPG_STATE__PAUSE) { + ret_code = 0; + + if (!(reg_data & UVD_DPG_PAUSE__NJ_PAUSE_DPG_ACK_MASK)) + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_POWER_STATUS, + UVD_POWER_STATUS__UVD_POWER_STATUS_TILES_OFF, + UVD_POWER_STATUS__UVD_POWER_STATUS_MASK, ret_code); + + if (!ret_code) { + /* Make sure JPRG Snoop is disabled before sending the pause */ + reg_data2 = RREG32_SOC15(UVD, 0, mmUVD_POWER_STATUS); + reg_data2 |= UVD_POWER_STATUS__JRBC_SNOOP_DIS_MASK; + WREG32_SOC15(UVD, 0, mmUVD_POWER_STATUS, reg_data2); + + /* pause DPG jpeg */ + reg_data |= UVD_DPG_PAUSE__JPEG_PAUSE_DPG_REQ_MASK; + WREG32_SOC15(UVD, 0, mmUVD_DPG_PAUSE, reg_data); + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_DPG_PAUSE, + UVD_DPG_PAUSE__JPEG_PAUSE_DPG_ACK_MASK, + UVD_DPG_PAUSE__JPEG_PAUSE_DPG_ACK_MASK, ret_code); + + /* Restore */ + ring = &adev->vcn.ring_jpeg; + WREG32_SOC15(UVD, 0, mmUVD_LMI_JRBC_RB_VMID, 0); + WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_CNTL, + UVD_JRBC_RB_CNTL__RB_NO_FETCH_MASK | + UVD_JRBC_RB_CNTL__RB_RPTR_WR_EN_MASK); + WREG32_SOC15(UVD, 0, mmUVD_LMI_JRBC_RB_64BIT_BAR_LOW, + lower_32_bits(ring->gpu_addr)); + WREG32_SOC15(UVD, 0, mmUVD_LMI_JRBC_RB_64BIT_BAR_HIGH, + upper_32_bits(ring->gpu_addr)); + WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_RPTR, ring->wptr); + WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_WPTR, ring->wptr); + WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_CNTL, + UVD_JRBC_RB_CNTL__RB_RPTR_WR_EN_MASK); + + ring = &adev->vcn.ring_dec; + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_WPTR, + RREG32_SOC15(UVD, 0, mmUVD_SCRATCH2)); + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_POWER_STATUS, + UVD_PGFSM_CONFIG__UVDM_UVDU_PWR_ON, + UVD_POWER_STATUS__UVD_POWER_STATUS_MASK, ret_code); + } + } else { + /* unpause dpg jpeg, no need to wait */ + reg_data &= ~UVD_DPG_PAUSE__JPEG_PAUSE_DPG_REQ_MASK; + WREG32_SOC15(UVD, 0, mmUVD_DPG_PAUSE, reg_data); + } + adev->vcn.pause_state.jpeg = new_state->jpeg; + } + + return 0; +} + static void amdgpu_vcn_idle_work_handler(struct work_struct *work) { struct amdgpu_device *adev = container_of(work, struct amdgpu_device, vcn.idle_work.work); - unsigned fences = amdgpu_fence_count_emitted(&adev->vcn.ring_dec); - unsigned i; + unsigned int fences = 0; + unsigned int i; for (i = 0; i < adev->vcn.num_enc_rings; ++i) { fences += amdgpu_fence_count_emitted(&adev->vcn.ring_enc[i]); } + if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) { + struct dpg_pause_state new_state; + + if (fences) + new_state.fw_based = VCN_DPG_STATE__PAUSE; + else + new_state.fw_based = VCN_DPG_STATE__UNPAUSE; + + if (amdgpu_fence_count_emitted(&adev->vcn.ring_jpeg)) + new_state.jpeg = VCN_DPG_STATE__PAUSE; + else + new_state.jpeg = VCN_DPG_STATE__UNPAUSE; + + amdgpu_vcn_pause_dpg_mode(adev, &new_state); + } + fences += amdgpu_fence_count_emitted(&adev->vcn.ring_jpeg); + fences += amdgpu_fence_count_emitted(&adev->vcn.ring_dec); if (fences == 0) { amdgpu_gfx_off_ctrl(adev, true); @@ -250,6 +393,22 @@ void amdgpu_vcn_ring_begin_use(struct amdgpu_ring *ring) amdgpu_device_ip_set_powergating_state(adev, AMD_IP_BLOCK_TYPE_VCN, AMD_PG_STATE_UNGATE); } + + if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) { + struct dpg_pause_state new_state; + + if (ring->funcs->type == AMDGPU_RING_TYPE_VCN_ENC) + new_state.fw_based = VCN_DPG_STATE__PAUSE; + else + new_state.fw_based = adev->vcn.pause_state.fw_based; + + if (ring->funcs->type == AMDGPU_RING_TYPE_VCN_JPEG) + new_state.jpeg = VCN_DPG_STATE__PAUSE; + else + new_state.jpeg = adev->vcn.pause_state.jpeg; + + amdgpu_vcn_pause_dpg_mode(adev, &new_state); + } } void amdgpu_vcn_ring_end_use(struct amdgpu_ring *ring) @@ -264,7 +423,7 @@ int amdgpu_vcn_dec_ring_test_ring(struct amdgpu_ring *ring) unsigned i; int r; - WREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_CONTEXT_ID), 0xCAFEDEAD); + WREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9), 0xCAFEDEAD); r = amdgpu_ring_alloc(ring, 3); if (r) { DRM_ERROR("amdgpu: cp failed to lock ring %d (%d).\n", @@ -272,11 +431,11 @@ int amdgpu_vcn_dec_ring_test_ring(struct amdgpu_ring *ring) return r; } amdgpu_ring_write(ring, - PACKET0(SOC15_REG_OFFSET(UVD, 0, mmUVD_CONTEXT_ID), 0)); + PACKET0(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9), 0)); amdgpu_ring_write(ring, 0xDEADBEEF); amdgpu_ring_commit(ring); for (i = 0; i < adev->usec_timeout; i++) { - tmp = RREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_CONTEXT_ID)); + tmp = RREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9)); if (tmp == 0xDEADBEEF) break; DRM_UDELAY(1); @@ -616,7 +775,7 @@ int amdgpu_vcn_jpeg_ring_test_ring(struct amdgpu_ring *ring) unsigned i; int r; - WREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_CONTEXT_ID), 0xCAFEDEAD); + WREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9), 0xCAFEDEAD); r = amdgpu_ring_alloc(ring, 3); if (r) { @@ -626,12 +785,12 @@ int amdgpu_vcn_jpeg_ring_test_ring(struct amdgpu_ring *ring) } amdgpu_ring_write(ring, - PACKETJ(SOC15_REG_OFFSET(UVD, 0, mmUVD_CONTEXT_ID), 0, 0, 0)); + PACKETJ(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9), 0, 0, 0)); amdgpu_ring_write(ring, 0xDEADBEEF); amdgpu_ring_commit(ring); for (i = 0; i < adev->usec_timeout; i++) { - tmp = RREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_CONTEXT_ID)); + tmp = RREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9)); if (tmp == 0xDEADBEEF) break; DRM_UDELAY(1); @@ -665,7 +824,7 @@ static int amdgpu_vcn_jpeg_set_reg(struct amdgpu_ring *ring, uint32_t handle, ib = &job->ibs[0]; - ib->ptr[0] = PACKETJ(SOC15_REG_OFFSET(UVD, 0, mmUVD_JPEG_PITCH), 0, 0, PACKETJ_TYPE0); + ib->ptr[0] = PACKETJ(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9), 0, 0, PACKETJ_TYPE0); ib->ptr[1] = 0xDEADBEEF; for (i = 2; i < 16; i += 2) { ib->ptr[i] = PACKETJ(0, 0, 0, PACKETJ_TYPE6); @@ -714,7 +873,7 @@ int amdgpu_vcn_jpeg_ring_test_ib(struct amdgpu_ring *ring, long timeout) r = 0; for (i = 0; i < adev->usec_timeout; i++) { - tmp = RREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_JPEG_PITCH)); + tmp = RREG32(SOC15_REG_OFFSET(UVD, 0, mmUVD_SCRATCH9)); if (tmp == 0xDEADBEEF) break; DRM_UDELAY(1); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h index 0b0b8638d73f..a0ad19af9080 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h @@ -24,9 +24,9 @@ #ifndef __AMDGPU_VCN_H__ #define __AMDGPU_VCN_H__ -#define AMDGPU_VCN_STACK_SIZE (200*1024) -#define AMDGPU_VCN_HEAP_SIZE (256*1024) -#define AMDGPU_VCN_SESSION_SIZE (50*1024) +#define AMDGPU_VCN_STACK_SIZE (128*1024) +#define AMDGPU_VCN_CONTEXT_SIZE (512*1024) + #define AMDGPU_VCN_FIRMWARE_OFFSET 256 #define AMDGPU_VCN_MAX_ENC_RINGS 3 @@ -56,6 +56,16 @@ enum engine_status_constants { UVD_STATUS__RBC_BUSY = 0x1, }; +enum internal_dpg_state { + VCN_DPG_STATE__UNPAUSE = 0, + VCN_DPG_STATE__PAUSE, +}; + +struct dpg_pause_state { + enum internal_dpg_state fw_based; + enum internal_dpg_state jpeg; +}; + struct amdgpu_vcn { struct amdgpu_bo *vcpu_bo; void *cpu_addr; @@ -69,6 +79,8 @@ struct amdgpu_vcn { struct amdgpu_ring ring_jpeg; struct amdgpu_irq_src irq; unsigned num_enc_rings; + enum amd_powergating_state cur_state; + struct dpg_pause_state pause_state; }; int amdgpu_vcn_sw_init(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 6904d794d60a..352b30409060 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -542,7 +542,8 @@ static void amdgpu_vm_pt_next_leaf(struct amdgpu_device *adev, struct amdgpu_vm_pt_cursor *cursor) { amdgpu_vm_pt_next(adev, cursor); - while (amdgpu_vm_pt_descendant(adev, cursor)); + if (cursor->pfn != ~0ll) + while (amdgpu_vm_pt_descendant(adev, cursor)); } /** @@ -3234,8 +3235,10 @@ void amdgpu_vm_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm) } rbtree_postorder_for_each_entry_safe(mapping, tmp, &vm->va.rb_root, rb) { + /* Don't remove the mapping here, we don't want to trigger a + * rebalance and the tree is about to be destroyed anyway. + */ list_del(&mapping->list); - amdgpu_vm_it_remove(mapping, &vm->va); kfree(mapping); } list_for_each_entry_safe(mapping, tmp, &vm->freed, list) { diff --git a/drivers/gpu/drm/amd/amdgpu/ci_dpm.c b/drivers/gpu/drm/amd/amdgpu/ci_dpm.c index d2469453dca2..79220a91abe3 100644 --- a/drivers/gpu/drm/amd/amdgpu/ci_dpm.c +++ b/drivers/gpu/drm/amd/amdgpu/ci_dpm.c @@ -6277,12 +6277,12 @@ static int ci_dpm_sw_init(void *handle) int ret; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - ret = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 230, + ret = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 230, &adev->pm.dpm.thermal.irq); if (ret) return ret; - ret = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 231, + ret = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 231, &adev->pm.dpm.thermal.irq); if (ret) return ret; diff --git a/drivers/gpu/drm/amd/amdgpu/cik.c b/drivers/gpu/drm/amd/amdgpu/cik.c index 78ab939ae5d8..f41f5f57e9f3 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik.c +++ b/drivers/gpu/drm/amd/amdgpu/cik.c @@ -2002,6 +2002,8 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &cik_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v7_0_ip_block); amdgpu_device_ip_block_add(adev, &cik_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v7_2_ip_block); + amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); if (amdgpu_dpm == -1) amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); else @@ -2014,8 +2016,6 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v8_2_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v7_2_ip_block); - amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); amdgpu_device_ip_block_add(adev, &uvd_v4_2_ip_block); amdgpu_device_ip_block_add(adev, &vce_v2_0_ip_block); break; @@ -2023,6 +2023,8 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &cik_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v7_0_ip_block); amdgpu_device_ip_block_add(adev, &cik_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v7_3_ip_block); + amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); if (amdgpu_dpm == -1) amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); else @@ -2035,8 +2037,6 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v8_5_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v7_3_ip_block); - amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); amdgpu_device_ip_block_add(adev, &uvd_v4_2_ip_block); amdgpu_device_ip_block_add(adev, &vce_v2_0_ip_block); break; @@ -2044,6 +2044,8 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &cik_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v7_0_ip_block); amdgpu_device_ip_block_add(adev, &cik_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v7_1_ip_block); + amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); amdgpu_device_ip_block_add(adev, &kv_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -2053,8 +2055,7 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v8_1_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v7_1_ip_block); - amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); + amdgpu_device_ip_block_add(adev, &uvd_v4_2_ip_block); amdgpu_device_ip_block_add(adev, &vce_v2_0_ip_block); break; @@ -2063,6 +2064,8 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &cik_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v7_0_ip_block); amdgpu_device_ip_block_add(adev, &cik_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v7_2_ip_block); + amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); amdgpu_device_ip_block_add(adev, &kv_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -2072,8 +2075,6 @@ int cik_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v8_3_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v7_2_ip_block); - amdgpu_device_ip_block_add(adev, &cik_sdma_ip_block); amdgpu_device_ip_block_add(adev, &uvd_v4_2_ip_block); amdgpu_device_ip_block_add(adev, &vce_v2_0_ip_block); break; diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index 44d10c2172f6..b5775c6a857b 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -276,7 +276,7 @@ static void cik_ih_decode_iv(struct amdgpu_device *adev, dw[2] = le32_to_cpu(adev->irq.ih.ring[ring_index + 2]); dw[3] = le32_to_cpu(adev->irq.ih.ring[ring_index + 3]); - entry->client_id = AMDGPU_IH_CLIENTID_LEGACY; + entry->client_id = AMDGPU_IRQ_CLIENTID_LEGACY; entry->src_id = dw[0] & 0xff; entry->src_data[0] = dw[1] & 0xfffffff; entry->ring_id = dw[2] & 0xff; @@ -318,7 +318,7 @@ static int cik_ih_sw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - r = amdgpu_ih_ring_init(adev, 64 * 1024, false); + r = amdgpu_ih_ring_init(adev, &adev->irq.ih, 64 * 1024, false); if (r) return r; @@ -332,7 +332,7 @@ static int cik_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini(adev); - amdgpu_ih_ring_fini(adev); + amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; @@ -468,8 +468,7 @@ static const struct amdgpu_ih_funcs cik_ih_funcs = { static void cik_ih_set_interrupt_funcs(struct amdgpu_device *adev) { - if (adev->irq.ih_funcs == NULL) - adev->irq.ih_funcs = &cik_ih_funcs; + adev->irq.ih_funcs = &cik_ih_funcs; } const struct amdgpu_ip_block_version cik_ih_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c index 154b1499b07e..b918c8886b75 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c @@ -970,19 +970,19 @@ static int cik_sdma_sw_init(void *handle) } /* SDMA trap event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 224, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 224, &adev->sdma.trap_irq); if (r) return r; /* SDMA Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 241, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 241, &adev->sdma.illegal_inst_irq); if (r) return r; /* SDMA Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 247, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 247, &adev->sdma.illegal_inst_irq); if (r) return r; @@ -1370,10 +1370,8 @@ static const struct amdgpu_buffer_funcs cik_sdma_buffer_funcs = { static void cik_sdma_set_buffer_funcs(struct amdgpu_device *adev) { - if (adev->mman.buffer_funcs == NULL) { - adev->mman.buffer_funcs = &cik_sdma_buffer_funcs; - adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; - } + adev->mman.buffer_funcs = &cik_sdma_buffer_funcs; + adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; } static const struct amdgpu_vm_pte_funcs cik_sdma_vm_pte_funcs = { @@ -1389,15 +1387,13 @@ static void cik_sdma_set_vm_pte_funcs(struct amdgpu_device *adev) struct drm_gpu_scheduler *sched; unsigned i; - if (adev->vm_manager.vm_pte_funcs == NULL) { - adev->vm_manager.vm_pte_funcs = &cik_sdma_vm_pte_funcs; - for (i = 0; i < adev->sdma.num_instances; i++) { - sched = &adev->sdma.instance[i].ring.sched; - adev->vm_manager.vm_pte_rqs[i] = - &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; - } - adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; + adev->vm_manager.vm_pte_funcs = &cik_sdma_vm_pte_funcs; + for (i = 0; i < adev->sdma.num_instances; i++) { + sched = &adev->sdma.instance[i].ring.sched; + adev->vm_manager.vm_pte_rqs[i] = + &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; } + adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; } const struct amdgpu_ip_block_version cik_sdma_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index 960c29e17da6..df5ac4d85a00 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -255,7 +255,7 @@ static void cz_ih_decode_iv(struct amdgpu_device *adev, dw[2] = le32_to_cpu(adev->irq.ih.ring[ring_index + 2]); dw[3] = le32_to_cpu(adev->irq.ih.ring[ring_index + 3]); - entry->client_id = AMDGPU_IH_CLIENTID_LEGACY; + entry->client_id = AMDGPU_IRQ_CLIENTID_LEGACY; entry->src_id = dw[0] & 0xff; entry->src_data[0] = dw[1] & 0xfffffff; entry->ring_id = dw[2] & 0xff; @@ -297,7 +297,7 @@ static int cz_ih_sw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - r = amdgpu_ih_ring_init(adev, 64 * 1024, false); + r = amdgpu_ih_ring_init(adev, &adev->irq.ih, 64 * 1024, false); if (r) return r; @@ -311,7 +311,7 @@ static int cz_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini(adev); - amdgpu_ih_ring_fini(adev); + amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; @@ -449,8 +449,7 @@ static const struct amdgpu_ih_funcs cz_ih_funcs = { static void cz_ih_set_interrupt_funcs(struct amdgpu_device *adev) { - if (adev->irq.ih_funcs == NULL) - adev->irq.ih_funcs = &cz_ih_funcs; + adev->irq.ih_funcs = &cz_ih_funcs; } const struct amdgpu_ip_block_version cz_ih_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c index 3916aa6cc4ec..4cfecdce29a3 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c @@ -2746,19 +2746,19 @@ static int dce_v10_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; for (i = 0; i < adev->mode_info.num_crtc; i++) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); if (r) return r; } for (i = VISLANDS30_IV_SRCID_D1_GRPH_PFLIP; i < 20; i += 2) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i, &adev->pageflip_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i, &adev->pageflip_irq); if (r) return r; } /* HPD hotplug */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_HOTPLUG_DETECT_A, &adev->hpd_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_HOTPLUG_DETECT_A, &adev->hpd_irq); if (r) return r; @@ -3570,8 +3570,7 @@ static const struct amdgpu_display_funcs dce_v10_0_display_funcs = { static void dce_v10_0_set_display_funcs(struct amdgpu_device *adev) { - if (adev->mode_info.funcs == NULL) - adev->mode_info.funcs = &dce_v10_0_display_funcs; + adev->mode_info.funcs = &dce_v10_0_display_funcs; } static const struct amdgpu_irq_src_funcs dce_v10_0_crtc_irq_funcs = { diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c index 4ffb612a4e53..7c868916d90f 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c @@ -2867,19 +2867,19 @@ static int dce_v11_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; for (i = 0; i < adev->mode_info.num_crtc; i++) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); if (r) return r; } for (i = VISLANDS30_IV_SRCID_D1_GRPH_PFLIP; i < 20; i += 2) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i, &adev->pageflip_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i, &adev->pageflip_irq); if (r) return r; } /* HPD hotplug */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_HOTPLUG_DETECT_A, &adev->hpd_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_HOTPLUG_DETECT_A, &adev->hpd_irq); if (r) return r; @@ -3702,8 +3702,7 @@ static const struct amdgpu_display_funcs dce_v11_0_display_funcs = { static void dce_v11_0_set_display_funcs(struct amdgpu_device *adev) { - if (adev->mode_info.funcs == NULL) - adev->mode_info.funcs = &dce_v11_0_display_funcs; + adev->mode_info.funcs = &dce_v11_0_display_funcs; } static const struct amdgpu_irq_src_funcs dce_v11_0_crtc_irq_funcs = { diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c index 480c5348a14f..17eaaba36017 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c @@ -2616,19 +2616,19 @@ static int dce_v6_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; for (i = 0; i < adev->mode_info.num_crtc; i++) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); if (r) return r; } for (i = 8; i < 20; i += 2) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i, &adev->pageflip_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i, &adev->pageflip_irq); if (r) return r; } /* HPD hotplug */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 42, &adev->hpd_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 42, &adev->hpd_irq); if (r) return r; @@ -3376,8 +3376,7 @@ static const struct amdgpu_display_funcs dce_v6_0_display_funcs = { static void dce_v6_0_set_display_funcs(struct amdgpu_device *adev) { - if (adev->mode_info.funcs == NULL) - adev->mode_info.funcs = &dce_v6_0_display_funcs; + adev->mode_info.funcs = &dce_v6_0_display_funcs; } static const struct amdgpu_irq_src_funcs dce_v6_0_crtc_irq_funcs = { diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c index 797196476c94..8c0576978d36 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c @@ -2643,19 +2643,19 @@ static int dce_v8_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; for (i = 0; i < adev->mode_info.num_crtc; i++) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i + 1, &adev->crtc_irq); if (r) return r; } for (i = 8; i < 20; i += 2) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i, &adev->pageflip_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i, &adev->pageflip_irq); if (r) return r; } /* HPD hotplug */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 42, &adev->hpd_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 42, &adev->hpd_irq); if (r) return r; @@ -3458,8 +3458,7 @@ static const struct amdgpu_display_funcs dce_v8_0_display_funcs = { static void dce_v8_0_set_display_funcs(struct amdgpu_device *adev) { - if (adev->mode_info.funcs == NULL) - adev->mode_info.funcs = &dce_v8_0_display_funcs; + adev->mode_info.funcs = &dce_v8_0_display_funcs; } static const struct amdgpu_irq_src_funcs dce_v8_0_crtc_irq_funcs = { diff --git a/drivers/gpu/drm/amd/amdgpu/dce_virtual.c b/drivers/gpu/drm/amd/amdgpu/dce_virtual.c index 15257634a53a..fdace004544d 100644 --- a/drivers/gpu/drm/amd/amdgpu/dce_virtual.c +++ b/drivers/gpu/drm/amd/amdgpu/dce_virtual.c @@ -372,7 +372,7 @@ static int dce_virtual_sw_init(void *handle) int r, i; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SMU_DISP_TIMER2_TRIGGER, &adev->crtc_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SMU_DISP_TIMER2_TRIGGER, &adev->crtc_irq); if (r) return r; @@ -649,8 +649,7 @@ static const struct amdgpu_display_funcs dce_virtual_display_funcs = { static void dce_virtual_set_display_funcs(struct amdgpu_device *adev) { - if (adev->mode_info.funcs == NULL) - adev->mode_info.funcs = &dce_virtual_display_funcs; + adev->mode_info.funcs = &dce_virtual_display_funcs; } static int dce_virtual_pageflip(struct amdgpu_device *adev, diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c index de184a886057..d76eb27945dc 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c @@ -1552,7 +1552,7 @@ static void gfx_v6_0_config_init(struct amdgpu_device *adev) adev->gfx.config.double_offchip_lds_buf = 0; } -static void gfx_v6_0_gpu_init(struct amdgpu_device *adev) +static void gfx_v6_0_constants_init(struct amdgpu_device *adev) { u32 gb_addr_config = 0; u32 mc_shared_chmap, mc_arb_ramcfg; @@ -3094,15 +3094,15 @@ static int gfx_v6_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; int i, r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 181, &adev->gfx.eop_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 181, &adev->gfx.eop_irq); if (r) return r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 184, &adev->gfx.priv_reg_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 184, &adev->gfx.priv_reg_irq); if (r) return r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 185, &adev->gfx.priv_inst_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 185, &adev->gfx.priv_inst_irq); if (r) return r; @@ -3175,7 +3175,7 @@ static int gfx_v6_0_hw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - gfx_v6_0_gpu_init(adev); + gfx_v6_0_constants_init(adev); r = gfx_v6_0_rlc_resume(adev); if (r) diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c index fc39ebbc9d9f..0e72bc09939a 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c @@ -1886,14 +1886,14 @@ static void gfx_v7_0_config_init(struct amdgpu_device *adev) } /** - * gfx_v7_0_gpu_init - setup the 3D engine + * gfx_v7_0_constants_init - setup the 3D engine * * @adev: amdgpu_device pointer * - * Configures the 3D engine and tiling configuration - * registers so that the 3D engine is usable. + * init the gfx constants such as the 3D engine, tiling configuration + * registers, maximum number of quad pipes, render backends... */ -static void gfx_v7_0_gpu_init(struct amdgpu_device *adev) +static void gfx_v7_0_constants_init(struct amdgpu_device *adev) { u32 sh_mem_cfg, sh_static_mem_cfg, sh_mem_base; u32 tmp; @@ -4516,18 +4516,18 @@ static int gfx_v7_0_sw_init(void *handle) adev->gfx.mec.num_queue_per_pipe = 8; /* EOP Event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 181, &adev->gfx.eop_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 181, &adev->gfx.eop_irq); if (r) return r; /* Privileged reg */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 184, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 184, &adev->gfx.priv_reg_irq); if (r) return r; /* Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 185, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 185, &adev->gfx.priv_inst_irq); if (r) return r; @@ -4624,7 +4624,7 @@ static int gfx_v7_0_hw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - gfx_v7_0_gpu_init(adev); + gfx_v7_0_constants_init(adev); /* init rlc */ r = gfx_v7_0_rlc_resume(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c index 470dc80f4fe7..617b0c8908a3 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c @@ -1173,64 +1173,61 @@ static int gfx_v8_0_init_microcode(struct amdgpu_device *adev) } } - if (adev->firmware.load_type == AMDGPU_FW_LOAD_SMU) { - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_PFP]; - info->ucode_id = AMDGPU_UCODE_ID_CP_PFP; - info->fw = adev->gfx.pfp_fw; - header = (const struct common_firmware_header *)info->fw->data; - adev->firmware.fw_size += - ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); - - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_ME]; - info->ucode_id = AMDGPU_UCODE_ID_CP_ME; - info->fw = adev->gfx.me_fw; - header = (const struct common_firmware_header *)info->fw->data; - adev->firmware.fw_size += - ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); - - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_CE]; - info->ucode_id = AMDGPU_UCODE_ID_CP_CE; - info->fw = adev->gfx.ce_fw; - header = (const struct common_firmware_header *)info->fw->data; - adev->firmware.fw_size += - ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_PFP]; + info->ucode_id = AMDGPU_UCODE_ID_CP_PFP; + info->fw = adev->gfx.pfp_fw; + header = (const struct common_firmware_header *)info->fw->data; + adev->firmware.fw_size += + ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_ME]; + info->ucode_id = AMDGPU_UCODE_ID_CP_ME; + info->fw = adev->gfx.me_fw; + header = (const struct common_firmware_header *)info->fw->data; + adev->firmware.fw_size += + ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_CE]; + info->ucode_id = AMDGPU_UCODE_ID_CP_CE; + info->fw = adev->gfx.ce_fw; + header = (const struct common_firmware_header *)info->fw->data; + adev->firmware.fw_size += + ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_RLC_G]; + info->ucode_id = AMDGPU_UCODE_ID_RLC_G; + info->fw = adev->gfx.rlc_fw; + header = (const struct common_firmware_header *)info->fw->data; + adev->firmware.fw_size += + ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_MEC1]; + info->ucode_id = AMDGPU_UCODE_ID_CP_MEC1; + info->fw = adev->gfx.mec_fw; + header = (const struct common_firmware_header *)info->fw->data; + adev->firmware.fw_size += + ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + + /* we need account JT in */ + cp_hdr = (const struct gfx_firmware_header_v1_0 *)adev->gfx.mec_fw->data; + adev->firmware.fw_size += + ALIGN(le32_to_cpu(cp_hdr->jt_size) << 2, PAGE_SIZE); - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_RLC_G]; - info->ucode_id = AMDGPU_UCODE_ID_RLC_G; - info->fw = adev->gfx.rlc_fw; - header = (const struct common_firmware_header *)info->fw->data; + if (amdgpu_sriov_vf(adev)) { + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_STORAGE]; + info->ucode_id = AMDGPU_UCODE_ID_STORAGE; + info->fw = adev->gfx.mec_fw; adev->firmware.fw_size += - ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + ALIGN(le32_to_cpu(64 * PAGE_SIZE), PAGE_SIZE); + } - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_MEC1]; - info->ucode_id = AMDGPU_UCODE_ID_CP_MEC1; - info->fw = adev->gfx.mec_fw; + if (adev->gfx.mec2_fw) { + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_MEC2]; + info->ucode_id = AMDGPU_UCODE_ID_CP_MEC2; + info->fw = adev->gfx.mec2_fw; header = (const struct common_firmware_header *)info->fw->data; adev->firmware.fw_size += ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); - - /* we need account JT in */ - cp_hdr = (const struct gfx_firmware_header_v1_0 *)adev->gfx.mec_fw->data; - adev->firmware.fw_size += - ALIGN(le32_to_cpu(cp_hdr->jt_size) << 2, PAGE_SIZE); - - if (amdgpu_sriov_vf(adev)) { - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_STORAGE]; - info->ucode_id = AMDGPU_UCODE_ID_STORAGE; - info->fw = adev->gfx.mec_fw; - adev->firmware.fw_size += - ALIGN(le32_to_cpu(64 * PAGE_SIZE), PAGE_SIZE); - } - - if (adev->gfx.mec2_fw) { - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_MEC2]; - info->ucode_id = AMDGPU_UCODE_ID_CP_MEC2; - info->fw = adev->gfx.mec2_fw; - header = (const struct common_firmware_header *)info->fw->data; - adev->firmware.fw_size += - ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); - } - } out: @@ -2048,36 +2045,31 @@ static int gfx_v8_0_sw_init(void *handle) adev->gfx.mec.num_pipe_per_mec = 4; adev->gfx.mec.num_queue_per_pipe = 8; - /* KIQ event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_INT_IB2, &adev->gfx.kiq.irq); - if (r) - return r; - /* EOP Event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_END_OF_PIPE, &adev->gfx.eop_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_END_OF_PIPE, &adev->gfx.eop_irq); if (r) return r; /* Privileged reg */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_PRIV_REG_FAULT, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_PRIV_REG_FAULT, &adev->gfx.priv_reg_irq); if (r) return r; /* Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_PRIV_INSTR_FAULT, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_PRIV_INSTR_FAULT, &adev->gfx.priv_inst_irq); if (r) return r; /* Add CP EDC/ECC irq */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_ECC_ERROR, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CP_ECC_ERROR, &adev->gfx.cp_ecc_error_irq); if (r) return r; /* SQ interrupts. */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SQ_INTERRUPT_MSG, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SQ_INTERRUPT_MSG, &adev->gfx.sq_irq); if (r) { DRM_ERROR("amdgpu_irq_add() for SQ failed: %d\n", r); @@ -3835,7 +3827,7 @@ static void gfx_v8_0_config_init(struct amdgpu_device *adev) } } -static void gfx_v8_0_gpu_init(struct amdgpu_device *adev) +static void gfx_v8_0_constants_init(struct amdgpu_device *adev) { u32 tmp, sh_static_mem_cfg; int i; @@ -4181,65 +4173,11 @@ static void gfx_v8_0_rlc_start(struct amdgpu_device *adev) udelay(50); } -static int gfx_v8_0_rlc_load_microcode(struct amdgpu_device *adev) -{ - const struct rlc_firmware_header_v2_0 *hdr; - const __le32 *fw_data; - unsigned i, fw_size; - - if (!adev->gfx.rlc_fw) - return -EINVAL; - - hdr = (const struct rlc_firmware_header_v2_0 *)adev->gfx.rlc_fw->data; - amdgpu_ucode_print_rlc_hdr(&hdr->header); - - fw_data = (const __le32 *)(adev->gfx.rlc_fw->data + - le32_to_cpu(hdr->header.ucode_array_offset_bytes)); - fw_size = le32_to_cpu(hdr->header.ucode_size_bytes) / 4; - - WREG32(mmRLC_GPM_UCODE_ADDR, 0); - for (i = 0; i < fw_size; i++) - WREG32(mmRLC_GPM_UCODE_DATA, le32_to_cpup(fw_data++)); - WREG32(mmRLC_GPM_UCODE_ADDR, adev->gfx.rlc_fw_version); - - return 0; -} - static int gfx_v8_0_rlc_resume(struct amdgpu_device *adev) { - int r; - u32 tmp; - gfx_v8_0_rlc_stop(adev); - - /* disable CG */ - tmp = RREG32(mmRLC_CGCG_CGLS_CTRL); - tmp &= ~(RLC_CGCG_CGLS_CTRL__CGCG_EN_MASK | - RLC_CGCG_CGLS_CTRL__CGLS_EN_MASK); - WREG32(mmRLC_CGCG_CGLS_CTRL, tmp); - if (adev->asic_type == CHIP_POLARIS11 || - adev->asic_type == CHIP_POLARIS10 || - adev->asic_type == CHIP_POLARIS12 || - adev->asic_type == CHIP_VEGAM) { - tmp = RREG32(mmRLC_CGCG_CGLS_CTRL_3D); - tmp &= ~0x3; - WREG32(mmRLC_CGCG_CGLS_CTRL_3D, tmp); - } - - /* disable PG */ - WREG32(mmRLC_PG_CNTL, 0); - gfx_v8_0_rlc_reset(adev); gfx_v8_0_init_pg(adev); - - - if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { - /* legacy rlc firmware loading */ - r = gfx_v8_0_rlc_load_microcode(adev); - if (r) - return r; - } - gfx_v8_0_rlc_start(adev); return 0; @@ -4265,63 +4203,6 @@ static void gfx_v8_0_cp_gfx_enable(struct amdgpu_device *adev, bool enable) udelay(50); } -static int gfx_v8_0_cp_gfx_load_microcode(struct amdgpu_device *adev) -{ - const struct gfx_firmware_header_v1_0 *pfp_hdr; - const struct gfx_firmware_header_v1_0 *ce_hdr; - const struct gfx_firmware_header_v1_0 *me_hdr; - const __le32 *fw_data; - unsigned i, fw_size; - - if (!adev->gfx.me_fw || !adev->gfx.pfp_fw || !adev->gfx.ce_fw) - return -EINVAL; - - pfp_hdr = (const struct gfx_firmware_header_v1_0 *) - adev->gfx.pfp_fw->data; - ce_hdr = (const struct gfx_firmware_header_v1_0 *) - adev->gfx.ce_fw->data; - me_hdr = (const struct gfx_firmware_header_v1_0 *) - adev->gfx.me_fw->data; - - amdgpu_ucode_print_gfx_hdr(&pfp_hdr->header); - amdgpu_ucode_print_gfx_hdr(&ce_hdr->header); - amdgpu_ucode_print_gfx_hdr(&me_hdr->header); - - gfx_v8_0_cp_gfx_enable(adev, false); - - /* PFP */ - fw_data = (const __le32 *) - (adev->gfx.pfp_fw->data + - le32_to_cpu(pfp_hdr->header.ucode_array_offset_bytes)); - fw_size = le32_to_cpu(pfp_hdr->header.ucode_size_bytes) / 4; - WREG32(mmCP_PFP_UCODE_ADDR, 0); - for (i = 0; i < fw_size; i++) - WREG32(mmCP_PFP_UCODE_DATA, le32_to_cpup(fw_data++)); - WREG32(mmCP_PFP_UCODE_ADDR, adev->gfx.pfp_fw_version); - - /* CE */ - fw_data = (const __le32 *) - (adev->gfx.ce_fw->data + - le32_to_cpu(ce_hdr->header.ucode_array_offset_bytes)); - fw_size = le32_to_cpu(ce_hdr->header.ucode_size_bytes) / 4; - WREG32(mmCP_CE_UCODE_ADDR, 0); - for (i = 0; i < fw_size; i++) - WREG32(mmCP_CE_UCODE_DATA, le32_to_cpup(fw_data++)); - WREG32(mmCP_CE_UCODE_ADDR, adev->gfx.ce_fw_version); - - /* ME */ - fw_data = (const __le32 *) - (adev->gfx.me_fw->data + - le32_to_cpu(me_hdr->header.ucode_array_offset_bytes)); - fw_size = le32_to_cpu(me_hdr->header.ucode_size_bytes) / 4; - WREG32(mmCP_ME_RAM_WADDR, 0); - for (i = 0; i < fw_size; i++) - WREG32(mmCP_ME_RAM_DATA, le32_to_cpup(fw_data++)); - WREG32(mmCP_ME_RAM_WADDR, adev->gfx.me_fw_version); - - return 0; -} - static u32 gfx_v8_0_get_csb_size(struct amdgpu_device *adev) { u32 count = 0; @@ -4521,52 +4402,6 @@ static void gfx_v8_0_cp_compute_enable(struct amdgpu_device *adev, bool enable) udelay(50); } -static int gfx_v8_0_cp_compute_load_microcode(struct amdgpu_device *adev) -{ - const struct gfx_firmware_header_v1_0 *mec_hdr; - const __le32 *fw_data; - unsigned i, fw_size; - - if (!adev->gfx.mec_fw) - return -EINVAL; - - gfx_v8_0_cp_compute_enable(adev, false); - - mec_hdr = (const struct gfx_firmware_header_v1_0 *)adev->gfx.mec_fw->data; - amdgpu_ucode_print_gfx_hdr(&mec_hdr->header); - - fw_data = (const __le32 *) - (adev->gfx.mec_fw->data + - le32_to_cpu(mec_hdr->header.ucode_array_offset_bytes)); - fw_size = le32_to_cpu(mec_hdr->header.ucode_size_bytes) / 4; - - /* MEC1 */ - WREG32(mmCP_MEC_ME1_UCODE_ADDR, 0); - for (i = 0; i < fw_size; i++) - WREG32(mmCP_MEC_ME1_UCODE_DATA, le32_to_cpup(fw_data+i)); - WREG32(mmCP_MEC_ME1_UCODE_ADDR, adev->gfx.mec_fw_version); - - /* Loading MEC2 firmware is only necessary if MEC2 should run different microcode than MEC1. */ - if (adev->gfx.mec2_fw) { - const struct gfx_firmware_header_v1_0 *mec2_hdr; - - mec2_hdr = (const struct gfx_firmware_header_v1_0 *)adev->gfx.mec2_fw->data; - amdgpu_ucode_print_gfx_hdr(&mec2_hdr->header); - - fw_data = (const __le32 *) - (adev->gfx.mec2_fw->data + - le32_to_cpu(mec2_hdr->header.ucode_array_offset_bytes)); - fw_size = le32_to_cpu(mec2_hdr->header.ucode_size_bytes) / 4; - - WREG32(mmCP_MEC_ME2_UCODE_ADDR, 0); - for (i = 0; i < fw_size; i++) - WREG32(mmCP_MEC_ME2_UCODE_DATA, le32_to_cpup(fw_data+i)); - WREG32(mmCP_MEC_ME2_UCODE_ADDR, adev->gfx.mec2_fw_version); - } - - return 0; -} - /* KIQ functions */ static void gfx_v8_0_kiq_setting(struct amdgpu_ring *ring) { @@ -4892,7 +4727,7 @@ static int gfx_v8_0_kcq_init_queue(struct amdgpu_ring *ring) struct vi_mqd *mqd = ring->mqd_ptr; int mqd_idx = ring - &adev->gfx.compute_ring[0]; - if (!adev->in_gpu_reset && !adev->gfx.in_suspend) { + if (!adev->in_gpu_reset && !adev->in_suspend) { memset((void *)mqd, 0, sizeof(struct vi_mqd_allocation)); ((struct vi_mqd_allocation *)mqd)->dynamic_cu_mask = 0xFFFFFFFF; ((struct vi_mqd_allocation *)mqd)->dynamic_rb_mask = 0xFFFFFFFF; @@ -4980,8 +4815,10 @@ static int gfx_v8_0_kcq_resume(struct amdgpu_device *adev) if (r) goto done; - /* Test KCQs */ - for (i = 0; i < adev->gfx.num_compute_rings; i++) { + /* Test KCQs - reversing the order of rings seems to fix ring test failure + * after GPU reset + */ + for (i = adev->gfx.num_compute_rings - 1; i >= 0; i--) { ring = &adev->gfx.compute_ring[i]; ring->ready = true; r = amdgpu_ring_test_ring(ring); @@ -5000,17 +4837,6 @@ static int gfx_v8_0_cp_resume(struct amdgpu_device *adev) if (!(adev->flags & AMD_IS_APU)) gfx_v8_0_enable_gui_idle_interrupt(adev, false); - if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { - /* legacy firmware loading */ - r = gfx_v8_0_cp_gfx_load_microcode(adev); - if (r) - return r; - - r = gfx_v8_0_cp_compute_load_microcode(adev); - if (r) - return r; - } - r = gfx_v8_0_kiq_resume(adev); if (r) return r; @@ -5039,7 +4865,7 @@ static int gfx_v8_0_hw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; gfx_v8_0_init_golden_registers(adev); - gfx_v8_0_gpu_init(adev); + gfx_v8_0_constants_init(adev); r = gfx_v8_0_rlc_resume(adev); if (r) @@ -5080,6 +4906,55 @@ static int gfx_v8_0_kcq_disable(struct amdgpu_device *adev) return r; } +static bool gfx_v8_0_is_idle(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + if (REG_GET_FIELD(RREG32(mmGRBM_STATUS), GRBM_STATUS, GUI_ACTIVE) + || RREG32(mmGRBM_STATUS2) != 0x8) + return false; + else + return true; +} + +static bool gfx_v8_0_rlc_is_idle(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + if (RREG32(mmGRBM_STATUS2) != 0x8) + return false; + else + return true; +} + +static int gfx_v8_0_wait_for_rlc_idle(void *handle) +{ + unsigned int i; + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + for (i = 0; i < adev->usec_timeout; i++) { + if (gfx_v8_0_rlc_is_idle(handle)) + return 0; + + udelay(1); + } + return -ETIMEDOUT; +} + +static int gfx_v8_0_wait_for_idle(void *handle) +{ + unsigned int i; + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + for (i = 0; i < adev->usec_timeout; i++) { + if (gfx_v8_0_is_idle(handle)) + return 0; + + udelay(1); + } + return -ETIMEDOUT; +} + static int gfx_v8_0_hw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; @@ -5098,51 +4973,27 @@ static int gfx_v8_0_hw_fini(void *handle) pr_debug("For SRIOV client, shouldn't do anything.\n"); return 0; } - gfx_v8_0_cp_enable(adev, false); - gfx_v8_0_rlc_stop(adev); - + adev->gfx.rlc.funcs->enter_safe_mode(adev); + if (!gfx_v8_0_wait_for_idle(adev)) + gfx_v8_0_cp_enable(adev, false); + else + pr_err("cp is busy, skip halt cp\n"); + if (!gfx_v8_0_wait_for_rlc_idle(adev)) + gfx_v8_0_rlc_stop(adev); + else + pr_err("rlc is busy, skip halt rlc\n"); + adev->gfx.rlc.funcs->exit_safe_mode(adev); return 0; } static int gfx_v8_0_suspend(void *handle) { - struct amdgpu_device *adev = (struct amdgpu_device *)handle; - adev->gfx.in_suspend = true; - return gfx_v8_0_hw_fini(adev); + return gfx_v8_0_hw_fini(handle); } static int gfx_v8_0_resume(void *handle) { - int r; - struct amdgpu_device *adev = (struct amdgpu_device *)handle; - - r = gfx_v8_0_hw_init(adev); - adev->gfx.in_suspend = false; - return r; -} - -static bool gfx_v8_0_is_idle(void *handle) -{ - struct amdgpu_device *adev = (struct amdgpu_device *)handle; - - if (REG_GET_FIELD(RREG32(mmGRBM_STATUS), GRBM_STATUS, GUI_ACTIVE)) - return false; - else - return true; -} - -static int gfx_v8_0_wait_for_idle(void *handle) -{ - unsigned i; - struct amdgpu_device *adev = (struct amdgpu_device *)handle; - - for (i = 0; i < adev->usec_timeout; i++) { - if (gfx_v8_0_is_idle(handle)) - return 0; - - udelay(1); - } - return -ETIMEDOUT; + return gfx_v8_0_hw_init(handle); } static bool gfx_v8_0_check_soft_reset(void *handle) @@ -7013,52 +6864,6 @@ static int gfx_v8_0_sq_irq(struct amdgpu_device *adev, return 0; } -static int gfx_v8_0_kiq_set_interrupt_state(struct amdgpu_device *adev, - struct amdgpu_irq_src *src, - unsigned int type, - enum amdgpu_interrupt_state state) -{ - struct amdgpu_ring *ring = &(adev->gfx.kiq.ring); - - switch (type) { - case AMDGPU_CP_KIQ_IRQ_DRIVER0: - WREG32_FIELD(CPC_INT_CNTL, GENERIC2_INT_ENABLE, - state == AMDGPU_IRQ_STATE_DISABLE ? 0 : 1); - if (ring->me == 1) - WREG32_FIELD_OFFSET(CP_ME1_PIPE0_INT_CNTL, - ring->pipe, - GENERIC2_INT_ENABLE, - state == AMDGPU_IRQ_STATE_DISABLE ? 0 : 1); - else - WREG32_FIELD_OFFSET(CP_ME2_PIPE0_INT_CNTL, - ring->pipe, - GENERIC2_INT_ENABLE, - state == AMDGPU_IRQ_STATE_DISABLE ? 0 : 1); - break; - default: - BUG(); /* kiq only support GENERIC2_INT now */ - break; - } - return 0; -} - -static int gfx_v8_0_kiq_irq(struct amdgpu_device *adev, - struct amdgpu_irq_src *source, - struct amdgpu_iv_entry *entry) -{ - u8 me_id, pipe_id, queue_id; - struct amdgpu_ring *ring = &(adev->gfx.kiq.ring); - - me_id = (entry->ring_id & 0x0c) >> 2; - pipe_id = (entry->ring_id & 0x03) >> 0; - queue_id = (entry->ring_id & 0x70) >> 4; - DRM_DEBUG("IH: CPC GENERIC2_INT, me:%d, pipe:%d, queue:%d\n", - me_id, pipe_id, queue_id); - - amdgpu_fence_process(ring); - return 0; -} - static const struct amd_ip_funcs gfx_v8_0_ip_funcs = { .name = "gfx_v8_0", .early_init = gfx_v8_0_early_init, @@ -7209,11 +7014,6 @@ static const struct amdgpu_irq_src_funcs gfx_v8_0_priv_inst_irq_funcs = { .process = gfx_v8_0_priv_inst_irq, }; -static const struct amdgpu_irq_src_funcs gfx_v8_0_kiq_irq_funcs = { - .set = gfx_v8_0_kiq_set_interrupt_state, - .process = gfx_v8_0_kiq_irq, -}; - static const struct amdgpu_irq_src_funcs gfx_v8_0_cp_ecc_error_irq_funcs = { .set = gfx_v8_0_set_cp_ecc_int_state, .process = gfx_v8_0_cp_ecc_error_irq, @@ -7235,9 +7035,6 @@ static void gfx_v8_0_set_irq_funcs(struct amdgpu_device *adev) adev->gfx.priv_inst_irq.num_types = 1; adev->gfx.priv_inst_irq.funcs = &gfx_v8_0_priv_inst_irq_funcs; - adev->gfx.kiq.irq.num_types = AMDGPU_CP_KIQ_IRQ_LAST; - adev->gfx.kiq.irq.funcs = &gfx_v8_0_kiq_irq_funcs; - adev->gfx.cp_ecc_error_irq.num_types = 1; adev->gfx.cp_ecc_error_irq.funcs = &gfx_v8_0_cp_ecc_error_irq_funcs; diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c index f369d9603435..6d7baf59d6e1 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c @@ -97,6 +97,7 @@ MODULE_FIRMWARE("amdgpu/raven2_rlc.bin"); static const struct soc15_reg_golden golden_settings_gc_9_0[] = { SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG2, 0xf00fffff, 0x00000400), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0x80000000, 0x80000000), SOC15_REG_GOLDEN_VALUE(GC, 0, mmGB_GPU_ID, 0x0000000f, 0x00000000), SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_BINNER_EVENT_CNTL_3, 0x00000003, 0x82400024), SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_ENHANCE, 0x3fffffff, 0x00000001), @@ -133,7 +134,10 @@ static const struct soc15_reg_golden golden_settings_gc_9_0_vg10[] = SOC15_REG_GOLDEN_VALUE(GC, 0, mmRMI_UTCL1_CNTL2, 0x00030000, 0x00020000), SOC15_REG_GOLDEN_VALUE(GC, 0, mmSPI_CONFIG_CNTL_1, 0x0000000f, 0x01000107), SOC15_REG_GOLDEN_VALUE(GC, 0, mmTD_CNTL, 0x00001800, 0x00000800), - SOC15_REG_GOLDEN_VALUE(GC, 0, mmWD_UTCL1_CNTL, 0x08000000, 0x08000080) + SOC15_REG_GOLDEN_VALUE(GC, 0, mmWD_UTCL1_CNTL, 0x08000000, 0x08000080), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_MEC1_F32_INT_DIS, 0x00000000, 0x00000800), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_MEC2_F32_INT_DIS, 0x00000000, 0x00000800), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_DEBUG, 0x00000000, 0x00008000) }; static const struct soc15_reg_golden golden_settings_gc_9_0_vg20[] = @@ -173,7 +177,10 @@ static const struct soc15_reg_golden golden_settings_gc_9_1[] = SOC15_REG_GOLDEN_VALUE(GC, 0, mmTCP_CHAN_STEER_LO, 0xffffffff, 0x00003120), SOC15_REG_GOLDEN_VALUE(GC, 0, mmVGT_CACHE_INVALIDATION, 0x3fff3af3, 0x19200000), SOC15_REG_GOLDEN_VALUE(GC, 0, mmVGT_GS_MAX_WAVE_ID, 0x00000fff, 0x000000ff), - SOC15_REG_GOLDEN_VALUE(GC, 0, mmWD_UTCL1_CNTL, 0x08000000, 0x08000080) + SOC15_REG_GOLDEN_VALUE(GC, 0, mmWD_UTCL1_CNTL, 0x08000000, 0x08000080), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_MEC1_F32_INT_DIS, 0x00000000, 0x00000800), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_MEC2_F32_INT_DIS, 0x00000000, 0x00000800), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_DEBUG, 0x00000000, 0x00008000) }; static const struct soc15_reg_golden golden_settings_gc_9_1_rv1[] = @@ -247,7 +254,10 @@ static const struct soc15_reg_golden golden_settings_gc_9_2_1_vg12[] = SOC15_REG_GOLDEN_VALUE(GC, 0, mmSPI_CONFIG_CNTL_1, 0xffff03ff, 0x01000107), SOC15_REG_GOLDEN_VALUE(GC, 0, mmTCP_CHAN_STEER_HI, 0xffffffff, 0x00000000), SOC15_REG_GOLDEN_VALUE(GC, 0, mmTCP_CHAN_STEER_LO, 0xffffffff, 0x76325410), - SOC15_REG_GOLDEN_VALUE(GC, 0, mmTD_CNTL, 0x01bd9f33, 0x01000000) + SOC15_REG_GOLDEN_VALUE(GC, 0, mmTD_CNTL, 0x01bd9f33, 0x01000000), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_MEC1_F32_INT_DIS, 0x00000000, 0x00000800), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_MEC2_F32_INT_DIS, 0x00000000, 0x00000800), + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCP_DEBUG, 0x00000000, 0x00008000) }; static const u32 GFX_RLC_SRM_INDEX_CNTL_ADDR_OFFSETS[] = @@ -908,6 +918,50 @@ static void gfx_v9_0_get_csb_buffer(struct amdgpu_device *adev, buffer[count++] = cpu_to_le32(0); } +static void gfx_v9_0_init_always_on_cu_mask(struct amdgpu_device *adev) +{ + struct amdgpu_cu_info *cu_info = &adev->gfx.cu_info; + uint32_t pg_always_on_cu_num = 2; + uint32_t always_on_cu_num; + uint32_t i, j, k; + uint32_t mask, cu_bitmap, counter; + + if (adev->flags & AMD_IS_APU) + always_on_cu_num = 4; + else if (adev->asic_type == CHIP_VEGA12) + always_on_cu_num = 8; + else + always_on_cu_num = 12; + + mutex_lock(&adev->grbm_idx_mutex); + for (i = 0; i < adev->gfx.config.max_shader_engines; i++) { + for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { + mask = 1; + cu_bitmap = 0; + counter = 0; + gfx_v9_0_select_se_sh(adev, i, j, 0xffffffff); + + for (k = 0; k < adev->gfx.config.max_cu_per_sh; k ++) { + if (cu_info->bitmap[i][j] & mask) { + if (counter == pg_always_on_cu_num) + WREG32_SOC15(GC, 0, mmRLC_PG_ALWAYS_ON_CU_MASK, cu_bitmap); + if (counter < always_on_cu_num) + cu_bitmap |= mask; + else + break; + counter++; + } + mask <<= 1; + } + + WREG32_SOC15(GC, 0, mmRLC_LB_ALWAYS_ACTIVE_CU_MASK, cu_bitmap); + cu_info->ao_cu_bitmap[i][j] = cu_bitmap; + } + } + gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff); + mutex_unlock(&adev->grbm_idx_mutex); +} + static void gfx_v9_0_init_lbpw(struct amdgpu_device *adev) { uint32_t data; @@ -941,8 +995,59 @@ static void gfx_v9_0_init_lbpw(struct amdgpu_device *adev) data |= 0x00C00000; WREG32_SOC15(GC, 0, mmRLC_GPM_GENERAL_7, data); - /* set RLC_LB_ALWAYS_ACTIVE_CU_MASK = 0xFFF */ - WREG32_SOC15(GC, 0, mmRLC_LB_ALWAYS_ACTIVE_CU_MASK, 0xFFF); + /* + * RLC_LB_ALWAYS_ACTIVE_CU_MASK = 0xF (4 CUs AON for Raven), + * programmed in gfx_v9_0_init_always_on_cu_mask() + */ + + /* set RLC_LB_CNTL = 0x8000_0095, 31 bit is reserved, + * but used for RLC_LB_CNTL configuration */ + data = RLC_LB_CNTL__LB_CNT_SPIM_ACTIVE_MASK; + data |= REG_SET_FIELD(data, RLC_LB_CNTL, CU_MASK_USED_OFF_HYST, 0x09); + data |= REG_SET_FIELD(data, RLC_LB_CNTL, RESERVED, 0x80000); + WREG32_SOC15(GC, 0, mmRLC_LB_CNTL, data); + mutex_unlock(&adev->grbm_idx_mutex); + + gfx_v9_0_init_always_on_cu_mask(adev); +} + +static void gfx_v9_4_init_lbpw(struct amdgpu_device *adev) +{ + uint32_t data; + + /* set mmRLC_LB_THR_CONFIG_1/2/3/4 */ + WREG32_SOC15(GC, 0, mmRLC_LB_THR_CONFIG_1, 0x0000007F); + WREG32_SOC15(GC, 0, mmRLC_LB_THR_CONFIG_2, 0x033388F8); + WREG32_SOC15(GC, 0, mmRLC_LB_THR_CONFIG_3, 0x00000077); + WREG32_SOC15(GC, 0, mmRLC_LB_THR_CONFIG_4, (0x10 | 0x27 << 8 | 0x02FA << 16)); + + /* set mmRLC_LB_CNTR_INIT = 0x0000_0000 */ + WREG32_SOC15(GC, 0, mmRLC_LB_CNTR_INIT, 0x00000000); + + /* set mmRLC_LB_CNTR_MAX = 0x0000_0500 */ + WREG32_SOC15(GC, 0, mmRLC_LB_CNTR_MAX, 0x00000800); + + mutex_lock(&adev->grbm_idx_mutex); + /* set mmRLC_LB_INIT_CU_MASK thru broadcast mode to enable all SE/SH*/ + gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff); + WREG32_SOC15(GC, 0, mmRLC_LB_INIT_CU_MASK, 0xffffffff); + + /* set mmRLC_LB_PARAMS = 0x003F_1006 */ + data = REG_SET_FIELD(0, RLC_LB_PARAMS, FIFO_SAMPLES, 0x0003); + data |= REG_SET_FIELD(data, RLC_LB_PARAMS, PG_IDLE_SAMPLES, 0x0010); + data |= REG_SET_FIELD(data, RLC_LB_PARAMS, PG_IDLE_SAMPLE_INTERVAL, 0x033F); + WREG32_SOC15(GC, 0, mmRLC_LB_PARAMS, data); + + /* set mmRLC_GPM_GENERAL_7[31-16] = 0x00C0 */ + data = RREG32_SOC15(GC, 0, mmRLC_GPM_GENERAL_7); + data &= 0x0000FFFF; + data |= 0x00C00000; + WREG32_SOC15(GC, 0, mmRLC_GPM_GENERAL_7, data); + + /* + * RLC_LB_ALWAYS_ACTIVE_CU_MASK = 0xFFF (12 CUs AON), + * programmed in gfx_v9_0_init_always_on_cu_mask() + */ /* set RLC_LB_CNTL = 0x8000_0095, 31 bit is reserved, * but used for RLC_LB_CNTL configuration */ @@ -951,6 +1056,8 @@ static void gfx_v9_0_init_lbpw(struct amdgpu_device *adev) data |= REG_SET_FIELD(data, RLC_LB_CNTL, RESERVED, 0x80000); WREG32_SOC15(GC, 0, mmRLC_LB_CNTL, data); mutex_unlock(&adev->grbm_idx_mutex); + + gfx_v9_0_init_always_on_cu_mask(adev); } static void gfx_v9_0_enable_lbpw(struct amdgpu_device *adev, bool enable) @@ -1084,8 +1191,17 @@ static int gfx_v9_0_rlc_init(struct amdgpu_device *adev) rv_init_cp_jump_table(adev); amdgpu_bo_kunmap(adev->gfx.rlc.cp_table_obj); amdgpu_bo_unreserve(adev->gfx.rlc.cp_table_obj); + } + switch (adev->asic_type) { + case CHIP_RAVEN: gfx_v9_0_init_lbpw(adev); + break; + case CHIP_VEGA20: + gfx_v9_4_init_lbpw(adev); + break; + default: + break; } return 0; @@ -1605,11 +1721,6 @@ static int gfx_v9_0_sw_init(void *handle) adev->gfx.mec.num_pipe_per_mec = 4; adev->gfx.mec.num_queue_per_pipe = 8; - /* KIQ event */ - r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_GRBM_CP, GFX_9_0__SRCID__CP_IB2_INTERRUPT_PKT, &adev->gfx.kiq.irq); - if (r) - return r; - /* EOP Event */ r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_GRBM_CP, GFX_9_0__SRCID__CP_EOP_INTERRUPT, &adev->gfx.eop_irq); if (r) @@ -1847,7 +1958,7 @@ static void gfx_v9_0_init_compute_vmid(struct amdgpu_device *adev) mutex_unlock(&adev->srbm_mutex); } -static void gfx_v9_0_gpu_init(struct amdgpu_device *adev) +static void gfx_v9_0_constants_init(struct amdgpu_device *adev) { u32 tmp; int i; @@ -2403,7 +2514,8 @@ static int gfx_v9_0_rlc_resume(struct amdgpu_device *adev) return r; } - if (adev->asic_type == CHIP_RAVEN) { + if (adev->asic_type == CHIP_RAVEN || + adev->asic_type == CHIP_VEGA20) { if (amdgpu_lbpw != 0) gfx_v9_0_enable_lbpw(adev, true); else @@ -3091,7 +3203,7 @@ static int gfx_v9_0_kcq_init_queue(struct amdgpu_ring *ring) struct v9_mqd *mqd = ring->mqd_ptr; int mqd_idx = ring - &adev->gfx.compute_ring[0]; - if (!adev->in_gpu_reset && !adev->gfx.in_suspend) { + if (!adev->in_gpu_reset && !adev->in_suspend) { memset((void *)mqd, 0, sizeof(struct v9_mqd_allocation)); ((struct v9_mqd_allocation *)mqd)->dynamic_cu_mask = 0xFFFFFFFF; ((struct v9_mqd_allocation *)mqd)->dynamic_rb_mask = 0xFFFFFFFF; @@ -3235,7 +3347,7 @@ static int gfx_v9_0_hw_init(void *handle) gfx_v9_0_init_golden_registers(adev); - gfx_v9_0_gpu_init(adev); + gfx_v9_0_constants_init(adev); r = gfx_v9_0_csb_vram_pin(adev); if (r) @@ -3310,7 +3422,7 @@ static int gfx_v9_0_hw_fini(void *handle) /* Use deinitialize sequence from CAIL when unbinding device from driver, * otherwise KIQ is hanging when binding back */ - if (!adev->in_gpu_reset && !adev->gfx.in_suspend) { + if (!adev->in_gpu_reset && !adev->in_suspend) { mutex_lock(&adev->srbm_mutex); soc15_grbm_select(adev, adev->gfx.kiq.ring.me, adev->gfx.kiq.ring.pipe, @@ -3330,20 +3442,12 @@ static int gfx_v9_0_hw_fini(void *handle) static int gfx_v9_0_suspend(void *handle) { - struct amdgpu_device *adev = (struct amdgpu_device *)handle; - - adev->gfx.in_suspend = true; - return gfx_v9_0_hw_fini(adev); + return gfx_v9_0_hw_fini(handle); } static int gfx_v9_0_resume(void *handle) { - struct amdgpu_device *adev = (struct amdgpu_device *)handle; - int r; - - r = gfx_v9_0_hw_init(adev); - adev->gfx.in_suspend = false; - return r; + return gfx_v9_0_hw_init(handle); } static bool gfx_v9_0_is_idle(void *handle) @@ -4609,68 +4713,6 @@ static int gfx_v9_0_priv_inst_irq(struct amdgpu_device *adev, return 0; } -static int gfx_v9_0_kiq_set_interrupt_state(struct amdgpu_device *adev, - struct amdgpu_irq_src *src, - unsigned int type, - enum amdgpu_interrupt_state state) -{ - uint32_t tmp, target; - struct amdgpu_ring *ring = &(adev->gfx.kiq.ring); - - if (ring->me == 1) - target = SOC15_REG_OFFSET(GC, 0, mmCP_ME1_PIPE0_INT_CNTL); - else - target = SOC15_REG_OFFSET(GC, 0, mmCP_ME2_PIPE0_INT_CNTL); - target += ring->pipe; - - switch (type) { - case AMDGPU_CP_KIQ_IRQ_DRIVER0: - if (state == AMDGPU_IRQ_STATE_DISABLE) { - tmp = RREG32_SOC15(GC, 0, mmCPC_INT_CNTL); - tmp = REG_SET_FIELD(tmp, CPC_INT_CNTL, - GENERIC2_INT_ENABLE, 0); - WREG32_SOC15(GC, 0, mmCPC_INT_CNTL, tmp); - - tmp = RREG32(target); - tmp = REG_SET_FIELD(tmp, CP_ME2_PIPE0_INT_CNTL, - GENERIC2_INT_ENABLE, 0); - WREG32(target, tmp); - } else { - tmp = RREG32_SOC15(GC, 0, mmCPC_INT_CNTL); - tmp = REG_SET_FIELD(tmp, CPC_INT_CNTL, - GENERIC2_INT_ENABLE, 1); - WREG32_SOC15(GC, 0, mmCPC_INT_CNTL, tmp); - - tmp = RREG32(target); - tmp = REG_SET_FIELD(tmp, CP_ME2_PIPE0_INT_CNTL, - GENERIC2_INT_ENABLE, 1); - WREG32(target, tmp); - } - break; - default: - BUG(); /* kiq only support GENERIC2_INT now */ - break; - } - return 0; -} - -static int gfx_v9_0_kiq_irq(struct amdgpu_device *adev, - struct amdgpu_irq_src *source, - struct amdgpu_iv_entry *entry) -{ - u8 me_id, pipe_id, queue_id; - struct amdgpu_ring *ring = &(adev->gfx.kiq.ring); - - me_id = (entry->ring_id & 0x0c) >> 2; - pipe_id = (entry->ring_id & 0x03) >> 0; - queue_id = (entry->ring_id & 0x70) >> 4; - DRM_DEBUG("IH: CPC GENERIC2_INT, me:%d, pipe:%d, queue:%d\n", - me_id, pipe_id, queue_id); - - amdgpu_fence_process(ring); - return 0; -} - static const struct amd_ip_funcs gfx_v9_0_ip_funcs = { .name = "gfx_v9_0", .early_init = gfx_v9_0_early_init, @@ -4819,11 +4861,6 @@ static void gfx_v9_0_set_ring_funcs(struct amdgpu_device *adev) adev->gfx.compute_ring[i].funcs = &gfx_v9_0_ring_funcs_compute; } -static const struct amdgpu_irq_src_funcs gfx_v9_0_kiq_irq_funcs = { - .set = gfx_v9_0_kiq_set_interrupt_state, - .process = gfx_v9_0_kiq_irq, -}; - static const struct amdgpu_irq_src_funcs gfx_v9_0_eop_irq_funcs = { .set = gfx_v9_0_set_eop_interrupt_state, .process = gfx_v9_0_eop_irq, @@ -4849,9 +4886,6 @@ static void gfx_v9_0_set_irq_funcs(struct amdgpu_device *adev) adev->gfx.priv_inst_irq.num_types = 1; adev->gfx.priv_inst_irq.funcs = &gfx_v9_0_priv_inst_irq_funcs; - - adev->gfx.kiq.irq.num_types = AMDGPU_CP_KIQ_IRQ_LAST; - adev->gfx.kiq.irq.funcs = &gfx_v9_0_kiq_irq_funcs; } static void gfx_v9_0_set_rlc_funcs(struct amdgpu_device *adev) @@ -4871,7 +4905,20 @@ static void gfx_v9_0_set_rlc_funcs(struct amdgpu_device *adev) static void gfx_v9_0_set_gds_init(struct amdgpu_device *adev) { /* init asci gds info */ - adev->gds.mem.total_size = RREG32_SOC15(GC, 0, mmGDS_VMID0_SIZE); + switch (adev->asic_type) { + case CHIP_VEGA10: + case CHIP_VEGA12: + case CHIP_VEGA20: + adev->gds.mem.total_size = 0x10000; + break; + case CHIP_RAVEN: + adev->gds.mem.total_size = 0x1000; + break; + default: + adev->gds.mem.total_size = 0x10000; + break; + } + adev->gds.gws.total_size = 64; adev->gds.oa.total_size = 16; diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c index 65f58ebcf835..ceb7847b504f 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c @@ -82,7 +82,8 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev) * to get rid of the VM fault and hardware hang. */ WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR, - (max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18) + 0x1); + max((adev->gmc.vram_end >> 18) + 0x1, + adev->gmc.agp_end >> 18)); else WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR, max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18); diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c index 4411463ca719..e1c2b4e9c7b2 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c @@ -859,11 +859,11 @@ static int gmc_v6_0_sw_init(void *handle) adev->gmc.vram_type = gmc_v6_0_convert_vram_type(tmp); } - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 146, &adev->gmc.vm_fault); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 146, &adev->gmc.vm_fault); if (r) return r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 147, &adev->gmc.vm_fault); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 147, &adev->gmc.vm_fault); if (r) return r; @@ -1180,8 +1180,7 @@ static const struct amdgpu_irq_src_funcs gmc_v6_0_irq_funcs = { static void gmc_v6_0_set_gmc_funcs(struct amdgpu_device *adev) { - if (adev->gmc.gmc_funcs == NULL) - adev->gmc.gmc_funcs = &gmc_v6_0_gmc_funcs; + adev->gmc.gmc_funcs = &gmc_v6_0_gmc_funcs; } static void gmc_v6_0_set_irq_funcs(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index ae776ce9a415..910c4ce19cb3 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -991,11 +991,11 @@ static int gmc_v7_0_sw_init(void *handle) adev->gmc.vram_type = gmc_v7_0_convert_vram_type(tmp); } - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_PAGE_INV_FAULT, &adev->gmc.vm_fault); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_PAGE_INV_FAULT, &adev->gmc.vm_fault); if (r) return r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_MEM_PROT_FAULT, &adev->gmc.vm_fault); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_MEM_PROT_FAULT, &adev->gmc.vm_fault); if (r) return r; @@ -1388,8 +1388,7 @@ static const struct amdgpu_irq_src_funcs gmc_v7_0_irq_funcs = { static void gmc_v7_0_set_gmc_funcs(struct amdgpu_device *adev) { - if (adev->gmc.gmc_funcs == NULL) - adev->gmc.gmc_funcs = &gmc_v7_0_gmc_funcs; + adev->gmc.gmc_funcs = &gmc_v7_0_gmc_funcs; } static void gmc_v7_0_set_irq_funcs(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c index 53ae49b8bde8..1d3265c97b70 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c @@ -1095,11 +1095,11 @@ static int gmc_v8_0_sw_init(void *handle) adev->gmc.vram_type = gmc_v8_0_convert_vram_type(tmp); } - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_PAGE_INV_FAULT, &adev->gmc.vm_fault); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_PAGE_INV_FAULT, &adev->gmc.vm_fault); if (r) return r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_MEM_PROT_FAULT, &adev->gmc.vm_fault); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GFX_MEM_PROT_FAULT, &adev->gmc.vm_fault); if (r) return r; @@ -1733,8 +1733,7 @@ static const struct amdgpu_irq_src_funcs gmc_v8_0_irq_funcs = { static void gmc_v8_0_set_gmc_funcs(struct amdgpu_device *adev) { - if (adev->gmc.gmc_funcs == NULL) - adev->gmc.gmc_funcs = &gmc_v8_0_gmc_funcs; + adev->gmc.gmc_funcs = &gmc_v8_0_gmc_funcs; } static void gmc_v8_0_set_irq_funcs(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c index aad3c7c5fb3a..f35d7a554ad5 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c @@ -593,8 +593,7 @@ static const struct amdgpu_gmc_funcs gmc_v9_0_gmc_funcs = { static void gmc_v9_0_set_gmc_funcs(struct amdgpu_device *adev) { - if (adev->gmc.gmc_funcs == NULL) - adev->gmc.gmc_funcs = &gmc_v9_0_gmc_funcs; + adev->gmc.gmc_funcs = &gmc_v9_0_gmc_funcs; } static int gmc_v9_0_early_init(void *handle) diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index 842c4b677b4d..cf0fc61aebe6 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -255,7 +255,7 @@ static void iceland_ih_decode_iv(struct amdgpu_device *adev, dw[2] = le32_to_cpu(adev->irq.ih.ring[ring_index + 2]); dw[3] = le32_to_cpu(adev->irq.ih.ring[ring_index + 3]); - entry->client_id = AMDGPU_IH_CLIENTID_LEGACY; + entry->client_id = AMDGPU_IRQ_CLIENTID_LEGACY; entry->src_id = dw[0] & 0xff; entry->src_data[0] = dw[1] & 0xfffffff; entry->ring_id = dw[2] & 0xff; @@ -297,7 +297,7 @@ static int iceland_ih_sw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - r = amdgpu_ih_ring_init(adev, 64 * 1024, false); + r = amdgpu_ih_ring_init(adev, &adev->irq.ih, 64 * 1024, false); if (r) return r; @@ -311,7 +311,7 @@ static int iceland_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini(adev); - amdgpu_ih_ring_fini(adev); + amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; @@ -447,8 +447,7 @@ static const struct amdgpu_ih_funcs iceland_ih_funcs = { static void iceland_ih_set_interrupt_funcs(struct amdgpu_device *adev) { - if (adev->irq.ih_funcs == NULL) - adev->irq.ih_funcs = &iceland_ih_funcs; + adev->irq.ih_funcs = &iceland_ih_funcs; } const struct amdgpu_ip_block_version iceland_ih_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/kv_dpm.c b/drivers/gpu/drm/amd/amdgpu/kv_dpm.c index cb79a93c2eb7..d0e478f43443 100644 --- a/drivers/gpu/drm/amd/amdgpu/kv_dpm.c +++ b/drivers/gpu/drm/amd/amdgpu/kv_dpm.c @@ -2995,12 +2995,12 @@ static int kv_dpm_sw_init(void *handle) int ret; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - ret = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 230, + ret = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 230, &adev->pm.dpm.thermal.irq); if (ret) return ret; - ret = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 231, + ret = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 231, &adev->pm.dpm.thermal.irq); if (ret) return ret; diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c index 80698b5ffa4a..fd23ba1226a5 100644 --- a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c +++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c @@ -100,7 +100,8 @@ static void mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev) * to get rid of the VM fault and hardware hang. */ WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR, - (max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18) + 0x1); + max((adev->gmc.vram_end >> 18) + 0x1, + adev->gmc.agp_end >> 18)); else WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR, max(adev->gmc.vram_end, adev->gmc.agp_end) >> 18); @@ -279,7 +280,7 @@ void mmhub_v1_0_update_power_gating(struct amdgpu_device *adev, return; if (enable && adev->pg_flags & AMD_PG_SUPPORT_MMHUB) { - if (adev->powerplay.pp_funcs->set_powergating_by_smu) + if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->set_powergating_by_smu) amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_GMC, true); } diff --git a/drivers/gpu/drm/amd/amdgpu/mxgpu_vi.c b/drivers/gpu/drm/amd/amdgpu/mxgpu_vi.c index 842567b53df5..64e875d528dd 100644 --- a/drivers/gpu/drm/amd/amdgpu/mxgpu_vi.c +++ b/drivers/gpu/drm/amd/amdgpu/mxgpu_vi.c @@ -580,11 +580,11 @@ int xgpu_vi_mailbox_add_irq_id(struct amdgpu_device *adev) { int r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 135, &adev->virt.rcv_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 135, &adev->virt.rcv_irq); if (r) return r; - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 138, &adev->virt.ack_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 138, &adev->virt.ack_irq); if (r) { amdgpu_irq_put(adev, &adev->virt.rcv_irq, 0); return r; diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c index 9217af00bc8d..3f3fac2d50cd 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c @@ -306,11 +306,8 @@ static int psp_v11_0_ring_stop(struct psp_context *psp, enum psp_ring_type ring_type) { int ret = 0; - struct psp_ring *ring; struct amdgpu_device *adev = psp->adev; - ring = &psp->km_ring; - /* Write the ring destroy command to C2PMSG_64 */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_64, GFX_CTRL_CMD_ID_DESTROY_RINGS); diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c index c403bdf8ad70..2d4770e173dd 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c @@ -504,41 +504,6 @@ static int sdma_v2_4_rlc_resume(struct amdgpu_device *adev) return 0; } -/** - * sdma_v2_4_load_microcode - load the sDMA ME ucode - * - * @adev: amdgpu_device pointer - * - * Loads the sDMA0/1 ucode. - * Returns 0 for success, -EINVAL if the ucode is not available. - */ -static int sdma_v2_4_load_microcode(struct amdgpu_device *adev) -{ - const struct sdma_firmware_header_v1_0 *hdr; - const __le32 *fw_data; - u32 fw_size; - int i, j; - - /* halt the MEs */ - sdma_v2_4_enable(adev, false); - - for (i = 0; i < adev->sdma.num_instances; i++) { - if (!adev->sdma.instance[i].fw) - return -EINVAL; - hdr = (const struct sdma_firmware_header_v1_0 *)adev->sdma.instance[i].fw->data; - amdgpu_ucode_print_sdma_hdr(&hdr->header); - fw_size = le32_to_cpu(hdr->header.ucode_size_bytes) / 4; - fw_data = (const __le32 *) - (adev->sdma.instance[i].fw->data + - le32_to_cpu(hdr->header.ucode_array_offset_bytes)); - WREG32(mmSDMA0_UCODE_ADDR + sdma_offsets[i], 0); - for (j = 0; j < fw_size; j++) - WREG32(mmSDMA0_UCODE_DATA + sdma_offsets[i], le32_to_cpup(fw_data++)); - WREG32(mmSDMA0_UCODE_ADDR + sdma_offsets[i], adev->sdma.instance[i].fw_version); - } - - return 0; -} /** * sdma_v2_4_start - setup and start the async dma engines @@ -552,13 +517,6 @@ static int sdma_v2_4_start(struct amdgpu_device *adev) { int r; - - if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { - r = sdma_v2_4_load_microcode(adev); - if (r) - return r; - } - /* halt the engine before programing */ sdma_v2_4_enable(adev, false); @@ -898,19 +856,19 @@ static int sdma_v2_4_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; /* SDMA trap event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_TRAP, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_TRAP, &adev->sdma.trap_irq); if (r) return r; /* SDMA Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 241, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 241, &adev->sdma.illegal_inst_irq); if (r) return r; /* SDMA Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_SRBM_WRITE, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_SRBM_WRITE, &adev->sdma.illegal_inst_irq); if (r) return r; @@ -1296,10 +1254,8 @@ static const struct amdgpu_buffer_funcs sdma_v2_4_buffer_funcs = { static void sdma_v2_4_set_buffer_funcs(struct amdgpu_device *adev) { - if (adev->mman.buffer_funcs == NULL) { - adev->mman.buffer_funcs = &sdma_v2_4_buffer_funcs; - adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; - } + adev->mman.buffer_funcs = &sdma_v2_4_buffer_funcs; + adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; } static const struct amdgpu_vm_pte_funcs sdma_v2_4_vm_pte_funcs = { @@ -1315,15 +1271,13 @@ static void sdma_v2_4_set_vm_pte_funcs(struct amdgpu_device *adev) struct drm_gpu_scheduler *sched; unsigned i; - if (adev->vm_manager.vm_pte_funcs == NULL) { - adev->vm_manager.vm_pte_funcs = &sdma_v2_4_vm_pte_funcs; - for (i = 0; i < adev->sdma.num_instances; i++) { - sched = &adev->sdma.instance[i].ring.sched; - adev->vm_manager.vm_pte_rqs[i] = - &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; - } - adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; + adev->vm_manager.vm_pte_funcs = &sdma_v2_4_vm_pte_funcs; + for (i = 0; i < adev->sdma.num_instances; i++) { + sched = &adev->sdma.instance[i].ring.sched; + adev->vm_manager.vm_pte_rqs[i] = + &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; } + adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; } const struct amdgpu_ip_block_version sdma_v2_4_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c index 2677d6a1bf42..6fb3edaba0ec 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c @@ -318,14 +318,13 @@ static int sdma_v3_0_init_microcode(struct amdgpu_device *adev) if (adev->sdma.instance[i].feature_version >= 20) adev->sdma.instance[i].burst_nop = true; - if (adev->firmware.load_type == AMDGPU_FW_LOAD_SMU) { - info = &adev->firmware.ucode[AMDGPU_UCODE_ID_SDMA0 + i]; - info->ucode_id = AMDGPU_UCODE_ID_SDMA0 + i; - info->fw = adev->sdma.instance[i].fw; - header = (const struct common_firmware_header *)info->fw->data; - adev->firmware.fw_size += - ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); - } + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_SDMA0 + i]; + info->ucode_id = AMDGPU_UCODE_ID_SDMA0 + i; + info->fw = adev->sdma.instance[i].fw; + header = (const struct common_firmware_header *)info->fw->data; + adev->firmware.fw_size += + ALIGN(le32_to_cpu(header->ucode_size_bytes), PAGE_SIZE); + } out: if (err) { @@ -778,42 +777,6 @@ static int sdma_v3_0_rlc_resume(struct amdgpu_device *adev) } /** - * sdma_v3_0_load_microcode - load the sDMA ME ucode - * - * @adev: amdgpu_device pointer - * - * Loads the sDMA0/1 ucode. - * Returns 0 for success, -EINVAL if the ucode is not available. - */ -static int sdma_v3_0_load_microcode(struct amdgpu_device *adev) -{ - const struct sdma_firmware_header_v1_0 *hdr; - const __le32 *fw_data; - u32 fw_size; - int i, j; - - /* halt the MEs */ - sdma_v3_0_enable(adev, false); - - for (i = 0; i < adev->sdma.num_instances; i++) { - if (!adev->sdma.instance[i].fw) - return -EINVAL; - hdr = (const struct sdma_firmware_header_v1_0 *)adev->sdma.instance[i].fw->data; - amdgpu_ucode_print_sdma_hdr(&hdr->header); - fw_size = le32_to_cpu(hdr->header.ucode_size_bytes) / 4; - fw_data = (const __le32 *) - (adev->sdma.instance[i].fw->data + - le32_to_cpu(hdr->header.ucode_array_offset_bytes)); - WREG32(mmSDMA0_UCODE_ADDR + sdma_offsets[i], 0); - for (j = 0; j < fw_size; j++) - WREG32(mmSDMA0_UCODE_DATA + sdma_offsets[i], le32_to_cpup(fw_data++)); - WREG32(mmSDMA0_UCODE_ADDR + sdma_offsets[i], adev->sdma.instance[i].fw_version); - } - - return 0; -} - -/** * sdma_v3_0_start - setup and start the async dma engines * * @adev: amdgpu_device pointer @@ -825,12 +788,6 @@ static int sdma_v3_0_start(struct amdgpu_device *adev) { int r; - if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { - r = sdma_v3_0_load_microcode(adev); - if (r) - return r; - } - /* disable sdma engine before programing it */ sdma_v3_0_ctx_switch_enable(adev, false); sdma_v3_0_enable(adev, false); @@ -1177,19 +1134,19 @@ static int sdma_v3_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; /* SDMA trap event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_TRAP, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_TRAP, &adev->sdma.trap_irq); if (r) return r; /* SDMA Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 241, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 241, &adev->sdma.illegal_inst_irq); if (r) return r; /* SDMA Privileged inst */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_SRBM_WRITE, + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_SDMA_SRBM_WRITE, &adev->sdma.illegal_inst_irq); if (r) return r; @@ -1736,10 +1693,8 @@ static const struct amdgpu_buffer_funcs sdma_v3_0_buffer_funcs = { static void sdma_v3_0_set_buffer_funcs(struct amdgpu_device *adev) { - if (adev->mman.buffer_funcs == NULL) { - adev->mman.buffer_funcs = &sdma_v3_0_buffer_funcs; - adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; - } + adev->mman.buffer_funcs = &sdma_v3_0_buffer_funcs; + adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; } static const struct amdgpu_vm_pte_funcs sdma_v3_0_vm_pte_funcs = { @@ -1755,15 +1710,13 @@ static void sdma_v3_0_set_vm_pte_funcs(struct amdgpu_device *adev) struct drm_gpu_scheduler *sched; unsigned i; - if (adev->vm_manager.vm_pte_funcs == NULL) { - adev->vm_manager.vm_pte_funcs = &sdma_v3_0_vm_pte_funcs; - for (i = 0; i < adev->sdma.num_instances; i++) { - sched = &adev->sdma.instance[i].ring.sched; - adev->vm_manager.vm_pte_rqs[i] = - &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; - } - adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; + adev->vm_manager.vm_pte_funcs = &sdma_v3_0_vm_pte_funcs; + for (i = 0; i < adev->sdma.num_instances; i++) { + sched = &adev->sdma.instance[i].ring.sched; + adev->vm_manager.vm_pte_rqs[i] = + &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; } + adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; } const struct amdgpu_ip_block_version sdma_v3_0_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c index 2ea1f0d8f5be..7a8c9172d30a 100644 --- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c +++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c @@ -148,6 +148,7 @@ static const struct soc15_reg_golden golden_settings_sdma0_4_2[] = SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_RLC7_RB_RPTR_ADDR_LO, 0xfffffffd, 0x00000001), SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_RLC7_RB_WPTR_POLL_CNTL, 0xfffffff7, 0x00403000), SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_UTCL1_PAGE, 0x000003ff, 0x000003c0), + SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_UTCL1_WATERMK, 0xFE000000, 0x00000000), }; static const struct soc15_reg_golden golden_settings_sdma1_4_2[] = { @@ -177,6 +178,7 @@ static const struct soc15_reg_golden golden_settings_sdma1_4_2[] = { SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_RLC7_RB_RPTR_ADDR_LO, 0xfffffffd, 0x00000001), SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_RLC7_RB_WPTR_POLL_CNTL, 0xfffffff7, 0x00403000), SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_UTCL1_PAGE, 0x000003ff, 0x000003c0), + SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_UTCL1_WATERMK, 0xFE000000, 0x00000000), }; static const struct soc15_reg_golden golden_settings_sdma_rv1[] = @@ -818,7 +820,7 @@ sdma_v4_1_update_power_gating(struct amdgpu_device *adev, bool enable) uint32_t def, data; if (enable && (adev->pg_flags & AMD_PG_SUPPORT_SDMA)) { - /* disable idle interrupt */ + /* enable idle interrupt */ def = data = RREG32(SOC15_REG_OFFSET(SDMA0, 0, mmSDMA0_CNTL)); data |= SDMA0_CNTL__CTXEMPTY_INT_ENABLE_MASK; @@ -1320,9 +1322,15 @@ static int sdma_v4_0_sw_init(void *handle) DRM_INFO("use_doorbell being set to: [%s]\n", ring->use_doorbell?"true":"false"); - ring->doorbell_index = (i == 0) ? - (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) //get DWORD offset - : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // get DWORD offset + if (adev->asic_type == CHIP_VEGA10) + ring->doorbell_index = (i == 0) ? + (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE0 << 1) //get DWORD offset + : (AMDGPU_VEGA10_DOORBELL64_sDMA_ENGINE1 << 1); // get DWORD offset + else + ring->doorbell_index = (i == 0) ? + (AMDGPU_DOORBELL64_sDMA_ENGINE0 << 1) //get DWORD offset + : (AMDGPU_DOORBELL64_sDMA_ENGINE1 << 1); // get DWORD offset + sprintf(ring->name, "sdma%d", i); r = amdgpu_ring_init(adev, ring, 1024, @@ -1358,6 +1366,10 @@ static int sdma_v4_0_hw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; + if (adev->asic_type == CHIP_RAVEN && adev->powerplay.pp_funcs && + adev->powerplay.pp_funcs->set_powergating_by_smu) + amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_SDMA, false); + sdma_v4_0_init_golden_registers(adev); r = sdma_v4_0_start(adev); @@ -1375,6 +1387,10 @@ static int sdma_v4_0_hw_fini(void *handle) sdma_v4_0_ctx_switch_enable(adev, false); sdma_v4_0_enable(adev, false); + if (adev->asic_type == CHIP_RAVEN && adev->powerplay.pp_funcs + && adev->powerplay.pp_funcs->set_powergating_by_smu) + amdgpu_dpm_set_powergating_by_smu(adev, AMD_IP_BLOCK_TYPE_SDMA, true); + return 0; } @@ -1801,10 +1817,8 @@ static const struct amdgpu_buffer_funcs sdma_v4_0_buffer_funcs = { static void sdma_v4_0_set_buffer_funcs(struct amdgpu_device *adev) { - if (adev->mman.buffer_funcs == NULL) { - adev->mman.buffer_funcs = &sdma_v4_0_buffer_funcs; - adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; - } + adev->mman.buffer_funcs = &sdma_v4_0_buffer_funcs; + adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; } static const struct amdgpu_vm_pte_funcs sdma_v4_0_vm_pte_funcs = { @@ -1820,15 +1834,13 @@ static void sdma_v4_0_set_vm_pte_funcs(struct amdgpu_device *adev) struct drm_gpu_scheduler *sched; unsigned i; - if (adev->vm_manager.vm_pte_funcs == NULL) { - adev->vm_manager.vm_pte_funcs = &sdma_v4_0_vm_pte_funcs; - for (i = 0; i < adev->sdma.num_instances; i++) { - sched = &adev->sdma.instance[i].ring.sched; - adev->vm_manager.vm_pte_rqs[i] = - &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; - } - adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; + adev->vm_manager.vm_pte_funcs = &sdma_v4_0_vm_pte_funcs; + for (i = 0; i < adev->sdma.num_instances; i++) { + sched = &adev->sdma.instance[i].ring.sched; + adev->vm_manager.vm_pte_rqs[i] = + &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; } + adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; } const struct amdgpu_ip_block_version sdma_v4_0_ip_block = { diff --git a/drivers/gpu/drm/amd/amdgpu/si.c b/drivers/gpu/drm/amd/amdgpu/si.c index c364ef94cc36..f8408f88cd37 100644 --- a/drivers/gpu/drm/amd/amdgpu/si.c +++ b/drivers/gpu/drm/amd/amdgpu/si.c @@ -2057,13 +2057,13 @@ int si_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &si_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v6_0_ip_block); amdgpu_device_ip_block_add(adev, &si_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v6_0_ip_block); + amdgpu_device_ip_block_add(adev, &si_dma_ip_block); amdgpu_device_ip_block_add(adev, &si_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); else amdgpu_device_ip_block_add(adev, &dce_v6_0_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v6_0_ip_block); - amdgpu_device_ip_block_add(adev, &si_dma_ip_block); /* amdgpu_device_ip_block_add(adev, &uvd_v3_1_ip_block); */ /* amdgpu_device_ip_block_add(adev, &vce_v1_0_ip_block); */ break; @@ -2071,13 +2071,14 @@ int si_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &si_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v6_0_ip_block); amdgpu_device_ip_block_add(adev, &si_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v6_0_ip_block); + amdgpu_device_ip_block_add(adev, &si_dma_ip_block); amdgpu_device_ip_block_add(adev, &si_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); else amdgpu_device_ip_block_add(adev, &dce_v6_4_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v6_0_ip_block); - amdgpu_device_ip_block_add(adev, &si_dma_ip_block); + /* amdgpu_device_ip_block_add(adev, &uvd_v3_1_ip_block); */ /* amdgpu_device_ip_block_add(adev, &vce_v1_0_ip_block); */ break; @@ -2085,11 +2086,11 @@ int si_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &si_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v6_0_ip_block); amdgpu_device_ip_block_add(adev, &si_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v6_0_ip_block); + amdgpu_device_ip_block_add(adev, &si_dma_ip_block); amdgpu_device_ip_block_add(adev, &si_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v6_0_ip_block); - amdgpu_device_ip_block_add(adev, &si_dma_ip_block); break; default: BUG(); diff --git a/drivers/gpu/drm/amd/amdgpu/si_dma.c b/drivers/gpu/drm/amd/amdgpu/si_dma.c index fafaf259b17b..adbaea6da0d7 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_dma.c +++ b/drivers/gpu/drm/amd/amdgpu/si_dma.c @@ -502,12 +502,14 @@ static int si_dma_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; /* DMA0 trap event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 224, &adev->sdma.trap_irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 224, + &adev->sdma.trap_irq); if (r) return r; /* DMA1 trap event */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 244, &adev->sdma.trap_irq_1); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 244, + &adev->sdma.trap_irq); if (r) return r; @@ -649,17 +651,10 @@ static int si_dma_process_trap_irq(struct amdgpu_device *adev, struct amdgpu_irq_src *source, struct amdgpu_iv_entry *entry) { - amdgpu_fence_process(&adev->sdma.instance[0].ring); - - return 0; -} - -static int si_dma_process_trap_irq_1(struct amdgpu_device *adev, - struct amdgpu_irq_src *source, - struct amdgpu_iv_entry *entry) -{ - amdgpu_fence_process(&adev->sdma.instance[1].ring); - + if (entry->src_id == 224) + amdgpu_fence_process(&adev->sdma.instance[0].ring); + else + amdgpu_fence_process(&adev->sdma.instance[1].ring); return 0; } @@ -786,11 +781,6 @@ static const struct amdgpu_irq_src_funcs si_dma_trap_irq_funcs = { .process = si_dma_process_trap_irq, }; -static const struct amdgpu_irq_src_funcs si_dma_trap_irq_funcs_1 = { - .set = si_dma_set_trap_irq_state, - .process = si_dma_process_trap_irq_1, -}; - static const struct amdgpu_irq_src_funcs si_dma_illegal_inst_irq_funcs = { .process = si_dma_process_illegal_inst_irq, }; @@ -799,7 +789,6 @@ static void si_dma_set_irq_funcs(struct amdgpu_device *adev) { adev->sdma.trap_irq.num_types = AMDGPU_SDMA_IRQ_LAST; adev->sdma.trap_irq.funcs = &si_dma_trap_irq_funcs; - adev->sdma.trap_irq_1.funcs = &si_dma_trap_irq_funcs_1; adev->sdma.illegal_inst_irq.funcs = &si_dma_illegal_inst_irq_funcs; } @@ -863,10 +852,8 @@ static const struct amdgpu_buffer_funcs si_dma_buffer_funcs = { static void si_dma_set_buffer_funcs(struct amdgpu_device *adev) { - if (adev->mman.buffer_funcs == NULL) { - adev->mman.buffer_funcs = &si_dma_buffer_funcs; - adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; - } + adev->mman.buffer_funcs = &si_dma_buffer_funcs; + adev->mman.buffer_funcs_ring = &adev->sdma.instance[0].ring; } static const struct amdgpu_vm_pte_funcs si_dma_vm_pte_funcs = { @@ -882,15 +869,13 @@ static void si_dma_set_vm_pte_funcs(struct amdgpu_device *adev) struct drm_gpu_scheduler *sched; unsigned i; - if (adev->vm_manager.vm_pte_funcs == NULL) { - adev->vm_manager.vm_pte_funcs = &si_dma_vm_pte_funcs; - for (i = 0; i < adev->sdma.num_instances; i++) { - sched = &adev->sdma.instance[i].ring.sched; - adev->vm_manager.vm_pte_rqs[i] = - &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; - } - adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; + adev->vm_manager.vm_pte_funcs = &si_dma_vm_pte_funcs; + for (i = 0; i < adev->sdma.num_instances; i++) { + sched = &adev->sdma.instance[i].ring.sched; + adev->vm_manager.vm_pte_rqs[i] = + &sched->sched_rq[DRM_SCHED_PRIORITY_KERNEL]; } + adev->vm_manager.vm_pte_num_rqs = adev->sdma.num_instances; } const struct amdgpu_ip_block_version si_dma_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/si_dpm.c b/drivers/gpu/drm/amd/amdgpu/si_dpm.c index 1de96995e690..da58040fdbdc 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_dpm.c +++ b/drivers/gpu/drm/amd/amdgpu/si_dpm.c @@ -7687,11 +7687,11 @@ static int si_dpm_sw_init(void *handle) int ret; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - ret = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 230, &adev->pm.dpm.thermal.irq); + ret = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 230, &adev->pm.dpm.thermal.irq); if (ret) return ret; - ret = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 231, &adev->pm.dpm.thermal.irq); + ret = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 231, &adev->pm.dpm.thermal.irq); if (ret) return ret; diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 60dad63098a2..b3d7d9f83202 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -142,7 +142,7 @@ static void si_ih_decode_iv(struct amdgpu_device *adev, dw[2] = le32_to_cpu(adev->irq.ih.ring[ring_index + 2]); dw[3] = le32_to_cpu(adev->irq.ih.ring[ring_index + 3]); - entry->client_id = AMDGPU_IH_CLIENTID_LEGACY; + entry->client_id = AMDGPU_IRQ_CLIENTID_LEGACY; entry->src_id = dw[0] & 0xff; entry->src_data[0] = dw[1] & 0xfffffff; entry->ring_id = dw[2] & 0xff; @@ -170,7 +170,7 @@ static int si_ih_sw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - r = amdgpu_ih_ring_init(adev, 64 * 1024, false); + r = amdgpu_ih_ring_init(adev, &adev->irq.ih, 64 * 1024, false); if (r) return r; @@ -182,7 +182,7 @@ static int si_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini(adev); - amdgpu_ih_ring_fini(adev); + amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } @@ -308,8 +308,7 @@ static const struct amdgpu_ih_funcs si_ih_funcs = { static void si_ih_set_interrupt_funcs(struct amdgpu_device *adev) { - if (adev->irq.ih_funcs == NULL) - adev->irq.ih_funcs = &si_ih_funcs; + adev->irq.ih_funcs = &si_ih_funcs; } const struct amdgpu_ip_block_version si_ih_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c index 138c4810a3de..bf5e6a413dee 100644 --- a/drivers/gpu/drm/amd/amdgpu/soc15.c +++ b/drivers/gpu/drm/amd/amdgpu/soc15.c @@ -529,6 +529,8 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &psp_v11_0_ip_block); else amdgpu_device_ip_block_add(adev, &psp_v3_1_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block); if (!amdgpu_sriov_vf(adev)) amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) @@ -539,8 +541,6 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev) #else # warning "Enable CONFIG_DRM_AMD_DC for display support on SOC15." #endif - amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block); if (!(adev->asic_type == CHIP_VEGA20 && amdgpu_sriov_vf(adev))) { amdgpu_device_ip_block_add(adev, &uvd_v7_0_ip_block); amdgpu_device_ip_block_add(adev, &vce_v4_0_ip_block); @@ -551,6 +551,8 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &gmc_v9_0_ip_block); amdgpu_device_ip_block_add(adev, &vega10_ih_ip_block); amdgpu_device_ip_block_add(adev, &psp_v10_0_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block); amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -560,8 +562,6 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev) #else # warning "Enable CONFIG_DRM_AMD_DC for display support on SOC15." #endif - amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block); amdgpu_device_ip_block_add(adev, &vcn_v1_0_ip_block); break; default: @@ -739,7 +739,8 @@ static int soc15_common_early_init(void *handle) adev->pg_flags = AMD_PG_SUPPORT_SDMA | AMD_PG_SUPPORT_MMHUB | - AMD_PG_SUPPORT_VCN; + AMD_PG_SUPPORT_VCN | + AMD_PG_SUPPORT_VCN_DPG; } else { adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG | AMD_CG_SUPPORT_GFX_MGLS | diff --git a/drivers/gpu/drm/amd/amdgpu/soc15_common.h b/drivers/gpu/drm/amd/amdgpu/soc15_common.h index f5d602540673..958b10a57073 100644 --- a/drivers/gpu/drm/amd/amdgpu/soc15_common.h +++ b/drivers/gpu/drm/amd/amdgpu/soc15_common.h @@ -57,13 +57,33 @@ loop--; \ if (!loop) { \ DRM_ERROR("Register(%d) [%s] failed to reach value 0x%08x != 0x%08x\n", \ - inst, #reg, expected_value, (tmp_ & (mask))); \ + inst, #reg, (unsigned)expected_value, (unsigned)(tmp_ & (mask))); \ ret = -ETIMEDOUT; \ break; \ } \ } \ } while (0) +#define RREG32_SOC15_DPG_MODE(ip, inst, reg, mask, sram_sel) \ + ({ WREG32_SOC15(ip, inst, mmUVD_DPG_LMA_MASK, mask); \ + WREG32_SOC15(ip, inst, mmUVD_DPG_LMA_CTL, \ + UVD_DPG_LMA_CTL__MASK_EN_MASK | \ + ((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) \ + << UVD_DPG_LMA_CTL__READ_WRITE_ADDR__SHIFT) | \ + (sram_sel << UVD_DPG_LMA_CTL__SRAM_SEL__SHIFT)); \ + RREG32_SOC15(ip, inst, mmUVD_DPG_LMA_DATA); }) + +#define WREG32_SOC15_DPG_MODE(ip, inst, reg, value, mask, sram_sel) \ + do { \ + WREG32_SOC15(ip, inst, mmUVD_DPG_LMA_DATA, value); \ + WREG32_SOC15(ip, inst, mmUVD_DPG_LMA_MASK, mask); \ + WREG32_SOC15(ip, inst, mmUVD_DPG_LMA_CTL, \ + UVD_DPG_LMA_CTL__READ_WRITE_MASK | \ + ((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) \ + << UVD_DPG_LMA_CTL__READ_WRITE_ADDR__SHIFT) | \ + (sram_sel << UVD_DPG_LMA_CTL__SRAM_SEL__SHIFT)); \ + } while (0) + #endif diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index 52853d8a8fdd..3abffd06b5c7 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -266,7 +266,7 @@ static void tonga_ih_decode_iv(struct amdgpu_device *adev, dw[2] = le32_to_cpu(adev->irq.ih.ring[ring_index + 2]); dw[3] = le32_to_cpu(adev->irq.ih.ring[ring_index + 3]); - entry->client_id = AMDGPU_IH_CLIENTID_LEGACY; + entry->client_id = AMDGPU_IRQ_CLIENTID_LEGACY; entry->src_id = dw[0] & 0xff; entry->src_data[0] = dw[1] & 0xfffffff; entry->ring_id = dw[2] & 0xff; @@ -317,7 +317,7 @@ static int tonga_ih_sw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - r = amdgpu_ih_ring_init(adev, 64 * 1024, true); + r = amdgpu_ih_ring_init(adev, &adev->irq.ih, 64 * 1024, true); if (r) return r; @@ -334,7 +334,7 @@ static int tonga_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini(adev); - amdgpu_ih_ring_fini(adev); + amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); return 0; @@ -513,8 +513,7 @@ static const struct amdgpu_ih_funcs tonga_ih_funcs = { static void tonga_ih_set_interrupt_funcs(struct amdgpu_device *adev) { - if (adev->irq.ih_funcs == NULL) - adev->irq.ih_funcs = &tonga_ih_funcs; + adev->irq.ih_funcs = &tonga_ih_funcs; } const struct amdgpu_ip_block_version tonga_ih_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c index 8a926d1df939..1fc17bf39fed 100644 --- a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c +++ b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c @@ -108,7 +108,7 @@ static int uvd_v4_2_sw_init(void *handle) int r; /* UVD TRAP */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 124, &adev->uvd.inst->irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 124, &adev->uvd.inst->irq); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c index 50248059412e..fde6ad5ac9ab 100644 --- a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c +++ b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c @@ -105,7 +105,7 @@ static int uvd_v5_0_sw_init(void *handle) int r; /* UVD TRAP */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_UVD_SYSTEM_MESSAGE, &adev->uvd.inst->irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_UVD_SYSTEM_MESSAGE, &adev->uvd.inst->irq); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c index 6ae82cc2e55e..7a5b40275e8e 100644 --- a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c +++ b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c @@ -274,7 +274,7 @@ err: */ static int uvd_v6_0_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, - bool direct, struct dma_fence **fence) + struct dma_fence **fence) { const unsigned ib_size_dw = 16; struct amdgpu_job *job; @@ -310,11 +310,7 @@ static int uvd_v6_0_enc_get_destroy_msg(struct amdgpu_ring *ring, for (i = ib->length_dw; i < ib_size_dw; ++i) ib->ptr[i] = 0x0; - if (direct) - r = amdgpu_job_submit_direct(job, ring, &f); - else - r = amdgpu_job_submit(job, &ring->adev->vce.entity, - AMDGPU_FENCE_OWNER_UNDEFINED, &f); + r = amdgpu_job_submit_direct(job, ring, &f); if (r) goto err; @@ -345,7 +341,7 @@ static int uvd_v6_0_enc_ring_test_ib(struct amdgpu_ring *ring, long timeout) goto error; } - r = uvd_v6_0_enc_get_destroy_msg(ring, 1, true, &fence); + r = uvd_v6_0_enc_get_destroy_msg(ring, 1, &fence); if (r) { DRM_ERROR("amdgpu: failed to get destroy ib (%ld).\n", r); goto error; @@ -393,14 +389,14 @@ static int uvd_v6_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; /* UVD TRAP */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_UVD_SYSTEM_MESSAGE, &adev->uvd.inst->irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_UVD_SYSTEM_MESSAGE, &adev->uvd.inst->irq); if (r) return r; /* UVD ENC TRAP */ if (uvd_v6_0_enc_support(adev)) { for (i = 0; i < adev->uvd.num_enc_rings; ++i) { - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, i + VISLANDS30_IV_SRCID_UVD_ENC_GEN_PURP, &adev->uvd.inst->irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, i + VISLANDS30_IV_SRCID_UVD_ENC_GEN_PURP, &adev->uvd.inst->irq); if (r) return r; } diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c index a289f6a20b6b..58b39afcfb86 100644 --- a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c +++ b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c @@ -280,8 +280,8 @@ err: * * Close up a stream for HW test or if userspace failed to do so */ -int uvd_v7_0_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, - bool direct, struct dma_fence **fence) +static int uvd_v7_0_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, + struct dma_fence **fence) { const unsigned ib_size_dw = 16; struct amdgpu_job *job; @@ -317,11 +317,7 @@ int uvd_v7_0_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, for (i = ib->length_dw; i < ib_size_dw; ++i) ib->ptr[i] = 0x0; - if (direct) - r = amdgpu_job_submit_direct(job, ring, &f); - else - r = amdgpu_job_submit(job, &ring->adev->vce.entity, - AMDGPU_FENCE_OWNER_UNDEFINED, &f); + r = amdgpu_job_submit_direct(job, ring, &f); if (r) goto err; @@ -352,7 +348,7 @@ static int uvd_v7_0_enc_ring_test_ib(struct amdgpu_ring *ring, long timeout) goto error; } - r = uvd_v7_0_enc_get_destroy_msg(ring, 1, true, &fence); + r = uvd_v7_0_enc_get_destroy_msg(ring, 1, &fence); if (r) { DRM_ERROR("amdgpu: (%d)failed to get destroy ib (%ld).\n", ring->me, r); goto error; diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c index 7eaa54ba016b..ea28828360d3 100644 --- a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c @@ -417,7 +417,7 @@ static int vce_v2_0_sw_init(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; /* VCE */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, 167, &adev->vce.irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, 167, &adev->vce.irq); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c index c8390f9adfd6..6dbd39730070 100644 --- a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c @@ -423,7 +423,7 @@ static int vce_v3_0_sw_init(void *handle) int r, i; /* VCE */ - r = amdgpu_irq_add_id(adev, AMDGPU_IH_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_VCE_TRAP, &adev->vce.irq); + r = amdgpu_irq_add_id(adev, AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_VCE_TRAP, &adev->vce.irq); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c index 2664bb2c47c3..eae90922fdbe 100644 --- a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c +++ b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c @@ -37,6 +37,11 @@ #include "ivsrcid/vcn/irqsrcs_vcn_1_0.h" +#define mmUVD_RBC_XX_IB_REG_CHECK 0x05ab +#define mmUVD_RBC_XX_IB_REG_CHECK_BASE_IDX 1 +#define mmUVD_REG_XX_MASK 0x05ac +#define mmUVD_REG_XX_MASK_BASE_IDX 1 + static int vcn_v1_0_stop(struct amdgpu_device *adev); static void vcn_v1_0_set_dec_ring_funcs(struct amdgpu_device *adev); static void vcn_v1_0_set_enc_ring_funcs(struct amdgpu_device *adev); @@ -198,7 +203,8 @@ static int vcn_v1_0_hw_init(void *handle) done: if (!r) - DRM_INFO("VCN decode and encode initialized successfully.\n"); + DRM_INFO("VCN decode and encode initialized successfully(under %s).\n", + (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)?"DPG Mode":"SPG Mode"); return r; } @@ -266,17 +272,18 @@ static int vcn_v1_0_resume(void *handle) } /** - * vcn_v1_0_mc_resume - memory controller programming + * vcn_v1_0_mc_resume_spg_mode - memory controller programming * * @adev: amdgpu_device pointer * * Let the VCN memory controller know it's offsets */ -static void vcn_v1_0_mc_resume(struct amdgpu_device *adev) +static void vcn_v1_0_mc_resume_spg_mode(struct amdgpu_device *adev) { uint32_t size = AMDGPU_GPU_PAGE_ALIGN(adev->vcn.fw->size + 4); uint32_t offset; + /* cache window 0: fw */ if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { WREG32_SOC15(UVD, 0, mmUVD_LMI_VCPU_CACHE_64BIT_BAR_LOW, (adev->firmware.ucode[AMDGPU_UCODE_ID_VCN].tmr_mc_addr_lo)); @@ -296,20 +303,21 @@ static void vcn_v1_0_mc_resume(struct amdgpu_device *adev) WREG32_SOC15(UVD, 0, mmUVD_VCPU_CACHE_SIZE0, size); + /* cache window 1: stack */ WREG32_SOC15(UVD, 0, mmUVD_LMI_VCPU_CACHE1_64BIT_BAR_LOW, lower_32_bits(adev->vcn.gpu_addr + offset)); WREG32_SOC15(UVD, 0, mmUVD_LMI_VCPU_CACHE1_64BIT_BAR_HIGH, upper_32_bits(adev->vcn.gpu_addr + offset)); WREG32_SOC15(UVD, 0, mmUVD_VCPU_CACHE_OFFSET1, 0); - WREG32_SOC15(UVD, 0, mmUVD_VCPU_CACHE_SIZE1, AMDGPU_VCN_HEAP_SIZE); + WREG32_SOC15(UVD, 0, mmUVD_VCPU_CACHE_SIZE1, AMDGPU_VCN_STACK_SIZE); + /* cache window 2: context */ WREG32_SOC15(UVD, 0, mmUVD_LMI_VCPU_CACHE2_64BIT_BAR_LOW, - lower_32_bits(adev->vcn.gpu_addr + offset + AMDGPU_VCN_HEAP_SIZE)); + lower_32_bits(adev->vcn.gpu_addr + offset + AMDGPU_VCN_STACK_SIZE)); WREG32_SOC15(UVD, 0, mmUVD_LMI_VCPU_CACHE2_64BIT_BAR_HIGH, - upper_32_bits(adev->vcn.gpu_addr + offset + AMDGPU_VCN_HEAP_SIZE)); + upper_32_bits(adev->vcn.gpu_addr + offset + AMDGPU_VCN_STACK_SIZE)); WREG32_SOC15(UVD, 0, mmUVD_VCPU_CACHE_OFFSET2, 0); - WREG32_SOC15(UVD, 0, mmUVD_VCPU_CACHE_SIZE2, - AMDGPU_VCN_STACK_SIZE + (AMDGPU_VCN_SESSION_SIZE * 40)); + WREG32_SOC15(UVD, 0, mmUVD_VCPU_CACHE_SIZE2, AMDGPU_VCN_CONTEXT_SIZE); WREG32_SOC15(UVD, 0, mmUVD_UDEC_ADDR_CONFIG, adev->gfx.config.gb_addr_config); @@ -317,6 +325,96 @@ static void vcn_v1_0_mc_resume(struct amdgpu_device *adev) adev->gfx.config.gb_addr_config); WREG32_SOC15(UVD, 0, mmUVD_UDEC_DBW_ADDR_CONFIG, adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_UDEC_DBW_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_MIF_CURR_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_MIF_CURR_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_MIF_RECON1_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_MIF_RECON1_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_MIF_REF_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_MIF_REF_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_JPEG_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); + WREG32_SOC15(UVD, 0, mmUVD_JPEG_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config); +} + +static void vcn_v1_0_mc_resume_dpg_mode(struct amdgpu_device *adev) +{ + uint32_t size = AMDGPU_GPU_PAGE_ALIGN(adev->vcn.fw->size + 4); + uint32_t offset; + + /* cache window 0: fw */ + if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) { + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE_64BIT_BAR_LOW, + (adev->firmware.ucode[AMDGPU_UCODE_ID_VCN].tmr_mc_addr_lo), + 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE_64BIT_BAR_HIGH, + (adev->firmware.ucode[AMDGPU_UCODE_ID_VCN].tmr_mc_addr_hi), + 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CACHE_OFFSET0, 0, + 0xFFFFFFFF, 0); + offset = 0; + } else { + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE_64BIT_BAR_LOW, + lower_32_bits(adev->vcn.gpu_addr), 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE_64BIT_BAR_HIGH, + upper_32_bits(adev->vcn.gpu_addr), 0xFFFFFFFF, 0); + offset = size; + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CACHE_OFFSET0, + AMDGPU_UVD_FIRMWARE_OFFSET >> 3, 0xFFFFFFFF, 0); + } + + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CACHE_SIZE0, size, 0xFFFFFFFF, 0); + + /* cache window 1: stack */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE1_64BIT_BAR_LOW, + lower_32_bits(adev->vcn.gpu_addr + offset), 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE1_64BIT_BAR_HIGH, + upper_32_bits(adev->vcn.gpu_addr + offset), 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CACHE_OFFSET1, 0, + 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CACHE_SIZE1, AMDGPU_VCN_STACK_SIZE, + 0xFFFFFFFF, 0); + + /* cache window 2: context */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE2_64BIT_BAR_LOW, + lower_32_bits(adev->vcn.gpu_addr + offset + AMDGPU_VCN_STACK_SIZE), + 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_VCPU_CACHE2_64BIT_BAR_HIGH, + upper_32_bits(adev->vcn.gpu_addr + offset + AMDGPU_VCN_STACK_SIZE), + 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CACHE_OFFSET2, 0, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CACHE_SIZE2, AMDGPU_VCN_CONTEXT_SIZE, + 0xFFFFFFFF, 0); + + /* VCN global tiling registers */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_UDEC_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_UDEC_DB_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_UDEC_DBW_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_UDEC_DBW_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MIF_CURR_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MIF_CURR_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MIF_RECON1_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MIF_RECON1_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MIF_REF_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MIF_REF_UV_ADDR_CONFIG, + adev->gfx.config.gb_addr_config, 0xFFFFFFFF, 0); } /** @@ -519,6 +617,60 @@ static void vcn_v1_0_enable_clock_gating(struct amdgpu_device *adev) WREG32_SOC15(VCN, 0, mmUVD_SUVD_CGC_CTRL, data); } +static void vcn_v1_0_clock_gating_dpg_mode(struct amdgpu_device *adev, uint8_t sram_sel) +{ + uint32_t reg_data = 0; + + /* disable JPEG CGC */ + if (adev->cg_flags & AMD_CG_SUPPORT_VCN_MGCG) + reg_data = 1 << JPEG_CGC_CTRL__DYN_CLOCK_MODE__SHIFT; + else + reg_data = 0 << JPEG_CGC_CTRL__DYN_CLOCK_MODE__SHIFT; + reg_data |= 1 << JPEG_CGC_CTRL__CLK_GATE_DLY_TIMER__SHIFT; + reg_data |= 4 << JPEG_CGC_CTRL__CLK_OFF_DELAY__SHIFT; + WREG32_SOC15_DPG_MODE(UVD, 0, mmJPEG_CGC_CTRL, reg_data, 0xFFFFFFFF, sram_sel); + + WREG32_SOC15_DPG_MODE(UVD, 0, mmJPEG_CGC_GATE, 0, 0xFFFFFFFF, sram_sel); + + /* enable sw clock gating control */ + if (adev->cg_flags & AMD_CG_SUPPORT_VCN_MGCG) + reg_data = 1 << UVD_CGC_CTRL__DYN_CLOCK_MODE__SHIFT; + else + reg_data = 0 << UVD_CGC_CTRL__DYN_CLOCK_MODE__SHIFT; + reg_data |= 1 << UVD_CGC_CTRL__CLK_GATE_DLY_TIMER__SHIFT; + reg_data |= 4 << UVD_CGC_CTRL__CLK_OFF_DELAY__SHIFT; + reg_data &= ~(UVD_CGC_CTRL__UDEC_RE_MODE_MASK | + UVD_CGC_CTRL__UDEC_CM_MODE_MASK | + UVD_CGC_CTRL__UDEC_IT_MODE_MASK | + UVD_CGC_CTRL__UDEC_DB_MODE_MASK | + UVD_CGC_CTRL__UDEC_MP_MODE_MASK | + UVD_CGC_CTRL__SYS_MODE_MASK | + UVD_CGC_CTRL__UDEC_MODE_MASK | + UVD_CGC_CTRL__MPEG2_MODE_MASK | + UVD_CGC_CTRL__REGS_MODE_MASK | + UVD_CGC_CTRL__RBC_MODE_MASK | + UVD_CGC_CTRL__LMI_MC_MODE_MASK | + UVD_CGC_CTRL__LMI_UMC_MODE_MASK | + UVD_CGC_CTRL__IDCT_MODE_MASK | + UVD_CGC_CTRL__MPRD_MODE_MASK | + UVD_CGC_CTRL__MPC_MODE_MASK | + UVD_CGC_CTRL__LBSI_MODE_MASK | + UVD_CGC_CTRL__LRBBM_MODE_MASK | + UVD_CGC_CTRL__WCB_MODE_MASK | + UVD_CGC_CTRL__VCPU_MODE_MASK | + UVD_CGC_CTRL__SCPU_MODE_MASK); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_CGC_CTRL, reg_data, 0xFFFFFFFF, sram_sel); + + /* turn off clock gating */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_CGC_GATE, 0, 0xFFFFFFFF, sram_sel); + + /* turn on SUVD clock gating */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_SUVD_CGC_GATE, 1, 0xFFFFFFFF, sram_sel); + + /* turn on sw mode in UVD_SUVD_CGC_CTRL */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_SUVD_CGC_CTRL, 0, 0xFFFFFFFF, sram_sel); +} + static void vcn_1_0_disable_static_power_gating(struct amdgpu_device *adev) { uint32_t data = 0; @@ -614,7 +766,7 @@ static void vcn_1_0_enable_static_power_gating(struct amdgpu_device *adev) * * Setup and start the VCN block */ -static int vcn_v1_0_start(struct amdgpu_device *adev) +static int vcn_v1_0_start_spg_mode(struct amdgpu_device *adev) { struct amdgpu_ring *ring = &adev->vcn.ring_dec; uint32_t rb_bufsz, tmp; @@ -625,41 +777,24 @@ static int vcn_v1_0_start(struct amdgpu_device *adev) lmi_swap_cntl = 0; vcn_1_0_disable_static_power_gating(adev); + + tmp = RREG32_SOC15(UVD, 0, mmUVD_STATUS) | UVD_STATUS__UVD_BUSY; + WREG32_SOC15(UVD, 0, mmUVD_STATUS, tmp); + /* disable clock gating */ vcn_v1_0_disable_clock_gating(adev); - vcn_v1_0_mc_resume(adev); - /* disable interupt */ WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_MASTINT_EN), 0, ~UVD_MASTINT_EN__VCPU_EN_MASK); - /* stall UMC and register bus before resetting VCPU */ - WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_LMI_CTRL2), - UVD_LMI_CTRL2__STALL_ARB_UMC_MASK, - ~UVD_LMI_CTRL2__STALL_ARB_UMC_MASK); - mdelay(1); - - /* put LMI, VCPU, RBC etc... into reset */ - WREG32_SOC15(UVD, 0, mmUVD_SOFT_RESET, - UVD_SOFT_RESET__LMI_SOFT_RESET_MASK | - UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK | - UVD_SOFT_RESET__LBSI_SOFT_RESET_MASK | - UVD_SOFT_RESET__RBC_SOFT_RESET_MASK | - UVD_SOFT_RESET__CSM_SOFT_RESET_MASK | - UVD_SOFT_RESET__CXW_SOFT_RESET_MASK | - UVD_SOFT_RESET__TAP_SOFT_RESET_MASK | - UVD_SOFT_RESET__LMI_UMC_SOFT_RESET_MASK); - mdelay(5); - /* initialize VCN memory controller */ - WREG32_SOC15(UVD, 0, mmUVD_LMI_CTRL, - (0x40 << UVD_LMI_CTRL__WRITE_CLEAN_TIMER__SHIFT) | - UVD_LMI_CTRL__WRITE_CLEAN_TIMER_EN_MASK | - UVD_LMI_CTRL__DATA_COHERENCY_EN_MASK | - UVD_LMI_CTRL__VCPU_DATA_COHERENCY_EN_MASK | - UVD_LMI_CTRL__REQ_MODE_MASK | - 0x00100000L); + tmp = RREG32_SOC15(UVD, 0, mmUVD_LMI_CTRL); + WREG32_SOC15(UVD, 0, mmUVD_LMI_CTRL, tmp | + UVD_LMI_CTRL__WRITE_CLEAN_TIMER_EN_MASK | + UVD_LMI_CTRL__MASK_MC_URGENT_MASK | + UVD_LMI_CTRL__DATA_COHERENCY_EN_MASK | + UVD_LMI_CTRL__VCPU_DATA_COHERENCY_EN_MASK); #ifdef __BIG_ENDIAN /* swap (8 in 32) RB and IB */ @@ -667,41 +802,61 @@ static int vcn_v1_0_start(struct amdgpu_device *adev) #endif WREG32_SOC15(UVD, 0, mmUVD_LMI_SWAP_CNTL, lmi_swap_cntl); - WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUXA0, 0x40c2040); - WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUXA1, 0x0); - WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUXB0, 0x40c2040); - WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUXB1, 0x0); - WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_ALU, 0); - WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUX, 0x88); + tmp = RREG32_SOC15(UVD, 0, mmUVD_MPC_CNTL); + tmp &= ~UVD_MPC_CNTL__REPLACEMENT_MODE_MASK; + tmp |= 0x2 << UVD_MPC_CNTL__REPLACEMENT_MODE__SHIFT; + WREG32_SOC15(UVD, 0, mmUVD_MPC_CNTL, tmp); + + WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUXA0, + ((0x1 << UVD_MPC_SET_MUXA0__VARA_1__SHIFT) | + (0x2 << UVD_MPC_SET_MUXA0__VARA_2__SHIFT) | + (0x3 << UVD_MPC_SET_MUXA0__VARA_3__SHIFT) | + (0x4 << UVD_MPC_SET_MUXA0__VARA_4__SHIFT))); + + WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUXB0, + ((0x1 << UVD_MPC_SET_MUXB0__VARB_1__SHIFT) | + (0x2 << UVD_MPC_SET_MUXB0__VARB_2__SHIFT) | + (0x3 << UVD_MPC_SET_MUXB0__VARB_3__SHIFT) | + (0x4 << UVD_MPC_SET_MUXB0__VARB_4__SHIFT))); - /* take all subblocks out of reset, except VCPU */ - WREG32_SOC15(UVD, 0, mmUVD_SOFT_RESET, - UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK); - mdelay(5); + WREG32_SOC15(UVD, 0, mmUVD_MPC_SET_MUX, + ((0x0 << UVD_MPC_SET_MUX__SET_0__SHIFT) | + (0x1 << UVD_MPC_SET_MUX__SET_1__SHIFT) | + (0x2 << UVD_MPC_SET_MUX__SET_2__SHIFT))); + + vcn_v1_0_mc_resume_spg_mode(adev); + + WREG32_SOC15(UVD, 0, mmUVD_REG_XX_MASK, 0x10); + WREG32_SOC15(UVD, 0, mmUVD_RBC_XX_IB_REG_CHECK, + RREG32_SOC15(UVD, 0, mmUVD_RBC_XX_IB_REG_CHECK) | 0x3); /* enable VCPU clock */ - WREG32_SOC15(UVD, 0, mmUVD_VCPU_CNTL, - UVD_VCPU_CNTL__CLK_EN_MASK); + WREG32_SOC15(UVD, 0, mmUVD_VCPU_CNTL, UVD_VCPU_CNTL__CLK_EN_MASK); + + /* boot up the VCPU */ + WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_SOFT_RESET), 0, + ~UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK); /* enable UMC */ WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_LMI_CTRL2), 0, ~UVD_LMI_CTRL2__STALL_ARB_UMC_MASK); - /* boot up the VCPU */ - WREG32_SOC15(UVD, 0, mmUVD_SOFT_RESET, 0); - mdelay(10); + tmp = RREG32_SOC15(UVD, 0, mmUVD_SOFT_RESET); + tmp &= ~UVD_SOFT_RESET__LMI_SOFT_RESET_MASK; + tmp &= ~UVD_SOFT_RESET__LMI_UMC_SOFT_RESET_MASK; + WREG32_SOC15(UVD, 0, mmUVD_SOFT_RESET, tmp); for (i = 0; i < 10; ++i) { uint32_t status; for (j = 0; j < 100; ++j) { status = RREG32_SOC15(UVD, 0, mmUVD_STATUS); - if (status & 2) + if (status & UVD_STATUS__IDLE) break; mdelay(10); } r = 0; - if (status & 2) + if (status & UVD_STATUS__IDLE) break; DRM_ERROR("VCN decode not responding, trying to reset the VCPU!!!\n"); @@ -721,24 +876,22 @@ static int vcn_v1_0_start(struct amdgpu_device *adev) } /* enable master interrupt */ WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_MASTINT_EN), - (UVD_MASTINT_EN__VCPU_EN_MASK|UVD_MASTINT_EN__SYS_EN_MASK), - ~(UVD_MASTINT_EN__VCPU_EN_MASK|UVD_MASTINT_EN__SYS_EN_MASK)); + UVD_MASTINT_EN__VCPU_EN_MASK, ~UVD_MASTINT_EN__VCPU_EN_MASK); /* enable system interrupt for JRBC, TODO: move to set interrupt*/ WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_SYS_INT_EN), UVD_SYS_INT_EN__UVD_JRBC_EN_MASK, ~UVD_SYS_INT_EN__UVD_JRBC_EN_MASK); - /* clear the bit 4 of VCN_STATUS */ - WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_STATUS), 0, - ~(2 << UVD_STATUS__VCPU_REPORT__SHIFT)); + /* clear the busy bit of UVD_STATUS */ + tmp = RREG32_SOC15(UVD, 0, mmUVD_STATUS) & ~UVD_STATUS__UVD_BUSY; + WREG32_SOC15(UVD, 0, mmUVD_STATUS, tmp); /* force RBC into idle state */ rb_bufsz = order_base_2(ring->ring_size); tmp = REG_SET_FIELD(0, UVD_RBC_RB_CNTL, RB_BUFSZ, rb_bufsz); tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_BLKSZ, 1); tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_NO_FETCH, 1); - tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_WPTR_POLL_EN, 0); tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_NO_UPDATE, 1); tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_RPTR_WR_EN, 1); WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_CNTL, tmp); @@ -759,6 +912,8 @@ static int vcn_v1_0_start(struct amdgpu_device *adev) /* Initialize the ring buffer's read and write pointers */ WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_RPTR, 0); + WREG32_SOC15(UVD, 0, mmUVD_SCRATCH2, 0); + ring->wptr = RREG32_SOC15(UVD, 0, mmUVD_RBC_RB_RPTR); WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_WPTR, lower_32_bits(ring->wptr)); @@ -782,12 +937,13 @@ static int vcn_v1_0_start(struct amdgpu_device *adev) ring = &adev->vcn.ring_jpeg; WREG32_SOC15(UVD, 0, mmUVD_LMI_JRBC_RB_VMID, 0); - WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_CNTL, (0x00000001L | 0x00000002L)); + WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_CNTL, UVD_JRBC_RB_CNTL__RB_NO_FETCH_MASK | + UVD_JRBC_RB_CNTL__RB_RPTR_WR_EN_MASK); WREG32_SOC15(UVD, 0, mmUVD_LMI_JRBC_RB_64BIT_BAR_LOW, lower_32_bits(ring->gpu_addr)); WREG32_SOC15(UVD, 0, mmUVD_LMI_JRBC_RB_64BIT_BAR_HIGH, upper_32_bits(ring->gpu_addr)); WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_RPTR, 0); WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_WPTR, 0); - WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_CNTL, 0x00000002L); + WREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_CNTL, UVD_JRBC_RB_CNTL__RB_RPTR_WR_EN_MASK); /* initialize wptr */ ring->wptr = RREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_WPTR); @@ -799,6 +955,166 @@ static int vcn_v1_0_start(struct amdgpu_device *adev) return 0; } +static int vcn_v1_0_start_dpg_mode(struct amdgpu_device *adev) +{ + struct amdgpu_ring *ring = &adev->vcn.ring_dec; + uint32_t rb_bufsz, tmp; + uint32_t lmi_swap_cntl; + + /* disable byte swapping */ + lmi_swap_cntl = 0; + + vcn_1_0_enable_static_power_gating(adev); + + /* enable dynamic power gating mode */ + tmp = RREG32_SOC15(UVD, 0, mmUVD_POWER_STATUS); + tmp |= UVD_POWER_STATUS__UVD_PG_MODE_MASK; + tmp |= UVD_POWER_STATUS__UVD_PG_EN_MASK; + WREG32_SOC15(UVD, 0, mmUVD_POWER_STATUS, tmp); + + /* enable clock gating */ + vcn_v1_0_clock_gating_dpg_mode(adev, 0); + + /* enable VCPU clock */ + tmp = (0xFF << UVD_VCPU_CNTL__PRB_TIMEOUT_VAL__SHIFT); + tmp |= UVD_VCPU_CNTL__CLK_EN_MASK; + tmp |= UVD_VCPU_CNTL__MIF_WR_LOW_THRESHOLD_BP_MASK; + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_VCPU_CNTL, tmp, 0xFFFFFFFF, 0); + + /* disable interupt */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MASTINT_EN, + 0, UVD_MASTINT_EN__VCPU_EN_MASK, 0); + + /* initialize VCN memory controller */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_CTRL, + (8 << UVD_LMI_CTRL__WRITE_CLEAN_TIMER__SHIFT) | + UVD_LMI_CTRL__WRITE_CLEAN_TIMER_EN_MASK | + UVD_LMI_CTRL__DATA_COHERENCY_EN_MASK | + UVD_LMI_CTRL__VCPU_DATA_COHERENCY_EN_MASK | + UVD_LMI_CTRL__REQ_MODE_MASK | + UVD_LMI_CTRL__CRC_RESET_MASK | + UVD_LMI_CTRL__MASK_MC_URGENT_MASK | + 0x00100000L, 0xFFFFFFFF, 0); + +#ifdef __BIG_ENDIAN + /* swap (8 in 32) RB and IB */ + lmi_swap_cntl = 0xa; +#endif + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_SWAP_CNTL, lmi_swap_cntl, 0xFFFFFFFF, 0); + + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MPC_CNTL, + 0x2 << UVD_MPC_CNTL__REPLACEMENT_MODE__SHIFT, 0xFFFFFFFF, 0); + + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MPC_SET_MUXA0, + ((0x1 << UVD_MPC_SET_MUXA0__VARA_1__SHIFT) | + (0x2 << UVD_MPC_SET_MUXA0__VARA_2__SHIFT) | + (0x3 << UVD_MPC_SET_MUXA0__VARA_3__SHIFT) | + (0x4 << UVD_MPC_SET_MUXA0__VARA_4__SHIFT)), 0xFFFFFFFF, 0); + + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MPC_SET_MUXB0, + ((0x1 << UVD_MPC_SET_MUXB0__VARB_1__SHIFT) | + (0x2 << UVD_MPC_SET_MUXB0__VARB_2__SHIFT) | + (0x3 << UVD_MPC_SET_MUXB0__VARB_3__SHIFT) | + (0x4 << UVD_MPC_SET_MUXB0__VARB_4__SHIFT)), 0xFFFFFFFF, 0); + + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MPC_SET_MUX, + ((0x0 << UVD_MPC_SET_MUX__SET_0__SHIFT) | + (0x1 << UVD_MPC_SET_MUX__SET_1__SHIFT) | + (0x2 << UVD_MPC_SET_MUX__SET_2__SHIFT)), 0xFFFFFFFF, 0); + + vcn_v1_0_mc_resume_dpg_mode(adev); + + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_REG_XX_MASK, 0x10, 0xFFFFFFFF, 0); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_RBC_XX_IB_REG_CHECK, 0x3, 0xFFFFFFFF, 0); + + /* boot up the VCPU */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_SOFT_RESET, 0, 0xFFFFFFFF, 0); + + /* enable UMC */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_CTRL2, + 0x1F << UVD_LMI_CTRL2__RE_OFLD_MIF_WR_REQ_NUM__SHIFT, + 0xFFFFFFFF, 0); + + /* enable master interrupt */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_MASTINT_EN, + UVD_MASTINT_EN__VCPU_EN_MASK, UVD_MASTINT_EN__VCPU_EN_MASK, 0); + + vcn_v1_0_clock_gating_dpg_mode(adev, 1); + /* setup mmUVD_LMI_CTRL */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_LMI_CTRL, + (8 << UVD_LMI_CTRL__WRITE_CLEAN_TIMER__SHIFT) | + UVD_LMI_CTRL__WRITE_CLEAN_TIMER_EN_MASK | + UVD_LMI_CTRL__DATA_COHERENCY_EN_MASK | + UVD_LMI_CTRL__VCPU_DATA_COHERENCY_EN_MASK | + UVD_LMI_CTRL__REQ_MODE_MASK | + UVD_LMI_CTRL__CRC_RESET_MASK | + UVD_LMI_CTRL__MASK_MC_URGENT_MASK | + 0x00100000L, 0xFFFFFFFF, 1); + + tmp = adev->gfx.config.gb_addr_config; + /* setup VCN global tiling registers */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_JPEG_ADDR_CONFIG, tmp, 0xFFFFFFFF, 1); + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_JPEG_UV_ADDR_CONFIG, tmp, 0xFFFFFFFF, 1); + + /* enable System Interrupt for JRBC */ + WREG32_SOC15_DPG_MODE(UVD, 0, mmUVD_SYS_INT_EN, + UVD_SYS_INT_EN__UVD_JRBC_EN_MASK, 0xFFFFFFFF, 1); + + /* force RBC into idle state */ + rb_bufsz = order_base_2(ring->ring_size); + tmp = REG_SET_FIELD(0, UVD_RBC_RB_CNTL, RB_BUFSZ, rb_bufsz); + tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_BLKSZ, 1); + tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_NO_FETCH, 1); + tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_NO_UPDATE, 1); + tmp = REG_SET_FIELD(tmp, UVD_RBC_RB_CNTL, RB_RPTR_WR_EN, 1); + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_CNTL, tmp); + + /* set the write pointer delay */ + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_WPTR_CNTL, 0); + + /* set the wb address */ + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_RPTR_ADDR, + (upper_32_bits(ring->gpu_addr) >> 2)); + + /* programm the RB_BASE for ring buffer */ + WREG32_SOC15(UVD, 0, mmUVD_LMI_RBC_RB_64BIT_BAR_LOW, + lower_32_bits(ring->gpu_addr)); + WREG32_SOC15(UVD, 0, mmUVD_LMI_RBC_RB_64BIT_BAR_HIGH, + upper_32_bits(ring->gpu_addr)); + + /* Initialize the ring buffer's read and write pointers */ + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_RPTR, 0); + + WREG32_SOC15(UVD, 0, mmUVD_SCRATCH2, 0); + + ring->wptr = RREG32_SOC15(UVD, 0, mmUVD_RBC_RB_RPTR); + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_WPTR, + lower_32_bits(ring->wptr)); + + WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_RBC_RB_CNTL), 0, + ~UVD_RBC_RB_CNTL__RB_NO_FETCH_MASK); + + /* initialize wptr */ + ring->wptr = RREG32_SOC15(UVD, 0, mmUVD_JRBC_RB_WPTR); + + /* copy patch commands to the jpeg ring */ + vcn_v1_0_jpeg_ring_set_patch_ring(ring, + (ring->wptr + ring->max_dw * amdgpu_sched_hw_submission)); + + return 0; +} + +static int vcn_v1_0_start(struct amdgpu_device *adev) +{ + int r; + + if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) + r = vcn_v1_0_start_dpg_mode(adev); + else + r = vcn_v1_0_start_spg_mode(adev); + return r; +} + /** * vcn_v1_0_stop - stop VCN block * @@ -806,41 +1122,90 @@ static int vcn_v1_0_start(struct amdgpu_device *adev) * * stop the VCN block */ -static int vcn_v1_0_stop(struct amdgpu_device *adev) +static int vcn_v1_0_stop_spg_mode(struct amdgpu_device *adev) { - /* force RBC into idle state */ - WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_CNTL, 0x11010101); + int ret_code, tmp; - /* Stall UMC and register bus before resetting VCPU */ - WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_LMI_CTRL2), - UVD_LMI_CTRL2__STALL_ARB_UMC_MASK, - ~UVD_LMI_CTRL2__STALL_ARB_UMC_MASK); - mdelay(1); + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_STATUS, UVD_STATUS__IDLE, 0x7, ret_code); + + tmp = UVD_LMI_STATUS__VCPU_LMI_WRITE_CLEAN_MASK | + UVD_LMI_STATUS__READ_CLEAN_MASK | + UVD_LMI_STATUS__WRITE_CLEAN_MASK | + UVD_LMI_STATUS__WRITE_CLEAN_RAW_MASK; + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_LMI_STATUS, tmp, tmp, ret_code); /* put VCPU into reset */ - WREG32_SOC15(UVD, 0, mmUVD_SOFT_RESET, - UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK); - mdelay(5); + WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_SOFT_RESET), + UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK, + ~UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK); + + tmp = UVD_LMI_STATUS__UMC_READ_CLEAN_RAW_MASK | + UVD_LMI_STATUS__UMC_WRITE_CLEAN_RAW_MASK; + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_LMI_STATUS, tmp, tmp, ret_code); /* disable VCPU clock */ - WREG32_SOC15(UVD, 0, mmUVD_VCPU_CNTL, 0x0); + WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_VCPU_CNTL), 0, + ~UVD_VCPU_CNTL__CLK_EN_MASK); - /* Unstall UMC and register bus */ - WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_LMI_CTRL2), 0, - ~UVD_LMI_CTRL2__STALL_ARB_UMC_MASK); + /* reset LMI UMC/LMI */ + WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_SOFT_RESET), + UVD_SOFT_RESET__LMI_UMC_SOFT_RESET_MASK, + ~UVD_SOFT_RESET__LMI_UMC_SOFT_RESET_MASK); + + WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_SOFT_RESET), + UVD_SOFT_RESET__LMI_SOFT_RESET_MASK, + ~UVD_SOFT_RESET__LMI_SOFT_RESET_MASK); - WREG32_SOC15(VCN, 0, mmUVD_STATUS, 0); + WREG32_SOC15(UVD, 0, mmUVD_STATUS, 0); vcn_v1_0_enable_clock_gating(adev); vcn_1_0_enable_static_power_gating(adev); return 0; } +static int vcn_v1_0_stop_dpg_mode(struct amdgpu_device *adev) +{ + int ret_code = 0; + + /* Wait for power status to be UVD_POWER_STATUS__UVD_POWER_STATUS_TILES_OFF */ + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_POWER_STATUS, + UVD_POWER_STATUS__UVD_POWER_STATUS_TILES_OFF, + UVD_POWER_STATUS__UVD_POWER_STATUS_MASK, ret_code); + + if (!ret_code) { + int tmp = RREG32_SOC15(UVD, 0, mmUVD_RBC_RB_WPTR) & 0x7FFFFFFF; + /* wait for read ptr to be equal to write ptr */ + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_RBC_RB_RPTR, tmp, 0xFFFFFFFF, ret_code); + + SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_POWER_STATUS, + UVD_POWER_STATUS__UVD_POWER_STATUS_TILES_OFF, + UVD_POWER_STATUS__UVD_POWER_STATUS_MASK, ret_code); + } + + /* disable dynamic power gating mode */ + WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_POWER_STATUS), 0, + ~UVD_POWER_STATUS__UVD_PG_MODE_MASK); + + return 0; +} + +static int vcn_v1_0_stop(struct amdgpu_device *adev) +{ + int r; + + if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) + r = vcn_v1_0_stop_dpg_mode(adev); + else + r = vcn_v1_0_stop_spg_mode(adev); + + return r; +} + static bool vcn_v1_0_is_idle(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - return (RREG32_SOC15(VCN, 0, mmUVD_STATUS) == 0x2); + return (RREG32_SOC15(VCN, 0, mmUVD_STATUS) == UVD_STATUS__IDLE); } static int vcn_v1_0_wait_for_idle(void *handle) @@ -848,7 +1213,8 @@ static int vcn_v1_0_wait_for_idle(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; int ret = 0; - SOC15_WAIT_ON_RREG(VCN, 0, mmUVD_STATUS, 0x2, 0x2, ret); + SOC15_WAIT_ON_RREG(VCN, 0, mmUVD_STATUS, UVD_STATUS__IDLE, + UVD_STATUS__IDLE, ret); return ret; } @@ -910,6 +1276,10 @@ static void vcn_v1_0_dec_ring_set_wptr(struct amdgpu_ring *ring) { struct amdgpu_device *adev = ring->adev; + if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) + WREG32_SOC15(UVD, 0, mmUVD_SCRATCH2, + lower_32_bits(ring->wptr) | 0x80000000); + WREG32_SOC15(UVD, 0, mmUVD_RBC_RB_WPTR, lower_32_bits(ring->wptr)); } @@ -1633,12 +2003,20 @@ static int vcn_v1_0_set_powergating_state(void *handle, * revisit this when there is a cleaner line between * the smc and the hw blocks */ + int ret; struct amdgpu_device *adev = (struct amdgpu_device *)handle; + if(state == adev->vcn.cur_state) + return 0; + if (state == AMD_PG_STATE_GATE) - return vcn_v1_0_stop(adev); + ret = vcn_v1_0_stop(adev); else - return vcn_v1_0_start(adev); + ret = vcn_v1_0_start(adev); + + if(!ret) + adev->vcn.cur_state = state; + return ret; } static const struct amd_ip_funcs vcn_v1_0_ip_funcs = { diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index acbe5a770207..a99f71797aa3 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -380,7 +380,7 @@ static int vega10_ih_sw_init(void *handle) int r; struct amdgpu_device *adev = (struct amdgpu_device *)handle; - r = amdgpu_ih_ring_init(adev, 256 * 1024, true); + r = amdgpu_ih_ring_init(adev, &adev->irq.ih, 256 * 1024, true); if (r) return r; @@ -397,7 +397,7 @@ static int vega10_ih_sw_fini(void *handle) struct amdgpu_device *adev = (struct amdgpu_device *)handle; amdgpu_irq_fini(adev); - amdgpu_ih_ring_fini(adev); + amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; } @@ -494,8 +494,7 @@ static const struct amdgpu_ih_funcs vega10_ih_funcs = { static void vega10_ih_set_interrupt_funcs(struct amdgpu_device *adev) { - if (adev->irq.ih_funcs == NULL) - adev->irq.ih_funcs = &vega10_ih_funcs; + adev->irq.ih_funcs = &vega10_ih_funcs; } const struct amdgpu_ip_block_version vega10_ih_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/vi.c b/drivers/gpu/drm/amd/amdgpu/vi.c index 88b57a5e9489..07880d35e9de 100644 --- a/drivers/gpu/drm/amd/amdgpu/vi.c +++ b/drivers/gpu/drm/amd/amdgpu/vi.c @@ -1596,16 +1596,18 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &vi_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v7_4_ip_block); amdgpu_device_ip_block_add(adev, &iceland_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v2_4_ip_block); amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v2_4_ip_block); break; case CHIP_FIJI: amdgpu_device_ip_block_add(adev, &vi_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v8_5_ip_block); amdgpu_device_ip_block_add(adev, &tonga_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -1615,8 +1617,6 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v10_1_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); if (!amdgpu_sriov_vf(adev)) { amdgpu_device_ip_block_add(adev, &uvd_v6_0_ip_block); amdgpu_device_ip_block_add(adev, &vce_v3_0_ip_block); @@ -1626,6 +1626,8 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &vi_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v8_0_ip_block); amdgpu_device_ip_block_add(adev, &tonga_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -1635,8 +1637,6 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v10_0_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); if (!amdgpu_sriov_vf(adev)) { amdgpu_device_ip_block_add(adev, &uvd_v5_0_ip_block); amdgpu_device_ip_block_add(adev, &vce_v3_0_ip_block); @@ -1649,6 +1649,8 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &vi_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v8_1_ip_block); amdgpu_device_ip_block_add(adev, &tonga_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v3_1_ip_block); amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -1658,8 +1660,6 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v11_2_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v3_1_ip_block); amdgpu_device_ip_block_add(adev, &uvd_v6_3_ip_block); amdgpu_device_ip_block_add(adev, &vce_v3_4_ip_block); break; @@ -1667,6 +1667,8 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &vi_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v8_0_ip_block); amdgpu_device_ip_block_add(adev, &cz_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -1676,8 +1678,6 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v11_0_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v8_0_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); amdgpu_device_ip_block_add(adev, &uvd_v6_0_ip_block); amdgpu_device_ip_block_add(adev, &vce_v3_1_ip_block); #if defined(CONFIG_DRM_AMD_ACP) @@ -1688,6 +1688,8 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) amdgpu_device_ip_block_add(adev, &vi_common_ip_block); amdgpu_device_ip_block_add(adev, &gmc_v8_0_ip_block); amdgpu_device_ip_block_add(adev, &cz_ih_ip_block); + amdgpu_device_ip_block_add(adev, &gfx_v8_1_ip_block); + amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); amdgpu_device_ip_block_add(adev, &pp_smu_ip_block); if (adev->enable_virtual_display) amdgpu_device_ip_block_add(adev, &dce_virtual_ip_block); @@ -1697,8 +1699,6 @@ int vi_set_ip_blocks(struct amdgpu_device *adev) #endif else amdgpu_device_ip_block_add(adev, &dce_v11_0_ip_block); - amdgpu_device_ip_block_add(adev, &gfx_v8_1_ip_block); - amdgpu_device_ip_block_add(adev, &sdma_v3_0_ip_block); amdgpu_device_ip_block_add(adev, &uvd_v6_2_ip_block); amdgpu_device_ip_block_add(adev, &vce_v3_4_ip_block); #if defined(CONFIG_DRM_AMD_ACP) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index 758398bdb39b..14d5b5fa822d 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -447,6 +447,24 @@ static int kfd_ioctl_set_cu_mask(struct file *filp, struct kfd_process *p, return retval; } +static int kfd_ioctl_get_queue_wave_state(struct file *filep, + struct kfd_process *p, void *data) +{ + struct kfd_ioctl_get_queue_wave_state_args *args = data; + int r; + + mutex_lock(&p->mutex); + + r = pqm_get_wave_state(&p->pqm, args->queue_id, + (void __user *)args->ctl_stack_address, + &args->ctl_stack_used_size, + &args->save_area_used_size); + + mutex_unlock(&p->mutex); + + return r; +} + static int kfd_ioctl_set_memory_policy(struct file *filep, struct kfd_process *p, void *data) { @@ -1615,6 +1633,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { AMDKFD_IOCTL_DEF(AMDKFD_IOC_SET_CU_MASK, kfd_ioctl_set_cu_mask, 0), + AMDKFD_IOCTL_DEF(AMDKFD_IOC_GET_QUEUE_WAVE_STATE, + kfd_ioctl_get_queue_wave_state, 0) + }; #define AMDKFD_CORE_IOCTL_COUNT ARRAY_SIZE(amdkfd_ioctls) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c index d4560f1869bd..56412b0e7e1c 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c @@ -647,6 +647,7 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev, num_of_cache_types = ARRAY_SIZE(polaris11_cache_info); break; case CHIP_VEGA10: + case CHIP_VEGA20: pcache_info = vega10_cache_info; num_of_cache_types = ARRAY_SIZE(vega10_cache_info); break; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c index b4d9e6b4f583..a9f18ea7e354 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c @@ -53,6 +53,7 @@ static const struct kfd_device_info kaveri_device_info = { .needs_iommu_device = true, .needs_pci_atomics = false, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info carrizo_device_info = { @@ -69,6 +70,7 @@ static const struct kfd_device_info carrizo_device_info = { .needs_iommu_device = true, .needs_pci_atomics = false, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info raven_device_info = { @@ -84,6 +86,7 @@ static const struct kfd_device_info raven_device_info = { .needs_iommu_device = true, .needs_pci_atomics = true, .num_sdma_engines = 1, + .num_sdma_queues_per_engine = 2, }; #endif @@ -101,6 +104,7 @@ static const struct kfd_device_info hawaii_device_info = { .needs_iommu_device = false, .needs_pci_atomics = false, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info tonga_device_info = { @@ -116,21 +120,7 @@ static const struct kfd_device_info tonga_device_info = { .needs_iommu_device = false, .needs_pci_atomics = true, .num_sdma_engines = 2, -}; - -static const struct kfd_device_info tonga_vf_device_info = { - .asic_family = CHIP_TONGA, - .max_pasid_bits = 16, - .max_no_of_hqd = 24, - .doorbell_size = 4, - .ih_ring_entry_size = 4 * sizeof(uint32_t), - .event_interrupt_class = &event_interrupt_class_cik, - .num_of_watch_points = 4, - .mqd_size_aligned = MQD_SIZE_ALIGNED, - .supports_cwsr = false, - .needs_iommu_device = false, - .needs_pci_atomics = false, - .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info fiji_device_info = { @@ -146,6 +136,7 @@ static const struct kfd_device_info fiji_device_info = { .needs_iommu_device = false, .needs_pci_atomics = true, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info fiji_vf_device_info = { @@ -161,6 +152,7 @@ static const struct kfd_device_info fiji_vf_device_info = { .needs_iommu_device = false, .needs_pci_atomics = false, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; @@ -177,6 +169,7 @@ static const struct kfd_device_info polaris10_device_info = { .needs_iommu_device = false, .needs_pci_atomics = true, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info polaris10_vf_device_info = { @@ -192,6 +185,7 @@ static const struct kfd_device_info polaris10_vf_device_info = { .needs_iommu_device = false, .needs_pci_atomics = false, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info polaris11_device_info = { @@ -207,6 +201,7 @@ static const struct kfd_device_info polaris11_device_info = { .needs_iommu_device = false, .needs_pci_atomics = true, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info vega10_device_info = { @@ -222,6 +217,7 @@ static const struct kfd_device_info vega10_device_info = { .needs_iommu_device = false, .needs_pci_atomics = false, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; static const struct kfd_device_info vega10_vf_device_info = { @@ -237,8 +233,24 @@ static const struct kfd_device_info vega10_vf_device_info = { .needs_iommu_device = false, .needs_pci_atomics = false, .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 2, }; +static const struct kfd_device_info vega20_device_info = { + .asic_family = CHIP_VEGA20, + .max_pasid_bits = 16, + .max_no_of_hqd = 24, + .doorbell_size = 8, + .ih_ring_entry_size = 8 * sizeof(uint32_t), + .event_interrupt_class = &event_interrupt_class_v9, + .num_of_watch_points = 4, + .mqd_size_aligned = MQD_SIZE_ALIGNED, + .supports_cwsr = true, + .needs_iommu_device = false, + .needs_pci_atomics = false, + .num_sdma_engines = 2, + .num_sdma_queues_per_engine = 8, +}; struct kfd_deviceid { unsigned short did; @@ -293,7 +305,6 @@ static const struct kfd_deviceid supported_devices[] = { { 0x6928, &tonga_device_info }, /* Tonga */ { 0x6929, &tonga_device_info }, /* Tonga */ { 0x692B, &tonga_device_info }, /* Tonga */ - { 0x692F, &tonga_vf_device_info }, /* Tonga vf */ { 0x6938, &tonga_device_info }, /* Tonga */ { 0x6939, &tonga_device_info }, /* Tonga */ { 0x7300, &fiji_device_info }, /* Fiji */ @@ -328,6 +339,12 @@ static const struct kfd_deviceid supported_devices[] = { { 0x6868, &vega10_device_info }, /* Vega10 */ { 0x686C, &vega10_vf_device_info }, /* Vega10 vf*/ { 0x687F, &vega10_device_info }, /* Vega10 */ + { 0x66a0, &vega20_device_info }, /* Vega20 */ + { 0x66a1, &vega20_device_info }, /* Vega20 */ + { 0x66a2, &vega20_device_info }, /* Vega20 */ + { 0x66a3, &vega20_device_info }, /* Vega20 */ + { 0x66a7, &vega20_device_info }, /* Vega20 */ + { 0x66af, &vega20_device_info } /* Vega20 */ }; static int kfd_gtt_sa_init(struct kfd_dev *kfd, unsigned int buf_size, @@ -366,6 +383,10 @@ struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, return NULL; } + kfd = kzalloc(sizeof(*kfd), GFP_KERNEL); + if (!kfd) + return NULL; + /* Allow BIF to recode atomics to PCIe 3.0 AtomicOps. * 32 and 64-bit requests are possible and must be * supported. @@ -377,12 +398,10 @@ struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, dev_info(kfd_device, "skipped device %x:%x, PCI rejects atomics\n", pdev->vendor, pdev->device); + kfree(kfd); return NULL; - } - - kfd = kzalloc(sizeof(*kfd), GFP_KERNEL); - if (!kfd) - return NULL; + } else if (!ret) + kfd->pci_atomic_requested = true; kfd->kgd = kgd; kfd->device_info = device_info; @@ -419,6 +438,10 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, { unsigned int size; + kfd->mec_fw_version = kfd->kfd2kgd->get_fw_version(kfd->kgd, + KGD_ENGINE_MEC1); + kfd->sdma_fw_version = kfd->kfd2kgd->get_fw_version(kfd->kgd, + KGD_ENGINE_SDMA1); kfd->shared_resources = *gpu_resources; kfd->vm_info.first_vmid_kfd = ffs(gpu_resources->compute_vmid_bitmap)-1; @@ -457,7 +480,8 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd, if (kfd->kfd2kgd->init_gtt_mem_allocation( kfd->kgd, size, &kfd->gtt_mem, - &kfd->gtt_start_gpu_addr, &kfd->gtt_start_cpu_ptr)){ + &kfd->gtt_start_gpu_addr, &kfd->gtt_start_cpu_ptr, + false)) { dev_err(kfd_device, "Could not allocate %d bytes\n", size); goto out; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c index ec0d62a16e53..a3b933967171 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c @@ -109,7 +109,7 @@ static unsigned int get_num_sdma_engines(struct device_queue_manager *dqm) unsigned int get_num_sdma_queues(struct device_queue_manager *dqm) { return dqm->dev->device_info->num_sdma_engines - * KFD_SDMA_QUEUES_PER_ENGINE; + * dqm->dev->device_info->num_sdma_queues_per_engine; } void program_sh_mem_settings(struct device_queue_manager *dqm, @@ -358,8 +358,8 @@ static int create_compute_queue_nocpsch(struct device_queue_manager *dqm, struct queue *q, struct qcm_process_device *qpd) { - int retval; struct mqd_manager *mqd_mgr; + int retval; mqd_mgr = dqm->ops.get_mqd_manager(dqm, KFD_MQD_TYPE_COMPUTE); if (!mqd_mgr) @@ -387,8 +387,12 @@ static int create_compute_queue_nocpsch(struct device_queue_manager *dqm, if (!q->properties.is_active) return 0; - retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe, q->queue, - &q->properties, q->process->mm); + if (WARN(q->process->mm != current->mm, + "should only run in user thread")) + retval = -EFAULT; + else + retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe, q->queue, + &q->properties, current->mm); if (retval) goto out_uninit_mqd; @@ -545,9 +549,15 @@ static int update_queue(struct device_queue_manager *dqm, struct queue *q) retval = map_queues_cpsch(dqm); else if (q->properties.is_active && (q->properties.type == KFD_QUEUE_TYPE_COMPUTE || - q->properties.type == KFD_QUEUE_TYPE_SDMA)) - retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe, q->queue, - &q->properties, q->process->mm); + q->properties.type == KFD_QUEUE_TYPE_SDMA)) { + if (WARN(q->process->mm != current->mm, + "should only run in user thread")) + retval = -EFAULT; + else + retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, + q->pipe, q->queue, + &q->properties, current->mm); + } out_unlock: dqm_unlock(dqm); @@ -653,10 +663,11 @@ out: static int restore_process_queues_nocpsch(struct device_queue_manager *dqm, struct qcm_process_device *qpd) { + struct mm_struct *mm = NULL; struct queue *q; struct mqd_manager *mqd_mgr; struct kfd_process_device *pdd; - uint32_t pd_base; + uint64_t pd_base; int retval = 0; pdd = qpd_to_pdd(qpd); @@ -676,7 +687,7 @@ static int restore_process_queues_nocpsch(struct device_queue_manager *dqm, /* Update PD Base in QPD */ qpd->page_table_base = pd_base; - pr_debug("Updated PD address to 0x%08x\n", pd_base); + pr_debug("Updated PD address to 0x%llx\n", pd_base); if (!list_empty(&qpd->queues_list)) { dqm->dev->kfd2kgd->set_vm_context_page_table_base( @@ -686,6 +697,15 @@ static int restore_process_queues_nocpsch(struct device_queue_manager *dqm, kfd_flush_tlb(pdd); } + /* Take a safe reference to the mm_struct, which may otherwise + * disappear even while the kfd_process is still referenced. + */ + mm = get_task_mm(pdd->process->lead_thread); + if (!mm) { + retval = -EFAULT; + goto out; + } + /* activate all active queues on the qpd */ list_for_each_entry(q, &qpd->queues_list, list) { if (!q->properties.is_evicted) @@ -700,14 +720,15 @@ static int restore_process_queues_nocpsch(struct device_queue_manager *dqm, q->properties.is_evicted = false; q->properties.is_active = true; retval = mqd_mgr->load_mqd(mqd_mgr, q->mqd, q->pipe, - q->queue, &q->properties, - q->process->mm); + q->queue, &q->properties, mm); if (retval) goto out; dqm->queue_count++; } qpd->evicted = 0; out: + if (mm) + mmput(mm); dqm_unlock(dqm); return retval; } @@ -717,7 +738,7 @@ static int restore_process_queues_cpsch(struct device_queue_manager *dqm, { struct queue *q; struct kfd_process_device *pdd; - uint32_t pd_base; + uint64_t pd_base; int retval = 0; pdd = qpd_to_pdd(qpd); @@ -737,7 +758,7 @@ static int restore_process_queues_cpsch(struct device_queue_manager *dqm, /* Update PD Base in QPD */ qpd->page_table_base = pd_base; - pr_debug("Updated PD address to 0x%08x\n", pd_base); + pr_debug("Updated PD address to 0x%llx\n", pd_base); /* activate all active queues on the qpd */ list_for_each_entry(q, &qpd->queues_list, list) { @@ -761,7 +782,7 @@ static int register_process(struct device_queue_manager *dqm, { struct device_process_node *n; struct kfd_process_device *pdd; - uint32_t pd_base; + uint64_t pd_base; int retval; n = kzalloc(sizeof(*n), GFP_KERNEL); @@ -779,6 +800,7 @@ static int register_process(struct device_queue_manager *dqm, /* Update PD Base in QPD */ qpd->page_table_base = pd_base; + pr_debug("Updated PD address to 0x%llx\n", pd_base); retval = dqm->asic_ops.update_qpd(dqm, qpd); @@ -1342,9 +1364,6 @@ static int destroy_queue_cpsch(struct device_queue_manager *dqm, { int retval; struct mqd_manager *mqd_mgr; - bool preempt_all_queues; - - preempt_all_queues = false; retval = 0; @@ -1528,6 +1547,41 @@ static int process_termination_nocpsch(struct device_queue_manager *dqm, return retval; } +static int get_wave_state(struct device_queue_manager *dqm, + struct queue *q, + void __user *ctl_stack, + u32 *ctl_stack_used_size, + u32 *save_area_used_size) +{ + struct mqd_manager *mqd; + int r; + + dqm_lock(dqm); + + if (q->properties.type != KFD_QUEUE_TYPE_COMPUTE || + q->properties.is_active || !q->device->cwsr_enabled) { + r = -EINVAL; + goto dqm_unlock; + } + + mqd = dqm->ops.get_mqd_manager(dqm, KFD_MQD_TYPE_COMPUTE); + if (!mqd) { + r = -ENOMEM; + goto dqm_unlock; + } + + if (!mqd->get_wave_state) { + r = -EINVAL; + goto dqm_unlock; + } + + r = mqd->get_wave_state(mqd, q->mqd, ctl_stack, ctl_stack_used_size, + save_area_used_size); + +dqm_unlock: + dqm_unlock(dqm); + return r; +} static int process_termination_cpsch(struct device_queue_manager *dqm, struct qcm_process_device *qpd) @@ -1649,6 +1703,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev) dqm->ops.process_termination = process_termination_cpsch; dqm->ops.evict_process_queues = evict_process_queues_cpsch; dqm->ops.restore_process_queues = restore_process_queues_cpsch; + dqm->ops.get_wave_state = get_wave_state; break; case KFD_SCHED_POLICY_NO_HWS: /* initialize dqm for no cp scheduling */ @@ -1668,6 +1723,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev) dqm->ops.evict_process_queues = evict_process_queues_nocpsch; dqm->ops.restore_process_queues = restore_process_queues_nocpsch; + dqm->ops.get_wave_state = get_wave_state; break; default: pr_err("Invalid scheduling policy %d\n", dqm->sched_policy); @@ -1695,6 +1751,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev) break; case CHIP_VEGA10: + case CHIP_VEGA20: case CHIP_RAVEN: device_queue_manager_init_v9(&dqm->asic_ops); break; @@ -1806,7 +1863,9 @@ int dqm_debugfs_hqds(struct seq_file *m, void *data) } for (pipe = 0; pipe < get_num_sdma_engines(dqm); pipe++) { - for (queue = 0; queue < KFD_SDMA_QUEUES_PER_ENGINE; queue++) { + for (queue = 0; + queue < dqm->dev->device_info->num_sdma_queues_per_engine; + queue++) { r = dqm->dev->kfd2kgd->hqd_sdma_dump( dqm->dev->kgd, pipe, queue, &dump, &n_regs); if (r) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h index 00da3169a004..70e38a2e23b9 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h @@ -33,7 +33,6 @@ #define KFD_UNMAP_LATENCY_MS (4000) #define QUEUE_PREEMPT_DEFAULT_TIMEOUT_MS (2 * KFD_UNMAP_LATENCY_MS + 1000) -#define KFD_SDMA_QUEUES_PER_ENGINE (2) struct device_process_node { struct qcm_process_device *qpd; @@ -82,6 +81,8 @@ struct device_process_node { * * @restore_process_queues: Restore all evicted queues queues of a process * + * @get_wave_state: Retrieves context save state and optionally copies the + * control stack, if kept in the MQD, to the given userspace address. */ struct device_queue_manager_ops { @@ -137,6 +138,12 @@ struct device_queue_manager_ops { struct qcm_process_device *qpd); int (*restore_process_queues)(struct device_queue_manager *dqm, struct qcm_process_device *qpd); + + int (*get_wave_state)(struct device_queue_manager *dqm, + struct queue *q, + void __user *ctl_stack, + u32 *ctl_stack_used_size, + u32 *save_area_used_size); }; struct device_queue_manager_asic_ops { diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c index 97d5423c5673..3d66cec414af 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c @@ -400,6 +400,7 @@ int kfd_init_apertures(struct kfd_process *process) kfd_init_apertures_vi(pdd, id); break; case CHIP_VEGA10: + case CHIP_VEGA20: case CHIP_RAVEN: kfd_init_apertures_v9(pdd, id); break; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c index 7a61f38c09e6..01494752c36a 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c @@ -62,9 +62,20 @@ int kfd_iommu_device_init(struct kfd_dev *kfd) struct amd_iommu_device_info iommu_info; unsigned int pasid_limit; int err; + struct kfd_topology_device *top_dev; - if (!kfd->device_info->needs_iommu_device) + top_dev = kfd_topology_device_by_id(kfd->id); + + /* + * Overwrite ATS capability according to needs_iommu_device to fix + * potential missing corresponding bit in CRAT of BIOS. + */ + if (!kfd->device_info->needs_iommu_device) { + top_dev->node_props.capability &= ~HSA_CAP_ATS_PRESENT; return 0; + } + + top_dev->node_props.capability |= HSA_CAP_ATS_PRESENT; iommu_info.flags = 0; err = amd_iommu_device_info(kfd->pdev, &iommu_info); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c index 9f84b4d9fb88..6c31f7370193 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue.c @@ -322,6 +322,7 @@ struct kernel_queue *kernel_queue_init(struct kfd_dev *dev, break; case CHIP_VEGA10: + case CHIP_VEGA20: case CHIP_RAVEN: kernel_queue_init_v9(&kq->ops_asic_specific); break; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue_v9.c index 684a3bf07efd..33830b1a5a54 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue_v9.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_kernel_queue_v9.c @@ -71,8 +71,7 @@ static int pm_map_process_v9(struct packet_manager *pm, uint32_t *buffer, struct qcm_process_device *qpd) { struct pm4_mes_map_process *packet; - uint64_t vm_page_table_base_addr = - (uint64_t)(qpd->page_table_base) << 12; + uint64_t vm_page_table_base_addr = qpd->page_table_base; packet = (struct pm4_mes_map_process *)buffer; memset(buffer, 0, sizeof(struct pm4_mes_map_process)); diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c index 3bc25ab84f34..e33019a7a883 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.c @@ -39,6 +39,7 @@ struct mqd_manager *mqd_manager_init(enum KFD_MQD_TYPE type, case CHIP_POLARIS11: return mqd_manager_init_vi_tonga(type, dev); case CHIP_VEGA10: + case CHIP_VEGA20: case CHIP_RAVEN: return mqd_manager_init_v9(type, dev); default: diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h index 4e84052d4e21..f8261313ae7b 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h @@ -43,6 +43,9 @@ * * @is_occupied: Checks if the relevant HQD slot is occupied. * + * @get_wave_state: Retrieves context save state and optionally copies the + * control stack, if kept in the MQD, to the given userspace address. + * * @mqd_mutex: Mqd manager mutex. * * @dev: The kfd device structure coupled with this module. @@ -85,6 +88,11 @@ struct mqd_manager { uint64_t queue_address, uint32_t pipe_id, uint32_t queue_id); + int (*get_wave_state)(struct mqd_manager *mm, void *mqd, + void __user *ctl_stack, + u32 *ctl_stack_used_size, + u32 *save_area_used_size); + #if defined(CONFIG_DEBUG_FS) int (*debugfs_show_mqd)(struct seq_file *m, void *data); #endif diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c index f5fc3675f21e..f381c1cb27bd 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c @@ -88,7 +88,7 @@ static int init_mqd(struct mqd_manager *mm, void **mqd, ALIGN(sizeof(struct v9_mqd), PAGE_SIZE), &((*mqd_mem_obj)->gtt_mem), &((*mqd_mem_obj)->gpu_addr), - (void *)&((*mqd_mem_obj)->cpu_ptr)); + (void *)&((*mqd_mem_obj)->cpu_ptr), true); } else retval = kfd_gtt_sa_allocate(mm->dev, sizeof(struct v9_mqd), mqd_mem_obj); @@ -266,6 +266,28 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd, pipe_id, queue_id); } +static int get_wave_state(struct mqd_manager *mm, void *mqd, + void __user *ctl_stack, + u32 *ctl_stack_used_size, + u32 *save_area_used_size) +{ + struct v9_mqd *m; + + /* Control stack is located one page after MQD. */ + void *mqd_ctl_stack = (void *)((uintptr_t)mqd + PAGE_SIZE); + + m = get_mqd(mqd); + + *ctl_stack_used_size = m->cp_hqd_cntl_stack_size - + m->cp_hqd_cntl_stack_offset; + *save_area_used_size = m->cp_hqd_wg_state_offset; + + if (copy_to_user(ctl_stack, mqd_ctl_stack, m->cp_hqd_cntl_stack_size)) + return -EFAULT; + + return 0; +} + static int init_mqd_hiq(struct mqd_manager *mm, void **mqd, struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr, struct queue_properties *q) @@ -435,6 +457,7 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type, mqd->update_mqd = update_mqd; mqd->destroy_mqd = destroy_mqd; mqd->is_occupied = is_occupied; + mqd->get_wave_state = get_wave_state; #if defined(CONFIG_DEBUG_FS) mqd->debugfs_show_mqd = debugfs_show_mqd; #endif diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c index b81fda3754da..6469b3456f00 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c @@ -269,6 +269,28 @@ static bool is_occupied(struct mqd_manager *mm, void *mqd, pipe_id, queue_id); } +static int get_wave_state(struct mqd_manager *mm, void *mqd, + void __user *ctl_stack, + u32 *ctl_stack_used_size, + u32 *save_area_used_size) +{ + struct vi_mqd *m; + + m = get_mqd(mqd); + + *ctl_stack_used_size = m->cp_hqd_cntl_stack_size - + m->cp_hqd_cntl_stack_offset; + *save_area_used_size = m->cp_hqd_wg_state_offset - + m->cp_hqd_cntl_stack_size; + + /* Control stack is not copied to user mode for GFXv8 because + * it's part of the context save area that is already + * accessible to user mode + */ + + return 0; +} + static int init_mqd_hiq(struct mqd_manager *mm, void **mqd, struct kfd_mem_obj **mqd_mem_obj, uint64_t *gart_addr, struct queue_properties *q) @@ -436,6 +458,7 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type, mqd->update_mqd = update_mqd; mqd->destroy_mqd = destroy_mqd; mqd->is_occupied = is_occupied; + mqd->get_wave_state = get_wave_state; #if defined(CONFIG_DEBUG_FS) mqd->debugfs_show_mqd = debugfs_show_mqd; #endif diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c index 1092631765cb..c6080ed3b6a7 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c @@ -229,6 +229,7 @@ int pm_init(struct packet_manager *pm, struct device_queue_manager *dqm) pm->pmf = &kfd_vi_pm_funcs; break; case CHIP_VEGA10: + case CHIP_VEGA20: case CHIP_RAVEN: pm->pmf = &kfd_v9_pm_funcs; break; diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index 05283c99d731..53ff86d45d91 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -176,6 +176,7 @@ struct kfd_device_info { bool needs_iommu_device; bool needs_pci_atomics; unsigned int num_sdma_engines; + unsigned int num_sdma_queues_per_engine; }; struct kfd_mem_obj { @@ -247,6 +248,10 @@ struct kfd_dev { /* Debug manager */ struct kfd_dbgmgr *dbgmgr; + /* Firmware versions */ + uint16_t mec_fw_version; + uint16_t sdma_fw_version; + /* Maximum process number mapped to HW scheduler */ unsigned int max_proc_per_quantum; @@ -257,6 +262,8 @@ struct kfd_dev { /* xGMI */ uint64_t hive_id; + + bool pci_atomic_requested; }; /* KGD2KFD callbacks */ @@ -500,11 +507,11 @@ struct qcm_process_device { * All the memory management data should be here too */ uint64_t gds_context_area; + uint64_t page_table_base; uint32_t sh_mem_config; uint32_t sh_mem_bases; uint32_t sh_mem_ape1_base; uint32_t sh_mem_ape1_limit; - uint32_t page_table_base; uint32_t gds_size; uint32_t num_gws; uint32_t num_oac; @@ -782,6 +789,7 @@ int kfd_topology_add_device(struct kfd_dev *gpu); int kfd_topology_remove_device(struct kfd_dev *gpu); struct kfd_topology_device *kfd_topology_device_by_proximity_domain( uint32_t proximity_domain); +struct kfd_topology_device *kfd_topology_device_by_id(uint32_t gpu_id); struct kfd_dev *kfd_device_by_id(uint32_t gpu_id); struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev); int kfd_topology_enum_kfd_devices(uint8_t idx, struct kfd_dev **kdev); @@ -855,6 +863,11 @@ int pqm_set_cu_mask(struct process_queue_manager *pqm, unsigned int qid, struct queue_properties *p); struct kernel_queue *pqm_get_kernel_queue(struct process_queue_manager *pqm, unsigned int qid); +int pqm_get_wave_state(struct process_queue_manager *pqm, + unsigned int qid, + void __user *ctl_stack, + u32 *ctl_stack_used_size, + u32 *save_area_used_size); int amdkfd_fence_wait_timeout(unsigned int *fence_addr, unsigned int fence_value, diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c index c8cad9c078ae..fcaaf93681ac 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c @@ -408,6 +408,28 @@ struct kernel_queue *pqm_get_kernel_queue( return NULL; } +int pqm_get_wave_state(struct process_queue_manager *pqm, + unsigned int qid, + void __user *ctl_stack, + u32 *ctl_stack_used_size, + u32 *save_area_used_size) +{ + struct process_queue_node *pqn; + + pqn = get_queue_by_qid(pqm, qid); + if (!pqn) { + pr_debug("amdkfd: No queue %d exists for operation\n", + qid); + return -EFAULT; + } + + return pqn->q->device->dqm->ops.get_wave_state(pqn->q->device->dqm, + pqn->q, + ctl_stack, + ctl_stack_used_size, + save_area_used_size); +} + #if defined(CONFIG_DEBUG_FS) int pqm_debugfs_mqds(struct seq_file *m, void *data) diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c index 19ecc8233d66..e3843c5929ed 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c @@ -63,22 +63,33 @@ struct kfd_topology_device *kfd_topology_device_by_proximity_domain( return device; } -struct kfd_dev *kfd_device_by_id(uint32_t gpu_id) +struct kfd_topology_device *kfd_topology_device_by_id(uint32_t gpu_id) { - struct kfd_topology_device *top_dev; - struct kfd_dev *device = NULL; + struct kfd_topology_device *top_dev = NULL; + struct kfd_topology_device *ret = NULL; down_read(&topology_lock); list_for_each_entry(top_dev, &topology_device_list, list) if (top_dev->gpu_id == gpu_id) { - device = top_dev->gpu; + ret = top_dev; break; } up_read(&topology_lock); - return device; + return ret; +} + +struct kfd_dev *kfd_device_by_id(uint32_t gpu_id) +{ + struct kfd_topology_device *top_dev; + + top_dev = kfd_topology_device_by_id(gpu_id); + if (!top_dev) + return NULL; + + return top_dev->gpu; } struct kfd_dev *kfd_device_by_pci_dev(const struct pci_dev *pdev) @@ -471,11 +482,11 @@ static ssize_t node_show(struct kobject *kobj, struct attribute *attr, (unsigned long long int) 0); sysfs_show_32bit_prop(buffer, "fw_version", - dev->gpu->kfd2kgd->get_fw_version( - dev->gpu->kgd, - KGD_ENGINE_MEC1)); + dev->gpu->mec_fw_version); sysfs_show_32bit_prop(buffer, "capability", dev->node_props.capability); + sysfs_show_32bit_prop(buffer, "sdma_fw_version", + dev->gpu->sdma_fw_version); } return sysfs_show_32bit_prop(buffer, "max_engine_clk_ccompute", @@ -1116,17 +1127,40 @@ static void kfd_fill_mem_clk_max_info(struct kfd_topology_device *dev) static void kfd_fill_iolink_non_crat_info(struct kfd_topology_device *dev) { - struct kfd_iolink_properties *link; + struct kfd_iolink_properties *link, *cpu_link; + struct kfd_topology_device *cpu_dev; + uint32_t cap; + uint32_t cpu_flag = CRAT_IOLINK_FLAGS_ENABLED; + uint32_t flag = CRAT_IOLINK_FLAGS_ENABLED; if (!dev || !dev->gpu) return; - /* GPU only creates direck links so apply flags setting to all */ - if (dev->gpu->device_info->asic_family == CHIP_HAWAII) - list_for_each_entry(link, &dev->io_link_props, list) - link->flags = CRAT_IOLINK_FLAGS_ENABLED | - CRAT_IOLINK_FLAGS_NO_ATOMICS_32_BIT | - CRAT_IOLINK_FLAGS_NO_ATOMICS_64_BIT; + pcie_capability_read_dword(dev->gpu->pdev, + PCI_EXP_DEVCAP2, &cap); + + if (!(cap & (PCI_EXP_DEVCAP2_ATOMIC_COMP32 | + PCI_EXP_DEVCAP2_ATOMIC_COMP64))) + cpu_flag |= CRAT_IOLINK_FLAGS_NO_ATOMICS_32_BIT | + CRAT_IOLINK_FLAGS_NO_ATOMICS_64_BIT; + + if (!dev->gpu->pci_atomic_requested || + dev->gpu->device_info->asic_family == CHIP_HAWAII) + flag |= CRAT_IOLINK_FLAGS_NO_ATOMICS_32_BIT | + CRAT_IOLINK_FLAGS_NO_ATOMICS_64_BIT; + + /* GPU only creates direct links so apply flags setting to all */ + list_for_each_entry(link, &dev->io_link_props, list) { + link->flags = flag; + cpu_dev = kfd_topology_device_by_proximity_domain( + link->node_to); + if (cpu_dev) { + list_for_each_entry(cpu_link, + &cpu_dev->io_link_props, list) + if (cpu_link->node_to == link->node_from) + cpu_link->flags = cpu_flag; + } + } } int kfd_topology_add_device(struct kfd_dev *gpu) @@ -1244,6 +1278,7 @@ int kfd_topology_add_device(struct kfd_dev *gpu) HSA_CAP_DOORBELL_TYPE_TOTALBITS_MASK); break; case CHIP_VEGA10: + case CHIP_VEGA20: case CHIP_RAVEN: dev->node_props.capability |= ((HSA_CAP_DOORBELL_TYPE_2_0 << HSA_CAP_DOORBELL_TYPE_TOTALBITS_SHIFT) & diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 6c849e055038..b0df6dc9a775 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -338,14 +338,6 @@ static int dm_set_powergating_state(void *handle, /* Prototypes of private functions */ static int dm_early_init(void* handle); -static void hotplug_notify_work_func(struct work_struct *work) -{ - struct amdgpu_display_manager *dm = container_of(work, struct amdgpu_display_manager, mst_hotplug_work); - struct drm_device *dev = dm->ddev; - - drm_kms_helper_hotplug_event(dev); -} - /* Allocate memory for FBC compressed data */ static void amdgpu_dm_fbc_init(struct drm_connector *connector) { @@ -447,8 +439,6 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) goto error; } - INIT_WORK(&adev->dm.mst_hotplug_work, hotplug_notify_work_func); - adev->dm.freesync_module = mod_freesync_create(adev->dm.dc); if (!adev->dm.freesync_module) { DRM_ERROR( @@ -728,6 +718,87 @@ amdgpu_dm_find_first_crtc_matching_connector(struct drm_atomic_state *state, return NULL; } +static void emulated_link_detect(struct dc_link *link) +{ + struct dc_sink_init_data sink_init_data = { 0 }; + struct display_sink_capability sink_caps = { 0 }; + enum dc_edid_status edid_status; + struct dc_context *dc_ctx = link->ctx; + struct dc_sink *sink = NULL; + struct dc_sink *prev_sink = NULL; + + link->type = dc_connection_none; + prev_sink = link->local_sink; + + if (prev_sink != NULL) + dc_sink_retain(prev_sink); + + switch (link->connector_signal) { + case SIGNAL_TYPE_HDMI_TYPE_A: { + sink_caps.transaction_type = DDC_TRANSACTION_TYPE_I2C; + sink_caps.signal = SIGNAL_TYPE_HDMI_TYPE_A; + break; + } + + case SIGNAL_TYPE_DVI_SINGLE_LINK: { + sink_caps.transaction_type = DDC_TRANSACTION_TYPE_I2C; + sink_caps.signal = SIGNAL_TYPE_DVI_SINGLE_LINK; + break; + } + + case SIGNAL_TYPE_DVI_DUAL_LINK: { + sink_caps.transaction_type = DDC_TRANSACTION_TYPE_I2C; + sink_caps.signal = SIGNAL_TYPE_DVI_DUAL_LINK; + break; + } + + case SIGNAL_TYPE_LVDS: { + sink_caps.transaction_type = DDC_TRANSACTION_TYPE_I2C; + sink_caps.signal = SIGNAL_TYPE_LVDS; + break; + } + + case SIGNAL_TYPE_EDP: { + sink_caps.transaction_type = + DDC_TRANSACTION_TYPE_I2C_OVER_AUX; + sink_caps.signal = SIGNAL_TYPE_EDP; + break; + } + + case SIGNAL_TYPE_DISPLAY_PORT: { + sink_caps.transaction_type = + DDC_TRANSACTION_TYPE_I2C_OVER_AUX; + sink_caps.signal = SIGNAL_TYPE_VIRTUAL; + break; + } + + default: + DC_ERROR("Invalid connector type! signal:%d\n", + link->connector_signal); + return; + } + + sink_init_data.link = link; + sink_init_data.sink_signal = sink_caps.signal; + + sink = dc_sink_create(&sink_init_data); + if (!sink) { + DC_ERROR("Failed to create sink!\n"); + return; + } + + link->local_sink = sink; + + edid_status = dm_helpers_read_local_edid( + link->ctx, + link, + sink); + + if (edid_status != EDID_OK) + DC_ERROR("Failed to read EDID"); + +} + static int dm_resume(void *handle) { struct amdgpu_device *adev = handle; @@ -741,6 +812,7 @@ static int dm_resume(void *handle) struct drm_plane *plane; struct drm_plane_state *new_plane_state; struct dm_plane_state *dm_new_plane_state; + enum dc_connection_type new_connection_type = dc_connection_none; int ret; int i; @@ -771,7 +843,13 @@ static int dm_resume(void *handle) continue; mutex_lock(&aconnector->hpd_lock); - dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD); + if (!dc_link_detect_sink(aconnector->dc_link, &new_connection_type)) + DRM_ERROR("KMS: Failed to detect connector\n"); + + if (aconnector->base.force && new_connection_type == dc_connection_none) + emulated_link_detect(aconnector->dc_link); + else + dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD); if (aconnector->fake_enable && aconnector->dc_link->local_sink) aconnector->fake_enable = false; @@ -1020,6 +1098,7 @@ static void handle_hpd_irq(void *param) struct amdgpu_dm_connector *aconnector = (struct amdgpu_dm_connector *)param; struct drm_connector *connector = &aconnector->base; struct drm_device *dev = connector->dev; + enum dc_connection_type new_connection_type = dc_connection_none; /* * In case of failure or MST no need to update connector status or notify the OS @@ -1030,7 +1109,21 @@ static void handle_hpd_irq(void *param) if (aconnector->fake_enable) aconnector->fake_enable = false; - if (dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD)) { + if (!dc_link_detect_sink(aconnector->dc_link, &new_connection_type)) + DRM_ERROR("KMS: Failed to detect connector\n"); + + if (aconnector->base.force && new_connection_type == dc_connection_none) { + emulated_link_detect(aconnector->dc_link); + + + drm_modeset_lock_all(dev); + dm_restore_drm_connector_state(dev, connector); + drm_modeset_unlock_all(dev); + + if (aconnector->base.force == DRM_FORCE_UNSPECIFIED) + drm_kms_helper_hotplug_event(dev); + + } else if (dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD)) { amdgpu_dm_update_connector_after_detect(aconnector); @@ -1130,6 +1223,7 @@ static void handle_hpd_rx_irq(void *param) struct drm_device *dev = connector->dev; struct dc_link *dc_link = aconnector->dc_link; bool is_mst_root_connector = aconnector->mst_mgr.mst_state; + enum dc_connection_type new_connection_type = dc_connection_none; /* * TODO:Temporary add mutex to protect hpd interrupt not have a gpio @@ -1142,7 +1236,24 @@ static void handle_hpd_rx_irq(void *param) if (dc_link_handle_hpd_rx_irq(dc_link, NULL, NULL) && !is_mst_root_connector) { /* Downstream Port status changed. */ - if (dc_link_detect(dc_link, DETECT_REASON_HPDRX)) { + if (!dc_link_detect_sink(dc_link, &new_connection_type)) + DRM_ERROR("KMS: Failed to detect connector\n"); + + if (aconnector->base.force && new_connection_type == dc_connection_none) { + emulated_link_detect(dc_link); + + if (aconnector->fake_enable) + aconnector->fake_enable = false; + + amdgpu_dm_update_connector_after_detect(aconnector); + + + drm_modeset_lock_all(dev); + dm_restore_drm_connector_state(dev, connector); + drm_modeset_unlock_all(dev); + + drm_kms_helper_hotplug_event(dev); + } else if (dc_link_detect(dc_link, DETECT_REASON_HPDRX)) { if (aconnector->fake_enable) aconnector->fake_enable = false; @@ -1214,7 +1325,7 @@ static int dce110_register_irq_handlers(struct amdgpu_device *adev) struct dc_interrupt_params int_params = {0}; int r; int i; - unsigned client_id = AMDGPU_IH_CLIENTID_LEGACY; + unsigned client_id = AMDGPU_IRQ_CLIENTID_LEGACY; if (adev->asic_type == CHIP_VEGA10 || adev->asic_type == CHIP_VEGA12 || @@ -1413,6 +1524,13 @@ static int amdgpu_dm_backlight_update_status(struct backlight_device *bd) { struct amdgpu_display_manager *dm = bl_get_data(bd); + /* + * PWM interperts 0 as 100% rather than 0% because of HW + * limitation for level 0.So limiting minimum brightness level + * to 1. + */ + if (bd->props.brightness < 1) + return 1; if (dc_link_set_backlight_level(dm->backlight_link, bd->props.brightness, 0, 0)) return 0; @@ -1539,6 +1657,7 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev) struct amdgpu_mode_info *mode_info = &adev->mode_info; uint32_t link_cnt; int32_t total_overlay_planes, total_primary_planes; + enum dc_connection_type new_connection_type = dc_connection_none; link_cnt = dm->dc->caps.max_links; if (amdgpu_dm_mode_config_init(dm->adev)) { @@ -1605,7 +1724,14 @@ static int amdgpu_dm_initialize_drm_device(struct amdgpu_device *adev) link = dc_get_link_at_index(dm->dc, i); - if (dc_link_detect(link, DETECT_REASON_BOOT)) { + if (!dc_link_detect_sink(link, &new_connection_type)) + DRM_ERROR("KMS: Failed to detect connector\n"); + + if (aconnector->base.force && new_connection_type == dc_connection_none) { + emulated_link_detect(link); + amdgpu_dm_update_connector_after_detect(aconnector); + + } else if (dc_link_detect(link, DETECT_REASON_BOOT)) { amdgpu_dm_update_connector_after_detect(aconnector); register_backlight_device(dm, link); } @@ -2648,7 +2774,7 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector, if (dm_state && dm_state->freesync_capable) stream->ignore_msa_timing_param = true; finish: - if (sink && sink->sink_signal == SIGNAL_TYPE_VIRTUAL) + if (sink && sink->sink_signal == SIGNAL_TYPE_VIRTUAL && aconnector->base.force != DRM_FORCE_ON) dc_sink_release(sink); return stream; @@ -4079,6 +4205,7 @@ static void amdgpu_dm_do_flip(struct drm_crtc *crtc, /* TODO eliminate or rename surface_update */ struct dc_surface_update surface_updates[1] = { {0} }; struct dm_crtc_state *acrtc_state = to_dm_crtc_state(crtc->state); + struct dc_stream_status *stream_status; /* Prepare wait for target vblank early - before the fence-waits */ @@ -4134,7 +4261,19 @@ static void amdgpu_dm_do_flip(struct drm_crtc *crtc, spin_unlock_irqrestore(&crtc->dev->event_lock, flags); - surface_updates->surface = dc_stream_get_status(acrtc_state->stream)->plane_states[0]; + stream_status = dc_stream_get_status(acrtc_state->stream); + if (!stream_status) { + DRM_ERROR("No stream status for CRTC: id=%d\n", + acrtc->crtc_id); + return; + } + + surface_updates->surface = stream_status->plane_states[0]; + if (!surface_updates->surface) { + DRM_ERROR("No surface for CRTC: id=%d\n", + acrtc->crtc_id); + return; + } surface_updates->flip_addr = &addr; dc_commit_updates_for_stream(adev->dm.dc, @@ -4608,12 +4747,18 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state) } spin_unlock_irqrestore(&adev->ddev->event_lock, flags); - /* Signal HW programming completion */ - drm_atomic_helper_commit_hw_done(state); if (wait_for_vblank) drm_atomic_helper_wait_for_flip_done(dev, state); + /* + * FIXME: + * Delay hw_done() until flip_done() is signaled. This is to block + * another commit from freeing the CRTC state while we're still + * waiting on flip_done. + */ + drm_atomic_helper_commit_hw_done(state); + drm_atomic_helper_cleanup_planes(dev, state); /* @@ -4797,6 +4942,8 @@ void set_freesync_on_stream(struct amdgpu_display_manager *dm, mod_freesync_build_vrr_infopacket(dm->freesync_module, new_stream, &vrr, + packet_type_fs1, + NULL, &vrr_infopacket); new_crtc_state->adjust = vrr.adjust; diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h index d4f1bdf93207..978b34a5011c 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h @@ -108,8 +108,6 @@ struct amdgpu_display_manager { const struct dc_link *backlight_link; - struct work_struct mst_hotplug_work; - struct mod_freesync *freesync_module; /** diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c index 0ef4a40d2247..9a7ac58eb18e 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c @@ -705,7 +705,8 @@ int connector_debugfs_init(struct amdgpu_dm_connector *connector) int i; struct dentry *ent, *dir = connector->base.debugfs_entry; - if (connector->base.connector_type == DRM_MODE_CONNECTOR_DisplayPort) { + if (connector->base.connector_type == DRM_MODE_CONNECTOR_DisplayPort || + connector->base.connector_type == DRM_MODE_CONNECTOR_eDP) { for (i = 0; i < ARRAY_SIZE(dp_debugfs_entries); i++) { ent = debugfs_create_file(dp_debugfs_entries[i].name, 0644, diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c index a910f01838ab..a212178f2edc 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c @@ -36,17 +36,13 @@ * Private declarations. *****************************************************************************/ -struct handler_common_data { +struct amdgpu_dm_irq_handler_data { struct list_head list; interrupt_handler handler; void *handler_arg; /* DM which this handler belongs to */ struct amdgpu_display_manager *dm; -}; - -struct amdgpu_dm_irq_handler_data { - struct handler_common_data hcd; /* DAL irq source which registered for this interrupt. */ enum dc_irq_source irq_source; }; @@ -61,7 +57,7 @@ struct amdgpu_dm_irq_handler_data { * Private functions. *****************************************************************************/ -static void init_handler_common_data(struct handler_common_data *hcd, +static void init_handler_common_data(struct amdgpu_dm_irq_handler_data *hcd, void (*ih)(void *), void *args, struct amdgpu_display_manager *dm) @@ -85,11 +81,9 @@ static void dm_irq_work_func(struct work_struct *work) struct amdgpu_dm_irq_handler_data *handler_data; list_for_each(entry, handler_list) { - handler_data = - list_entry( - entry, - struct amdgpu_dm_irq_handler_data, - hcd.list); + handler_data = list_entry(entry, + struct amdgpu_dm_irq_handler_data, + list); DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n", handler_data->irq_source); @@ -97,7 +91,7 @@ static void dm_irq_work_func(struct work_struct *work) DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n", handler_data->irq_source); - handler_data->hcd.handler(handler_data->hcd.handler_arg); + handler_data->handler(handler_data->handler_arg); } /* Call a DAL subcomponent which registered for interrupt notification @@ -137,11 +131,11 @@ static struct list_head *remove_irq_handler(struct amdgpu_device *adev, list_for_each_safe(entry, tmp, hnd_list) { handler = list_entry(entry, struct amdgpu_dm_irq_handler_data, - hcd.list); + list); if (ih == handler) { /* Found our handler. Remove it from the list. */ - list_del(&handler->hcd.list); + list_del(&handler->list); handler_removed = true; break; } @@ -230,8 +224,7 @@ void *amdgpu_dm_irq_register_interrupt(struct amdgpu_device *adev, memset(handler_data, 0, sizeof(*handler_data)); - init_handler_common_data(&handler_data->hcd, ih, handler_args, - &adev->dm); + init_handler_common_data(handler_data, ih, handler_args, &adev->dm); irq_source = int_params->irq_source; @@ -250,7 +243,7 @@ void *amdgpu_dm_irq_register_interrupt(struct amdgpu_device *adev, break; } - list_add_tail(&handler_data->hcd.list, hnd_list); + list_add_tail(&handler_data->list, hnd_list); DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags); @@ -462,15 +455,13 @@ static void amdgpu_dm_irq_immediate_work(struct amdgpu_device *adev, entry, &adev->dm.irq_handler_list_high_tab[irq_source]) { - handler_data = - list_entry( - entry, - struct amdgpu_dm_irq_handler_data, - hcd.list); + handler_data = list_entry(entry, + struct amdgpu_dm_irq_handler_data, + list); /* Call a subcomponent which registered for immediate * interrupt notification */ - handler_data->hcd.handler(handler_data->hcd.handler_arg); + handler_data->handler(handler_data->handler_arg); } DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags); diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c index 6d16b4a0353d..12001a006b2d 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c @@ -101,10 +101,12 @@ bool dm_pp_apply_display_requirements( adev->pm.pm_display_cfg.displays[i].controller_id = dc_cfg->pipe_idx + 1; } - if (adev->powerplay.pp_funcs->display_configuration_change) + if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->display_configuration_change) adev->powerplay.pp_funcs->display_configuration_change( adev->powerplay.pp_handle, &adev->pm.pm_display_cfg); + + amdgpu_pm_compute_clocks(adev); } return true; @@ -302,7 +304,7 @@ bool dm_pp_get_clock_levels_by_type( struct amd_pp_simple_clock_info validation_clks = { 0 }; uint32_t i; - if (adev->powerplay.pp_funcs->get_clock_by_type) { + if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->get_clock_by_type) { if (adev->powerplay.pp_funcs->get_clock_by_type(pp_handle, dc_to_pp_clock_type(clk_type), &pp_clks)) { /* Error in pplib. Provide default values. */ @@ -313,7 +315,7 @@ bool dm_pp_get_clock_levels_by_type( pp_to_dc_clock_levels(&pp_clks, dc_clks, clk_type); - if (adev->powerplay.pp_funcs->get_display_mode_validation_clocks) { + if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->get_display_mode_validation_clocks) { if (adev->powerplay.pp_funcs->get_display_mode_validation_clocks( pp_handle, &validation_clks)) { /* Error in pplib. Provide default values. */ @@ -396,6 +398,9 @@ bool dm_pp_get_clock_levels_by_type_with_voltage( struct pp_clock_levels_with_voltage pp_clk_info = {0}; const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs; + if (!pp_funcs || !pp_funcs->get_clock_by_type_with_voltage) + return false; + if (pp_funcs->get_clock_by_type_with_voltage(pp_handle, dc_to_pp_clock_type(clk_type), &pp_clk_info)) @@ -436,7 +441,7 @@ bool dm_pp_apply_clock_for_voltage_request( if (!pp_clock_request.clock_type) return false; - if (adev->powerplay.pp_funcs->display_clock_voltage_request) + if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->display_clock_voltage_request) ret = adev->powerplay.pp_funcs->display_clock_voltage_request( adev->powerplay.pp_handle, &pp_clock_request); @@ -453,7 +458,7 @@ bool dm_pp_get_static_clocks( struct amd_pp_clock_info pp_clk_info = {0}; int ret = 0; - if (adev->powerplay.pp_funcs->get_current_clocks) + if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->get_current_clocks) ret = adev->powerplay.pp_funcs->get_current_clocks( adev->powerplay.pp_handle, &pp_clk_info); @@ -503,6 +508,9 @@ void pp_rv_set_wm_ranges(struct pp_smu *pp, wm_with_clock_ranges.num_wm_dmif_sets = ranges->num_reader_wm_sets; wm_with_clock_ranges.num_wm_mcif_sets = ranges->num_writer_wm_sets; + if (!pp_funcs || !pp_funcs->set_watermarks_for_clocks_ranges) + return; + for (i = 0; i < wm_with_clock_ranges.num_wm_dmif_sets; i++) { if (ranges->reader_wm_sets[i].wm_inst > 3) wm_dce_clocks[i].wm_set_id = WM_SET_A; diff --git a/drivers/gpu/drm/amd/display/dc/calcs/dcn_calc_auto.c b/drivers/gpu/drm/amd/display/dc/calcs/dcn_calc_auto.c index 5e2ea12fbb73..d0fc54f8fb1c 100644 --- a/drivers/gpu/drm/amd/display/dc/calcs/dcn_calc_auto.c +++ b/drivers/gpu/drm/amd/display/dc/calcs/dcn_calc_auto.c @@ -1625,11 +1625,11 @@ void dispclkdppclkdcfclk_deep_sleep_prefetch_parameters_watermarks_and_performan else { v->dsty_after_scaler = 0.0; } - v->v_update_offset_pix =dcn_bw_ceil2(v->htotal[k] / 4.0, 1.0); + v->v_update_offset_pix[k] = dcn_bw_ceil2(v->htotal[k] / 4.0, 1.0); v->total_repeater_delay_time = v->max_inter_dcn_tile_repeaters * (2.0 / v->dppclk + 3.0 / v->dispclk); - v->v_update_width_pix = (14.0 / v->dcf_clk_deep_sleep + 12.0 / v->dppclk + v->total_repeater_delay_time) * v->pixel_clock[k]; - v->v_ready_offset_pix =dcn_bw_max2(150.0 / v->dppclk, v->total_repeater_delay_time + 20.0 / v->dcf_clk_deep_sleep + 10.0 / v->dppclk) * v->pixel_clock[k]; - v->t_setup = (v->v_update_offset_pix + v->v_update_width_pix + v->v_ready_offset_pix) / v->pixel_clock[k]; + v->v_update_width_pix[k] = (14.0 / v->dcf_clk_deep_sleep + 12.0 / v->dppclk + v->total_repeater_delay_time) * v->pixel_clock[k]; + v->v_ready_offset_pix[k] = dcn_bw_max2(150.0 / v->dppclk, v->total_repeater_delay_time + 20.0 / v->dcf_clk_deep_sleep + 10.0 / v->dppclk) * v->pixel_clock[k]; + v->t_setup = (v->v_update_offset_pix[k] + v->v_update_width_pix[k] + v->v_ready_offset_pix[k]) / v->pixel_clock[k]; v->v_startup[k] =dcn_bw_min2(v->v_startup_lines, v->max_vstartup_lines[k]); if (v->prefetch_mode == 0.0) { v->t_wait =dcn_bw_max3(v->dram_clock_change_latency + v->urgent_latency, v->sr_enter_plus_exit_time, v->urgent_latency); diff --git a/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c b/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c index 80ec09eef44f..3208188b7ed4 100644 --- a/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c +++ b/drivers/gpu/drm/amd/display/dc/calcs/dcn_calcs.c @@ -1096,9 +1096,9 @@ bool dcn_validate_bandwidth( if (pipe->top_pipe && pipe->top_pipe->plane_state == pipe->plane_state) continue; - pipe->pipe_dlg_param.vupdate_width = v->v_update_width[input_idx][v->dpp_per_plane[input_idx] == 2 ? 1 : 0]; - pipe->pipe_dlg_param.vupdate_offset = v->v_update_offset[input_idx][v->dpp_per_plane[input_idx] == 2 ? 1 : 0]; - pipe->pipe_dlg_param.vready_offset = v->v_ready_offset[input_idx][v->dpp_per_plane[input_idx] == 2 ? 1 : 0]; + pipe->pipe_dlg_param.vupdate_width = v->v_update_width_pix[input_idx]; + pipe->pipe_dlg_param.vupdate_offset = v->v_update_offset_pix[input_idx]; + pipe->pipe_dlg_param.vready_offset = v->v_ready_offset_pix[input_idx]; pipe->pipe_dlg_param.vstartup_start = v->v_startup[input_idx]; pipe->pipe_dlg_param.htotal = pipe->stream->timing.h_total; @@ -1137,9 +1137,9 @@ bool dcn_validate_bandwidth( TIMING_3D_FORMAT_SIDE_BY_SIDE))) { if (hsplit_pipe && hsplit_pipe->plane_state == pipe->plane_state) { /* update previously split pipe */ - hsplit_pipe->pipe_dlg_param.vupdate_width = v->v_update_width[input_idx][v->dpp_per_plane[input_idx] == 2 ? 1 : 0]; - hsplit_pipe->pipe_dlg_param.vupdate_offset = v->v_update_offset[input_idx][v->dpp_per_plane[input_idx] == 2 ? 1 : 0]; - hsplit_pipe->pipe_dlg_param.vready_offset = v->v_ready_offset[input_idx][v->dpp_per_plane[input_idx] == 2 ? 1 : 0]; + hsplit_pipe->pipe_dlg_param.vupdate_width = v->v_update_width_pix[input_idx]; + hsplit_pipe->pipe_dlg_param.vupdate_offset = v->v_update_offset_pix[input_idx]; + hsplit_pipe->pipe_dlg_param.vready_offset = v->v_ready_offset_pix[input_idx]; hsplit_pipe->pipe_dlg_param.vstartup_start = v->v_startup[input_idx]; hsplit_pipe->pipe_dlg_param.htotal = pipe->stream->timing.h_total; diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c index 1c438eedf77a..7c491c91465f 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc.c @@ -60,6 +60,7 @@ #define DC_LOGGER \ dc->ctx->logger +const static char DC_BUILD_ID[] = "production-build"; /******************************************************************************* * Private functions @@ -460,9 +461,25 @@ void dc_link_set_preferred_link_settings(struct dc *dc, struct dc_link_settings *link_setting, struct dc_link *link) { + int i; + struct pipe_ctx *pipe; + struct dc_stream_state *link_stream; struct dc_link_settings store_settings = *link_setting; - struct dc_stream_state *link_stream = - link->dc->current_state->res_ctx.pipe_ctx[0].stream; + + for (i = 0; i < MAX_PIPES; i++) { + pipe = &dc->current_state->res_ctx.pipe_ctx[i]; + if (pipe->stream && pipe->stream->sink + && pipe->stream->sink->link) { + if (pipe->stream->sink->link == link) + break; + } + } + + /* Stream not found */ + if (i == MAX_PIPES) + return; + + link_stream = link->dc->current_state->res_ctx.pipe_ctx[i].stream; link->preferred_link_setting = store_settings; if (link_stream) @@ -742,6 +759,8 @@ struct dc *dc_create(const struct dc_init_data *init_params) dc->config = init_params->flags; + dc->build_id = DC_BUILD_ID; + DC_LOG_DC("Display Core initialized\n"); @@ -1094,32 +1113,6 @@ static bool is_surface_in_context( return false; } -static unsigned int pixel_format_to_bpp(enum surface_pixel_format format) -{ - switch (format) { - case SURFACE_PIXEL_FORMAT_VIDEO_420_YCbCr: - case SURFACE_PIXEL_FORMAT_VIDEO_420_YCrCb: - return 12; - case SURFACE_PIXEL_FORMAT_GRPH_ARGB1555: - case SURFACE_PIXEL_FORMAT_GRPH_RGB565: - case SURFACE_PIXEL_FORMAT_VIDEO_420_10bpc_YCbCr: - case SURFACE_PIXEL_FORMAT_VIDEO_420_10bpc_YCrCb: - return 16; - case SURFACE_PIXEL_FORMAT_GRPH_ARGB8888: - case SURFACE_PIXEL_FORMAT_GRPH_ABGR8888: - case SURFACE_PIXEL_FORMAT_GRPH_ARGB2101010: - case SURFACE_PIXEL_FORMAT_GRPH_ABGR2101010: - return 32; - case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616: - case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616F: - case SURFACE_PIXEL_FORMAT_GRPH_ABGR16161616F: - return 64; - default: - ASSERT_CRITICAL(false); - return -1; - } -} - static enum surface_update_type get_plane_info_update_type(const struct dc_surface_update *u) { union surface_update_flags *update_flags = &u->surface->update_flags; @@ -1153,16 +1146,13 @@ static enum surface_update_type get_plane_info_update_type(const struct dc_surfa || u->plane_info->dcc.grph.meta_pitch != u->surface->dcc.grph.meta_pitch) update_flags->bits.dcc_change = 1; - if (pixel_format_to_bpp(u->plane_info->format) != - pixel_format_to_bpp(u->surface->format)) + if (resource_pixel_format_to_bpp(u->plane_info->format) != + resource_pixel_format_to_bpp(u->surface->format)) /* different bytes per element will require full bandwidth * and DML calculation */ update_flags->bits.bpp_change = 1; - if (u->gamma && dce_use_lut(u->plane_info->format)) - update_flags->bits.gamma_change = 1; - if (memcmp(&u->plane_info->tiling_info, &u->surface->tiling_info, sizeof(union dc_tiling_info)) != 0) { update_flags->bits.swizzle_change = 1; @@ -1179,7 +1169,6 @@ static enum surface_update_type get_plane_info_update_type(const struct dc_surfa if (update_flags->bits.rotation_change || update_flags->bits.stereo_format_change || update_flags->bits.pixel_format_change - || update_flags->bits.gamma_change || update_flags->bits.bpp_change || update_flags->bits.bandwidth_change || update_flags->bits.output_tf_change) @@ -1269,13 +1258,26 @@ static enum surface_update_type det_surface_update(const struct dc *dc, if (u->coeff_reduction_factor) update_flags->bits.coeff_reduction_change = 1; + if (u->gamma) { + enum surface_pixel_format format = SURFACE_PIXEL_FORMAT_GRPH_BEGIN; + + if (u->plane_info) + format = u->plane_info->format; + else if (u->surface) + format = u->surface->format; + + if (dce_use_lut(format)) + update_flags->bits.gamma_change = 1; + } + if (update_flags->bits.in_transfer_func_change) { type = UPDATE_TYPE_MED; elevate_update_type(&overall_type, type); } if (update_flags->bits.input_csc_change - || update_flags->bits.coeff_reduction_change) { + || update_flags->bits.coeff_reduction_change + || update_flags->bits.gamma_change) { type = UPDATE_TYPE_FULL; elevate_update_type(&overall_type, type); } @@ -1379,7 +1381,7 @@ static void notify_display_count_to_smu( * sent as part of pplib_apply_display_requirements. * So just return. */ - if (!pp_smu->set_display_count) + if (!pp_smu || !pp_smu->set_display_count) return; display_count = 0; @@ -1834,3 +1836,16 @@ void dc_link_remove_remote_sink(struct dc_link *link, struct dc_sink *sink) } } } + +void get_clock_requirements_for_state(struct dc_state *state, struct AsicStateEx *info) +{ + info->displayClock = (unsigned int)state->bw.dcn.clk.dispclk_khz; + info->engineClock = (unsigned int)state->bw.dcn.clk.dcfclk_khz; + info->memoryClock = (unsigned int)state->bw.dcn.clk.dramclk_khz; + info->maxSupportedDppClock = (unsigned int)state->bw.dcn.clk.max_supported_dppclk_khz; + info->dppClock = (unsigned int)state->bw.dcn.clk.dppclk_khz; + info->socClock = (unsigned int)state->bw.dcn.clk.socclk_khz; + info->dcfClockDeepSleep = (unsigned int)state->bw.dcn.clk.dcfclk_deep_sleep_khz; + info->fClock = (unsigned int)state->bw.dcn.clk.fclk_khz; + info->phyClock = (unsigned int)state->bw.dcn.clk.phyclk_khz; +}
\ No newline at end of file diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link.c b/drivers/gpu/drm/amd/display/dc/core/dc_link.c index bd58dbae7d3e..fb04a4ad141f 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc_link.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc_link.c @@ -198,7 +198,7 @@ static bool program_hpd_filter( return result; } -static bool detect_sink(struct dc_link *link, enum dc_connection_type *type) +bool dc_link_detect_sink(struct dc_link *link, enum dc_connection_type *type) { uint32_t is_hpd_high = 0; struct gpio *hpd_pin; @@ -612,7 +612,7 @@ bool dc_link_detect(struct dc_link *link, enum dc_detect_reason reason) if (link->connector_signal == SIGNAL_TYPE_VIRTUAL) return false; - if (false == detect_sink(link, &new_connection_type)) { + if (false == dc_link_detect_sink(link, &new_connection_type)) { BREAK_TO_DEBUGGER(); return false; } @@ -2559,23 +2559,24 @@ void core_link_enable_stream( pipe_ctx->stream_res.stream_enc, &stream->timing); - resource_build_info_frame(pipe_ctx); - core_dc->hwss.update_info_frame(pipe_ctx); + if (!IS_FPGA_MAXIMUS_DC(core_dc->ctx->dce_environment)) { + resource_build_info_frame(pipe_ctx); + core_dc->hwss.update_info_frame(pipe_ctx); - /* eDP lit up by bios already, no need to enable again. */ - if (pipe_ctx->stream->signal == SIGNAL_TYPE_EDP && - pipe_ctx->stream->apply_edp_fast_boot_optimization) { - pipe_ctx->stream->apply_edp_fast_boot_optimization = false; - pipe_ctx->stream->dpms_off = false; - return; - } + /* eDP lit up by bios already, no need to enable again. */ + if (pipe_ctx->stream->signal == SIGNAL_TYPE_EDP && + pipe_ctx->stream->apply_edp_fast_boot_optimization) { + pipe_ctx->stream->apply_edp_fast_boot_optimization = false; + pipe_ctx->stream->dpms_off = false; + return; + } - if (pipe_ctx->stream->dpms_off) - return; + if (pipe_ctx->stream->dpms_off) + return; - status = enable_link(state, pipe_ctx); + status = enable_link(state, pipe_ctx); - if (status != DC_OK) { + if (status != DC_OK) { DC_LOG_WARNING("enabling link %u failed: %d\n", pipe_ctx->stream->sink->link->link_index, status); @@ -2590,23 +2591,26 @@ void core_link_enable_stream( BREAK_TO_DEBUGGER(); return; } - } + } - core_dc->hwss.enable_audio_stream(pipe_ctx); + core_dc->hwss.enable_audio_stream(pipe_ctx); - /* turn off otg test pattern if enable */ - if (pipe_ctx->stream_res.tg->funcs->set_test_pattern) - pipe_ctx->stream_res.tg->funcs->set_test_pattern(pipe_ctx->stream_res.tg, - CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, - COLOR_DEPTH_UNDEFINED); + /* turn off otg test pattern if enable */ + if (pipe_ctx->stream_res.tg->funcs->set_test_pattern) + pipe_ctx->stream_res.tg->funcs->set_test_pattern(pipe_ctx->stream_res.tg, + CONTROLLER_DP_TEST_PATTERN_VIDEOMODE, + COLOR_DEPTH_UNDEFINED); - core_dc->hwss.enable_stream(pipe_ctx); + core_dc->hwss.enable_stream(pipe_ctx); - if (pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) - allocate_mst_payload(pipe_ctx); + if (pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) + allocate_mst_payload(pipe_ctx); + + core_dc->hwss.unblank_stream(pipe_ctx, + &pipe_ctx->stream->sink->link->cur_link_settings); + + } - core_dc->hwss.unblank_stream(pipe_ctx, - &pipe_ctx->stream->sink->link->cur_link_settings); } void core_link_disable_stream(struct pipe_ctx *pipe_ctx, int option) diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c index 2d6a4300bfa4..b6fe29b9fb65 100644 --- a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c +++ b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c @@ -1975,6 +1975,9 @@ static void calculate_phy_pix_clks(struct dc_stream_state *stream) else stream->phy_pix_clk = stream->timing.pix_clk_khz; + + if (stream->timing.timing_3d_format == TIMING_3D_FORMAT_HW_FRAME_PACKING) + stream->phy_pix_clk *= 2; } enum dc_status resource_map_pool_resources( @@ -2096,6 +2099,14 @@ enum dc_status dc_validate_global_state( if (pipe_ctx->stream != stream) continue; + if (dc->res_pool->funcs->get_default_swizzle_mode && + pipe_ctx->plane_state && + pipe_ctx->plane_state->tiling_info.gfx9.swizzle == DC_SW_UNKNOWN) { + result = dc->res_pool->funcs->get_default_swizzle_mode(pipe_ctx->plane_state); + if (result != DC_OK) + return result; + } + /* Switch to dp clock source only if there is * no non dp stream that shares the same timing * with the dp stream. @@ -2885,3 +2896,32 @@ enum dc_status dc_validate_plane(struct dc *dc, const struct dc_plane_state *pla return res; } + +unsigned int resource_pixel_format_to_bpp(enum surface_pixel_format format) +{ + switch (format) { + case SURFACE_PIXEL_FORMAT_GRPH_PALETA_256_COLORS: + return 8; + case SURFACE_PIXEL_FORMAT_VIDEO_420_YCbCr: + case SURFACE_PIXEL_FORMAT_VIDEO_420_YCrCb: + return 12; + case SURFACE_PIXEL_FORMAT_GRPH_ARGB1555: + case SURFACE_PIXEL_FORMAT_GRPH_RGB565: + case SURFACE_PIXEL_FORMAT_VIDEO_420_10bpc_YCbCr: + case SURFACE_PIXEL_FORMAT_VIDEO_420_10bpc_YCrCb: + return 16; + case SURFACE_PIXEL_FORMAT_GRPH_ARGB8888: + case SURFACE_PIXEL_FORMAT_GRPH_ABGR8888: + case SURFACE_PIXEL_FORMAT_GRPH_ARGB2101010: + case SURFACE_PIXEL_FORMAT_GRPH_ABGR2101010: + case SURFACE_PIXEL_FORMAT_GRPH_ABGR2101010_XR_BIAS: + return 32; + case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616: + case SURFACE_PIXEL_FORMAT_GRPH_ARGB16161616F: + case SURFACE_PIXEL_FORMAT_GRPH_ABGR16161616F: + return 64; + default: + ASSERT_CRITICAL(false); + return -1; + } +} diff --git a/drivers/gpu/drm/amd/display/dc/dc.h b/drivers/gpu/drm/amd/display/dc/dc.h index 7691139363a9..199527171100 100644 --- a/drivers/gpu/drm/amd/display/dc/dc.h +++ b/drivers/gpu/drm/amd/display/dc/dc.h @@ -38,13 +38,12 @@ #include "inc/compressor.h" #include "dml/display_mode_lib.h" -#define DC_VER "3.1.66" +#define DC_VER "3.1.68" #define MAX_SURFACES 3 #define MAX_STREAMS 6 #define MAX_SINKS_PER_LINK 4 - /******************************************************************************* * Display Core Interfaces ******************************************************************************/ @@ -208,6 +207,7 @@ struct dc_clocks { int dcfclk_deep_sleep_khz; int fclk_khz; int phyclk_khz; + int dramclk_khz; }; struct dc_debug_options { @@ -315,6 +315,8 @@ struct dc { struct compressor *fbc_compressor; struct dc_debug_data debug_data; + + const char *build_id; }; enum frame_buffer_mode { @@ -599,6 +601,8 @@ struct dc_validation_set { enum dc_status dc_validate_plane(struct dc *dc, const struct dc_plane_state *plane_state); +void get_clock_requirements_for_state(struct dc_state *state, struct AsicStateEx *info); + enum dc_status dc_validate_global_state( struct dc *dc, struct dc_state *new_ctx); diff --git a/drivers/gpu/drm/amd/display/dc/dc_hw_types.h b/drivers/gpu/drm/amd/display/dc/dc_hw_types.h index 57f57cf0fe2a..7825e4b5e97c 100644 --- a/drivers/gpu/drm/amd/display/dc/dc_hw_types.h +++ b/drivers/gpu/drm/amd/display/dc/dc_hw_types.h @@ -289,7 +289,8 @@ enum swizzle_mode_values { DC_SW_VAR_S_X = 29, DC_SW_VAR_D_X = 30, DC_SW_VAR_R_X = 31, - DC_SW_MAX + DC_SW_MAX = 32, + DC_SW_UNKNOWN = DC_SW_MAX }; union dc_tiling_info { diff --git a/drivers/gpu/drm/amd/display/dc/dc_link.h b/drivers/gpu/drm/amd/display/dc/dc_link.h index 438fb35d87b8..3bfdccceb524 100644 --- a/drivers/gpu/drm/amd/display/dc/dc_link.h +++ b/drivers/gpu/drm/amd/display/dc/dc_link.h @@ -216,6 +216,7 @@ void dc_link_enable_hpd_filter(struct dc_link *link, bool enable); bool dc_link_is_dp_sink_present(struct dc_link *link); +bool dc_link_detect_sink(struct dc_link *link, enum dc_connection_type *type); /* * DPCD access interfaces */ diff --git a/drivers/gpu/drm/amd/display/dc/dc_types.h b/drivers/gpu/drm/amd/display/dc/dc_types.h index 4fb62780a696..6e12d640d020 100644 --- a/drivers/gpu/drm/amd/display/dc/dc_types.h +++ b/drivers/gpu/drm/amd/display/dc/dc_types.h @@ -659,4 +659,16 @@ enum i2c_mot_mode { I2C_MOT_FALSE }; +struct AsicStateEx { + unsigned int memoryClock; + unsigned int displayClock; + unsigned int engineClock; + unsigned int maxSupportedDppClock; + unsigned int dppClock; + unsigned int socClock; + unsigned int dcfClockDeepSleep; + unsigned int fClock; + unsigned int phyClock; +}; + #endif /* DC_TYPES_H_ */ diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c b/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c index 3f5b2e6f7553..aaeb7faac0c4 100644 --- a/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c +++ b/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c @@ -312,7 +312,7 @@ static void process_channel_reply( /* in case HPD is LOW, exit AUX transaction */ if ((sw_status & AUX_SW_STATUS__AUX_SW_HPD_DISCON_MASK)) { - reply->status = AUX_CHANNEL_OPERATION_FAILED_HPD_DISCON; + reply->status = AUX_TRANSACTION_REPLY_HPD_DISCON; return; } diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_clocks.c b/drivers/gpu/drm/amd/display/dc/dce/dce_clocks.c index bf6261a1584b..d89a097ba936 100644 --- a/drivers/gpu/drm/amd/display/dc/dce/dce_clocks.c +++ b/drivers/gpu/drm/amd/display/dc/dce/dce_clocks.c @@ -468,6 +468,9 @@ static void dce12_update_clocks(struct dccg *dccg, { struct dm_pp_clock_for_voltage_req clock_voltage_req = {0}; + /* TODO: Investigate why this is needed to fix display corruption. */ + new_clocks->dispclk_khz = new_clocks->dispclk_khz * 115 / 100; + if (should_set_clock(safe_to_lower, new_clocks->dispclk_khz, dccg->clks.dispclk_khz)) { clock_voltage_req.clk_type = DM_PP_CLOCK_TYPE_DISPLAY_CLK; clock_voltage_req.clocks_in_khz = new_clocks->dispclk_khz; @@ -661,6 +664,11 @@ static void dce_update_clocks(struct dccg *dccg, bool safe_to_lower) { struct dm_pp_power_level_change_request level_change_req; + struct dce_dccg *clk_dce = TO_DCE_CLOCKS(dccg); + + /* TODO: Investigate why this is needed to fix display corruption. */ + if (!clk_dce->dfs_bypass_active) + new_clocks->dispclk_khz = new_clocks->dispclk_khz * 115 / 100; level_change_req.power_level = dce_get_required_clocks_state(dccg, new_clocks); /* get max clock state from PPLIB */ diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c index 4942590e8b9c..366bc8c2c643 100644 --- a/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dce/dce_link_encoder.c @@ -662,21 +662,10 @@ bool dce110_link_encoder_validate_dp_output( const struct dce110_link_encoder *enc110, const struct dc_crtc_timing *crtc_timing) { - /* default RGB only */ - if (crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB) - return true; - - if (enc110->base.features.flags.bits.IS_YCBCR_CAPABLE) - return true; - - /* for DCE 8.x or later DP Y-only feature, - * we need ASIC cap + FeatureSupportDPYonly, not support 666 */ - if (crtc_timing->flags.Y_ONLY && - enc110->base.features.flags.bits.IS_YCBCR_CAPABLE && - crtc_timing->display_color_depth != COLOR_DEPTH_666) - return true; + if (crtc_timing->pixel_encoding == PIXEL_ENCODING_YCBCR420) + return false; - return false; + return true; } void dce110_link_encoder_construct( diff --git a/drivers/gpu/drm/amd/display/dc/dce100/dce100_resource.c b/drivers/gpu/drm/amd/display/dc/dce100/dce100_resource.c index b1cc38827f09..14754a87156c 100644 --- a/drivers/gpu/drm/amd/display/dc/dce100/dce100_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dce100/dce100_resource.c @@ -551,8 +551,7 @@ static const struct encoder_feature_support link_enc_feature = { .max_hdmi_deep_color = COLOR_DEPTH_121212, .max_hdmi_pixel_clock = 300000, .flags.bits.IS_HBR2_CAPABLE = true, - .flags.bits.IS_TPS3_CAPABLE = true, - .flags.bits.IS_YCBCR_CAPABLE = true + .flags.bits.IS_TPS3_CAPABLE = true }; struct link_encoder *dce100_link_encoder_create( @@ -690,7 +689,9 @@ static void destruct(struct dce110_resource_pool *pool) kfree(DCE110TG_FROM_TG(pool->base.timing_generators[i])); pool->base.timing_generators[i] = NULL; } + } + for (i = 0; i < pool->base.res_cap->num_ddc; i++) { if (pool->base.engines[i] != NULL) dce110_engine_destroy(&pool->base.engines[i]); if (pool->base.hw_i2cs[i] != NULL) { diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c index dc1eed5ba996..b75ede5f84f7 100644 --- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c +++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c @@ -1377,26 +1377,13 @@ static enum dc_status apply_single_controller_ctx_to_hw( /* */ dc->hwss.enable_stream_timing(pipe_ctx, context, dc); - /* FPGA does not program backend */ - if (IS_FPGA_MAXIMUS_DC(dc->ctx->dce_environment)) { - pipe_ctx->stream_res.opp->funcs->opp_set_dyn_expansion( - pipe_ctx->stream_res.opp, - COLOR_SPACE_YCBCR601, - stream->timing.display_color_depth, - pipe_ctx->stream->signal); - - pipe_ctx->stream_res.opp->funcs->opp_program_fmt( - pipe_ctx->stream_res.opp, - &stream->bit_depth_params, - &stream->clamping); - return DC_OK; - } /* TODO: move to stream encoder */ if (pipe_ctx->stream->signal != SIGNAL_TYPE_VIRTUAL) if (DC_OK != bios_parser_crtc_source_select(pipe_ctx)) { BREAK_TO_DEBUGGER(); return DC_ERROR_UNEXPECTED; } + pipe_ctx->stream_res.opp->funcs->opp_set_dyn_expansion( pipe_ctx->stream_res.opp, COLOR_SPACE_YCBCR601, @@ -2550,7 +2537,7 @@ static void pplib_apply_display_requirements( dc->prev_display_config = *pp_display_cfg; } -void dce110_set_bandwidth( +static void dce110_set_bandwidth( struct dc *dc, struct dc_state *context, bool decrease_allowed) diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.h index e4c5db75c4c6..d6db3dbd9015 100644 --- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.h +++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.h @@ -68,11 +68,6 @@ void dce110_fill_display_configs( const struct dc_state *context, struct dm_pp_display_configuration *pp_display_cfg); -void dce110_set_bandwidth( - struct dc *dc, - struct dc_state *context, - bool decrease_allowed); - uint32_t dce110_get_min_vblank_time_us(const struct dc_state *context); void dp_receiver_power_ctrl(struct dc_link *link, bool on); diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c index b44cc7042249..e3624ca24574 100644 --- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c @@ -568,10 +568,9 @@ static struct input_pixel_processor *dce110_ipp_create( static const struct encoder_feature_support link_enc_feature = { .max_hdmi_deep_color = COLOR_DEPTH_121212, - .max_hdmi_pixel_clock = 594000, + .max_hdmi_pixel_clock = 300000, .flags.bits.IS_HBR2_CAPABLE = true, - .flags.bits.IS_TPS3_CAPABLE = true, - .flags.bits.IS_YCBCR_CAPABLE = true + .flags.bits.IS_TPS3_CAPABLE = true }; static struct link_encoder *dce110_link_encoder_create( @@ -720,7 +719,9 @@ static void destruct(struct dce110_resource_pool *pool) kfree(DCE110TG_FROM_TG(pool->base.timing_generators[i])); pool->base.timing_generators[i] = NULL; } + } + for (i = 0; i < pool->base.res_cap->num_ddc; i++) { if (pool->base.engines[i] != NULL) dce110_engine_destroy(&pool->base.engines[i]); if (pool->base.hw_i2cs[i] != NULL) { diff --git a/drivers/gpu/drm/amd/display/dc/dce112/dce112_resource.c b/drivers/gpu/drm/amd/display/dc/dce112/dce112_resource.c index 0f8332ea1160..3ce79c208ddf 100644 --- a/drivers/gpu/drm/amd/display/dc/dce112/dce112_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dce112/dce112_resource.c @@ -555,8 +555,7 @@ static const struct encoder_feature_support link_enc_feature = { .flags.bits.IS_HBR2_CAPABLE = true, .flags.bits.IS_HBR3_CAPABLE = true, .flags.bits.IS_TPS3_CAPABLE = true, - .flags.bits.IS_TPS4_CAPABLE = true, - .flags.bits.IS_YCBCR_CAPABLE = true + .flags.bits.IS_TPS4_CAPABLE = true }; struct link_encoder *dce112_link_encoder_create( @@ -694,9 +693,6 @@ static void destruct(struct dce110_resource_pool *pool) if (pool->base.opps[i] != NULL) dce110_opp_destroy(&pool->base.opps[i]); - if (pool->base.engines[i] != NULL) - dce110_engine_destroy(&pool->base.engines[i]); - if (pool->base.transforms[i] != NULL) dce112_transform_destroy(&pool->base.transforms[i]); @@ -712,6 +708,11 @@ static void destruct(struct dce110_resource_pool *pool) kfree(DCE110TG_FROM_TG(pool->base.timing_generators[i])); pool->base.timing_generators[i] = NULL; } + } + + for (i = 0; i < pool->base.res_cap->num_ddc; i++) { + if (pool->base.engines[i] != NULL) + dce110_engine_destroy(&pool->base.engines[i]); if (pool->base.hw_i2cs[i] != NULL) { kfree(pool->base.hw_i2cs[i]); pool->base.hw_i2cs[i] = NULL; diff --git a/drivers/gpu/drm/amd/display/dc/dce120/dce120_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dce120/dce120_hw_sequencer.c index 5853522a6182..eb0f5f9a973b 100644 --- a/drivers/gpu/drm/amd/display/dc/dce120/dce120_hw_sequencer.c +++ b/drivers/gpu/drm/amd/display/dc/dce120/dce120_hw_sequencer.c @@ -244,17 +244,6 @@ static void dce120_update_dchub( dh_data->dchub_info_valid = false; } -static void dce120_set_bandwidth( - struct dc *dc, - struct dc_state *context, - bool decrease_allowed) -{ - if (context->stream_count <= 0) - return; - - dce110_set_bandwidth(dc, context, decrease_allowed); -} - void dce120_hw_sequencer_construct(struct dc *dc) { /* All registers used by dce11.2 match those in dce11 in offset and @@ -263,6 +252,5 @@ void dce120_hw_sequencer_construct(struct dc *dc) dce110_hw_sequencer_construct(dc); dc->hwss.enable_display_power_gating = dce120_enable_display_power_gating; dc->hwss.update_dchub = dce120_update_dchub; - dc->hwss.set_bandwidth = dce120_set_bandwidth; } diff --git a/drivers/gpu/drm/amd/display/dc/dce120/dce120_resource.c b/drivers/gpu/drm/amd/display/dc/dce120/dce120_resource.c index 59055801af44..79ab5f9f9115 100644 --- a/drivers/gpu/drm/amd/display/dc/dce120/dce120_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dce120/dce120_resource.c @@ -533,7 +533,9 @@ static void destruct(struct dce110_resource_pool *pool) kfree(DCE110TG_FROM_TG(pool->base.timing_generators[i])); pool->base.timing_generators[i] = NULL; } + } + for (i = 0; i < pool->base.res_cap->num_ddc; i++) { if (pool->base.engines[i] != NULL) dce110_engine_destroy(&pool->base.engines[i]); if (pool->base.hw_i2cs[i] != NULL) { @@ -609,7 +611,6 @@ static const struct encoder_feature_support link_enc_feature = { .flags.bits.IS_HBR3_CAPABLE = true, .flags.bits.IS_TPS3_CAPABLE = true, .flags.bits.IS_TPS4_CAPABLE = true, - .flags.bits.IS_YCBCR_CAPABLE = true }; static struct link_encoder *dce120_link_encoder_create( diff --git a/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c b/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c index 1dc590ccc5f9..d68f951f9869 100644 --- a/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c @@ -650,8 +650,7 @@ static const struct encoder_feature_support link_enc_feature = { .max_hdmi_deep_color = COLOR_DEPTH_121212, .max_hdmi_pixel_clock = 297000, .flags.bits.IS_HBR2_CAPABLE = true, - .flags.bits.IS_TPS3_CAPABLE = true, - .flags.bits.IS_YCBCR_CAPABLE = true + .flags.bits.IS_TPS3_CAPABLE = true }; struct link_encoder *dce80_link_encoder_create( @@ -739,7 +738,9 @@ static void destruct(struct dce110_resource_pool *pool) kfree(DCE110TG_FROM_TG(pool->base.timing_generators[i])); pool->base.timing_generators[i] = NULL; } + } + for (i = 0; i < pool->base.res_cap->num_ddc; i++) { if (pool->base.engines[i] != NULL) dce110_engine_destroy(&pool->base.engines[i]); if (pool->base.hw_i2cs[i] != NULL) { diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.c index 1ea91e153d3a..4254e7e1a509 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.c +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.c @@ -87,6 +87,23 @@ void hubbub1_wm_read_state(struct hubbub *hubbub, s->dram_clk_chanage = REG_READ(DCHUBBUB_ARB_ALLOW_DRAM_CLK_CHANGE_WATERMARK_D); } +void hubbub1_disable_allow_self_refresh(struct hubbub *hubbub) +{ + REG_UPDATE(DCHUBBUB_ARB_DRAM_STATE_CNTL, + DCHUBBUB_ARB_ALLOW_SELF_REFRESH_FORCE_ENABLE, 0); +} + +bool hububu1_is_allow_self_refresh_enabled(struct hubbub *hubbub) +{ + uint32_t enable = 0; + + REG_GET(DCHUBBUB_ARB_DRAM_STATE_CNTL, + DCHUBBUB_ARB_ALLOW_SELF_REFRESH_FORCE_ENABLE, &enable); + + return true ? false : enable; +} + + bool hubbub1_verify_allow_pstate_change_high( struct hubbub *hubbub) { @@ -116,7 +133,43 @@ bool hubbub1_verify_allow_pstate_change_high( forced_pstate_allow = false; } - /* RV1: + /* RV2: + * dchubbubdebugind, at: 0xB + * description + * 0: Pipe0 Plane0 Allow Pstate Change + * 1: Pipe0 Plane1 Allow Pstate Change + * 2: Pipe0 Cursor0 Allow Pstate Change + * 3: Pipe0 Cursor1 Allow Pstate Change + * 4: Pipe1 Plane0 Allow Pstate Change + * 5: Pipe1 Plane1 Allow Pstate Change + * 6: Pipe1 Cursor0 Allow Pstate Change + * 7: Pipe1 Cursor1 Allow Pstate Change + * 8: Pipe2 Plane0 Allow Pstate Change + * 9: Pipe2 Plane1 Allow Pstate Change + * 10: Pipe2 Cursor0 Allow Pstate Change + * 11: Pipe2 Cursor1 Allow Pstate Change + * 12: Pipe3 Plane0 Allow Pstate Change + * 13: Pipe3 Plane1 Allow Pstate Change + * 14: Pipe3 Cursor0 Allow Pstate Change + * 15: Pipe3 Cursor1 Allow Pstate Change + * 16: Pipe4 Plane0 Allow Pstate Change + * 17: Pipe4 Plane1 Allow Pstate Change + * 18: Pipe4 Cursor0 Allow Pstate Change + * 19: Pipe4 Cursor1 Allow Pstate Change + * 20: Pipe5 Plane0 Allow Pstate Change + * 21: Pipe5 Plane1 Allow Pstate Change + * 22: Pipe5 Cursor0 Allow Pstate Change + * 23: Pipe5 Cursor1 Allow Pstate Change + * 24: Pipe6 Plane0 Allow Pstate Change + * 25: Pipe6 Plane1 Allow Pstate Change + * 26: Pipe6 Cursor0 Allow Pstate Change + * 27: Pipe6 Cursor1 Allow Pstate Change + * 28: WB0 Allow Pstate Change + * 29: WB1 Allow Pstate Change + * 30: Arbiter's allow_pstate_change + * 31: SOC pstate change request" + * + * RV1: * dchubbubdebugind, at: 0x7 * description "3-0: Pipe0 cursor0 QOS * 7-4: Pipe1 cursor0 QOS @@ -140,7 +193,6 @@ bool hubbub1_verify_allow_pstate_change_high( * 31: SOC pstate change request */ - REG_WRITE(DCHUBBUB_TEST_DEBUG_INDEX, hubbub->debug_test_index_pstate); for (i = 0; i < pstate_wait_timeout_us; i++) { @@ -802,5 +854,9 @@ void hubbub1_construct(struct hubbub *hubbub, hubbub->masks = hubbub_mask; hubbub->debug_test_index_pstate = 0x7; +#if defined(CONFIG_DRM_AMD_DC_DCN1_01) + if (ctx->dce_version == DCN_VERSION_1_01) + hubbub->debug_test_index_pstate = 0xB; +#endif } diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h index d6e596eef4c5..d0f03d152913 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubbub.h @@ -203,6 +203,10 @@ void hubbub1_program_watermarks( unsigned int refclk_mhz, bool safe_to_lower); +void hubbub1_disable_allow_self_refresh(struct hubbub *hubbub); + +bool hububu1_is_allow_self_refresh_enabled(struct hubbub *hubub); + void hubbub1_toggle_watermark_change_req( struct hubbub *hubbub); diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c index 6bd4ec39f869..193184affefb 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c @@ -44,6 +44,7 @@ #include "dcn10_hubp.h" #include "dcn10_hubbub.h" #include "dcn10_cm_common.h" +#include "dc_link_dp.h" #define DC_LOGGER_INIT(logger) @@ -996,7 +997,21 @@ static void dcn10_init_hw(struct dc *dc) } else { if (!dcb->funcs->is_accelerated_mode(dcb)) { + bool allow_self_fresh_force_enable = + hububu1_is_allow_self_refresh_enabled(dc->res_pool->hubbub); + bios_golden_init(dc); + + /* WA for making DF sleep when idle after resume from S0i3. + * DCHUBBUB_ARB_ALLOW_SELF_REFRESH_FORCE_ENABLE is set to 1 by + * command table, if DCHUBBUB_ARB_ALLOW_SELF_REFRESH_FORCE_ENABLE = 0 + * before calling command table and it changed to 1 after, + * it should be set back to 0. + */ + if (allow_self_fresh_force_enable == false && + hububu1_is_allow_self_refresh_enabled(dc->res_pool->hubbub)) + hubbub1_disable_allow_self_refresh(dc->res_pool->hubbub); + disable_vga(dc->hwseq); } diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c index 6f675206a136..ba6a8686062f 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_link_encoder.c @@ -606,22 +606,10 @@ bool dcn10_link_encoder_validate_dp_output( const struct dcn10_link_encoder *enc10, const struct dc_crtc_timing *crtc_timing) { - /* default RGB only */ - if (crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB) - return true; - - if (enc10->base.features.flags.bits.IS_YCBCR_CAPABLE) - return true; - - /* for DCE 8.x or later DP Y-only feature, - * we need ASIC cap + FeatureSupportDPYonly, not support 666 - */ - if (crtc_timing->flags.Y_ONLY && - enc10->base.features.flags.bits.IS_YCBCR_CAPABLE && - crtc_timing->display_color_depth != COLOR_DEPTH_666) - return true; + if (crtc_timing->pixel_encoding == PIXEL_ENCODING_YCBCR420) + return false; - return false; + return true; } void dcn10_link_encoder_construct( diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c index 411f89218e01..54626682bab2 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c @@ -98,7 +98,6 @@ static uint32_t get_start_vline(struct timing_generator *optc, const struct dc_c struct dc_crtc_timing patched_crtc_timing; int vesa_sync_start; int asic_blank_end; - int interlace_factor; int vertical_line_start; patched_crtc_timing = *dc_crtc_timing; @@ -112,16 +111,13 @@ static uint32_t get_start_vline(struct timing_generator *optc, const struct dc_c vesa_sync_start - patched_crtc_timing.h_border_left; - interlace_factor = patched_crtc_timing.flags.INTERLACE ? 2 : 1; - vesa_sync_start = patched_crtc_timing.v_addressable + patched_crtc_timing.v_border_bottom + patched_crtc_timing.v_front_porch; asic_blank_end = (patched_crtc_timing.v_total - vesa_sync_start - - patched_crtc_timing.v_border_top) - * interlace_factor; + patched_crtc_timing.v_border_top); vertical_line_start = asic_blank_end - optc->dlg_otg_param.vstartup_start + 1; if (vertical_line_start < 0) { @@ -154,7 +150,7 @@ void optc1_program_vline_interrupt( req_delta_lines--; if (req_delta_lines > vsync_line) - start_line = dc_crtc_timing->v_total - (req_delta_lines - vsync_line) - 1; + start_line = dc_crtc_timing->v_total - (req_delta_lines - vsync_line) + 2; else start_line = vsync_line - req_delta_lines; @@ -186,7 +182,6 @@ void optc1_program_timing( uint32_t v_sync_end; uint32_t v_init, v_fp2; uint32_t h_sync_polarity, v_sync_polarity; - uint32_t interlace_factor; uint32_t start_point = 0; uint32_t field_num = 0; uint32_t h_div_2; @@ -237,16 +232,8 @@ void optc1_program_timing( REG_UPDATE(OTG_H_SYNC_A_CNTL, OTG_H_SYNC_A_POL, h_sync_polarity); - /* Load vertical timing */ + v_total = patched_crtc_timing.v_total - 1; - /* CRTC_V_TOTAL = v_total - 1 */ - if (patched_crtc_timing.flags.INTERLACE) { - interlace_factor = 2; - v_total = 2 * patched_crtc_timing.v_total; - } else { - interlace_factor = 1; - v_total = patched_crtc_timing.v_total - 1; - } REG_SET(OTG_V_TOTAL, 0, OTG_V_TOTAL, v_total); @@ -259,7 +246,7 @@ void optc1_program_timing( OTG_V_TOTAL_MIN, v_total); /* v_sync_start = 0, v_sync_end = v_sync_width */ - v_sync_end = patched_crtc_timing.v_sync_width * interlace_factor; + v_sync_end = patched_crtc_timing.v_sync_width; REG_UPDATE_2(OTG_V_SYNC_A, OTG_V_SYNC_A_START, 0, @@ -271,15 +258,13 @@ void optc1_program_timing( asic_blank_end = (patched_crtc_timing.v_total - vesa_sync_start - - patched_crtc_timing.v_border_top) - * interlace_factor; + patched_crtc_timing.v_border_top); /* v_blank_start = v_blank_end + v_active */ asic_blank_start = asic_blank_end + (patched_crtc_timing.v_border_top + patched_crtc_timing.v_addressable + - patched_crtc_timing.v_border_bottom) - * interlace_factor; + patched_crtc_timing.v_border_bottom); REG_UPDATE_2(OTG_V_BLANK_START_END, OTG_V_BLANK_START, asic_blank_start, @@ -301,7 +286,7 @@ void optc1_program_timing( 0 : 1; REG_UPDATE(OTG_V_SYNC_A_CNTL, - OTG_V_SYNC_A_POL, v_sync_polarity); + OTG_V_SYNC_A_POL, v_sync_polarity); v_init = asic_blank_start; if (optc->dlg_otg_param.signal == SIGNAL_TYPE_DISPLAY_PORT || @@ -532,7 +517,6 @@ bool optc1_validate_timing( struct timing_generator *optc, const struct dc_crtc_timing *timing) { - uint32_t interlace_factor; uint32_t v_blank; uint32_t h_blank; uint32_t min_v_blank; @@ -540,10 +524,8 @@ bool optc1_validate_timing( ASSERT(timing != NULL); - interlace_factor = timing->flags.INTERLACE ? 2 : 1; v_blank = (timing->v_total - timing->v_addressable - - timing->v_border_top - timing->v_border_bottom) * - interlace_factor; + timing->v_border_top - timing->v_border_bottom); h_blank = (timing->h_total - timing->h_addressable - timing->h_border_right - diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c index cb1b134b8fcb..a71453a15ae3 100644 --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c @@ -507,6 +507,18 @@ static const struct resource_caps res_cap = { .num_ddc = 4, }; +#if defined(CONFIG_DRM_AMD_DC_DCN1_01) +static const struct resource_caps rv2_res_cap = { + .num_timing_generator = 3, + .num_opp = 3, + .num_video_plane = 3, + .num_audio = 3, + .num_stream_encoder = 3, + .num_pll = 3, + .num_ddc = 3, +}; +#endif + static const struct dc_debug_options debug_defaults_drv = { .sanity_checks = true, .disable_dmcu = true, @@ -711,8 +723,7 @@ static const struct encoder_feature_support link_enc_feature = { .flags.bits.IS_HBR2_CAPABLE = true, .flags.bits.IS_HBR3_CAPABLE = true, .flags.bits.IS_TPS3_CAPABLE = true, - .flags.bits.IS_TPS4_CAPABLE = true, - .flags.bits.IS_YCBCR_CAPABLE = true + .flags.bits.IS_TPS4_CAPABLE = true }; struct link_encoder *dcn10_link_encoder_create( @@ -897,7 +908,9 @@ static void destruct(struct dcn10_resource_pool *pool) kfree(DCN10TG_FROM_TG(pool->base.timing_generators[i])); pool->base.timing_generators[i] = NULL; } + } + for (i = 0; i < pool->base.res_cap->num_ddc; i++) { if (pool->base.engines[i] != NULL) pool->base.engines[i]->funcs->destroy_engine(&pool->base.engines[i]); if (pool->base.hw_i2cs[i] != NULL) { @@ -1119,6 +1132,24 @@ static enum dc_status dcn10_validate_plane(const struct dc_plane_state *plane_st return DC_OK; } +static enum dc_status dcn10_get_default_swizzle_mode(struct dc_plane_state *plane_state) +{ + enum dc_status result = DC_OK; + + enum surface_pixel_format surf_pix_format = plane_state->format; + unsigned int bpp = resource_pixel_format_to_bpp(surf_pix_format); + + enum swizzle_mode_values swizzle = DC_SW_LINEAR; + + if (bpp == 64) + swizzle = DC_SW_64KB_D; + else + swizzle = DC_SW_64KB_S; + + plane_state->tiling_info.gfx9.swizzle = swizzle; + return result; +} + static const struct dc_cap_funcs cap_funcs = { .get_dcc_compression_cap = dcn10_get_dcc_compression_cap }; @@ -1129,7 +1160,8 @@ static const struct resource_funcs dcn10_res_pool_funcs = { .validate_bandwidth = dcn_validate_bandwidth, .acquire_idle_pipe_for_layer = dcn10_acquire_idle_pipe_for_layer, .validate_plane = dcn10_validate_plane, - .add_stream_to_ctx = dcn10_add_stream_to_ctx + .add_stream_to_ctx = dcn10_add_stream_to_ctx, + .get_default_swizzle_mode = dcn10_get_default_swizzle_mode }; static uint32_t read_pipe_fuses(struct dc_context *ctx) @@ -1152,7 +1184,12 @@ static bool construct( ctx->dc_bios->regs = &bios_regs; - pool->base.res_cap = &res_cap; +#if defined(CONFIG_DRM_AMD_DC_DCN1_01) + if (ctx->dce_version == DCN_VERSION_1_01) + pool->base.res_cap = &rv2_res_cap; + else +#endif + pool->base.res_cap = &res_cap; pool->base.funcs = &dcn10_res_pool_funcs; /* diff --git a/drivers/gpu/drm/amd/display/dc/i2caux/dce110/aux_engine_dce110.c b/drivers/gpu/drm/amd/display/dc/i2caux/dce110/aux_engine_dce110.c index 8eee8ace1259..59c3ed43d609 100644 --- a/drivers/gpu/drm/amd/display/dc/i2caux/dce110/aux_engine_dce110.c +++ b/drivers/gpu/drm/amd/display/dc/i2caux/dce110/aux_engine_dce110.c @@ -346,7 +346,7 @@ static void process_channel_reply( /* in case HPD is LOW, exit AUX transaction */ if ((sw_status & AUX_SW_STATUS__AUX_SW_HPD_DISCON_MASK)) { - reply->status = AUX_CHANNEL_OPERATION_FAILED_HPD_DISCON; + reply->status = AUX_TRANSACTION_REPLY_HPD_DISCON; return; } diff --git a/drivers/gpu/drm/amd/display/dc/inc/core_types.h b/drivers/gpu/drm/amd/display/dc/inc/core_types.h index 831a1bdf622c..c1976c175b57 100644 --- a/drivers/gpu/drm/amd/display/dc/inc/core_types.h +++ b/drivers/gpu/drm/amd/display/dc/inc/core_types.h @@ -120,6 +120,9 @@ struct resource_funcs { struct dc *dc, struct dc_state *new_ctx, struct dc_stream_state *stream); + enum dc_status (*get_default_swizzle_mode)( + struct dc_plane_state *plane_state); + }; struct audio_support{ diff --git a/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h b/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h index ddbb673caa08..e688eb9b975c 100644 --- a/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h +++ b/drivers/gpu/drm/amd/display/dc/inc/dcn_calcs.h @@ -504,10 +504,10 @@ struct dcn_bw_internal_vars { float prefetch_mode; float dstx_after_scaler; float dsty_after_scaler; - float v_update_offset_pix; + float v_update_offset_pix[number_of_planes_minus_one + 1]; float total_repeater_delay_time; - float v_update_width_pix; - float v_ready_offset_pix; + float v_update_width_pix[number_of_planes_minus_one + 1]; + float v_ready_offset_pix[number_of_planes_minus_one + 1]; float t_setup; float t_wait; float bandwidth_available_for_immediate_flip; diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h b/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h index 58818920ed41..e28e9770e0a3 100644 --- a/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h +++ b/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h @@ -58,7 +58,6 @@ struct encoder_feature_support { uint32_t IS_HBR3_CAPABLE:1; uint32_t IS_TPS3_CAPABLE:1; uint32_t IS_TPS4_CAPABLE:1; - uint32_t IS_YCBCR_CAPABLE:1; uint32_t HDMI_6GB_EN:1; } bits; uint32_t raw; diff --git a/drivers/gpu/drm/amd/display/dc/inc/resource.h b/drivers/gpu/drm/amd/display/dc/inc/resource.h index 76d00c6dbca9..33b99e3ab10d 100644 --- a/drivers/gpu/drm/amd/display/dc/inc/resource.h +++ b/drivers/gpu/drm/amd/display/dc/inc/resource.h @@ -172,4 +172,7 @@ void update_audio_usage( const struct resource_pool *pool, struct audio *audio, bool acquired); + +unsigned int resource_pixel_format_to_bpp(enum surface_pixel_format format); + #endif /* DRIVERS_GPU_DRM_AMD_DC_DEV_DC_INC_RESOURCE_H_ */ diff --git a/drivers/gpu/drm/amd/display/include/signal_types.h b/drivers/gpu/drm/amd/display/include/signal_types.h index 03476b142d8e..f56d2891475f 100644 --- a/drivers/gpu/drm/amd/display/include/signal_types.h +++ b/drivers/gpu/drm/amd/display/include/signal_types.h @@ -102,4 +102,9 @@ static inline bool dc_is_audio_capable_signal(enum signal_type signal) dc_is_hdmi_signal(signal)); } +static inline bool dc_is_virtual_signal(enum signal_type signal) +{ + return (signal == SIGNAL_TYPE_VIRTUAL); +} + #endif diff --git a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c index 15427f4fc990..cdcefd087487 100644 --- a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c +++ b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c @@ -1069,10 +1069,14 @@ static void build_evenly_distributed_points( struct dividers dividers) { struct gamma_pixel *p = points; - struct gamma_pixel *p_last = p + numberof_points - 1; + struct gamma_pixel *p_last; uint32_t i = 0; + // This function should not gets called with 0 as a parameter + ASSERT(numberof_points > 0); + p_last = p + numberof_points - 1; + do { struct fixed31_32 value = dc_fixpt_from_fraction(i, numberof_points - 1); @@ -1083,7 +1087,7 @@ static void build_evenly_distributed_points( ++p; ++i; - } while (i != numberof_points); + } while (i < numberof_points); p->r = dc_fixpt_div(p_last->r, dividers.divider1); p->g = dc_fixpt_div(p_last->g, dividers.divider1); diff --git a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c index e1688902a1b0..4018c7180d00 100644 --- a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c +++ b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c @@ -480,22 +480,11 @@ bool mod_freesync_get_v_position(struct mod_freesync *mod_freesync, return false; } -void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync, - const struct dc_stream_state *stream, - const struct mod_vrr_params *vrr, - struct dc_info_packet *infopacket) +static void build_vrr_infopacket_header_v1(enum signal_type signal, + struct dc_info_packet *infopacket, + unsigned int *payload_size) { - /* SPD info packet for FreeSync */ - unsigned char checksum = 0; - unsigned int idx, payload_size = 0; - - /* Check if Freesync is supported. Return if false. If true, - * set the corresponding bit in the info packet - */ - if (!vrr->supported || !vrr->send_vsif) - return; - - if (dc_is_hdmi_signal(stream->signal)) { + if (dc_is_hdmi_signal(signal)) { /* HEADER */ @@ -510,9 +499,9 @@ void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync, /* HB2 = [Bits 7:5 = 0] [Bits 4:0 = Length = 0x08] */ infopacket->hb2 = 0x08; - payload_size = 0x08; + *payload_size = 0x08; - } else if (dc_is_dp_signal(stream->signal)) { + } else if (dc_is_dp_signal(signal)) { /* HEADER */ @@ -536,9 +525,62 @@ void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync, */ infopacket->hb3 = 0x04; - payload_size = 0x1B; + *payload_size = 0x1B; } +} + +static void build_vrr_infopacket_header_v2(enum signal_type signal, + struct dc_info_packet *infopacket, + unsigned int *payload_size) +{ + if (dc_is_hdmi_signal(signal)) { + + /* HEADER */ + + /* HB0 = Packet Type = 0x83 (Source Product + * Descriptor InfoFrame) + */ + infopacket->hb0 = DC_HDMI_INFOFRAME_TYPE_SPD; + + /* HB1 = Version = 0x02 */ + infopacket->hb1 = 0x02; + + /* HB2 = [Bits 7:5 = 0] [Bits 4:0 = Length = 0x09] */ + infopacket->hb2 = 0x09; + + *payload_size = 0x0A; + } else if (dc_is_dp_signal(signal)) { + + /* HEADER */ + + /* HB0 = Secondary-data Packet ID = 0 - Only non-zero + * when used to associate audio related info packets + */ + infopacket->hb0 = 0x00; + + /* HB1 = Packet Type = 0x83 (Source Product + * Descriptor InfoFrame) + */ + infopacket->hb1 = DC_HDMI_INFOFRAME_TYPE_SPD; + + /* HB2 = [Bits 7:0 = Least significant eight bits - + * For INFOFRAME, the value must be 1Bh] + */ + infopacket->hb2 = 0x1B; + + /* HB3 = [Bits 7:2 = INFOFRAME SDP Version Number = 0x2] + * [Bits 1:0 = Most significant two bits = 0x00] + */ + infopacket->hb3 = 0x08; + + *payload_size = 0x1B; + } +} + +static void build_vrr_infopacket_data(const struct mod_vrr_params *vrr, + struct dc_info_packet *infopacket) +{ /* PB1 = 0x1A (24bit AMD IEEE OUI (0x00001A) - Byte 0) */ infopacket->sb[1] = 0x1A; @@ -576,15 +618,39 @@ void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync, */ infopacket->sb[8] = (unsigned char)(vrr->max_refresh_in_uhz / 1000000); - /* PB9 - PB27 = Reserved */ + //FreeSync HDR + infopacket->sb[9] = 0; + infopacket->sb[10] = 0; +} + +static void build_vrr_infopacket_fs2_data(enum color_transfer_func app_tf, + struct dc_info_packet *infopacket) +{ + if (app_tf != transfer_func_unknown) { + infopacket->valid = true; + + infopacket->sb[6] |= 0x08; // PB6 = [Bit 3 = Native Color Active] + + if (app_tf == transfer_func_gamma_22) { + infopacket->sb[9] |= 0x04; // PB6 = [Bit 2 = Gamma 2.2 EOTF Active] + } + } +} + +static void build_vrr_infopacket_checksum(unsigned int *payload_size, + struct dc_info_packet *infopacket) +{ /* Calculate checksum */ + unsigned int idx = 0; + unsigned char checksum = 0; + checksum += infopacket->hb0; checksum += infopacket->hb1; checksum += infopacket->hb2; checksum += infopacket->hb3; - for (idx = 1; idx <= payload_size; idx++) + for (idx = 1; idx <= *payload_size; idx++) checksum += infopacket->sb[idx]; /* PB0 = Checksum (one byte complement) */ @@ -593,6 +659,64 @@ void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync, infopacket->valid = true; } +static void build_vrr_infopacket_v1(enum signal_type signal, + const struct mod_vrr_params *vrr, + struct dc_info_packet *infopacket) +{ + /* SPD info packet for FreeSync */ + unsigned int payload_size = 0; + + build_vrr_infopacket_header_v1(signal, infopacket, &payload_size); + build_vrr_infopacket_data(vrr, infopacket); + build_vrr_infopacket_checksum(&payload_size, infopacket); + + infopacket->valid = true; +} + +static void build_vrr_infopacket_v2(enum signal_type signal, + const struct mod_vrr_params *vrr, + const enum color_transfer_func *app_tf, + struct dc_info_packet *infopacket) +{ + unsigned int payload_size = 0; + + build_vrr_infopacket_header_v2(signal, infopacket, &payload_size); + build_vrr_infopacket_data(vrr, infopacket); + + if (app_tf != NULL) + build_vrr_infopacket_fs2_data(*app_tf, infopacket); + + build_vrr_infopacket_checksum(&payload_size, infopacket); + + infopacket->valid = true; +} + +void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync, + const struct dc_stream_state *stream, + const struct mod_vrr_params *vrr, + enum vrr_packet_type packet_type, + const enum color_transfer_func *app_tf, + struct dc_info_packet *infopacket) +{ + /* SPD info packet for FreeSync */ + + /* Check if Freesync is supported. Return if false. If true, + * set the corresponding bit in the info packet + */ + if (!vrr->supported || !vrr->send_vsif) + return; + + switch (packet_type) { + case packet_type_fs2: + build_vrr_infopacket_v2(stream->signal, vrr, app_tf, infopacket); + break; + case packet_type_vrr: + case packet_type_fs1: + default: + build_vrr_infopacket_v1(stream->signal, vrr, infopacket); + } +} + void mod_freesync_build_vrr_params(struct mod_freesync *mod_freesync, const struct dc_stream_state *stream, struct mod_freesync_config *in_config, diff --git a/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h b/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h index a0f32cde721c..949a8b62aa98 100644 --- a/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h +++ b/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h @@ -54,7 +54,7 @@ #ifndef MOD_FREESYNC_H_ #define MOD_FREESYNC_H_ -#include "dm_services.h" +#include "mod_shared.h" // Access structures struct mod_freesync { @@ -144,6 +144,8 @@ void mod_freesync_get_settings(struct mod_freesync *mod_freesync, void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync, const struct dc_stream_state *stream, const struct mod_vrr_params *vrr, + enum vrr_packet_type packet_type, + const enum color_transfer_func *app_tf, struct dc_info_packet *infopacket); void mod_freesync_build_vrr_params(struct mod_freesync *mod_freesync, diff --git a/drivers/gpu/drm/amd/display/modules/inc/mod_shared.h b/drivers/gpu/drm/amd/display/modules/inc/mod_shared.h new file mode 100644 index 000000000000..238c431ae483 --- /dev/null +++ b/drivers/gpu/drm/amd/display/modules/inc/mod_shared.h @@ -0,0 +1,49 @@ +/* + * Copyright 2016 Advanced Micro Devices, Inc. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: AMD + * + */ + + +#ifndef MOD_SHARED_H_ +#define MOD_SHARED_H_ + +enum color_transfer_func { + transfer_func_unknown, + transfer_func_srgb, + transfer_func_bt709, + transfer_func_pq2084, + transfer_func_pq2084_interim, + transfer_func_linear_0_1, + transfer_func_linear_0_125, + transfer_func_dolbyvision, + transfer_func_gamma_22, + transfer_func_gamma_26 +}; + +enum vrr_packet_type { + packet_type_vrr, + packet_type_fs1, + packet_type_fs2 +}; + +#endif /* MOD_SHARED_H_ */ diff --git a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c index 52378fc69079..ff8bfb9b43b0 100644 --- a/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c +++ b/drivers/gpu/drm/amd/display/modules/info_packet/info_packet.c @@ -48,9 +48,12 @@ static void mod_build_vsc_infopacket(const struct dc_stream_state *stream, unsigned int i; unsigned int pixelEncoding = 0; unsigned int colorimetryFormat = 0; + bool stereo3dSupport = false; - if (stream->timing.timing_3d_format != TIMING_3D_FORMAT_NONE && stream->view_format != VIEW_3D_FORMAT_NONE) + if (stream->timing.timing_3d_format != TIMING_3D_FORMAT_NONE && stream->view_format != VIEW_3D_FORMAT_NONE) { vscPacketRevision = 1; + stereo3dSupport = true; + } /*VSC packet set to 2 when DP revision >= 1.2*/ if (stream->psr_version != 0) @@ -94,12 +97,59 @@ static void mod_build_vsc_infopacket(const struct dc_stream_state *stream, info_packet->hb2 = 0x01; // 01h = Revision number. VSC SDP supporting 3D stereo only info_packet->hb3 = 0x01; // 01h = VSC SDP supporting 3D stereo only (HB2 = 01h). - if (stream->timing.timing_3d_format == TIMING_3D_FORMAT_INBAND_FA) - info_packet->sb[0] = 0x1; - info_packet->valid = true; } + if (stereo3dSupport) { + /* ==============================================================================================================| + * A. STEREO 3D + * ==============================================================================================================| + * VSC Payload (1 byte) From DP1.2 spec + * + * Bits 3:0 (Stereo Interface Method Code) | Bits 7:4 (Stereo Interface Method Specific Parameter) + * ----------------------------------------------------------------------------------------------------- + * 0 = Non Stereo Video | Must be set to 0x0 + * ----------------------------------------------------------------------------------------------------- + * 1 = Frame/Field Sequential | 0x0: L + R view indication based on MISC1 bit 2:1 + * | 0x1: Right when Stereo Signal = 1 + * | 0x2: Left when Stereo Signal = 1 + * | (others reserved) + * ----------------------------------------------------------------------------------------------------- + * 2 = Stacked Frame | 0x0: Left view is on top and right view on bottom + * | (others reserved) + * ----------------------------------------------------------------------------------------------------- + * 3 = Pixel Interleaved | 0x0: horiz interleaved, right view pixels on even lines + * | 0x1: horiz interleaved, right view pixels on odd lines + * | 0x2: checker board, start with left view pixel + * | 0x3: vertical interleaved, start with left view pixels + * | 0x4: vertical interleaved, start with right view pixels + * | (others reserved) + * ----------------------------------------------------------------------------------------------------- + * 4 = Side-by-side | 0x0: left half represents left eye view + * | 0x1: left half represents right eye view + */ + switch (stream->timing.timing_3d_format) { + case TIMING_3D_FORMAT_HW_FRAME_PACKING: + case TIMING_3D_FORMAT_SW_FRAME_PACKING: + case TIMING_3D_FORMAT_TOP_AND_BOTTOM: + case TIMING_3D_FORMAT_TB_SW_PACKED: + info_packet->sb[0] = 0x02; // Stacked Frame, Left view is on top and right view on bottom. + break; + case TIMING_3D_FORMAT_DP_HDMI_INBAND_FA: + case TIMING_3D_FORMAT_INBAND_FA: + info_packet->sb[0] = 0x01; // Frame/Field Sequential, L + R view indication based on MISC1 bit 2:1 + break; + case TIMING_3D_FORMAT_SIDE_BY_SIDE: + case TIMING_3D_FORMAT_SBS_SW_PACKED: + info_packet->sb[0] = 0x04; // Side-by-side + break; + default: + info_packet->sb[0] = 0x00; // No Stereo Video, Shall be cleared to 0x0. + break; + } + + } + /* 05h = VSC SDP supporting 3D stereo, PSR2, and Pixel Encoding/Colorimetry Format indication. * Added in DP1.3, a DP Source device is allowed to indicate the pixel encoding/colorimetry * format to the DP Sink device with VSC SDP only when the DP Sink device supports it diff --git a/drivers/gpu/drm/amd/display/modules/stats/stats.c b/drivers/gpu/drm/amd/display/modules/stats/stats.c index 3d4c1b1ab8c4..03121ca64fe4 100644 --- a/drivers/gpu/drm/amd/display/modules/stats/stats.c +++ b/drivers/gpu/drm/amd/display/modules/stats/stats.c @@ -186,12 +186,8 @@ void mod_stats_destroy(struct mod_stats *mod_stats) if (mod_stats != NULL) { struct core_stats *core_stats = MOD_STATS_TO_CORE(mod_stats); - if (core_stats->time != NULL) - kfree(core_stats->time); - - if (core_stats->events != NULL) - kfree(core_stats->events); - + kfree(core_stats->time); + kfree(core_stats->events); kfree(core_stats); } } diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 86b167ec9863..2083c308007c 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -109,6 +109,7 @@ enum amd_powergating_state { #define AMD_PG_SUPPORT_GFX_PIPELINE (1 << 12) #define AMD_PG_SUPPORT_MMHUB (1 << 13) #define AMD_PG_SUPPORT_VCN (1 << 14) +#define AMD_PG_SUPPORT_VCN_DPG (1 << 15) enum PP_FEATURE_MASK { PP_SCLK_DPM_MASK = 0x1, diff --git a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_0_offset.h index 4ce090db7ef7..529b37db274c 100644 --- a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_0_offset.h +++ b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_0_offset.h @@ -2449,6 +2449,8 @@ #define mmCP_ECC_FIRSTOCCURRENCE_RING2_BASE_IDX 0 #define mmGB_EDC_MODE 0x107e #define mmGB_EDC_MODE_BASE_IDX 0 +#define mmCP_DEBUG 0x107f +#define mmCP_DEBUG_BASE_IDX 0 #define mmCP_CPF_DEBUG 0x1080 #define mmCP_PQ_WPTR_POLL_CNTL 0x1083 #define mmCP_PQ_WPTR_POLL_CNTL_BASE_IDX 0 diff --git a/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_offset.h index efd2704d0f8f..0d6891095f62 100644 --- a/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_offset.h +++ b/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_offset.h @@ -175,4 +175,7 @@ #define mmSMUSVI0_PLANE0_CURRENTVID_BASE_IDX 0 #define mmSMUSVI0_PLANE0_CURRENTVID 0x0013 +#define mmSMUSVI0_TEL_PLANE0_BASE_IDX 0 +#define mmSMUSVI0_TEL_PLANE0 0x0004 + #endif diff --git a/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_sh_mask.h index 2487ab9621e9..b1d9d8be1119 100644 --- a/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_sh_mask.h +++ b/drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_9_0_sh_mask.h @@ -258,4 +258,7 @@ #define SMUSVI0_PLANE0_CURRENTVID__CURRENT_SVI0_PLANE0_VID__SHIFT 0x18 #define SMUSVI0_PLANE0_CURRENTVID__CURRENT_SVI0_PLANE0_VID_MASK 0xFF000000L +#define SMUSVI0_TEL_PLANE0__SVI0_PLANE0_VDDCOR__SHIFT 0x10 +#define SMUSVI0_TEL_PLANE0__SVI0_PLANE0_VDDCOR_MASK 0x01FF0000L + #endif diff --git a/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_offset.h b/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_offset.h index 510ec3c70626..a9eb57a53e59 100644 --- a/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_offset.h +++ b/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_offset.h @@ -26,6 +26,18 @@ #define mmCG_MULT_THERMAL_STATUS 0x005f #define mmCG_MULT_THERMAL_STATUS_BASE_IDX 0 +#define mmCG_FDO_CTRL0 0x0067 +#define mmCG_FDO_CTRL0_BASE_IDX 0 + +#define mmCG_FDO_CTRL1 0x0068 +#define mmCG_FDO_CTRL1_BASE_IDX 0 + +#define mmCG_FDO_CTRL2 0x0069 +#define mmCG_FDO_CTRL2_BASE_IDX 0 + +#define mmCG_TACH_CTRL 0x006a +#define mmCG_TACH_CTRL_BASE_IDX 0 + #define mmTHM_THERMAL_INT_ENA 0x000a #define mmTHM_THERMAL_INT_ENA_BASE_IDX 0 #define mmTHM_THERMAL_INT_CTRL 0x000b diff --git a/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_sh_mask.h index f69533fa6abf..d130d92aee19 100644 --- a/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_sh_mask.h +++ b/drivers/gpu/drm/amd/include/asic_reg/thm/thm_11_0_2_sh_mask.h @@ -28,6 +28,16 @@ #define CG_MULT_THERMAL_STATUS__CTF_TEMP__SHIFT 0x9 #define CG_MULT_THERMAL_STATUS__ASIC_MAX_TEMP_MASK 0x000001FFL #define CG_MULT_THERMAL_STATUS__CTF_TEMP_MASK 0x0003FE00L +#define CG_FDO_CTRL2__TMIN__SHIFT 0x0 +#define CG_FDO_CTRL2__TMIN_MASK 0x000000FFL +#define CG_FDO_CTRL2__FDO_PWM_MODE__SHIFT 0xb +#define CG_FDO_CTRL2__FDO_PWM_MODE_MASK 0x00003800L +#define CG_FDO_CTRL1__FMAX_DUTY100__SHIFT 0x0 +#define CG_FDO_CTRL1__FMAX_DUTY100_MASK 0x000000FFL +#define CG_FDO_CTRL0__FDO_STATIC_DUTY__SHIFT 0x0 +#define CG_FDO_CTRL0__FDO_STATIC_DUTY_MASK 0x000000FFL +#define CG_TACH_CTRL__TARGET_PERIOD__SHIFT 0x3 +#define CG_TACH_CTRL__TARGET_PERIOD_MASK 0xFFFFFFF8L //THM_THERMAL_INT_ENA #define THM_THERMAL_INT_ENA__THERM_INTH_SET__SHIFT 0x0 diff --git a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_offset.h index 216a401028de..442ca7c471a5 100644 --- a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_offset.h +++ b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_offset.h @@ -33,6 +33,14 @@ #define mmUVD_POWER_STATUS_BASE_IDX 1 #define mmCC_UVD_HARVESTING 0x00c7 #define mmCC_UVD_HARVESTING_BASE_IDX 1 +#define mmUVD_DPG_LMA_CTL 0x00d1 +#define mmUVD_DPG_LMA_CTL_BASE_IDX 1 +#define mmUVD_DPG_LMA_DATA 0x00d2 +#define mmUVD_DPG_LMA_DATA_BASE_IDX 1 +#define mmUVD_DPG_LMA_MASK 0x00d3 +#define mmUVD_DPG_LMA_MASK_BASE_IDX 1 +#define mmUVD_DPG_PAUSE 0x00d4 +#define mmUVD_DPG_PAUSE_BASE_IDX 1 #define mmUVD_SCRATCH1 0x00d5 #define mmUVD_SCRATCH1_BASE_IDX 1 #define mmUVD_SCRATCH2 0x00d6 @@ -74,6 +82,18 @@ #define mmUVD_LCM_CGC_CNTRL 0x0123 #define mmUVD_LCM_CGC_CNTRL_BASE_IDX 1 +#define mmUVD_MIF_CURR_UV_ADDR_CONFIG 0x0184 +#define mmUVD_MIF_CURR_UV_ADDR_CONFIG_BASE_IDX 1 +#define mmUVD_MIF_REF_UV_ADDR_CONFIG 0x0185 +#define mmUVD_MIF_REF_UV_ADDR_CONFIG_BASE_IDX 1 +#define mmUVD_MIF_RECON1_UV_ADDR_CONFIG 0x0186 +#define mmUVD_MIF_RECON1_UV_ADDR_CONFIG_BASE_IDX 1 +#define mmUVD_MIF_CURR_ADDR_CONFIG 0x0192 +#define mmUVD_MIF_CURR_ADDR_CONFIG_BASE_IDX 1 +#define mmUVD_MIF_REF_ADDR_CONFIG 0x0193 +#define mmUVD_MIF_REF_ADDR_CONFIG_BASE_IDX 1 +#define mmUVD_MIF_RECON1_ADDR_CONFIG 0x01c5 +#define mmUVD_MIF_RECON1_ADDR_CONFIG_BASE_IDX 1 // addressBlock: uvd_uvdnpdec // base address: 0x20000 @@ -319,6 +339,8 @@ #define mmUVD_LMI_VM_CTRL_BASE_IDX 1 #define mmUVD_LMI_SWAP_CNTL 0x056d #define mmUVD_LMI_SWAP_CNTL_BASE_IDX 1 +#define mmUVD_MPC_CNTL 0x0577 +#define mmUVD_MPC_CNTL_BASE_IDX 1 #define mmUVD_MPC_SET_MUXA0 0x0579 #define mmUVD_MPC_SET_MUXA0_BASE_IDX 1 #define mmUVD_MPC_SET_MUXA1 0x057a diff --git a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_sh_mask.h index 124383dac284..63457f9df4c5 100644 --- a/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_sh_mask.h +++ b/drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_1_0_sh_mask.h @@ -87,6 +87,26 @@ //CC_UVD_HARVESTING #define CC_UVD_HARVESTING__UVD_DISABLE__SHIFT 0x1 #define CC_UVD_HARVESTING__UVD_DISABLE_MASK 0x00000002L +//UVD_DPG_LMA_CTL +#define UVD_DPG_LMA_CTL__READ_WRITE__SHIFT 0x0 +#define UVD_DPG_LMA_CTL__MASK_EN__SHIFT 0x1 +#define UVD_DPG_LMA_CTL__ADDR_AUTO_INCREMENT__SHIFT 0x2 +#define UVD_DPG_LMA_CTL__SRAM_SEL__SHIFT 0x4 +#define UVD_DPG_LMA_CTL__READ_WRITE_ADDR__SHIFT 0x10 +#define UVD_DPG_LMA_CTL__READ_WRITE_MASK 0x00000001L +#define UVD_DPG_LMA_CTL__MASK_EN_MASK 0x00000002L +#define UVD_DPG_LMA_CTL__ADDR_AUTO_INCREMENT_MASK 0x00000004L +#define UVD_DPG_LMA_CTL__SRAM_SEL_MASK 0x00000010L +#define UVD_DPG_LMA_CTL__READ_WRITE_ADDR_MASK 0xFFFF0000L +//UVD_DPG_PAUSE +#define UVD_DPG_PAUSE__JPEG_PAUSE_DPG_REQ__SHIFT 0x0 +#define UVD_DPG_PAUSE__JPEG_PAUSE_DPG_ACK__SHIFT 0x1 +#define UVD_DPG_PAUSE__NJ_PAUSE_DPG_REQ__SHIFT 0x2 +#define UVD_DPG_PAUSE__NJ_PAUSE_DPG_ACK__SHIFT 0x3 +#define UVD_DPG_PAUSE__JPEG_PAUSE_DPG_REQ_MASK 0x00000001L +#define UVD_DPG_PAUSE__JPEG_PAUSE_DPG_ACK_MASK 0x00000002L +#define UVD_DPG_PAUSE__NJ_PAUSE_DPG_REQ_MASK 0x00000004L +#define UVD_DPG_PAUSE__NJ_PAUSE_DPG_ACK_MASK 0x00000008L //UVD_SCRATCH1 #define UVD_SCRATCH1__SCRATCH1_DATA__SHIFT 0x0 #define UVD_SCRATCH1__SCRATCH1_DATA_MASK 0xFFFFFFFFL @@ -965,6 +985,7 @@ #define UVD_LMI_CTRL2__STALL_ARB_UMC__SHIFT 0x8 #define UVD_LMI_CTRL2__MC_READ_ID_SEL__SHIFT 0x9 #define UVD_LMI_CTRL2__MC_WRITE_ID_SEL__SHIFT 0xb +#define UVD_LMI_CTRL2__RE_OFLD_MIF_WR_REQ_NUM__SHIFT 0x11 #define UVD_LMI_CTRL2__SPH_DIS_MASK 0x00000001L #define UVD_LMI_CTRL2__STALL_ARB_MASK 0x00000002L #define UVD_LMI_CTRL2__ASSERT_UMC_URGENT_MASK 0x00000004L @@ -973,6 +994,7 @@ #define UVD_LMI_CTRL2__STALL_ARB_UMC_MASK 0x00000100L #define UVD_LMI_CTRL2__MC_READ_ID_SEL_MASK 0x00000600L #define UVD_LMI_CTRL2__MC_WRITE_ID_SEL_MASK 0x00001800L +#define UVD_LMI_CTRL2__RE_OFLD_MIF_WR_REQ_NUM_MASK 0x01FE0000L //UVD_MASTINT_EN #define UVD_MASTINT_EN__OVERRUN_RST__SHIFT 0x0 #define UVD_MASTINT_EN__VCPU_EN__SHIFT 0x1 @@ -983,6 +1005,7 @@ #define UVD_MASTINT_EN__SYS_EN_MASK 0x00000004L #define UVD_MASTINT_EN__INT_OVERRUN_MASK 0x007FFFF0L //UVD_SYS_INT_EN +#define UVD_SYS_INT_EN__UVD_JRBC_EN__SHIFT 0x4 #define UVD_SYS_INT_EN__UVD_JRBC_EN_MASK 0x00000010L //JPEG_CGC_CTRL #define JPEG_CGC_CTRL__DYN_CLOCK_MODE__SHIFT 0x0 @@ -1024,6 +1047,19 @@ #define UVD_LMI_CTRL__DB_IT_DATA_COHERENCY_EN_MASK 0x01000000L #define UVD_LMI_CTRL__IT_IT_DATA_COHERENCY_EN_MASK 0x02000000L #define UVD_LMI_CTRL__RFU_MASK 0xF8000000L +//UVD_LMI_STATUS +#define UVD_LMI_STATUS__READ_CLEAN__SHIFT 0x0 +#define UVD_LMI_STATUS__WRITE_CLEAN__SHIFT 0x1 +#define UVD_LMI_STATUS__WRITE_CLEAN_RAW__SHIFT 0x2 +#define UVD_LMI_STATUS__VCPU_LMI_WRITE_CLEAN__SHIFT 0x3 +#define UVD_LMI_STATUS__UMC_WRITE_CLEAN_RAW__SHIFT 0x6 +#define UVD_LMI_STATUS__UMC_READ_CLEAN_RAW__SHIFT 0x9 +#define UVD_LMI_STATUS__READ_CLEAN_MASK 0x00000001L +#define UVD_LMI_STATUS__WRITE_CLEAN_MASK 0x00000002L +#define UVD_LMI_STATUS__WRITE_CLEAN_RAW_MASK 0x00000004L +#define UVD_LMI_STATUS__VCPU_LMI_WRITE_CLEAN_MASK 0x00000008L +#define UVD_LMI_STATUS__UMC_WRITE_CLEAN_RAW_MASK 0x00000040L +#define UVD_LMI_STATUS__UMC_READ_CLEAN_RAW_MASK 0x00000200L //UVD_LMI_SWAP_CNTL #define UVD_LMI_SWAP_CNTL__RB_MC_SWAP__SHIFT 0x0 #define UVD_LMI_SWAP_CNTL__IB_MC_SWAP__SHIFT 0x2 @@ -1057,6 +1093,9 @@ #define UVD_LMI_SWAP_CNTL__RB_WR_MC_SWAP_MASK 0x0C000000L #define UVD_LMI_SWAP_CNTL__RE_MC_SWAP_MASK 0x30000000L #define UVD_LMI_SWAP_CNTL__MP_MC_SWAP_MASK 0xC0000000L +//UVD_MPC_CNTL +#define UVD_MPC_CNTL__REPLACEMENT_MODE__SHIFT 0x3 +#define UVD_MPC_CNTL__REPLACEMENT_MODE_MASK 0x00000038L //UVD_MPC_SET_MUXA0 #define UVD_MPC_SET_MUXA0__VARA_0__SHIFT 0x0 #define UVD_MPC_SET_MUXA0__VARA_1__SHIFT 0x6 @@ -1138,7 +1177,11 @@ #define UVD_VCPU_CACHE_SIZE2__CACHE_SIZE2_MASK 0x001FFFFFL //UVD_VCPU_CNTL #define UVD_VCPU_CNTL__CLK_EN__SHIFT 0x9 +#define UVD_VCPU_CNTL__MIF_WR_LOW_THRESHOLD_BP__SHIFT 0x11 +#define UVD_VCPU_CNTL__PRB_TIMEOUT_VAL__SHIFT 0x14 #define UVD_VCPU_CNTL__CLK_EN_MASK 0x00000200L +#define UVD_VCPU_CNTL__MIF_WR_LOW_THRESHOLD_BP_MASK 0x00020000L +#define UVD_VCPU_CNTL__PRB_TIMEOUT_VAL_MASK 0x0FF00000L //UVD_SOFT_RESET #define UVD_SOFT_RESET__RBC_SOFT_RESET__SHIFT 0x0 #define UVD_SOFT_RESET__LBSI_SOFT_RESET__SHIFT 0x1 diff --git a/drivers/gpu/drm/amd/include/atomfirmware.h b/drivers/gpu/drm/amd/include/atomfirmware.h index 8ae7adb7329b..d2e7c0fa96c2 100644 --- a/drivers/gpu/drm/amd/include/atomfirmware.h +++ b/drivers/gpu/drm/amd/include/atomfirmware.h @@ -1532,6 +1532,94 @@ struct atom_smc_dpm_info_v4_3 uint32_t boardreserved[10]; }; +struct smudpm_i2ccontrollerconfig_t { + uint32_t enabled; + uint32_t slaveaddress; + uint32_t controllerport; + uint32_t controllername; + uint32_t thermalthrottler; + uint32_t i2cprotocol; + uint32_t i2cspeed; +}; + +struct atom_smc_dpm_info_v4_4 +{ + struct atom_common_table_header table_header; + uint32_t i2c_padding[3]; + + uint16_t maxvoltagestepgfx; + uint16_t maxvoltagestepsoc; + + uint8_t vddgfxvrmapping; + uint8_t vddsocvrmapping; + uint8_t vddmem0vrmapping; + uint8_t vddmem1vrmapping; + + uint8_t gfxulvphasesheddingmask; + uint8_t soculvphasesheddingmask; + uint8_t externalsensorpresent; + uint8_t padding8_v; + + uint16_t gfxmaxcurrent; + uint8_t gfxoffset; + uint8_t padding_telemetrygfx; + + uint16_t socmaxcurrent; + uint8_t socoffset; + uint8_t padding_telemetrysoc; + + uint16_t mem0maxcurrent; + uint8_t mem0offset; + uint8_t padding_telemetrymem0; + + uint16_t mem1maxcurrent; + uint8_t mem1offset; + uint8_t padding_telemetrymem1; + + + uint8_t acdcgpio; + uint8_t acdcpolarity; + uint8_t vr0hotgpio; + uint8_t vr0hotpolarity; + + uint8_t vr1hotgpio; + uint8_t vr1hotpolarity; + uint8_t padding1; + uint8_t padding2; + + + uint8_t ledpin0; + uint8_t ledpin1; + uint8_t ledpin2; + uint8_t padding8_4; + + + uint8_t pllgfxclkspreadenabled; + uint8_t pllgfxclkspreadpercent; + uint16_t pllgfxclkspreadfreq; + + + uint8_t uclkspreadenabled; + uint8_t uclkspreadpercent; + uint16_t uclkspreadfreq; + + + uint8_t fclkspreadenabled; + uint8_t fclkspreadpercent; + uint16_t fclkspreadfreq; + + + uint8_t fllgfxclkspreadenabled; + uint8_t fllgfxclkspreadpercent; + uint16_t fllgfxclkspreadfreq; + + + struct smudpm_i2ccontrollerconfig_t i2ccontrollers[7]; + + + uint32_t boardreserved[10]; +}; + /* *************************************************************************** Data Table asic_profiling_info structure diff --git a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h index cb4deb28bb7f..64ecffd52126 100644 --- a/drivers/gpu/drm/amd/include/kgd_kfd_interface.h +++ b/drivers/gpu/drm/amd/include/kgd_kfd_interface.h @@ -146,10 +146,10 @@ struct kgd2kfd_shared_resources { * is reserved: (D & reserved_doorbell_mask) == reserved_doorbell_val * * KFD currently uses 1024 (= 0x3ff) doorbells per process. If - * doorbells 0x0f0-0x0f7 and 0x2f-0x2f7 are reserved, that means - * mask would be set to 0x1f8 and val set to 0x0f0. + * doorbells 0x0e0-0x0ff and 0x2e0-0x2ff are reserved, that means + * mask would be set to 0x1e0 and val set to 0x0e0. */ - unsigned int sdma_doorbell[2][2]; + unsigned int sdma_doorbell[2][8]; unsigned int reserved_doorbell_mask; unsigned int reserved_doorbell_val; @@ -322,7 +322,7 @@ struct tile_config { struct kfd2kgd_calls { int (*init_gtt_mem_allocation)(struct kgd_dev *kgd, size_t size, void **mem_obj, uint64_t *gpu_addr, - void **cpu_ptr); + void **cpu_ptr, bool mqd_gfx9); void (*free_gtt_mem)(struct kgd_dev *kgd, void *mem_obj); @@ -409,9 +409,9 @@ struct kfd2kgd_calls { struct dma_fence **ef); void (*destroy_process_vm)(struct kgd_dev *kgd, void *vm); void (*release_process_vm)(struct kgd_dev *kgd, void *vm); - uint32_t (*get_process_page_dir)(void *vm); + uint64_t (*get_process_page_dir)(void *vm); void (*set_vm_context_page_table_base)(struct kgd_dev *kgd, - uint32_t vmid, uint32_t page_table_base); + uint32_t vmid, uint64_t page_table_base); int (*alloc_memory_of_gpu)(struct kgd_dev *kgd, uint64_t va, uint64_t size, void *vm, struct kgd_mem **mem, uint64_t *offset, diff --git a/drivers/gpu/drm/amd/include/kgd_pp_interface.h b/drivers/gpu/drm/amd/include/kgd_pp_interface.h index 448dee481a38..980e696989b1 100644 --- a/drivers/gpu/drm/amd/include/kgd_pp_interface.h +++ b/drivers/gpu/drm/amd/include/kgd_pp_interface.h @@ -113,6 +113,9 @@ enum amd_pp_sensors { AMDGPU_PP_SENSOR_GPU_POWER, AMDGPU_PP_SENSOR_STABLE_PSTATE_SCLK, AMDGPU_PP_SENSOR_STABLE_PSTATE_MCLK, + AMDGPU_PP_SENSOR_ENABLED_SMC_FEATURES_MASK, + AMDGPU_PP_SENSOR_MIN_FAN_RPM, + AMDGPU_PP_SENSOR_MAX_FAN_RPM, }; enum amd_pp_task { @@ -227,6 +230,7 @@ struct amd_pm_funcs { enum amd_dpm_forced_level (*get_performance_level)(void *handle); enum amd_pm_state_type (*get_current_power_state)(void *handle); int (*get_fan_speed_rpm)(void *handle, uint32_t *rpm); + int (*set_fan_speed_rpm)(void *handle, uint32_t rpm); int (*get_pp_num_states)(void *handle, struct pp_states_info *data); int (*get_pp_table)(void *handle, char **table); int (*set_pp_table)(void *handle, const char *buf, size_t size); @@ -271,6 +275,7 @@ struct amd_pm_funcs { int (*get_display_mode_validation_clocks)(void *handle, struct amd_pp_simple_clock_info *clocks); int (*notify_smu_enable_pwe)(void *handle); + int (*enable_mgpu_fan_boost)(void *handle); }; #endif diff --git a/drivers/gpu/drm/amd/powerplay/amd_powerplay.c b/drivers/gpu/drm/amd/powerplay/amd_powerplay.c index da4ebff5b74d..d6aa1d414320 100644 --- a/drivers/gpu/drm/amd/powerplay/amd_powerplay.c +++ b/drivers/gpu/drm/amd/powerplay/amd_powerplay.c @@ -109,11 +109,8 @@ static int pp_sw_fini(void *handle) hwmgr_sw_fini(hwmgr); - if (adev->firmware.load_type == AMDGPU_FW_LOAD_SMU) { - release_firmware(adev->pm.fw); - adev->pm.fw = NULL; - amdgpu_ucode_fini_bo(adev); - } + release_firmware(adev->pm.fw); + adev->pm.fw = NULL; return 0; } @@ -124,9 +121,6 @@ static int pp_hw_init(void *handle) struct amdgpu_device *adev = handle; struct pp_hwmgr *hwmgr = adev->powerplay.pp_handle; - if (adev->firmware.load_type == AMDGPU_FW_LOAD_SMU) - amdgpu_ucode_init_bo(adev); - ret = hwmgr_hw_init(hwmgr); if (ret) @@ -273,8 +267,23 @@ const struct amdgpu_ip_block_version pp_smu_ip_block = .funcs = &pp_ip_funcs, }; +/* This interface only be supported On Vi, + * because only smu7/8 can help to load gfx/sdma fw, + * smu need to be enabled before load other ip's fw. + * so call start smu to load smu7 fw and other ip's fw + */ static int pp_dpm_load_fw(void *handle) { + struct pp_hwmgr *hwmgr = handle; + + if (!hwmgr || !hwmgr->smumgr_funcs || !hwmgr->smumgr_funcs->start_smu) + return -EINVAL; + + if (hwmgr->smumgr_funcs->start_smu(hwmgr)) { + pr_err("fw load failed\n"); + return -EINVAL; + } + return 0; } @@ -576,6 +585,24 @@ static int pp_dpm_get_fan_speed_rpm(void *handle, uint32_t *rpm) return ret; } +static int pp_dpm_set_fan_speed_rpm(void *handle, uint32_t rpm) +{ + struct pp_hwmgr *hwmgr = handle; + int ret = 0; + + if (!hwmgr || !hwmgr->pm_en) + return -EINVAL; + + if (hwmgr->hwmgr_func->set_fan_speed_rpm == NULL) { + pr_info("%s was not implemented.\n", __func__); + return 0; + } + mutex_lock(&hwmgr->smu_lock); + ret = hwmgr->hwmgr_func->set_fan_speed_rpm(hwmgr, rpm); + mutex_unlock(&hwmgr->smu_lock); + return ret; +} + static int pp_dpm_get_pp_num_states(void *handle, struct pp_states_info *data) { @@ -696,11 +723,14 @@ static int pp_dpm_force_clock_level(void *handle, pr_info("%s was not implemented.\n", __func__); return 0; } + + if (hwmgr->dpm_level != AMD_DPM_FORCED_LEVEL_MANUAL) { + pr_info("force clock level is for dpm manual mode only.\n"); + return -EINVAL; + } + mutex_lock(&hwmgr->smu_lock); - if (hwmgr->dpm_level == AMD_DPM_FORCED_LEVEL_MANUAL) - ret = hwmgr->hwmgr_func->force_clock_level(hwmgr, type, mask); - else - ret = -EINVAL; + ret = hwmgr->hwmgr_func->force_clock_level(hwmgr, type, mask); mutex_unlock(&hwmgr->smu_lock); return ret; } @@ -813,6 +843,12 @@ static int pp_dpm_read_sensor(void *handle, int idx, case AMDGPU_PP_SENSOR_STABLE_PSTATE_MCLK: *((uint32_t *)value) = hwmgr->pstate_mclk; return 0; + case AMDGPU_PP_SENSOR_MIN_FAN_RPM: + *((uint32_t *)value) = hwmgr->thermal_controller.fanInfo.ulMinRPM; + return 0; + case AMDGPU_PP_SENSOR_MAX_FAN_RPM: + *((uint32_t *)value) = hwmgr->thermal_controller.fanInfo.ulMaxRPM; + return 0; default: mutex_lock(&hwmgr->smu_lock); ret = hwmgr->hwmgr_func->read_sensor(hwmgr, idx, value, size); @@ -861,9 +897,14 @@ static int pp_set_power_profile_mode(void *handle, long *input, uint32_t size) pr_info("%s was not implemented.\n", __func__); return ret; } + + if (hwmgr->dpm_level != AMD_DPM_FORCED_LEVEL_MANUAL) { + pr_info("power profile setting is for manual dpm mode only.\n"); + return ret; + } + mutex_lock(&hwmgr->smu_lock); - if (hwmgr->dpm_level == AMD_DPM_FORCED_LEVEL_MANUAL) - ret = hwmgr->hwmgr_func->set_power_profile_mode(hwmgr, input, size); + ret = hwmgr->hwmgr_func->set_power_profile_mode(hwmgr, input, size); mutex_unlock(&hwmgr->smu_lock); return ret; } @@ -925,6 +966,7 @@ static int pp_dpm_switch_power_profile(void *handle, static int pp_set_power_limit(void *handle, uint32_t limit) { struct pp_hwmgr *hwmgr = handle; + uint32_t max_power_limit; if (!hwmgr || !hwmgr->pm_en) return -EINVAL; @@ -937,7 +979,13 @@ static int pp_set_power_limit(void *handle, uint32_t limit) if (limit == 0) limit = hwmgr->default_power_limit; - if (limit > hwmgr->default_power_limit) + max_power_limit = hwmgr->default_power_limit; + if (hwmgr->od_enabled) { + max_power_limit *= (100 + hwmgr->platform_descriptor.TDPODLimit); + max_power_limit /= 100; + } + + if (limit > max_power_limit) return -EINVAL; mutex_lock(&hwmgr->smu_lock); @@ -956,8 +1004,13 @@ static int pp_get_power_limit(void *handle, uint32_t *limit, bool default_limit) mutex_lock(&hwmgr->smu_lock); - if (default_limit) + if (default_limit) { *limit = hwmgr->default_power_limit; + if (hwmgr->od_enabled) { + *limit *= (100 + hwmgr->platform_descriptor.TDPODLimit); + *limit /= 100; + } + } else *limit = hwmgr->power_limit; @@ -1196,6 +1249,21 @@ static void pp_dpm_powergate_acp(void *handle, bool gate) hwmgr->hwmgr_func->powergate_acp(hwmgr, gate); } +static void pp_dpm_powergate_sdma(void *handle, bool gate) +{ + struct pp_hwmgr *hwmgr = handle; + + if (!hwmgr) + return; + + if (hwmgr->hwmgr_func->powergate_sdma == NULL) { + pr_info("%s was not implemented.\n", __func__); + return; + } + + hwmgr->hwmgr_func->powergate_sdma(hwmgr, gate); +} + static int pp_set_powergating_by_smu(void *handle, uint32_t block_type, bool gate) { @@ -1218,6 +1286,9 @@ static int pp_set_powergating_by_smu(void *handle, case AMD_IP_BLOCK_TYPE_ACP: pp_dpm_powergate_acp(handle, gate); break; + case AMD_IP_BLOCK_TYPE_SDMA: + pp_dpm_powergate_sdma(handle, gate); + break; default: break; } @@ -1243,6 +1314,24 @@ static int pp_notify_smu_enable_pwe(void *handle) return 0; } +static int pp_enable_mgpu_fan_boost(void *handle) +{ + struct pp_hwmgr *hwmgr = handle; + + if (!hwmgr) + return -EINVAL; + + if (!hwmgr->pm_en || + hwmgr->hwmgr_func->enable_mgpu_fan_boost == NULL) + return 0; + + mutex_lock(&hwmgr->smu_lock); + hwmgr->hwmgr_func->enable_mgpu_fan_boost(hwmgr); + mutex_unlock(&hwmgr->smu_lock); + + return 0; +} + static const struct amd_pm_funcs pp_dpm_funcs = { .load_firmware = pp_dpm_load_fw, .wait_for_fw_loading_complete = pp_dpm_fw_loading_complete, @@ -1255,6 +1344,7 @@ static const struct amd_pm_funcs pp_dpm_funcs = { .set_fan_speed_percent = pp_dpm_set_fan_speed_percent, .get_fan_speed_percent = pp_dpm_get_fan_speed_percent, .get_fan_speed_rpm = pp_dpm_get_fan_speed_rpm, + .set_fan_speed_rpm = pp_dpm_set_fan_speed_rpm, .get_pp_num_states = pp_dpm_get_pp_num_states, .get_pp_table = pp_dpm_get_pp_table, .set_pp_table = pp_dpm_set_pp_table, @@ -1287,4 +1377,5 @@ static const struct amd_pm_funcs pp_dpm_funcs = { .display_clock_voltage_request = pp_display_clock_voltage_request, .get_display_mode_validation_clocks = pp_get_display_mode_validation_clocks, .notify_smu_enable_pwe = pp_notify_smu_enable_pwe, + .enable_mgpu_fan_boost = pp_enable_mgpu_fan_boost, }; diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c index 7500a3e61dba..47ac92369739 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/hwmgr.c @@ -89,7 +89,6 @@ int hwmgr_early_init(struct pp_hwmgr *hwmgr) hwmgr_init_default_caps(hwmgr); hwmgr_set_user_specify_caps(hwmgr); hwmgr->fan_ctrl_is_in_default_mode = true; - hwmgr->reload_fw = 1; hwmgr_init_workload_prority(hwmgr); switch (hwmgr->chip_family) { @@ -209,17 +208,6 @@ int hwmgr_hw_init(struct pp_hwmgr *hwmgr) { int ret = 0; - if (!hwmgr || !hwmgr->smumgr_funcs) - return -EINVAL; - - if (hwmgr->smumgr_funcs->start_smu) { - ret = hwmgr->smumgr_funcs->start_smu(hwmgr); - if (ret) { - pr_err("smc start failed\n"); - return -EINVAL; - } - } - if (!hwmgr->pm_en) return 0; @@ -320,13 +308,6 @@ int hwmgr_resume(struct pp_hwmgr *hwmgr) if (!hwmgr) return -EINVAL; - if (hwmgr->smumgr_funcs && hwmgr->smumgr_funcs->start_smu) { - if (hwmgr->smumgr_funcs->start_smu(hwmgr)) { - pr_err("smc start failed\n"); - return -EINVAL; - } - } - if (!hwmgr->pm_en) return 0; diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.c b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.c index d27c1c9df286..4588bddf8b33 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.c @@ -488,7 +488,8 @@ int pp_atomfwctrl_get_gpio_information(struct pp_hwmgr *hwmgr, return 0; } -int pp_atomfwctrl_get_clk_information_by_clkid(struct pp_hwmgr *hwmgr, BIOS_CLKID id, uint32_t *frequency) +int pp_atomfwctrl_get_clk_information_by_clkid(struct pp_hwmgr *hwmgr, + uint8_t id, uint32_t *frequency) { struct amdgpu_device *adev = hwmgr->adev; struct atom_get_smu_clock_info_parameters_v3_1 parameters; diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.h b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.h index 22e21668c93a..fe9e8ceef50e 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.h +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/ppatomfwctrl.h @@ -236,7 +236,7 @@ int pp_atomfwctrl_get_vbios_bootup_values(struct pp_hwmgr *hwmgr, int pp_atomfwctrl_get_smc_dpm_information(struct pp_hwmgr *hwmgr, struct pp_atomfwctrl_smc_dpm_parameters *param); int pp_atomfwctrl_get_clk_information_by_clkid(struct pp_hwmgr *hwmgr, - BIOS_CLKID id, uint32_t *frequency); + uint8_t id, uint32_t *frequency); #endif diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/smu10_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/smu10_hwmgr.c index 9808bd48b386..dd18cb710391 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/smu10_hwmgr.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/smu10_hwmgr.c @@ -552,6 +552,8 @@ static int smu10_dpm_force_dpm_level(struct pp_hwmgr *hwmgr, { struct smu10_hwmgr *data = hwmgr->backend; struct amdgpu_device *adev = hwmgr->adev; + uint32_t min_sclk = hwmgr->display_config->min_core_set_clock; + uint32_t min_mclk = hwmgr->display_config->min_mem_set_clock/100; if (hwmgr->smu_version < 0x1E3700) { pr_info("smu firmware version too old, can not set dpm level\n"); @@ -563,6 +565,13 @@ static int smu10_dpm_force_dpm_level(struct pp_hwmgr *hwmgr, (adev->rev_id >= 8)) return 0; + if (min_sclk < data->gfx_min_freq_limit) + min_sclk = data->gfx_min_freq_limit; + + min_sclk /= 100; /* transfer 10KHz to MHz */ + if (min_mclk < data->clock_table.FClocks[0].Freq) + min_mclk = data->clock_table.FClocks[0].Freq; + switch (level) { case AMD_DPM_FORCED_LEVEL_HIGH: case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK: @@ -595,18 +604,18 @@ static int smu10_dpm_force_dpm_level(struct pp_hwmgr *hwmgr, case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK: smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetHardMinGfxClk, - data->gfx_min_freq_limit/100); + min_sclk); smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetSoftMaxGfxClk, - data->gfx_min_freq_limit/100); + min_sclk); break; case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK: smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetHardMinFclkByFreq, - SMU10_UMD_PSTATE_MIN_FCLK); + min_mclk); smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetSoftMaxFclkByFreq, - SMU10_UMD_PSTATE_MIN_FCLK); + min_mclk); break; case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD: smum_send_msg_to_smc_with_parameter(hwmgr, @@ -638,12 +647,12 @@ static int smu10_dpm_force_dpm_level(struct pp_hwmgr *hwmgr, case AMD_DPM_FORCED_LEVEL_AUTO: smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetHardMinGfxClk, - data->gfx_min_freq_limit/100); + min_sclk); smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetHardMinFclkByFreq, hwmgr->display_config->num_display > 3 ? SMU10_UMD_PSTATE_PEAK_FCLK : - SMU10_UMD_PSTATE_MIN_FCLK); + min_mclk); smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetHardMinSocclkByFreq, @@ -674,10 +683,10 @@ static int smu10_dpm_force_dpm_level(struct pp_hwmgr *hwmgr, data->gfx_min_freq_limit/100); smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetHardMinFclkByFreq, - SMU10_UMD_PSTATE_MIN_FCLK); + min_mclk); smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetSoftMaxFclkByFreq, - SMU10_UMD_PSTATE_MIN_FCLK); + min_mclk); break; case AMD_DPM_FORCED_LEVEL_MANUAL: case AMD_DPM_FORCED_LEVEL_PROFILE_EXIT: @@ -1144,6 +1153,14 @@ static int smu10_powergate_mmhub(struct pp_hwmgr *hwmgr) return smum_send_msg_to_smc(hwmgr, PPSMC_MSG_PowerGateMmHub); } +static int smu10_powergate_sdma(struct pp_hwmgr *hwmgr, bool gate) +{ + if (gate) + return smum_send_msg_to_smc(hwmgr, PPSMC_MSG_PowerDownSdma); + else + return smum_send_msg_to_smc(hwmgr, PPSMC_MSG_PowerUpSdma); +} + static void smu10_powergate_vcn(struct pp_hwmgr *hwmgr, bool bgate) { if (bgate) { @@ -1199,6 +1216,7 @@ static const struct pp_hwmgr_func smu10_hwmgr_funcs = { .smus_notify_pwe = smu10_smus_notify_pwe, .display_clock_voltage_request = smu10_display_clock_voltage_request, .powergate_gfx = smu10_gfx_off_control, + .powergate_sdma = smu10_powergate_sdma, }; int smu10_init_function_pointers(struct pp_hwmgr *hwmgr) diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_hwmgr.c index 04b7da0e39a6..ed35ec0341e6 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_hwmgr.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_hwmgr.c @@ -3588,9 +3588,10 @@ static int smu7_find_dpm_states_clocks_in_dpm_table(struct pp_hwmgr *hwmgr, cons break; } - if (i >= sclk_table->count) + if (i >= sclk_table->count) { data->need_update_smu7_dpm_table |= DPMTABLE_OD_UPDATE_SCLK; - else { + sclk_table->dpm_levels[i-1].value = sclk; + } else { /* TODO: Check SCLK in DAL's minimum clocks * in case DeepSleep divider update is required. */ @@ -3605,9 +3606,10 @@ static int smu7_find_dpm_states_clocks_in_dpm_table(struct pp_hwmgr *hwmgr, cons break; } - if (i >= mclk_table->count) + if (i >= mclk_table->count) { data->need_update_smu7_dpm_table |= DPMTABLE_OD_UPDATE_MCLK; - + mclk_table->dpm_levels[i-1].value = mclk; + } if (data->display_timing.num_existing_displays != hwmgr->display_config->num_display) data->need_update_smu7_dpm_table |= DPMTABLE_UPDATE_MCLK; @@ -4106,17 +4108,17 @@ static int smu7_register_irq_handlers(struct pp_hwmgr *hwmgr) source->funcs = &smu7_irq_funcs; amdgpu_irq_add_id((struct amdgpu_device *)(hwmgr->adev), - AMDGPU_IH_CLIENTID_LEGACY, + AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CG_TSS_THERMAL_LOW_TO_HIGH, source); amdgpu_irq_add_id((struct amdgpu_device *)(hwmgr->adev), - AMDGPU_IH_CLIENTID_LEGACY, + AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_CG_TSS_THERMAL_HIGH_TO_LOW, source); /* Register CTF(GPIO_19) interrupt */ amdgpu_irq_add_id((struct amdgpu_device *)(hwmgr->adev), - AMDGPU_IH_CLIENTID_LEGACY, + AMDGPU_IRQ_CLIENTID_LEGACY, VISLANDS30_IV_SRCID_GPIO_19, source); @@ -5035,6 +5037,18 @@ static int smu7_get_performance_level(struct pp_hwmgr *hwmgr, const struct pp_hw return 0; } +static int smu7_power_off_asic(struct pp_hwmgr *hwmgr) +{ + int result; + + result = smu7_disable_dpm_tasks(hwmgr); + PP_ASSERT_WITH_CODE((0 == result), + "[disable_dpm_tasks] Failed to disable DPM!", + ); + + return result; +} + static const struct pp_hwmgr_func smu7_hwmgr_funcs = { .backend_init = &smu7_hwmgr_backend_init, .backend_fini = &smu7_hwmgr_backend_fini, @@ -5092,6 +5106,7 @@ static const struct pp_hwmgr_func smu7_hwmgr_funcs = { .get_power_profile_mode = smu7_get_power_profile_mode, .set_power_profile_mode = smu7_set_power_profile_mode, .get_performance_level = smu7_get_performance_level, + .power_off_asic = smu7_power_off_asic, }; uint8_t smu7_get_sleep_divider_id_from_clock(uint32_t clock, diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_thermal.c b/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_thermal.c index 44527755e747..5bdc0df5a9f4 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_thermal.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/smu7_thermal.c @@ -260,6 +260,7 @@ int smu7_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t speed) if (hwmgr->thermal_controller.fanInfo.bNoFan || (hwmgr->thermal_controller.fanInfo. ucTachometerPulsesPerRevolution == 0) || + speed == 0 || (speed < hwmgr->thermal_controller.fanInfo.ulMinRPM) || (speed > hwmgr->thermal_controller.fanInfo.ulMaxRPM)) return 0; @@ -272,7 +273,7 @@ int smu7_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t speed) tach_period = 60 * crystal_clock_freq * 10000 / (8 * speed); PHM_WRITE_VFPF_INDIRECT_FIELD(hwmgr->device, CGS_IND_REG__SMC, - CG_TACH_STATUS, TACH_PERIOD, tach_period); + CG_TACH_CTRL, TARGET_PERIOD, tach_period); return smu7_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC_RPM); } diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/smu8_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/smu8_hwmgr.c index b8637049198d..fef111ddb736 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/smu8_hwmgr.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/smu8_hwmgr.c @@ -880,7 +880,7 @@ static int smu8_set_power_state_tasks(struct pp_hwmgr *hwmgr, const void *input) smu8_update_low_mem_pstate(hwmgr, input); return 0; -}; +} static int smu8_setup_asic_task(struct pp_hwmgr *hwmgr) @@ -934,14 +934,6 @@ static void smu8_reset_cc6_data(struct pp_hwmgr *hwmgr) hw_data->cc6_settings.cpu_pstate_disable = false; } -static int smu8_power_off_asic(struct pp_hwmgr *hwmgr) -{ - smu8_power_up_display_clock_sys_pll(hwmgr); - smu8_clear_nb_dpm_flag(hwmgr); - smu8_reset_cc6_data(hwmgr); - return 0; -}; - static void smu8_program_voting_clients(struct pp_hwmgr *hwmgr) { cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, @@ -1011,6 +1003,17 @@ static void smu8_reset_acp_boot_level(struct pp_hwmgr *hwmgr) data->acp_boot_level = 0xff; } +static int smu8_enable_dpm_tasks(struct pp_hwmgr *hwmgr) +{ + smu8_program_voting_clients(hwmgr); + if (smu8_start_dpm(hwmgr)) + return -EINVAL; + smu8_program_bootup_state(hwmgr); + smu8_reset_acp_boot_level(hwmgr); + + return 0; +} + static int smu8_disable_dpm_tasks(struct pp_hwmgr *hwmgr) { smu8_disable_nb_dpm(hwmgr); @@ -1020,18 +1023,16 @@ static int smu8_disable_dpm_tasks(struct pp_hwmgr *hwmgr) return -EINVAL; return 0; -}; +} -static int smu8_enable_dpm_tasks(struct pp_hwmgr *hwmgr) +static int smu8_power_off_asic(struct pp_hwmgr *hwmgr) { - smu8_program_voting_clients(hwmgr); - if (smu8_start_dpm(hwmgr)) - return -EINVAL; - smu8_program_bootup_state(hwmgr); - smu8_reset_acp_boot_level(hwmgr); - + smu8_disable_dpm_tasks(hwmgr); + smu8_power_up_display_clock_sys_pll(hwmgr); + smu8_clear_nb_dpm_flag(hwmgr); + smu8_reset_cc6_data(hwmgr); return 0; -}; +} static int smu8_apply_state_adjust_rules(struct pp_hwmgr *hwmgr, struct pp_power_state *prequest_ps, @@ -1227,14 +1228,17 @@ static int smu8_dpm_force_dpm_level(struct pp_hwmgr *hwmgr, static int smu8_dpm_powerdown_uvd(struct pp_hwmgr *hwmgr) { - if (PP_CAP(PHM_PlatformCaps_UVDPowerGating)) + if (PP_CAP(PHM_PlatformCaps_UVDPowerGating)) { + smu8_nbdpm_pstate_enable_disable(hwmgr, true, true); return smum_send_msg_to_smc(hwmgr, PPSMC_MSG_UVDPowerOFF); + } return 0; } static int smu8_dpm_powerup_uvd(struct pp_hwmgr *hwmgr) { if (PP_CAP(PHM_PlatformCaps_UVDPowerGating)) { + smu8_nbdpm_pstate_enable_disable(hwmgr, false, true); return smum_send_msg_to_smc_with_parameter( hwmgr, PPSMC_MSG_UVDPowerON, diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.c b/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.c index 2aab1b475945..99a33c33a32c 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.c @@ -39,6 +39,50 @@ uint16_t convert_to_vddc(uint8_t vid) return (uint16_t) ((6200 - (vid * 25)) / VOLTAGE_SCALE); } +int phm_copy_clock_limits_array( + struct pp_hwmgr *hwmgr, + uint32_t **pptable_info_array, + const uint32_t *pptable_array, + uint32_t power_saving_clock_count) +{ + uint32_t array_size, i; + uint32_t *table; + + array_size = sizeof(uint32_t) * power_saving_clock_count; + table = kzalloc(array_size, GFP_KERNEL); + if (NULL == table) + return -ENOMEM; + + for (i = 0; i < power_saving_clock_count; i++) + table[i] = le32_to_cpu(pptable_array[i]); + + *pptable_info_array = table; + + return 0; +} + +int phm_copy_overdrive_settings_limits_array( + struct pp_hwmgr *hwmgr, + uint32_t **pptable_info_array, + const uint32_t *pptable_array, + uint32_t od_setting_count) +{ + uint32_t array_size, i; + uint32_t *table; + + array_size = sizeof(uint32_t) * od_setting_count; + table = kzalloc(array_size, GFP_KERNEL); + if (NULL == table) + return -ENOMEM; + + for (i = 0; i < od_setting_count; i++) + table[i] = le32_to_cpu(pptable_array[i]); + + *pptable_info_array = table; + + return 0; +} + uint32_t phm_set_field_to_u32(u32 offset, u32 original_data, u32 field, u32 size) { u32 mask = 0; @@ -545,7 +589,7 @@ int phm_irq_process(struct amdgpu_device *adev, uint32_t client_id = entry->client_id; uint32_t src_id = entry->src_id; - if (client_id == AMDGPU_IH_CLIENTID_LEGACY) { + if (client_id == AMDGPU_IRQ_CLIENTID_LEGACY) { if (src_id == VISLANDS30_IV_SRCID_CG_TSS_THERMAL_LOW_TO_HIGH) pr_warn("GPU over temperature range detected on PCIe %d:%d.%d!\n", PCI_BUS_NUM(adev->pdev->devfn), @@ -674,7 +718,7 @@ int smu_set_watermarks_for_clocks_ranges(void *wt_table, table->WatermarkRow[1][i].MaxClock = cpu_to_le16((uint16_t) (wm_with_clock_ranges->wm_dmif_clocks_ranges[i].wm_max_dcfclk_clk_in_khz) / - 100); + 1000); table->WatermarkRow[1][i].MinUclk = cpu_to_le16((uint16_t) (wm_with_clock_ranges->wm_dmif_clocks_ranges[i].wm_min_mem_clk_in_khz) / diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.h b/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.h index 5454289d5226..ad33983a8064 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.h +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/smu_helper.h @@ -47,6 +47,18 @@ struct watermarks { uint32_t padding[7]; }; +int phm_copy_clock_limits_array( + struct pp_hwmgr *hwmgr, + uint32_t **pptable_info_array, + const uint32_t *pptable_array, + uint32_t power_saving_clock_count); + +int phm_copy_overdrive_settings_limits_array( + struct pp_hwmgr *hwmgr, + uint32_t **pptable_info_array, + const uint32_t *pptable_array, + uint32_t od_setting_count); + extern int phm_wait_for_register_unequal(struct pp_hwmgr *hwmgr, uint32_t index, uint32_t value, uint32_t mask); diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.c index ca9be583fb62..8c4db86bb4b7 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.c @@ -39,6 +39,7 @@ #include "soc15_common.h" #include "pppcielanes.h" #include "vega10_hwmgr.h" +#include "vega10_smumgr.h" #include "vega10_processpptables.h" #include "vega10_pptable.h" #include "vega10_thermal.h" @@ -1332,7 +1333,6 @@ static int vega10_setup_default_dpm_tables(struct pp_hwmgr *hwmgr) if (hwmgr->platform_descriptor.overdriveLimit.memoryClock == 0) hwmgr->platform_descriptor.overdriveLimit.memoryClock = dpm_table->dpm_levels[dpm_table->count-1].value; - vega10_init_dpm_state(&(dpm_table->dpm_state)); data->dpm_table.eclk_table.count = 0; @@ -3248,6 +3248,37 @@ static int vega10_apply_state_adjust_rules(struct pp_hwmgr *hwmgr, static int vega10_find_dpm_states_clocks_in_dpm_table(struct pp_hwmgr *hwmgr, const void *input) { struct vega10_hwmgr *data = hwmgr->backend; + const struct phm_set_power_state_input *states = + (const struct phm_set_power_state_input *)input; + const struct vega10_power_state *vega10_ps = + cast_const_phw_vega10_power_state(states->pnew_state); + struct vega10_single_dpm_table *sclk_table = &(data->dpm_table.gfx_table); + uint32_t sclk = vega10_ps->performance_levels + [vega10_ps->performance_level_count - 1].gfx_clock; + struct vega10_single_dpm_table *mclk_table = &(data->dpm_table.mem_table); + uint32_t mclk = vega10_ps->performance_levels + [vega10_ps->performance_level_count - 1].mem_clock; + uint32_t i; + + for (i = 0; i < sclk_table->count; i++) { + if (sclk == sclk_table->dpm_levels[i].value) + break; + } + + if (i >= sclk_table->count) { + data->need_update_dpm_table |= DPMTABLE_OD_UPDATE_SCLK; + sclk_table->dpm_levels[i-1].value = sclk; + } + + for (i = 0; i < mclk_table->count; i++) { + if (mclk == mclk_table->dpm_levels[i].value) + break; + } + + if (i >= mclk_table->count) { + data->need_update_dpm_table |= DPMTABLE_OD_UPDATE_MCLK; + mclk_table->dpm_levels[i-1].value = mclk; + } if (data->display_timing.num_existing_displays != hwmgr->display_config->num_display) data->need_update_dpm_table |= DPMTABLE_UPDATE_MCLK; @@ -3713,6 +3744,11 @@ static int vega10_read_sensor(struct pp_hwmgr *hwmgr, int idx, SMUSVI0_PLANE0_CURRENTVID__CURRENT_SVI0_PLANE0_VID__SHIFT; *((uint32_t *)value) = (uint32_t)convert_to_vddc((uint8_t)val_vid); return 0; + case AMDGPU_PP_SENSOR_ENABLED_SMC_FEATURES_MASK: + ret = vega10_get_enabled_smc_features(hwmgr, (uint64_t *)value); + if (!ret) + *size = 8; + break; default: ret = -EINVAL; break; @@ -4523,11 +4559,13 @@ static int vega10_set_sclk_od(struct pp_hwmgr *hwmgr, uint32_t value) if (vega10_ps->performance_levels [vega10_ps->performance_level_count - 1].gfx_clock > - hwmgr->platform_descriptor.overdriveLimit.engineClock) + hwmgr->platform_descriptor.overdriveLimit.engineClock) { vega10_ps->performance_levels [vega10_ps->performance_level_count - 1].gfx_clock = hwmgr->platform_descriptor.overdriveLimit.engineClock; - + pr_warn("max sclk supported by vbios is %d\n", + hwmgr->platform_descriptor.overdriveLimit.engineClock); + } return 0; } @@ -4575,10 +4613,13 @@ static int vega10_set_mclk_od(struct pp_hwmgr *hwmgr, uint32_t value) if (vega10_ps->performance_levels [vega10_ps->performance_level_count - 1].mem_clock > - hwmgr->platform_descriptor.overdriveLimit.memoryClock) + hwmgr->platform_descriptor.overdriveLimit.memoryClock) { vega10_ps->performance_levels [vega10_ps->performance_level_count - 1].mem_clock = hwmgr->platform_descriptor.overdriveLimit.memoryClock; + pr_warn("max mclk supported by vbios is %d\n", + hwmgr->platform_descriptor.overdriveLimit.memoryClock); + } return 0; } @@ -4940,16 +4981,6 @@ static const struct pp_hwmgr_func vega10_hwmgr_funcs = { .get_performance_level = vega10_get_performance_level, }; -int vega10_enable_smc_features(struct pp_hwmgr *hwmgr, - bool enable, uint32_t feature_mask) -{ - int msg = enable ? PPSMC_MSG_EnableSmuFeatures : - PPSMC_MSG_DisableSmuFeatures; - - return smum_send_msg_to_smc_with_parameter(hwmgr, - msg, feature_mask); -} - int vega10_hwmgr_init(struct pp_hwmgr *hwmgr) { hwmgr->hwmgr_func = &vega10_hwmgr_funcs; diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.h b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.h index 339820da9e6a..89870556de1b 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.h +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_hwmgr.h @@ -441,7 +441,5 @@ int vega10_update_uvd_dpm(struct pp_hwmgr *hwmgr, bool bgate); int vega10_update_samu_dpm(struct pp_hwmgr *hwmgr, bool bgate); int vega10_update_acp_dpm(struct pp_hwmgr *hwmgr, bool bgate); int vega10_enable_disable_vce_dpm(struct pp_hwmgr *hwmgr, bool enable); -int vega10_enable_smc_features(struct pp_hwmgr *hwmgr, - bool enable, uint32_t feature_mask); #endif /* _VEGA10_HWMGR_H_ */ diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_powertune.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_powertune.c index 22364875a943..2d88abf97e7b 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_powertune.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_powertune.c @@ -23,6 +23,7 @@ #include "hwmgr.h" #include "vega10_hwmgr.h" +#include "vega10_smumgr.h" #include "vega10_powertune.h" #include "vega10_ppsmc.h" #include "vega10_inc.h" diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_processpptables.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_processpptables.c index 16b1a9cf6cf0..b8747a5c9204 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_processpptables.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_processpptables.c @@ -451,23 +451,23 @@ static int get_tdp_table( le16_to_cpu(power_tune_table_v2->usLoadLineResistance); } else { power_tune_table_v3 = (ATOM_Vega10_PowerTune_Table_V3 *)table; - tdp_table->usMaximumPowerDeliveryLimit = power_tune_table_v3->usSocketPowerLimit; - tdp_table->usTDC = power_tune_table_v3->usTdcLimit; - tdp_table->usEDCLimit = power_tune_table_v3->usEdcLimit; - tdp_table->usSoftwareShutdownTemp = power_tune_table_v3->usSoftwareShutdownTemp; - tdp_table->usTemperatureLimitTedge = power_tune_table_v3->usTemperatureLimitTedge; - tdp_table->usTemperatureLimitHotspot = power_tune_table_v3->usTemperatureLimitHotSpot; - tdp_table->usTemperatureLimitLiquid1 = power_tune_table_v3->usTemperatureLimitLiquid1; - tdp_table->usTemperatureLimitLiquid2 = power_tune_table_v3->usTemperatureLimitLiquid2; - tdp_table->usTemperatureLimitHBM = power_tune_table_v3->usTemperatureLimitHBM; - tdp_table->usTemperatureLimitVrVddc = power_tune_table_v3->usTemperatureLimitVrSoc; - tdp_table->usTemperatureLimitVrMvdd = power_tune_table_v3->usTemperatureLimitVrMem; - tdp_table->usTemperatureLimitPlx = power_tune_table_v3->usTemperatureLimitPlx; + tdp_table->usMaximumPowerDeliveryLimit = le16_to_cpu(power_tune_table_v3->usSocketPowerLimit); + tdp_table->usTDC = le16_to_cpu(power_tune_table_v3->usTdcLimit); + tdp_table->usEDCLimit = le16_to_cpu(power_tune_table_v3->usEdcLimit); + tdp_table->usSoftwareShutdownTemp = le16_to_cpu(power_tune_table_v3->usSoftwareShutdownTemp); + tdp_table->usTemperatureLimitTedge = le16_to_cpu(power_tune_table_v3->usTemperatureLimitTedge); + tdp_table->usTemperatureLimitHotspot = le16_to_cpu(power_tune_table_v3->usTemperatureLimitHotSpot); + tdp_table->usTemperatureLimitLiquid1 = le16_to_cpu(power_tune_table_v3->usTemperatureLimitLiquid1); + tdp_table->usTemperatureLimitLiquid2 = le16_to_cpu(power_tune_table_v3->usTemperatureLimitLiquid2); + tdp_table->usTemperatureLimitHBM = le16_to_cpu(power_tune_table_v3->usTemperatureLimitHBM); + tdp_table->usTemperatureLimitVrVddc = le16_to_cpu(power_tune_table_v3->usTemperatureLimitVrSoc); + tdp_table->usTemperatureLimitVrMvdd = le16_to_cpu(power_tune_table_v3->usTemperatureLimitVrMem); + tdp_table->usTemperatureLimitPlx = le16_to_cpu(power_tune_table_v3->usTemperatureLimitPlx); tdp_table->ucLiquid1_I2C_address = power_tune_table_v3->ucLiquid1_I2C_address; tdp_table->ucLiquid2_I2C_address = power_tune_table_v3->ucLiquid2_I2C_address; - tdp_table->usBoostStartTemperature = power_tune_table_v3->usBoostStartTemperature; - tdp_table->usBoostStopTemperature = power_tune_table_v3->usBoostStopTemperature; - tdp_table->ulBoostClock = power_tune_table_v3->ulBoostClock; + tdp_table->usBoostStartTemperature = le16_to_cpu(power_tune_table_v3->usBoostStartTemperature); + tdp_table->usBoostStopTemperature = le16_to_cpu(power_tune_table_v3->usBoostStopTemperature); + tdp_table->ulBoostClock = le32_to_cpu(power_tune_table_v3->ulBoostClock); get_scl_sda_value(power_tune_table_v3->ucLiquid_I2C_Line, &scl, &sda); diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_thermal.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_thermal.c index aa044c1955fe..3f807d6c95ce 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_thermal.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega10_thermal.c @@ -23,6 +23,7 @@ #include "vega10_thermal.h" #include "vega10_hwmgr.h" +#include "vega10_smumgr.h" #include "vega10_ppsmc.h" #include "vega10_inc.h" #include "soc15_common.h" @@ -311,6 +312,7 @@ int vega10_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t speed) int result = 0; if (hwmgr->thermal_controller.fanInfo.bNoFan || + speed == 0 || (speed < hwmgr->thermal_controller.fanInfo.ulMinRPM) || (speed > hwmgr->thermal_controller.fanInfo.ulMaxRPM)) return -1; @@ -321,9 +323,9 @@ int vega10_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t speed) if (!result) { crystal_clock_freq = amdgpu_asic_get_xclk((struct amdgpu_device *)hwmgr->adev); tach_period = 60 * crystal_clock_freq * 10000 / (8 * speed); - WREG32_SOC15(THM, 0, mmCG_TACH_STATUS, - REG_SET_FIELD(RREG32_SOC15(THM, 0, mmCG_TACH_STATUS), - CG_TACH_STATUS, TACH_PERIOD, + WREG32_SOC15(THM, 0, mmCG_TACH_CTRL, + REG_SET_FIELD(RREG32_SOC15(THM, 0, mmCG_TACH_CTRL), + CG_TACH_CTRL, TARGET_PERIOD, tach_period)); } return vega10_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC_RPM); diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_hwmgr.c index 0789d64246ca..74bc37308dc0 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_hwmgr.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_hwmgr.c @@ -745,8 +745,8 @@ static int vega12_init_smc_table(struct pp_hwmgr *hwmgr) memcpy(pp_table, pptable_information->smc_pptable, sizeof(PPTable_t)); - result = vega12_copy_table_to_smc(hwmgr, - (uint8_t *)pp_table, TABLE_PPTABLE); + result = smum_smc_table_manager(hwmgr, + (uint8_t *)pp_table, TABLE_PPTABLE, false); PP_ASSERT_WITH_CODE(!result, "Failed to upload PPtable!", return result); @@ -1317,7 +1317,11 @@ static int vega12_read_sensor(struct pp_hwmgr *hwmgr, int idx, break; case AMDGPU_PP_SENSOR_GPU_POWER: ret = vega12_get_gpu_power(hwmgr, (uint32_t *)value); - + break; + case AMDGPU_PP_SENSOR_ENABLED_SMC_FEATURES_MASK: + ret = vega12_get_enabled_smc_features(hwmgr, (uint64_t *)value); + if (!ret) + *size = 8; break; default: ret = -EINVAL; @@ -2103,8 +2107,8 @@ static int vega12_display_configuration_changed_task(struct pp_hwmgr *hwmgr) if ((data->water_marks_bitmap & WaterMarksExist) && !(data->water_marks_bitmap & WaterMarksLoaded)) { - result = vega12_copy_table_to_smc(hwmgr, - (uint8_t *)wm_table, TABLE_WATERMARKS); + result = smum_smc_table_manager(hwmgr, + (uint8_t *)wm_table, TABLE_WATERMARKS, false); PP_ASSERT_WITH_CODE(result, "Failed to update WMTABLE!", return EINVAL); data->water_marks_bitmap |= WaterMarksLoaded; } @@ -2352,6 +2356,13 @@ static int vega12_gfx_off_control(struct pp_hwmgr *hwmgr, bool enable) return vega12_disable_gfx_off(hwmgr); } +static int vega12_get_performance_level(struct pp_hwmgr *hwmgr, const struct pp_hw_power_state *state, + PHM_PerformanceLevelDesignation designation, uint32_t index, + PHM_PerformanceLevel *level) +{ + return 0; +} + static const struct pp_hwmgr_func vega12_hwmgr_funcs = { .backend_init = vega12_hwmgr_backend_init, .backend_fini = vega12_hwmgr_backend_fini, @@ -2402,6 +2413,7 @@ static const struct pp_hwmgr_func vega12_hwmgr_funcs = { .register_irq_handlers = smu9_register_irq_handlers, .start_thermal_controller = vega12_start_thermal_controller, .powergate_gfx = vega12_gfx_off_control, + .get_performance_level = vega12_get_performance_level, }; int vega12_hwmgr_init(struct pp_hwmgr *hwmgr) diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_processpptables.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_processpptables.c index cb3a5b1737c8..9817f7a5ed29 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_processpptables.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega12_processpptables.c @@ -99,50 +99,6 @@ static int set_platform_caps(struct pp_hwmgr *hwmgr, uint32_t powerplay_caps) return 0; } -static int copy_clock_limits_array( - struct pp_hwmgr *hwmgr, - uint32_t **pptable_info_array, - const uint32_t *pptable_array) -{ - uint32_t array_size, i; - uint32_t *table; - - array_size = sizeof(uint32_t) * ATOM_VEGA12_PPCLOCK_COUNT; - - table = kzalloc(array_size, GFP_KERNEL); - if (NULL == table) - return -ENOMEM; - - for (i = 0; i < ATOM_VEGA12_PPCLOCK_COUNT; i++) - table[i] = pptable_array[i]; - - *pptable_info_array = table; - - return 0; -} - -static int copy_overdrive_settings_limits_array( - struct pp_hwmgr *hwmgr, - uint32_t **pptable_info_array, - const uint32_t *pptable_array) -{ - uint32_t array_size, i; - uint32_t *table; - - array_size = sizeof(uint32_t) * ATOM_VEGA12_ODSETTING_COUNT; - - table = kzalloc(array_size, GFP_KERNEL); - if (NULL == table) - return -ENOMEM; - - for (i = 0; i < ATOM_VEGA12_ODSETTING_COUNT; i++) - table[i] = pptable_array[i]; - - *pptable_info_array = table; - - return 0; -} - static int append_vbios_pptable(struct pp_hwmgr *hwmgr, PPTable_t *ppsmc_pptable) { struct pp_atomfwctrl_smc_dpm_parameters smc_dpm_table; @@ -250,14 +206,22 @@ static int init_powerplay_table_information( phm_cap_set(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_MicrocodeFanControl); - if (powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_GFXCLKFMAX] > VEGA12_ENGINECLOCK_HARDMAX) + if (le32_to_cpu(powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_GFXCLKFMAX]) > VEGA12_ENGINECLOCK_HARDMAX) hwmgr->platform_descriptor.overdriveLimit.engineClock = VEGA12_ENGINECLOCK_HARDMAX; else - hwmgr->platform_descriptor.overdriveLimit.engineClock = powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_GFXCLKFMAX]; - hwmgr->platform_descriptor.overdriveLimit.memoryClock = powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_UCLKFMAX]; - - copy_overdrive_settings_limits_array(hwmgr, &pptable_information->od_settings_max, powerplay_table->ODSettingsMax); - copy_overdrive_settings_limits_array(hwmgr, &pptable_information->od_settings_min, powerplay_table->ODSettingsMin); + hwmgr->platform_descriptor.overdriveLimit.engineClock = + le32_to_cpu(powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_GFXCLKFMAX]); + hwmgr->platform_descriptor.overdriveLimit.memoryClock = + le32_to_cpu(powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_UCLKFMAX]); + + phm_copy_overdrive_settings_limits_array(hwmgr, + &pptable_information->od_settings_max, + powerplay_table->ODSettingsMax, + ATOM_VEGA12_ODSETTING_COUNT); + phm_copy_overdrive_settings_limits_array(hwmgr, + &pptable_information->od_settings_min, + powerplay_table->ODSettingsMin, + ATOM_VEGA12_ODSETTING_COUNT); /* hwmgr->platformDescriptor.minOverdriveVDDC = 0; hwmgr->platformDescriptor.maxOverdriveVDDC = 0; @@ -267,15 +231,15 @@ static int init_powerplay_table_information( && hwmgr->platform_descriptor.overdriveLimit.memoryClock > 0) phm_cap_set(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_ACOverdriveSupport); - pptable_information->us_small_power_limit1 = powerplay_table->usSmallPowerLimit1; - pptable_information->us_small_power_limit2 = powerplay_table->usSmallPowerLimit2; - pptable_information->us_boost_power_limit = powerplay_table->usBoostPowerLimit; - pptable_information->us_od_turbo_power_limit = powerplay_table->usODTurboPowerLimit; - pptable_information->us_od_powersave_power_limit = powerplay_table->usODPowerSavePowerLimit; + pptable_information->us_small_power_limit1 = le16_to_cpu(powerplay_table->usSmallPowerLimit1); + pptable_information->us_small_power_limit2 = le16_to_cpu(powerplay_table->usSmallPowerLimit2); + pptable_information->us_boost_power_limit = le16_to_cpu(powerplay_table->usBoostPowerLimit); + pptable_information->us_od_turbo_power_limit = le16_to_cpu(powerplay_table->usODTurboPowerLimit); + pptable_information->us_od_powersave_power_limit = le16_to_cpu(powerplay_table->usODPowerSavePowerLimit); - pptable_information->us_software_shutdown_temp = powerplay_table->usSoftwareShutdownTemp; + pptable_information->us_software_shutdown_temp = le16_to_cpu(powerplay_table->usSoftwareShutdownTemp); - hwmgr->platform_descriptor.TDPODLimit = (uint16_t)powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_POWERPERCENTAGE]; + hwmgr->platform_descriptor.TDPODLimit = le32_to_cpu(powerplay_table->ODSettingsMax[ATOM_VEGA12_ODSETTING_POWERPERCENTAGE]); disable_power_control = 0; if (!disable_power_control) { @@ -285,8 +249,8 @@ static int init_powerplay_table_information( PHM_PlatformCaps_PowerControl); } - copy_clock_limits_array(hwmgr, &pptable_information->power_saving_clock_max, powerplay_table->PowerSavingClockMax); - copy_clock_limits_array(hwmgr, &pptable_information->power_saving_clock_min, powerplay_table->PowerSavingClockMin); + phm_copy_clock_limits_array(hwmgr, &pptable_information->power_saving_clock_max, powerplay_table->PowerSavingClockMax, ATOM_VEGA12_PPCLOCK_COUNT); + phm_copy_clock_limits_array(hwmgr, &pptable_information->power_saving_clock_min, powerplay_table->PowerSavingClockMin, ATOM_VEGA12_PPCLOCK_COUNT); pptable_information->smc_pptable = (PPTable_t *)kmalloc(sizeof(PPTable_t), GFP_KERNEL); if (pptable_information->smc_pptable == NULL) diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c index d45cbfe8e184..57143d51e3ee 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_hwmgr.c @@ -46,6 +46,9 @@ #include "ppinterrupt.h" #include "pp_overdriver.h" #include "pp_thermal.h" +#include "soc15_common.h" +#include "smuio/smuio_9_0_offset.h" +#include "smuio/smuio_9_0_sh_mask.h" static void vega20_set_default_registry_data(struct pp_hwmgr *hwmgr) { @@ -461,7 +464,7 @@ static int vega20_get_number_of_dpm_level(struct pp_hwmgr *hwmgr, "[GetNumOfDpmLevel] failed to get dpm levels!", return ret); - vega20_read_arg_from_smc(hwmgr, num_of_levels); + *num_of_levels = smum_get_argument(hwmgr); PP_ASSERT_WITH_CODE(*num_of_levels > 0, "[GetNumOfDpmLevel] number of clk levels is invalid!", return -EINVAL); @@ -481,7 +484,7 @@ static int vega20_get_dpm_frequency_by_index(struct pp_hwmgr *hwmgr, "[GetDpmFreqByIndex] failed to get dpm freq by index!", return ret); - vega20_read_arg_from_smc(hwmgr, clk); + *clk = smum_get_argument(hwmgr); PP_ASSERT_WITH_CODE(*clk, "[GetDpmFreqByIndex] clk value is invalid!", return -EINVAL); @@ -743,8 +746,8 @@ static int vega20_init_smc_table(struct pp_hwmgr *hwmgr) memcpy(pp_table, pptable_information->smc_pptable, sizeof(PPTable_t)); - result = vega20_copy_table_to_smc(hwmgr, - (uint8_t *)pp_table, TABLE_PPTABLE); + result = smum_smc_table_manager(hwmgr, + (uint8_t *)pp_table, TABLE_PPTABLE, false); PP_ASSERT_WITH_CODE(!result, "[InitSMCTable] Failed to upload PPtable!", return result); @@ -1044,7 +1047,7 @@ static int vega20_od8_get_gfx_clock_base_voltage( "[GetBaseVoltage] failed to get GFXCLK AVFS voltage from SMU!", return ret); - vega20_read_arg_from_smc(hwmgr, voltage); + *voltage = smum_get_argument(hwmgr); *voltage = *voltage / VOLTAGE_SCALE; return 0; @@ -1067,7 +1070,7 @@ static int vega20_od8_initialize_default_settings( vega20_od8_set_feature_id(hwmgr); /* Set default values */ - ret = vega20_copy_table_from_smc(hwmgr, (uint8_t *)od_table, TABLE_OVERDRIVE); + ret = smum_smc_table_manager(hwmgr, (uint8_t *)od_table, TABLE_OVERDRIVE, true); PP_ASSERT_WITH_CODE(!ret, "Failed to export over drive table!", return ret); @@ -1195,7 +1198,7 @@ static int vega20_od8_initialize_default_settings( } } - ret = vega20_copy_table_to_smc(hwmgr, (uint8_t *)od_table, TABLE_OVERDRIVE); + ret = smum_smc_table_manager(hwmgr, (uint8_t *)od_table, TABLE_OVERDRIVE, false); PP_ASSERT_WITH_CODE(!ret, "Failed to import over drive table!", return ret); @@ -1214,7 +1217,7 @@ static int vega20_od8_set_settings( struct vega20_od8_single_setting *od8_settings = data->od8_settings.od8_settings_array; - ret = vega20_copy_table_from_smc(hwmgr, (uint8_t *)(&od_table), TABLE_OVERDRIVE); + ret = smum_smc_table_manager(hwmgr, (uint8_t *)(&od_table), TABLE_OVERDRIVE, true); PP_ASSERT_WITH_CODE(!ret, "Failed to export over drive table!", return ret); @@ -1271,7 +1274,7 @@ static int vega20_od8_set_settings( break; } - ret = vega20_copy_table_to_smc(hwmgr, (uint8_t *)(&od_table), TABLE_OVERDRIVE); + ret = smum_smc_table_manager(hwmgr, (uint8_t *)(&od_table), TABLE_OVERDRIVE, false); PP_ASSERT_WITH_CODE(!ret, "Failed to import over drive table!", return ret); @@ -1401,7 +1404,7 @@ static int vega20_get_max_sustainable_clock(struct pp_hwmgr *hwmgr, (clock_select << 16))) == 0, "[GetMaxSustainableClock] Failed to get max DC clock from SMC!", return ret); - vega20_read_arg_from_smc(hwmgr, clock); + *clock = smum_get_argument(hwmgr); /* if DC limit is zero, return AC limit */ if (*clock == 0) { @@ -1410,7 +1413,7 @@ static int vega20_get_max_sustainable_clock(struct pp_hwmgr *hwmgr, (clock_select << 16))) == 0, "[GetMaxSustainableClock] failed to get max AC clock from SMC!", return ret); - vega20_read_arg_from_smc(hwmgr, clock); + *clock = smum_get_argument(hwmgr); } return 0; @@ -1474,6 +1477,19 @@ static int vega20_init_max_sustainable_clocks(struct pp_hwmgr *hwmgr) return 0; } +static int vega20_enable_mgpu_fan_boost(struct pp_hwmgr *hwmgr) +{ + int result; + + result = smum_send_msg_to_smc(hwmgr, + PPSMC_MSG_SetMGpuFanBoostLimitRpm); + PP_ASSERT_WITH_CODE(!result, + "[EnableMgpuFan] Failed to enable mgpu fan boost!", + return result); + + return 0; +} + static void vega20_init_powergate_state(struct pp_hwmgr *hwmgr) { struct vega20_hwmgr *data = @@ -1544,6 +1560,14 @@ static int vega20_enable_dpm_tasks(struct pp_hwmgr *hwmgr) "[EnableDPMTasks] Failed to populate umdpstate clocks!", return result); + result = smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_GetPptLimit, + POWER_SOURCE_AC << 16); + PP_ASSERT_WITH_CODE(!result, + "[GetPptLimit] get default PPT limit failed!", + return result); + hwmgr->power_limit = + hwmgr->default_power_limit = smum_get_argument(hwmgr); + return 0; } @@ -1770,14 +1794,14 @@ static int vega20_get_clock_ranges(struct pp_hwmgr *hwmgr, PPSMC_MSG_GetMaxDpmFreq, (clock_select << 16))) == 0, "[GetClockRanges] Failed to get max clock from SMC!", return ret); - vega20_read_arg_from_smc(hwmgr, clock); + *clock = smum_get_argument(hwmgr); } else { PP_ASSERT_WITH_CODE((ret = smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_GetMinDpmFreq, (clock_select << 16))) == 0, "[GetClockRanges] Failed to get min clock from SMC!", return ret); - vega20_read_arg_from_smc(hwmgr, clock); + *clock = smum_get_argument(hwmgr); } return 0; @@ -1841,7 +1865,7 @@ static int vega20_get_gpu_power(struct pp_hwmgr *hwmgr, int ret = 0; SmuMetrics_t metrics_table; - ret = vega20_copy_table_from_smc(hwmgr, (uint8_t *)&metrics_table, TABLE_SMU_METRICS); + ret = smum_smc_table_manager(hwmgr, (uint8_t *)&metrics_table, TABLE_SMU_METRICS, true); PP_ASSERT_WITH_CODE(!ret, "Failed to export SMU METRICS table!", return ret); @@ -1851,38 +1875,20 @@ static int vega20_get_gpu_power(struct pp_hwmgr *hwmgr, return ret; } -static int vega20_get_current_gfx_clk_freq(struct pp_hwmgr *hwmgr, uint32_t *gfx_freq) -{ - uint32_t gfx_clk = 0; - int ret = 0; - - *gfx_freq = 0; - - PP_ASSERT_WITH_CODE((ret = smum_send_msg_to_smc_with_parameter(hwmgr, - PPSMC_MSG_GetDpmClockFreq, (PPCLK_GFXCLK << 16))) == 0, - "[GetCurrentGfxClkFreq] Attempt to get Current GFXCLK Frequency Failed!", - return ret); - vega20_read_arg_from_smc(hwmgr, &gfx_clk); - - *gfx_freq = gfx_clk * 100; - - return 0; -} - -static int vega20_get_current_mclk_freq(struct pp_hwmgr *hwmgr, uint32_t *mclk_freq) +static int vega20_get_current_clk_freq(struct pp_hwmgr *hwmgr, + PPCLK_e clk_id, uint32_t *clk_freq) { - uint32_t mem_clk = 0; int ret = 0; - *mclk_freq = 0; + *clk_freq = 0; PP_ASSERT_WITH_CODE((ret = smum_send_msg_to_smc_with_parameter(hwmgr, - PPSMC_MSG_GetDpmClockFreq, (PPCLK_UCLK << 16))) == 0, - "[GetCurrentMClkFreq] Attempt to get Current MCLK Frequency Failed!", + PPSMC_MSG_GetDpmClockFreq, (clk_id << 16))) == 0, + "[GetCurrentClkFreq] Attempt to get Current Frequency Failed!", return ret); - vega20_read_arg_from_smc(hwmgr, &mem_clk); + *clk_freq = smum_get_argument(hwmgr); - *mclk_freq = mem_clk * 100; + *clk_freq = *clk_freq * 100; return 0; } @@ -1893,7 +1899,7 @@ static int vega20_get_current_activity_percent(struct pp_hwmgr *hwmgr, int ret = 0; SmuMetrics_t metrics_table; - ret = vega20_copy_table_from_smc(hwmgr, (uint8_t *)&metrics_table, TABLE_SMU_METRICS); + ret = smum_smc_table_manager(hwmgr, (uint8_t *)&metrics_table, TABLE_SMU_METRICS, true); PP_ASSERT_WITH_CODE(!ret, "Failed to export SMU METRICS table!", return ret); @@ -1907,16 +1913,22 @@ static int vega20_read_sensor(struct pp_hwmgr *hwmgr, int idx, void *value, int *size) { struct vega20_hwmgr *data = (struct vega20_hwmgr *)(hwmgr->backend); + struct amdgpu_device *adev = hwmgr->adev; + uint32_t val_vid; int ret = 0; switch (idx) { case AMDGPU_PP_SENSOR_GFX_SCLK: - ret = vega20_get_current_gfx_clk_freq(hwmgr, (uint32_t *)value); + ret = vega20_get_current_clk_freq(hwmgr, + PPCLK_GFXCLK, + (uint32_t *)value); if (!ret) *size = 4; break; case AMDGPU_PP_SENSOR_GFX_MCLK: - ret = vega20_get_current_mclk_freq(hwmgr, (uint32_t *)value); + ret = vega20_get_current_clk_freq(hwmgr, + PPCLK_UCLK, + (uint32_t *)value); if (!ret) *size = 4; break; @@ -1941,6 +1953,18 @@ static int vega20_read_sensor(struct pp_hwmgr *hwmgr, int idx, *size = 16; ret = vega20_get_gpu_power(hwmgr, (uint32_t *)value); break; + case AMDGPU_PP_SENSOR_VDDGFX: + val_vid = (RREG32_SOC15(SMUIO, 0, mmSMUSVI0_TEL_PLANE0) & + SMUSVI0_TEL_PLANE0__SVI0_PLANE0_VDDCOR_MASK) >> + SMUSVI0_TEL_PLANE0__SVI0_PLANE0_VDDCOR__SHIFT; + *((uint32_t *)value) = + (uint32_t)convert_to_vddc((uint8_t)val_vid); + break; + case AMDGPU_PP_SENSOR_ENABLED_SMC_FEATURES_MASK: + ret = vega20_get_enabled_smc_features(hwmgr, (uint64_t *)value); + if (!ret) + *size = 8; + break; default: ret = -EINVAL; break; @@ -1974,7 +1998,6 @@ int vega20_display_clock_voltage_request(struct pp_hwmgr *hwmgr, if (data->smu_features[GNLD_DPM_DCEFCLK].enabled) { switch (clk_type) { case amd_pp_dcef_clock: - clk_freq = clock_req->clock_freq_in_khz / 100; clk_select = PPCLK_DCEFCLK; break; case amd_pp_disp_clock: @@ -2003,11 +2026,20 @@ int vega20_display_clock_voltage_request(struct pp_hwmgr *hwmgr, return result; } +static int vega20_get_performance_level(struct pp_hwmgr *hwmgr, const struct pp_hw_power_state *state, + PHM_PerformanceLevelDesignation designation, uint32_t index, + PHM_PerformanceLevel *level) +{ + return 0; +} + static int vega20_notify_smc_display_config_after_ps_adjustment( struct pp_hwmgr *hwmgr) { struct vega20_hwmgr *data = (struct vega20_hwmgr *)(hwmgr->backend); + struct vega20_single_dpm_table *dpm_table = + &data->dpm_table.mem_table; struct PP_Clocks min_clocks = {0}; struct pp_display_clock_request clock_req; int ret = 0; @@ -2025,7 +2057,7 @@ static int vega20_notify_smc_display_config_after_ps_adjustment( if (data->smu_features[GNLD_DPM_DCEFCLK].supported) { clock_req.clock_type = amd_pp_dcef_clock; - clock_req.clock_freq_in_khz = min_clocks.dcefClock; + clock_req.clock_freq_in_khz = min_clocks.dcefClock * 10; if (!vega20_display_clock_voltage_request(hwmgr, &clock_req)) { if (data->smu_features[GNLD_DS_DCEFCLK].supported) PP_ASSERT_WITH_CODE((ret = smum_send_msg_to_smc_with_parameter( @@ -2038,6 +2070,15 @@ static int vega20_notify_smc_display_config_after_ps_adjustment( } } + if (data->smu_features[GNLD_DPM_UCLK].enabled) { + dpm_table->dpm_state.hard_min_level = min_clocks.memoryClock / 100; + PP_ASSERT_WITH_CODE(!(ret = smum_send_msg_to_smc_with_parameter(hwmgr, + PPSMC_MSG_SetHardMinByFreq, + (PPCLK_UCLK << 16 ) | dpm_table->dpm_state.hard_min_level)), + "[SetHardMinFreq] Set hard min uclk failed!", + return ret); + } + return 0; } @@ -2264,6 +2305,25 @@ static uint32_t vega20_get_fan_control_mode(struct pp_hwmgr *hwmgr) return AMD_FAN_CTRL_AUTO; } +static void vega20_set_fan_control_mode(struct pp_hwmgr *hwmgr, uint32_t mode) +{ + switch (mode) { + case AMD_FAN_CTRL_NONE: + vega20_fan_ctrl_set_fan_speed_percent(hwmgr, 100); + break; + case AMD_FAN_CTRL_MANUAL: + if (PP_CAP(PHM_PlatformCaps_MicrocodeFanControl)) + vega20_fan_ctrl_stop_smc_fan_control(hwmgr); + break; + case AMD_FAN_CTRL_AUTO: + if (PP_CAP(PHM_PlatformCaps_MicrocodeFanControl)) + vega20_fan_ctrl_start_smc_fan_control(hwmgr); + break; + default: + break; + } +} + static int vega20_get_dal_power_level(struct pp_hwmgr *hwmgr, struct amd_pp_simple_clock_info *info) { @@ -2296,7 +2356,7 @@ static int vega20_get_sclks(struct pp_hwmgr *hwmgr, for (i = 0; i < count; i++) { clocks->data[i].clocks_in_khz = - dpm_table->dpm_levels[i].value * 100; + dpm_table->dpm_levels[i].value * 1000; clocks->data[i].latency_in_us = 0; } @@ -2326,7 +2386,7 @@ static int vega20_get_memclocks(struct pp_hwmgr *hwmgr, for (i = 0; i < count; i++) { clocks->data[i].clocks_in_khz = data->mclk_latency_table.entries[i].frequency = - dpm_table->dpm_levels[i].value * 100; + dpm_table->dpm_levels[i].value * 1000; clocks->data[i].latency_in_us = data->mclk_latency_table.entries[i].latency = vega20_get_mem_latency(hwmgr, dpm_table->dpm_levels[i].value); @@ -2351,7 +2411,7 @@ static int vega20_get_dcefclocks(struct pp_hwmgr *hwmgr, for (i = 0; i < count; i++) { clocks->data[i].clocks_in_khz = - dpm_table->dpm_levels[i].value * 100; + dpm_table->dpm_levels[i].value * 1000; clocks->data[i].latency_in_us = 0; } @@ -2374,7 +2434,7 @@ static int vega20_get_socclocks(struct pp_hwmgr *hwmgr, for (i = 0; i < count; i++) { clocks->data[i].clocks_in_khz = - dpm_table->dpm_levels[i].value * 100; + dpm_table->dpm_levels[i].value * 1000; clocks->data[i].latency_in_us = 0; } @@ -2525,11 +2585,11 @@ static int vega20_odn_edit_dpm_table(struct pp_hwmgr *hwmgr, return -EINVAL; } - if (input_clk < clocks.data[0].clocks_in_khz / 100 || + if (input_clk < clocks.data[0].clocks_in_khz / 1000 || input_clk > od8_settings[OD8_SETTING_UCLK_FMAX].max_value) { pr_info("clock freq %d is not within allowed range [%d - %d]\n", input_clk, - clocks.data[0].clocks_in_khz / 100, + clocks.data[0].clocks_in_khz / 1000, od8_settings[OD8_SETTING_UCLK_FMAX].max_value); return -EINVAL; } @@ -2612,18 +2672,18 @@ static int vega20_odn_edit_dpm_table(struct pp_hwmgr *hwmgr, data->gfxclk_overdrive = false; data->memclk_overdrive = false; - ret = vega20_copy_table_from_smc(hwmgr, - (uint8_t *)od_table, - TABLE_OVERDRIVE); + ret = smum_smc_table_manager(hwmgr, + (uint8_t *)od_table, + TABLE_OVERDRIVE, true); PP_ASSERT_WITH_CODE(!ret, "Failed to export overdrive table!", return ret); break; case PP_OD_COMMIT_DPM_TABLE: - ret = vega20_copy_table_to_smc(hwmgr, - (uint8_t *)od_table, - TABLE_OVERDRIVE); + ret = smum_smc_table_manager(hwmgr, + (uint8_t *)od_table, + TABLE_OVERDRIVE, false); PP_ASSERT_WITH_CODE(!ret, "Failed to import overdrive table!", return ret); @@ -2669,7 +2729,7 @@ static int vega20_print_clock_levels(struct pp_hwmgr *hwmgr, switch (type) { case PP_SCLK: - ret = vega20_get_current_gfx_clk_freq(hwmgr, &now); + ret = vega20_get_current_clk_freq(hwmgr, PPCLK_GFXCLK, &now); PP_ASSERT_WITH_CODE(!ret, "Attempt to get current gfx clk Failed!", return ret); @@ -2681,12 +2741,12 @@ static int vega20_print_clock_levels(struct pp_hwmgr *hwmgr, for (i = 0; i < clocks.num_levels; i++) size += sprintf(buf + size, "%d: %uMhz %s\n", - i, clocks.data[i].clocks_in_khz / 100, + i, clocks.data[i].clocks_in_khz / 1000, (clocks.data[i].clocks_in_khz == now) ? "*" : ""); break; case PP_MCLK: - ret = vega20_get_current_mclk_freq(hwmgr, &now); + ret = vega20_get_current_clk_freq(hwmgr, PPCLK_UCLK, &now); PP_ASSERT_WITH_CODE(!ret, "Attempt to get current mclk freq Failed!", return ret); @@ -2698,7 +2758,7 @@ static int vega20_print_clock_levels(struct pp_hwmgr *hwmgr, for (i = 0; i < clocks.num_levels; i++) size += sprintf(buf + size, "%d: %uMhz %s\n", - i, clocks.data[i].clocks_in_khz / 100, + i, clocks.data[i].clocks_in_khz / 1000, (clocks.data[i].clocks_in_khz == now) ? "*" : ""); break; @@ -2763,7 +2823,7 @@ static int vega20_print_clock_levels(struct pp_hwmgr *hwmgr, return ret); size += sprintf(buf + size, "MCLK: %7uMhz %10uMhz\n", - clocks.data[0].clocks_in_khz / 100, + clocks.data[0].clocks_in_khz / 1000, od8_settings[OD8_SETTING_UCLK_FMAX].max_value); } @@ -2847,8 +2907,8 @@ static int vega20_display_configuration_changed_task(struct pp_hwmgr *hwmgr) if ((data->water_marks_bitmap & WaterMarksExist) && !(data->water_marks_bitmap & WaterMarksLoaded)) { - result = vega20_copy_table_to_smc(hwmgr, - (uint8_t *)wm_table, TABLE_WATERMARKS); + result = smum_smc_table_manager(hwmgr, + (uint8_t *)wm_table, TABLE_WATERMARKS, false); PP_ASSERT_WITH_CODE(!result, "Failed to update WMTABLE!", return result); @@ -3118,6 +3178,34 @@ static int vega20_power_off_asic(struct pp_hwmgr *hwmgr) return result; } +static int conv_power_profile_to_pplib_workload(int power_profile) +{ + int pplib_workload = 0; + + switch (power_profile) { + case PP_SMC_POWER_PROFILE_FULLSCREEN3D: + pplib_workload = WORKLOAD_PPLIB_FULL_SCREEN_3D_BIT; + break; + case PP_SMC_POWER_PROFILE_POWERSAVING: + pplib_workload = WORKLOAD_PPLIB_POWER_SAVING_BIT; + break; + case PP_SMC_POWER_PROFILE_VIDEO: + pplib_workload = WORKLOAD_PPLIB_VIDEO_BIT; + break; + case PP_SMC_POWER_PROFILE_VR: + pplib_workload = WORKLOAD_PPLIB_VR_BIT; + break; + case PP_SMC_POWER_PROFILE_COMPUTE: + pplib_workload = WORKLOAD_PPLIB_COMPUTE_BIT; + break; + case PP_SMC_POWER_PROFILE_CUSTOM: + pplib_workload = WORKLOAD_PPLIB_CUSTOM_BIT; + break; + } + + return pplib_workload; +} + static int vega20_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf) { DpmActivityMonitorCoeffInt_t activity_monitor; @@ -3153,14 +3241,14 @@ static int vega20_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf) for (i = 0; i <= PP_SMC_POWER_PROFILE_CUSTOM; i++) { /* conv PP_SMC_POWER_PROFILE* to WORKLOAD_PPLIB_*_BIT */ - workload_type = i + 1; + workload_type = conv_power_profile_to_pplib_workload(i); result = vega20_get_activity_monitor_coeff(hwmgr, (uint8_t *)(&activity_monitor), workload_type); PP_ASSERT_WITH_CODE(!result, "[GetPowerProfile] Failed to get activity monitor!", return result); - size += sprintf(buf + size, "%2d(%14s%s)\n", + size += sprintf(buf + size, "%2d %14s%s:\n", i, profile_name[i], (i == hwmgr->power_profile_mode) ? "*" : " "); size += sprintf(buf + size, "%19s %d(%13s) %7d %7d %7d %7d %7d %7d %7d %7d %7d\n", @@ -3226,10 +3314,15 @@ static int vega20_get_power_profile_mode(struct pp_hwmgr *hwmgr, char *buf) static int vega20_set_power_profile_mode(struct pp_hwmgr *hwmgr, long *input, uint32_t size) { DpmActivityMonitorCoeffInt_t activity_monitor; - int result = 0; + int workload_type, result = 0; hwmgr->power_profile_mode = input[size]; + if (hwmgr->power_profile_mode > PP_SMC_POWER_PROFILE_CUSTOM) { + pr_err("Invalid power profile mode %d\n", hwmgr->power_profile_mode); + return -EINVAL; + } + if (hwmgr->power_profile_mode == PP_SMC_POWER_PROFILE_CUSTOM) { if (size < 10) return -EINVAL; @@ -3296,8 +3389,11 @@ static int vega20_set_power_profile_mode(struct pp_hwmgr *hwmgr, long *input, ui return result); } + /* conv PP_SMC_POWER_PROFILE* to WORKLOAD_PPLIB_*_BIT */ + workload_type = + conv_power_profile_to_pplib_workload(hwmgr->power_profile_mode); smum_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_SetWorkloadMask, - 1 << hwmgr->power_profile_mode); + 1 << workload_type); return 0; } @@ -3383,6 +3479,8 @@ static const struct pp_hwmgr_func vega20_hwmgr_funcs = { vega20_set_watermarks_for_clocks_ranges, .display_clock_voltage_request = vega20_display_clock_voltage_request, + .get_performance_level = + vega20_get_performance_level, /* UMD pstate, profile related */ .force_dpm_level = vega20_dpm_force_dpm_level, @@ -3427,15 +3525,25 @@ static const struct pp_hwmgr_func vega20_hwmgr_funcs = { .disable_smc_firmware_ctf = vega20_thermal_disable_alert, /* fan control related */ + .get_fan_speed_percent = + vega20_fan_ctrl_get_fan_speed_percent, + .set_fan_speed_percent = + vega20_fan_ctrl_set_fan_speed_percent, .get_fan_speed_info = vega20_fan_ctrl_get_fan_speed_info, .get_fan_speed_rpm = vega20_fan_ctrl_get_fan_speed_rpm, + .set_fan_speed_rpm = + vega20_fan_ctrl_set_fan_speed_rpm, .get_fan_control_mode = vega20_get_fan_control_mode, + .set_fan_control_mode = + vega20_set_fan_control_mode, /* smu memory related */ .notify_cac_buffer_info = vega20_notify_cac_buffer_info, + .enable_mgpu_fan_boost = + vega20_enable_mgpu_fan_boost, }; int vega20_hwmgr_init(struct pp_hwmgr *hwmgr) diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_processpptables.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_processpptables.c index 5f1f7a32ac24..97f8a1a970c3 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_processpptables.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_processpptables.c @@ -100,9 +100,8 @@ static void dump_pptable(PPTable_t *pptable) pr_info("PpmTemperatureThreshold = %d\n", pptable->PpmTemperatureThreshold); pr_info("MemoryOnPackage = 0x%02x\n", pptable->MemoryOnPackage); - pr_info("padding8_limits[0] = 0x%02x\n", pptable->padding8_limits[0]); - pr_info("padding8_limits[1] = 0x%02x\n", pptable->padding8_limits[1]); - pr_info("padding8_limits[2] = 0x%02x\n", pptable->padding8_limits[2]); + pr_info("padding8_limits = 0x%02x\n", pptable->padding8_limits); + pr_info("Tvr_SocLimit = %d\n", pptable->Tvr_SocLimit); pr_info("UlvVoltageOffsetSoc = %d\n", pptable->UlvVoltageOffsetSoc); pr_info("UlvVoltageOffsetGfx = %d\n", pptable->UlvVoltageOffsetGfx); @@ -417,8 +416,8 @@ static void dump_pptable(PPTable_t *pptable) pr_info("FanGainEdge = %d\n", pptable->FanGainEdge); pr_info("FanGainHotspot = %d\n", pptable->FanGainHotspot); pr_info("FanGainLiquid = %d\n", pptable->FanGainLiquid); - pr_info("FanGainVrVddc = %d\n", pptable->FanGainVrVddc); - pr_info("FanGainVrMvdd = %d\n", pptable->FanGainVrMvdd); + pr_info("FanGainVrGfx = %d\n", pptable->FanGainVrGfx); + pr_info("FanGainVrSoc = %d\n", pptable->FanGainVrSoc); pr_info("FanGainPlx = %d\n", pptable->FanGainPlx); pr_info("FanGainHbm = %d\n", pptable->FanGainHbm); pr_info("FanPwmMin = %d\n", pptable->FanPwmMin); @@ -533,23 +532,20 @@ static void dump_pptable(PPTable_t *pptable) pr_info("MinVoltageUlvGfx = %d\n", pptable->MinVoltageUlvGfx); pr_info("MinVoltageUlvSoc = %d\n", pptable->MinVoltageUlvSoc); - for (i = 0; i < 14; i++) - pr_info("Reserved[%d] = 0x%x\n", i, pptable->Reserved[i]); + pr_info("MGpuFanBoostLimitRpm = %d\n", pptable->MGpuFanBoostLimitRpm); + pr_info("padding16_Fan = %d\n", pptable->padding16_Fan); + + pr_info("FanGainVrMem0 = %d\n", pptable->FanGainVrMem0); + pr_info("FanGainVrMem0 = %d\n", pptable->FanGainVrMem0); - pr_info("Liquid1_I2C_address = 0x%x\n", pptable->Liquid1_I2C_address); - pr_info("Liquid2_I2C_address = 0x%x\n", pptable->Liquid2_I2C_address); - pr_info("Vr_I2C_address = 0x%x\n", pptable->Vr_I2C_address); - pr_info("Plx_I2C_address = 0x%x\n", pptable->Plx_I2C_address); + pr_info("DcBtcGb[AVFS_VOLTAGE_GFX] = 0x%x\n", pptable->DcBtcGb[AVFS_VOLTAGE_GFX]); + pr_info("DcBtcGb[AVFS_VOLTAGE_SOC] = 0x%x\n", pptable->DcBtcGb[AVFS_VOLTAGE_SOC]); - pr_info("Liquid_I2C_LineSCL = 0x%x\n", pptable->Liquid_I2C_LineSCL); - pr_info("Liquid_I2C_LineSDA = 0x%x\n", pptable->Liquid_I2C_LineSDA); - pr_info("Vr_I2C_LineSCL = 0x%x\n", pptable->Vr_I2C_LineSCL); - pr_info("Vr_I2C_LineSDA = 0x%x\n", pptable->Vr_I2C_LineSDA); + for (i = 0; i < 11; i++) + pr_info("Reserved[%d] = 0x%x\n", i, pptable->Reserved[i]); - pr_info("Plx_I2C_LineSCL = 0x%x\n", pptable->Plx_I2C_LineSCL); - pr_info("Plx_I2C_LineSDA = 0x%x\n", pptable->Plx_I2C_LineSDA); - pr_info("VrSensorPresent = 0x%x\n", pptable->VrSensorPresent); - pr_info("LiquidSensorPresent = 0x%x\n", pptable->LiquidSensorPresent); + for (i = 0; i < 3; i++) + pr_info("Padding32[%d] = 0x%x\n", i, pptable->Padding32[i]); pr_info("MaxVoltageStepGfx = 0x%x\n", pptable->MaxVoltageStepGfx); pr_info("MaxVoltageStepSoc = 0x%x\n", pptable->MaxVoltageStepSoc); @@ -611,6 +607,24 @@ static void dump_pptable(PPTable_t *pptable) pr_info("FllGfxclkSpreadPercent = %d\n", pptable->FllGfxclkSpreadPercent); pr_info("FllGfxclkSpreadFreq = %d\n", pptable->FllGfxclkSpreadFreq); + for (i = 0; i < I2C_CONTROLLER_NAME_COUNT; i++) { + pr_info("I2cControllers[%d]:\n", i); + pr_info(" .Enabled = %d\n", + pptable->I2cControllers[i].Enabled); + pr_info(" .SlaveAddress = 0x%x\n", + pptable->I2cControllers[i].SlaveAddress); + pr_info(" .ControllerPort = %d\n", + pptable->I2cControllers[i].ControllerPort); + pr_info(" .ControllerName = %d\n", + pptable->I2cControllers[i].ControllerName); + pr_info(" .ThermalThrottler = %d\n", + pptable->I2cControllers[i].ThermalThrottler); + pr_info(" .I2cProtocol = %d\n", + pptable->I2cControllers[i].I2cProtocol); + pr_info(" .I2cSpeed = %d\n", + pptable->I2cControllers[i].I2cSpeed); + } + for (i = 0; i < 10; i++) pr_info("BoardReserved[%d] = 0x%x\n", i, pptable->BoardReserved[i]); @@ -628,8 +642,14 @@ static int check_powerplay_tables( "Unsupported PPTable format!", return -1); PP_ASSERT_WITH_CODE(powerplay_table->sHeader.structuresize > 0, "Invalid PowerPlay Table!", return -1); - PP_ASSERT_WITH_CODE(powerplay_table->smcPPTable.Version == PPTABLE_V20_SMU_VERSION, - "Unmatch PPTable version, vbios update may be needed!", return -1); + + if (powerplay_table->smcPPTable.Version != PPTABLE_V20_SMU_VERSION) { + pr_info("Unmatch PPTable version: " + "pptable from VBIOS is V%d while driver supported is V%d!", + powerplay_table->smcPPTable.Version, + PPTABLE_V20_SMU_VERSION); + return -EINVAL; + } //dump_pptable(&powerplay_table->smcPPTable); @@ -661,50 +681,6 @@ static int set_platform_caps(struct pp_hwmgr *hwmgr, uint32_t powerplay_caps) return 0; } -static int copy_clock_limits_array( - struct pp_hwmgr *hwmgr, - uint32_t **pptable_info_array, - const uint32_t *pptable_array, - uint32_t power_saving_clock_count) -{ - uint32_t array_size, i; - uint32_t *table; - - array_size = sizeof(uint32_t) * power_saving_clock_count; - table = kzalloc(array_size, GFP_KERNEL); - if (NULL == table) - return -ENOMEM; - - for (i = 0; i < power_saving_clock_count; i++) - table[i] = pptable_array[i]; - - *pptable_info_array = table; - - return 0; -} - -static int copy_overdrive_settings_limits_array( - struct pp_hwmgr *hwmgr, - uint32_t **pptable_info_array, - const uint32_t *pptable_array, - uint32_t od_setting_count) -{ - uint32_t array_size, i; - uint32_t *table; - - array_size = sizeof(uint32_t) * od_setting_count; - table = kzalloc(array_size, GFP_KERNEL); - if (NULL == table) - return -ENOMEM; - - for (i = 0; i < od_setting_count; i++) - table[i] = pptable_array[i]; - - *pptable_info_array = table; - - return 0; -} - static int copy_overdrive_feature_capabilities_array( struct pp_hwmgr *hwmgr, uint8_t **pptable_info_array, @@ -721,7 +697,7 @@ static int copy_overdrive_feature_capabilities_array( return -ENOMEM; for (i = 0; i < od_feature_count; i++) { - table[i] = pptable_array[i]; + table[i] = le32_to_cpu(pptable_array[i]); if (table[i]) od_supported = true; } @@ -737,29 +713,15 @@ static int copy_overdrive_feature_capabilities_array( static int append_vbios_pptable(struct pp_hwmgr *hwmgr, PPTable_t *ppsmc_pptable) { - struct atom_smc_dpm_info_v4_3 *smc_dpm_table; + struct atom_smc_dpm_info_v4_4 *smc_dpm_table; int index = GetIndexIntoMasterDataTable(smc_dpm_info); + int i; PP_ASSERT_WITH_CODE( smc_dpm_table = smu_atom_get_data_table(hwmgr->adev, index, NULL, NULL, NULL), "[appendVbiosPPTable] Failed to retrieve Smc Dpm Table from VBIOS!", return -1); - ppsmc_pptable->Liquid1_I2C_address = smc_dpm_table->liquid1_i2c_address; - ppsmc_pptable->Liquid2_I2C_address = smc_dpm_table->liquid2_i2c_address; - ppsmc_pptable->Vr_I2C_address = smc_dpm_table->vr_i2c_address; - ppsmc_pptable->Plx_I2C_address = smc_dpm_table->plx_i2c_address; - - ppsmc_pptable->Liquid_I2C_LineSCL = smc_dpm_table->liquid_i2c_linescl; - ppsmc_pptable->Liquid_I2C_LineSDA = smc_dpm_table->liquid_i2c_linesda; - ppsmc_pptable->Vr_I2C_LineSCL = smc_dpm_table->vr_i2c_linescl; - ppsmc_pptable->Vr_I2C_LineSDA = smc_dpm_table->vr_i2c_linesda; - - ppsmc_pptable->Plx_I2C_LineSCL = smc_dpm_table->plx_i2c_linescl; - ppsmc_pptable->Plx_I2C_LineSDA = smc_dpm_table->plx_i2c_linesda; - ppsmc_pptable->VrSensorPresent = smc_dpm_table->vrsensorpresent; - ppsmc_pptable->LiquidSensorPresent = smc_dpm_table->liquidsensorpresent; - ppsmc_pptable->MaxVoltageStepGfx = smc_dpm_table->maxvoltagestepgfx; ppsmc_pptable->MaxVoltageStepSoc = smc_dpm_table->maxvoltagestepsoc; @@ -818,6 +780,21 @@ static int append_vbios_pptable(struct pp_hwmgr *hwmgr, PPTable_t *ppsmc_pptable ppsmc_pptable->FllGfxclkSpreadPercent = smc_dpm_table->fllgfxclkspreadpercent; ppsmc_pptable->FllGfxclkSpreadFreq = smc_dpm_table->fllgfxclkspreadfreq; + for (i = 0; i < I2C_CONTROLLER_NAME_COUNT; i++) { + ppsmc_pptable->I2cControllers[i].Enabled = + smc_dpm_table->i2ccontrollers[i].enabled; + ppsmc_pptable->I2cControllers[i].SlaveAddress = + smc_dpm_table->i2ccontrollers[i].slaveaddress; + ppsmc_pptable->I2cControllers[i].ControllerPort = + smc_dpm_table->i2ccontrollers[i].controllerport; + ppsmc_pptable->I2cControllers[i].ThermalThrottler = + smc_dpm_table->i2ccontrollers[i].thermalthrottler; + ppsmc_pptable->I2cControllers[i].I2cProtocol = + smc_dpm_table->i2ccontrollers[i].i2cprotocol; + ppsmc_pptable->I2cControllers[i].I2cSpeed = + smc_dpm_table->i2ccontrollers[i].i2cspeed; + } + return 0; } @@ -834,6 +811,8 @@ static int init_powerplay_table_information( hwmgr->thermal_controller.ucType = powerplay_table->ucThermalControllerType; pptable_information->uc_thermal_controller_type = powerplay_table->ucThermalControllerType; + hwmgr->thermal_controller.fanInfo.ulMinRPM = 0; + hwmgr->thermal_controller.fanInfo.ulMaxRPM = powerplay_table->smcPPTable.FanMaximumRpm; set_hw_cap(hwmgr, ATOM_VEGA20_PP_THERMALCONTROLLER_NONE != hwmgr->thermal_controller.ucType, @@ -842,34 +821,40 @@ static int init_powerplay_table_information( phm_cap_set(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_MicrocodeFanControl); if (powerplay_table->OverDrive8Table.ucODTableRevision == 1) { - od_feature_count = (powerplay_table->OverDrive8Table.ODFeatureCount > ATOM_VEGA20_ODFEATURE_COUNT) ? - ATOM_VEGA20_ODFEATURE_COUNT : powerplay_table->OverDrive8Table.ODFeatureCount; - od_setting_count = (powerplay_table->OverDrive8Table.ODSettingCount > ATOM_VEGA20_ODSETTING_COUNT) ? - ATOM_VEGA20_ODSETTING_COUNT : powerplay_table->OverDrive8Table.ODSettingCount; + od_feature_count = + (le32_to_cpu(powerplay_table->OverDrive8Table.ODFeatureCount) > + ATOM_VEGA20_ODFEATURE_COUNT) ? + ATOM_VEGA20_ODFEATURE_COUNT : + le32_to_cpu(powerplay_table->OverDrive8Table.ODFeatureCount); + od_setting_count = + (le32_to_cpu(powerplay_table->OverDrive8Table.ODSettingCount) > + ATOM_VEGA20_ODSETTING_COUNT) ? + ATOM_VEGA20_ODSETTING_COUNT : + le32_to_cpu(powerplay_table->OverDrive8Table.ODSettingCount); copy_overdrive_feature_capabilities_array(hwmgr, &pptable_information->od_feature_capabilities, powerplay_table->OverDrive8Table.ODFeatureCapabilities, od_feature_count); - copy_overdrive_settings_limits_array(hwmgr, + phm_copy_overdrive_settings_limits_array(hwmgr, &pptable_information->od_settings_max, powerplay_table->OverDrive8Table.ODSettingsMax, od_setting_count); - copy_overdrive_settings_limits_array(hwmgr, + phm_copy_overdrive_settings_limits_array(hwmgr, &pptable_information->od_settings_min, powerplay_table->OverDrive8Table.ODSettingsMin, od_setting_count); } - pptable_information->us_small_power_limit1 = powerplay_table->usSmallPowerLimit1; - pptable_information->us_small_power_limit2 = powerplay_table->usSmallPowerLimit2; - pptable_information->us_boost_power_limit = powerplay_table->usBoostPowerLimit; - pptable_information->us_od_turbo_power_limit = powerplay_table->usODTurboPowerLimit; - pptable_information->us_od_powersave_power_limit = powerplay_table->usODPowerSavePowerLimit; + pptable_information->us_small_power_limit1 = le16_to_cpu(powerplay_table->usSmallPowerLimit1); + pptable_information->us_small_power_limit2 = le16_to_cpu(powerplay_table->usSmallPowerLimit2); + pptable_information->us_boost_power_limit = le16_to_cpu(powerplay_table->usBoostPowerLimit); + pptable_information->us_od_turbo_power_limit = le16_to_cpu(powerplay_table->usODTurboPowerLimit); + pptable_information->us_od_powersave_power_limit = le16_to_cpu(powerplay_table->usODPowerSavePowerLimit); - pptable_information->us_software_shutdown_temp = powerplay_table->usSoftwareShutdownTemp; + pptable_information->us_software_shutdown_temp = le16_to_cpu(powerplay_table->usSoftwareShutdownTemp); - hwmgr->platform_descriptor.TDPODLimit = (uint16_t)powerplay_table->OverDrive8Table.ODSettingsMax[ATOM_VEGA20_ODSETTING_POWERPERCENTAGE]; + hwmgr->platform_descriptor.TDPODLimit = le32_to_cpu(powerplay_table->OverDrive8Table.ODSettingsMax[ATOM_VEGA20_ODSETTING_POWERPERCENTAGE]); disable_power_control = 0; if (!disable_power_control && hwmgr->platform_descriptor.TDPODLimit) @@ -877,13 +862,16 @@ static int init_powerplay_table_information( phm_cap_set(hwmgr->platform_descriptor.platformCaps, PHM_PlatformCaps_PowerControl); if (powerplay_table->PowerSavingClockTable.ucTableRevision == 1) { - power_saving_clock_count = (powerplay_table->PowerSavingClockTable.PowerSavingClockCount >= ATOM_VEGA20_PPCLOCK_COUNT) ? - ATOM_VEGA20_PPCLOCK_COUNT : powerplay_table->PowerSavingClockTable.PowerSavingClockCount; - copy_clock_limits_array(hwmgr, + power_saving_clock_count = + (le32_to_cpu(powerplay_table->PowerSavingClockTable.PowerSavingClockCount) >= + ATOM_VEGA20_PPCLOCK_COUNT) ? + ATOM_VEGA20_PPCLOCK_COUNT : + le32_to_cpu(powerplay_table->PowerSavingClockTable.PowerSavingClockCount); + phm_copy_clock_limits_array(hwmgr, &pptable_information->power_saving_clock_max, powerplay_table->PowerSavingClockTable.PowerSavingClockMax, power_saving_clock_count); - copy_clock_limits_array(hwmgr, + phm_copy_clock_limits_array(hwmgr, &pptable_information->power_saving_clock_min, powerplay_table->PowerSavingClockTable.PowerSavingClockMin, power_saving_clock_count); @@ -893,7 +881,10 @@ static int init_powerplay_table_information( if (pptable_information->smc_pptable == NULL) return -ENOMEM; - memcpy(pptable_information->smc_pptable, &(powerplay_table->smcPPTable), sizeof(PPTable_t)); + memcpy(pptable_information->smc_pptable, + &(powerplay_table->smcPPTable), + sizeof(PPTable_t)); + result = append_vbios_pptable(hwmgr, (pptable_information->smc_pptable)); diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.c b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.c index 2984ddd5428c..ede54e87e287 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.c +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.c @@ -29,6 +29,78 @@ #include "soc15_common.h" #include "pp_debug.h" +static int vega20_disable_fan_control_feature(struct pp_hwmgr *hwmgr) +{ + struct vega20_hwmgr *data = hwmgr->backend; + int ret = 0; + + if (data->smu_features[GNLD_FAN_CONTROL].supported) { + ret = vega20_enable_smc_features( + hwmgr, false, + data->smu_features[GNLD_FAN_CONTROL]. + smu_feature_bitmap); + PP_ASSERT_WITH_CODE(!ret, + "Disable FAN CONTROL feature Failed!", + return ret); + data->smu_features[GNLD_FAN_CONTROL].enabled = false; + } + + return ret; +} + +int vega20_fan_ctrl_stop_smc_fan_control(struct pp_hwmgr *hwmgr) +{ + struct vega20_hwmgr *data = hwmgr->backend; + + if (data->smu_features[GNLD_FAN_CONTROL].supported) + return vega20_disable_fan_control_feature(hwmgr); + + return 0; +} + +static int vega20_enable_fan_control_feature(struct pp_hwmgr *hwmgr) +{ + struct vega20_hwmgr *data = hwmgr->backend; + int ret = 0; + + if (data->smu_features[GNLD_FAN_CONTROL].supported) { + ret = vega20_enable_smc_features( + hwmgr, true, + data->smu_features[GNLD_FAN_CONTROL]. + smu_feature_bitmap); + PP_ASSERT_WITH_CODE(!ret, + "Enable FAN CONTROL feature Failed!", + return ret); + data->smu_features[GNLD_FAN_CONTROL].enabled = true; + } + + return ret; +} + +int vega20_fan_ctrl_start_smc_fan_control(struct pp_hwmgr *hwmgr) +{ + struct vega20_hwmgr *data = hwmgr->backend; + + if (data->smu_features[GNLD_FAN_CONTROL].supported) + return vega20_enable_fan_control_feature(hwmgr); + + return 0; +} + +static int vega20_fan_ctrl_set_static_mode(struct pp_hwmgr *hwmgr, uint32_t mode) +{ + struct amdgpu_device *adev = hwmgr->adev; + + WREG32_SOC15(THM, 0, mmCG_FDO_CTRL2, + REG_SET_FIELD(RREG32_SOC15(THM, 0, mmCG_FDO_CTRL2), + CG_FDO_CTRL2, TMIN, 0)); + WREG32_SOC15(THM, 0, mmCG_FDO_CTRL2, + REG_SET_FIELD(RREG32_SOC15(THM, 0, mmCG_FDO_CTRL2), + CG_FDO_CTRL2, FDO_PWM_MODE, mode)); + + return 0; +} + static int vega20_get_current_rpm(struct pp_hwmgr *hwmgr, uint32_t *current_rpm) { int ret = 0; @@ -37,20 +109,67 @@ static int vega20_get_current_rpm(struct pp_hwmgr *hwmgr, uint32_t *current_rpm) PPSMC_MSG_GetCurrentRpm)) == 0, "Attempt to get current RPM from SMC Failed!", return ret); - PP_ASSERT_WITH_CODE((ret = vega20_read_arg_from_smc(hwmgr, - current_rpm)) == 0, - "Attempt to read current RPM from SMC Failed!", - return ret); + *current_rpm = smum_get_argument(hwmgr); return 0; } +int vega20_fan_ctrl_get_fan_speed_percent(struct pp_hwmgr *hwmgr, + uint32_t *speed) +{ + struct vega20_hwmgr *data = (struct vega20_hwmgr *)(hwmgr->backend); + PPTable_t *pp_table = &(data->smc_state_table.pp_table); + uint32_t current_rpm, percent = 0; + int ret = 0; + + ret = vega20_get_current_rpm(hwmgr, ¤t_rpm); + if (ret) + return ret; + + percent = current_rpm * 100 / pp_table->FanMaximumRpm; + + *speed = percent > 100 ? 100 : percent; + + return 0; +} + +int vega20_fan_ctrl_set_fan_speed_percent(struct pp_hwmgr *hwmgr, + uint32_t speed) +{ + struct amdgpu_device *adev = hwmgr->adev; + uint32_t duty100; + uint32_t duty; + uint64_t tmp64; + + if (speed > 100) + speed = 100; + + if (PP_CAP(PHM_PlatformCaps_MicrocodeFanControl)) + vega20_fan_ctrl_stop_smc_fan_control(hwmgr); + + duty100 = REG_GET_FIELD(RREG32_SOC15(THM, 0, mmCG_FDO_CTRL1), + CG_FDO_CTRL1, FMAX_DUTY100); + + if (duty100 == 0) + return -EINVAL; + + tmp64 = (uint64_t)speed * duty100; + do_div(tmp64, 100); + duty = (uint32_t)tmp64; + + WREG32_SOC15(THM, 0, mmCG_FDO_CTRL0, + REG_SET_FIELD(RREG32_SOC15(THM, 0, mmCG_FDO_CTRL0), + CG_FDO_CTRL0, FDO_STATIC_DUTY, duty)); + + return vega20_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC); +} + int vega20_fan_ctrl_get_fan_speed_info(struct pp_hwmgr *hwmgr, struct phm_fan_speed_info *fan_speed_info) { memset(fan_speed_info, 0, sizeof(*fan_speed_info)); - fan_speed_info->supports_percent_read = false; - fan_speed_info->supports_percent_write = false; + fan_speed_info->supports_percent_read = true; + fan_speed_info->supports_percent_write = true; fan_speed_info->supports_rpm_read = true; fan_speed_info->supports_rpm_write = true; @@ -64,6 +183,31 @@ int vega20_fan_ctrl_get_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t *speed) return vega20_get_current_rpm(hwmgr, speed); } +int vega20_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t speed) +{ + struct amdgpu_device *adev = hwmgr->adev; + uint32_t tach_period, crystal_clock_freq; + int result = 0; + + if (!speed) + return -EINVAL; + + if (PP_CAP(PHM_PlatformCaps_MicrocodeFanControl)) { + result = vega20_fan_ctrl_stop_smc_fan_control(hwmgr); + if (result) + return result; + } + + crystal_clock_freq = amdgpu_asic_get_xclk((struct amdgpu_device *)hwmgr->adev); + tach_period = 60 * crystal_clock_freq * 10000 / (8 * speed); + WREG32_SOC15(THM, 0, mmCG_TACH_CTRL, + REG_SET_FIELD(RREG32_SOC15(THM, 0, mmCG_TACH_CTRL), + CG_TACH_CTRL, TARGET_PERIOD, + tach_period)); + + return vega20_fan_ctrl_set_static_mode(hwmgr, FDO_PWM_MODE_STATIC_RPM); +} + /** * Reads the remote temperature from the SIslands thermal controller. * diff --git a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.h b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.h index 2a6d49fec4e0..2d1769bbd24e 100644 --- a/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.h +++ b/drivers/gpu/drm/amd/powerplay/hwmgr/vega20_thermal.h @@ -50,15 +50,22 @@ struct vega20_temperature { #define FDO_PWM_MODE_STATIC_RPM 5 extern int vega20_thermal_get_temperature(struct pp_hwmgr *hwmgr); -extern int vega20_thermal_stop_thermal_controller(struct pp_hwmgr *hwmgr); extern int vega20_fan_ctrl_get_fan_speed_info(struct pp_hwmgr *hwmgr, struct phm_fan_speed_info *fan_speed_info); -extern int vega20_fan_ctrl_reset_fan_speed_to_default(struct pp_hwmgr *hwmgr); extern int vega20_fan_ctrl_get_fan_speed_rpm(struct pp_hwmgr *hwmgr, uint32_t *speed); +extern int vega20_fan_ctrl_set_fan_speed_rpm(struct pp_hwmgr *hwmgr, + uint32_t speed); +extern int vega20_fan_ctrl_get_fan_speed_percent(struct pp_hwmgr *hwmgr, + uint32_t *speed); +extern int vega20_fan_ctrl_set_fan_speed_percent(struct pp_hwmgr *hwmgr, + uint32_t speed); +extern int vega20_fan_ctrl_stop_smc_fan_control(struct pp_hwmgr *hwmgr); +extern int vega20_fan_ctrl_start_smc_fan_control(struct pp_hwmgr *hwmgr); extern int vega20_thermal_disable_alert(struct pp_hwmgr *hwmgr); extern int vega20_start_thermal_controller(struct pp_hwmgr *hwmgr, struct PP_TemperatureRange *range); +extern int vega20_thermal_stop_thermal_controller(struct pp_hwmgr *hwmgr); #endif diff --git a/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h b/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h index a6d92128b19c..e5a60aa44b5d 100644 --- a/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h +++ b/drivers/gpu/drm/amd/powerplay/inc/hwmgr.h @@ -328,6 +328,8 @@ struct pp_hwmgr_func { int (*set_power_limit)(struct pp_hwmgr *hwmgr, uint32_t n); int (*powergate_mmhub)(struct pp_hwmgr *hwmgr); int (*smus_notify_pwe)(struct pp_hwmgr *hwmgr); + int (*powergate_sdma)(struct pp_hwmgr *hwmgr, bool bgate); + int (*enable_mgpu_fan_boost)(struct pp_hwmgr *hwmgr); }; struct pp_table_func { @@ -732,7 +734,6 @@ struct pp_hwmgr { void *smu_backend; const struct pp_smumgr_func *smumgr_funcs; bool is_kicker; - bool reload_fw; enum PP_DAL_POWERLEVEL dal_power_level; struct phm_dynamic_state_info dyn_state; diff --git a/drivers/gpu/drm/amd/powerplay/inc/smu11_driver_if.h b/drivers/gpu/drm/amd/powerplay/inc/smu11_driver_if.h index 71191deb4e76..63d5cf691549 100644 --- a/drivers/gpu/drm/amd/powerplay/inc/smu11_driver_if.h +++ b/drivers/gpu/drm/amd/powerplay/inc/smu11_driver_if.h @@ -27,9 +27,9 @@ // *** IMPORTANT *** // SMU TEAM: Always increment the interface version if // any structure is changed in this file -#define SMU11_DRIVER_IF_VERSION 0x11 +#define SMU11_DRIVER_IF_VERSION 0x12 -#define PPTABLE_V20_SMU_VERSION 2 +#define PPTABLE_V20_SMU_VERSION 3 #define NUM_GFXCLK_DPM_LEVELS 16 #define NUM_VCLK_DPM_LEVELS 8 @@ -165,7 +165,7 @@ #define FEATURE_DS_FCLK_MASK (1 << FEATURE_DS_FCLK_BIT ) #define FEATURE_DS_MP1CLK_MASK (1 << FEATURE_DS_MP1CLK_BIT ) #define FEATURE_DS_MP0CLK_MASK (1 << FEATURE_DS_MP0CLK_BIT ) - +#define FEATURE_XGMI_MASK (1 << FEATURE_XGMI_BIT ) #define DPM_OVERRIDE_DISABLE_SOCCLK_PID 0x00000001 #define DPM_OVERRIDE_DISABLE_UCLK_PID 0x00000002 @@ -186,6 +186,9 @@ #define DPM_OVERRIDE_ENABLE_GFXOFF_UCLK_SWITCH 0x00010000 #define DPM_OVERRIDE_ENABLE_GFXOFF_FCLK_SWITCH 0x00020000 +#define I2C_CONTROLLER_ENABLED 1 +#define I2C_CONTROLLER_DISABLED 0 + #define VR_MAPPING_VR_SELECT_MASK 0x01 #define VR_MAPPING_VR_SELECT_SHIFT 0x00 @@ -208,15 +211,17 @@ #define THROTTLER_STATUS_TEMP_HOTSPOT_BIT 2 #define THROTTLER_STATUS_TEMP_HBM_BIT 3 #define THROTTLER_STATUS_TEMP_VR_GFX_BIT 4 -#define THROTTLER_STATUS_TEMP_VR_MEM_BIT 5 -#define THROTTLER_STATUS_TEMP_LIQUID_BIT 6 -#define THROTTLER_STATUS_TEMP_PLX_BIT 7 -#define THROTTLER_STATUS_TEMP_SKIN_BIT 8 -#define THROTTLER_STATUS_TDC_GFX_BIT 9 -#define THROTTLER_STATUS_TDC_SOC_BIT 10 -#define THROTTLER_STATUS_PPT_BIT 11 -#define THROTTLER_STATUS_FIT_BIT 12 -#define THROTTLER_STATUS_PPM_BIT 13 +#define THROTTLER_STATUS_TEMP_VR_SOC_BIT 5 +#define THROTTLER_STATUS_TEMP_VR_MEM0_BIT 6 +#define THROTTLER_STATUS_TEMP_VR_MEM1_BIT 7 +#define THROTTLER_STATUS_TEMP_LIQUID_BIT 8 +#define THROTTLER_STATUS_TEMP_PLX_BIT 9 +#define THROTTLER_STATUS_TEMP_SKIN_BIT 10 +#define THROTTLER_STATUS_TDC_GFX_BIT 11 +#define THROTTLER_STATUS_TDC_SOC_BIT 12 +#define THROTTLER_STATUS_PPT_BIT 13 +#define THROTTLER_STATUS_FIT_BIT 14 +#define THROTTLER_STATUS_PPM_BIT 15 #define TABLE_TRANSFER_OK 0x0 @@ -236,6 +241,58 @@ #define XGMI_STATE_D0 1 #define XGMI_STATE_D3 0 +typedef enum { + I2C_CONTROLLER_PORT_0 = 0, + I2C_CONTROLLER_PORT_1 = 1, +} I2cControllerPort_e; + +typedef enum { + I2C_CONTROLLER_NAME_VR_GFX = 0, + I2C_CONTROLLER_NAME_VR_SOC, + I2C_CONTROLLER_NAME_VR_VDDCI, + I2C_CONTROLLER_NAME_VR_HBM, + I2C_CONTROLLER_NAME_LIQUID_0, + I2C_CONTROLLER_NAME_LIQUID_1, + I2C_CONTROLLER_NAME_PLX, + I2C_CONTROLLER_NAME_COUNT, +} I2cControllerName_e; + +typedef enum { + I2C_CONTROLLER_THROTTLER_TYPE_NONE = 0, + I2C_CONTROLLER_THROTTLER_VR_GFX, + I2C_CONTROLLER_THROTTLER_VR_SOC, + I2C_CONTROLLER_THROTTLER_VR_VDDCI, + I2C_CONTROLLER_THROTTLER_VR_HBM, + I2C_CONTROLLER_THROTTLER_LIQUID_0, + I2C_CONTROLLER_THROTTLER_LIQUID_1, + I2C_CONTROLLER_THROTTLER_PLX, +} I2cControllerThrottler_e; + +typedef enum { + I2C_CONTROLLER_PROTOCOL_VR_XPDE132G5, + I2C_CONTROLLER_PROTOCOL_VR_IR35217, + I2C_CONTROLLER_PROTOCOL_TMP_TMP102A, + I2C_CONTROLLER_PROTOCOL_SPARE_0, + I2C_CONTROLLER_PROTOCOL_SPARE_1, + I2C_CONTROLLER_PROTOCOL_SPARE_2, +} I2cControllerProtocol_e; + +typedef enum { + I2C_CONTROLLER_SPEED_SLOW = 0, + I2C_CONTROLLER_SPEED_FAST = 1, +} I2cControllerSpeed_e; + +typedef struct { + uint32_t Enabled; + uint32_t SlaveAddress; + uint32_t ControllerPort; + uint32_t ControllerName; + + uint32_t ThermalThrottler; + uint32_t I2cProtocol; + uint32_t I2cSpeed; +} I2cControllerConfig_t; + typedef struct { uint32_t a; uint32_t b; @@ -269,6 +326,12 @@ typedef enum { } PPCLK_e; typedef enum { + POWER_SOURCE_AC, + POWER_SOURCE_DC, + POWER_SOURCE_COUNT, +} POWER_SOURCE_e; + +typedef enum { VOLTAGE_MODE_AVFS = 0, VOLTAGE_MODE_AVFS_SS, VOLTAGE_MODE_SS, @@ -328,8 +391,8 @@ typedef struct { uint16_t PpmTemperatureThreshold; uint8_t MemoryOnPackage; - uint8_t padding8_limits[3]; - + uint8_t padding8_limits; + uint16_t Tvr_SocLimit; uint16_t UlvVoltageOffsetSoc; uint16_t UlvVoltageOffsetGfx; @@ -400,8 +463,8 @@ typedef struct { uint16_t FanGainEdge; uint16_t FanGainHotspot; uint16_t FanGainLiquid; - uint16_t FanGainVrVddc; - uint16_t FanGainVrMvdd; + uint16_t FanGainVrGfx; + uint16_t FanGainVrSoc; uint16_t FanGainPlx; uint16_t FanGainHbm; uint16_t FanPwmMin; @@ -438,7 +501,7 @@ typedef struct { uint8_t DcBtcEnabled[AVFS_VOLTAGE_COUNT]; uint8_t Padding8_GfxBtc[2]; - uint16_t DcBtcMin[AVFS_VOLTAGE_COUNT]; + int16_t DcBtcMin[AVFS_VOLTAGE_COUNT]; uint16_t DcBtcMax[AVFS_VOLTAGE_COUNT]; @@ -461,24 +524,14 @@ typedef struct { uint16_t MGpuFanBoostLimitRpm; uint16_t padding16_Fan; - uint32_t Reserved[13]; - + uint16_t FanGainVrMem0; + uint16_t FanGainVrMem1; + uint16_t DcBtcGb[AVFS_VOLTAGE_COUNT]; - uint8_t Liquid1_I2C_address; - uint8_t Liquid2_I2C_address; - uint8_t Vr_I2C_address; - uint8_t Plx_I2C_address; + uint32_t Reserved[11]; - uint8_t Liquid_I2C_LineSCL; - uint8_t Liquid_I2C_LineSDA; - uint8_t Vr_I2C_LineSCL; - uint8_t Vr_I2C_LineSDA; - - uint8_t Plx_I2C_LineSCL; - uint8_t Plx_I2C_LineSDA; - uint8_t VrSensorPresent; - uint8_t LiquidSensorPresent; + uint32_t Padding32[3]; uint16_t MaxVoltageStepGfx; uint16_t MaxVoltageStepSoc; @@ -545,6 +598,8 @@ typedef struct { uint8_t FllGfxclkSpreadPercent; uint16_t FllGfxclkSpreadFreq; + I2cControllerConfig_t I2cControllers[I2C_CONTROLLER_NAME_COUNT]; + uint32_t BoardReserved[10]; @@ -601,7 +656,9 @@ typedef struct { uint16_t TemperatureHotspot ; uint16_t TemperatureHBM ; uint16_t TemperatureVrGfx ; - uint16_t TemperatureVrMem ; + uint16_t TemperatureVrSoc ; + uint16_t TemperatureVrMem0 ; + uint16_t TemperatureVrMem1 ; uint16_t TemperatureLiquid ; uint16_t TemperaturePlx ; uint32_t ThrottlerStatus ; diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/ci_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/ci_smumgr.c index 18643e06bc6f..669bd0c2a16c 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/ci_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/ci_smumgr.c @@ -2269,11 +2269,13 @@ static uint32_t ci_get_offsetof(uint32_t type, uint32_t member) case DRAM_LOG_BUFF_SIZE: return offsetof(SMU7_SoftRegisters, DRAM_LOG_BUFF_SIZE); } + break; case SMU_Discrete_DpmTable: switch (member) { case LowSclkInterruptThreshold: return offsetof(SMU7_Discrete_DpmTable, LowSclkInterruptT); } + break; } pr_debug("can't get the offset of type %x member %x\n", type, member); return 0; diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c index ec14798e87b6..bc8375cbf297 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/fiji_smumgr.c @@ -302,16 +302,6 @@ static int fiji_start_smu(struct pp_hwmgr *hwmgr) hwmgr->avfs_supported = false; } - /* To initialize all clock gating before RLC loaded and running.*/ - amdgpu_device_ip_set_clockgating_state(hwmgr->adev, - AMD_IP_BLOCK_TYPE_GFX, AMD_CG_STATE_GATE); - amdgpu_device_ip_set_clockgating_state(hwmgr->adev, - AMD_IP_BLOCK_TYPE_GMC, AMD_CG_STATE_GATE); - amdgpu_device_ip_set_clockgating_state(hwmgr->adev, - AMD_IP_BLOCK_TYPE_SDMA, AMD_CG_STATE_GATE); - amdgpu_device_ip_set_clockgating_state(hwmgr->adev, - AMD_IP_BLOCK_TYPE_COMMON, AMD_CG_STATE_GATE); - /* Setup SoftRegsStart here for register lookup in case * DummyBackEnd is used and ProcessFirmwareHeader is not executed */ @@ -2331,6 +2321,7 @@ static uint32_t fiji_get_offsetof(uint32_t type, uint32_t member) case DRAM_LOG_BUFF_SIZE: return offsetof(SMU73_SoftRegisters, DRAM_LOG_BUFF_SIZE); } + break; case SMU_Discrete_DpmTable: switch (member) { case UvdBootLevel: @@ -2340,6 +2331,7 @@ static uint32_t fiji_get_offsetof(uint32_t type, uint32_t member) case LowSclkInterruptThreshold: return offsetof(SMU73_Discrete_DpmTable, LowSclkInterruptThreshold); } + break; } pr_warn("can't get the offset of type %x member %x\n", type, member); return 0; diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/iceland_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/iceland_smumgr.c index 73aa368a454e..375ccf6ff5f2 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/iceland_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/iceland_smumgr.c @@ -232,26 +232,25 @@ static int iceland_request_smu_load_specific_fw(struct pp_hwmgr *hwmgr, static int iceland_start_smu(struct pp_hwmgr *hwmgr) { + struct iceland_smumgr *priv = hwmgr->smu_backend; int result; - result = iceland_smu_upload_firmware_image(hwmgr); - if (result) - return result; - result = iceland_smu_start_smc(hwmgr); - if (result) - return result; - if (!smu7_is_smc_ram_running(hwmgr)) { - pr_info("smu not running, upload firmware again \n"); result = iceland_smu_upload_firmware_image(hwmgr); if (result) return result; - result = iceland_smu_start_smc(hwmgr); - if (result) - return result; + iceland_smu_start_smc(hwmgr); } + /* Setup SoftRegsStart here to visit the register UcodeLoadStatus + * to check fw loading state + */ + smu7_read_smc_sram_dword(hwmgr, + SMU71_FIRMWARE_HEADER_LOCATION + + offsetof(SMU71_Firmware_Header, SoftRegisters), + &(priv->smu7_data.soft_regs_start), 0x40000); + result = smu7_request_smu_load_fw(hwmgr); return result; @@ -2237,11 +2236,13 @@ static uint32_t iceland_get_offsetof(uint32_t type, uint32_t member) case DRAM_LOG_BUFF_SIZE: return offsetof(SMU71_SoftRegisters, DRAM_LOG_BUFF_SIZE); } + break; case SMU_Discrete_DpmTable: switch (member) { case LowSclkInterruptThreshold: return offsetof(SMU71_Discrete_DpmTable, LowSclkInterruptThreshold); } + break; } pr_warn("can't get the offset of type %x member %x\n", type, member); return 0; @@ -2662,7 +2663,7 @@ const struct pp_smumgr_func iceland_smu_funcs = { .smu_fini = &smu7_smu_fini, .start_smu = &iceland_start_smu, .check_fw_load_finish = &smu7_check_fw_load_finish, - .request_smu_load_fw = &smu7_reload_firmware, + .request_smu_load_fw = &smu7_request_smu_load_fw, .request_smu_load_specific_fw = &iceland_request_smu_load_specific_fw, .send_msg_to_smc = &smu7_send_msg_to_smc, .send_msg_to_smc_with_parameter = &smu7_send_msg_to_smc_with_parameter, diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/smu10_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/smu10_smumgr.c index 6f961dec2088..d0eb8ab50148 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/smu10_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/smu10_smumgr.c @@ -186,40 +186,12 @@ static int smu10_verify_smc_interface(struct pp_hwmgr *hwmgr) return 0; } -/* sdma is disabled by default in vbios, need to re-enable in driver */ -static void smu10_smc_enable_sdma(struct pp_hwmgr *hwmgr) -{ - smu10_send_msg_to_smc(hwmgr, - PPSMC_MSG_PowerUpSdma); -} - -static void smu10_smc_disable_sdma(struct pp_hwmgr *hwmgr) -{ - smu10_send_msg_to_smc(hwmgr, - PPSMC_MSG_PowerDownSdma); -} - -/* vcn is disabled by default in vbios, need to re-enable in driver */ -static void smu10_smc_enable_vcn(struct pp_hwmgr *hwmgr) -{ - smu10_send_msg_to_smc_with_parameter(hwmgr, - PPSMC_MSG_PowerUpVcn, 0); -} - -static void smu10_smc_disable_vcn(struct pp_hwmgr *hwmgr) -{ - smu10_send_msg_to_smc_with_parameter(hwmgr, - PPSMC_MSG_PowerDownVcn, 0); -} - static int smu10_smu_fini(struct pp_hwmgr *hwmgr) { struct smu10_smumgr *priv = (struct smu10_smumgr *)(hwmgr->smu_backend); if (priv) { - smu10_smc_disable_sdma(hwmgr); - smu10_smc_disable_vcn(hwmgr); amdgpu_bo_free_kernel(&priv->smu_tables.entry[SMU10_WMTABLE].handle, &priv->smu_tables.entry[SMU10_WMTABLE].mc_addr, &priv->smu_tables.entry[SMU10_WMTABLE].table); @@ -243,8 +215,7 @@ static int smu10_start_smu(struct pp_hwmgr *hwmgr) if (smu10_verify_smc_interface(hwmgr)) return -EINVAL; - smu10_smc_enable_sdma(hwmgr); - smu10_smc_enable_vcn(hwmgr); + return 0; } diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/smu7_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/smu7_smumgr.c index 186dafc7f166..3f51d545e8ff 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/smu7_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/smu7_smumgr.c @@ -302,44 +302,6 @@ int smu7_write_smc_sram_dword(struct pp_hwmgr *hwmgr, uint32_t smc_addr, uint32_ return 0; } -/* Convert the firmware type to SMU type mask. For MEC, we need to check all MEC related type */ - -static uint32_t smu7_get_mask_for_firmware_type(uint32_t fw_type) -{ - uint32_t result = 0; - - switch (fw_type) { - case UCODE_ID_SDMA0: - result = UCODE_ID_SDMA0_MASK; - break; - case UCODE_ID_SDMA1: - result = UCODE_ID_SDMA1_MASK; - break; - case UCODE_ID_CP_CE: - result = UCODE_ID_CP_CE_MASK; - break; - case UCODE_ID_CP_PFP: - result = UCODE_ID_CP_PFP_MASK; - break; - case UCODE_ID_CP_ME: - result = UCODE_ID_CP_ME_MASK; - break; - case UCODE_ID_CP_MEC: - case UCODE_ID_CP_MEC_JT1: - case UCODE_ID_CP_MEC_JT2: - result = UCODE_ID_CP_MEC_MASK; - break; - case UCODE_ID_RLC_G: - result = UCODE_ID_RLC_G_MASK; - break; - default: - pr_info("UCode type is out of range! \n"); - result = 0; - } - - return result; -} - static int smu7_populate_single_firmware_entry(struct pp_hwmgr *hwmgr, uint32_t fw_type, struct SMU_Entry *entry) @@ -381,10 +343,7 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr) uint32_t fw_to_load; int r = 0; - if (!hwmgr->reload_fw) { - pr_info("skip reloading...\n"); - return 0; - } + amdgpu_ucode_init_bo(hwmgr->adev); if (smu_data->soft_regs_start) cgs_write_ind_register(hwmgr->device, CGS_IND_REG__SMC, @@ -467,10 +426,13 @@ int smu7_request_smu_load_fw(struct pp_hwmgr *hwmgr) smu7_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_DRV_DRAM_ADDR_HI, upper_32_bits(smu_data->header_buffer.mc_addr)); smu7_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_DRV_DRAM_ADDR_LO, lower_32_bits(smu_data->header_buffer.mc_addr)); - if (smu7_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_LoadUcodes, fw_to_load)) - pr_err("Fail to Request SMU Load uCode"); + smu7_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_LoadUcodes, fw_to_load); - return r; + r = smu7_check_fw_load_finish(hwmgr, fw_to_load); + if (!r) + return 0; + + pr_err("SMU load firmware failed\n"); failed: kfree(smu_data->toc); @@ -482,13 +444,12 @@ failed: int smu7_check_fw_load_finish(struct pp_hwmgr *hwmgr, uint32_t fw_type) { struct smu7_smumgr *smu_data = (struct smu7_smumgr *)(hwmgr->smu_backend); - uint32_t fw_mask = smu7_get_mask_for_firmware_type(fw_type); uint32_t ret; ret = phm_wait_on_indirect_register(hwmgr, mmSMC_IND_INDEX_11, smu_data->soft_regs_start + smum_get_offsetof(hwmgr, SMU_SoftRegisters, UcodeLoadStatus), - fw_mask, fw_mask); + fw_type, fw_type); return ret; } diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/smu8_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/smu8_smumgr.c index f7e3bc22bb93..09b844ec3eab 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/smu8_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/smu8_smumgr.c @@ -71,7 +71,11 @@ static int smu8_send_msg_to_smc_async(struct pp_hwmgr *hwmgr, uint16_t msg) result = PHM_WAIT_FIELD_UNEQUAL(hwmgr, SMU_MP1_SRBM2P_RESP_0, CONTENT, 0); if (result != 0) { + /* Read the last message to SMU, to report actual cause */ + uint32_t val = cgs_read_register(hwmgr->device, + mmSMU_MP1_SRBM2P_MSG_0); pr_err("smu8_send_msg_to_smc_async (0x%04x) failed\n", msg); + pr_err("SMU still servicing msg (0x%04x)\n", val); return result; } @@ -658,11 +662,10 @@ static int smu8_request_smu_load_fw(struct pp_hwmgr *hwmgr) { struct smu8_smumgr *smu8_smu = hwmgr->smu_backend; uint32_t smc_address; + uint32_t fw_to_check = 0; + int ret; - if (!hwmgr->reload_fw) { - pr_info("skip reloading...\n"); - return 0; - } + amdgpu_ucode_init_bo(hwmgr->adev); smu8_smu_populate_firmware_entries(hwmgr); @@ -689,28 +692,9 @@ static int smu8_request_smu_load_fw(struct pp_hwmgr *hwmgr) smu8_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_ExecuteJob, smu8_smu->toc_entry_power_profiling_index); - return smu8_send_msg_to_smc_with_parameter(hwmgr, + smu8_send_msg_to_smc_with_parameter(hwmgr, PPSMC_MSG_ExecuteJob, smu8_smu->toc_entry_initialize_index); -} - -static int smu8_start_smu(struct pp_hwmgr *hwmgr) -{ - int ret = 0; - uint32_t fw_to_check = 0; - struct amdgpu_device *adev = hwmgr->adev; - - uint32_t index = SMN_MP1_SRAM_START_ADDR + - SMU8_FIRMWARE_HEADER_LOCATION + - offsetof(struct SMU8_Firmware_Header, Version); - - - if (hwmgr == NULL || hwmgr->device == NULL) - return -EINVAL; - - cgs_write_register(hwmgr->device, mmMP0PUB_IND_INDEX, index); - hwmgr->smu_version = cgs_read_register(hwmgr->device, mmMP0PUB_IND_DATA); - adev->pm.fw_version = hwmgr->smu_version >> 8; fw_to_check = UCODE_ID_RLC_G_MASK | UCODE_ID_SDMA0_MASK | @@ -724,17 +708,38 @@ static int smu8_start_smu(struct pp_hwmgr *hwmgr) if (hwmgr->chip_id == CHIP_STONEY) fw_to_check &= ~(UCODE_ID_SDMA1_MASK | UCODE_ID_CP_MEC_JT2_MASK); - ret = smu8_request_smu_load_fw(hwmgr); - if (ret) + ret = smu8_check_fw_load_finish(hwmgr, fw_to_check); + if (ret) { pr_err("SMU firmware load failed\n"); - - smu8_check_fw_load_finish(hwmgr, fw_to_check); + return ret; + } ret = smu8_load_mec_firmware(hwmgr); - if (ret) + if (ret) { pr_err("Mec Firmware load failed\n"); + return ret; + } - return ret; + return 0; +} + +static int smu8_start_smu(struct pp_hwmgr *hwmgr) +{ + struct amdgpu_device *adev = hwmgr->adev; + + uint32_t index = SMN_MP1_SRAM_START_ADDR + + SMU8_FIRMWARE_HEADER_LOCATION + + offsetof(struct SMU8_Firmware_Header, Version); + + + if (hwmgr == NULL || hwmgr->device == NULL) + return -EINVAL; + + cgs_write_register(hwmgr->device, mmMP0PUB_IND_INDEX, index); + hwmgr->smu_version = cgs_read_register(hwmgr->device, mmMP0PUB_IND_DATA); + adev->pm.fw_version = hwmgr->smu_version >> 8; + + return smu8_request_smu_load_fw(hwmgr); } static int smu8_smu_init(struct pp_hwmgr *hwmgr) diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c index ae8378ed32ee..3ed6c5f1e5cf 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/tonga_smumgr.c @@ -192,6 +192,7 @@ static int tonga_start_in_non_protection_mode(struct pp_hwmgr *hwmgr) static int tonga_start_smu(struct pp_hwmgr *hwmgr) { + struct tonga_smumgr *priv = hwmgr->smu_backend; int result; /* Only start SMC if SMC RAM is not running */ @@ -209,6 +210,14 @@ static int tonga_start_smu(struct pp_hwmgr *hwmgr) } } + /* Setup SoftRegsStart here to visit the register UcodeLoadStatus + * to check fw loading state + */ + smu7_read_smc_sram_dword(hwmgr, + SMU72_FIRMWARE_HEADER_LOCATION + + offsetof(SMU72_Firmware_Header, SoftRegisters), + &(priv->smu7_data.soft_regs_start), 0x40000); + result = smu7_request_smu_load_fw(hwmgr); return result; @@ -2619,6 +2628,7 @@ static uint32_t tonga_get_offsetof(uint32_t type, uint32_t member) case DRAM_LOG_BUFF_SIZE: return offsetof(SMU72_SoftRegisters, DRAM_LOG_BUFF_SIZE); } + break; case SMU_Discrete_DpmTable: switch (member) { case UvdBootLevel: @@ -2628,6 +2638,7 @@ static uint32_t tonga_get_offsetof(uint32_t type, uint32_t member) case LowSclkInterruptThreshold: return offsetof(SMU72_Discrete_DpmTable, LowSclkInterruptThreshold); } + break; } pr_warn("can't get the offset of type %x member %x\n", type, member); return 0; diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.c index 5d19115f410c..c81acc3192ad 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.c @@ -88,8 +88,18 @@ static int vega10_copy_table_to_smc(struct pp_hwmgr *hwmgr, return 0; } -static int vega10_get_smc_features(struct pp_hwmgr *hwmgr, - uint32_t *features_enabled) +int vega10_enable_smc_features(struct pp_hwmgr *hwmgr, + bool enable, uint32_t feature_mask) +{ + int msg = enable ? PPSMC_MSG_EnableSmuFeatures : + PPSMC_MSG_DisableSmuFeatures; + + return smum_send_msg_to_smc_with_parameter(hwmgr, + msg, feature_mask); +} + +int vega10_get_enabled_smc_features(struct pp_hwmgr *hwmgr, + uint64_t *features_enabled) { if (features_enabled == NULL) return -EINVAL; @@ -102,9 +112,9 @@ static int vega10_get_smc_features(struct pp_hwmgr *hwmgr, static bool vega10_is_dpm_running(struct pp_hwmgr *hwmgr) { - uint32_t features_enabled = 0; + uint64_t features_enabled = 0; - vega10_get_smc_features(hwmgr, &features_enabled); + vega10_get_enabled_smc_features(hwmgr, &features_enabled); if (features_enabled & SMC_DPM_FEATURES) return true; diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.h b/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.h index 424e868bc768..bad760f22624 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.h +++ b/drivers/gpu/drm/amd/powerplay/smumgr/vega10_smumgr.h @@ -42,6 +42,10 @@ struct vega10_smumgr { struct smu_table_array smu_tables; }; +int vega10_enable_smc_features(struct pp_hwmgr *hwmgr, + bool enable, uint32_t feature_mask); +int vega10_get_enabled_smc_features(struct pp_hwmgr *hwmgr, + uint64_t *features_enabled); #endif diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.c index 7f0e2109f40d..ddb801517667 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.c @@ -37,8 +37,8 @@ * @param hwmgr the address of the HW manager * @param table_id the driver's table ID to copy from */ -int vega12_copy_table_from_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id) +static int vega12_copy_table_from_smc(struct pp_hwmgr *hwmgr, + uint8_t *table, int16_t table_id) { struct vega12_smumgr *priv = (struct vega12_smumgr *)(hwmgr->smu_backend); @@ -75,8 +75,8 @@ int vega12_copy_table_from_smc(struct pp_hwmgr *hwmgr, * @param hwmgr the address of the HW manager * @param table_id the table to copy from */ -int vega12_copy_table_to_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id) +static int vega12_copy_table_to_smc(struct pp_hwmgr *hwmgr, + uint8_t *table, int16_t table_id) { struct vega12_smumgr *priv = (struct vega12_smumgr *)(hwmgr->smu_backend); @@ -351,6 +351,19 @@ static int vega12_start_smu(struct pp_hwmgr *hwmgr) return 0; } +static int vega12_smc_table_manager(struct pp_hwmgr *hwmgr, uint8_t *table, + uint16_t table_id, bool rw) +{ + int ret; + + if (rw) + ret = vega12_copy_table_from_smc(hwmgr, table, table_id); + else + ret = vega12_copy_table_to_smc(hwmgr, table, table_id); + + return ret; +} + const struct pp_smumgr_func vega12_smu_funcs = { .smu_init = &vega12_smu_init, .smu_fini = &vega12_smu_fini, @@ -362,4 +375,5 @@ const struct pp_smumgr_func vega12_smu_funcs = { .upload_pptable_settings = NULL, .is_dpm_running = vega12_is_dpm_running, .get_argument = smu9_get_argument, + .smc_table_manager = vega12_smc_table_manager, }; diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.h b/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.h index b285cbc04019..aeec965ce81f 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.h +++ b/drivers/gpu/drm/amd/powerplay/smumgr/vega12_smumgr.h @@ -48,10 +48,6 @@ struct vega12_smumgr { #define SMU_FEATURES_HIGH_MASK 0xFFFFFFFF00000000 #define SMU_FEATURES_HIGH_SHIFT 32 -int vega12_copy_table_from_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id); -int vega12_copy_table_to_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id); int vega12_enable_smc_features(struct pp_hwmgr *hwmgr, bool enable, uint64_t feature_mask); int vega12_get_enabled_smc_features(struct pp_hwmgr *hwmgr, diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.c index fe7f71079e0e..b7ff7d4d6f44 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.c @@ -148,19 +148,11 @@ static int vega20_send_msg_to_smc_with_parameter(struct pp_hwmgr *hwmgr, return (ret == PPSMC_Result_OK) ? 0 : -EIO; } -/* - * Retrieve an argument from SMC. - * @param hwmgr the address of the powerplay hardware manager. - * @param arg pointer to store the argument from SMC. - * @return Always return 0. - */ -int vega20_read_arg_from_smc(struct pp_hwmgr *hwmgr, uint32_t *arg) +static uint32_t vega20_get_argument(struct pp_hwmgr *hwmgr) { struct amdgpu_device *adev = hwmgr->adev; - *arg = RREG32_SOC15(MP1, 0, mmMP1_SMN_C2PMSG_82); - - return 0; + return RREG32_SOC15(MP1, 0, mmMP1_SMN_C2PMSG_82); } /* @@ -168,8 +160,8 @@ int vega20_read_arg_from_smc(struct pp_hwmgr *hwmgr, uint32_t *arg) * @param hwmgr the address of the HW manager * @param table_id the driver's table ID to copy from */ -int vega20_copy_table_from_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id) +static int vega20_copy_table_from_smc(struct pp_hwmgr *hwmgr, + uint8_t *table, int16_t table_id) { struct vega20_smumgr *priv = (struct vega20_smumgr *)(hwmgr->smu_backend); @@ -208,8 +200,8 @@ int vega20_copy_table_from_smc(struct pp_hwmgr *hwmgr, * @param hwmgr the address of the HW manager * @param table_id the table to copy from */ -int vega20_copy_table_to_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id) +static int vega20_copy_table_to_smc(struct pp_hwmgr *hwmgr, + uint8_t *table, int16_t table_id) { struct vega20_smumgr *priv = (struct vega20_smumgr *)(hwmgr->smu_backend); @@ -345,18 +337,12 @@ int vega20_get_enabled_smc_features(struct pp_hwmgr *hwmgr, PPSMC_MSG_GetEnabledSmuFeaturesLow)) == 0, "[GetEnabledSMCFeatures] Attemp to get SMU features Low failed!", return ret); - PP_ASSERT_WITH_CODE((ret = vega20_read_arg_from_smc(hwmgr, - &smc_features_low)) == 0, - "[GetEnabledSMCFeatures] Attemp to read SMU features Low argument failed!", - return ret); + smc_features_low = vega20_get_argument(hwmgr); PP_ASSERT_WITH_CODE((ret = vega20_send_msg_to_smc(hwmgr, PPSMC_MSG_GetEnabledSmuFeaturesHigh)) == 0, "[GetEnabledSMCFeatures] Attemp to get SMU features High failed!", return ret); - PP_ASSERT_WITH_CODE((ret = vega20_read_arg_from_smc(hwmgr, - &smc_features_high)) == 0, - "[GetEnabledSMCFeatures] Attemp to read SMU features High argument failed!", - return ret); + smc_features_high = vega20_get_argument(hwmgr); *features_enabled = ((((uint64_t)smc_features_low << SMU_FEATURES_LOW_SHIFT) & SMU_FEATURES_LOW_MASK) | (((uint64_t)smc_features_high << SMU_FEATURES_HIGH_SHIFT) & SMU_FEATURES_HIGH_MASK)); @@ -574,6 +560,19 @@ static bool vega20_is_dpm_running(struct pp_hwmgr *hwmgr) return false; } +static int vega20_smc_table_manager(struct pp_hwmgr *hwmgr, uint8_t *table, + uint16_t table_id, bool rw) +{ + int ret; + + if (rw) + ret = vega20_copy_table_from_smc(hwmgr, table, table_id); + else + ret = vega20_copy_table_to_smc(hwmgr, table, table_id); + + return ret; +} + const struct pp_smumgr_func vega20_smu_funcs = { .smu_init = &vega20_smu_init, .smu_fini = &vega20_smu_fini, @@ -584,4 +583,6 @@ const struct pp_smumgr_func vega20_smu_funcs = { .download_pptable_settings = NULL, .upload_pptable_settings = NULL, .is_dpm_running = vega20_is_dpm_running, + .get_argument = vega20_get_argument, + .smc_table_manager = vega20_smc_table_manager, }; diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.h b/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.h index 505eb0d82e3b..77349c3f0162 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.h +++ b/drivers/gpu/drm/amd/powerplay/smumgr/vega20_smumgr.h @@ -47,11 +47,6 @@ struct vega20_smumgr { #define SMU_FEATURES_HIGH_MASK 0xFFFFFFFF00000000 #define SMU_FEATURES_HIGH_SHIFT 32 -int vega20_read_arg_from_smc(struct pp_hwmgr *hwmgr, uint32_t *arg); -int vega20_copy_table_from_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id); -int vega20_copy_table_to_smc(struct pp_hwmgr *hwmgr, - uint8_t *table, int16_t table_id); int vega20_enable_smc_features(struct pp_hwmgr *hwmgr, bool enable, uint64_t feature_mask); int vega20_get_enabled_smc_features(struct pp_hwmgr *hwmgr, diff --git a/drivers/gpu/drm/amd/powerplay/smumgr/vegam_smumgr.c b/drivers/gpu/drm/amd/powerplay/smumgr/vegam_smumgr.c index 3d415fabbd93..9f71512b2510 100644 --- a/drivers/gpu/drm/amd/powerplay/smumgr/vegam_smumgr.c +++ b/drivers/gpu/drm/amd/powerplay/smumgr/vegam_smumgr.c @@ -2185,6 +2185,7 @@ static uint32_t vegam_get_offsetof(uint32_t type, uint32_t member) case DRAM_LOG_BUFF_SIZE: return offsetof(SMU75_SoftRegisters, DRAM_LOG_BUFF_SIZE); } + break; case SMU_Discrete_DpmTable: switch (member) { case UvdBootLevel: @@ -2194,6 +2195,7 @@ static uint32_t vegam_get_offsetof(uint32_t type, uint32_t member) case LowSclkInterruptThreshold: return offsetof(SMU75_Discrete_DpmTable, LowSclkInterruptThreshold); } + break; } pr_warn("can't get the offset of type %x member %x\n", type, member); return 0; diff --git a/drivers/gpu/drm/arm/hdlcd_drv.c b/drivers/gpu/drm/arm/hdlcd_drv.c index 0ed1cde98cf8..dfad8d06d108 100644 --- a/drivers/gpu/drm/arm/hdlcd_drv.c +++ b/drivers/gpu/drm/arm/hdlcd_drv.c @@ -103,7 +103,6 @@ setup_fail: static const struct drm_mode_config_funcs hdlcd_mode_config_funcs = { .fb_create = drm_gem_fb_create, - .output_poll_changed = drm_fb_helper_output_poll_changed, .atomic_check = drm_atomic_helper_check, .atomic_commit = drm_atomic_helper_commit, }; @@ -233,7 +232,6 @@ static struct drm_driver hdlcd_driver = { .driver_features = DRIVER_HAVE_IRQ | DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME | DRIVER_ATOMIC, - .lastclose = drm_fb_helper_lastclose, .irq_handler = hdlcd_irq, .irq_preinstall = hdlcd_irq_preinstall, .irq_postinstall = hdlcd_irq_postinstall, @@ -308,19 +306,15 @@ static int hdlcd_drm_bind(struct device *dev) drm_mode_config_reset(drm); drm_kms_helper_poll_init(drm); - ret = drm_fb_cma_fbdev_init(drm, 32, 0); - if (ret) - goto err_fbdev; - ret = drm_dev_register(drm, 0); if (ret) goto err_register; + drm_fbdev_generic_setup(drm, 32); + return 0; err_register: - drm_fb_cma_fbdev_fini(drm); -err_fbdev: drm_kms_helper_poll_fini(drm); err_vblank: pm_runtime_disable(drm->dev); @@ -346,7 +340,6 @@ static void hdlcd_drm_unbind(struct device *dev) struct hdlcd_drm_private *hdlcd = drm->dev_private; drm_dev_unregister(drm); - drm_fb_cma_fbdev_fini(drm); drm_kms_helper_poll_fini(drm); component_unbind_all(dev, drm); of_node_put(hdlcd->crtc.port); diff --git a/drivers/gpu/drm/arm/malidp_crtc.c b/drivers/gpu/drm/arm/malidp_crtc.c index ef44202fb43f..e1b72782848c 100644 --- a/drivers/gpu/drm/arm/malidp_crtc.c +++ b/drivers/gpu/drm/arm/malidp_crtc.c @@ -348,19 +348,20 @@ static int malidp_crtc_atomic_check(struct drm_crtc *crtc, /* * check if there is enough rotation memory available for planes - * that need 90° and 270° rotation. Each plane has set its required - * memory size in the ->plane_check() callback, here we only make - * sure that the sums are less that the total usable memory. + * that need 90° and 270° rotion or planes that are compressed. + * Each plane has set its required memory size in the ->plane_check() + * callback, here we only make sure that the sums are less that the + * total usable memory. * * The rotation memory allocation algorithm (for each plane): - * a. If no more rotated planes exist, all remaining rotate - * memory in the bank is available for use by the plane. - * b. If other rotated planes exist, and plane's layer ID is - * DE_VIDEO1, it can use all the memory from first bank if - * secondary rotation memory bank is available, otherwise it can + * a. If no more rotated or compressed planes exist, all remaining + * rotate memory in the bank is available for use by the plane. + * b. If other rotated or compressed planes exist, and plane's + * layer ID is DE_VIDEO1, it can use all the memory from first bank + * if secondary rotation memory bank is available, otherwise it can * use up to half the bank's memory. - * c. If other rotated planes exist, and plane's layer ID is not - * DE_VIDEO1, it can use half of the available memory + * c. If other rotated or compressed planes exist, and plane's layer ID + * is not DE_VIDEO1, it can use half of the available memory. * * Note: this algorithm assumes that the order in which the planes are * checked always has DE_VIDEO1 plane first in the list if it is @@ -372,7 +373,9 @@ static int malidp_crtc_atomic_check(struct drm_crtc *crtc, /* first count the number of rotated planes */ drm_atomic_crtc_state_for_each_plane_state(plane, pstate, state) { - if (pstate->rotation & MALIDP_ROTATED_MASK) + struct drm_framebuffer *fb = pstate->fb; + + if ((pstate->rotation & MALIDP_ROTATED_MASK) || fb->modifier) rotated_planes++; } @@ -388,8 +391,9 @@ static int malidp_crtc_atomic_check(struct drm_crtc *crtc, drm_atomic_crtc_state_for_each_plane_state(plane, pstate, state) { struct malidp_plane *mp = to_malidp_plane(plane); struct malidp_plane_state *ms = to_malidp_plane_state(pstate); + struct drm_framebuffer *fb = pstate->fb; - if (pstate->rotation & MALIDP_ROTATED_MASK) { + if ((pstate->rotation & MALIDP_ROTATED_MASK) || fb->modifier) { /* process current plane */ rotated_planes--; diff --git a/drivers/gpu/drm/arm/malidp_drv.c b/drivers/gpu/drm/arm/malidp_drv.c index 08b5bb219816..505f316a192e 100644 --- a/drivers/gpu/drm/arm/malidp_drv.c +++ b/drivers/gpu/drm/arm/malidp_drv.c @@ -37,6 +37,8 @@ #include "malidp_hw.h" #define MALIDP_CONF_VALID_TIMEOUT 250 +#define AFBC_HEADER_SIZE 16 +#define AFBC_SUPERBLK_ALIGNMENT 128 static void malidp_write_gamma_table(struct malidp_hw_device *hwdev, u32 data[MALIDP_COEFFTAB_NUM_COEFFS]) @@ -258,9 +260,134 @@ static const struct drm_mode_config_helper_funcs malidp_mode_config_helpers = { .atomic_commit_tail = malidp_atomic_commit_tail, }; +static bool +malidp_verify_afbc_framebuffer_caps(struct drm_device *dev, + const struct drm_mode_fb_cmd2 *mode_cmd) +{ + const struct drm_format_info *info; + + if ((mode_cmd->modifier[0] >> 56) != DRM_FORMAT_MOD_VENDOR_ARM) { + DRM_DEBUG_KMS("Unknown modifier (not Arm)\n"); + return false; + } + + if (mode_cmd->modifier[0] & + ~DRM_FORMAT_MOD_ARM_AFBC(AFBC_MOD_VALID_BITS)) { + DRM_DEBUG_KMS("Unsupported modifiers\n"); + return false; + } + + info = drm_get_format_info(dev, mode_cmd); + if (!info) { + DRM_DEBUG_KMS("Unable to get the format information\n"); + return false; + } + + if (info->num_planes != 1) { + DRM_DEBUG_KMS("AFBC buffers expect one plane\n"); + return false; + } + + if (mode_cmd->offsets[0] != 0) { + DRM_DEBUG_KMS("AFBC buffers' plane offset should be 0\n"); + return false; + } + + switch (mode_cmd->modifier[0] & AFBC_FORMAT_MOD_BLOCK_SIZE_MASK) { + case AFBC_FORMAT_MOD_BLOCK_SIZE_16x16: + if ((mode_cmd->width % 16) || (mode_cmd->height % 16)) { + DRM_DEBUG_KMS("AFBC buffers must be aligned to 16 pixels\n"); + return false; + } + break; + default: + DRM_DEBUG_KMS("Unsupported AFBC block size\n"); + return false; + } + + return true; +} + +static bool +malidp_verify_afbc_framebuffer_size(struct drm_device *dev, + struct drm_file *file, + const struct drm_mode_fb_cmd2 *mode_cmd) +{ + int n_superblocks = 0; + const struct drm_format_info *info; + struct drm_gem_object *objs = NULL; + u32 afbc_superblock_size = 0, afbc_superblock_height = 0; + u32 afbc_superblock_width = 0, afbc_size = 0; + + switch (mode_cmd->modifier[0] & AFBC_FORMAT_MOD_BLOCK_SIZE_MASK) { + case AFBC_FORMAT_MOD_BLOCK_SIZE_16x16: + afbc_superblock_height = 16; + afbc_superblock_width = 16; + break; + default: + DRM_DEBUG_KMS("AFBC superblock size is not supported\n"); + return false; + } + + info = drm_get_format_info(dev, mode_cmd); + + n_superblocks = (mode_cmd->width / afbc_superblock_width) * + (mode_cmd->height / afbc_superblock_height); + + afbc_superblock_size = info->cpp[0] * afbc_superblock_width * + afbc_superblock_height; + + afbc_size = ALIGN(n_superblocks * AFBC_HEADER_SIZE, AFBC_SUPERBLK_ALIGNMENT); + afbc_size += n_superblocks * ALIGN(afbc_superblock_size, AFBC_SUPERBLK_ALIGNMENT); + + if (mode_cmd->width * info->cpp[0] != mode_cmd->pitches[0]) { + DRM_DEBUG_KMS("Invalid value of pitch (=%u) should be same as width (=%u) * cpp (=%u)\n", + mode_cmd->pitches[0], mode_cmd->width, info->cpp[0]); + return false; + } + + objs = drm_gem_object_lookup(file, mode_cmd->handles[0]); + if (!objs) { + DRM_DEBUG_KMS("Failed to lookup GEM object\n"); + return false; + } + + if (objs->size < afbc_size) { + DRM_DEBUG_KMS("buffer size (%zu) too small for AFBC buffer size = %u\n", + objs->size, afbc_size); + drm_gem_object_put_unlocked(objs); + return false; + } + + drm_gem_object_put_unlocked(objs); + + return true; +} + +static bool +malidp_verify_afbc_framebuffer(struct drm_device *dev, struct drm_file *file, + const struct drm_mode_fb_cmd2 *mode_cmd) +{ + if (malidp_verify_afbc_framebuffer_caps(dev, mode_cmd)) + return malidp_verify_afbc_framebuffer_size(dev, file, mode_cmd); + + return false; +} + +struct drm_framebuffer * +malidp_fb_create(struct drm_device *dev, struct drm_file *file, + const struct drm_mode_fb_cmd2 *mode_cmd) +{ + if (mode_cmd->modifier[0]) { + if (!malidp_verify_afbc_framebuffer(dev, file, mode_cmd)) + return ERR_PTR(-EINVAL); + } + + return drm_gem_fb_create(dev, file, mode_cmd); +} + static const struct drm_mode_config_funcs malidp_mode_config_funcs = { - .fb_create = drm_gem_fb_create, - .output_poll_changed = drm_fb_helper_output_poll_changed, + .fb_create = malidp_fb_create, .atomic_check = drm_atomic_helper_check, .atomic_commit = drm_atomic_helper_commit, }; @@ -450,7 +577,6 @@ static int malidp_debugfs_init(struct drm_minor *minor) static struct drm_driver malidp_driver = { .driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_ATOMIC | DRIVER_PRIME, - .lastclose = drm_fb_helper_lastclose, .gem_free_object_unlocked = drm_gem_cma_free_object, .gem_vm_ops = &drm_gem_cma_vm_ops, .dumb_create = malidp_dumb_create, @@ -754,6 +880,7 @@ static int malidp_bind(struct device *dev) drm->irq_enabled = true; ret = drm_vblank_init(drm, drm->mode_config.num_crtc); + drm_crtc_vblank_reset(&malidp->crtc); if (ret < 0) { DRM_ERROR("failed to initialise vblank\n"); goto vblank_fail; @@ -762,22 +889,18 @@ static int malidp_bind(struct device *dev) drm_mode_config_reset(drm); - ret = drm_fb_cma_fbdev_init(drm, 32, 0); - if (ret) - goto fbdev_fail; - drm_kms_helper_poll_init(drm); ret = drm_dev_register(drm, 0); if (ret) goto register_fail; + drm_fbdev_generic_setup(drm, 32); + return 0; register_fail: - drm_fb_cma_fbdev_fini(drm); drm_kms_helper_poll_fini(drm); -fbdev_fail: pm_runtime_get_sync(dev); vblank_fail: malidp_se_irq_fini(hwdev); @@ -814,7 +937,6 @@ static void malidp_unbind(struct device *dev) struct malidp_hw_device *hwdev = malidp->dev; drm_dev_unregister(drm); - drm_fb_cma_fbdev_fini(drm); drm_kms_helper_poll_fini(drm); pm_runtime_get_sync(dev); drm_crtc_vblank_off(&malidp->crtc); diff --git a/drivers/gpu/drm/arm/malidp_drv.h b/drivers/gpu/drm/arm/malidp_drv.h index e3eb0cb1f385..b76c86f18a56 100644 --- a/drivers/gpu/drm/arm/malidp_drv.h +++ b/drivers/gpu/drm/arm/malidp_drv.h @@ -55,6 +55,12 @@ struct malidp_plane { const struct malidp_layer *layer; }; +enum mmu_prefetch_mode { + MALIDP_PREFETCH_MODE_NONE, + MALIDP_PREFETCH_MODE_PARTIAL, + MALIDP_PREFETCH_MODE_FULL, +}; + struct malidp_plane_state { struct drm_plane_state base; @@ -63,6 +69,8 @@ struct malidp_plane_state { /* internal format ID */ u8 format; u8 n_planes; + enum mmu_prefetch_mode mmu_prefetch_mode; + u32 mmu_prefetch_pgsize; }; #define to_malidp_plane(x) container_of(x, struct malidp_plane, base) diff --git a/drivers/gpu/drm/arm/malidp_hw.c b/drivers/gpu/drm/arm/malidp_hw.c index c94a4422e0e9..7aad7dd80d8c 100644 --- a/drivers/gpu/drm/arm/malidp_hw.c +++ b/drivers/gpu/drm/arm/malidp_hw.c @@ -84,16 +84,48 @@ static const struct malidp_format_id malidp550_de_formats[] = { }; static const struct malidp_layer malidp500_layers[] = { - { DE_VIDEO1, MALIDP500_DE_LV_BASE, MALIDP500_DE_LV_PTR_BASE, MALIDP_DE_LV_STRIDE0, MALIDP500_LV_YUV2RGB }, - { DE_GRAPHICS1, MALIDP500_DE_LG1_BASE, MALIDP500_DE_LG1_PTR_BASE, MALIDP_DE_LG_STRIDE, 0 }, - { DE_GRAPHICS2, MALIDP500_DE_LG2_BASE, MALIDP500_DE_LG2_PTR_BASE, MALIDP_DE_LG_STRIDE, 0 }, + /* id, base address, fb pointer address base, stride offset, + * yuv2rgb matrix offset, mmu control register offset, rotation_features + */ + { DE_VIDEO1, MALIDP500_DE_LV_BASE, MALIDP500_DE_LV_PTR_BASE, + MALIDP_DE_LV_STRIDE0, MALIDP500_LV_YUV2RGB, 0, ROTATE_ANY }, + { DE_GRAPHICS1, MALIDP500_DE_LG1_BASE, MALIDP500_DE_LG1_PTR_BASE, + MALIDP_DE_LG_STRIDE, 0, 0, ROTATE_ANY }, + { DE_GRAPHICS2, MALIDP500_DE_LG2_BASE, MALIDP500_DE_LG2_PTR_BASE, + MALIDP_DE_LG_STRIDE, 0, 0, ROTATE_ANY }, }; static const struct malidp_layer malidp550_layers[] = { - { DE_VIDEO1, MALIDP550_DE_LV1_BASE, MALIDP550_DE_LV1_PTR_BASE, MALIDP_DE_LV_STRIDE0, MALIDP550_LV_YUV2RGB }, - { DE_GRAPHICS1, MALIDP550_DE_LG_BASE, MALIDP550_DE_LG_PTR_BASE, MALIDP_DE_LG_STRIDE, 0 }, - { DE_VIDEO2, MALIDP550_DE_LV2_BASE, MALIDP550_DE_LV2_PTR_BASE, MALIDP_DE_LV_STRIDE0, MALIDP550_LV_YUV2RGB }, - { DE_SMART, MALIDP550_DE_LS_BASE, MALIDP550_DE_LS_PTR_BASE, MALIDP550_DE_LS_R1_STRIDE, 0 }, + /* id, base address, fb pointer address base, stride offset, + * yuv2rgb matrix offset, mmu control register offset, rotation_features + */ + { DE_VIDEO1, MALIDP550_DE_LV1_BASE, MALIDP550_DE_LV1_PTR_BASE, + MALIDP_DE_LV_STRIDE0, MALIDP550_LV_YUV2RGB, 0, ROTATE_ANY }, + { DE_GRAPHICS1, MALIDP550_DE_LG_BASE, MALIDP550_DE_LG_PTR_BASE, + MALIDP_DE_LG_STRIDE, 0, 0, ROTATE_ANY }, + { DE_VIDEO2, MALIDP550_DE_LV2_BASE, MALIDP550_DE_LV2_PTR_BASE, + MALIDP_DE_LV_STRIDE0, MALIDP550_LV_YUV2RGB, 0, ROTATE_ANY }, + { DE_SMART, MALIDP550_DE_LS_BASE, MALIDP550_DE_LS_PTR_BASE, + MALIDP550_DE_LS_R1_STRIDE, 0, 0, ROTATE_NONE }, +}; + +static const struct malidp_layer malidp650_layers[] = { + /* id, base address, fb pointer address base, stride offset, + * yuv2rgb matrix offset, mmu control register offset, + * rotation_features + */ + { DE_VIDEO1, MALIDP550_DE_LV1_BASE, MALIDP550_DE_LV1_PTR_BASE, + MALIDP_DE_LV_STRIDE0, MALIDP550_LV_YUV2RGB, + MALIDP650_DE_LV_MMU_CTRL, ROTATE_ANY }, + { DE_GRAPHICS1, MALIDP550_DE_LG_BASE, MALIDP550_DE_LG_PTR_BASE, + MALIDP_DE_LG_STRIDE, 0, MALIDP650_DE_LG_MMU_CTRL, + ROTATE_COMPRESSED }, + { DE_VIDEO2, MALIDP550_DE_LV2_BASE, MALIDP550_DE_LV2_PTR_BASE, + MALIDP_DE_LV_STRIDE0, MALIDP550_LV_YUV2RGB, + MALIDP650_DE_LV_MMU_CTRL, ROTATE_ANY }, + { DE_SMART, MALIDP550_DE_LS_BASE, MALIDP550_DE_LS_PTR_BASE, + MALIDP550_DE_LS_R1_STRIDE, 0, MALIDP650_DE_LS_MMU_CTRL, + ROTATE_NONE }, }; #define SE_N_SCALING_COEFFS 96 @@ -288,10 +320,6 @@ static void malidp500_modeset(struct malidp_hw_device *hwdev, struct videomode * static int malidp500_rotmem_required(struct malidp_hw_device *hwdev, u16 w, u16 h, u32 fmt) { - /* RGB888 or BGR888 can't be rotated */ - if ((fmt == DRM_FORMAT_RGB888) || (fmt == DRM_FORMAT_BGR888)) - return -EINVAL; - /* * Each layer needs enough rotation memory to fit 8 lines * worth of pixel data. Required size is then: @@ -384,7 +412,8 @@ static long malidp500_se_calc_mclk(struct malidp_hw_device *hwdev, static int malidp500_enable_memwrite(struct malidp_hw_device *hwdev, dma_addr_t *addrs, s32 *pitches, - int num_planes, u16 w, u16 h, u32 fmt_id) + int num_planes, u16 w, u16 h, u32 fmt_id, + const s16 *rgb2yuv_coeffs) { u32 base = MALIDP500_SE_MEMWRITE_BASE; u32 de_base = malidp_get_block_base(hwdev, MALIDP_DE_BLOCK); @@ -416,6 +445,16 @@ static int malidp500_enable_memwrite(struct malidp_hw_device *hwdev, malidp_hw_write(hwdev, MALIDP_DE_H_ACTIVE(w) | MALIDP_DE_V_ACTIVE(h), MALIDP500_SE_MEMWRITE_OUT_SIZE); + + if (rgb2yuv_coeffs) { + int i; + + for (i = 0; i < MALIDP_COLORADJ_NUM_COEFFS; i++) { + malidp_hw_write(hwdev, rgb2yuv_coeffs[i], + MALIDP500_SE_RGB_YUV_COEFFS + i * 4); + } + } + malidp_hw_setbits(hwdev, MALIDP_SE_MEMWRITE_EN, MALIDP500_SE_CONTROL); return 0; @@ -568,10 +607,6 @@ static int malidp550_rotmem_required(struct malidp_hw_device *hwdev, u16 w, u16 { u32 bytes_per_col; - /* raw RGB888 or BGR888 can't be rotated */ - if ((fmt == DRM_FORMAT_RGB888) || (fmt == DRM_FORMAT_BGR888)) - return -EINVAL; - switch (fmt) { /* 8 lines at 4 bytes per pixel */ case DRM_FORMAT_ARGB2101010: @@ -658,7 +693,8 @@ static long malidp550_se_calc_mclk(struct malidp_hw_device *hwdev, static int malidp550_enable_memwrite(struct malidp_hw_device *hwdev, dma_addr_t *addrs, s32 *pitches, - int num_planes, u16 w, u16 h, u32 fmt_id) + int num_planes, u16 w, u16 h, u32 fmt_id, + const s16 *rgb2yuv_coeffs) { u32 base = MALIDP550_SE_MEMWRITE_BASE; u32 de_base = malidp_get_block_base(hwdev, MALIDP_DE_BLOCK); @@ -689,6 +725,15 @@ static int malidp550_enable_memwrite(struct malidp_hw_device *hwdev, malidp_hw_setbits(hwdev, MALIDP550_SE_MEMWRITE_ONESHOT | MALIDP_SE_MEMWRITE_EN, MALIDP550_SE_CONTROL); + if (rgb2yuv_coeffs) { + int i; + + for (i = 0; i < MALIDP_COLORADJ_NUM_COEFFS; i++) { + malidp_hw_write(hwdev, rgb2yuv_coeffs[i], + MALIDP550_SE_RGB_YUV_COEFFS + i * 4); + } + } + return 0; } @@ -832,8 +877,8 @@ const struct malidp_hw malidp_device[MALIDP_MAX_DEVICES] = { .dc_base = MALIDP550_DC_BASE, .out_depth_base = MALIDP550_DE_OUTPUT_DEPTH, .features = MALIDP_REGMAP_HAS_CLEARIRQ, - .n_layers = ARRAY_SIZE(malidp550_layers), - .layers = malidp550_layers, + .n_layers = ARRAY_SIZE(malidp650_layers), + .layers = malidp650_layers, .de_irq_map = { .irq_mask = MALIDP_DE_IRQ_UNDERRUN | MALIDP650_DE_IRQ_DRIFT | diff --git a/drivers/gpu/drm/arm/malidp_hw.h b/drivers/gpu/drm/arm/malidp_hw.h index ad2e96915d44..40155e2ea9d9 100644 --- a/drivers/gpu/drm/arm/malidp_hw.h +++ b/drivers/gpu/drm/arm/malidp_hw.h @@ -36,6 +36,12 @@ enum { SE_MEMWRITE = BIT(5), }; +enum rotation_features { + ROTATE_NONE, /* does not support rotation at all */ + ROTATE_ANY, /* supports rotation on any buffers */ + ROTATE_COMPRESSED, /* supports rotation only on compressed buffers */ +}; + struct malidp_format_id { u32 format; /* DRM fourcc */ u8 layer; /* bitmask of layers supporting it */ @@ -62,6 +68,8 @@ struct malidp_layer { u16 ptr; /* address offset for the pointer register */ u16 stride_offset; /* offset to the first stride register. */ s16 yuv2rgb_offset; /* offset to the YUV->RGB matrix entries */ + u16 mmu_ctrl_offset; /* offset to the MMU control register */ + enum rotation_features rot; /* type of rotation supported */ }; enum malidp_scaling_coeff_set { @@ -191,7 +199,8 @@ struct malidp_hw { * @param fmt_id - internal format ID of output buffer */ int (*enable_memwrite)(struct malidp_hw_device *hwdev, dma_addr_t *addrs, - s32 *pitches, int num_planes, u16 w, u16 h, u32 fmt_id); + s32 *pitches, int num_planes, u16 w, u16 h, u32 fmt_id, + const s16 *rgb2yuv_coeffs); /* * Disable the writing to memory of the next frame's content. @@ -379,4 +388,9 @@ static inline void malidp_se_set_enh_coeffs(struct malidp_hw_device *hwdev) #define MALIDP_GAMMA_LUT_SIZE 4096 +#define AFBC_MOD_VALID_BITS (AFBC_FORMAT_MOD_BLOCK_SIZE_MASK | \ + AFBC_FORMAT_MOD_YTR | AFBC_FORMAT_MOD_SPLIT | \ + AFBC_FORMAT_MOD_SPARSE | AFBC_FORMAT_MOD_CBR | \ + AFBC_FORMAT_MOD_TILED | AFBC_FORMAT_MOD_SC) + #endif /* __MALIDP_HW_H__ */ diff --git a/drivers/gpu/drm/arm/malidp_mw.c b/drivers/gpu/drm/arm/malidp_mw.c index ba6ae66387c9..91472e5e0c8b 100644 --- a/drivers/gpu/drm/arm/malidp_mw.c +++ b/drivers/gpu/drm/arm/malidp_mw.c @@ -26,6 +26,8 @@ struct malidp_mw_connector_state { s32 pitches[2]; u8 format; u8 n_planes; + bool rgb2yuv_initialized; + const s16 *rgb2yuv_coeffs; }; static int malidp_mw_connector_get_modes(struct drm_connector *connector) @@ -84,7 +86,7 @@ static void malidp_mw_connector_destroy(struct drm_connector *connector) static struct drm_connector_state * malidp_mw_connector_duplicate_state(struct drm_connector *connector) { - struct malidp_mw_connector_state *mw_state; + struct malidp_mw_connector_state *mw_state, *mw_current_state; if (WARN_ON(!connector->state)) return NULL; @@ -93,7 +95,10 @@ malidp_mw_connector_duplicate_state(struct drm_connector *connector) if (!mw_state) return NULL; - /* No need to preserve any of our driver-local data */ + mw_current_state = to_mw_state(connector->state); + mw_state->rgb2yuv_coeffs = mw_current_state->rgb2yuv_coeffs; + mw_state->rgb2yuv_initialized = mw_current_state->rgb2yuv_initialized; + __drm_atomic_helper_connector_duplicate_state(connector, &mw_state->base); return &mw_state->base; @@ -108,6 +113,13 @@ static const struct drm_connector_funcs malidp_mw_connector_funcs = { .atomic_destroy_state = drm_atomic_helper_connector_destroy_state, }; +static const s16 rgb2yuv_coeffs_bt709_limited[MALIDP_COLORADJ_NUM_COEFFS] = { + 47, 157, 16, + -26, -87, 112, + 112, -102, -10, + 16, 128, 128 +}; + static int malidp_mw_encoder_atomic_check(struct drm_encoder *encoder, struct drm_crtc_state *crtc_state, @@ -157,6 +169,9 @@ malidp_mw_encoder_atomic_check(struct drm_encoder *encoder, } mw_state->n_planes = n_planes; + if (fb->format->is_yuv) + mw_state->rgb2yuv_coeffs = rgb2yuv_coeffs_bt709_limited; + return 0; } @@ -239,10 +254,12 @@ void malidp_mw_atomic_commit(struct drm_device *drm, drm_writeback_queue_job(mw_conn, conn_state->writeback_job); conn_state->writeback_job = NULL; - hwdev->hw->enable_memwrite(hwdev, mw_state->addrs, mw_state->pitches, mw_state->n_planes, - fb->width, fb->height, mw_state->format); + fb->width, fb->height, mw_state->format, + !mw_state->rgb2yuv_initialized ? + mw_state->rgb2yuv_coeffs : NULL); + mw_state->rgb2yuv_initialized = !!mw_state->rgb2yuv_coeffs; } else { DRM_DEV_DEBUG_DRIVER(drm->dev, "Disable memwrite\n"); hwdev->hw->disable_memwrite(hwdev); diff --git a/drivers/gpu/drm/arm/malidp_planes.c b/drivers/gpu/drm/arm/malidp_planes.c index 49c37f6dd63e..837a24d56675 100644 --- a/drivers/gpu/drm/arm/malidp_planes.c +++ b/drivers/gpu/drm/arm/malidp_planes.c @@ -10,11 +10,14 @@ * ARM Mali DP plane manipulation routines. */ +#include <linux/iommu.h> + #include <drm/drmP.h> #include <drm/drm_atomic.h> #include <drm/drm_atomic_helper.h> #include <drm/drm_fb_cma_helper.h> #include <drm/drm_gem_cma_helper.h> +#include <drm/drm_gem_framebuffer_helper.h> #include <drm/drm_plane_helper.h> #include <drm/drm_print.h> @@ -36,6 +39,7 @@ #define LAYER_COMP_MASK (0x3 << 12) #define LAYER_COMP_PIXEL (0x3 << 12) #define LAYER_COMP_PLANE (0x2 << 12) +#define LAYER_PMUL_ENABLE (0x1 << 14) #define LAYER_ALPHA_OFFSET (16) #define LAYER_ALPHA_MASK (0xff) #define LAYER_ALPHA(x) (((x) & LAYER_ALPHA_MASK) << LAYER_ALPHA_OFFSET) @@ -56,6 +60,13 @@ */ #define MALIDP_ALPHA_LUT 0xffaa5500 +/* page sizes the MMU prefetcher can support */ +#define MALIDP_MMU_PREFETCH_PARTIAL_PGSIZES (SZ_4K | SZ_64K) +#define MALIDP_MMU_PREFETCH_FULL_PGSIZES (SZ_1M | SZ_2M) + +/* readahead for partial-frame prefetch */ +#define MALIDP_MMU_PREFETCH_READAHEAD 8 + static void malidp_de_plane_destroy(struct drm_plane *plane) { struct malidp_plane *mp = to_malidp_plane(plane); @@ -100,6 +111,9 @@ drm_plane_state *malidp_duplicate_plane_state(struct drm_plane *plane) state->format = m_state->format; state->n_planes = m_state->n_planes; + state->mmu_prefetch_mode = m_state->mmu_prefetch_mode; + state->mmu_prefetch_pgsize = m_state->mmu_prefetch_pgsize; + return &state->base; } @@ -112,6 +126,12 @@ static void malidp_destroy_plane_state(struct drm_plane *plane, kfree(m_state); } +static const char * const prefetch_mode_names[] = { + [MALIDP_PREFETCH_MODE_NONE] = "MMU_PREFETCH_NONE", + [MALIDP_PREFETCH_MODE_PARTIAL] = "MMU_PREFETCH_PARTIAL", + [MALIDP_PREFETCH_MODE_FULL] = "MMU_PREFETCH_FULL", +}; + static void malidp_plane_atomic_print_state(struct drm_printer *p, const struct drm_plane_state *state) { @@ -120,6 +140,9 @@ static void malidp_plane_atomic_print_state(struct drm_printer *p, drm_printf(p, "\trotmem_size=%u\n", ms->rotmem_size); drm_printf(p, "\tformat_id=%u\n", ms->format); drm_printf(p, "\tn_planes=%u\n", ms->n_planes); + drm_printf(p, "\tmmu_prefetch_mode=%s\n", + prefetch_mode_names[ms->mmu_prefetch_mode]); + drm_printf(p, "\tmmu_prefetch_pgsize=%d\n", ms->mmu_prefetch_pgsize); } static const struct drm_plane_funcs malidp_de_plane_funcs = { @@ -173,6 +196,199 @@ static int malidp_se_check_scaling(struct malidp_plane *mp, return 0; } +static u32 malidp_get_pgsize_bitmap(struct malidp_plane *mp) +{ + u32 pgsize_bitmap = 0; + + if (iommu_present(&platform_bus_type)) { + struct iommu_domain *mmu_dom = + iommu_get_domain_for_dev(mp->base.dev->dev); + + if (mmu_dom) + pgsize_bitmap = mmu_dom->pgsize_bitmap; + } + + return pgsize_bitmap; +} + +/* + * Check if the framebuffer is entirely made up of pages at least pgsize in + * size. Only a heuristic: assumes that each scatterlist entry has been aligned + * to the largest page size smaller than its length and that the MMU maps to + * the largest page size possible. + */ +static bool malidp_check_pages_threshold(struct malidp_plane_state *ms, + u32 pgsize) +{ + int i; + + for (i = 0; i < ms->n_planes; i++) { + struct drm_gem_object *obj; + struct drm_gem_cma_object *cma_obj; + struct sg_table *sgt; + struct scatterlist *sgl; + + obj = drm_gem_fb_get_obj(ms->base.fb, i); + cma_obj = to_drm_gem_cma_obj(obj); + + if (cma_obj->sgt) + sgt = cma_obj->sgt; + else + sgt = obj->dev->driver->gem_prime_get_sg_table(obj); + + if (!sgt) + return false; + + sgl = sgt->sgl; + + while (sgl) { + if (sgl->length < pgsize) { + if (!cma_obj->sgt) + kfree(sgt); + return false; + } + + sgl = sg_next(sgl); + } + if (!cma_obj->sgt) + kfree(sgt); + } + + return true; +} + +/* + * Check if it is possible to enable partial-frame MMU prefetch given the + * current format, AFBC state and rotation. + */ +static bool malidp_partial_prefetch_supported(u32 format, u64 modifier, + unsigned int rotation) +{ + bool afbc, sparse; + + /* rotation and horizontal flip not supported for partial prefetch */ + if (rotation & (DRM_MODE_ROTATE_90 | DRM_MODE_ROTATE_180 | + DRM_MODE_ROTATE_270 | DRM_MODE_REFLECT_X)) + return false; + + afbc = modifier & DRM_FORMAT_MOD_ARM_AFBC(0); + sparse = modifier & AFBC_FORMAT_MOD_SPARSE; + + switch (format) { + case DRM_FORMAT_ARGB2101010: + case DRM_FORMAT_RGBA1010102: + case DRM_FORMAT_BGRA1010102: + case DRM_FORMAT_ARGB8888: + case DRM_FORMAT_RGBA8888: + case DRM_FORMAT_BGRA8888: + case DRM_FORMAT_XRGB8888: + case DRM_FORMAT_XBGR8888: + case DRM_FORMAT_RGBX8888: + case DRM_FORMAT_BGRX8888: + case DRM_FORMAT_RGB888: + case DRM_FORMAT_RGBA5551: + case DRM_FORMAT_RGB565: + /* always supported */ + return true; + + case DRM_FORMAT_ABGR2101010: + case DRM_FORMAT_ABGR8888: + case DRM_FORMAT_ABGR1555: + case DRM_FORMAT_BGR565: + /* supported, but if AFBC then must be sparse mode */ + return (!afbc) || (afbc && sparse); + + case DRM_FORMAT_BGR888: + /* supported, but not for AFBC */ + return !afbc; + + case DRM_FORMAT_YUYV: + case DRM_FORMAT_UYVY: + case DRM_FORMAT_NV12: + case DRM_FORMAT_YUV420: + /* not supported */ + return false; + + default: + return false; + } +} + +/* + * Select the preferred MMU prefetch mode. Full-frame prefetch is preferred as + * long as the framebuffer is all large pages. Otherwise partial-frame prefetch + * is selected as long as it is supported for the current format. The selected + * page size for prefetch is returned in pgsize_bitmap. + */ +static enum mmu_prefetch_mode malidp_mmu_prefetch_select_mode + (struct malidp_plane_state *ms, u32 *pgsize_bitmap) +{ + u32 pgsizes; + + /* get the full-frame prefetch page size(s) supported by the MMU */ + pgsizes = *pgsize_bitmap & MALIDP_MMU_PREFETCH_FULL_PGSIZES; + + while (pgsizes) { + u32 largest_pgsize = 1 << __fls(pgsizes); + + if (malidp_check_pages_threshold(ms, largest_pgsize)) { + *pgsize_bitmap = largest_pgsize; + return MALIDP_PREFETCH_MODE_FULL; + } + + pgsizes -= largest_pgsize; + } + + /* get the partial-frame prefetch page size(s) supported by the MMU */ + pgsizes = *pgsize_bitmap & MALIDP_MMU_PREFETCH_PARTIAL_PGSIZES; + + if (malidp_partial_prefetch_supported(ms->base.fb->format->format, + ms->base.fb->modifier, + ms->base.rotation)) { + /* partial prefetch using the smallest page size */ + *pgsize_bitmap = 1 << __ffs(pgsizes); + return MALIDP_PREFETCH_MODE_PARTIAL; + } + *pgsize_bitmap = 0; + return MALIDP_PREFETCH_MODE_NONE; +} + +static u32 malidp_calc_mmu_control_value(enum mmu_prefetch_mode mode, + u8 readahead, u8 n_planes, u32 pgsize) +{ + u32 mmu_ctrl = 0; + + if (mode != MALIDP_PREFETCH_MODE_NONE) { + mmu_ctrl |= MALIDP_MMU_CTRL_EN; + + if (mode == MALIDP_PREFETCH_MODE_PARTIAL) { + mmu_ctrl |= MALIDP_MMU_CTRL_MODE; + mmu_ctrl |= MALIDP_MMU_CTRL_PP_NUM_REQ(readahead); + } + + if (pgsize == SZ_64K || pgsize == SZ_2M) { + int i; + + for (i = 0; i < n_planes; i++) + mmu_ctrl |= MALIDP_MMU_CTRL_PX_PS(i); + } + } + + return mmu_ctrl; +} + +static void malidp_de_prefetch_settings(struct malidp_plane *mp, + struct malidp_plane_state *ms) +{ + if (!mp->layer->mmu_ctrl_offset) + return; + + /* get the page sizes supported by the MMU */ + ms->mmu_prefetch_pgsize = malidp_get_pgsize_bitmap(mp); + ms->mmu_prefetch_mode = + malidp_mmu_prefetch_select_mode(ms, &ms->mmu_prefetch_pgsize); +} + static int malidp_de_plane_check(struct drm_plane *plane, struct drm_plane_state *state) { @@ -180,6 +396,7 @@ static int malidp_de_plane_check(struct drm_plane *plane, struct malidp_plane_state *ms = to_malidp_plane_state(state); bool rotated = state->rotation & MALIDP_ROTATED_MASK; struct drm_framebuffer *fb; + u16 pixel_alpha = state->pixel_blend_mode; int i, ret; if (!state->crtc || !state->fb) @@ -223,11 +440,20 @@ static int malidp_de_plane_check(struct drm_plane *plane, if (ret) return ret; - /* packed RGB888 / BGR888 can't be rotated or flipped */ - if (state->rotation != DRM_MODE_ROTATE_0 && - (fb->format->format == DRM_FORMAT_RGB888 || - fb->format->format == DRM_FORMAT_BGR888)) - return -EINVAL; + /* validate the rotation constraints for each layer */ + if (state->rotation != DRM_MODE_ROTATE_0) { + if (mp->layer->rot == ROTATE_NONE) + return -EINVAL; + if ((mp->layer->rot == ROTATE_COMPRESSED) && !(fb->modifier)) + return -EINVAL; + /* + * packed RGB888 / BGR888 can't be rotated or flipped + * unless they are stored in a compressed way + */ + if ((fb->format->format == DRM_FORMAT_RGB888 || + fb->format->format == DRM_FORMAT_BGR888) && !(fb->modifier)) + return -EINVAL; + } ms->rotmem_size = 0; if (state->rotation & MALIDP_ROTATED_MASK) { @@ -242,6 +468,14 @@ static int malidp_de_plane_check(struct drm_plane *plane, ms->rotmem_size = val; } + /* HW can't support plane + pixel blending */ + if ((state->alpha != DRM_BLEND_ALPHA_OPAQUE) && + (pixel_alpha != DRM_MODE_BLEND_PIXEL_NONE) && + fb->format->has_alpha) + return -EINVAL; + + malidp_de_prefetch_settings(mp, ms); + return 0; } @@ -318,22 +552,42 @@ static void malidp_de_set_color_encoding(struct malidp_plane *plane, } } +static void malidp_de_set_mmu_control(struct malidp_plane *mp, + struct malidp_plane_state *ms) +{ + u32 mmu_ctrl; + + /* check hardware supports MMU prefetch */ + if (!mp->layer->mmu_ctrl_offset) + return; + + mmu_ctrl = malidp_calc_mmu_control_value(ms->mmu_prefetch_mode, + MALIDP_MMU_PREFETCH_READAHEAD, + ms->n_planes, + ms->mmu_prefetch_pgsize); + + malidp_hw_write(mp->hwdev, mmu_ctrl, + mp->layer->base + mp->layer->mmu_ctrl_offset); +} + static void malidp_de_plane_update(struct drm_plane *plane, struct drm_plane_state *old_state) { struct malidp_plane *mp; struct malidp_plane_state *ms = to_malidp_plane_state(plane->state); + struct drm_plane_state *state = plane->state; + u16 pixel_alpha = state->pixel_blend_mode; + u8 plane_alpha = state->alpha >> 8; u32 src_w, src_h, dest_w, dest_h, val; int i; - bool format_has_alpha = plane->state->fb->format->has_alpha; mp = to_malidp_plane(plane); /* convert src values from Q16 fixed point to integer */ - src_w = plane->state->src_w >> 16; - src_h = plane->state->src_h >> 16; - dest_w = plane->state->crtc_w; - dest_h = plane->state->crtc_h; + src_w = state->src_w >> 16; + src_h = state->src_h >> 16; + dest_w = state->crtc_w; + dest_h = state->crtc_h; val = malidp_hw_read(mp->hwdev, mp->layer->base); val = (val & ~LAYER_FORMAT_MASK) | ms->format; @@ -342,14 +596,17 @@ static void malidp_de_plane_update(struct drm_plane *plane, for (i = 0; i < ms->n_planes; i++) { /* calculate the offset for the layer's plane registers */ u16 ptr = mp->layer->ptr + (i << 4); - dma_addr_t fb_addr = drm_fb_cma_get_gem_addr(plane->state->fb, - plane->state, i); + dma_addr_t fb_addr = drm_fb_cma_get_gem_addr(state->fb, + state, i); malidp_hw_write(mp->hwdev, lower_32_bits(fb_addr), ptr); malidp_hw_write(mp->hwdev, upper_32_bits(fb_addr), ptr + 4); } + + malidp_de_set_mmu_control(mp, ms); + malidp_de_set_plane_pitches(mp, ms->n_planes, - plane->state->fb->pitches); + state->fb->pitches); if ((plane->state->color_encoding != old_state->color_encoding) || (plane->state->color_range != old_state->color_range)) @@ -362,52 +619,56 @@ static void malidp_de_plane_update(struct drm_plane *plane, malidp_hw_write(mp->hwdev, LAYER_H_VAL(dest_w) | LAYER_V_VAL(dest_h), mp->layer->base + MALIDP_LAYER_COMP_SIZE); - malidp_hw_write(mp->hwdev, LAYER_H_VAL(plane->state->crtc_x) | - LAYER_V_VAL(plane->state->crtc_y), + malidp_hw_write(mp->hwdev, LAYER_H_VAL(state->crtc_x) | + LAYER_V_VAL(state->crtc_y), mp->layer->base + MALIDP_LAYER_OFFSET); - if (mp->layer->id == DE_SMART) + if (mp->layer->id == DE_SMART) { + /* + * Enable the first rectangle in the SMART layer to be + * able to use it as a drm plane. + */ + malidp_hw_write(mp->hwdev, 1, + mp->layer->base + MALIDP550_LS_ENABLE); malidp_hw_write(mp->hwdev, LAYER_H_VAL(src_w) | LAYER_V_VAL(src_h), mp->layer->base + MALIDP550_LS_R1_IN_SIZE); + } /* first clear the rotation bits */ val = malidp_hw_read(mp->hwdev, mp->layer->base + MALIDP_LAYER_CONTROL); val &= ~LAYER_ROT_MASK; /* setup the rotation and axis flip bits */ - if (plane->state->rotation & DRM_MODE_ROTATE_MASK) + if (state->rotation & DRM_MODE_ROTATE_MASK) val |= ilog2(plane->state->rotation & DRM_MODE_ROTATE_MASK) << LAYER_ROT_OFFSET; - if (plane->state->rotation & DRM_MODE_REFLECT_X) + if (state->rotation & DRM_MODE_REFLECT_X) val |= LAYER_H_FLIP; - if (plane->state->rotation & DRM_MODE_REFLECT_Y) + if (state->rotation & DRM_MODE_REFLECT_Y) val |= LAYER_V_FLIP; - val &= ~LAYER_COMP_MASK; - if (format_has_alpha) { - - /* - * always enable pixel alpha blending until we have a way - * to change blend modes - */ - val |= LAYER_COMP_PIXEL; - } else { + val &= ~(LAYER_COMP_MASK | LAYER_PMUL_ENABLE | LAYER_ALPHA(0xff)); - /* - * do not enable pixel alpha blending as the color channel - * does not have any alpha information - */ + if (state->alpha != DRM_BLEND_ALPHA_OPAQUE) { val |= LAYER_COMP_PLANE; - - /* Set layer alpha coefficient to 0xff ie fully opaque */ - val |= LAYER_ALPHA(0xff); + } else if (state->fb->format->has_alpha) { + /* We only care about blend mode if the format has alpha */ + switch (pixel_alpha) { + case DRM_MODE_BLEND_PREMULTI: + val |= LAYER_COMP_PIXEL | LAYER_PMUL_ENABLE; + break; + case DRM_MODE_BLEND_COVERAGE: + val |= LAYER_COMP_PIXEL; + break; + } } + val |= LAYER_ALPHA(plane_alpha); val &= ~LAYER_FLOWCFG(LAYER_FLOWCFG_MASK); - if (plane->state->crtc) { + if (state->crtc) { struct malidp_crtc_state *m = - to_malidp_crtc_state(plane->state->crtc->state); + to_malidp_crtc_state(state->crtc->state); if (m->scaler_config.scale_enable && m->scaler_config.plane_src_id == mp->layer->id) @@ -446,6 +707,9 @@ int malidp_de_planes_init(struct drm_device *drm) unsigned long crtcs = 1 << drm->mode_config.num_crtc; unsigned long flags = DRM_MODE_ROTATE_0 | DRM_MODE_ROTATE_90 | DRM_MODE_ROTATE_180 | DRM_MODE_ROTATE_270 | DRM_MODE_REFLECT_X | DRM_MODE_REFLECT_Y; + unsigned int blend_caps = BIT(DRM_MODE_BLEND_PIXEL_NONE) | + BIT(DRM_MODE_BLEND_PREMULTI) | + BIT(DRM_MODE_BLEND_COVERAGE); u32 *formats; int ret, i, j, n; @@ -483,13 +747,10 @@ int malidp_de_planes_init(struct drm_device *drm) plane->hwdev = malidp->dev; plane->layer = &map->layers[i]; + drm_plane_create_alpha_property(&plane->base); + drm_plane_create_blend_mode_property(&plane->base, blend_caps); + if (id == DE_SMART) { - /* - * Enable the first rectangle in the SMART layer to be - * able to use it as a drm plane. - */ - malidp_hw_write(malidp->dev, 1, - plane->layer->base + MALIDP550_LS_ENABLE); /* Skip the features which the SMART layer doesn't have. */ continue; } diff --git a/drivers/gpu/drm/arm/malidp_regs.h b/drivers/gpu/drm/arm/malidp_regs.h index 3579d36b2a71..7ce3e141464d 100644 --- a/drivers/gpu/drm/arm/malidp_regs.h +++ b/drivers/gpu/drm/arm/malidp_regs.h @@ -205,6 +205,7 @@ #define MALIDP500_SE_BASE 0x00c00 #define MALIDP500_SE_CONTROL 0x00c0c #define MALIDP500_SE_MEMWRITE_OUT_SIZE 0x00c2c +#define MALIDP500_SE_RGB_YUV_COEFFS 0x00C74 #define MALIDP500_SE_MEMWRITE_BASE 0x00e00 #define MALIDP500_DC_IRQ_BASE 0x00f00 #define MALIDP500_CONFIG_VALID 0x00f00 @@ -238,6 +239,7 @@ #define MALIDP550_SE_CONTROL 0x08010 #define MALIDP550_SE_MEMWRITE_ONESHOT (1 << 7) #define MALIDP550_SE_MEMWRITE_OUT_SIZE 0x08030 +#define MALIDP550_SE_RGB_YUV_COEFFS 0x08078 #define MALIDP550_SE_MEMWRITE_BASE 0x08100 #define MALIDP550_DC_BASE 0x0c000 #define MALIDP550_DC_CONTROL 0x0c010 @@ -245,6 +247,17 @@ #define MALIDP550_CONFIG_VALID 0x0c014 #define MALIDP550_CONFIG_ID 0x0ffd4 +/* register offsets specific to DP650 */ +#define MALIDP650_DE_LV_MMU_CTRL 0x000D0 +#define MALIDP650_DE_LG_MMU_CTRL 0x00048 +#define MALIDP650_DE_LS_MMU_CTRL 0x00078 + +/* bit masks to set the MMU control register */ +#define MALIDP_MMU_CTRL_EN (1 << 0) +#define MALIDP_MMU_CTRL_MODE (1 << 4) +#define MALIDP_MMU_CTRL_PX_PS(x) (1 << (8 + (x))) +#define MALIDP_MMU_CTRL_PP_NUM_REQ(x) (((x) & 0x7f) << 12) + /* * Starting with DP550 the register map blocks has been standardised to the * following layout: diff --git a/drivers/gpu/drm/bochs/bochs.h b/drivers/gpu/drm/bochs/bochs.h index b4f6bb521900..e7a69077e45a 100644 --- a/drivers/gpu/drm/bochs/bochs.h +++ b/drivers/gpu/drm/bochs/bochs.h @@ -58,6 +58,7 @@ struct bochs_device { void __iomem *fb_map; unsigned long fb_base; unsigned long fb_size; + unsigned long qext_size; /* mode */ u16 xres; @@ -121,7 +122,8 @@ int bochs_hw_init(struct drm_device *dev); void bochs_hw_fini(struct drm_device *dev); void bochs_hw_setmode(struct bochs_device *bochs, - struct drm_display_mode *mode); + struct drm_display_mode *mode, + const struct drm_format_info *format); void bochs_hw_setbase(struct bochs_device *bochs, int x, int y, u64 addr); diff --git a/drivers/gpu/drm/bochs/bochs_fbdev.c b/drivers/gpu/drm/bochs/bochs_fbdev.c index 8f4d6c052f7b..dd3c7df267da 100644 --- a/drivers/gpu/drm/bochs/bochs_fbdev.c +++ b/drivers/gpu/drm/bochs/bochs_fbdev.c @@ -63,9 +63,8 @@ static int bochsfb_create(struct drm_fb_helper *helper, mode_cmd.width = sizes->surface_width; mode_cmd.height = sizes->surface_height; - mode_cmd.pitches[0] = mode_cmd.width * ((sizes->surface_bpp + 7) / 8); - mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp, - sizes->surface_depth); + mode_cmd.pitches[0] = sizes->surface_width * 4; + mode_cmd.pixel_format = DRM_FORMAT_HOST_XRGB8888; size = mode_cmd.pitches[0] * mode_cmd.height; /* alloc, pin & map bo */ @@ -137,8 +136,19 @@ static const struct drm_fb_helper_funcs bochs_fb_helper_funcs = { .fb_probe = bochsfb_create, }; +static struct drm_framebuffer * +bochs_gem_fb_create(struct drm_device *dev, struct drm_file *file, + const struct drm_mode_fb_cmd2 *mode_cmd) +{ + if (mode_cmd->pixel_format != DRM_FORMAT_XRGB8888 && + mode_cmd->pixel_format != DRM_FORMAT_BGRX8888) + return ERR_PTR(-EINVAL); + + return drm_gem_fb_create(dev, file, mode_cmd); +} + const struct drm_mode_config_funcs bochs_mode_funcs = { - .fb_create = drm_gem_fb_create, + .fb_create = bochs_gem_fb_create, }; int bochs_fbdev_init(struct bochs_device *bochs) diff --git a/drivers/gpu/drm/bochs/bochs_hw.c b/drivers/gpu/drm/bochs/bochs_hw.c index 16e4f1caccca..cacff73a64ab 100644 --- a/drivers/gpu/drm/bochs/bochs_hw.c +++ b/drivers/gpu/drm/bochs/bochs_hw.c @@ -47,11 +47,33 @@ static void bochs_dispi_write(struct bochs_device *bochs, u16 reg, u16 val) } } +static void bochs_hw_set_big_endian(struct bochs_device *bochs) +{ + if (bochs->qext_size < 8) + return; + + writel(0xbebebebe, bochs->mmio + 0x604); +} + +static void bochs_hw_set_little_endian(struct bochs_device *bochs) +{ + if (bochs->qext_size < 8) + return; + + writel(0x1e1e1e1e, bochs->mmio + 0x604); +} + +#ifdef __BIG_ENDIAN +#define bochs_hw_set_native_endian(_b) bochs_hw_set_big_endian(_b) +#else +#define bochs_hw_set_native_endian(_b) bochs_hw_set_little_endian(_b) +#endif + int bochs_hw_init(struct drm_device *dev) { struct bochs_device *bochs = dev->dev_private; struct pci_dev *pdev = dev->pdev; - unsigned long addr, size, mem, ioaddr, iosize, qext_size; + unsigned long addr, size, mem, ioaddr, iosize; u16 id; if (pdev->resource[2].flags & IORESOURCE_MEM) { @@ -117,19 +139,14 @@ int bochs_hw_init(struct drm_device *dev) ioaddr); if (bochs->mmio && pdev->revision >= 2) { - qext_size = readl(bochs->mmio + 0x600); - if (qext_size < 4 || qext_size > iosize) + bochs->qext_size = readl(bochs->mmio + 0x600); + if (bochs->qext_size < 4 || bochs->qext_size > iosize) { + bochs->qext_size = 0; goto noext; - DRM_DEBUG("Found qemu ext regs, size %ld\n", qext_size); - if (qext_size >= 8) { -#ifdef __BIG_ENDIAN - writel(0xbebebebe, bochs->mmio + 0x604); -#else - writel(0x1e1e1e1e, bochs->mmio + 0x604); -#endif - DRM_DEBUG(" qext endian: 0x%x\n", - readl(bochs->mmio + 0x604)); } + DRM_DEBUG("Found qemu ext regs, size %ld\n", + bochs->qext_size); + bochs_hw_set_native_endian(bochs); } noext: @@ -150,7 +167,8 @@ void bochs_hw_fini(struct drm_device *dev) } void bochs_hw_setmode(struct bochs_device *bochs, - struct drm_display_mode *mode) + struct drm_display_mode *mode, + const struct drm_format_info *format) { bochs->xres = mode->hdisplay; bochs->yres = mode->vdisplay; @@ -158,8 +176,12 @@ void bochs_hw_setmode(struct bochs_device *bochs, bochs->stride = mode->hdisplay * (bochs->bpp / 8); bochs->yres_virtual = bochs->fb_size / bochs->stride; - DRM_DEBUG_DRIVER("%dx%d @ %d bpp, vy %d\n", + DRM_DEBUG_DRIVER("%dx%d @ %d bpp, format %c%c%c%c, vy %d\n", bochs->xres, bochs->yres, bochs->bpp, + (format->format >> 0) & 0xff, + (format->format >> 8) & 0xff, + (format->format >> 16) & 0xff, + (format->format >> 24) & 0xff, bochs->yres_virtual); bochs_vga_writeb(bochs, 0x3c0, 0x20); /* unblank */ @@ -177,6 +199,20 @@ void bochs_hw_setmode(struct bochs_device *bochs, bochs_dispi_write(bochs, VBE_DISPI_INDEX_ENABLE, VBE_DISPI_ENABLED | VBE_DISPI_LFB_ENABLED); + + switch (format->format) { + case DRM_FORMAT_XRGB8888: + bochs_hw_set_little_endian(bochs); + break; + case DRM_FORMAT_BGRX8888: + bochs_hw_set_big_endian(bochs); + break; + default: + /* should not happen */ + DRM_ERROR("%s: Huh? Got framebuffer format 0x%x", + __func__, format->format); + break; + }; } void bochs_hw_setbase(struct bochs_device *bochs, diff --git a/drivers/gpu/drm/bochs/bochs_kms.c b/drivers/gpu/drm/bochs/bochs_kms.c index ea9a43d31bf1..9bc5b438aefd 100644 --- a/drivers/gpu/drm/bochs/bochs_kms.c +++ b/drivers/gpu/drm/bochs/bochs_kms.c @@ -77,7 +77,10 @@ static int bochs_crtc_mode_set(struct drm_crtc *crtc, struct bochs_device *bochs = container_of(crtc, struct bochs_device, crtc); - bochs_hw_setmode(bochs, mode); + if (WARN_ON(crtc->primary->fb == NULL)) + return -EINVAL; + + bochs_hw_setmode(bochs, mode, crtc->primary->fb->format); bochs_crtc_mode_set_base(crtc, x, y, old_fb); return 0; } @@ -126,12 +129,44 @@ static const struct drm_crtc_helper_funcs bochs_helper_funcs = { .commit = bochs_crtc_commit, }; +static const uint32_t bochs_formats[] = { + DRM_FORMAT_XRGB8888, + DRM_FORMAT_BGRX8888, +}; + +static struct drm_plane *bochs_primary_plane(struct drm_device *dev) +{ + struct drm_plane *primary; + int ret; + + primary = kzalloc(sizeof(*primary), GFP_KERNEL); + if (primary == NULL) { + DRM_DEBUG_KMS("Failed to allocate primary plane\n"); + return NULL; + } + + ret = drm_universal_plane_init(dev, primary, 0, + &drm_primary_helper_funcs, + bochs_formats, + ARRAY_SIZE(bochs_formats), + NULL, + DRM_PLANE_TYPE_PRIMARY, NULL); + if (ret) { + kfree(primary); + primary = NULL; + } + + return primary; +} + static void bochs_crtc_init(struct drm_device *dev) { struct bochs_device *bochs = dev->dev_private; struct drm_crtc *crtc = &bochs->crtc; + struct drm_plane *primary = bochs_primary_plane(dev); - drm_crtc_init(dev, crtc, &bochs_crtc_funcs); + drm_crtc_init_with_planes(dev, crtc, primary, NULL, + &bochs_crtc_funcs, NULL); drm_crtc_helper_add(crtc, &bochs_helper_funcs); } @@ -250,6 +285,7 @@ int bochs_kms_init(struct bochs_device *bochs) bochs->dev->mode_config.fb_base = bochs->fb_base; bochs->dev->mode_config.preferred_depth = 24; bochs->dev->mode_config.prefer_shadow = 0; + bochs->dev->mode_config.quirk_addfb_prefer_host_byte_order = true; bochs->dev->mode_config.funcs = &bochs_mode_funcs; diff --git a/drivers/gpu/drm/bridge/thc63lvd1024.c b/drivers/gpu/drm/bridge/thc63lvd1024.c index c8b9edd5a7f4..b083a740565c 100644 --- a/drivers/gpu/drm/bridge/thc63lvd1024.c +++ b/drivers/gpu/drm/bridge/thc63lvd1024.c @@ -45,6 +45,23 @@ static int thc63_attach(struct drm_bridge *bridge) return drm_bridge_attach(bridge->encoder, thc63->next, bridge); } +static enum drm_mode_status thc63_mode_valid(struct drm_bridge *bridge, + const struct drm_display_mode *mode) +{ + /* + * The THC63LVD1024 clock frequency range is 8 to 135 MHz in single-in + * mode. Note that the limits are different in dual-in, single-out mode, + * and will need to be adjusted accordingly. + */ + if (mode->clock < 8000) + return MODE_CLOCK_LOW; + + if (mode->clock > 135000) + return MODE_CLOCK_HIGH; + + return MODE_OK; +} + static void thc63_enable(struct drm_bridge *bridge) { struct thc63_dev *thc63 = to_thc63(bridge); @@ -77,6 +94,7 @@ static void thc63_disable(struct drm_bridge *bridge) static const struct drm_bridge_funcs thc63_bridge_func = { .attach = thc63_attach, + .mode_valid = thc63_mode_valid, .enable = thc63_enable, .disable = thc63_disable, }; diff --git a/drivers/gpu/drm/bridge/ti-sn65dsi86.c b/drivers/gpu/drm/bridge/ti-sn65dsi86.c index f8a931cf3665..680566d97adc 100644 --- a/drivers/gpu/drm/bridge/ti-sn65dsi86.c +++ b/drivers/gpu/drm/bridge/ti-sn65dsi86.c @@ -458,18 +458,6 @@ static void ti_sn_bridge_enable(struct drm_bridge *bridge) unsigned int val; int ret; - /* - * FIXME: - * This 70ms was found necessary by experimentation. If it's not - * present, link training fails. It seems like it can go anywhere from - * pre_enable() up to semi-auto link training initiation below. - * - * Neither the datasheet for the bridge nor the panel tested mention a - * delay of this magnitude in the timing requirements. So for now, add - * the mystery delay until someone figures out a better fix. - */ - msleep(70); - /* DSI_A lane config */ val = CHA_DSI_LANES(4 - pdata->dsi->lanes); regmap_update_bits(pdata->regmap, SN_DSI_LANES_REG, @@ -536,7 +524,22 @@ static void ti_sn_bridge_pre_enable(struct drm_bridge *bridge) /* configure bridge ref_clk */ ti_sn_bridge_set_refclk_freq(pdata); - /* in case drm_panel is connected then HPD is not supported */ + /* + * HPD on this bridge chip is a bit useless. This is an eDP bridge + * so the HPD is an internal signal that's only there to signal that + * the panel is done powering up. ...but the bridge chip debounces + * this signal by between 100 ms and 400 ms (depending on process, + * voltage, and temperate--I measured it at about 200 ms). One + * particular panel asserted HPD 84 ms after it was powered on meaning + * that we saw HPD 284 ms after power on. ...but the same panel said + * that instead of looking at HPD you could just hardcode a delay of + * 200 ms. We'll assume that the panel driver will have the hardcoded + * delay in its prepare and always disable HPD. + * + * If HPD somehow makes sense on some future panel we'll have to + * change this to be conditional on someone specifying that HPD should + * be used. + */ regmap_update_bits(pdata->regmap, SN_HPD_DISABLE_REG, HPD_DISABLE, HPD_DISABLE); diff --git a/drivers/gpu/drm/drm_atomic.c b/drivers/gpu/drm/drm_atomic.c index 7ada75919756..2870ae205237 100644 --- a/drivers/gpu/drm/drm_atomic.c +++ b/drivers/gpu/drm/drm_atomic.c @@ -1106,7 +1106,7 @@ static void __drm_state_dump(struct drm_device *dev, struct drm_printer *p, struct drm_connector *connector; struct drm_connector_list_iter conn_iter; - if (!drm_core_check_feature(dev, DRIVER_ATOMIC)) + if (!drm_drv_uses_atomic_modeset(dev)) return; list_for_each_entry(plane, &config->plane_list, head) { diff --git a/drivers/gpu/drm/drm_atomic_helper.c b/drivers/gpu/drm/drm_atomic_helper.c index 4225b1ca407b..1cc3a045ec2f 100644 --- a/drivers/gpu/drm/drm_atomic_helper.c +++ b/drivers/gpu/drm/drm_atomic_helper.c @@ -3588,9 +3588,7 @@ void __drm_atomic_helper_plane_reset(struct drm_plane *plane, state->plane = plane; state->rotation = DRM_MODE_ROTATE_0; - /* Reset the alpha value to fully opaque if it matters */ - if (plane->alpha_property) - state->alpha = plane->alpha_property->values[1]; + state->alpha = DRM_BLEND_ALPHA_OPAQUE; state->pixel_blend_mode = DRM_MODE_BLEND_PREMULTI; plane->state = state; diff --git a/drivers/gpu/drm/drm_debugfs.c b/drivers/gpu/drm/drm_debugfs.c index 6f28fe58f169..373bd4c2b698 100644 --- a/drivers/gpu/drm/drm_debugfs.c +++ b/drivers/gpu/drm/drm_debugfs.c @@ -151,7 +151,7 @@ int drm_debugfs_init(struct drm_minor *minor, int minor_id, return ret; } - if (drm_core_check_feature(dev, DRIVER_ATOMIC)) { + if (drm_drv_uses_atomic_modeset(dev)) { ret = drm_atomic_debugfs_init(minor); if (ret) { DRM_ERROR("Failed to create atomic debugfs files\n"); diff --git a/drivers/gpu/drm/drm_edid.c b/drivers/gpu/drm/drm_edid.c index 3c9fc99648b7..08821dfa3327 100644 --- a/drivers/gpu/drm/drm_edid.c +++ b/drivers/gpu/drm/drm_edid.c @@ -119,6 +119,9 @@ static const struct edid_quirk { /* SDC panel of Lenovo B50-80 reports 8 bpc, but is a 6 bpc panel */ { "SDC", 0x3652, EDID_QUIRK_FORCE_6BPC }, + /* BOE model 0x0771 reports 8 bpc, but is a 6 bpc panel */ + { "BOE", 0x0771, EDID_QUIRK_FORCE_6BPC }, + /* Belinea 10 15 55 */ { "MAX", 1516, EDID_QUIRK_PREFER_LARGE_60 }, { "MAX", 0x77e, EDID_QUIRK_PREFER_LARGE_60 }, diff --git a/drivers/gpu/drm/drm_fb_helper.c b/drivers/gpu/drm/drm_fb_helper.c index 8e95d0f7c71d..9b111e846847 100644 --- a/drivers/gpu/drm/drm_fb_helper.c +++ b/drivers/gpu/drm/drm_fb_helper.c @@ -56,6 +56,25 @@ MODULE_PARM_DESC(drm_fbdev_overalloc, "Overallocation of the fbdev buffer (%) [default=" __MODULE_STRING(CONFIG_DRM_FBDEV_OVERALLOC) "]"); +/* + * In order to keep user-space compatibility, we want in certain use-cases + * to keep leaking the fbdev physical address to the user-space program + * handling the fbdev buffer. + * This is a bad habit essentially kept into closed source opengl driver + * that should really be moved into open-source upstream projects instead + * of using legacy physical addresses in user space to communicate with + * other out-of-tree kernel modules. + * + * This module_param *should* be removed as soon as possible and be + * considered as a broken and legacy behaviour from a modern fbdev device. + */ +#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM) +static bool drm_leak_fbdev_smem = false; +module_param_unsafe(drm_leak_fbdev_smem, bool, 0600); +MODULE_PARM_DESC(fbdev_emulation, + "Allow unsafe leaking fbdev physical smem address [default=false]"); +#endif + static LIST_HEAD(kernel_fb_helper_list); static DEFINE_MUTEX(kernel_fb_helper_lock); @@ -2370,7 +2389,6 @@ static int drm_pick_crtcs(struct drm_fb_helper *fb_helper, { int c, o; struct drm_connector *connector; - const struct drm_connector_helper_funcs *connector_funcs; int my_score, best_score, score; struct drm_fb_helper_crtc **crtcs, *crtc; struct drm_fb_helper_connector *fb_helper_conn; @@ -2399,8 +2417,6 @@ static int drm_pick_crtcs(struct drm_fb_helper *fb_helper, if (drm_has_preferred_mode(fb_helper_conn, width, height)) my_score++; - connector_funcs = connector->helper_private; - /* * select a crtc for this connector and then attempt to configure * remaining connectors @@ -2673,8 +2689,12 @@ __drm_fb_helper_initial_config_and_unlock(struct drm_fb_helper *fb_helper, info = fb_helper->fbdev; info->var.pixclock = 0; - /* don't leak any physical addresses to userspace */ - info->flags |= FBINFO_HIDE_SMEM_START; + /* Shamelessly allow physical address leaking to userspace */ +#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM) + if (!drm_leak_fbdev_smem) +#endif + /* don't leak any physical addresses to userspace */ + info->flags |= FBINFO_HIDE_SMEM_START; /* Need to drop locks to avoid recursive deadlock in * register_framebuffer. This is ok because the only thing left to do is @@ -2846,7 +2866,7 @@ int drm_fb_helper_fbdev_setup(struct drm_device *dev, if (!max_conn_count) max_conn_count = dev->mode_config.num_connector; if (!max_conn_count) { - DRM_DEV_ERROR(dev->dev, "No connectors\n"); + DRM_DEV_ERROR(dev->dev, "fbdev: No connectors\n"); return -EINVAL; } @@ -2854,13 +2874,13 @@ int drm_fb_helper_fbdev_setup(struct drm_device *dev, ret = drm_fb_helper_init(dev, fb_helper, max_conn_count); if (ret < 0) { - DRM_DEV_ERROR(dev->dev, "Failed to initialize fbdev helper\n"); + DRM_DEV_ERROR(dev->dev, "fbdev: Failed to initialize (ret=%d)\n", ret); return ret; } ret = drm_fb_helper_single_add_all_connectors(fb_helper); if (ret < 0) { - DRM_DEV_ERROR(dev->dev, "Failed to add connectors\n"); + DRM_DEV_ERROR(dev->dev, "fbdev: Failed to add connectors (ret=%d)\n", ret); goto err_drm_fb_helper_fini; } @@ -2869,7 +2889,7 @@ int drm_fb_helper_fbdev_setup(struct drm_device *dev, ret = drm_fb_helper_initial_config(fb_helper, preferred_bpp); if (ret < 0) { - DRM_DEV_ERROR(dev->dev, "Failed to set fbdev configuration\n"); + DRM_DEV_ERROR(dev->dev, "fbdev: Failed to set configuration (ret=%d)\n", ret); goto err_drm_fb_helper_fini; } @@ -3084,6 +3104,12 @@ int drm_fb_helper_generic_probe(struct drm_fb_helper *fb_helper, fbi->screen_size = fb->height * fb->pitches[0]; fbi->fix.smem_len = fbi->screen_size; fbi->screen_buffer = buffer->vaddr; + /* Shamelessly leak the physical address to user-space */ +#if IS_ENABLED(CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM) + if (drm_leak_fbdev_smem && fbi->fix.smem_start == 0) + fbi->fix.smem_start = + page_to_phys(virt_to_page(fbi->screen_buffer)); +#endif strcpy(fbi->fix.id, "DRM emulated"); drm_fb_helper_fill_fix(fbi, fb->pitches[0], fb->format->depth); diff --git a/drivers/gpu/drm/drm_fourcc.c b/drivers/gpu/drm/drm_fourcc.c index be1d6aaef651..90a1c846fc25 100644 --- a/drivers/gpu/drm/drm_fourcc.c +++ b/drivers/gpu/drm/drm_fourcc.c @@ -96,6 +96,41 @@ uint32_t drm_mode_legacy_fb_format(uint32_t bpp, uint32_t depth) EXPORT_SYMBOL(drm_mode_legacy_fb_format); /** + * drm_driver_legacy_fb_format - compute drm fourcc code from legacy description + * @bpp: bits per pixels + * @depth: bit depth per pixel + * @native: use host native byte order + * + * Computes a drm fourcc pixel format code for the given @bpp/@depth values. + * Unlike drm_mode_legacy_fb_format() this looks at the drivers mode_config, + * and depending on the quirk_addfb_prefer_host_byte_order flag it returns + * little endian byte order or host byte order framebuffer formats. + */ +uint32_t drm_driver_legacy_fb_format(struct drm_device *dev, + uint32_t bpp, uint32_t depth) +{ + uint32_t fmt = drm_mode_legacy_fb_format(bpp, depth); + + if (dev->mode_config.quirk_addfb_prefer_host_byte_order) { + if (fmt == DRM_FORMAT_XRGB8888) + fmt = DRM_FORMAT_HOST_XRGB8888; + if (fmt == DRM_FORMAT_ARGB8888) + fmt = DRM_FORMAT_HOST_ARGB8888; + if (fmt == DRM_FORMAT_RGB565) + fmt = DRM_FORMAT_HOST_RGB565; + if (fmt == DRM_FORMAT_XRGB1555) + fmt = DRM_FORMAT_HOST_XRGB1555; + } + + if (dev->mode_config.quirk_addfb_prefer_xbgr_30bpp && + fmt == DRM_FORMAT_XRGB2101010) + fmt = DRM_FORMAT_XBGR2101010; + + return fmt; +} +EXPORT_SYMBOL(drm_driver_legacy_fb_format); + +/** * drm_get_format_name - fill a string with a drm fourcc format's name * @format: format to compute name of * @buf: caller-supplied buffer diff --git a/drivers/gpu/drm/drm_framebuffer.c b/drivers/gpu/drm/drm_framebuffer.c index 1ee3d6b44280..3bf729d0aae5 100644 --- a/drivers/gpu/drm/drm_framebuffer.c +++ b/drivers/gpu/drm/drm_framebuffer.c @@ -116,7 +116,7 @@ int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or, if (!drm_core_check_feature(dev, DRIVER_MODESET)) return -EOPNOTSUPP; - r.pixel_format = drm_mode_legacy_fb_format(or->bpp, or->depth); + r.pixel_format = drm_driver_legacy_fb_format(dev, or->bpp, or->depth); if (r.pixel_format == DRM_FORMAT_INVALID) { DRM_DEBUG("bad {bpp:%d, depth:%d}\n", or->bpp, or->depth); return -EINVAL; @@ -129,21 +129,6 @@ int drm_mode_addfb(struct drm_device *dev, struct drm_mode_fb_cmd *or, r.pitches[0] = or->pitch; r.handles[0] = or->handle; - if (dev->mode_config.quirk_addfb_prefer_xbgr_30bpp && - r.pixel_format == DRM_FORMAT_XRGB2101010) - r.pixel_format = DRM_FORMAT_XBGR2101010; - - if (dev->mode_config.quirk_addfb_prefer_host_byte_order) { - if (r.pixel_format == DRM_FORMAT_XRGB8888) - r.pixel_format = DRM_FORMAT_HOST_XRGB8888; - if (r.pixel_format == DRM_FORMAT_ARGB8888) - r.pixel_format = DRM_FORMAT_HOST_ARGB8888; - if (r.pixel_format == DRM_FORMAT_RGB565) - r.pixel_format = DRM_FORMAT_HOST_RGB565; - if (r.pixel_format == DRM_FORMAT_XRGB1555) - r.pixel_format = DRM_FORMAT_HOST_XRGB1555; - } - ret = drm_mode_addfb2(dev, &r, file_priv); if (ret) return ret; diff --git a/drivers/gpu/drm/drm_gem_framebuffer_helper.c b/drivers/gpu/drm/drm_gem_framebuffer_helper.c index 7607f9cd6f77..ded7a379ac35 100644 --- a/drivers/gpu/drm/drm_gem_framebuffer_helper.c +++ b/drivers/gpu/drm/drm_gem_framebuffer_helper.c @@ -316,8 +316,8 @@ drm_gem_fbdev_fb_create(struct drm_device *dev, if (pitch_align) mode_cmd.pitches[0] = roundup(mode_cmd.pitches[0], pitch_align); - mode_cmd.pixel_format = drm_mode_legacy_fb_format(sizes->surface_bpp, - sizes->surface_depth); + mode_cmd.pixel_format = drm_driver_legacy_fb_format(dev, sizes->surface_bpp, + sizes->surface_depth); if (obj->size < mode_cmd.pitches[0] * mode_cmd.height) return ERR_PTR(-EINVAL); diff --git a/drivers/gpu/drm/drm_ioctl.c b/drivers/gpu/drm/drm_ioctl.c index 60dfbfae6a02..94bd872d56c4 100644 --- a/drivers/gpu/drm/drm_ioctl.c +++ b/drivers/gpu/drm/drm_ioctl.c @@ -306,6 +306,12 @@ drm_setclientcap(struct drm_device *dev, void *data, struct drm_file *file_priv) { struct drm_set_client_cap *req = data; + /* No render-only settable capabilities for now */ + + /* Below caps that only works with KMS drivers */ + if (!drm_core_check_feature(dev, DRIVER_MODESET)) + return -EOPNOTSUPP; + switch (req->capability) { case DRM_CLIENT_CAP_STEREO_3D: if (req->value > 1) diff --git a/drivers/gpu/drm/drm_panel.c b/drivers/gpu/drm/drm_panel.c index 0db486d10d1c..c33f95e08e1b 100644 --- a/drivers/gpu/drm/drm_panel.c +++ b/drivers/gpu/drm/drm_panel.c @@ -24,7 +24,6 @@ #include <linux/err.h> #include <linux/module.h> -#include <drm/drm_device.h> #include <drm/drm_crtc.h> #include <drm/drm_panel.h> @@ -105,13 +104,6 @@ int drm_panel_attach(struct drm_panel *panel, struct drm_connector *connector) if (panel->connector) return -EBUSY; - panel->link = device_link_add(connector->dev->dev, panel->dev, 0); - if (!panel->link) { - dev_err(panel->dev, "failed to link panel to %s\n", - dev_name(connector->dev->dev)); - return -EINVAL; - } - panel->connector = connector; panel->drm = connector->dev; @@ -133,8 +125,6 @@ EXPORT_SYMBOL(drm_panel_attach); */ int drm_panel_detach(struct drm_panel *panel) { - device_link_del(panel->link); - panel->connector = NULL; panel->drm = NULL; diff --git a/drivers/gpu/drm/drm_panel_orientation_quirks.c b/drivers/gpu/drm/drm_panel_orientation_quirks.c index fe9c6c731e87..ee4a5e1221f1 100644 --- a/drivers/gpu/drm/drm_panel_orientation_quirks.c +++ b/drivers/gpu/drm/drm_panel_orientation_quirks.c @@ -30,6 +30,12 @@ struct drm_dmi_panel_orientation_data { int orientation; }; +static const struct drm_dmi_panel_orientation_data acer_s1003 = { + .width = 800, + .height = 1280, + .orientation = DRM_MODE_PANEL_ORIENTATION_RIGHT_UP, +}; + static const struct drm_dmi_panel_orientation_data asus_t100ha = { .width = 800, .height = 1280, @@ -67,7 +73,13 @@ static const struct drm_dmi_panel_orientation_data lcd800x1280_rightside_up = { }; static const struct dmi_system_id orientation_data[] = { - { /* Asus T100HA */ + { /* Acer One 10 (S1003) */ + .matches = { + DMI_EXACT_MATCH(DMI_SYS_VENDOR, "Acer"), + DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "One S1003"), + }, + .driver_data = (void *)&acer_s1003, + }, { /* Asus T100HA */ .matches = { DMI_EXACT_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."), DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "T100HAN"), diff --git a/drivers/gpu/drm/drm_syncobj.c b/drivers/gpu/drm/drm_syncobj.c index 497729202bfe..5c2091dbd230 100644 --- a/drivers/gpu/drm/drm_syncobj.c +++ b/drivers/gpu/drm/drm_syncobj.c @@ -66,20 +66,9 @@ static const char *drm_syncobj_stub_fence_get_name(struct dma_fence *fence) return "syncobjstub"; } -static bool drm_syncobj_stub_fence_enable_signaling(struct dma_fence *fence) -{ - return !dma_fence_is_signaled(fence); -} - -static void drm_syncobj_stub_fence_release(struct dma_fence *f) -{ - kfree(f); -} static const struct dma_fence_ops drm_syncobj_stub_fence_ops = { .get_driver_name = drm_syncobj_stub_fence_get_name, .get_timeline_name = drm_syncobj_stub_fence_get_name, - .enable_signaling = drm_syncobj_stub_fence_enable_signaling, - .release = drm_syncobj_stub_fence_release, }; @@ -124,6 +113,8 @@ static int drm_syncobj_fence_get_or_add_callback(struct drm_syncobj *syncobj, { int ret; + WARN_ON(*fence); + *fence = drm_syncobj_fence_get(syncobj); if (*fence) return 1; @@ -683,7 +674,6 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs, { struct syncobj_wait_entry *entries; struct dma_fence *fence; - signed long ret; uint32_t signaled_count, i; entries = kcalloc(count, sizeof(*entries), GFP_KERNEL); @@ -703,7 +693,7 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs, if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT) { continue; } else { - ret = -EINVAL; + timeout = -EINVAL; goto cleanup_entries; } } @@ -715,12 +705,6 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs, } } - /* Initialize ret to the max of timeout and 1. That way, the - * default return value indicates a successful wait and not a - * timeout. - */ - ret = max_t(signed long, timeout, 1); - if (signaled_count == count || (signaled_count > 0 && !(flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_ALL))) @@ -735,6 +719,9 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs, if (flags & DRM_SYNCOBJ_WAIT_FLAGS_WAIT_FOR_SUBMIT) { for (i = 0; i < count; ++i) { + if (entries[i].fence) + continue; + drm_syncobj_fence_get_or_add_callback(syncobjs[i], &entries[i].fence, &entries[i].syncobj_cb, @@ -771,18 +758,17 @@ static signed long drm_syncobj_array_wait_timeout(struct drm_syncobj **syncobjs, goto done_waiting; if (timeout == 0) { - /* If we are doing a 0 timeout wait and we got - * here, then we just timed out. - */ - ret = 0; + timeout = -ETIME; goto done_waiting; } - ret = schedule_timeout(ret); + if (signal_pending(current)) { + timeout = -ERESTARTSYS; + goto done_waiting; + } - if (ret > 0 && signal_pending(current)) - ret = -ERESTARTSYS; - } while (ret > 0); + timeout = schedule_timeout(timeout); + } while (1); done_waiting: __set_current_state(TASK_RUNNING); @@ -799,7 +785,7 @@ cleanup_entries: } kfree(entries); - return ret; + return timeout; } /** @@ -840,19 +826,16 @@ static int drm_syncobj_array_wait(struct drm_device *dev, struct drm_syncobj **syncobjs) { signed long timeout = drm_timeout_abs_to_jiffies(wait->timeout_nsec); - signed long ret = 0; uint32_t first = ~0; - ret = drm_syncobj_array_wait_timeout(syncobjs, - wait->count_handles, - wait->flags, - timeout, &first); - if (ret < 0) - return ret; + timeout = drm_syncobj_array_wait_timeout(syncobjs, + wait->count_handles, + wait->flags, + timeout, &first); + if (timeout < 0) + return timeout; wait->first_signaled = first; - if (ret == 0) - return -ETIME; return 0; } diff --git a/drivers/gpu/drm/etnaviv/etnaviv_drv.c b/drivers/gpu/drm/etnaviv/etnaviv_drv.c index 9b2720b41571..83c1f46670bf 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_drv.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_drv.c @@ -592,8 +592,6 @@ static int etnaviv_pdev_probe(struct platform_device *pdev) struct device *dev = &pdev->dev; struct component_match *match = NULL; - dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32)); - if (!dev->platform_data) { struct device_node *core_node; @@ -655,13 +653,30 @@ static int __init etnaviv_init(void) for_each_compatible_node(np, NULL, "vivante,gc") { if (!of_device_is_available(np)) continue; - pdev = platform_device_register_simple("etnaviv", -1, - NULL, 0); - if (IS_ERR(pdev)) { - ret = PTR_ERR(pdev); + + pdev = platform_device_alloc("etnaviv", -1); + if (!pdev) { + ret = -ENOMEM; + of_node_put(np); + goto unregister_platform_driver; + } + pdev->dev.coherent_dma_mask = DMA_BIT_MASK(40); + pdev->dev.dma_mask = &pdev->dev.coherent_dma_mask; + + /* + * Apply the same DMA configuration to the virtual etnaviv + * device as the GPU we found. This assumes that all Vivante + * GPUs in the system share the same DMA constraints. + */ + of_dma_configure(&pdev->dev, np, true); + + ret = platform_device_add(pdev); + if (ret) { + platform_device_put(pdev); of_node_put(np); goto unregister_platform_driver; } + etnaviv_drm = pdev; of_node_put(np); break; diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c index 69e9b431bf1f..e7c3ed6c9a2e 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c @@ -105,7 +105,7 @@ static void etnaviv_sched_timedout_job(struct drm_sched_job *sched_job) change = dma_addr - gpu->hangcheck_dma_addr; if (change < 0 || change > 16) { gpu->hangcheck_dma_addr = dma_addr; - schedule_delayed_work(&sched_job->work_tdr, + schedule_delayed_work(&sched_job->sched->work_tdr, sched_job->sched->timeout); return; } diff --git a/drivers/gpu/drm/exynos/exynos_drm_drv.c b/drivers/gpu/drm/exynos/exynos_drm_drv.c index b599f74692e5..6f76baf4550a 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_drv.c +++ b/drivers/gpu/drm/exynos/exynos_drm_drv.c @@ -149,37 +149,15 @@ static struct drm_driver exynos_drm_driver = { static int exynos_drm_suspend(struct device *dev) { struct drm_device *drm_dev = dev_get_drvdata(dev); - struct exynos_drm_private *private; - - if (!drm_dev) - return 0; - - private = drm_dev->dev_private; - - drm_kms_helper_poll_disable(drm_dev); - exynos_drm_fbdev_suspend(drm_dev); - private->suspend_state = drm_atomic_helper_suspend(drm_dev); - if (IS_ERR(private->suspend_state)) { - exynos_drm_fbdev_resume(drm_dev); - drm_kms_helper_poll_enable(drm_dev); - return PTR_ERR(private->suspend_state); - } - return 0; + return drm_mode_config_helper_suspend(drm_dev); } static void exynos_drm_resume(struct device *dev) { struct drm_device *drm_dev = dev_get_drvdata(dev); - struct exynos_drm_private *private; - - if (!drm_dev) - return; - private = drm_dev->dev_private; - drm_atomic_helper_resume(drm_dev, private->suspend_state); - exynos_drm_fbdev_resume(drm_dev); - drm_kms_helper_poll_enable(drm_dev); + drm_mode_config_helper_resume(drm_dev); } static const struct dev_pm_ops exynos_drm_pm_ops = { diff --git a/drivers/gpu/drm/exynos/exynos_drm_drv.h b/drivers/gpu/drm/exynos/exynos_drm_drv.h index c737c4bd2c19..ec9604f1272b 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_drv.h +++ b/drivers/gpu/drm/exynos/exynos_drm_drv.h @@ -92,6 +92,8 @@ struct exynos_drm_plane { #define EXYNOS_DRM_PLANE_CAP_SCALE (1 << 1) #define EXYNOS_DRM_PLANE_CAP_ZPOS (1 << 2) #define EXYNOS_DRM_PLANE_CAP_TILE (1 << 3) +#define EXYNOS_DRM_PLANE_CAP_PIX_BLEND (1 << 4) +#define EXYNOS_DRM_PLANE_CAP_WIN_BLEND (1 << 5) /* * Exynos DRM plane configuration structure. @@ -195,7 +197,6 @@ struct drm_exynos_file_private { */ struct exynos_drm_private { struct drm_fb_helper *fb_helper; - struct drm_atomic_state *suspend_state; struct device *g2d_dev; struct device *dma_dev; diff --git a/drivers/gpu/drm/exynos/exynos_drm_dsi.c b/drivers/gpu/drm/exynos/exynos_drm_dsi.c index 781b82c2c579..07af7758066d 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_dsi.c +++ b/drivers/gpu/drm/exynos/exynos_drm_dsi.c @@ -255,6 +255,7 @@ struct exynos_dsi { struct mipi_dsi_host dsi_host; struct drm_connector connector; struct drm_panel *panel; + struct drm_bridge *out_bridge; struct device *dev; void __iomem *reg_base; @@ -279,7 +280,7 @@ struct exynos_dsi { struct list_head transfer_list; const struct exynos_dsi_driver_data *driver_data; - struct device_node *bridge_node; + struct device_node *in_bridge_node; }; #define host_to_dsi(host) container_of(host, struct exynos_dsi, dsi_host) @@ -1382,29 +1383,37 @@ static void exynos_dsi_enable(struct drm_encoder *encoder) return; pm_runtime_get_sync(dsi->dev); - dsi->state |= DSIM_STATE_ENABLED; - ret = drm_panel_prepare(dsi->panel); - if (ret < 0) { - dsi->state &= ~DSIM_STATE_ENABLED; - pm_runtime_put_sync(dsi->dev); - return; + if (dsi->panel) { + ret = drm_panel_prepare(dsi->panel); + if (ret < 0) + goto err_put_sync; + } else { + drm_bridge_pre_enable(dsi->out_bridge); } exynos_dsi_set_display_mode(dsi); exynos_dsi_set_display_enable(dsi, true); - ret = drm_panel_enable(dsi->panel); - if (ret < 0) { - dsi->state &= ~DSIM_STATE_ENABLED; - exynos_dsi_set_display_enable(dsi, false); - drm_panel_unprepare(dsi->panel); - pm_runtime_put_sync(dsi->dev); - return; + if (dsi->panel) { + ret = drm_panel_enable(dsi->panel); + if (ret < 0) + goto err_display_disable; + } else { + drm_bridge_enable(dsi->out_bridge); } dsi->state |= DSIM_STATE_VIDOUT_AVAILABLE; + return; + +err_display_disable: + exynos_dsi_set_display_enable(dsi, false); + drm_panel_unprepare(dsi->panel); + +err_put_sync: + dsi->state &= ~DSIM_STATE_ENABLED; + pm_runtime_put(dsi->dev); } static void exynos_dsi_disable(struct drm_encoder *encoder) @@ -1417,11 +1426,11 @@ static void exynos_dsi_disable(struct drm_encoder *encoder) dsi->state &= ~DSIM_STATE_VIDOUT_AVAILABLE; drm_panel_disable(dsi->panel); + drm_bridge_disable(dsi->out_bridge); exynos_dsi_set_display_enable(dsi, false); drm_panel_unprepare(dsi->panel); - + drm_bridge_post_disable(dsi->out_bridge); dsi->state &= ~DSIM_STATE_ENABLED; - pm_runtime_put_sync(dsi->dev); } @@ -1499,7 +1508,30 @@ static int exynos_dsi_host_attach(struct mipi_dsi_host *host, struct mipi_dsi_device *device) { struct exynos_dsi *dsi = host_to_dsi(host); - struct drm_device *drm = dsi->connector.dev; + struct drm_encoder *encoder = &dsi->encoder; + struct drm_device *drm = encoder->dev; + struct drm_bridge *out_bridge; + + out_bridge = of_drm_find_bridge(device->dev.of_node); + if (out_bridge) { + drm_bridge_attach(encoder, out_bridge, NULL); + dsi->out_bridge = out_bridge; + encoder->bridge = NULL; + } else { + int ret = exynos_dsi_create_connector(encoder); + + if (ret) { + DRM_ERROR("failed to create connector ret = %d\n", ret); + drm_encoder_cleanup(encoder); + return ret; + } + + dsi->panel = of_drm_find_panel(device->dev.of_node); + if (dsi->panel) { + drm_panel_attach(dsi->panel, &dsi->connector); + dsi->connector.status = connector_status_connected; + } + } /* * This is a temporary solution and should be made by more generic way. @@ -1518,14 +1550,6 @@ static int exynos_dsi_host_attach(struct mipi_dsi_host *host, dsi->lanes = device->lanes; dsi->format = device->format; dsi->mode_flags = device->mode_flags; - dsi->panel = of_drm_find_panel(device->dev.of_node); - if (IS_ERR(dsi->panel)) - dsi->panel = NULL; - - if (dsi->panel) { - drm_panel_attach(dsi->panel, &dsi->connector); - dsi->connector.status = connector_status_connected; - } exynos_drm_crtc_get_by_type(drm, EXYNOS_DISPLAY_TYPE_LCD)->i80_mode = !(dsi->mode_flags & MIPI_DSI_MODE_VIDEO); @@ -1541,19 +1565,21 @@ static int exynos_dsi_host_detach(struct mipi_dsi_host *host, struct mipi_dsi_device *device) { struct exynos_dsi *dsi = host_to_dsi(host); - struct drm_device *drm = dsi->connector.dev; - - mutex_lock(&drm->mode_config.mutex); + struct drm_device *drm = dsi->encoder.dev; if (dsi->panel) { + mutex_lock(&drm->mode_config.mutex); exynos_dsi_disable(&dsi->encoder); drm_panel_detach(dsi->panel); dsi->panel = NULL; dsi->connector.status = connector_status_disconnected; + mutex_unlock(&drm->mode_config.mutex); + } else { + if (dsi->out_bridge->funcs->detach) + dsi->out_bridge->funcs->detach(dsi->out_bridge); + dsi->out_bridge = NULL; } - mutex_unlock(&drm->mode_config.mutex); - if (drm->mode_config.poll_enabled) drm_kms_helper_hotplug_event(drm); @@ -1634,7 +1660,7 @@ static int exynos_dsi_parse_dt(struct exynos_dsi *dsi) if (ret < 0) return ret; - dsi->bridge_node = of_graph_get_remote_node(node, DSI_PORT_IN, 0); + dsi->in_bridge_node = of_graph_get_remote_node(node, DSI_PORT_IN, 0); return 0; } @@ -1645,7 +1671,7 @@ static int exynos_dsi_bind(struct device *dev, struct device *master, struct drm_encoder *encoder = dev_get_drvdata(dev); struct exynos_dsi *dsi = encoder_to_dsi(encoder); struct drm_device *drm_dev = data; - struct drm_bridge *bridge; + struct drm_bridge *in_bridge; int ret; drm_encoder_init(drm_dev, encoder, &exynos_dsi_encoder_funcs, @@ -1657,17 +1683,10 @@ static int exynos_dsi_bind(struct device *dev, struct device *master, if (ret < 0) return ret; - ret = exynos_dsi_create_connector(encoder); - if (ret) { - DRM_ERROR("failed to create connector ret = %d\n", ret); - drm_encoder_cleanup(encoder); - return ret; - } - - if (dsi->bridge_node) { - bridge = of_drm_find_bridge(dsi->bridge_node); - if (bridge) - drm_bridge_attach(encoder, bridge, NULL); + if (dsi->in_bridge_node) { + in_bridge = of_drm_find_bridge(dsi->in_bridge_node); + if (in_bridge) + drm_bridge_attach(encoder, in_bridge, NULL); } return mipi_dsi_host_register(&dsi->dsi_host); @@ -1786,7 +1805,7 @@ static int exynos_dsi_remove(struct platform_device *pdev) { struct exynos_dsi *dsi = platform_get_drvdata(pdev); - of_node_put(dsi->bridge_node); + of_node_put(dsi->in_bridge_node); pm_runtime_disable(&pdev->dev); diff --git a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c index 132dd52d0ac7..918dd2c82209 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_fbdev.c +++ b/drivers/gpu/drm/exynos/exynos_drm_fbdev.c @@ -270,20 +270,3 @@ void exynos_drm_fbdev_fini(struct drm_device *dev) private->fb_helper = NULL; } -void exynos_drm_fbdev_suspend(struct drm_device *dev) -{ - struct exynos_drm_private *private = dev->dev_private; - - console_lock(); - drm_fb_helper_set_suspend(private->fb_helper, 1); - console_unlock(); -} - -void exynos_drm_fbdev_resume(struct drm_device *dev) -{ - struct exynos_drm_private *private = dev->dev_private; - - console_lock(); - drm_fb_helper_set_suspend(private->fb_helper, 0); - console_unlock(); -} diff --git a/drivers/gpu/drm/exynos/exynos_drm_fbdev.h b/drivers/gpu/drm/exynos/exynos_drm_fbdev.h index b33847223a85..6840b6aadbc0 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_fbdev.h +++ b/drivers/gpu/drm/exynos/exynos_drm_fbdev.h @@ -19,8 +19,6 @@ int exynos_drm_fbdev_init(struct drm_device *dev); void exynos_drm_fbdev_fini(struct drm_device *dev); -void exynos_drm_fbdev_suspend(struct drm_device *drm); -void exynos_drm_fbdev_resume(struct drm_device *drm); #else @@ -39,14 +37,6 @@ static inline void exynos_drm_fbdev_restore_mode(struct drm_device *dev) #define exynos_drm_output_poll_changed (NULL) -static inline void exynos_drm_fbdev_suspend(struct drm_device *drm) -{ -} - -static inline void exynos_drm_fbdev_resume(struct drm_device *drm) -{ -} - #endif #endif diff --git a/drivers/gpu/drm/exynos/exynos_drm_gsc.c b/drivers/gpu/drm/exynos/exynos_drm_gsc.c index 7ba414b52faa..ce15d46bfce8 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_gsc.c +++ b/drivers/gpu/drm/exynos/exynos_drm_gsc.c @@ -448,7 +448,7 @@ static void gsc_handle_irq(struct gsc_context *ctx, bool enable, } -static void gsc_src_set_fmt(struct gsc_context *ctx, u32 fmt) +static void gsc_src_set_fmt(struct gsc_context *ctx, u32 fmt, bool tiled) { u32 cfg; @@ -514,6 +514,9 @@ static void gsc_src_set_fmt(struct gsc_context *ctx, u32 fmt) break; } + if (tiled) + cfg |= (GSC_IN_TILE_C_16x8 | GSC_IN_TILE_MODE); + gsc_write(cfg, GSC_IN_CON); } @@ -632,7 +635,7 @@ static void gsc_src_set_addr(struct gsc_context *ctx, u32 buf_id, gsc_src_set_buf_seq(ctx, buf_id, true); } -static void gsc_dst_set_fmt(struct gsc_context *ctx, u32 fmt) +static void gsc_dst_set_fmt(struct gsc_context *ctx, u32 fmt, bool tiled) { u32 cfg; @@ -698,6 +701,9 @@ static void gsc_dst_set_fmt(struct gsc_context *ctx, u32 fmt) break; } + if (tiled) + cfg |= (GSC_IN_TILE_C_16x8 | GSC_OUT_TILE_MODE); + gsc_write(cfg, GSC_OUT_CON); } @@ -1122,11 +1128,11 @@ static int gsc_commit(struct exynos_drm_ipp *ipp, return ret; } - gsc_src_set_fmt(ctx, task->src.buf.fourcc); + gsc_src_set_fmt(ctx, task->src.buf.fourcc, task->src.buf.modifier); gsc_src_set_transf(ctx, task->transform.rotation); gsc_src_set_size(ctx, &task->src); gsc_src_set_addr(ctx, 0, &task->src); - gsc_dst_set_fmt(ctx, task->dst.buf.fourcc); + gsc_dst_set_fmt(ctx, task->dst.buf.fourcc, task->dst.buf.modifier); gsc_dst_set_size(ctx, &task->dst); gsc_dst_set_addr(ctx, 0, &task->dst); gsc_set_prescaler(ctx, &ctx->sc, &task->src.rect, &task->dst.rect); @@ -1200,6 +1206,10 @@ static const unsigned int gsc_formats[] = { DRM_FORMAT_YUV420, DRM_FORMAT_YVU420, DRM_FORMAT_YUV422, }; +static const unsigned int gsc_tiled_formats[] = { + DRM_FORMAT_NV12, DRM_FORMAT_NV21, +}; + static int gsc_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -1207,23 +1217,24 @@ static int gsc_probe(struct platform_device *pdev) struct exynos_drm_ipp_formats *formats; struct gsc_context *ctx; struct resource *res; - int ret, i; + int num_formats, ret, i, j; ctx = devm_kzalloc(dev, sizeof(*ctx), GFP_KERNEL); if (!ctx) return -ENOMEM; - formats = devm_kcalloc(dev, - ARRAY_SIZE(gsc_formats), sizeof(*formats), - GFP_KERNEL); - if (!formats) - return -ENOMEM; - driver_data = (struct gsc_driverdata *)of_device_get_match_data(dev); ctx->dev = dev; ctx->num_clocks = driver_data->num_clocks; ctx->clk_names = driver_data->clk_names; + /* construct formats/limits array */ + num_formats = ARRAY_SIZE(gsc_formats) + ARRAY_SIZE(gsc_tiled_formats); + formats = devm_kcalloc(dev, num_formats, sizeof(*formats), GFP_KERNEL); + if (!formats) + return -ENOMEM; + + /* linear formats */ for (i = 0; i < ARRAY_SIZE(gsc_formats); i++) { formats[i].fourcc = gsc_formats[i]; formats[i].type = DRM_EXYNOS_IPP_FORMAT_SOURCE | @@ -1231,8 +1242,19 @@ static int gsc_probe(struct platform_device *pdev) formats[i].limits = driver_data->limits; formats[i].num_limits = driver_data->num_limits; } + + /* tiled formats */ + for (j = i, i = 0; i < ARRAY_SIZE(gsc_tiled_formats); j++, i++) { + formats[j].fourcc = gsc_tiled_formats[i]; + formats[j].modifier = DRM_FORMAT_MOD_SAMSUNG_16_16_TILE; + formats[j].type = DRM_EXYNOS_IPP_FORMAT_SOURCE | + DRM_EXYNOS_IPP_FORMAT_DESTINATION; + formats[j].limits = driver_data->limits; + formats[j].num_limits = driver_data->num_limits; + } + ctx->formats = formats; - ctx->num_formats = ARRAY_SIZE(gsc_formats); + ctx->num_formats = num_formats; /* clock control */ for (i = 0; i < ctx->num_clocks; i++) { diff --git a/drivers/gpu/drm/exynos/exynos_drm_plane.c b/drivers/gpu/drm/exynos/exynos_drm_plane.c index dba29aec59b4..df0508e0e49e 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_plane.c +++ b/drivers/gpu/drm/exynos/exynos_drm_plane.c @@ -131,16 +131,14 @@ static void exynos_drm_plane_reset(struct drm_plane *plane) if (plane->state) { exynos_state = to_exynos_plane_state(plane->state); - if (exynos_state->base.fb) - drm_framebuffer_put(exynos_state->base.fb); + __drm_atomic_helper_plane_destroy_state(plane->state); kfree(exynos_state); plane->state = NULL; } exynos_state = kzalloc(sizeof(*exynos_state), GFP_KERNEL); if (exynos_state) { - plane->state = &exynos_state->base; - plane->state->plane = plane; + __drm_atomic_helper_plane_reset(plane, &exynos_state->base); plane->state->zpos = exynos_plane->config->zpos; } } @@ -300,6 +298,10 @@ int exynos_plane_init(struct drm_device *dev, const struct exynos_drm_plane_config *config) { int err; + unsigned int supported_modes = BIT(DRM_MODE_BLEND_PIXEL_NONE) | + BIT(DRM_MODE_BLEND_PREMULTI) | + BIT(DRM_MODE_BLEND_COVERAGE); + struct drm_plane *plane = &exynos_plane->base; err = drm_universal_plane_init(dev, &exynos_plane->base, 1 << dev->mode_config.num_crtc, @@ -320,5 +322,11 @@ int exynos_plane_init(struct drm_device *dev, exynos_plane_attach_zpos_property(&exynos_plane->base, config->zpos, !(config->capabilities & EXYNOS_DRM_PLANE_CAP_ZPOS)); + if (config->capabilities & EXYNOS_DRM_PLANE_CAP_PIX_BLEND) + drm_plane_create_blend_mode_property(plane, supported_modes); + + if (config->capabilities & EXYNOS_DRM_PLANE_CAP_WIN_BLEND) + drm_plane_create_alpha_property(plane); + return 0; } diff --git a/drivers/gpu/drm/exynos/exynos_drm_scaler.c b/drivers/gpu/drm/exynos/exynos_drm_scaler.c index 0ddb6eec7b11..cd66774e817d 100644 --- a/drivers/gpu/drm/exynos/exynos_drm_scaler.c +++ b/drivers/gpu/drm/exynos/exynos_drm_scaler.c @@ -49,56 +49,46 @@ struct scaler_context { const struct scaler_data *scaler_data; }; -static u32 scaler_get_format(u32 drm_fmt) +struct scaler_format { + u32 drm_fmt; + u32 internal_fmt; + u32 chroma_tile_w; + u32 chroma_tile_h; +}; + +static const struct scaler_format scaler_formats[] = { + { DRM_FORMAT_NV12, SCALER_YUV420_2P_UV, 8, 8 }, + { DRM_FORMAT_NV21, SCALER_YUV420_2P_VU, 8, 8 }, + { DRM_FORMAT_YUV420, SCALER_YUV420_3P, 8, 8 }, + { DRM_FORMAT_YUYV, SCALER_YUV422_1P_YUYV, 16, 16 }, + { DRM_FORMAT_UYVY, SCALER_YUV422_1P_UYVY, 16, 16 }, + { DRM_FORMAT_YVYU, SCALER_YUV422_1P_YVYU, 16, 16 }, + { DRM_FORMAT_NV16, SCALER_YUV422_2P_UV, 8, 16 }, + { DRM_FORMAT_NV61, SCALER_YUV422_2P_VU, 8, 16 }, + { DRM_FORMAT_YUV422, SCALER_YUV422_3P, 8, 16 }, + { DRM_FORMAT_NV24, SCALER_YUV444_2P_UV, 16, 16 }, + { DRM_FORMAT_NV42, SCALER_YUV444_2P_VU, 16, 16 }, + { DRM_FORMAT_YUV444, SCALER_YUV444_3P, 16, 16 }, + { DRM_FORMAT_RGB565, SCALER_RGB_565, 0, 0 }, + { DRM_FORMAT_XRGB1555, SCALER_ARGB1555, 0, 0 }, + { DRM_FORMAT_ARGB1555, SCALER_ARGB1555, 0, 0 }, + { DRM_FORMAT_XRGB4444, SCALER_ARGB4444, 0, 0 }, + { DRM_FORMAT_ARGB4444, SCALER_ARGB4444, 0, 0 }, + { DRM_FORMAT_XRGB8888, SCALER_ARGB8888, 0, 0 }, + { DRM_FORMAT_ARGB8888, SCALER_ARGB8888, 0, 0 }, + { DRM_FORMAT_RGBX8888, SCALER_RGBA8888, 0, 0 }, + { DRM_FORMAT_RGBA8888, SCALER_RGBA8888, 0, 0 }, +}; + +static const struct scaler_format *scaler_get_format(u32 drm_fmt) { - switch (drm_fmt) { - case DRM_FORMAT_NV12: - return SCALER_YUV420_2P_UV; - case DRM_FORMAT_NV21: - return SCALER_YUV420_2P_VU; - case DRM_FORMAT_YUV420: - return SCALER_YUV420_3P; - case DRM_FORMAT_YUYV: - return SCALER_YUV422_1P_YUYV; - case DRM_FORMAT_UYVY: - return SCALER_YUV422_1P_UYVY; - case DRM_FORMAT_YVYU: - return SCALER_YUV422_1P_YVYU; - case DRM_FORMAT_NV16: - return SCALER_YUV422_2P_UV; - case DRM_FORMAT_NV61: - return SCALER_YUV422_2P_VU; - case DRM_FORMAT_YUV422: - return SCALER_YUV422_3P; - case DRM_FORMAT_NV24: - return SCALER_YUV444_2P_UV; - case DRM_FORMAT_NV42: - return SCALER_YUV444_2P_VU; - case DRM_FORMAT_YUV444: - return SCALER_YUV444_3P; - case DRM_FORMAT_RGB565: - return SCALER_RGB_565; - case DRM_FORMAT_XRGB1555: - return SCALER_ARGB1555; - case DRM_FORMAT_ARGB1555: - return SCALER_ARGB1555; - case DRM_FORMAT_XRGB4444: - return SCALER_ARGB4444; - case DRM_FORMAT_ARGB4444: - return SCALER_ARGB4444; - case DRM_FORMAT_XRGB8888: - return SCALER_ARGB8888; - case DRM_FORMAT_ARGB8888: - return SCALER_ARGB8888; - case DRM_FORMAT_RGBX8888: - return SCALER_RGBA8888; - case DRM_FORMAT_RGBA8888: - return SCALER_RGBA8888; - default: - break; - } + int i; - return 0; + for (i = 0; i < ARRAY_SIZE(scaler_formats); i++) + if (scaler_formats[i].drm_fmt == drm_fmt) + return &scaler_formats[i]; + + return NULL; } static inline int scaler_reset(struct scaler_context *scaler) @@ -152,11 +142,11 @@ static inline void scaler_enable_int(struct scaler_context *scaler) } static inline void scaler_set_src_fmt(struct scaler_context *scaler, - u32 src_fmt) + u32 src_fmt, u32 tile) { u32 val; - val = SCALER_SRC_CFG_SET_COLOR_FORMAT(src_fmt); + val = SCALER_SRC_CFG_SET_COLOR_FORMAT(src_fmt) | (tile << 10); scaler_write(val, SCALER_SRC_CFG); } @@ -188,15 +178,20 @@ static inline void scaler_set_src_span(struct scaler_context *scaler, scaler_write(val, SCALER_SRC_SPAN); } -static inline void scaler_set_src_luma_pos(struct scaler_context *scaler, - struct drm_exynos_ipp_task_rect *src_pos) +static inline void scaler_set_src_luma_chroma_pos(struct scaler_context *scaler, + struct drm_exynos_ipp_task_rect *src_pos, + const struct scaler_format *fmt) { u32 val; val = SCALER_SRC_Y_POS_SET_YH_POS(src_pos->x << 2); val |= SCALER_SRC_Y_POS_SET_YV_POS(src_pos->y << 2); scaler_write(val, SCALER_SRC_Y_POS); - scaler_write(val, SCALER_SRC_C_POS); /* ATTENTION! */ + val = SCALER_SRC_C_POS_SET_CH_POS( + (src_pos->x * fmt->chroma_tile_w / 16) << 2); + val |= SCALER_SRC_C_POS_SET_CV_POS( + (src_pos->y * fmt->chroma_tile_h / 16) << 2); + scaler_write(val, SCALER_SRC_C_POS); } static inline void scaler_set_src_wh(struct scaler_context *scaler, @@ -366,11 +361,12 @@ static int scaler_commit(struct exynos_drm_ipp *ipp, struct scaler_context *scaler = container_of(ipp, struct scaler_context, ipp); - u32 src_fmt = scaler_get_format(task->src.buf.fourcc); struct drm_exynos_ipp_task_rect *src_pos = &task->src.rect; - - u32 dst_fmt = scaler_get_format(task->dst.buf.fourcc); struct drm_exynos_ipp_task_rect *dst_pos = &task->dst.rect; + const struct scaler_format *src_fmt, *dst_fmt; + + src_fmt = scaler_get_format(task->src.buf.fourcc); + dst_fmt = scaler_get_format(task->dst.buf.fourcc); pm_runtime_get_sync(scaler->dev); if (scaler_reset(scaler)) { @@ -380,13 +376,14 @@ static int scaler_commit(struct exynos_drm_ipp *ipp, scaler->task = task; - scaler_set_src_fmt(scaler, src_fmt); + scaler_set_src_fmt( + scaler, src_fmt->internal_fmt, task->src.buf.modifier != 0); scaler_set_src_base(scaler, &task->src); scaler_set_src_span(scaler, &task->src); - scaler_set_src_luma_pos(scaler, src_pos); + scaler_set_src_luma_chroma_pos(scaler, src_pos, src_fmt); scaler_set_src_wh(scaler, src_pos); - scaler_set_dst_fmt(scaler, dst_fmt); + scaler_set_dst_fmt(scaler, dst_fmt->internal_fmt); scaler_set_dst_base(scaler, &task->dst); scaler_set_dst_span(scaler, &task->dst); scaler_set_dst_luma_pos(scaler, dst_pos); @@ -617,6 +614,16 @@ static const struct drm_exynos_ipp_limit scaler_5420_one_pixel_limits[] = { .v = { 65536 * 1 / 4, 65536 * 16 }) }, }; +static const struct drm_exynos_ipp_limit scaler_5420_tile_limits[] = { + { IPP_SIZE_LIMIT(BUFFER, .h = { 16, SZ_8K }, .v = { 16, SZ_8K })}, + { IPP_SIZE_LIMIT(AREA, .h.align = 16, .v.align = 16) }, + { IPP_SCALE_LIMIT(.h = {1, 1}, .v = {1, 1})}, + { } +}; + +#define IPP_SRCDST_TILE_FORMAT(f, l) \ + IPP_SRCDST_MFORMAT(f, DRM_FORMAT_MOD_SAMSUNG_16_16_TILE, (l)) + static const struct exynos_drm_ipp_formats exynos5420_formats[] = { /* SCALER_YUV420_2P_UV */ { IPP_SRCDST_FORMAT(NV21, scaler_5420_two_pixel_hv_limits) }, @@ -680,6 +687,18 @@ static const struct exynos_drm_ipp_formats exynos5420_formats[] = { /* SCALER_RGBA8888 */ { IPP_SRCDST_FORMAT(RGBA8888, scaler_5420_one_pixel_limits) }, + + /* SCALER_YUV420_2P_UV TILE */ + { IPP_SRCDST_TILE_FORMAT(NV21, scaler_5420_tile_limits) }, + + /* SCALER_YUV420_2P_VU TILE */ + { IPP_SRCDST_TILE_FORMAT(NV12, scaler_5420_tile_limits) }, + + /* SCALER_YUV420_3P TILE */ + { IPP_SRCDST_TILE_FORMAT(YUV420, scaler_5420_tile_limits) }, + + /* SCALER_YUV422_1P_YUYV TILE */ + { IPP_SRCDST_TILE_FORMAT(YUYV, scaler_5420_tile_limits) }, }; static const struct scaler_data exynos5420_data = { diff --git a/drivers/gpu/drm/exynos/exynos_mixer.c b/drivers/gpu/drm/exynos/exynos_mixer.c index ffbf4a950f69..e3a4ecbc503b 100644 --- a/drivers/gpu/drm/exynos/exynos_mixer.c +++ b/drivers/gpu/drm/exynos/exynos_mixer.c @@ -131,14 +131,18 @@ static const struct exynos_drm_plane_config plane_configs[MIXER_WIN_NR] = { .pixel_formats = mixer_formats, .num_pixel_formats = ARRAY_SIZE(mixer_formats), .capabilities = EXYNOS_DRM_PLANE_CAP_DOUBLE | - EXYNOS_DRM_PLANE_CAP_ZPOS, + EXYNOS_DRM_PLANE_CAP_ZPOS | + EXYNOS_DRM_PLANE_CAP_PIX_BLEND | + EXYNOS_DRM_PLANE_CAP_WIN_BLEND, }, { .zpos = 1, .type = DRM_PLANE_TYPE_CURSOR, .pixel_formats = mixer_formats, .num_pixel_formats = ARRAY_SIZE(mixer_formats), .capabilities = EXYNOS_DRM_PLANE_CAP_DOUBLE | - EXYNOS_DRM_PLANE_CAP_ZPOS, + EXYNOS_DRM_PLANE_CAP_ZPOS | + EXYNOS_DRM_PLANE_CAP_PIX_BLEND | + EXYNOS_DRM_PLANE_CAP_WIN_BLEND, }, { .zpos = 2, .type = DRM_PLANE_TYPE_OVERLAY, @@ -146,7 +150,8 @@ static const struct exynos_drm_plane_config plane_configs[MIXER_WIN_NR] = { .num_pixel_formats = ARRAY_SIZE(vp_formats), .capabilities = EXYNOS_DRM_PLANE_CAP_SCALE | EXYNOS_DRM_PLANE_CAP_ZPOS | - EXYNOS_DRM_PLANE_CAP_TILE, + EXYNOS_DRM_PLANE_CAP_TILE | + EXYNOS_DRM_PLANE_CAP_WIN_BLEND, }, }; @@ -309,31 +314,42 @@ static void vp_default_filter(struct mixer_context *ctx) } static void mixer_cfg_gfx_blend(struct mixer_context *ctx, unsigned int win, - bool alpha) + unsigned int pixel_alpha, unsigned int alpha) { + u32 win_alpha = alpha >> 8; u32 val; val = MXR_GRP_CFG_COLOR_KEY_DISABLE; /* no blank key */ - if (alpha) { - /* blending based on pixel alpha */ + switch (pixel_alpha) { + case DRM_MODE_BLEND_PIXEL_NONE: + break; + case DRM_MODE_BLEND_COVERAGE: + val |= MXR_GRP_CFG_PIXEL_BLEND_EN; + break; + case DRM_MODE_BLEND_PREMULTI: + default: val |= MXR_GRP_CFG_BLEND_PRE_MUL; val |= MXR_GRP_CFG_PIXEL_BLEND_EN; + break; + } + + if (alpha != DRM_BLEND_ALPHA_OPAQUE) { + val |= MXR_GRP_CFG_WIN_BLEND_EN; + val |= win_alpha; } mixer_reg_writemask(ctx, MXR_GRAPHIC_CFG(win), val, MXR_GRP_CFG_MISC_MASK); } -static void mixer_cfg_vp_blend(struct mixer_context *ctx) +static void mixer_cfg_vp_blend(struct mixer_context *ctx, unsigned int alpha) { - u32 val; + u32 win_alpha = alpha >> 8; + u32 val = 0; - /* - * No blending at the moment since the NV12/NV21 pixelformats don't - * have an alpha channel. However the mixer supports a global alpha - * value for a layer. Once this functionality is exposed, we can - * support blending of the video layer through this. - */ - val = 0; + if (alpha != DRM_BLEND_ALPHA_OPAQUE) { + val |= MXR_VID_CFG_BLEND_EN; + val |= win_alpha; + } mixer_reg_write(ctx, MXR_VIDEO_CFG, val); } @@ -529,7 +545,7 @@ static void vp_video_buffer(struct mixer_context *ctx, vp_reg_write(ctx, VP_BOT_C_PTR, chroma_addr[1]); mixer_cfg_layer(ctx, plane->index, priority, true); - mixer_cfg_vp_blend(ctx); + mixer_cfg_vp_blend(ctx, state->base.alpha); spin_unlock_irqrestore(&ctx->reg_slock, flags); @@ -553,10 +569,16 @@ static void mixer_graph_buffer(struct mixer_context *ctx, unsigned int win = plane->index; unsigned int x_ratio = 0, y_ratio = 0; unsigned int dst_x_offset, dst_y_offset; + unsigned int pixel_alpha; dma_addr_t dma_addr; unsigned int fmt; u32 val; + if (fb->format->has_alpha) + pixel_alpha = state->base.pixel_blend_mode; + else + pixel_alpha = DRM_MODE_BLEND_PIXEL_NONE; + switch (fb->format->format) { case DRM_FORMAT_XRGB4444: case DRM_FORMAT_ARGB4444: @@ -616,7 +638,7 @@ static void mixer_graph_buffer(struct mixer_context *ctx, mixer_reg_write(ctx, MXR_GRAPHIC_BASE(win), dma_addr); mixer_cfg_layer(ctx, win, priority, true); - mixer_cfg_gfx_blend(ctx, win, fb->format->has_alpha); + mixer_cfg_gfx_blend(ctx, win, pixel_alpha, state->base.alpha); /* layer update mandatory for mixer 16.0.33.0 */ if (ctx->mxr_ver == MXR_VER_16_0_33_0 || diff --git a/drivers/gpu/drm/exynos/regs-mixer.h b/drivers/gpu/drm/exynos/regs-mixer.h index 189cfa2470a8..d2b8194a07bf 100644 --- a/drivers/gpu/drm/exynos/regs-mixer.h +++ b/drivers/gpu/drm/exynos/regs-mixer.h @@ -109,12 +109,15 @@ #define MXR_CFG_SCAN_HD (1 << 0) #define MXR_CFG_SCAN_MASK 0x47 +/* bits for MXR_VIDEO_CFG */ +#define MXR_VID_CFG_BLEND_EN (1 << 16) + /* bits for MXR_GRAPHICn_CFG */ #define MXR_GRP_CFG_COLOR_KEY_DISABLE (1 << 21) #define MXR_GRP_CFG_BLEND_PRE_MUL (1 << 20) #define MXR_GRP_CFG_WIN_BLEND_EN (1 << 17) #define MXR_GRP_CFG_PIXEL_BLEND_EN (1 << 16) -#define MXR_GRP_CFG_MISC_MASK ((3 << 16) | (3 << 20)) +#define MXR_GRP_CFG_MISC_MASK ((3 << 16) | (3 << 20) | 0xff) #define MXR_GRP_CFG_FORMAT_VAL(x) MXR_MASK_VAL(x, 11, 8) #define MXR_GRP_CFG_FORMAT_MASK MXR_GRP_CFG_FORMAT_VAL(~0) #define MXR_GRP_CFG_ALPHA_VAL(x) MXR_MASK_VAL(x, 7, 0) diff --git a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c index 80232321a244..0496be5212e1 100644 --- a/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c +++ b/drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_drv.c @@ -353,12 +353,12 @@ static int fsl_dcu_drm_probe(struct platform_device *pdev) ret = drm_dev_register(drm, 0); if (ret < 0) - goto unref; + goto put; return 0; -unref: - drm_dev_unref(drm); +put: + drm_dev_put(drm); unregister_pix_clk: clk_unregister(fsl_dev->pix_clk); disable_clk: @@ -371,7 +371,7 @@ static int fsl_dcu_drm_remove(struct platform_device *pdev) struct fsl_dcu_drm_device *fsl_dev = platform_get_drvdata(pdev); drm_dev_unregister(fsl_dev->drm); - drm_dev_unref(fsl_dev->drm); + drm_dev_put(fsl_dev->drm); clk_disable_unprepare(fsl_dev->clk); clk_unregister(fsl_dev->pix_clk); diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c index d4f6f1f9df5b..68c0c297b3a5 100644 --- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c +++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c @@ -37,7 +37,7 @@ static const struct file_operations hibmc_fops = { .llseek = no_llseek, }; -irqreturn_t hibmc_drm_interrupt(int irq, void *arg) +static irqreturn_t hibmc_drm_interrupt(int irq, void *arg) { struct drm_device *dev = (struct drm_device *)arg; struct hibmc_drm_private *priv = @@ -74,30 +74,16 @@ static int __maybe_unused hibmc_pm_suspend(struct device *dev) { struct pci_dev *pdev = to_pci_dev(dev); struct drm_device *drm_dev = pci_get_drvdata(pdev); - struct hibmc_drm_private *priv = drm_dev->dev_private; - - drm_kms_helper_poll_disable(drm_dev); - priv->suspend_state = drm_atomic_helper_suspend(drm_dev); - if (IS_ERR(priv->suspend_state)) { - DRM_ERROR("drm_atomic_helper_suspend failed: %ld\n", - PTR_ERR(priv->suspend_state)); - drm_kms_helper_poll_enable(drm_dev); - return PTR_ERR(priv->suspend_state); - } - return 0; + return drm_mode_config_helper_suspend(drm_dev); } static int __maybe_unused hibmc_pm_resume(struct device *dev) { struct pci_dev *pdev = to_pci_dev(dev); struct drm_device *drm_dev = pci_get_drvdata(pdev); - struct hibmc_drm_private *priv = drm_dev->dev_private; - drm_atomic_helper_resume(drm_dev, priv->suspend_state); - drm_kms_helper_poll_enable(drm_dev); - - return 0; + return drm_mode_config_helper_resume(drm_dev); } static const struct dev_pm_ops hibmc_pm_ops = { @@ -387,7 +373,7 @@ err_unload: err_disable: pci_disable_device(pdev); err_free: - drm_dev_unref(dev); + drm_dev_put(dev); return ret; } @@ -398,11 +384,11 @@ static void hibmc_pci_remove(struct pci_dev *pdev) drm_dev_unregister(dev); hibmc_unload(dev); - drm_dev_unref(dev); + drm_dev_put(dev); } static struct pci_device_id hibmc_pci_table[] = { - {0x19e5, 0x1711, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0}, + { PCI_VDEVICE(HUAWEI, 0x1711) }, {0,} }; diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h index e195521eb41e..45c25a488f42 100644 --- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h +++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.h @@ -47,7 +47,6 @@ struct hibmc_drm_private { /* drm */ struct drm_device *dev; bool mode_config_initialized; - struct drm_atomic_state *suspend_state; /* ttm */ struct drm_global_reference mem_global_ref; diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_fbdev.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_fbdev.c index b92595c477ef..edcca1761500 100644 --- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_fbdev.c +++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_fbdev.c @@ -71,7 +71,6 @@ static int hibmc_drm_fb_create(struct drm_fb_helper *helper, DRM_DEBUG_DRIVER("surface width(%d), height(%d) and bpp(%d)\n", sizes->surface_width, sizes->surface_height, sizes->surface_bpp); - sizes->surface_depth = 32; bytes_per_pixel = DIV_ROUND_UP(sizes->surface_bpp, 8); @@ -122,6 +121,7 @@ static int hibmc_drm_fb_create(struct drm_fb_helper *helper, hi_fbdev->fb = hibmc_framebuffer_init(priv->dev, &mode_cmd, gobj); if (IS_ERR(hi_fbdev->fb)) { ret = PTR_ERR(hi_fbdev->fb); + hi_fbdev->fb = NULL; DRM_ERROR("failed to initialize framebuffer: %d\n", ret); goto out_release_fbi; } diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c index 4871025f7573..2e3e0bdb8932 100644 --- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c +++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_ttm.c @@ -283,7 +283,7 @@ static void hibmc_bo_unref(struct hibmc_bo **bo) return; tbo = &((*bo)->bo); - ttm_bo_unref(&tbo); + ttm_bo_put(tbo); *bo = NULL; } diff --git a/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c index ddb0403f1975..e6a62d5a00a3 100644 --- a/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c +++ b/drivers/gpu/drm/hisilicon/kirin/kirin_drm_drv.c @@ -193,7 +193,7 @@ static int kirin_drm_bind(struct device *dev) ret = kirin_drm_kms_init(drm_dev); if (ret) - goto err_drm_dev_unref; + goto err_drm_dev_put; ret = drm_dev_register(drm_dev, 0); if (ret) @@ -203,8 +203,8 @@ static int kirin_drm_bind(struct device *dev) err_kms_cleanup: kirin_drm_kms_cleanup(drm_dev); -err_drm_dev_unref: - drm_dev_unref(drm_dev); +err_drm_dev_put: + drm_dev_put(drm_dev); return ret; } @@ -215,7 +215,7 @@ static void kirin_drm_unbind(struct device *dev) drm_dev_unregister(drm_dev); kirin_drm_kms_cleanup(drm_dev); - drm_dev_unref(drm_dev); + drm_dev_put(drm_dev); } static const struct component_master_ops kirin_drm_ops = { diff --git a/drivers/gpu/drm/i915/gvt/dmabuf.c b/drivers/gpu/drm/i915/gvt/dmabuf.c index 6e3f56684f4e..51ed99a37803 100644 --- a/drivers/gpu/drm/i915/gvt/dmabuf.c +++ b/drivers/gpu/drm/i915/gvt/dmabuf.c @@ -170,20 +170,22 @@ static struct drm_i915_gem_object *vgpu_create_gem(struct drm_device *dev, unsigned int tiling_mode = 0; unsigned int stride = 0; - switch (info->drm_format_mod << 10) { - case PLANE_CTL_TILED_LINEAR: + switch (info->drm_format_mod) { + case DRM_FORMAT_MOD_LINEAR: tiling_mode = I915_TILING_NONE; break; - case PLANE_CTL_TILED_X: + case I915_FORMAT_MOD_X_TILED: tiling_mode = I915_TILING_X; stride = info->stride; break; - case PLANE_CTL_TILED_Y: + case I915_FORMAT_MOD_Y_TILED: + case I915_FORMAT_MOD_Yf_TILED: tiling_mode = I915_TILING_Y; stride = info->stride; break; default: - gvt_dbg_core("not supported tiling mode\n"); + gvt_dbg_core("invalid drm_format_mod %llx for tiling\n", + info->drm_format_mod); } obj->tiling_and_stride = tiling_mode | stride; } else { @@ -222,9 +224,26 @@ static int vgpu_get_plane_info(struct drm_device *dev, info->height = p.height; info->stride = p.stride; info->drm_format = p.drm_format; - info->drm_format_mod = p.tiled; + + switch (p.tiled) { + case PLANE_CTL_TILED_LINEAR: + info->drm_format_mod = DRM_FORMAT_MOD_LINEAR; + break; + case PLANE_CTL_TILED_X: + info->drm_format_mod = I915_FORMAT_MOD_X_TILED; + break; + case PLANE_CTL_TILED_Y: + info->drm_format_mod = I915_FORMAT_MOD_Y_TILED; + break; + case PLANE_CTL_TILED_YF: + info->drm_format_mod = I915_FORMAT_MOD_Yf_TILED; + break; + default: + gvt_vgpu_err("invalid tiling mode: %x\n", p.tiled); + } + info->size = (((p.stride * p.height * p.bpp) / 8) + - (PAGE_SIZE - 1)) >> PAGE_SHIFT; + (PAGE_SIZE - 1)) >> PAGE_SHIFT; } else if (plane_id == DRM_PLANE_TYPE_CURSOR) { ret = intel_vgpu_decode_cursor_plane(vgpu, &c); if (ret) diff --git a/drivers/gpu/drm/i915/gvt/fb_decoder.c b/drivers/gpu/drm/i915/gvt/fb_decoder.c index face664be3e8..481896fb712a 100644 --- a/drivers/gpu/drm/i915/gvt/fb_decoder.c +++ b/drivers/gpu/drm/i915/gvt/fb_decoder.c @@ -220,8 +220,7 @@ int intel_vgpu_decode_primary_plane(struct intel_vgpu *vgpu, if (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv) || IS_BROXTON(dev_priv)) { - plane->tiled = (val & PLANE_CTL_TILED_MASK) >> - _PLANE_CTL_TILED_SHIFT; + plane->tiled = val & PLANE_CTL_TILED_MASK; fmt = skl_format_to_drm( val & PLANE_CTL_FORMAT_MASK, val & PLANE_CTL_ORDER_RGBX, @@ -260,7 +259,7 @@ int intel_vgpu_decode_primary_plane(struct intel_vgpu *vgpu, return -EINVAL; } - plane->stride = intel_vgpu_get_stride(vgpu, pipe, (plane->tiled << 10), + plane->stride = intel_vgpu_get_stride(vgpu, pipe, plane->tiled, (IS_SKYLAKE(dev_priv) || IS_KABYLAKE(dev_priv) || IS_BROXTON(dev_priv)) ? diff --git a/drivers/gpu/drm/i915/gvt/fb_decoder.h b/drivers/gpu/drm/i915/gvt/fb_decoder.h index cb055f3c81a2..60c155085029 100644 --- a/drivers/gpu/drm/i915/gvt/fb_decoder.h +++ b/drivers/gpu/drm/i915/gvt/fb_decoder.h @@ -101,7 +101,7 @@ struct intel_gvt; /* color space conversion and gamma correction are not included */ struct intel_vgpu_primary_plane_format { u8 enabled; /* plane is enabled */ - u8 tiled; /* X-tiled */ + u32 tiled; /* tiling mode: linear, X-tiled, Y tiled, etc */ u8 bpp; /* bits per pixel */ u32 hw_format; /* format field in the PRI_CTL register */ u32 drm_format; /* format in DRM definition */ diff --git a/drivers/gpu/drm/i915/gvt/handlers.c b/drivers/gpu/drm/i915/gvt/handlers.c index d26258786e3f..90f50f67909a 100644 --- a/drivers/gpu/drm/i915/gvt/handlers.c +++ b/drivers/gpu/drm/i915/gvt/handlers.c @@ -1297,6 +1297,19 @@ static int power_well_ctl_mmio_write(struct intel_vgpu *vgpu, return 0; } +static int gen9_dbuf_ctl_mmio_write(struct intel_vgpu *vgpu, + unsigned int offset, void *p_data, unsigned int bytes) +{ + write_vreg(vgpu, offset, p_data, bytes); + + if (vgpu_vreg(vgpu, offset) & DBUF_POWER_REQUEST) + vgpu_vreg(vgpu, offset) |= DBUF_POWER_STATE; + else + vgpu_vreg(vgpu, offset) &= ~DBUF_POWER_STATE; + + return 0; +} + static int fpga_dbg_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, void *p_data, unsigned int bytes) { @@ -1526,9 +1539,15 @@ static int bxt_phy_ctl_family_write(struct intel_vgpu *vgpu, u32 v = *(u32 *)p_data; u32 data = v & COMMON_RESET_DIS ? BXT_PHY_LANE_ENABLED : 0; - vgpu_vreg(vgpu, _BXT_PHY_CTL_DDI_A) = data; - vgpu_vreg(vgpu, _BXT_PHY_CTL_DDI_B) = data; - vgpu_vreg(vgpu, _BXT_PHY_CTL_DDI_C) = data; + switch (offset) { + case _PHY_CTL_FAMILY_EDP: + vgpu_vreg(vgpu, _BXT_PHY_CTL_DDI_A) = data; + break; + case _PHY_CTL_FAMILY_DDI: + vgpu_vreg(vgpu, _BXT_PHY_CTL_DDI_B) = data; + vgpu_vreg(vgpu, _BXT_PHY_CTL_DDI_C) = data; + break; + } vgpu_vreg(vgpu, offset) = v; @@ -2801,6 +2820,8 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) MMIO_D(HSW_PWR_WELL_CTL1, D_SKL_PLUS); MMIO_DH(HSW_PWR_WELL_CTL2, D_SKL_PLUS, NULL, skl_power_well_ctl_write); + MMIO_DH(DBUF_CTL, D_SKL_PLUS, NULL, gen9_dbuf_ctl_mmio_write); + MMIO_D(_MMIO(0xa210), D_SKL_PLUS); MMIO_D(GEN9_MEDIA_PG_IDLE_HYSTERESIS, D_SKL_PLUS); MMIO_D(GEN9_RENDER_PG_IDLE_HYSTERESIS, D_SKL_PLUS); @@ -2976,8 +2997,6 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) NULL, gen9_trtte_write); MMIO_DH(_MMIO(0x4dfc), D_SKL_PLUS, NULL, gen9_trtt_chicken_write); - MMIO_D(_MMIO(0x45008), D_SKL_PLUS); - MMIO_D(_MMIO(0x46430), D_SKL_PLUS); MMIO_D(_MMIO(0x46520), D_SKL_PLUS); @@ -3014,7 +3033,9 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) MMIO_D(_MMIO(0x44500), D_SKL_PLUS); MMIO_DFH(GEN9_CSFE_CHICKEN1_RCS, D_SKL_PLUS, F_CMD_ACCESS, NULL, NULL); MMIO_DFH(GEN8_HDC_CHICKEN1, D_SKL_PLUS, F_MODE_MASK | F_CMD_ACCESS, - NULL, NULL); + NULL, NULL); + MMIO_DFH(GEN9_WM_CHICKEN3, D_SKL_PLUS, F_MODE_MASK | F_CMD_ACCESS, + NULL, NULL); MMIO_D(_MMIO(0x4ab8), D_KBL); MMIO_D(_MMIO(0x2248), D_KBL | D_SKL); @@ -3178,6 +3199,7 @@ static int init_bxt_mmio_info(struct intel_gvt *gvt) MMIO_D(BXT_DSI_PLL_ENABLE, D_BXT); MMIO_D(GEN9_CLKGATE_DIS_0, D_BXT); + MMIO_D(GEN9_CLKGATE_DIS_4, D_BXT); MMIO_D(HSW_TVIDEO_DIP_GCP(TRANSCODER_A), D_BXT); MMIO_D(HSW_TVIDEO_DIP_GCP(TRANSCODER_B), D_BXT); diff --git a/drivers/gpu/drm/i915/gvt/kvmgt.c b/drivers/gpu/drm/i915/gvt/kvmgt.c index 71751be329e3..c1072143da1d 100644 --- a/drivers/gpu/drm/i915/gvt/kvmgt.c +++ b/drivers/gpu/drm/i915/gvt/kvmgt.c @@ -32,6 +32,7 @@ #include <linux/device.h> #include <linux/mm.h> #include <linux/mmu_context.h> +#include <linux/sched/mm.h> #include <linux/types.h> #include <linux/list.h> #include <linux/rbtree.h> @@ -1792,16 +1793,21 @@ static int kvmgt_rw_gpa(unsigned long handle, unsigned long gpa, info = (struct kvmgt_guest_info *)handle; kvm = info->kvm; - if (kthread) + if (kthread) { + if (!mmget_not_zero(kvm->mm)) + return -EFAULT; use_mm(kvm->mm); + } idx = srcu_read_lock(&kvm->srcu); ret = write ? kvm_write_guest(kvm, gpa, buf, len) : kvm_read_guest(kvm, gpa, buf, len); srcu_read_unlock(&kvm->srcu, idx); - if (kthread) + if (kthread) { unuse_mm(kvm->mm); + mmput(kvm->mm); + } return ret; } @@ -1827,6 +1833,8 @@ static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn) { struct kvmgt_guest_info *info; struct kvm *kvm; + int idx; + bool ret; if (!handle_valid(handle)) return false; @@ -1834,8 +1842,11 @@ static bool kvmgt_is_valid_gfn(unsigned long handle, unsigned long gfn) info = (struct kvmgt_guest_info *)handle; kvm = info->kvm; - return kvm_is_visible_gfn(kvm, gfn); + idx = srcu_read_lock(&kvm->srcu); + ret = kvm_is_visible_gfn(kvm, gfn); + srcu_read_unlock(&kvm->srcu, idx); + return ret; } struct intel_gvt_mpt kvmgt_mpt = { diff --git a/drivers/gpu/drm/i915/gvt/mmio.c b/drivers/gpu/drm/i915/gvt/mmio.c index 4db817c21ed8..43f65848ecd6 100644 --- a/drivers/gpu/drm/i915/gvt/mmio.c +++ b/drivers/gpu/drm/i915/gvt/mmio.c @@ -245,6 +245,34 @@ void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu, bool dmlr) /* set the bit 0:2(Core C-State ) to C0 */ vgpu_vreg_t(vgpu, GEN6_GT_CORE_STATUS) = 0; + + if (IS_BROXTON(vgpu->gvt->dev_priv)) { + vgpu_vreg_t(vgpu, BXT_P_CR_GT_DISP_PWRON) &= + ~(BIT(0) | BIT(1)); + vgpu_vreg_t(vgpu, BXT_PORT_CL1CM_DW0(DPIO_PHY0)) &= + ~PHY_POWER_GOOD; + vgpu_vreg_t(vgpu, BXT_PORT_CL1CM_DW0(DPIO_PHY1)) &= + ~PHY_POWER_GOOD; + vgpu_vreg_t(vgpu, BXT_PHY_CTL_FAMILY(DPIO_PHY0)) &= + ~BIT(30); + vgpu_vreg_t(vgpu, BXT_PHY_CTL_FAMILY(DPIO_PHY1)) &= + ~BIT(30); + vgpu_vreg_t(vgpu, BXT_PHY_CTL(PORT_A)) &= + ~BXT_PHY_LANE_ENABLED; + vgpu_vreg_t(vgpu, BXT_PHY_CTL(PORT_A)) |= + BXT_PHY_CMNLANE_POWERDOWN_ACK | + BXT_PHY_LANE_POWERDOWN_ACK; + vgpu_vreg_t(vgpu, BXT_PHY_CTL(PORT_B)) &= + ~BXT_PHY_LANE_ENABLED; + vgpu_vreg_t(vgpu, BXT_PHY_CTL(PORT_B)) |= + BXT_PHY_CMNLANE_POWERDOWN_ACK | + BXT_PHY_LANE_POWERDOWN_ACK; + vgpu_vreg_t(vgpu, BXT_PHY_CTL(PORT_C)) &= + ~BXT_PHY_LANE_ENABLED; + vgpu_vreg_t(vgpu, BXT_PHY_CTL(PORT_C)) |= + BXT_PHY_CMNLANE_POWERDOWN_ACK | + BXT_PHY_LANE_POWERDOWN_ACK; + } } else { #define GVT_GEN8_MMIO_RESET_OFFSET (0x44200) /* only reset the engine related, so starting with 0x44200 diff --git a/drivers/gpu/drm/i915/gvt/mmio_context.c b/drivers/gpu/drm/i915/gvt/mmio_context.c index 7e702c6a32af..10e63eea5492 100644 --- a/drivers/gpu/drm/i915/gvt/mmio_context.c +++ b/drivers/gpu/drm/i915/gvt/mmio_context.c @@ -549,11 +549,9 @@ void intel_gvt_switch_mmio(struct intel_vgpu *pre, * performace for batch mmio read/write, so we need * handle forcewake mannually. */ - intel_runtime_pm_get(dev_priv); intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL); switch_mmio(pre, next, ring_id); intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL); - intel_runtime_pm_put(dev_priv); } /** diff --git a/drivers/gpu/drm/i915/gvt/opregion.c b/drivers/gpu/drm/i915/gvt/opregion.c index 82586c8e434f..276db53f1bf1 100644 --- a/drivers/gpu/drm/i915/gvt/opregion.c +++ b/drivers/gpu/drm/i915/gvt/opregion.c @@ -42,8 +42,6 @@ #define DEVICE_TYPE_EFP3 0x20 #define DEVICE_TYPE_EFP4 0x10 -#define DEV_SIZE 38 - struct opregion_header { u8 signature[16]; u32 size; @@ -63,6 +61,10 @@ struct bdb_data_header { u16 size; /* data size */ } __packed; +/* For supporting windows guest with opregion, here hardcode the emulated + * bdb header version as '186', and the corresponding child_device_config + * length should be '33' but not '38'. + */ struct efp_child_device_config { u16 handle; u16 device_type; @@ -109,12 +111,6 @@ struct efp_child_device_config { u8 mipi_bridge_type; /* 171 */ u16 device_class_ext; u8 dvo_function; - u8 dp_usb_type_c:1; /* 195 */ - u8 skip6:7; - u8 dp_usb_type_c_2x_gpio_index; /* 195 */ - u16 dp_usb_type_c_2x_gpio_pin; /* 195 */ - u8 iboost_dp:4; /* 196 */ - u8 iboost_hdmi:4; /* 196 */ } __packed; struct vbt { @@ -155,7 +151,7 @@ static void virt_vbt_generation(struct vbt *v) v->header.bdb_offset = offsetof(struct vbt, bdb_header); strcpy(&v->bdb_header.signature[0], "BIOS_DATA_BLOCK"); - v->bdb_header.version = 186; /* child_dev_size = 38 */ + v->bdb_header.version = 186; /* child_dev_size = 33 */ v->bdb_header.header_size = sizeof(v->bdb_header); v->bdb_header.bdb_size = sizeof(struct vbt) - sizeof(struct vbt_header) @@ -169,11 +165,13 @@ static void virt_vbt_generation(struct vbt *v) /* child device */ num_child = 4; /* each port has one child */ + v->general_definitions.child_dev_size = + sizeof(struct efp_child_device_config); v->general_definitions_header.id = BDB_GENERAL_DEFINITIONS; /* size will include child devices */ v->general_definitions_header.size = - sizeof(struct bdb_general_definitions) + num_child * DEV_SIZE; - v->general_definitions.child_dev_size = DEV_SIZE; + sizeof(struct bdb_general_definitions) + + num_child * v->general_definitions.child_dev_size; /* portA */ v->child0.handle = DEVICE_TYPE_EFP1; diff --git a/drivers/gpu/drm/i915/gvt/sched_policy.c b/drivers/gpu/drm/i915/gvt/sched_policy.c index 09d7bb72b4ff..c32e7d5e8629 100644 --- a/drivers/gpu/drm/i915/gvt/sched_policy.c +++ b/drivers/gpu/drm/i915/gvt/sched_policy.c @@ -47,11 +47,15 @@ static bool vgpu_has_pending_workload(struct intel_vgpu *vgpu) return false; } +/* We give 2 seconds higher prio for vGPU during start */ +#define GVT_SCHED_VGPU_PRI_TIME 2 + struct vgpu_sched_data { struct list_head lru_list; struct intel_vgpu *vgpu; bool active; - + bool pri_sched; + ktime_t pri_time; ktime_t sched_in_time; ktime_t sched_time; ktime_t left_ts; @@ -183,6 +187,14 @@ static struct intel_vgpu *find_busy_vgpu(struct gvt_sched_data *sched_data) if (!vgpu_has_pending_workload(vgpu_data->vgpu)) continue; + if (vgpu_data->pri_sched) { + if (ktime_before(ktime_get(), vgpu_data->pri_time)) { + vgpu = vgpu_data->vgpu; + break; + } else + vgpu_data->pri_sched = false; + } + /* Return the vGPU only if it has time slice left */ if (vgpu_data->left_ts > 0) { vgpu = vgpu_data->vgpu; @@ -202,6 +214,7 @@ static void tbs_sched_func(struct gvt_sched_data *sched_data) struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler; struct vgpu_sched_data *vgpu_data; struct intel_vgpu *vgpu = NULL; + /* no active vgpu or has already had a target */ if (list_empty(&sched_data->lru_runq_head) || scheduler->next_vgpu) goto out; @@ -209,12 +222,13 @@ static void tbs_sched_func(struct gvt_sched_data *sched_data) vgpu = find_busy_vgpu(sched_data); if (vgpu) { scheduler->next_vgpu = vgpu; - - /* Move the last used vGPU to the tail of lru_list */ vgpu_data = vgpu->sched_data; - list_del_init(&vgpu_data->lru_list); - list_add_tail(&vgpu_data->lru_list, - &sched_data->lru_runq_head); + if (!vgpu_data->pri_sched) { + /* Move the last used vGPU to the tail of lru_list */ + list_del_init(&vgpu_data->lru_list); + list_add_tail(&vgpu_data->lru_list, + &sched_data->lru_runq_head); + } } else { scheduler->next_vgpu = gvt->idle_vgpu; } @@ -328,11 +342,17 @@ static void tbs_sched_start_schedule(struct intel_vgpu *vgpu) { struct gvt_sched_data *sched_data = vgpu->gvt->scheduler.sched_data; struct vgpu_sched_data *vgpu_data = vgpu->sched_data; + ktime_t now; if (!list_empty(&vgpu_data->lru_list)) return; - list_add_tail(&vgpu_data->lru_list, &sched_data->lru_runq_head); + now = ktime_get(); + vgpu_data->pri_time = ktime_add(now, + ktime_set(GVT_SCHED_VGPU_PRI_TIME, 0)); + vgpu_data->pri_sched = true; + + list_add(&vgpu_data->lru_list, &sched_data->lru_runq_head); if (!hrtimer_active(&sched_data->timer)) hrtimer_start(&sched_data->timer, ktime_add_ns(ktime_get(), @@ -426,6 +446,7 @@ void intel_vgpu_stop_schedule(struct intel_vgpu *vgpu) &vgpu->gvt->scheduler; int ring_id; struct vgpu_sched_data *vgpu_data = vgpu->sched_data; + struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv; if (!vgpu_data->active) return; @@ -444,6 +465,7 @@ void intel_vgpu_stop_schedule(struct intel_vgpu *vgpu) scheduler->current_vgpu = NULL; } + intel_runtime_pm_get(dev_priv); spin_lock_bh(&scheduler->mmio_context_lock); for (ring_id = 0; ring_id < I915_NUM_ENGINES; ring_id++) { if (scheduler->engine_owner[ring_id] == vgpu) { @@ -452,5 +474,6 @@ void intel_vgpu_stop_schedule(struct intel_vgpu *vgpu) } } spin_unlock_bh(&scheduler->mmio_context_lock); + intel_runtime_pm_put(dev_priv); mutex_unlock(&vgpu->gvt->sched_lock); } diff --git a/drivers/gpu/drm/i915/gvt/vgpu.c b/drivers/gpu/drm/i915/gvt/vgpu.c index a4e8e3cf74fd..c628be05fbfe 100644 --- a/drivers/gpu/drm/i915/gvt/vgpu.c +++ b/drivers/gpu/drm/i915/gvt/vgpu.c @@ -281,6 +281,7 @@ void intel_gvt_destroy_vgpu(struct intel_vgpu *vgpu) intel_vgpu_clean_submission(vgpu); intel_vgpu_clean_display(vgpu); intel_vgpu_clean_opregion(vgpu); + intel_vgpu_reset_ggtt(vgpu, true); intel_vgpu_clean_gtt(vgpu); intel_gvt_hypervisor_detach_vgpu(vgpu); intel_vgpu_free_resource(vgpu); diff --git a/drivers/gpu/drm/imx/imx-drm-core.c b/drivers/gpu/drm/imx/imx-drm-core.c index 5ea0c82f9957..0e6942f21a4e 100644 --- a/drivers/gpu/drm/imx/imx-drm-core.c +++ b/drivers/gpu/drm/imx/imx-drm-core.c @@ -35,10 +35,8 @@ #define MAX_CRTC 4 -#if IS_ENABLED(CONFIG_DRM_FBDEV_EMULATION) static int legacyfb_depth = 16; module_param(legacyfb_depth, int, 0444); -#endif DEFINE_DRM_GEM_CMA_FOPS(imx_drm_driver_fops); @@ -86,7 +84,6 @@ static int imx_drm_atomic_check(struct drm_device *dev, static const struct drm_mode_config_funcs imx_drm_mode_config_funcs = { .fb_create = drm_gem_fb_create, - .output_poll_changed = drm_fb_helper_output_poll_changed, .atomic_check = imx_drm_atomic_check, .atomic_commit = drm_atomic_helper_commit, }; @@ -165,7 +162,6 @@ static const struct drm_ioctl_desc imx_drm_ioctls[] = { static struct drm_driver imx_drm_driver = { .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME | DRIVER_ATOMIC, - .lastclose = drm_fb_helper_lastclose, .gem_free_object_unlocked = drm_gem_cma_free_object, .gem_vm_ops = &drm_gem_cma_vm_ops, .dumb_create = drm_gem_cma_dumb_create, @@ -263,30 +259,23 @@ static int imx_drm_bind(struct device *dev) * The fb helper takes copies of key hardware information, so the * crtcs/connectors/encoders must not change after this point. */ -#if IS_ENABLED(CONFIG_DRM_FBDEV_EMULATION) if (legacyfb_depth != 16 && legacyfb_depth != 32) { dev_warn(dev, "Invalid legacyfb_depth. Defaulting to 16bpp\n"); legacyfb_depth = 16; } - ret = drm_fb_cma_fbdev_init(drm, legacyfb_depth, MAX_CRTC); - if (ret) - goto err_unbind; -#endif drm_kms_helper_poll_init(drm); ret = drm_dev_register(drm, 0); if (ret) - goto err_fbhelper; + goto err_poll_fini; + + drm_fbdev_generic_setup(drm, legacyfb_depth); return 0; -err_fbhelper: +err_poll_fini: drm_kms_helper_poll_fini(drm); -#if IS_ENABLED(CONFIG_DRM_FBDEV_EMULATION) - drm_fb_cma_fbdev_fini(drm); -err_unbind: -#endif component_unbind_all(drm->dev, drm); err_kms: drm_mode_config_cleanup(drm); @@ -303,8 +292,6 @@ static void imx_drm_unbind(struct device *dev) drm_kms_helper_poll_fini(drm); - drm_fb_cma_fbdev_fini(drm); - drm_mode_config_cleanup(drm); component_unbind_all(drm->dev, drm); diff --git a/drivers/gpu/drm/mediatek/Makefile b/drivers/gpu/drm/mediatek/Makefile index ce83c396a742..82ae49c64221 100644 --- a/drivers/gpu/drm/mediatek/Makefile +++ b/drivers/gpu/drm/mediatek/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 + mediatek-drm-y := mtk_disp_color.o \ mtk_disp_ovl.o \ mtk_disp_rdma.o \ @@ -18,6 +19,8 @@ obj-$(CONFIG_DRM_MEDIATEK) += mediatek-drm.o mediatek-drm-hdmi-objs := mtk_cec.o \ mtk_hdmi.o \ mtk_hdmi_ddc.o \ - mtk_mt8173_hdmi_phy.o + mtk_mt2701_hdmi_phy.o \ + mtk_mt8173_hdmi_phy.o \ + mtk_hdmi_phy.o obj-$(CONFIG_DRM_MEDIATEK_HDMI) += mediatek-drm-hdmi.o diff --git a/drivers/gpu/drm/mediatek/mtk_dpi.c b/drivers/gpu/drm/mediatek/mtk_dpi.c index 6c0ea39d5739..62a9d47df948 100644 --- a/drivers/gpu/drm/mediatek/mtk_dpi.c +++ b/drivers/gpu/drm/mediatek/mtk_dpi.c @@ -14,10 +14,12 @@ #include <drm/drmP.h> #include <drm/drm_crtc.h> #include <drm/drm_crtc_helper.h> +#include <drm/drm_of.h> #include <linux/kernel.h> #include <linux/component.h> #include <linux/platform_device.h> #include <linux/of.h> +#include <linux/of_device.h> #include <linux/of_graph.h> #include <linux/interrupt.h> #include <linux/types.h> @@ -72,12 +74,12 @@ struct mtk_dpi { struct clk *tvd_clk; int irq; struct drm_display_mode mode; + const struct mtk_dpi_conf *conf; enum mtk_dpi_out_color_format color_format; enum mtk_dpi_out_yc_map yc_map; enum mtk_dpi_out_bit_num bit_num; enum mtk_dpi_out_channel_swap channel_swap; - bool power_sta; - u8 power_ctl; + int refcount; }; static inline struct mtk_dpi *mtk_dpi_from_encoder(struct drm_encoder *e) @@ -90,11 +92,6 @@ enum mtk_dpi_polarity { MTK_DPI_POLARITY_FALLING, }; -enum mtk_dpi_power_ctl { - DPI_POWER_START = BIT(0), - DPI_POWER_ENABLE = BIT(1), -}; - struct mtk_dpi_polarities { enum mtk_dpi_polarity de_pol; enum mtk_dpi_polarity ck_pol; @@ -116,6 +113,12 @@ struct mtk_dpi_yc_limit { u16 c_bottom; }; +struct mtk_dpi_conf { + unsigned int (*cal_factor)(int clock); + u32 reg_h_fre_con; + bool edge_sel_en; +}; + static void mtk_dpi_mask(struct mtk_dpi *dpi, u32 offset, u32 val, u32 mask) { u32 tmp = readl(dpi->regs + offset) & ~mask; @@ -341,7 +344,13 @@ static void mtk_dpi_config_swap_input(struct mtk_dpi *dpi, bool enable) static void mtk_dpi_config_2n_h_fre(struct mtk_dpi *dpi) { - mtk_dpi_mask(dpi, DPI_H_FRE_CON, H_FRE_2N, H_FRE_2N); + mtk_dpi_mask(dpi, dpi->conf->reg_h_fre_con, H_FRE_2N, H_FRE_2N); +} + +static void mtk_dpi_config_disable_edge(struct mtk_dpi *dpi) +{ + if (dpi->conf->edge_sel_en) + mtk_dpi_mask(dpi, dpi->conf->reg_h_fre_con, 0, EDGE_SEL_EN); } static void mtk_dpi_config_color_format(struct mtk_dpi *dpi, @@ -367,40 +376,30 @@ static void mtk_dpi_config_color_format(struct mtk_dpi *dpi, } } -static void mtk_dpi_power_off(struct mtk_dpi *dpi, enum mtk_dpi_power_ctl pctl) +static void mtk_dpi_power_off(struct mtk_dpi *dpi) { - dpi->power_ctl &= ~pctl; - - if ((dpi->power_ctl & DPI_POWER_START) || - (dpi->power_ctl & DPI_POWER_ENABLE)) + if (WARN_ON(dpi->refcount == 0)) return; - if (!dpi->power_sta) + if (--dpi->refcount != 0) return; mtk_dpi_disable(dpi); clk_disable_unprepare(dpi->pixel_clk); clk_disable_unprepare(dpi->engine_clk); - dpi->power_sta = false; } -static int mtk_dpi_power_on(struct mtk_dpi *dpi, enum mtk_dpi_power_ctl pctl) +static int mtk_dpi_power_on(struct mtk_dpi *dpi) { int ret; - dpi->power_ctl |= pctl; - - if (!(dpi->power_ctl & DPI_POWER_START) && - !(dpi->power_ctl & DPI_POWER_ENABLE)) - return 0; - - if (dpi->power_sta) + if (++dpi->refcount != 1) return 0; ret = clk_prepare_enable(dpi->engine_clk); if (ret) { dev_err(dpi->dev, "Failed to enable engine clock: %d\n", ret); - goto err_eng; + goto err_refcount; } ret = clk_prepare_enable(dpi->pixel_clk); @@ -410,13 +409,12 @@ static int mtk_dpi_power_on(struct mtk_dpi *dpi, enum mtk_dpi_power_ctl pctl) } mtk_dpi_enable(dpi); - dpi->power_sta = true; return 0; err_pixel: clk_disable_unprepare(dpi->engine_clk); -err_eng: - dpi->power_ctl &= ~pctl; +err_refcount: + dpi->refcount--; return ret; } @@ -435,15 +433,7 @@ static int mtk_dpi_set_display_mode(struct mtk_dpi *dpi, unsigned int factor; /* let pll_rate can fix the valid range of tvdpll (1G~2GHz) */ - - if (mode->clock <= 27000) - factor = 3 << 4; - else if (mode->clock <= 84000) - factor = 3 << 3; - else if (mode->clock <= 167000) - factor = 3 << 2; - else - factor = 3 << 1; + factor = dpi->conf->cal_factor(mode->clock); drm_display_mode_to_videomode(mode, &vm); pll_rate = vm.pixelclock * factor; @@ -518,6 +508,7 @@ static int mtk_dpi_set_display_mode(struct mtk_dpi *dpi, mtk_dpi_config_yc_map(dpi, dpi->yc_map); mtk_dpi_config_color_format(dpi, dpi->color_format); mtk_dpi_config_2n_h_fre(dpi); + mtk_dpi_config_disable_edge(dpi); mtk_dpi_sw_reset(dpi, false); return 0; @@ -552,14 +543,14 @@ static void mtk_dpi_encoder_disable(struct drm_encoder *encoder) { struct mtk_dpi *dpi = mtk_dpi_from_encoder(encoder); - mtk_dpi_power_off(dpi, DPI_POWER_ENABLE); + mtk_dpi_power_off(dpi); } static void mtk_dpi_encoder_enable(struct drm_encoder *encoder) { struct mtk_dpi *dpi = mtk_dpi_from_encoder(encoder); - mtk_dpi_power_on(dpi, DPI_POWER_ENABLE); + mtk_dpi_power_on(dpi); mtk_dpi_set_display_mode(dpi, &dpi->mode); } @@ -582,14 +573,14 @@ static void mtk_dpi_start(struct mtk_ddp_comp *comp) { struct mtk_dpi *dpi = container_of(comp, struct mtk_dpi, ddp_comp); - mtk_dpi_power_on(dpi, DPI_POWER_START); + mtk_dpi_power_on(dpi); } static void mtk_dpi_stop(struct mtk_ddp_comp *comp) { struct mtk_dpi *dpi = container_of(comp, struct mtk_dpi, ddp_comp); - mtk_dpi_power_off(dpi, DPI_POWER_START); + mtk_dpi_power_off(dpi); } static const struct mtk_ddp_comp_funcs mtk_dpi_funcs = { @@ -656,12 +647,46 @@ static const struct component_ops mtk_dpi_component_ops = { .unbind = mtk_dpi_unbind, }; +static unsigned int mt8173_calculate_factor(int clock) +{ + if (clock <= 27000) + return 3 << 4; + else if (clock <= 84000) + return 3 << 3; + else if (clock <= 167000) + return 3 << 2; + else + return 3 << 1; +} + +static unsigned int mt2701_calculate_factor(int clock) +{ + if (clock <= 64000) + return 16; + else if (clock <= 128000) + return 8; + else if (clock <= 256000) + return 4; + else + return 2; +} + +static const struct mtk_dpi_conf mt8173_conf = { + .cal_factor = mt8173_calculate_factor, + .reg_h_fre_con = 0xe0, +}; + +static const struct mtk_dpi_conf mt2701_conf = { + .cal_factor = mt2701_calculate_factor, + .reg_h_fre_con = 0xb0, + .edge_sel_en = true, +}; + static int mtk_dpi_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; struct mtk_dpi *dpi; struct resource *mem; - struct device_node *bridge_node; int comp_id; int ret; @@ -670,6 +695,7 @@ static int mtk_dpi_probe(struct platform_device *pdev) return -ENOMEM; dpi->dev = dev; + dpi->conf = (struct mtk_dpi_conf *)of_device_get_match_data(dev); mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); dpi->regs = devm_ioremap_resource(dev, mem); @@ -706,16 +732,12 @@ static int mtk_dpi_probe(struct platform_device *pdev) return -EINVAL; } - bridge_node = of_graph_get_remote_node(dev->of_node, 0, 0); - if (!bridge_node) - return -ENODEV; - - dev_info(dev, "Found bridge node: %pOF\n", bridge_node); + ret = drm_of_find_panel_or_bridge(dev->of_node, 0, 0, + NULL, &dpi->bridge); + if (ret) + return ret; - dpi->bridge = of_drm_find_bridge(bridge_node); - of_node_put(bridge_node); - if (!dpi->bridge) - return -EPROBE_DEFER; + dev_info(dev, "Found bridge node: %pOF\n", dpi->bridge->of_node); comp_id = mtk_ddp_comp_get_id(dev->of_node, MTK_DPI); if (comp_id < 0) { @@ -749,8 +771,13 @@ static int mtk_dpi_remove(struct platform_device *pdev) } static const struct of_device_id mtk_dpi_of_ids[] = { - { .compatible = "mediatek,mt8173-dpi", }, - {} + { .compatible = "mediatek,mt2701-dpi", + .data = &mt2701_conf, + }, + { .compatible = "mediatek,mt8173-dpi", + .data = &mt8173_conf, + }, + { }, }; struct platform_driver mtk_dpi_driver = { diff --git a/drivers/gpu/drm/mediatek/mtk_dpi_regs.h b/drivers/gpu/drm/mediatek/mtk_dpi_regs.h index 4b6ad4751a31..d9db8c4cacd7 100644 --- a/drivers/gpu/drm/mediatek/mtk_dpi_regs.h +++ b/drivers/gpu/drm/mediatek/mtk_dpi_regs.h @@ -223,6 +223,6 @@ #define ESAV_CODE2 (0xFFF << 0) #define ESAV_CODE3_MSB BIT(16) -#define DPI_H_FRE_CON 0xE0 +#define EDGE_SEL_EN BIT(5) #define H_FRE_2N BIT(25) #endif /* __MTK_DPI_REGS_H */ diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp.c b/drivers/gpu/drm/mediatek/mtk_drm_ddp.c index 546b3e3b300b..579ce28d801d 100644 --- a/drivers/gpu/drm/mediatek/mtk_drm_ddp.c +++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp.c @@ -39,6 +39,7 @@ #define DISP_REG_CONFIG_DISP_OVL_MOUT_EN 0x030 #define DISP_REG_CONFIG_OUT_SEL 0x04c #define DISP_REG_CONFIG_DSI_SEL 0x050 +#define DISP_REG_CONFIG_DPI_SEL 0x064 #define DISP_REG_MUTEX_EN(n) (0x20 + 0x20 * (n)) #define DISP_REG_MUTEX(n) (0x24 + 0x20 * (n)) @@ -136,7 +137,10 @@ #define OVL_MOUT_EN_RDMA 0x1 #define BLS_TO_DSI_RDMA1_TO_DPI1 0x8 +#define BLS_TO_DPI_RDMA1_TO_DSI 0x2 #define DSI_SEL_IN_BLS 0x0 +#define DPI_SEL_IN_BLS 0x0 +#define DSI_SEL_IN_RDMA 0x1 struct mtk_disp_mutex { int id; @@ -339,9 +343,17 @@ static void mtk_ddp_sout_sel(void __iomem *config_regs, enum mtk_ddp_comp_id cur, enum mtk_ddp_comp_id next) { - if (cur == DDP_COMPONENT_BLS && next == DDP_COMPONENT_DSI0) + if (cur == DDP_COMPONENT_BLS && next == DDP_COMPONENT_DSI0) { writel_relaxed(BLS_TO_DSI_RDMA1_TO_DPI1, config_regs + DISP_REG_CONFIG_OUT_SEL); + } else if (cur == DDP_COMPONENT_BLS && next == DDP_COMPONENT_DPI0) { + writel_relaxed(BLS_TO_DPI_RDMA1_TO_DSI, + config_regs + DISP_REG_CONFIG_OUT_SEL); + writel_relaxed(DSI_SEL_IN_RDMA, + config_regs + DISP_REG_CONFIG_DSI_SEL); + writel_relaxed(DPI_SEL_IN_BLS, + config_regs + DISP_REG_CONFIG_DPI_SEL); + } } void mtk_ddp_add_comp_to_path(void __iomem *config_regs, diff --git a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c index ff974d82a4a6..54ca794db3e9 100644 --- a/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c +++ b/drivers/gpu/drm/mediatek/mtk_drm_ddp_comp.c @@ -294,7 +294,7 @@ int mtk_ddp_comp_init(struct device *dev, struct device_node *node, comp->irq = of_irq_get(node, 0); comp->clk = of_clk_get(node, 0); if (IS_ERR(comp->clk)) - comp->clk = NULL; + return PTR_ERR(comp->clk); /* Only DMA capable components need the LARB property */ comp->larb_dev = NULL; diff --git a/drivers/gpu/drm/mediatek/mtk_drm_drv.c b/drivers/gpu/drm/mediatek/mtk_drm_drv.c index 47ec604289b7..6422e99952fe 100644 --- a/drivers/gpu/drm/mediatek/mtk_drm_drv.c +++ b/drivers/gpu/drm/mediatek/mtk_drm_drv.c @@ -424,6 +424,8 @@ static const struct of_device_id mtk_ddp_comp_dt_ids[] = { .data = (void *)MTK_DSI }, { .compatible = "mediatek,mt8173-dsi", .data = (void *)MTK_DSI }, + { .compatible = "mediatek,mt2701-dpi", + .data = (void *)MTK_DPI }, { .compatible = "mediatek,mt8173-dpi", .data = (void *)MTK_DPI }, { .compatible = "mediatek,mt2701-disp-mutex", diff --git a/drivers/gpu/drm/mediatek/mtk_hdmi.c b/drivers/gpu/drm/mediatek/mtk_hdmi.c index 2d45d1dd9554..11e3644da79a 100644 --- a/drivers/gpu/drm/mediatek/mtk_hdmi.c +++ b/drivers/gpu/drm/mediatek/mtk_hdmi.c @@ -233,6 +233,7 @@ static void mtk_hdmi_hw_vid_black(struct mtk_hdmi *hdmi, bool black) static void mtk_hdmi_hw_make_reg_writable(struct mtk_hdmi *hdmi, bool enable) { struct arm_smccc_res res; + struct mtk_hdmi_phy *hdmi_phy = phy_get_drvdata(hdmi->phy); /* * MT8173 HDMI hardware has an output control bit to enable/disable HDMI @@ -240,8 +241,13 @@ static void mtk_hdmi_hw_make_reg_writable(struct mtk_hdmi *hdmi, bool enable) * The ARM trusted firmware provides an API for the HDMI driver to set * this control bit to enable HDMI output in supervisor mode. */ - arm_smccc_smc(MTK_SIP_SET_AUTHORIZED_SECURE_REG, 0x14000904, 0x80000000, - 0, 0, 0, 0, 0, &res); + if (hdmi_phy->conf && hdmi_phy->conf->tz_disabled) + regmap_update_bits(hdmi->sys_regmap, + hdmi->sys_offset + HDMI_SYS_CFG20, + 0x80008005, enable ? 0x80000005 : 0x8000); + else + arm_smccc_smc(MTK_SIP_SET_AUTHORIZED_SECURE_REG, 0x14000904, + 0x80000000, 0, 0, 0, 0, 0, &res); regmap_update_bits(hdmi->sys_regmap, hdmi->sys_offset + HDMI_SYS_CFG20, HDMI_PCLK_FREE_RUN, enable ? HDMI_PCLK_FREE_RUN : 0); @@ -1575,6 +1581,11 @@ static int mtk_hdmi_audio_hw_params(struct device *dev, void *data, hdmi_params.aud_i2s_fmt = HDMI_I2S_MODE_I2S_24BIT; hdmi_params.aud_mclk = HDMI_AUD_MCLK_128FS; break; + case HDMI_SPDIF: + hdmi_params.aud_codec = HDMI_AUDIO_CODING_TYPE_PCM; + hdmi_params.aud_sampe_size = HDMI_AUDIO_SAMPLE_SIZE_16; + hdmi_params.aud_input_type = HDMI_AUD_INPUT_SPDIF; + break; default: dev_err(hdmi->dev, "%s: Invalid DAI format %d\n", __func__, daifmt->fmt); diff --git a/drivers/gpu/drm/mediatek/mtk_hdmi.h b/drivers/gpu/drm/mediatek/mtk_hdmi.h index 6371b3de1ff6..3e9fb8d19802 100644 --- a/drivers/gpu/drm/mediatek/mtk_hdmi.h +++ b/drivers/gpu/drm/mediatek/mtk_hdmi.h @@ -13,11 +13,11 @@ */ #ifndef _MTK_HDMI_CTRL_H #define _MTK_HDMI_CTRL_H +#include "mtk_hdmi_phy.h" struct platform_driver; extern struct platform_driver mtk_cec_driver; extern struct platform_driver mtk_hdmi_ddc_driver; -extern struct platform_driver mtk_hdmi_phy_driver; #endif /* _MTK_HDMI_CTRL_H */ diff --git a/drivers/gpu/drm/mediatek/mtk_hdmi_phy.c b/drivers/gpu/drm/mediatek/mtk_hdmi_phy.c new file mode 100644 index 000000000000..4ef9c57ffd44 --- /dev/null +++ b/drivers/gpu/drm/mediatek/mtk_hdmi_phy.c @@ -0,0 +1,235 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2018 MediaTek Inc. + * Author: Jie Qiu <jie.qiu@mediatek.com> + */ + +#include "mtk_hdmi_phy.h" + +static int mtk_hdmi_phy_power_on(struct phy *phy); +static int mtk_hdmi_phy_power_off(struct phy *phy); + +static const struct phy_ops mtk_hdmi_phy_dev_ops = { + .power_on = mtk_hdmi_phy_power_on, + .power_off = mtk_hdmi_phy_power_off, + .owner = THIS_MODULE, +}; + +long mtk_hdmi_pll_round_rate(struct clk_hw *hw, unsigned long rate, + unsigned long *parent_rate) +{ + struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); + + hdmi_phy->pll_rate = rate; + if (rate <= 74250000) + *parent_rate = rate; + else + *parent_rate = rate / 2; + + return rate; +} + +unsigned long mtk_hdmi_pll_recalc_rate(struct clk_hw *hw, + unsigned long parent_rate) +{ + struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); + + return hdmi_phy->pll_rate; +} + +void mtk_hdmi_phy_clear_bits(struct mtk_hdmi_phy *hdmi_phy, u32 offset, + u32 bits) +{ + void __iomem *reg = hdmi_phy->regs + offset; + u32 tmp; + + tmp = readl(reg); + tmp &= ~bits; + writel(tmp, reg); +} + +void mtk_hdmi_phy_set_bits(struct mtk_hdmi_phy *hdmi_phy, u32 offset, + u32 bits) +{ + void __iomem *reg = hdmi_phy->regs + offset; + u32 tmp; + + tmp = readl(reg); + tmp |= bits; + writel(tmp, reg); +} + +void mtk_hdmi_phy_mask(struct mtk_hdmi_phy *hdmi_phy, u32 offset, + u32 val, u32 mask) +{ + void __iomem *reg = hdmi_phy->regs + offset; + u32 tmp; + + tmp = readl(reg); + tmp = (tmp & ~mask) | (val & mask); + writel(tmp, reg); +} + +inline struct mtk_hdmi_phy *to_mtk_hdmi_phy(struct clk_hw *hw) +{ + return container_of(hw, struct mtk_hdmi_phy, pll_hw); +} + +static int mtk_hdmi_phy_power_on(struct phy *phy) +{ + struct mtk_hdmi_phy *hdmi_phy = phy_get_drvdata(phy); + int ret; + + ret = clk_prepare_enable(hdmi_phy->pll); + if (ret < 0) + return ret; + + hdmi_phy->conf->hdmi_phy_enable_tmds(hdmi_phy); + return 0; +} + +static int mtk_hdmi_phy_power_off(struct phy *phy) +{ + struct mtk_hdmi_phy *hdmi_phy = phy_get_drvdata(phy); + + hdmi_phy->conf->hdmi_phy_disable_tmds(hdmi_phy); + clk_disable_unprepare(hdmi_phy->pll); + + return 0; +} + +static const struct phy_ops * +mtk_hdmi_phy_dev_get_ops(const struct mtk_hdmi_phy *hdmi_phy) +{ + if (hdmi_phy && hdmi_phy->conf && + hdmi_phy->conf->hdmi_phy_enable_tmds && + hdmi_phy->conf->hdmi_phy_disable_tmds) + return &mtk_hdmi_phy_dev_ops; + + dev_err(hdmi_phy->dev, "Failed to get dev ops of phy\n"); + return NULL; +} + +static void mtk_hdmi_phy_clk_get_ops(struct mtk_hdmi_phy *hdmi_phy, + const struct clk_ops **ops) +{ + if (hdmi_phy && hdmi_phy->conf && hdmi_phy->conf->hdmi_phy_clk_ops) + *ops = hdmi_phy->conf->hdmi_phy_clk_ops; + else + dev_err(hdmi_phy->dev, "Failed to get clk ops of phy\n"); +} + +static int mtk_hdmi_phy_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct mtk_hdmi_phy *hdmi_phy; + struct resource *mem; + struct clk *ref_clk; + const char *ref_clk_name; + struct clk_init_data clk_init = { + .num_parents = 1, + .parent_names = (const char * const *)&ref_clk_name, + .flags = CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, + }; + + struct phy *phy; + struct phy_provider *phy_provider; + int ret; + + hdmi_phy = devm_kzalloc(dev, sizeof(*hdmi_phy), GFP_KERNEL); + if (!hdmi_phy) + return -ENOMEM; + + mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); + hdmi_phy->regs = devm_ioremap_resource(dev, mem); + if (IS_ERR(hdmi_phy->regs)) { + ret = PTR_ERR(hdmi_phy->regs); + dev_err(dev, "Failed to get memory resource: %d\n", ret); + return ret; + } + + ref_clk = devm_clk_get(dev, "pll_ref"); + if (IS_ERR(ref_clk)) { + ret = PTR_ERR(ref_clk); + dev_err(&pdev->dev, "Failed to get PLL reference clock: %d\n", + ret); + return ret; + } + ref_clk_name = __clk_get_name(ref_clk); + + ret = of_property_read_string(dev->of_node, "clock-output-names", + &clk_init.name); + if (ret < 0) { + dev_err(dev, "Failed to read clock-output-names: %d\n", ret); + return ret; + } + + hdmi_phy->dev = dev; + hdmi_phy->conf = + (struct mtk_hdmi_phy_conf *)of_device_get_match_data(dev); + mtk_hdmi_phy_clk_get_ops(hdmi_phy, &clk_init.ops); + hdmi_phy->pll_hw.init = &clk_init; + hdmi_phy->pll = devm_clk_register(dev, &hdmi_phy->pll_hw); + if (IS_ERR(hdmi_phy->pll)) { + ret = PTR_ERR(hdmi_phy->pll); + dev_err(dev, "Failed to register PLL: %d\n", ret); + return ret; + } + + ret = of_property_read_u32(dev->of_node, "mediatek,ibias", + &hdmi_phy->ibias); + if (ret < 0) { + dev_err(&pdev->dev, "Failed to get ibias: %d\n", ret); + return ret; + } + + ret = of_property_read_u32(dev->of_node, "mediatek,ibias_up", + &hdmi_phy->ibias_up); + if (ret < 0) { + dev_err(&pdev->dev, "Failed to get ibias up: %d\n", ret); + return ret; + } + + dev_info(dev, "Using default TX DRV impedance: 4.2k/36\n"); + hdmi_phy->drv_imp_clk = 0x30; + hdmi_phy->drv_imp_d2 = 0x30; + hdmi_phy->drv_imp_d1 = 0x30; + hdmi_phy->drv_imp_d0 = 0x30; + + phy = devm_phy_create(dev, NULL, mtk_hdmi_phy_dev_get_ops(hdmi_phy)); + if (IS_ERR(phy)) { + dev_err(dev, "Failed to create HDMI PHY\n"); + return PTR_ERR(phy); + } + phy_set_drvdata(phy, hdmi_phy); + + phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate); + if (IS_ERR(phy_provider)) { + dev_err(dev, "Failed to register HDMI PHY\n"); + return PTR_ERR(phy_provider); + } + + return of_clk_add_provider(dev->of_node, of_clk_src_simple_get, + hdmi_phy->pll); +} + +static const struct of_device_id mtk_hdmi_phy_match[] = { + { .compatible = "mediatek,mt2701-hdmi-phy", + .data = &mtk_hdmi_phy_2701_conf, + }, + { .compatible = "mediatek,mt8173-hdmi-phy", + .data = &mtk_hdmi_phy_8173_conf, + }, + {}, +}; + +struct platform_driver mtk_hdmi_phy_driver = { + .probe = mtk_hdmi_phy_probe, + .driver = { + .name = "mediatek-hdmi-phy", + .of_match_table = mtk_hdmi_phy_match, + }, +}; + +MODULE_DESCRIPTION("MediaTek HDMI PHY Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/gpu/drm/mediatek/mtk_hdmi_phy.h b/drivers/gpu/drm/mediatek/mtk_hdmi_phy.h new file mode 100644 index 000000000000..f39b1fc66612 --- /dev/null +++ b/drivers/gpu/drm/mediatek/mtk_hdmi_phy.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright (c) 2018 MediaTek Inc. + * Author: Chunhui Dai <chunhui.dai@mediatek.com> + */ + +#ifndef _MTK_HDMI_PHY_H +#define _MTK_HDMI_PHY_H +#include <linux/clk.h> +#include <linux/clk-provider.h> +#include <linux/delay.h> +#include <linux/io.h> +#include <linux/mfd/syscon.h> +#include <linux/module.h> +#include <linux/of_device.h> +#include <linux/phy/phy.h> +#include <linux/platform_device.h> +#include <linux/types.h> + +struct mtk_hdmi_phy; + +struct mtk_hdmi_phy_conf { + bool tz_disabled; + const struct clk_ops *hdmi_phy_clk_ops; + void (*hdmi_phy_enable_tmds)(struct mtk_hdmi_phy *hdmi_phy); + void (*hdmi_phy_disable_tmds)(struct mtk_hdmi_phy *hdmi_phy); +}; + +struct mtk_hdmi_phy { + void __iomem *regs; + struct device *dev; + struct mtk_hdmi_phy_conf *conf; + struct clk *pll; + struct clk_hw pll_hw; + unsigned long pll_rate; + unsigned char drv_imp_clk; + unsigned char drv_imp_d2; + unsigned char drv_imp_d1; + unsigned char drv_imp_d0; + unsigned int ibias; + unsigned int ibias_up; +}; + +void mtk_hdmi_phy_clear_bits(struct mtk_hdmi_phy *hdmi_phy, u32 offset, + u32 bits); +void mtk_hdmi_phy_set_bits(struct mtk_hdmi_phy *hdmi_phy, u32 offset, + u32 bits); +void mtk_hdmi_phy_mask(struct mtk_hdmi_phy *hdmi_phy, u32 offset, + u32 val, u32 mask); +struct mtk_hdmi_phy *to_mtk_hdmi_phy(struct clk_hw *hw); +long mtk_hdmi_pll_round_rate(struct clk_hw *hw, unsigned long rate, + unsigned long *parent_rate); +unsigned long mtk_hdmi_pll_recalc_rate(struct clk_hw *hw, + unsigned long parent_rate); + +extern struct platform_driver mtk_hdmi_phy_driver; +extern struct mtk_hdmi_phy_conf mtk_hdmi_phy_8173_conf; +extern struct mtk_hdmi_phy_conf mtk_hdmi_phy_2701_conf; + +#endif /* _MTK_HDMI_PHY_H */ diff --git a/drivers/gpu/drm/mediatek/mtk_mt2701_hdmi_phy.c b/drivers/gpu/drm/mediatek/mtk_mt2701_hdmi_phy.c new file mode 100644 index 000000000000..fcc42dc6ea7f --- /dev/null +++ b/drivers/gpu/drm/mediatek/mtk_mt2701_hdmi_phy.c @@ -0,0 +1,212 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2018 MediaTek Inc. + * Author: Chunhui Dai <chunhui.dai@mediatek.com> + */ + +#include "mtk_hdmi_phy.h" + +#define HDMI_CON0 0x00 +#define RG_HDMITX_DRV_IBIAS 0 +#define RG_HDMITX_DRV_IBIAS_MASK (0x3f << 0) +#define RG_HDMITX_EN_SER 12 +#define RG_HDMITX_EN_SER_MASK (0x0f << 12) +#define RG_HDMITX_EN_SLDO 16 +#define RG_HDMITX_EN_SLDO_MASK (0x0f << 16) +#define RG_HDMITX_EN_PRED 20 +#define RG_HDMITX_EN_PRED_MASK (0x0f << 20) +#define RG_HDMITX_EN_IMP 24 +#define RG_HDMITX_EN_IMP_MASK (0x0f << 24) +#define RG_HDMITX_EN_DRV 28 +#define RG_HDMITX_EN_DRV_MASK (0x0f << 28) + +#define HDMI_CON1 0x04 +#define RG_HDMITX_PRED_IBIAS 18 +#define RG_HDMITX_PRED_IBIAS_MASK (0x0f << 18) +#define RG_HDMITX_PRED_IMP (0x01 << 22) +#define RG_HDMITX_DRV_IMP 26 +#define RG_HDMITX_DRV_IMP_MASK (0x3f << 26) + +#define HDMI_CON2 0x08 +#define RG_HDMITX_EN_TX_CKLDO (0x01 << 0) +#define RG_HDMITX_EN_TX_POSDIV (0x01 << 1) +#define RG_HDMITX_TX_POSDIV 3 +#define RG_HDMITX_TX_POSDIV_MASK (0x03 << 3) +#define RG_HDMITX_EN_MBIAS (0x01 << 6) +#define RG_HDMITX_MBIAS_LPF_EN (0x01 << 7) + +#define HDMI_CON4 0x10 +#define RG_HDMITX_RESERVE_MASK (0xffffffff << 0) + +#define HDMI_CON6 0x18 +#define RG_HTPLL_BR 0 +#define RG_HTPLL_BR_MASK (0x03 << 0) +#define RG_HTPLL_BC 2 +#define RG_HTPLL_BC_MASK (0x03 << 2) +#define RG_HTPLL_BP 4 +#define RG_HTPLL_BP_MASK (0x0f << 4) +#define RG_HTPLL_IR 8 +#define RG_HTPLL_IR_MASK (0x0f << 8) +#define RG_HTPLL_IC 12 +#define RG_HTPLL_IC_MASK (0x0f << 12) +#define RG_HTPLL_POSDIV 16 +#define RG_HTPLL_POSDIV_MASK (0x03 << 16) +#define RG_HTPLL_PREDIV 18 +#define RG_HTPLL_PREDIV_MASK (0x03 << 18) +#define RG_HTPLL_FBKSEL 20 +#define RG_HTPLL_FBKSEL_MASK (0x03 << 20) +#define RG_HTPLL_RLH_EN (0x01 << 22) +#define RG_HTPLL_FBKDIV 24 +#define RG_HTPLL_FBKDIV_MASK (0x7f << 24) +#define RG_HTPLL_EN (0x01 << 31) + +#define HDMI_CON7 0x1c +#define RG_HTPLL_AUTOK_EN (0x01 << 23) +#define RG_HTPLL_DIVEN 28 +#define RG_HTPLL_DIVEN_MASK (0x07 << 28) + +static int mtk_hdmi_pll_prepare(struct clk_hw *hw) +{ + struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); + + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON7, RG_HTPLL_AUTOK_EN); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_RLH_EN); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_POSDIV_MASK); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_MBIAS); + usleep_range(80, 100); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_EN); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_CKLDO); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SLDO_MASK); + usleep_range(80, 100); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_MBIAS_LPF_EN); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_POSDIV); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SER_MASK); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_PRED_MASK); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_DRV_MASK); + usleep_range(80, 100); + return 0; +} + +static void mtk_hdmi_pll_unprepare(struct clk_hw *hw) +{ + struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); + + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_DRV_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_PRED_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SER_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_POSDIV); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_MBIAS_LPF_EN); + usleep_range(80, 100); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SLDO_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_CKLDO); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_EN); + usleep_range(80, 100); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_MBIAS); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_POSDIV_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_RLH_EN); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON7, RG_HTPLL_AUTOK_EN); + usleep_range(80, 100); +} + +static int mtk_hdmi_pll_set_rate(struct clk_hw *hw, unsigned long rate, + unsigned long parent_rate) +{ + struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); + u32 pos_div; + + if (rate <= 64000000) + pos_div = 3; + else if (rate <= 12800000) + pos_div = 1; + else + pos_div = 1; + + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_PREDIV_MASK); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_POSDIV_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON6, (0x1 << RG_HTPLL_IC), + RG_HTPLL_IC_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON6, (0x1 << RG_HTPLL_IR), + RG_HTPLL_IR_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON2, (pos_div << RG_HDMITX_TX_POSDIV), + RG_HDMITX_TX_POSDIV_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON6, (1 << RG_HTPLL_FBKSEL), + RG_HTPLL_FBKSEL_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON6, (19 << RG_HTPLL_FBKDIV), + RG_HTPLL_FBKDIV_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON7, (0x2 << RG_HTPLL_DIVEN), + RG_HTPLL_DIVEN_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON6, (0xc << RG_HTPLL_BP), + RG_HTPLL_BP_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON6, (0x2 << RG_HTPLL_BC), + RG_HTPLL_BC_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON6, (0x1 << RG_HTPLL_BR), + RG_HTPLL_BR_MASK); + + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON1, RG_HDMITX_PRED_IMP); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON1, (0x3 << RG_HDMITX_PRED_IBIAS), + RG_HDMITX_PRED_IBIAS_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_IMP_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON1, (0x28 << RG_HDMITX_DRV_IMP), + RG_HDMITX_DRV_IMP_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON4, 0x28, RG_HDMITX_RESERVE_MASK); + mtk_hdmi_phy_mask(hdmi_phy, HDMI_CON0, (0xa << RG_HDMITX_DRV_IBIAS), + RG_HDMITX_DRV_IBIAS_MASK); + return 0; +} + +static const struct clk_ops mtk_hdmi_phy_pll_ops = { + .prepare = mtk_hdmi_pll_prepare, + .unprepare = mtk_hdmi_pll_unprepare, + .set_rate = mtk_hdmi_pll_set_rate, + .round_rate = mtk_hdmi_pll_round_rate, + .recalc_rate = mtk_hdmi_pll_recalc_rate, +}; + +static void mtk_hdmi_phy_enable_tmds(struct mtk_hdmi_phy *hdmi_phy) +{ + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON7, RG_HTPLL_AUTOK_EN); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_RLH_EN); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_POSDIV_MASK); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_MBIAS); + usleep_range(80, 100); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_EN); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_CKLDO); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SLDO_MASK); + usleep_range(80, 100); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_MBIAS_LPF_EN); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_POSDIV); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SER_MASK); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_PRED_MASK); + mtk_hdmi_phy_set_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_DRV_MASK); + usleep_range(80, 100); +} + +static void mtk_hdmi_phy_disable_tmds(struct mtk_hdmi_phy *hdmi_phy) +{ + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_DRV_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_PRED_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SER_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_POSDIV); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_MBIAS_LPF_EN); + usleep_range(80, 100); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON0, RG_HDMITX_EN_SLDO_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_TX_CKLDO); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_EN); + usleep_range(80, 100); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON2, RG_HDMITX_EN_MBIAS); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_POSDIV_MASK); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON6, RG_HTPLL_RLH_EN); + mtk_hdmi_phy_clear_bits(hdmi_phy, HDMI_CON7, RG_HTPLL_AUTOK_EN); + usleep_range(80, 100); +} + +struct mtk_hdmi_phy_conf mtk_hdmi_phy_2701_conf = { + .tz_disabled = true, + .hdmi_phy_clk_ops = &mtk_hdmi_phy_pll_ops, + .hdmi_phy_enable_tmds = mtk_hdmi_phy_enable_tmds, + .hdmi_phy_disable_tmds = mtk_hdmi_phy_disable_tmds, +}; + +MODULE_AUTHOR("Chunhui Dai <chunhui.dai@mediatek.com>"); +MODULE_DESCRIPTION("MediaTek HDMI PHY Driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/gpu/drm/mediatek/mtk_mt8173_hdmi_phy.c b/drivers/gpu/drm/mediatek/mtk_mt8173_hdmi_phy.c index 51cb9cfb6646..ed5916b27658 100644 --- a/drivers/gpu/drm/mediatek/mtk_mt8173_hdmi_phy.c +++ b/drivers/gpu/drm/mediatek/mtk_mt8173_hdmi_phy.c @@ -12,15 +12,7 @@ * GNU General Public License for more details. */ -#include <linux/clk.h> -#include <linux/clk-provider.h> -#include <linux/delay.h> -#include <linux/io.h> -#include <linux/mfd/syscon.h> -#include <linux/module.h> -#include <linux/phy/phy.h> -#include <linux/platform_device.h> -#include <linux/types.h> +#include "mtk_hdmi_phy.h" #define HDMI_CON0 0x00 #define RG_HDMITX_PLL_EN BIT(31) @@ -123,20 +115,6 @@ #define RGS_HDMITX_5T1_EDG (0xf << 4) #define RGS_HDMITX_PLUG_TST BIT(0) -struct mtk_hdmi_phy { - void __iomem *regs; - struct device *dev; - struct clk *pll; - struct clk_hw pll_hw; - unsigned long pll_rate; - u8 drv_imp_clk; - u8 drv_imp_d2; - u8 drv_imp_d1; - u8 drv_imp_d0; - u32 ibias; - u32 ibias_up; -}; - static const u8 PREDIV[3][4] = { {0x0, 0x0, 0x0, 0x0}, /* 27Mhz */ {0x1, 0x1, 0x1, 0x1}, /* 74Mhz */ @@ -185,44 +163,6 @@ static const u8 HTPLLBR[3][4] = { {0x1, 0x2, 0x2, 0x1} /* 148Mhz */ }; -static void mtk_hdmi_phy_clear_bits(struct mtk_hdmi_phy *hdmi_phy, u32 offset, - u32 bits) -{ - void __iomem *reg = hdmi_phy->regs + offset; - u32 tmp; - - tmp = readl(reg); - tmp &= ~bits; - writel(tmp, reg); -} - -static void mtk_hdmi_phy_set_bits(struct mtk_hdmi_phy *hdmi_phy, u32 offset, - u32 bits) -{ - void __iomem *reg = hdmi_phy->regs + offset; - u32 tmp; - - tmp = readl(reg); - tmp |= bits; - writel(tmp, reg); -} - -static void mtk_hdmi_phy_mask(struct mtk_hdmi_phy *hdmi_phy, u32 offset, - u32 val, u32 mask) -{ - void __iomem *reg = hdmi_phy->regs + offset; - u32 tmp; - - tmp = readl(reg); - tmp = (tmp & ~mask) | (val & mask); - writel(tmp, reg); -} - -static inline struct mtk_hdmi_phy *to_mtk_hdmi_phy(struct clk_hw *hw) -{ - return container_of(hw, struct mtk_hdmi_phy, pll_hw); -} - static int mtk_hdmi_pll_prepare(struct clk_hw *hw) { struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); @@ -345,29 +285,7 @@ static int mtk_hdmi_pll_set_rate(struct clk_hw *hw, unsigned long rate, return 0; } -static long mtk_hdmi_pll_round_rate(struct clk_hw *hw, unsigned long rate, - unsigned long *parent_rate) -{ - struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); - - hdmi_phy->pll_rate = rate; - if (rate <= 74250000) - *parent_rate = rate; - else - *parent_rate = rate / 2; - - return rate; -} - -static unsigned long mtk_hdmi_pll_recalc_rate(struct clk_hw *hw, - unsigned long parent_rate) -{ - struct mtk_hdmi_phy *hdmi_phy = to_mtk_hdmi_phy(hw); - - return hdmi_phy->pll_rate; -} - -static const struct clk_ops mtk_hdmi_pll_ops = { +static const struct clk_ops mtk_hdmi_phy_pll_ops = { .prepare = mtk_hdmi_pll_prepare, .unprepare = mtk_hdmi_pll_unprepare, .set_rate = mtk_hdmi_pll_set_rate, @@ -390,142 +308,10 @@ static void mtk_hdmi_phy_disable_tmds(struct mtk_hdmi_phy *hdmi_phy) RG_HDMITX_SER_EN); } -static int mtk_hdmi_phy_power_on(struct phy *phy) -{ - struct mtk_hdmi_phy *hdmi_phy = phy_get_drvdata(phy); - int ret; - - ret = clk_prepare_enable(hdmi_phy->pll); - if (ret < 0) - return ret; - - mtk_hdmi_phy_enable_tmds(hdmi_phy); - - return 0; -} - -static int mtk_hdmi_phy_power_off(struct phy *phy) -{ - struct mtk_hdmi_phy *hdmi_phy = phy_get_drvdata(phy); - - mtk_hdmi_phy_disable_tmds(hdmi_phy); - clk_disable_unprepare(hdmi_phy->pll); - - return 0; -} - -static const struct phy_ops mtk_hdmi_phy_ops = { - .power_on = mtk_hdmi_phy_power_on, - .power_off = mtk_hdmi_phy_power_off, - .owner = THIS_MODULE, -}; - -static int mtk_hdmi_phy_probe(struct platform_device *pdev) -{ - struct device *dev = &pdev->dev; - struct mtk_hdmi_phy *hdmi_phy; - struct resource *mem; - struct clk *ref_clk; - const char *ref_clk_name; - struct clk_init_data clk_init = { - .ops = &mtk_hdmi_pll_ops, - .num_parents = 1, - .parent_names = (const char * const *)&ref_clk_name, - .flags = CLK_SET_RATE_PARENT | CLK_SET_RATE_GATE, - }; - struct phy *phy; - struct phy_provider *phy_provider; - int ret; - - hdmi_phy = devm_kzalloc(dev, sizeof(*hdmi_phy), GFP_KERNEL); - if (!hdmi_phy) - return -ENOMEM; - - mem = platform_get_resource(pdev, IORESOURCE_MEM, 0); - hdmi_phy->regs = devm_ioremap_resource(dev, mem); - if (IS_ERR(hdmi_phy->regs)) { - ret = PTR_ERR(hdmi_phy->regs); - dev_err(dev, "Failed to get memory resource: %d\n", ret); - return ret; - } - - ref_clk = devm_clk_get(dev, "pll_ref"); - if (IS_ERR(ref_clk)) { - ret = PTR_ERR(ref_clk); - dev_err(&pdev->dev, "Failed to get PLL reference clock: %d\n", - ret); - return ret; - } - ref_clk_name = __clk_get_name(ref_clk); - - ret = of_property_read_string(dev->of_node, "clock-output-names", - &clk_init.name); - if (ret < 0) { - dev_err(dev, "Failed to read clock-output-names: %d\n", ret); - return ret; - } - - hdmi_phy->pll_hw.init = &clk_init; - hdmi_phy->pll = devm_clk_register(dev, &hdmi_phy->pll_hw); - if (IS_ERR(hdmi_phy->pll)) { - ret = PTR_ERR(hdmi_phy->pll); - dev_err(dev, "Failed to register PLL: %d\n", ret); - return ret; - } - - ret = of_property_read_u32(dev->of_node, "mediatek,ibias", - &hdmi_phy->ibias); - if (ret < 0) { - dev_err(&pdev->dev, "Failed to get ibias: %d\n", ret); - return ret; - } - - ret = of_property_read_u32(dev->of_node, "mediatek,ibias_up", - &hdmi_phy->ibias_up); - if (ret < 0) { - dev_err(&pdev->dev, "Failed to get ibias up: %d\n", ret); - return ret; - } - - dev_info(dev, "Using default TX DRV impedance: 4.2k/36\n"); - hdmi_phy->drv_imp_clk = 0x30; - hdmi_phy->drv_imp_d2 = 0x30; - hdmi_phy->drv_imp_d1 = 0x30; - hdmi_phy->drv_imp_d0 = 0x30; - - phy = devm_phy_create(dev, NULL, &mtk_hdmi_phy_ops); - if (IS_ERR(phy)) { - dev_err(dev, "Failed to create HDMI PHY\n"); - return PTR_ERR(phy); - } - phy_set_drvdata(phy, hdmi_phy); - - phy_provider = devm_of_phy_provider_register(dev, of_phy_simple_xlate); - if (IS_ERR(phy_provider)) - return PTR_ERR(phy_provider); - - hdmi_phy->dev = dev; - return of_clk_add_provider(dev->of_node, of_clk_src_simple_get, - hdmi_phy->pll); -} - -static int mtk_hdmi_phy_remove(struct platform_device *pdev) -{ - return 0; -} - -static const struct of_device_id mtk_hdmi_phy_match[] = { - { .compatible = "mediatek,mt8173-hdmi-phy", }, - {}, -}; - -struct platform_driver mtk_hdmi_phy_driver = { - .probe = mtk_hdmi_phy_probe, - .remove = mtk_hdmi_phy_remove, - .driver = { - .name = "mediatek-hdmi-phy", - .of_match_table = mtk_hdmi_phy_match, - }, +struct mtk_hdmi_phy_conf mtk_hdmi_phy_8173_conf = { + .hdmi_phy_clk_ops = &mtk_hdmi_phy_pll_ops, + .hdmi_phy_enable_tmds = mtk_hdmi_phy_enable_tmds, + .hdmi_phy_disable_tmds = mtk_hdmi_phy_disable_tmds, }; MODULE_AUTHOR("Jie Qiu <jie.qiu@mediatek.com>"); diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile index 261fa79d456d..19ab521d4c3a 100644 --- a/drivers/gpu/drm/msm/Makefile +++ b/drivers/gpu/drm/msm/Makefile @@ -58,7 +58,6 @@ msm-y := \ disp/dpu1/dpu_formats.o \ disp/dpu1/dpu_hw_blk.o \ disp/dpu1/dpu_hw_catalog.o \ - disp/dpu1/dpu_hw_cdm.o \ disp/dpu1/dpu_hw_ctl.o \ disp/dpu1/dpu_hw_interrupts.o \ disp/dpu1/dpu_hw_intf.o \ diff --git a/drivers/gpu/drm/msm/adreno/a2xx.xml.h b/drivers/gpu/drm/msm/adreno/a2xx.xml.h index 4bff0a740c7d..12b0ba270b5e 100644 --- a/drivers/gpu/drm/msm/adreno/a2xx.xml.h +++ b/drivers/gpu/drm/msm/adreno/a2xx.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: diff --git a/drivers/gpu/drm/msm/adreno/a3xx.xml.h b/drivers/gpu/drm/msm/adreno/a3xx.xml.h index 645a19aef399..a89f7bb8b5cc 100644 --- a/drivers/gpu/drm/msm/adreno/a3xx.xml.h +++ b/drivers/gpu/drm/msm/adreno/a3xx.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: diff --git a/drivers/gpu/drm/msm/adreno/a4xx.xml.h b/drivers/gpu/drm/msm/adreno/a4xx.xml.h index 19565e87aa7b..858690f52854 100644 --- a/drivers/gpu/drm/msm/adreno/a4xx.xml.h +++ b/drivers/gpu/drm/msm/adreno/a4xx.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: diff --git a/drivers/gpu/drm/msm/adreno/a5xx.xml.h b/drivers/gpu/drm/msm/adreno/a5xx.xml.h index 182d37ff3794..b4944cc0e62f 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx.xml.h +++ b/drivers/gpu/drm/msm/adreno/a5xx.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: diff --git a/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c b/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c index 059ec7d394d0..d2127b1c4ece 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_debugfs.c @@ -132,14 +132,14 @@ reset_set(void *data, u64 val) if (a5xx_gpu->pm4_bo) { if (a5xx_gpu->pm4_iova) msm_gem_put_iova(a5xx_gpu->pm4_bo, gpu->aspace); - drm_gem_object_unreference(a5xx_gpu->pm4_bo); + drm_gem_object_put(a5xx_gpu->pm4_bo); a5xx_gpu->pm4_bo = NULL; } if (a5xx_gpu->pfp_bo) { if (a5xx_gpu->pfp_iova) msm_gem_put_iova(a5xx_gpu->pfp_bo, gpu->aspace); - drm_gem_object_unreference(a5xx_gpu->pfp_bo); + drm_gem_object_put(a5xx_gpu->pfp_bo); a5xx_gpu->pfp_bo = NULL; } diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index ab1d9308c311..48b5304f460c 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -1234,7 +1234,7 @@ static void a5xx_crashdumper_free(struct msm_gpu *gpu, msm_gem_put_iova(dumper->bo, gpu->aspace); msm_gem_put_vaddr(dumper->bo); - drm_gem_object_unreference(dumper->bo); + drm_gem_object_put(dumper->bo); } static int a5xx_crashdumper_run(struct msm_gpu *gpu, @@ -1436,12 +1436,22 @@ static struct msm_ringbuffer *a5xx_active_ring(struct msm_gpu *gpu) return a5xx_gpu->cur_ring; } -static int a5xx_gpu_busy(struct msm_gpu *gpu, uint64_t *value) +static unsigned long a5xx_gpu_busy(struct msm_gpu *gpu) { - *value = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO, - REG_A5XX_RBBM_PERFCTR_RBBM_0_HI); + u64 busy_cycles, busy_time; - return 0; + busy_cycles = gpu_read64(gpu, REG_A5XX_RBBM_PERFCTR_RBBM_0_LO, + REG_A5XX_RBBM_PERFCTR_RBBM_0_HI); + + busy_time = busy_cycles - gpu->devfreq.busy_cycles; + do_div(busy_time, clk_get_rate(gpu->core_clk) / 1000000); + + gpu->devfreq.busy_cycles = busy_cycles; + + if (WARN_ON(busy_time > ~0LU)) + return ~0LU; + + return (unsigned long)busy_time; } static const struct adreno_gpu_funcs funcs = { diff --git a/drivers/gpu/drm/msm/adreno/a5xx_power.c b/drivers/gpu/drm/msm/adreno/a5xx_power.c index e9c0e56dbec0..7a41e1c147e4 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_power.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_power.c @@ -323,7 +323,7 @@ err: if (a5xx_gpu->gpmu_iova) msm_gem_put_iova(a5xx_gpu->gpmu_bo, gpu->aspace); if (a5xx_gpu->gpmu_bo) - drm_gem_object_unreference(a5xx_gpu->gpmu_bo); + drm_gem_object_put(a5xx_gpu->gpmu_bo); a5xx_gpu->gpmu_bo = NULL; a5xx_gpu->gpmu_iova = 0; diff --git a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c index 970c7963ae29..4c357ead1be6 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_preempt.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_preempt.c @@ -208,6 +208,13 @@ void a5xx_preempt_hw_init(struct msm_gpu *gpu) struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu); int i; + /* Always come up on rb 0 */ + a5xx_gpu->cur_ring = gpu->rb[0]; + + /* No preemption if we only have one ring */ + if (gpu->nr_rings == 1) + return; + for (i = 0; i < gpu->nr_rings; i++) { a5xx_gpu->preempt[i]->wptr = 0; a5xx_gpu->preempt[i]->rptr = 0; @@ -220,9 +227,6 @@ void a5xx_preempt_hw_init(struct msm_gpu *gpu) /* Reset the preemption state */ set_preempt_state(a5xx_gpu, PREEMPT_NONE); - - /* Always come up on rb 0 */ - a5xx_gpu->cur_ring = gpu->rb[0]; } static int preempt_init_ring(struct a5xx_gpu *a5xx_gpu, @@ -272,7 +276,7 @@ void a5xx_preempt_fini(struct msm_gpu *gpu) if (a5xx_gpu->preempt_iova[i]) msm_gem_put_iova(a5xx_gpu->preempt_bo[i], gpu->aspace); - drm_gem_object_unreference(a5xx_gpu->preempt_bo[i]); + drm_gem_object_put(a5xx_gpu->preempt_bo[i]); a5xx_gpu->preempt_bo[i] = NULL; } } diff --git a/drivers/gpu/drm/msm/adreno/a6xx.xml.h b/drivers/gpu/drm/msm/adreno/a6xx.xml.h index 87eab51f7000..a6f7c40454a6 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx.xml.h +++ b/drivers/gpu/drm/msm/adreno/a6xx.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: @@ -268,8 +268,687 @@ enum a6xx_depth_format { DEPTH6_32 = 4, }; +enum a6xx_shader_id { + A6XX_TP0_TMO_DATA = 9, + A6XX_TP0_SMO_DATA = 10, + A6XX_TP0_MIPMAP_BASE_DATA = 11, + A6XX_TP1_TMO_DATA = 25, + A6XX_TP1_SMO_DATA = 26, + A6XX_TP1_MIPMAP_BASE_DATA = 27, + A6XX_SP_INST_DATA = 41, + A6XX_SP_LB_0_DATA = 42, + A6XX_SP_LB_1_DATA = 43, + A6XX_SP_LB_2_DATA = 44, + A6XX_SP_LB_3_DATA = 45, + A6XX_SP_LB_4_DATA = 46, + A6XX_SP_LB_5_DATA = 47, + A6XX_SP_CB_BINDLESS_DATA = 48, + A6XX_SP_CB_LEGACY_DATA = 49, + A6XX_SP_UAV_DATA = 50, + A6XX_SP_INST_TAG = 51, + A6XX_SP_CB_BINDLESS_TAG = 52, + A6XX_SP_TMO_UMO_TAG = 53, + A6XX_SP_SMO_TAG = 54, + A6XX_SP_STATE_DATA = 55, + A6XX_HLSQ_CHUNK_CVS_RAM = 73, + A6XX_HLSQ_CHUNK_CPS_RAM = 74, + A6XX_HLSQ_CHUNK_CVS_RAM_TAG = 75, + A6XX_HLSQ_CHUNK_CPS_RAM_TAG = 76, + A6XX_HLSQ_ICB_CVS_CB_BASE_TAG = 77, + A6XX_HLSQ_ICB_CPS_CB_BASE_TAG = 78, + A6XX_HLSQ_CVS_MISC_RAM = 80, + A6XX_HLSQ_CPS_MISC_RAM = 81, + A6XX_HLSQ_INST_RAM = 82, + A6XX_HLSQ_GFX_CVS_CONST_RAM = 83, + A6XX_HLSQ_GFX_CPS_CONST_RAM = 84, + A6XX_HLSQ_CVS_MISC_RAM_TAG = 85, + A6XX_HLSQ_CPS_MISC_RAM_TAG = 86, + A6XX_HLSQ_INST_RAM_TAG = 87, + A6XX_HLSQ_GFX_CVS_CONST_RAM_TAG = 88, + A6XX_HLSQ_GFX_CPS_CONST_RAM_TAG = 89, + A6XX_HLSQ_PWR_REST_RAM = 90, + A6XX_HLSQ_PWR_REST_TAG = 91, + A6XX_HLSQ_DATAPATH_META = 96, + A6XX_HLSQ_FRONTEND_META = 97, + A6XX_HLSQ_INDIRECT_META = 98, + A6XX_HLSQ_BACKEND_META = 99, +}; + +enum a6xx_debugbus_id { + A6XX_DBGBUS_CP = 1, + A6XX_DBGBUS_RBBM = 2, + A6XX_DBGBUS_VBIF = 3, + A6XX_DBGBUS_HLSQ = 4, + A6XX_DBGBUS_UCHE = 5, + A6XX_DBGBUS_DPM = 6, + A6XX_DBGBUS_TESS = 7, + A6XX_DBGBUS_PC = 8, + A6XX_DBGBUS_VFDP = 9, + A6XX_DBGBUS_VPC = 10, + A6XX_DBGBUS_TSE = 11, + A6XX_DBGBUS_RAS = 12, + A6XX_DBGBUS_VSC = 13, + A6XX_DBGBUS_COM = 14, + A6XX_DBGBUS_LRZ = 16, + A6XX_DBGBUS_A2D = 17, + A6XX_DBGBUS_CCUFCHE = 18, + A6XX_DBGBUS_GMU_CX = 19, + A6XX_DBGBUS_RBP = 20, + A6XX_DBGBUS_DCS = 21, + A6XX_DBGBUS_DBGC = 22, + A6XX_DBGBUS_CX = 23, + A6XX_DBGBUS_GMU_GX = 24, + A6XX_DBGBUS_TPFCHE = 25, + A6XX_DBGBUS_GBIF_GX = 26, + A6XX_DBGBUS_GPC = 29, + A6XX_DBGBUS_LARC = 30, + A6XX_DBGBUS_HLSQ_SPTP = 31, + A6XX_DBGBUS_RB_0 = 32, + A6XX_DBGBUS_RB_1 = 33, + A6XX_DBGBUS_UCHE_WRAPPER = 36, + A6XX_DBGBUS_CCU_0 = 40, + A6XX_DBGBUS_CCU_1 = 41, + A6XX_DBGBUS_VFD_0 = 56, + A6XX_DBGBUS_VFD_1 = 57, + A6XX_DBGBUS_VFD_2 = 58, + A6XX_DBGBUS_VFD_3 = 59, + A6XX_DBGBUS_SP_0 = 64, + A6XX_DBGBUS_SP_1 = 65, + A6XX_DBGBUS_TPL1_0 = 72, + A6XX_DBGBUS_TPL1_1 = 73, + A6XX_DBGBUS_TPL1_2 = 74, + A6XX_DBGBUS_TPL1_3 = 75, +}; + enum a6xx_cp_perfcounter_select { PERF_CP_ALWAYS_COUNT = 0, + PERF_CP_BUSY_GFX_CORE_IDLE = 1, + PERF_CP_BUSY_CYCLES = 2, + PERF_CP_NUM_PREEMPTIONS = 3, + PERF_CP_PREEMPTION_REACTION_DELAY = 4, + PERF_CP_PREEMPTION_SWITCH_OUT_TIME = 5, + PERF_CP_PREEMPTION_SWITCH_IN_TIME = 6, + PERF_CP_DEAD_DRAWS_IN_BIN_RENDER = 7, + PERF_CP_PREDICATED_DRAWS_KILLED = 8, + PERF_CP_MODE_SWITCH = 9, + PERF_CP_ZPASS_DONE = 10, + PERF_CP_CONTEXT_DONE = 11, + PERF_CP_CACHE_FLUSH = 12, + PERF_CP_LONG_PREEMPTIONS = 13, + PERF_CP_SQE_I_CACHE_STARVE = 14, + PERF_CP_SQE_IDLE = 15, + PERF_CP_SQE_PM4_STARVE_RB_IB = 16, + PERF_CP_SQE_PM4_STARVE_SDS = 17, + PERF_CP_SQE_MRB_STARVE = 18, + PERF_CP_SQE_RRB_STARVE = 19, + PERF_CP_SQE_VSD_STARVE = 20, + PERF_CP_VSD_DECODE_STARVE = 21, + PERF_CP_SQE_PIPE_OUT_STALL = 22, + PERF_CP_SQE_SYNC_STALL = 23, + PERF_CP_SQE_PM4_WFI_STALL = 24, + PERF_CP_SQE_SYS_WFI_STALL = 25, + PERF_CP_SQE_T4_EXEC = 26, + PERF_CP_SQE_LOAD_STATE_EXEC = 27, + PERF_CP_SQE_SAVE_SDS_STATE = 28, + PERF_CP_SQE_DRAW_EXEC = 29, + PERF_CP_SQE_CTXT_REG_BUNCH_EXEC = 30, + PERF_CP_SQE_EXEC_PROFILED = 31, + PERF_CP_MEMORY_POOL_EMPTY = 32, + PERF_CP_MEMORY_POOL_SYNC_STALL = 33, + PERF_CP_MEMORY_POOL_ABOVE_THRESH = 34, + PERF_CP_AHB_WR_STALL_PRE_DRAWS = 35, + PERF_CP_AHB_STALL_SQE_GMU = 36, + PERF_CP_AHB_STALL_SQE_WR_OTHER = 37, + PERF_CP_AHB_STALL_SQE_RD_OTHER = 38, + PERF_CP_CLUSTER0_EMPTY = 39, + PERF_CP_CLUSTER1_EMPTY = 40, + PERF_CP_CLUSTER2_EMPTY = 41, + PERF_CP_CLUSTER3_EMPTY = 42, + PERF_CP_CLUSTER4_EMPTY = 43, + PERF_CP_CLUSTER5_EMPTY = 44, + PERF_CP_PM4_DATA = 45, + PERF_CP_PM4_HEADERS = 46, + PERF_CP_VBIF_READ_BEATS = 47, + PERF_CP_VBIF_WRITE_BEATS = 48, + PERF_CP_SQE_INSTR_COUNTER = 49, +}; + +enum a6xx_rbbm_perfcounter_select { + PERF_RBBM_ALWAYS_COUNT = 0, + PERF_RBBM_ALWAYS_ON = 1, + PERF_RBBM_TSE_BUSY = 2, + PERF_RBBM_RAS_BUSY = 3, + PERF_RBBM_PC_DCALL_BUSY = 4, + PERF_RBBM_PC_VSD_BUSY = 5, + PERF_RBBM_STATUS_MASKED = 6, + PERF_RBBM_COM_BUSY = 7, + PERF_RBBM_DCOM_BUSY = 8, + PERF_RBBM_VBIF_BUSY = 9, + PERF_RBBM_VSC_BUSY = 10, + PERF_RBBM_TESS_BUSY = 11, + PERF_RBBM_UCHE_BUSY = 12, + PERF_RBBM_HLSQ_BUSY = 13, +}; + +enum a6xx_pc_perfcounter_select { + PERF_PC_BUSY_CYCLES = 0, + PERF_PC_WORKING_CYCLES = 1, + PERF_PC_STALL_CYCLES_VFD = 2, + PERF_PC_STALL_CYCLES_TSE = 3, + PERF_PC_STALL_CYCLES_VPC = 4, + PERF_PC_STALL_CYCLES_UCHE = 5, + PERF_PC_STALL_CYCLES_TESS = 6, + PERF_PC_STALL_CYCLES_TSE_ONLY = 7, + PERF_PC_STALL_CYCLES_VPC_ONLY = 8, + PERF_PC_PASS1_TF_STALL_CYCLES = 9, + PERF_PC_STARVE_CYCLES_FOR_INDEX = 10, + PERF_PC_STARVE_CYCLES_FOR_TESS_FACTOR = 11, + PERF_PC_STARVE_CYCLES_FOR_VIZ_STREAM = 12, + PERF_PC_STARVE_CYCLES_FOR_POSITION = 13, + PERF_PC_STARVE_CYCLES_DI = 14, + PERF_PC_VIS_STREAMS_LOADED = 15, + PERF_PC_INSTANCES = 16, + PERF_PC_VPC_PRIMITIVES = 17, + PERF_PC_DEAD_PRIM = 18, + PERF_PC_LIVE_PRIM = 19, + PERF_PC_VERTEX_HITS = 20, + PERF_PC_IA_VERTICES = 21, + PERF_PC_IA_PRIMITIVES = 22, + PERF_PC_GS_PRIMITIVES = 23, + PERF_PC_HS_INVOCATIONS = 24, + PERF_PC_DS_INVOCATIONS = 25, + PERF_PC_VS_INVOCATIONS = 26, + PERF_PC_GS_INVOCATIONS = 27, + PERF_PC_DS_PRIMITIVES = 28, + PERF_PC_VPC_POS_DATA_TRANSACTION = 29, + PERF_PC_3D_DRAWCALLS = 30, + PERF_PC_2D_DRAWCALLS = 31, + PERF_PC_NON_DRAWCALL_GLOBAL_EVENTS = 32, + PERF_TESS_BUSY_CYCLES = 33, + PERF_TESS_WORKING_CYCLES = 34, + PERF_TESS_STALL_CYCLES_PC = 35, + PERF_TESS_STARVE_CYCLES_PC = 36, + PERF_PC_TSE_TRANSACTION = 37, + PERF_PC_TSE_VERTEX = 38, + PERF_PC_TESS_PC_UV_TRANS = 39, + PERF_PC_TESS_PC_UV_PATCHES = 40, + PERF_PC_TESS_FACTOR_TRANS = 41, +}; + +enum a6xx_vfd_perfcounter_select { + PERF_VFD_BUSY_CYCLES = 0, + PERF_VFD_STALL_CYCLES_UCHE = 1, + PERF_VFD_STALL_CYCLES_VPC_ALLOC = 2, + PERF_VFD_STALL_CYCLES_SP_INFO = 3, + PERF_VFD_STALL_CYCLES_SP_ATTR = 4, + PERF_VFD_STARVE_CYCLES_UCHE = 5, + PERF_VFD_RBUFFER_FULL = 6, + PERF_VFD_ATTR_INFO_FIFO_FULL = 7, + PERF_VFD_DECODED_ATTRIBUTE_BYTES = 8, + PERF_VFD_NUM_ATTRIBUTES = 9, + PERF_VFD_UPPER_SHADER_FIBERS = 10, + PERF_VFD_LOWER_SHADER_FIBERS = 11, + PERF_VFD_MODE_0_FIBERS = 12, + PERF_VFD_MODE_1_FIBERS = 13, + PERF_VFD_MODE_2_FIBERS = 14, + PERF_VFD_MODE_3_FIBERS = 15, + PERF_VFD_MODE_4_FIBERS = 16, + PERF_VFD_TOTAL_VERTICES = 17, + PERF_VFDP_STALL_CYCLES_VFD = 18, + PERF_VFDP_STALL_CYCLES_VFD_INDEX = 19, + PERF_VFDP_STALL_CYCLES_VFD_PROG = 20, + PERF_VFDP_STARVE_CYCLES_PC = 21, + PERF_VFDP_VS_STAGE_WAVES = 22, +}; + +enum a6xx_hslq_perfcounter_select { + PERF_HLSQ_BUSY_CYCLES = 0, + PERF_HLSQ_STALL_CYCLES_UCHE = 1, + PERF_HLSQ_STALL_CYCLES_SP_STATE = 2, + PERF_HLSQ_STALL_CYCLES_SP_FS_STAGE = 3, + PERF_HLSQ_UCHE_LATENCY_CYCLES = 4, + PERF_HLSQ_UCHE_LATENCY_COUNT = 5, + PERF_HLSQ_FS_STAGE_1X_WAVES = 6, + PERF_HLSQ_FS_STAGE_2X_WAVES = 7, + PERF_HLSQ_QUADS = 8, + PERF_HLSQ_CS_INVOCATIONS = 9, + PERF_HLSQ_COMPUTE_DRAWCALLS = 10, + PERF_HLSQ_FS_DATA_WAIT_PROGRAMMING = 11, + PERF_HLSQ_DUAL_FS_PROG_ACTIVE = 12, + PERF_HLSQ_DUAL_VS_PROG_ACTIVE = 13, + PERF_HLSQ_FS_BATCH_COUNT_ZERO = 14, + PERF_HLSQ_VS_BATCH_COUNT_ZERO = 15, + PERF_HLSQ_WAVE_PENDING_NO_QUAD = 16, + PERF_HLSQ_WAVE_PENDING_NO_PRIM_BASE = 17, + PERF_HLSQ_STALL_CYCLES_VPC = 18, + PERF_HLSQ_PIXELS = 19, + PERF_HLSQ_DRAW_MODE_SWITCH_VSFS_SYNC = 20, +}; + +enum a6xx_vpc_perfcounter_select { + PERF_VPC_BUSY_CYCLES = 0, + PERF_VPC_WORKING_CYCLES = 1, + PERF_VPC_STALL_CYCLES_UCHE = 2, + PERF_VPC_STALL_CYCLES_VFD_WACK = 3, + PERF_VPC_STALL_CYCLES_HLSQ_PRIM_ALLOC = 4, + PERF_VPC_STALL_CYCLES_PC = 5, + PERF_VPC_STALL_CYCLES_SP_LM = 6, + PERF_VPC_STARVE_CYCLES_SP = 7, + PERF_VPC_STARVE_CYCLES_LRZ = 8, + PERF_VPC_PC_PRIMITIVES = 9, + PERF_VPC_SP_COMPONENTS = 10, + PERF_VPC_STALL_CYCLES_VPCRAM_POS = 11, + PERF_VPC_LRZ_ASSIGN_PRIMITIVES = 12, + PERF_VPC_RB_VISIBLE_PRIMITIVES = 13, + PERF_VPC_LM_TRANSACTION = 14, + PERF_VPC_STREAMOUT_TRANSACTION = 15, + PERF_VPC_VS_BUSY_CYCLES = 16, + PERF_VPC_PS_BUSY_CYCLES = 17, + PERF_VPC_VS_WORKING_CYCLES = 18, + PERF_VPC_PS_WORKING_CYCLES = 19, + PERF_VPC_STARVE_CYCLES_RB = 20, + PERF_VPC_NUM_VPCRAM_READ_POS = 21, + PERF_VPC_WIT_FULL_CYCLES = 22, + PERF_VPC_VPCRAM_FULL_CYCLES = 23, + PERF_VPC_LM_FULL_WAIT_FOR_INTP_END = 24, + PERF_VPC_NUM_VPCRAM_WRITE = 25, + PERF_VPC_NUM_VPCRAM_READ_SO = 26, + PERF_VPC_NUM_ATTR_REQ_LM = 27, +}; + +enum a6xx_tse_perfcounter_select { + PERF_TSE_BUSY_CYCLES = 0, + PERF_TSE_CLIPPING_CYCLES = 1, + PERF_TSE_STALL_CYCLES_RAS = 2, + PERF_TSE_STALL_CYCLES_LRZ_BARYPLANE = 3, + PERF_TSE_STALL_CYCLES_LRZ_ZPLANE = 4, + PERF_TSE_STARVE_CYCLES_PC = 5, + PERF_TSE_INPUT_PRIM = 6, + PERF_TSE_INPUT_NULL_PRIM = 7, + PERF_TSE_TRIVAL_REJ_PRIM = 8, + PERF_TSE_CLIPPED_PRIM = 9, + PERF_TSE_ZERO_AREA_PRIM = 10, + PERF_TSE_FACENESS_CULLED_PRIM = 11, + PERF_TSE_ZERO_PIXEL_PRIM = 12, + PERF_TSE_OUTPUT_NULL_PRIM = 13, + PERF_TSE_OUTPUT_VISIBLE_PRIM = 14, + PERF_TSE_CINVOCATION = 15, + PERF_TSE_CPRIMITIVES = 16, + PERF_TSE_2D_INPUT_PRIM = 17, + PERF_TSE_2D_ALIVE_CYCLES = 18, + PERF_TSE_CLIP_PLANES = 19, +}; + +enum a6xx_ras_perfcounter_select { + PERF_RAS_BUSY_CYCLES = 0, + PERF_RAS_SUPERTILE_ACTIVE_CYCLES = 1, + PERF_RAS_STALL_CYCLES_LRZ = 2, + PERF_RAS_STARVE_CYCLES_TSE = 3, + PERF_RAS_SUPER_TILES = 4, + PERF_RAS_8X4_TILES = 5, + PERF_RAS_MASKGEN_ACTIVE = 6, + PERF_RAS_FULLY_COVERED_SUPER_TILES = 7, + PERF_RAS_FULLY_COVERED_8X4_TILES = 8, + PERF_RAS_PRIM_KILLED_INVISILBE = 9, + PERF_RAS_SUPERTILE_GEN_ACTIVE_CYCLES = 10, + PERF_RAS_LRZ_INTF_WORKING_CYCLES = 11, + PERF_RAS_BLOCKS = 12, +}; + +enum a6xx_uche_perfcounter_select { + PERF_UCHE_BUSY_CYCLES = 0, + PERF_UCHE_STALL_CYCLES_ARBITER = 1, + PERF_UCHE_VBIF_LATENCY_CYCLES = 2, + PERF_UCHE_VBIF_LATENCY_SAMPLES = 3, + PERF_UCHE_VBIF_READ_BEATS_TP = 4, + PERF_UCHE_VBIF_READ_BEATS_VFD = 5, + PERF_UCHE_VBIF_READ_BEATS_HLSQ = 6, + PERF_UCHE_VBIF_READ_BEATS_LRZ = 7, + PERF_UCHE_VBIF_READ_BEATS_SP = 8, + PERF_UCHE_READ_REQUESTS_TP = 9, + PERF_UCHE_READ_REQUESTS_VFD = 10, + PERF_UCHE_READ_REQUESTS_HLSQ = 11, + PERF_UCHE_READ_REQUESTS_LRZ = 12, + PERF_UCHE_READ_REQUESTS_SP = 13, + PERF_UCHE_WRITE_REQUESTS_LRZ = 14, + PERF_UCHE_WRITE_REQUESTS_SP = 15, + PERF_UCHE_WRITE_REQUESTS_VPC = 16, + PERF_UCHE_WRITE_REQUESTS_VSC = 17, + PERF_UCHE_EVICTS = 18, + PERF_UCHE_BANK_REQ0 = 19, + PERF_UCHE_BANK_REQ1 = 20, + PERF_UCHE_BANK_REQ2 = 21, + PERF_UCHE_BANK_REQ3 = 22, + PERF_UCHE_BANK_REQ4 = 23, + PERF_UCHE_BANK_REQ5 = 24, + PERF_UCHE_BANK_REQ6 = 25, + PERF_UCHE_BANK_REQ7 = 26, + PERF_UCHE_VBIF_READ_BEATS_CH0 = 27, + PERF_UCHE_VBIF_READ_BEATS_CH1 = 28, + PERF_UCHE_GMEM_READ_BEATS = 29, + PERF_UCHE_TPH_REF_FULL = 30, + PERF_UCHE_TPH_VICTIM_FULL = 31, + PERF_UCHE_TPH_EXT_FULL = 32, + PERF_UCHE_VBIF_STALL_WRITE_DATA = 33, + PERF_UCHE_DCMP_LATENCY_SAMPLES = 34, + PERF_UCHE_DCMP_LATENCY_CYCLES = 35, + PERF_UCHE_VBIF_READ_BEATS_PC = 36, + PERF_UCHE_READ_REQUESTS_PC = 37, + PERF_UCHE_RAM_READ_REQ = 38, + PERF_UCHE_RAM_WRITE_REQ = 39, +}; + +enum a6xx_tp_perfcounter_select { + PERF_TP_BUSY_CYCLES = 0, + PERF_TP_STALL_CYCLES_UCHE = 1, + PERF_TP_LATENCY_CYCLES = 2, + PERF_TP_LATENCY_TRANS = 3, + PERF_TP_FLAG_CACHE_REQUEST_SAMPLES = 4, + PERF_TP_FLAG_CACHE_REQUEST_LATENCY = 5, + PERF_TP_L1_CACHELINE_REQUESTS = 6, + PERF_TP_L1_CACHELINE_MISSES = 7, + PERF_TP_SP_TP_TRANS = 8, + PERF_TP_TP_SP_TRANS = 9, + PERF_TP_OUTPUT_PIXELS = 10, + PERF_TP_FILTER_WORKLOAD_16BIT = 11, + PERF_TP_FILTER_WORKLOAD_32BIT = 12, + PERF_TP_QUADS_RECEIVED = 13, + PERF_TP_QUADS_OFFSET = 14, + PERF_TP_QUADS_SHADOW = 15, + PERF_TP_QUADS_ARRAY = 16, + PERF_TP_QUADS_GRADIENT = 17, + PERF_TP_QUADS_1D = 18, + PERF_TP_QUADS_2D = 19, + PERF_TP_QUADS_BUFFER = 20, + PERF_TP_QUADS_3D = 21, + PERF_TP_QUADS_CUBE = 22, + PERF_TP_DIVERGENT_QUADS_RECEIVED = 23, + PERF_TP_PRT_NON_RESIDENT_EVENTS = 24, + PERF_TP_OUTPUT_PIXELS_POINT = 25, + PERF_TP_OUTPUT_PIXELS_BILINEAR = 26, + PERF_TP_OUTPUT_PIXELS_MIP = 27, + PERF_TP_OUTPUT_PIXELS_ANISO = 28, + PERF_TP_OUTPUT_PIXELS_ZERO_LOD = 29, + PERF_TP_FLAG_CACHE_REQUESTS = 30, + PERF_TP_FLAG_CACHE_MISSES = 31, + PERF_TP_L1_5_L2_REQUESTS = 32, + PERF_TP_2D_OUTPUT_PIXELS = 33, + PERF_TP_2D_OUTPUT_PIXELS_POINT = 34, + PERF_TP_2D_OUTPUT_PIXELS_BILINEAR = 35, + PERF_TP_2D_FILTER_WORKLOAD_16BIT = 36, + PERF_TP_2D_FILTER_WORKLOAD_32BIT = 37, + PERF_TP_TPA2TPC_TRANS = 38, + PERF_TP_L1_MISSES_ASTC_1TILE = 39, + PERF_TP_L1_MISSES_ASTC_2TILE = 40, + PERF_TP_L1_MISSES_ASTC_4TILE = 41, + PERF_TP_L1_5_L2_COMPRESS_REQS = 42, + PERF_TP_L1_5_L2_COMPRESS_MISS = 43, + PERF_TP_L1_BANK_CONFLICT = 44, + PERF_TP_L1_5_MISS_LATENCY_CYCLES = 45, + PERF_TP_L1_5_MISS_LATENCY_TRANS = 46, + PERF_TP_QUADS_CONSTANT_MULTIPLIED = 47, + PERF_TP_FRONTEND_WORKING_CYCLES = 48, + PERF_TP_L1_TAG_WORKING_CYCLES = 49, + PERF_TP_L1_DATA_WRITE_WORKING_CYCLES = 50, + PERF_TP_PRE_L1_DECOM_WORKING_CYCLES = 51, + PERF_TP_BACKEND_WORKING_CYCLES = 52, + PERF_TP_FLAG_CACHE_WORKING_CYCLES = 53, + PERF_TP_L1_5_CACHE_WORKING_CYCLES = 54, + PERF_TP_STARVE_CYCLES_SP = 55, + PERF_TP_STARVE_CYCLES_UCHE = 56, +}; + +enum a6xx_sp_perfcounter_select { + PERF_SP_BUSY_CYCLES = 0, + PERF_SP_ALU_WORKING_CYCLES = 1, + PERF_SP_EFU_WORKING_CYCLES = 2, + PERF_SP_STALL_CYCLES_VPC = 3, + PERF_SP_STALL_CYCLES_TP = 4, + PERF_SP_STALL_CYCLES_UCHE = 5, + PERF_SP_STALL_CYCLES_RB = 6, + PERF_SP_NON_EXECUTION_CYCLES = 7, + PERF_SP_WAVE_CONTEXTS = 8, + PERF_SP_WAVE_CONTEXT_CYCLES = 9, + PERF_SP_FS_STAGE_WAVE_CYCLES = 10, + PERF_SP_FS_STAGE_WAVE_SAMPLES = 11, + PERF_SP_VS_STAGE_WAVE_CYCLES = 12, + PERF_SP_VS_STAGE_WAVE_SAMPLES = 13, + PERF_SP_FS_STAGE_DURATION_CYCLES = 14, + PERF_SP_VS_STAGE_DURATION_CYCLES = 15, + PERF_SP_WAVE_CTRL_CYCLES = 16, + PERF_SP_WAVE_LOAD_CYCLES = 17, + PERF_SP_WAVE_EMIT_CYCLES = 18, + PERF_SP_WAVE_NOP_CYCLES = 19, + PERF_SP_WAVE_WAIT_CYCLES = 20, + PERF_SP_WAVE_FETCH_CYCLES = 21, + PERF_SP_WAVE_IDLE_CYCLES = 22, + PERF_SP_WAVE_END_CYCLES = 23, + PERF_SP_WAVE_LONG_SYNC_CYCLES = 24, + PERF_SP_WAVE_SHORT_SYNC_CYCLES = 25, + PERF_SP_WAVE_JOIN_CYCLES = 26, + PERF_SP_LM_LOAD_INSTRUCTIONS = 27, + PERF_SP_LM_STORE_INSTRUCTIONS = 28, + PERF_SP_LM_ATOMICS = 29, + PERF_SP_GM_LOAD_INSTRUCTIONS = 30, + PERF_SP_GM_STORE_INSTRUCTIONS = 31, + PERF_SP_GM_ATOMICS = 32, + PERF_SP_VS_STAGE_TEX_INSTRUCTIONS = 33, + PERF_SP_VS_STAGE_EFU_INSTRUCTIONS = 34, + PERF_SP_VS_STAGE_FULL_ALU_INSTRUCTIONS = 35, + PERF_SP_VS_STAGE_HALF_ALU_INSTRUCTIONS = 36, + PERF_SP_FS_STAGE_TEX_INSTRUCTIONS = 37, + PERF_SP_FS_STAGE_CFLOW_INSTRUCTIONS = 38, + PERF_SP_FS_STAGE_EFU_INSTRUCTIONS = 39, + PERF_SP_FS_STAGE_FULL_ALU_INSTRUCTIONS = 40, + PERF_SP_FS_STAGE_HALF_ALU_INSTRUCTIONS = 41, + PERF_SP_FS_STAGE_BARY_INSTRUCTIONS = 42, + PERF_SP_VS_INSTRUCTIONS = 43, + PERF_SP_FS_INSTRUCTIONS = 44, + PERF_SP_ADDR_LOCK_COUNT = 45, + PERF_SP_UCHE_READ_TRANS = 46, + PERF_SP_UCHE_WRITE_TRANS = 47, + PERF_SP_EXPORT_VPC_TRANS = 48, + PERF_SP_EXPORT_RB_TRANS = 49, + PERF_SP_PIXELS_KILLED = 50, + PERF_SP_ICL1_REQUESTS = 51, + PERF_SP_ICL1_MISSES = 52, + PERF_SP_HS_INSTRUCTIONS = 53, + PERF_SP_DS_INSTRUCTIONS = 54, + PERF_SP_GS_INSTRUCTIONS = 55, + PERF_SP_CS_INSTRUCTIONS = 56, + PERF_SP_GPR_READ = 57, + PERF_SP_GPR_WRITE = 58, + PERF_SP_FS_STAGE_HALF_EFU_INSTRUCTIONS = 59, + PERF_SP_VS_STAGE_HALF_EFU_INSTRUCTIONS = 60, + PERF_SP_LM_BANK_CONFLICTS = 61, + PERF_SP_TEX_CONTROL_WORKING_CYCLES = 62, + PERF_SP_LOAD_CONTROL_WORKING_CYCLES = 63, + PERF_SP_FLOW_CONTROL_WORKING_CYCLES = 64, + PERF_SP_LM_WORKING_CYCLES = 65, + PERF_SP_DISPATCHER_WORKING_CYCLES = 66, + PERF_SP_SEQUENCER_WORKING_CYCLES = 67, + PERF_SP_LOW_EFFICIENCY_STARVED_BY_TP = 68, + PERF_SP_STARVE_CYCLES_HLSQ = 69, + PERF_SP_NON_EXECUTION_LS_CYCLES = 70, + PERF_SP_WORKING_EU = 71, + PERF_SP_ANY_EU_WORKING = 72, + PERF_SP_WORKING_EU_FS_STAGE = 73, + PERF_SP_ANY_EU_WORKING_FS_STAGE = 74, + PERF_SP_WORKING_EU_VS_STAGE = 75, + PERF_SP_ANY_EU_WORKING_VS_STAGE = 76, + PERF_SP_WORKING_EU_CS_STAGE = 77, + PERF_SP_ANY_EU_WORKING_CS_STAGE = 78, + PERF_SP_GPR_READ_PREFETCH = 79, + PERF_SP_GPR_READ_CONFLICT = 80, + PERF_SP_GPR_WRITE_CONFLICT = 81, + PERF_SP_GM_LOAD_LATENCY_CYCLES = 82, + PERF_SP_GM_LOAD_LATENCY_SAMPLES = 83, + PERF_SP_EXECUTABLE_WAVES = 84, +}; + +enum a6xx_rb_perfcounter_select { + PERF_RB_BUSY_CYCLES = 0, + PERF_RB_STALL_CYCLES_HLSQ = 1, + PERF_RB_STALL_CYCLES_FIFO0_FULL = 2, + PERF_RB_STALL_CYCLES_FIFO1_FULL = 3, + PERF_RB_STALL_CYCLES_FIFO2_FULL = 4, + PERF_RB_STARVE_CYCLES_SP = 5, + PERF_RB_STARVE_CYCLES_LRZ_TILE = 6, + PERF_RB_STARVE_CYCLES_CCU = 7, + PERF_RB_STARVE_CYCLES_Z_PLANE = 8, + PERF_RB_STARVE_CYCLES_BARY_PLANE = 9, + PERF_RB_Z_WORKLOAD = 10, + PERF_RB_HLSQ_ACTIVE = 11, + PERF_RB_Z_READ = 12, + PERF_RB_Z_WRITE = 13, + PERF_RB_C_READ = 14, + PERF_RB_C_WRITE = 15, + PERF_RB_TOTAL_PASS = 16, + PERF_RB_Z_PASS = 17, + PERF_RB_Z_FAIL = 18, + PERF_RB_S_FAIL = 19, + PERF_RB_BLENDED_FXP_COMPONENTS = 20, + PERF_RB_BLENDED_FP16_COMPONENTS = 21, + PERF_RB_PS_INVOCATIONS = 22, + PERF_RB_2D_ALIVE_CYCLES = 23, + PERF_RB_2D_STALL_CYCLES_A2D = 24, + PERF_RB_2D_STARVE_CYCLES_SRC = 25, + PERF_RB_2D_STARVE_CYCLES_SP = 26, + PERF_RB_2D_STARVE_CYCLES_DST = 27, + PERF_RB_2D_VALID_PIXELS = 28, + PERF_RB_3D_PIXELS = 29, + PERF_RB_BLENDER_WORKING_CYCLES = 30, + PERF_RB_ZPROC_WORKING_CYCLES = 31, + PERF_RB_CPROC_WORKING_CYCLES = 32, + PERF_RB_SAMPLER_WORKING_CYCLES = 33, + PERF_RB_STALL_CYCLES_CCU_COLOR_READ = 34, + PERF_RB_STALL_CYCLES_CCU_COLOR_WRITE = 35, + PERF_RB_STALL_CYCLES_CCU_DEPTH_READ = 36, + PERF_RB_STALL_CYCLES_CCU_DEPTH_WRITE = 37, + PERF_RB_STALL_CYCLES_VPC = 38, + PERF_RB_2D_INPUT_TRANS = 39, + PERF_RB_2D_OUTPUT_RB_DST_TRANS = 40, + PERF_RB_2D_OUTPUT_RB_SRC_TRANS = 41, + PERF_RB_BLENDED_FP32_COMPONENTS = 42, + PERF_RB_COLOR_PIX_TILES = 43, + PERF_RB_STALL_CYCLES_CCU = 44, + PERF_RB_EARLY_Z_ARB3_GRANT = 45, + PERF_RB_LATE_Z_ARB3_GRANT = 46, + PERF_RB_EARLY_Z_SKIP_GRANT = 47, +}; + +enum a6xx_vsc_perfcounter_select { + PERF_VSC_BUSY_CYCLES = 0, + PERF_VSC_WORKING_CYCLES = 1, + PERF_VSC_STALL_CYCLES_UCHE = 2, + PERF_VSC_EOT_NUM = 3, + PERF_VSC_INPUT_TILES = 4, +}; + +enum a6xx_ccu_perfcounter_select { + PERF_CCU_BUSY_CYCLES = 0, + PERF_CCU_STALL_CYCLES_RB_DEPTH_RETURN = 1, + PERF_CCU_STALL_CYCLES_RB_COLOR_RETURN = 2, + PERF_CCU_STARVE_CYCLES_FLAG_RETURN = 3, + PERF_CCU_DEPTH_BLOCKS = 4, + PERF_CCU_COLOR_BLOCKS = 5, + PERF_CCU_DEPTH_BLOCK_HIT = 6, + PERF_CCU_COLOR_BLOCK_HIT = 7, + PERF_CCU_PARTIAL_BLOCK_READ = 8, + PERF_CCU_GMEM_READ = 9, + PERF_CCU_GMEM_WRITE = 10, + PERF_CCU_DEPTH_READ_FLAG0_COUNT = 11, + PERF_CCU_DEPTH_READ_FLAG1_COUNT = 12, + PERF_CCU_DEPTH_READ_FLAG2_COUNT = 13, + PERF_CCU_DEPTH_READ_FLAG3_COUNT = 14, + PERF_CCU_DEPTH_READ_FLAG4_COUNT = 15, + PERF_CCU_DEPTH_READ_FLAG5_COUNT = 16, + PERF_CCU_DEPTH_READ_FLAG6_COUNT = 17, + PERF_CCU_DEPTH_READ_FLAG8_COUNT = 18, + PERF_CCU_COLOR_READ_FLAG0_COUNT = 19, + PERF_CCU_COLOR_READ_FLAG1_COUNT = 20, + PERF_CCU_COLOR_READ_FLAG2_COUNT = 21, + PERF_CCU_COLOR_READ_FLAG3_COUNT = 22, + PERF_CCU_COLOR_READ_FLAG4_COUNT = 23, + PERF_CCU_COLOR_READ_FLAG5_COUNT = 24, + PERF_CCU_COLOR_READ_FLAG6_COUNT = 25, + PERF_CCU_COLOR_READ_FLAG8_COUNT = 26, + PERF_CCU_2D_RD_REQ = 27, + PERF_CCU_2D_WR_REQ = 28, +}; + +enum a6xx_lrz_perfcounter_select { + PERF_LRZ_BUSY_CYCLES = 0, + PERF_LRZ_STARVE_CYCLES_RAS = 1, + PERF_LRZ_STALL_CYCLES_RB = 2, + PERF_LRZ_STALL_CYCLES_VSC = 3, + PERF_LRZ_STALL_CYCLES_VPC = 4, + PERF_LRZ_STALL_CYCLES_FLAG_PREFETCH = 5, + PERF_LRZ_STALL_CYCLES_UCHE = 6, + PERF_LRZ_LRZ_READ = 7, + PERF_LRZ_LRZ_WRITE = 8, + PERF_LRZ_READ_LATENCY = 9, + PERF_LRZ_MERGE_CACHE_UPDATING = 10, + PERF_LRZ_PRIM_KILLED_BY_MASKGEN = 11, + PERF_LRZ_PRIM_KILLED_BY_LRZ = 12, + PERF_LRZ_VISIBLE_PRIM_AFTER_LRZ = 13, + PERF_LRZ_FULL_8X8_TILES = 14, + PERF_LRZ_PARTIAL_8X8_TILES = 15, + PERF_LRZ_TILE_KILLED = 16, + PERF_LRZ_TOTAL_PIXEL = 17, + PERF_LRZ_VISIBLE_PIXEL_AFTER_LRZ = 18, + PERF_LRZ_FULLY_COVERED_TILES = 19, + PERF_LRZ_PARTIAL_COVERED_TILES = 20, + PERF_LRZ_FEEDBACK_ACCEPT = 21, + PERF_LRZ_FEEDBACK_DISCARD = 22, + PERF_LRZ_FEEDBACK_STALL = 23, + PERF_LRZ_STALL_CYCLES_RB_ZPLANE = 24, + PERF_LRZ_STALL_CYCLES_RB_BPLANE = 25, + PERF_LRZ_STALL_CYCLES_VC = 26, + PERF_LRZ_RAS_MASK_TRANS = 27, +}; + +enum a6xx_cmp_perfcounter_select { + PERF_CMPDECMP_STALL_CYCLES_ARB = 0, + PERF_CMPDECMP_VBIF_LATENCY_CYCLES = 1, + PERF_CMPDECMP_VBIF_LATENCY_SAMPLES = 2, + PERF_CMPDECMP_VBIF_READ_DATA_CCU = 3, + PERF_CMPDECMP_VBIF_WRITE_DATA_CCU = 4, + PERF_CMPDECMP_VBIF_READ_REQUEST = 5, + PERF_CMPDECMP_VBIF_WRITE_REQUEST = 6, + PERF_CMPDECMP_VBIF_READ_DATA = 7, + PERF_CMPDECMP_VBIF_WRITE_DATA = 8, + PERF_CMPDECMP_FLAG_FETCH_CYCLES = 9, + PERF_CMPDECMP_FLAG_FETCH_SAMPLES = 10, + PERF_CMPDECMP_DEPTH_WRITE_FLAG1_COUNT = 11, + PERF_CMPDECMP_DEPTH_WRITE_FLAG2_COUNT = 12, + PERF_CMPDECMP_DEPTH_WRITE_FLAG3_COUNT = 13, + PERF_CMPDECMP_DEPTH_WRITE_FLAG4_COUNT = 14, + PERF_CMPDECMP_DEPTH_WRITE_FLAG5_COUNT = 15, + PERF_CMPDECMP_DEPTH_WRITE_FLAG6_COUNT = 16, + PERF_CMPDECMP_DEPTH_WRITE_FLAG8_COUNT = 17, + PERF_CMPDECMP_COLOR_WRITE_FLAG1_COUNT = 18, + PERF_CMPDECMP_COLOR_WRITE_FLAG2_COUNT = 19, + PERF_CMPDECMP_COLOR_WRITE_FLAG3_COUNT = 20, + PERF_CMPDECMP_COLOR_WRITE_FLAG4_COUNT = 21, + PERF_CMPDECMP_COLOR_WRITE_FLAG5_COUNT = 22, + PERF_CMPDECMP_COLOR_WRITE_FLAG6_COUNT = 23, + PERF_CMPDECMP_COLOR_WRITE_FLAG8_COUNT = 24, + PERF_CMPDECMP_2D_STALL_CYCLES_VBIF_REQ = 25, + PERF_CMPDECMP_2D_STALL_CYCLES_VBIF_WR = 26, + PERF_CMPDECMP_2D_STALL_CYCLES_VBIF_RETURN = 27, + PERF_CMPDECMP_2D_RD_DATA = 28, + PERF_CMPDECMP_2D_WR_DATA = 29, + PERF_CMPDECMP_VBIF_READ_DATA_UCHE_CH0 = 30, + PERF_CMPDECMP_VBIF_READ_DATA_UCHE_CH1 = 31, + PERF_CMPDECMP_2D_OUTPUT_TRANS = 32, + PERF_CMPDECMP_VBIF_WRITE_DATA_UCHE = 33, + PERF_CMPDECMP_DEPTH_WRITE_FLAG0_COUNT = 34, + PERF_CMPDECMP_COLOR_WRITE_FLAG0_COUNT = 35, + PERF_CMPDECMP_COLOR_WRITE_FLAGALPHA_COUNT = 36, + PERF_CMPDECMP_2D_BUSY_CYCLES = 37, + PERF_CMPDECMP_2D_REORDER_STARVE_CYCLES = 38, + PERF_CMPDECMP_2D_PIXELS = 39, }; enum a6xx_tex_filter { @@ -1765,12 +2444,39 @@ static inline uint32_t A6XX_UCHE_CLIENT_PF_PERFSEL(uint32_t val) #define REG_A6XX_VBIF_VERSION 0x00003000 +#define REG_A6XX_VBIF_CLKON 0x00003001 +#define A6XX_VBIF_CLKON_FORCE_ON_TESTBUS 0x00000002 + #define REG_A6XX_VBIF_GATE_OFF_WRREQ_EN 0x0000302a #define REG_A6XX_VBIF_XIN_HALT_CTRL0 0x00003080 #define REG_A6XX_VBIF_XIN_HALT_CTRL1 0x00003081 +#define REG_A6XX_VBIF_TEST_BUS_OUT_CTRL 0x00003084 + +#define REG_A6XX_VBIF_TEST_BUS1_CTRL0 0x00003085 + +#define REG_A6XX_VBIF_TEST_BUS1_CTRL1 0x00003086 +#define A6XX_VBIF_TEST_BUS1_CTRL1_DATA_SEL__MASK 0x0000000f +#define A6XX_VBIF_TEST_BUS1_CTRL1_DATA_SEL__SHIFT 0 +static inline uint32_t A6XX_VBIF_TEST_BUS1_CTRL1_DATA_SEL(uint32_t val) +{ + return ((val) << A6XX_VBIF_TEST_BUS1_CTRL1_DATA_SEL__SHIFT) & A6XX_VBIF_TEST_BUS1_CTRL1_DATA_SEL__MASK; +} + +#define REG_A6XX_VBIF_TEST_BUS2_CTRL0 0x00003087 + +#define REG_A6XX_VBIF_TEST_BUS2_CTRL1 0x00003088 +#define A6XX_VBIF_TEST_BUS2_CTRL1_DATA_SEL__MASK 0x000001ff +#define A6XX_VBIF_TEST_BUS2_CTRL1_DATA_SEL__SHIFT 0 +static inline uint32_t A6XX_VBIF_TEST_BUS2_CTRL1_DATA_SEL(uint32_t val) +{ + return ((val) << A6XX_VBIF_TEST_BUS2_CTRL1_DATA_SEL__SHIFT) & A6XX_VBIF_TEST_BUS2_CTRL1_DATA_SEL__MASK; +} + +#define REG_A6XX_VBIF_TEST_BUS_OUT 0x0000308c + #define REG_A6XX_VBIF_PERF_CNT_SEL0 0x000030d0 #define REG_A6XX_VBIF_PERF_CNT_SEL1 0x000030d1 @@ -1813,313 +2519,79 @@ static inline uint32_t A6XX_UCHE_CLIENT_PF_PERFSEL(uint32_t val) #define REG_A6XX_VBIF_PERF_PWR_CNT_HIGH2 0x0000311a -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_A 0x00018400 - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_B 0x00018401 - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_C 0x00018402 - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_D 0x00018403 -#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__MASK 0x000000ff -#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__SHIFT 0 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__MASK 0x0000ff00 -#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__SHIFT 8 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__MASK; -} - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_CNTLT 0x00018404 -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK 0x0000003f -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT 0 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK 0x00007000 -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT 12 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK 0xf0000000 -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT 28 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK; -} - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_CNTLM 0x00018405 -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK 0x0f000000 -#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT 24 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK; -} - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_0 0x00018408 - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_1 0x00018409 - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_2 0x0001840a - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_3 0x0001840b - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_0 0x0001840c - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_1 0x0001840d - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_2 0x0001840e - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_3 0x0001840f - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0 0x00018410 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK 0x0000000f -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT 0 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK 0x000000f0 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT 4 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK 0x00000f00 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT 8 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK 0x0000f000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT 12 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK 0x000f0000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT 16 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK 0x00f00000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT 20 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK 0x0f000000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT 24 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK 0xf0000000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT 28 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK; -} - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1 0x00018411 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK 0x0000000f -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT 0 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK 0x000000f0 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT 4 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK 0x00000f00 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT 8 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK 0x0000f000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT 12 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK 0x000f0000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT 16 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK 0x00f00000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT 20 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK 0x0f000000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT 24 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK; -} -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK 0xf0000000 -#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT 28 -static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15(uint32_t val) -{ - return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK; -} - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_TRACE_BUF1 0x0001842f - -#define REG_A6XX_CX_DBGC_CFG_DBGBUS_TRACE_BUF2 0x00018430 - -#define REG_A6XX_PDC_GPU_ENABLE_PDC 0x00021140 - -#define REG_A6XX_PDC_GPU_SEQ_START_ADDR 0x00021148 - -#define REG_A6XX_PDC_GPU_TCS0_CONTROL 0x00021540 - -#define REG_A6XX_PDC_GPU_TCS0_CMD_ENABLE_BANK 0x00021541 - -#define REG_A6XX_PDC_GPU_TCS0_CMD_WAIT_FOR_CMPL_BANK 0x00021542 - -#define REG_A6XX_PDC_GPU_TCS0_CMD0_MSGID 0x00021543 - -#define REG_A6XX_PDC_GPU_TCS0_CMD0_ADDR 0x00021544 - -#define REG_A6XX_PDC_GPU_TCS0_CMD0_DATA 0x00021545 - -#define REG_A6XX_PDC_GPU_TCS1_CONTROL 0x00021572 - -#define REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK 0x00021573 - -#define REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK 0x00021574 - -#define REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID 0x00021575 - -#define REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR 0x00021576 - -#define REG_A6XX_PDC_GPU_TCS1_CMD0_DATA 0x00021577 - -#define REG_A6XX_PDC_GPU_TCS2_CONTROL 0x000215a4 - -#define REG_A6XX_PDC_GPU_TCS2_CMD_ENABLE_BANK 0x000215a5 - -#define REG_A6XX_PDC_GPU_TCS2_CMD_WAIT_FOR_CMPL_BANK 0x000215a6 - -#define REG_A6XX_PDC_GPU_TCS2_CMD0_MSGID 0x000215a7 - -#define REG_A6XX_PDC_GPU_TCS2_CMD0_ADDR 0x000215a8 - -#define REG_A6XX_PDC_GPU_TCS2_CMD0_DATA 0x000215a9 - -#define REG_A6XX_PDC_GPU_TCS3_CONTROL 0x000215d6 - -#define REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK 0x000215d7 - -#define REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK 0x000215d8 - -#define REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID 0x000215d9 - -#define REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR 0x000215da - -#define REG_A6XX_PDC_GPU_TCS3_CMD0_DATA 0x000215db - -#define REG_A6XX_PDC_GPU_SEQ_MEM_0 0x000a0000 - -#define REG_A6XX_X1_WINDOW_OFFSET 0x000088d4 -#define A6XX_X1_WINDOW_OFFSET_WINDOW_OFFSET_DISABLE 0x80000000 -#define A6XX_X1_WINDOW_OFFSET_X__MASK 0x00007fff -#define A6XX_X1_WINDOW_OFFSET_X__SHIFT 0 -static inline uint32_t A6XX_X1_WINDOW_OFFSET_X(uint32_t val) -{ - return ((val) << A6XX_X1_WINDOW_OFFSET_X__SHIFT) & A6XX_X1_WINDOW_OFFSET_X__MASK; -} -#define A6XX_X1_WINDOW_OFFSET_Y__MASK 0x7fff0000 -#define A6XX_X1_WINDOW_OFFSET_Y__SHIFT 16 -static inline uint32_t A6XX_X1_WINDOW_OFFSET_Y(uint32_t val) -{ - return ((val) << A6XX_X1_WINDOW_OFFSET_Y__SHIFT) & A6XX_X1_WINDOW_OFFSET_Y__MASK; -} - -#define REG_A6XX_X2_WINDOW_OFFSET 0x0000b4d1 -#define A6XX_X2_WINDOW_OFFSET_WINDOW_OFFSET_DISABLE 0x80000000 -#define A6XX_X2_WINDOW_OFFSET_X__MASK 0x00007fff -#define A6XX_X2_WINDOW_OFFSET_X__SHIFT 0 -static inline uint32_t A6XX_X2_WINDOW_OFFSET_X(uint32_t val) +#define REG_A6XX_RB_WINDOW_OFFSET2 0x000088d4 +#define A6XX_RB_WINDOW_OFFSET2_WINDOW_OFFSET_DISABLE 0x80000000 +#define A6XX_RB_WINDOW_OFFSET2_X__MASK 0x00007fff +#define A6XX_RB_WINDOW_OFFSET2_X__SHIFT 0 +static inline uint32_t A6XX_RB_WINDOW_OFFSET2_X(uint32_t val) { - return ((val) << A6XX_X2_WINDOW_OFFSET_X__SHIFT) & A6XX_X2_WINDOW_OFFSET_X__MASK; + return ((val) << A6XX_RB_WINDOW_OFFSET2_X__SHIFT) & A6XX_RB_WINDOW_OFFSET2_X__MASK; } -#define A6XX_X2_WINDOW_OFFSET_Y__MASK 0x7fff0000 -#define A6XX_X2_WINDOW_OFFSET_Y__SHIFT 16 -static inline uint32_t A6XX_X2_WINDOW_OFFSET_Y(uint32_t val) +#define A6XX_RB_WINDOW_OFFSET2_Y__MASK 0x7fff0000 +#define A6XX_RB_WINDOW_OFFSET2_Y__SHIFT 16 +static inline uint32_t A6XX_RB_WINDOW_OFFSET2_Y(uint32_t val) { - return ((val) << A6XX_X2_WINDOW_OFFSET_Y__SHIFT) & A6XX_X2_WINDOW_OFFSET_Y__MASK; + return ((val) << A6XX_RB_WINDOW_OFFSET2_Y__SHIFT) & A6XX_RB_WINDOW_OFFSET2_Y__MASK; } -#define REG_A6XX_X3_WINDOW_OFFSET 0x0000b307 -#define A6XX_X3_WINDOW_OFFSET_WINDOW_OFFSET_DISABLE 0x80000000 -#define A6XX_X3_WINDOW_OFFSET_X__MASK 0x00007fff -#define A6XX_X3_WINDOW_OFFSET_X__SHIFT 0 -static inline uint32_t A6XX_X3_WINDOW_OFFSET_X(uint32_t val) +#define REG_A6XX_SP_WINDOW_OFFSET 0x0000b4d1 +#define A6XX_SP_WINDOW_OFFSET_WINDOW_OFFSET_DISABLE 0x80000000 +#define A6XX_SP_WINDOW_OFFSET_X__MASK 0x00007fff +#define A6XX_SP_WINDOW_OFFSET_X__SHIFT 0 +static inline uint32_t A6XX_SP_WINDOW_OFFSET_X(uint32_t val) { - return ((val) << A6XX_X3_WINDOW_OFFSET_X__SHIFT) & A6XX_X3_WINDOW_OFFSET_X__MASK; + return ((val) << A6XX_SP_WINDOW_OFFSET_X__SHIFT) & A6XX_SP_WINDOW_OFFSET_X__MASK; } -#define A6XX_X3_WINDOW_OFFSET_Y__MASK 0x7fff0000 -#define A6XX_X3_WINDOW_OFFSET_Y__SHIFT 16 -static inline uint32_t A6XX_X3_WINDOW_OFFSET_Y(uint32_t val) +#define A6XX_SP_WINDOW_OFFSET_Y__MASK 0x7fff0000 +#define A6XX_SP_WINDOW_OFFSET_Y__SHIFT 16 +static inline uint32_t A6XX_SP_WINDOW_OFFSET_Y(uint32_t val) { - return ((val) << A6XX_X3_WINDOW_OFFSET_Y__SHIFT) & A6XX_X3_WINDOW_OFFSET_Y__MASK; + return ((val) << A6XX_SP_WINDOW_OFFSET_Y__SHIFT) & A6XX_SP_WINDOW_OFFSET_Y__MASK; } -#define REG_A6XX_X1_BIN_SIZE 0x000080a1 -#define A6XX_X1_BIN_SIZE_WIDTH__MASK 0x000000ff -#define A6XX_X1_BIN_SIZE_WIDTH__SHIFT 0 -static inline uint32_t A6XX_X1_BIN_SIZE_WIDTH(uint32_t val) +#define REG_A6XX_SP_TP_WINDOW_OFFSET 0x0000b307 +#define A6XX_SP_TP_WINDOW_OFFSET_WINDOW_OFFSET_DISABLE 0x80000000 +#define A6XX_SP_TP_WINDOW_OFFSET_X__MASK 0x00007fff +#define A6XX_SP_TP_WINDOW_OFFSET_X__SHIFT 0 +static inline uint32_t A6XX_SP_TP_WINDOW_OFFSET_X(uint32_t val) { - return ((val >> 5) << A6XX_X1_BIN_SIZE_WIDTH__SHIFT) & A6XX_X1_BIN_SIZE_WIDTH__MASK; + return ((val) << A6XX_SP_TP_WINDOW_OFFSET_X__SHIFT) & A6XX_SP_TP_WINDOW_OFFSET_X__MASK; } -#define A6XX_X1_BIN_SIZE_HEIGHT__MASK 0x0001ff00 -#define A6XX_X1_BIN_SIZE_HEIGHT__SHIFT 8 -static inline uint32_t A6XX_X1_BIN_SIZE_HEIGHT(uint32_t val) +#define A6XX_SP_TP_WINDOW_OFFSET_Y__MASK 0x7fff0000 +#define A6XX_SP_TP_WINDOW_OFFSET_Y__SHIFT 16 +static inline uint32_t A6XX_SP_TP_WINDOW_OFFSET_Y(uint32_t val) { - return ((val >> 4) << A6XX_X1_BIN_SIZE_HEIGHT__SHIFT) & A6XX_X1_BIN_SIZE_HEIGHT__MASK; + return ((val) << A6XX_SP_TP_WINDOW_OFFSET_Y__SHIFT) & A6XX_SP_TP_WINDOW_OFFSET_Y__MASK; } -#define REG_A6XX_X2_BIN_SIZE 0x00008800 -#define A6XX_X2_BIN_SIZE_WIDTH__MASK 0x000000ff -#define A6XX_X2_BIN_SIZE_WIDTH__SHIFT 0 -static inline uint32_t A6XX_X2_BIN_SIZE_WIDTH(uint32_t val) +#define REG_A6XX_GRAS_BIN_CONTROL 0x000080a1 +#define A6XX_GRAS_BIN_CONTROL_BINW__MASK 0x000000ff +#define A6XX_GRAS_BIN_CONTROL_BINW__SHIFT 0 +static inline uint32_t A6XX_GRAS_BIN_CONTROL_BINW(uint32_t val) { - return ((val >> 5) << A6XX_X2_BIN_SIZE_WIDTH__SHIFT) & A6XX_X2_BIN_SIZE_WIDTH__MASK; + return ((val >> 5) << A6XX_GRAS_BIN_CONTROL_BINW__SHIFT) & A6XX_GRAS_BIN_CONTROL_BINW__MASK; } -#define A6XX_X2_BIN_SIZE_HEIGHT__MASK 0x0001ff00 -#define A6XX_X2_BIN_SIZE_HEIGHT__SHIFT 8 -static inline uint32_t A6XX_X2_BIN_SIZE_HEIGHT(uint32_t val) +#define A6XX_GRAS_BIN_CONTROL_BINH__MASK 0x0001ff00 +#define A6XX_GRAS_BIN_CONTROL_BINH__SHIFT 8 +static inline uint32_t A6XX_GRAS_BIN_CONTROL_BINH(uint32_t val) { - return ((val >> 4) << A6XX_X2_BIN_SIZE_HEIGHT__SHIFT) & A6XX_X2_BIN_SIZE_HEIGHT__MASK; + return ((val >> 4) << A6XX_GRAS_BIN_CONTROL_BINH__SHIFT) & A6XX_GRAS_BIN_CONTROL_BINH__MASK; } +#define A6XX_GRAS_BIN_CONTROL_BINNING_PASS 0x00040000 +#define A6XX_GRAS_BIN_CONTROL_USE_VIZ 0x00200000 -#define REG_A6XX_X3_BIN_SIZE 0x000088d3 -#define A6XX_X3_BIN_SIZE_WIDTH__MASK 0x000000ff -#define A6XX_X3_BIN_SIZE_WIDTH__SHIFT 0 -static inline uint32_t A6XX_X3_BIN_SIZE_WIDTH(uint32_t val) +#define REG_A6XX_RB_BIN_CONTROL2 0x000088d3 +#define A6XX_RB_BIN_CONTROL2_BINW__MASK 0x000000ff +#define A6XX_RB_BIN_CONTROL2_BINW__SHIFT 0 +static inline uint32_t A6XX_RB_BIN_CONTROL2_BINW(uint32_t val) { - return ((val >> 5) << A6XX_X3_BIN_SIZE_WIDTH__SHIFT) & A6XX_X3_BIN_SIZE_WIDTH__MASK; + return ((val >> 5) << A6XX_RB_BIN_CONTROL2_BINW__SHIFT) & A6XX_RB_BIN_CONTROL2_BINW__MASK; } -#define A6XX_X3_BIN_SIZE_HEIGHT__MASK 0x0001ff00 -#define A6XX_X3_BIN_SIZE_HEIGHT__SHIFT 8 -static inline uint32_t A6XX_X3_BIN_SIZE_HEIGHT(uint32_t val) +#define A6XX_RB_BIN_CONTROL2_BINH__MASK 0x0001ff00 +#define A6XX_RB_BIN_CONTROL2_BINH__SHIFT 8 +static inline uint32_t A6XX_RB_BIN_CONTROL2_BINH(uint32_t val) { - return ((val >> 4) << A6XX_X3_BIN_SIZE_HEIGHT__SHIFT) & A6XX_X3_BIN_SIZE_HEIGHT__MASK; + return ((val >> 4) << A6XX_RB_BIN_CONTROL2_BINH__SHIFT) & A6XX_RB_BIN_CONTROL2_BINH__MASK; } #define REG_A6XX_VSC_BIN_SIZE 0x00000c02 @@ -2182,11 +2654,19 @@ static inline uint32_t A6XX_VSC_PIPE_CONFIG_REG_H(uint32_t val) return ((val) << A6XX_VSC_PIPE_CONFIG_REG_H__SHIFT) & A6XX_VSC_PIPE_CONFIG_REG_H__MASK; } -#define REG_A6XX_VSC_XXX_ADDRESS_LO 0x00000c30 +#define REG_A6XX_VSC_PIPE_DATA2_ADDRESS_LO 0x00000c30 -#define REG_A6XX_VSC_XXX_ADDRESS_HI 0x00000c31 +#define REG_A6XX_VSC_PIPE_DATA2_ADDRESS_HI 0x00000c31 -#define REG_A6XX_VSC_XXX_PITCH 0x00000c32 +#define REG_A6XX_VSC_PIPE_DATA2_PITCH 0x00000c32 + +#define REG_A6XX_VSC_PIPE_DATA2_ARRAY_PITCH 0x00000c33 +#define A6XX_VSC_PIPE_DATA2_ARRAY_PITCH__MASK 0xffffffff +#define A6XX_VSC_PIPE_DATA2_ARRAY_PITCH__SHIFT 0 +static inline uint32_t A6XX_VSC_PIPE_DATA2_ARRAY_PITCH(uint32_t val) +{ + return ((val >> 4) << A6XX_VSC_PIPE_DATA2_ARRAY_PITCH__SHIFT) & A6XX_VSC_PIPE_DATA2_ARRAY_PITCH__MASK; +} #define REG_A6XX_VSC_PIPE_DATA_ADDRESS_LO 0x00000c34 @@ -2194,18 +2674,29 @@ static inline uint32_t A6XX_VSC_PIPE_CONFIG_REG_H(uint32_t val) #define REG_A6XX_VSC_PIPE_DATA_PITCH 0x00000c36 +#define REG_A6XX_VSC_PIPE_DATA_ARRAY_PITCH 0x00000c37 +#define A6XX_VSC_PIPE_DATA_ARRAY_PITCH__MASK 0xffffffff +#define A6XX_VSC_PIPE_DATA_ARRAY_PITCH__SHIFT 0 +static inline uint32_t A6XX_VSC_PIPE_DATA_ARRAY_PITCH(uint32_t val) +{ + return ((val >> 4) << A6XX_VSC_PIPE_DATA_ARRAY_PITCH__SHIFT) & A6XX_VSC_PIPE_DATA_ARRAY_PITCH__MASK; +} + static inline uint32_t REG_A6XX_VSC_SIZE(uint32_t i0) { return 0x00000c78 + 0x1*i0; } static inline uint32_t REG_A6XX_VSC_SIZE_REG(uint32_t i0) { return 0x00000c78 + 0x1*i0; } #define REG_A6XX_UCHE_UNKNOWN_0E12 0x00000e12 +#define REG_A6XX_GRAS_UNKNOWN_8000 0x00008000 + #define REG_A6XX_GRAS_UNKNOWN_8001 0x00008001 #define REG_A6XX_GRAS_UNKNOWN_8004 0x00008004 #define REG_A6XX_GRAS_CNTL 0x00008005 #define A6XX_GRAS_CNTL_VARYING 0x00000001 +#define A6XX_GRAS_CNTL_UNK3 0x00000008 #define A6XX_GRAS_CNTL_XCOORD 0x00000040 #define A6XX_GRAS_CNTL_YCOORD 0x00000080 #define A6XX_GRAS_CNTL_ZCOORD 0x00000100 @@ -2308,6 +2799,9 @@ static inline uint32_t A6XX_GRAS_SU_POINT_SIZE(float val) return ((((int32_t)(val * 16.0))) << A6XX_GRAS_SU_POINT_SIZE__SHIFT) & A6XX_GRAS_SU_POINT_SIZE__MASK; } +#define REG_A6XX_GRAS_SU_DEPTH_PLANE_CNTL 0x00008094 +#define A6XX_GRAS_SU_DEPTH_PLANE_CNTL_FRAG_WRITES_Z 0x00000001 + #define REG_A6XX_GRAS_SU_POLY_OFFSET_SCALE 0x00008095 #define A6XX_GRAS_SU_POLY_OFFSET_SCALE__MASK 0xffffffff #define A6XX_GRAS_SU_POLY_OFFSET_SCALE__SHIFT 0 @@ -2344,6 +2838,8 @@ static inline uint32_t A6XX_GRAS_SU_DEPTH_BUFFER_INFO_DEPTH_FORMAT(enum a6xx_dep #define REG_A6XX_GRAS_UNKNOWN_809B 0x0000809b +#define REG_A6XX_GRAS_UNKNOWN_80A0 0x000080a0 + #define REG_A6XX_GRAS_RAS_MSAA_CNTL 0x000080a2 #define A6XX_GRAS_RAS_MSAA_CNTL_SAMPLES__MASK 0x00000003 #define A6XX_GRAS_RAS_MSAA_CNTL_SAMPLES__SHIFT 0 @@ -2464,6 +2960,8 @@ static inline uint32_t A6XX_GRAS_SC_WINDOW_SCISSOR_BR_Y(uint32_t val) #define A6XX_GRAS_LRZ_CNTL_LRZ_WRITE 0x00000002 #define A6XX_GRAS_LRZ_CNTL_GREATER 0x00000004 +#define REG_A6XX_GRAS_UNKNOWN_8101 0x00008101 + #define REG_A6XX_GRAS_2D_BLIT_INFO 0x00008102 #define A6XX_GRAS_2D_BLIT_INFO_COLOR_FORMAT__MASK 0x000000ff #define A6XX_GRAS_2D_BLIT_INFO_COLOR_FORMAT__SHIFT 0 @@ -2494,6 +2992,10 @@ static inline uint32_t A6XX_GRAS_LRZ_BUFFER_PITCH_ARRAY_PITCH(uint32_t val) #define REG_A6XX_GRAS_LRZ_FAST_CLEAR_BUFFER_BASE_HI 0x00008107 +#define REG_A6XX_GRAS_UNKNOWN_8109 0x00008109 + +#define REG_A6XX_GRAS_UNKNOWN_8110 0x00008110 + #define REG_A6XX_GRAS_2D_BLIT_CNTL 0x00008400 #define REG_A6XX_GRAS_2D_SRC_TL_X 0x00008401 @@ -2590,6 +3092,33 @@ static inline uint32_t A6XX_GRAS_RESOLVE_CNTL_2_Y(uint32_t val) #define REG_A6XX_GRAS_UNKNOWN_8600 0x00008600 +#define REG_A6XX_RB_BIN_CONTROL 0x00008800 +#define A6XX_RB_BIN_CONTROL_BINW__MASK 0x000000ff +#define A6XX_RB_BIN_CONTROL_BINW__SHIFT 0 +static inline uint32_t A6XX_RB_BIN_CONTROL_BINW(uint32_t val) +{ + return ((val >> 5) << A6XX_RB_BIN_CONTROL_BINW__SHIFT) & A6XX_RB_BIN_CONTROL_BINW__MASK; +} +#define A6XX_RB_BIN_CONTROL_BINH__MASK 0x0001ff00 +#define A6XX_RB_BIN_CONTROL_BINH__SHIFT 8 +static inline uint32_t A6XX_RB_BIN_CONTROL_BINH(uint32_t val) +{ + return ((val >> 4) << A6XX_RB_BIN_CONTROL_BINH__SHIFT) & A6XX_RB_BIN_CONTROL_BINH__MASK; +} +#define A6XX_RB_BIN_CONTROL_BINNING_PASS 0x00040000 +#define A6XX_RB_BIN_CONTROL_USE_VIZ 0x00200000 + +#define REG_A6XX_RB_RENDER_CNTL 0x00008801 +#define A6XX_RB_RENDER_CNTL_UNK4 0x00000010 +#define A6XX_RB_RENDER_CNTL_BINNING 0x00000080 +#define A6XX_RB_RENDER_CNTL_FLAG_DEPTH 0x00004000 +#define A6XX_RB_RENDER_CNTL_FLAG_MRTS__MASK 0x00ff0000 +#define A6XX_RB_RENDER_CNTL_FLAG_MRTS__SHIFT 16 +static inline uint32_t A6XX_RB_RENDER_CNTL_FLAG_MRTS(uint32_t val) +{ + return ((val) << A6XX_RB_RENDER_CNTL_FLAG_MRTS__SHIFT) & A6XX_RB_RENDER_CNTL_FLAG_MRTS__MASK; +} + #define REG_A6XX_RB_RAS_MSAA_CNTL 0x00008802 #define A6XX_RB_RAS_MSAA_CNTL_SAMPLES__MASK 0x00000003 #define A6XX_RB_RAS_MSAA_CNTL_SAMPLES__SHIFT 0 @@ -2615,6 +3144,7 @@ static inline uint32_t A6XX_RB_DEST_MSAA_CNTL_SAMPLES(enum a3xx_msaa_samples val #define REG_A6XX_RB_RENDER_CONTROL0 0x00008809 #define A6XX_RB_RENDER_CONTROL0_VARYING 0x00000001 +#define A6XX_RB_RENDER_CONTROL0_UNK3 0x00000008 #define A6XX_RB_RENDER_CONTROL0_XCOORD 0x00000040 #define A6XX_RB_RENDER_CONTROL0_YCOORD 0x00000080 #define A6XX_RB_RENDER_CONTROL0_ZCOORD 0x00000100 @@ -2747,6 +3277,10 @@ static inline uint32_t A6XX_RB_DITHER_CNTL_DITHER_MODE_MRT7(enum adreno_rb_dithe #define A6XX_RB_SRGB_CNTL_SRGB_MRT6 0x00000040 #define A6XX_RB_SRGB_CNTL_SRGB_MRT7 0x00000080 +#define REG_A6XX_RB_UNKNOWN_8810 0x00008810 + +#define REG_A6XX_RB_UNKNOWN_8811 0x00008811 + #define REG_A6XX_RB_UNKNOWN_8818 0x00008818 #define REG_A6XX_RB_UNKNOWN_8819 0x00008819 @@ -2837,7 +3371,6 @@ static inline uint32_t A6XX_RB_MRT_BUF_INFO_COLOR_SWAP(enum a3xx_color_swap val) { return ((val) << A6XX_RB_MRT_BUF_INFO_COLOR_SWAP__SHIFT) & A6XX_RB_MRT_BUF_INFO_COLOR_SWAP__MASK; } -#define A6XX_RB_MRT_BUF_INFO_COLOR_SRGB 0x00008000 static inline uint32_t REG_A6XX_RB_MRT_PITCH(uint32_t i0) { return 0x00008823 + 0x8*i0; } #define A6XX_RB_MRT_PITCH__MASK 0xffffffff @@ -2923,6 +3456,9 @@ static inline uint32_t A6XX_RB_BLEND_CNTL_SAMPLE_MASK(uint32_t val) return ((val) << A6XX_RB_BLEND_CNTL_SAMPLE_MASK__SHIFT) & A6XX_RB_BLEND_CNTL_SAMPLE_MASK__MASK; } +#define REG_A6XX_RB_DEPTH_PLANE_CNTL 0x00008870 +#define A6XX_RB_DEPTH_PLANE_CNTL_FRAG_WRITES_Z 0x00000001 + #define REG_A6XX_RB_DEPTH_CNTL 0x00008871 #define A6XX_RB_DEPTH_CNTL_Z_ENABLE 0x00000001 #define A6XX_RB_DEPTH_CNTL_Z_WRITE_ENABLE 0x00000002 @@ -3053,6 +3589,12 @@ static inline uint32_t A6XX_RB_STENCILREF_REF(uint32_t val) { return ((val) << A6XX_RB_STENCILREF_REF__SHIFT) & A6XX_RB_STENCILREF_REF__MASK; } +#define A6XX_RB_STENCILREF_BFREF__MASK 0x0000ff00 +#define A6XX_RB_STENCILREF_BFREF__SHIFT 8 +static inline uint32_t A6XX_RB_STENCILREF_BFREF(uint32_t val) +{ + return ((val) << A6XX_RB_STENCILREF_BFREF__SHIFT) & A6XX_RB_STENCILREF_BFREF__MASK; +} #define REG_A6XX_RB_STENCILMASK 0x00008888 #define A6XX_RB_STENCILMASK_MASK__MASK 0x000000ff @@ -3061,6 +3603,12 @@ static inline uint32_t A6XX_RB_STENCILMASK_MASK(uint32_t val) { return ((val) << A6XX_RB_STENCILMASK_MASK__SHIFT) & A6XX_RB_STENCILMASK_MASK__MASK; } +#define A6XX_RB_STENCILMASK_BFMASK__MASK 0x0000ff00 +#define A6XX_RB_STENCILMASK_BFMASK__SHIFT 8 +static inline uint32_t A6XX_RB_STENCILMASK_BFMASK(uint32_t val) +{ + return ((val) << A6XX_RB_STENCILMASK_BFMASK__SHIFT) & A6XX_RB_STENCILMASK_BFMASK__MASK; +} #define REG_A6XX_RB_STENCILWRMASK 0x00008889 #define A6XX_RB_STENCILWRMASK_WRMASK__MASK 0x000000ff @@ -3069,6 +3617,12 @@ static inline uint32_t A6XX_RB_STENCILWRMASK_WRMASK(uint32_t val) { return ((val) << A6XX_RB_STENCILWRMASK_WRMASK__SHIFT) & A6XX_RB_STENCILWRMASK_WRMASK__MASK; } +#define A6XX_RB_STENCILWRMASK_BFWRMASK__MASK 0x0000ff00 +#define A6XX_RB_STENCILWRMASK_BFWRMASK__SHIFT 8 +static inline uint32_t A6XX_RB_STENCILWRMASK_BFWRMASK(uint32_t val) +{ + return ((val) << A6XX_RB_STENCILWRMASK_BFWRMASK__SHIFT) & A6XX_RB_STENCILWRMASK_BFWRMASK__MASK; +} #define REG_A6XX_RB_WINDOW_OFFSET 0x00008890 #define A6XX_RB_WINDOW_OFFSET_WINDOW_OFFSET_DISABLE 0x80000000 @@ -3177,14 +3731,14 @@ static inline uint32_t A6XX_RB_BLIT_DST_ARRAY_PITCH(uint32_t val) #define REG_A6XX_RB_BLIT_INFO 0x000088e3 #define A6XX_RB_BLIT_INFO_UNK0 0x00000001 -#define A6XX_RB_BLIT_INFO_FAST_CLEAR 0x00000002 +#define A6XX_RB_BLIT_INFO_GMEM 0x00000002 #define A6XX_RB_BLIT_INFO_INTEGER 0x00000004 -#define A6XX_RB_BLIT_INFO_UNK3 0x00000008 -#define A6XX_RB_BLIT_INFO_MASK__MASK 0x000000f0 -#define A6XX_RB_BLIT_INFO_MASK__SHIFT 4 -static inline uint32_t A6XX_RB_BLIT_INFO_MASK(uint32_t val) +#define A6XX_RB_BLIT_INFO_DEPTH 0x00000008 +#define A6XX_RB_BLIT_INFO_CLEAR_MASK__MASK 0x000000f0 +#define A6XX_RB_BLIT_INFO_CLEAR_MASK__SHIFT 4 +static inline uint32_t A6XX_RB_BLIT_INFO_CLEAR_MASK(uint32_t val) { - return ((val) << A6XX_RB_BLIT_INFO_MASK__SHIFT) & A6XX_RB_BLIT_INFO_MASK__MASK; + return ((val) << A6XX_RB_BLIT_INFO_CLEAR_MASK__SHIFT) & A6XX_RB_BLIT_INFO_CLEAR_MASK__MASK; } #define REG_A6XX_RB_UNKNOWN_88F0 0x000088f0 @@ -3274,12 +3828,16 @@ static inline uint32_t A6XX_RB_2D_DST_SIZE_PITCH(uint32_t val) #define REG_A6XX_RB_UNKNOWN_8E01 0x00008e01 +#define REG_A6XX_RB_UNKNOWN_8E04 0x00008e04 + #define REG_A6XX_RB_CCU_CNTL 0x00008e07 #define REG_A6XX_VPC_UNKNOWN_9101 0x00009101 #define REG_A6XX_VPC_GS_SIV_CNTL 0x00009104 +#define REG_A6XX_VPC_UNKNOWN_9107 0x00009107 + #define REG_A6XX_VPC_UNKNOWN_9108 0x00009108 static inline uint32_t REG_A6XX_VPC_VARYING_INTERP(uint32_t i0) { return 0x00009200 + 0x1*i0; } @@ -3385,6 +3943,9 @@ static inline uint32_t A6XX_VPC_CNTL_0_NUMNONPOSVAR(uint32_t val) #define A6XX_VPC_SO_BUF_CNTL_BUF3 0x00000200 #define A6XX_VPC_SO_BUF_CNTL_ENABLE 0x00008000 +#define REG_A6XX_VPC_SO_OVERRIDE 0x00009306 +#define A6XX_VPC_SO_OVERRIDE_SO_DISABLE 0x00000001 + #define REG_A6XX_VPC_UNKNOWN_9600 0x00009600 #define REG_A6XX_VPC_UNKNOWN_9602 0x00009602 @@ -3397,8 +3958,14 @@ static inline uint32_t A6XX_VPC_CNTL_0_NUMNONPOSVAR(uint32_t val) #define REG_A6XX_PC_UNKNOWN_9805 0x00009805 +#define REG_A6XX_PC_UNKNOWN_9806 0x00009806 + +#define REG_A6XX_PC_UNKNOWN_9980 0x00009980 + #define REG_A6XX_PC_UNKNOWN_9981 0x00009981 +#define REG_A6XX_PC_UNKNOWN_9990 0x00009990 + #define REG_A6XX_PC_PRIMITIVE_CNTL_0 0x00009b00 #define A6XX_PC_PRIMITIVE_CNTL_0_PRIMITIVE_RESTART 0x00000001 #define A6XX_PC_PRIMITIVE_CNTL_0_PROVOKING_VTX_LAST 0x00000002 @@ -3410,6 +3977,7 @@ static inline uint32_t A6XX_PC_PRIMITIVE_CNTL_1_STRIDE_IN_VPC(uint32_t val) { return ((val) << A6XX_PC_PRIMITIVE_CNTL_1_STRIDE_IN_VPC__SHIFT) & A6XX_PC_PRIMITIVE_CNTL_1_STRIDE_IN_VPC__MASK; } +#define A6XX_PC_PRIMITIVE_CNTL_1_PSIZE 0x00000100 #define REG_A6XX_PC_UNKNOWN_9B06 0x00009b06 @@ -3488,6 +4056,8 @@ static inline uint32_t A6XX_VFD_CONTROL_3_REGID_TESSY(uint32_t val) #define REG_A6XX_VFD_UNKNOWN_A008 0x0000a008 +#define REG_A6XX_VFD_UNKNOWN_A009 0x0000a009 + #define REG_A6XX_VFD_INDEX_OFFSET 0x0000a00e #define REG_A6XX_VFD_INSTANCE_START_OFFSET 0x0000a00f @@ -3640,6 +4210,8 @@ static inline uint32_t A6XX_SP_VS_CTRL_REG0_THREADSIZE(enum a3xx_threadsize val) #define A6XX_SP_VS_CTRL_REG0_PIXLODENABLE 0x04000000 #define A6XX_SP_VS_CTRL_REG0_MERGEDREGS 0x80000000 +#define REG_A6XX_SP_UNKNOWN_A81B 0x0000a81b + #define REG_A6XX_SP_VS_OBJ_START_LO 0x0000a81c #define REG_A6XX_SP_VS_OBJ_START_HI 0x0000a81d @@ -3884,6 +4456,8 @@ static inline uint32_t A6XX_SP_FS_CTRL_REG0_THREADSIZE(enum a3xx_threadsize val) #define A6XX_SP_FS_CTRL_REG0_PIXLODENABLE 0x04000000 #define A6XX_SP_FS_CTRL_REG0_MERGEDREGS 0x80000000 +#define REG_A6XX_SP_UNKNOWN_A982 0x0000a982 + #define REG_A6XX_SP_FS_OBJ_START_LO 0x0000a983 #define REG_A6XX_SP_FS_OBJ_START_HI 0x0000a984 @@ -3979,7 +4553,8 @@ static inline uint32_t A6XX_SP_FS_MRT_REG_COLOR_FORMAT(enum a6xx_color_fmt val) } #define A6XX_SP_FS_MRT_REG_COLOR_SINT 0x00000100 #define A6XX_SP_FS_MRT_REG_COLOR_UINT 0x00000200 -#define A6XX_SP_FS_MRT_REG_COLOR_SRGB 0x00000400 + +#define REG_A6XX_SP_UNKNOWN_A99E 0x0000a99e #define REG_A6XX_SP_FS_TEX_COUNT 0x0000a9a7 @@ -4066,14 +4641,20 @@ static inline uint32_t A6XX_SP_FS_CONFIG_NSAMP(uint32_t val) #define REG_A6XX_SP_FS_INSTRLEN 0x0000ab05 +#define REG_A6XX_SP_UNKNOWN_AB20 0x0000ab20 + #define REG_A6XX_SP_UNKNOWN_AE00 0x0000ae00 +#define REG_A6XX_SP_UNKNOWN_AE03 0x0000ae03 + #define REG_A6XX_SP_UNKNOWN_AE04 0x0000ae04 #define REG_A6XX_SP_UNKNOWN_AE0F 0x0000ae0f #define REG_A6XX_SP_UNKNOWN_B182 0x0000b182 +#define REG_A6XX_SP_UNKNOWN_B183 0x0000b183 + #define REG_A6XX_SP_TP_RAS_MSAA_CNTL 0x0000b300 #define A6XX_SP_TP_RAS_MSAA_CNTL_SAMPLES__MASK 0x00000003 #define A6XX_SP_TP_RAS_MSAA_CNTL_SAMPLES__SHIFT 0 @@ -4097,6 +4678,8 @@ static inline uint32_t A6XX_SP_TP_DEST_MSAA_CNTL_SAMPLES(enum a3xx_msaa_samples #define REG_A6XX_SP_TP_UNKNOWN_B304 0x0000b304 +#define REG_A6XX_SP_TP_UNKNOWN_B309 0x0000b309 + #define REG_A6XX_SP_PS_2D_SRC_INFO 0x0000b4c0 #define A6XX_SP_PS_2D_SRC_INFO_COLOR_FORMAT__MASK 0x000000ff #define A6XX_SP_PS_2D_SRC_INFO_COLOR_FORMAT__SHIFT 0 @@ -4162,6 +4745,8 @@ static inline uint32_t A6XX_HLSQ_GS_CNTL_CONSTLEN(uint32_t val) return ((val >> 2) << A6XX_HLSQ_GS_CNTL_CONSTLEN__SHIFT) & A6XX_HLSQ_GS_CNTL_CONSTLEN__MASK; } +#define REG_A6XX_HLSQ_UNKNOWN_B980 0x0000b980 + #define REG_A6XX_HLSQ_CONTROL_1_REG 0x0000b982 #define REG_A6XX_HLSQ_CONTROL_2_REG 0x0000b983 @@ -4537,11 +5122,11 @@ static inline uint32_t A6XX_TEX_CONST_7_FLAG_LO(uint32_t val) } #define REG_A6XX_TEX_CONST_8 0x00000008 -#define A6XX_TEX_CONST_8_BASE_HI__MASK 0x0001ffff -#define A6XX_TEX_CONST_8_BASE_HI__SHIFT 0 -static inline uint32_t A6XX_TEX_CONST_8_BASE_HI(uint32_t val) +#define A6XX_TEX_CONST_8_FLAG_HI__MASK 0x0001ffff +#define A6XX_TEX_CONST_8_FLAG_HI__SHIFT 0 +static inline uint32_t A6XX_TEX_CONST_8_FLAG_HI(uint32_t val) { - return ((val) << A6XX_TEX_CONST_8_BASE_HI__SHIFT) & A6XX_TEX_CONST_8_BASE_HI__MASK; + return ((val) << A6XX_TEX_CONST_8_FLAG_HI__SHIFT) & A6XX_TEX_CONST_8_FLAG_HI__MASK; } #define REG_A6XX_TEX_CONST_9 0x00000009 @@ -4558,5 +5143,227 @@ static inline uint32_t A6XX_TEX_CONST_8_BASE_HI(uint32_t val) #define REG_A6XX_TEX_CONST_15 0x0000000f +#define REG_A6XX_PDC_GPU_ENABLE_PDC 0x00001140 + +#define REG_A6XX_PDC_GPU_SEQ_START_ADDR 0x00001148 + +#define REG_A6XX_PDC_GPU_TCS0_CONTROL 0x00001540 + +#define REG_A6XX_PDC_GPU_TCS0_CMD_ENABLE_BANK 0x00001541 + +#define REG_A6XX_PDC_GPU_TCS0_CMD_WAIT_FOR_CMPL_BANK 0x00001542 + +#define REG_A6XX_PDC_GPU_TCS0_CMD0_MSGID 0x00001543 + +#define REG_A6XX_PDC_GPU_TCS0_CMD0_ADDR 0x00001544 + +#define REG_A6XX_PDC_GPU_TCS0_CMD0_DATA 0x00001545 + +#define REG_A6XX_PDC_GPU_TCS1_CONTROL 0x00001572 + +#define REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK 0x00001573 + +#define REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK 0x00001574 + +#define REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID 0x00001575 + +#define REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR 0x00001576 + +#define REG_A6XX_PDC_GPU_TCS1_CMD0_DATA 0x00001577 + +#define REG_A6XX_PDC_GPU_TCS2_CONTROL 0x000015a4 + +#define REG_A6XX_PDC_GPU_TCS2_CMD_ENABLE_BANK 0x000015a5 + +#define REG_A6XX_PDC_GPU_TCS2_CMD_WAIT_FOR_CMPL_BANK 0x000015a6 + +#define REG_A6XX_PDC_GPU_TCS2_CMD0_MSGID 0x000015a7 + +#define REG_A6XX_PDC_GPU_TCS2_CMD0_ADDR 0x000015a8 + +#define REG_A6XX_PDC_GPU_TCS2_CMD0_DATA 0x000015a9 + +#define REG_A6XX_PDC_GPU_TCS3_CONTROL 0x000015d6 + +#define REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK 0x000015d7 + +#define REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK 0x000015d8 + +#define REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID 0x000015d9 + +#define REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR 0x000015da + +#define REG_A6XX_PDC_GPU_TCS3_CMD0_DATA 0x000015db + +#define REG_A6XX_PDC_GPU_SEQ_MEM_0 0x00000000 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_A 0x00000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_INDEX__MASK 0x000000ff +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_INDEX__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_INDEX(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_INDEX__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_INDEX__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_BLK_SEL__MASK 0x0000ff00 +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_BLK_SEL__SHIFT 8 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_BLK_SEL(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_BLK_SEL__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_SEL_A_PING_BLK_SEL__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_B 0x00000001 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_C 0x00000002 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_D 0x00000003 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_CNTLT 0x00000004 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK 0x0000003f +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK 0x00007000 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT 12 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK 0xf0000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT 28 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_CNTLM 0x00000005 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK 0x0f000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT 24 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_0 0x00000008 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_1 0x00000009 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_2 0x0000000a + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_3 0x0000000b + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_0 0x0000000c + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_1 0x0000000d + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_2 0x0000000e + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_3 0x0000000f + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0 0x00000010 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK 0x0000000f +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK 0x000000f0 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT 4 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK 0x00000f00 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT 8 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK 0x0000f000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT 12 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK 0x000f0000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT 16 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK 0x00f00000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT 20 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK 0x0f000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT 24 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK 0xf0000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT 28 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1 0x00000011 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK 0x0000000f +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK 0x000000f0 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT 4 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK 0x00000f00 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT 8 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK 0x0000f000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT 12 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK 0x000f0000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT 16 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK 0x00f00000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT 20 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK 0x0f000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT 24 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK 0xf0000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT 28 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_TRACE_BUF1 0x0000002f + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_TRACE_BUF2 0x00000030 + #endif /* A6XX_XML */ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c index bbb8126ec5c5..d4e98e5876bc 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c @@ -2,7 +2,6 @@ /* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ #include <linux/clk.h> -#include <linux/iopoll.h> #include <linux/pm_opp.h> #include <soc/qcom/cmd-db.h> @@ -42,9 +41,6 @@ static irqreturn_t a6xx_hfi_irq(int irq, void *data) status = gmu_read(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO); gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, status); - if (status & A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ) - tasklet_schedule(&gmu->hfi_tasklet); - if (status & A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) { dev_err_ratelimited(gmu->dev, "GMU firmware fault\n"); @@ -65,12 +61,14 @@ static bool a6xx_gmu_gx_is_on(struct a6xx_gmu *gmu) A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_CLK_OFF)); } -static int a6xx_gmu_set_freq(struct a6xx_gmu *gmu, int index) +static void __a6xx_gmu_set_freq(struct a6xx_gmu *gmu, int index) { + int ret; + gmu_write(gmu, REG_A6XX_GMU_DCVS_ACK_OPTION, 0); gmu_write(gmu, REG_A6XX_GMU_DCVS_PERF_SETTING, - ((index << 24) & 0xff) | (3 & 0xf)); + ((3 & 0xf) << 28) | index); /* * Send an invalid index as a vote for the bus bandwidth and let the @@ -82,7 +80,37 @@ static int a6xx_gmu_set_freq(struct a6xx_gmu *gmu, int index) a6xx_gmu_set_oob(gmu, GMU_OOB_DCVS_SET); a6xx_gmu_clear_oob(gmu, GMU_OOB_DCVS_SET); - return gmu_read(gmu, REG_A6XX_GMU_DCVS_RETURN); + ret = gmu_read(gmu, REG_A6XX_GMU_DCVS_RETURN); + if (ret) + dev_err(gmu->dev, "GMU set GPU frequency error: %d\n", ret); + + gmu->freq = gmu->gpu_freqs[index]; +} + +void a6xx_gmu_set_freq(struct msm_gpu *gpu, unsigned long freq) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + u32 perf_index = 0; + + if (freq == gmu->freq) + return; + + for (perf_index = 0; perf_index < gmu->nr_gpu_freqs - 1; perf_index++) + if (freq == gmu->gpu_freqs[perf_index]) + break; + + __a6xx_gmu_set_freq(gmu, perf_index); +} + +unsigned long a6xx_gmu_get_freq(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + + return gmu->freq; } static bool a6xx_gmu_check_idle_level(struct a6xx_gmu *gmu) @@ -135,9 +163,6 @@ static int a6xx_gmu_hfi_start(struct a6xx_gmu *gmu) u32 val; int ret; - gmu_rmw(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, - A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ, 0); - gmu_write(gmu, REG_A6XX_GMU_HFI_CTRL_INIT, 1); ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_HFI_CTRL_STATUS, val, @@ -348,8 +373,23 @@ static void a6xx_rpmh_stop(struct a6xx_gmu *gmu) gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 0); } +static inline void pdc_write(void __iomem *ptr, u32 offset, u32 value) +{ + return msm_writel(value, ptr + (offset << 2)); +} + +static void __iomem *a6xx_gmu_get_mmio(struct platform_device *pdev, + const char *name); + static void a6xx_gmu_rpmh_init(struct a6xx_gmu *gmu) { + struct platform_device *pdev = to_platform_device(gmu->dev); + void __iomem *pdcptr = a6xx_gmu_get_mmio(pdev, "gmu_pdc"); + void __iomem *seqptr = a6xx_gmu_get_mmio(pdev, "gmu_pdc_seq"); + + if (!pdcptr || !seqptr) + goto err; + /* Disable SDE clock gating */ gmu_write(gmu, REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0, BIT(24)); @@ -374,44 +414,48 @@ static void a6xx_gmu_rpmh_init(struct a6xx_gmu *gmu) gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 4, 0x0020e8a8); /* Load PDC sequencer uCode for power up and power down sequence */ - pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0, 0xfebea1e1); - pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 1, 0xa5a4a3a2); - pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 2, 0x8382a6e0); - pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 3, 0xbce3e284); - pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 4, 0x002081fc); + pdc_write(seqptr, REG_A6XX_PDC_GPU_SEQ_MEM_0, 0xfebea1e1); + pdc_write(seqptr, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 1, 0xa5a4a3a2); + pdc_write(seqptr, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 2, 0x8382a6e0); + pdc_write(seqptr, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 3, 0xbce3e284); + pdc_write(seqptr, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 4, 0x002081fc); /* Set TCS commands used by PDC sequence for low power modes */ - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK, 7); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK, 0); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CONTROL, 0); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID, 0x10108); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR, 0x30010); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA, 1); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 4, 0x10108); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 4, 0x30000); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 4, 0x0); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 8, 0x10108); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 8, 0x30080); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 8, 0x0); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK, 7); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK, 0); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CONTROL, 0); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID, 0x10108); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR, 0x30010); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA, 2); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 4, 0x10108); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 4, 0x30000); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 4, 0x3); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 8, 0x10108); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 8, 0x30080); - pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 8, 0x3); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK, 7); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CONTROL, 0); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID, 0x10108); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR, 0x30010); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA, 1); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 4, 0x10108); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 4, 0x30000); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 4, 0x0); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 8, 0x10108); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 8, 0x30080); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 8, 0x0); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK, 7); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CONTROL, 0); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID, 0x10108); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR, 0x30010); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA, 2); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 4, 0x10108); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 4, 0x30000); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 4, 0x3); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 8, 0x10108); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 8, 0x30080); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 8, 0x3); /* Setup GPU PDC */ - pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_START_ADDR, 0); - pdc_write(gmu, REG_A6XX_PDC_GPU_ENABLE_PDC, 0x80000001); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_SEQ_START_ADDR, 0); + pdc_write(pdcptr, REG_A6XX_PDC_GPU_ENABLE_PDC, 0x80000001); /* ensure no writes happen before the uCode is fully written */ wmb(); + +err: + devm_iounmap(gmu->dev, pdcptr); + devm_iounmap(gmu->dev, seqptr); } /* @@ -547,8 +591,7 @@ static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state) } #define A6XX_HFI_IRQ_MASK \ - (A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ | \ - A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) + (A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) #define A6XX_GMU_IRQ_MASK \ (A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE | \ @@ -626,7 +669,7 @@ int a6xx_gmu_reset(struct a6xx_gpu *a6xx_gpu) ret = a6xx_hfi_start(gmu, GMU_COLD_BOOT); /* Set the GPU back to the highest power frequency */ - a6xx_gmu_set_freq(gmu, gmu->nr_gpu_freqs - 1); + __a6xx_gmu_set_freq(gmu, gmu->nr_gpu_freqs - 1); out: if (ret) @@ -665,7 +708,7 @@ int a6xx_gmu_resume(struct a6xx_gpu *a6xx_gpu) ret = a6xx_hfi_start(gmu, status); /* Set the GPU to the highest power frequency */ - a6xx_gmu_set_freq(gmu, gmu->nr_gpu_freqs - 1); + __a6xx_gmu_set_freq(gmu, gmu->nr_gpu_freqs - 1); out: /* Make sure to turn off the boot OOB request on error */ @@ -1140,7 +1183,7 @@ int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node) gmu->dev = &pdev->dev; - of_dma_configure(gmu->dev, node, false); + of_dma_configure(gmu->dev, node, true); /* Fow now, don't do anything fancy until we get our feet under us */ gmu->idle_level = GMU_IDLE_STATE_ACTIVE; @@ -1170,11 +1213,7 @@ int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node) /* Map the GMU registers */ gmu->mmio = a6xx_gmu_get_mmio(pdev, "gmu"); - - /* Map the GPU power domain controller registers */ - gmu->pdc_mmio = a6xx_gmu_get_mmio(pdev, "gmu_pdc"); - - if (IS_ERR(gmu->mmio) || IS_ERR(gmu->pdc_mmio)) + if (IS_ERR(gmu->mmio)) goto err; /* Get the HFI and GMU interrupts */ @@ -1184,9 +1223,6 @@ int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node) if (gmu->hfi_irq < 0 || gmu->gmu_irq < 0) goto err; - /* Set up a tasklet to handle GMU HFI responses */ - tasklet_init(&gmu->hfi_tasklet, a6xx_hfi_task, (unsigned long) gmu); - /* Get the power levels for the GMU and GPU */ a6xx_gmu_pwrlevels_probe(gmu); diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h index d9a386c18799..35f765afae45 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h @@ -4,6 +4,7 @@ #ifndef _A6XX_GMU_H_ #define _A6XX_GMU_H_ +#include <linux/iopoll.h> #include <linux/interrupt.h> #include "msm_drv.h" #include "a6xx_hfi.h" @@ -47,7 +48,6 @@ struct a6xx_gmu { struct device *dev; void * __iomem mmio; - void * __iomem pdc_mmio; int hfi_irq; int gmu_irq; @@ -74,6 +74,8 @@ struct a6xx_gmu { unsigned long gmu_freqs[4]; u32 cx_arc_votes[4]; + unsigned long freq; + struct a6xx_hfi_queue queues[2]; struct tasklet_struct hfi_tasklet; @@ -89,11 +91,6 @@ static inline void gmu_write(struct a6xx_gmu *gmu, u32 offset, u32 value) return msm_writel(value, gmu->mmio + (offset << 2)); } -static inline void pdc_write(struct a6xx_gmu *gmu, u32 offset, u32 value) -{ - return msm_writel(value, gmu->pdc_mmio + (offset << 2)); -} - static inline void gmu_rmw(struct a6xx_gmu *gmu, u32 reg, u32 mask, u32 or) { u32 val = gmu_read(gmu, reg); @@ -103,6 +100,16 @@ static inline void gmu_rmw(struct a6xx_gmu *gmu, u32 reg, u32 mask, u32 or) gmu_write(gmu, reg, val | or); } +static inline u64 gmu_read64(struct a6xx_gmu *gmu, u32 lo, u32 hi) +{ + u64 val; + + val = (u64) msm_readl(gmu->mmio + (lo << 2)); + val |= ((u64) msm_readl(gmu->mmio + (hi << 2)) << 32); + + return val; +} + #define gmu_poll_timeout(gmu, addr, val, cond, interval, timeout) \ readl_poll_timeout((gmu)->mmio + ((addr) << 2), val, cond, \ interval, timeout) @@ -157,6 +164,4 @@ void a6xx_hfi_init(struct a6xx_gmu *gmu); int a6xx_hfi_start(struct a6xx_gmu *gmu, int boot_state); void a6xx_hfi_stop(struct a6xx_gmu *gmu); -void a6xx_hfi_task(unsigned long data); - #endif diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h index ef68098d2adc..db56f263ed77 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: @@ -167,8 +167,8 @@ static inline uint32_t A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_MIN_PASS_LENGTH(uint32_ #define REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS 0x000050d0 #define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWERING_OFF 0x00000001 #define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWERING_ON 0x00000002 -#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWER_ON 0x00000004 -#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWER_OFF 0x00000008 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWER_OFF 0x00000004 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWER_ON 0x00000008 #define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SP_CLOCK_OFF 0x00000010 #define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GMU_UP_POWER_STATE 0x00000020 #define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_GDSC_POWER_OFF 0x00000040 diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index c629f742a1d1..631257c297fd 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -7,6 +7,8 @@ #include "a6xx_gpu.h" #include "a6xx_gmu.xml.h" +#include <linux/devfreq.h> + static inline bool _a6xx_check_idle(struct msm_gpu *gpu) { struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); @@ -438,10 +440,8 @@ static int a6xx_hw_init(struct msm_gpu *gpu) gpu_write(gpu, REG_A6XX_CP_PROTECT(22), A6XX_PROTECT_RW(0x900, 0x4d)); gpu_write(gpu, REG_A6XX_CP_PROTECT(23), A6XX_PROTECT_RW(0x98d, 0x76)); gpu_write(gpu, REG_A6XX_CP_PROTECT(24), - A6XX_PROTECT_RDONLY(0x8d0, 0x23)); - gpu_write(gpu, REG_A6XX_CP_PROTECT(25), A6XX_PROTECT_RDONLY(0x980, 0x4)); - gpu_write(gpu, REG_A6XX_CP_PROTECT(26), A6XX_PROTECT_RW(0xa630, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(25), A6XX_PROTECT_RW(0xa630, 0x0)); /* Enable interrupts */ gpu_write(gpu, REG_A6XX_RBBM_INT_0_MASK, A6XX_INT_MASK); @@ -682,6 +682,8 @@ static int a6xx_pm_resume(struct msm_gpu *gpu) gpu->needs_hw_init = true; + msm_gpu_resume_devfreq(gpu); + return ret; } @@ -690,6 +692,8 @@ static int a6xx_pm_suspend(struct msm_gpu *gpu) struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + devfreq_suspend_device(gpu->devfreq.devfreq); + /* * Make sure the GMU is idle before continuing (because some transitions * may use VBIF @@ -744,7 +748,7 @@ static void a6xx_destroy(struct msm_gpu *gpu) if (a6xx_gpu->sqe_bo) { if (a6xx_gpu->sqe_iova) msm_gem_put_iova(a6xx_gpu->sqe_bo, gpu->aspace); - drm_gem_object_unreference_unlocked(a6xx_gpu->sqe_bo); + drm_gem_object_put_unlocked(a6xx_gpu->sqe_bo); } a6xx_gmu_remove(a6xx_gpu); @@ -753,6 +757,27 @@ static void a6xx_destroy(struct msm_gpu *gpu) kfree(a6xx_gpu); } +static unsigned long a6xx_gpu_busy(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + u64 busy_cycles, busy_time; + + busy_cycles = gmu_read64(&a6xx_gpu->gmu, + REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_0_L, + REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_0_H); + + busy_time = (busy_cycles - gpu->devfreq.busy_cycles) * 10; + do_div(busy_time, 192); + + gpu->devfreq.busy_cycles = busy_cycles; + + if (WARN_ON(busy_time > ~0LU)) + return ~0LU; + + return (unsigned long)busy_time; +} + static const struct adreno_gpu_funcs funcs = { .base = { .get_param = adreno_get_param, @@ -768,6 +793,9 @@ static const struct adreno_gpu_funcs funcs = { #if defined(CONFIG_DEBUG_FS) || defined(CONFIG_DEV_COREDUMP) .show = a6xx_show, #endif + .gpu_busy = a6xx_gpu_busy, + .gpu_get_freq = a6xx_gmu_get_freq, + .gpu_set_freq = a6xx_gmu_set_freq, }, .get_timestamp = a6xx_get_timestamp, }; @@ -799,7 +827,7 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) } /* Check if there is a GMU phandle and set it up */ - node = of_parse_phandle(pdev->dev.of_node, "gmu", 0); + node = of_parse_phandle(pdev->dev.of_node, "qcom,gmu", 0); /* FIXME: How do we gracefully handle this? */ BUG_ON(!node); diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h index dd69e5b0e692..4127dcebc202 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -56,5 +56,6 @@ void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state); int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node); void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu); - +void a6xx_gmu_set_freq(struct msm_gpu *gpu, unsigned long freq); +unsigned long a6xx_gmu_get_freq(struct msm_gpu *gpu); #endif /* __A6XX_GPU_H__ */ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c index f19ef4cb6ea4..6ff9baec2658 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c @@ -79,83 +79,72 @@ static int a6xx_hfi_queue_write(struct a6xx_gmu *gmu, return 0; } -struct a6xx_hfi_response { - u32 id; - u32 seqnum; - struct list_head node; - struct completion complete; - - u32 error; - u32 payload[16]; -}; +static int a6xx_hfi_wait_for_ack(struct a6xx_gmu *gmu, u32 id, u32 seqnum, + u32 *payload, u32 payload_size) +{ + struct a6xx_hfi_queue *queue = &gmu->queues[HFI_RESPONSE_QUEUE]; + u32 val; + int ret; -/* - * Incoming HFI ack messages can come in out of order so we need to store all - * the pending messages on a list until they are handled. - */ -static spinlock_t hfi_ack_lock = __SPIN_LOCK_UNLOCKED(message_lock); -static LIST_HEAD(hfi_ack_list); + /* Wait for a response */ + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO, val, + val & A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ, 100, 5000); -static void a6xx_hfi_handle_ack(struct a6xx_gmu *gmu, - struct a6xx_hfi_msg_response *msg) -{ - struct a6xx_hfi_response *resp; - u32 id, seqnum; - - /* msg->ret_header contains the header of the message being acked */ - id = HFI_HEADER_ID(msg->ret_header); - seqnum = HFI_HEADER_SEQNUM(msg->ret_header); - - spin_lock(&hfi_ack_lock); - list_for_each_entry(resp, &hfi_ack_list, node) { - if (resp->id == id && resp->seqnum == seqnum) { - resp->error = msg->error; - memcpy(resp->payload, msg->payload, - sizeof(resp->payload)); - - complete(&resp->complete); - spin_unlock(&hfi_ack_lock); - return; - } + if (ret) { + dev_err(gmu->dev, + "Message %s id %d timed out waiting for response\n", + a6xx_hfi_msg_id[id], seqnum); + return -ETIMEDOUT; } - spin_unlock(&hfi_ack_lock); - dev_err(gmu->dev, "Nobody was waiting for HFI message %d\n", seqnum); -} + /* Clear the interrupt */ + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, + A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ); -static void a6xx_hfi_handle_error(struct a6xx_gmu *gmu, - struct a6xx_hfi_msg_response *msg) -{ - struct a6xx_hfi_msg_error *error = (struct a6xx_hfi_msg_error *) msg; + for (;;) { + struct a6xx_hfi_msg_response resp; - dev_err(gmu->dev, "GMU firmware error %d\n", error->code); -} + /* Get the next packet */ + ret = a6xx_hfi_queue_read(queue, (u32 *) &resp, + sizeof(resp) >> 2); -void a6xx_hfi_task(unsigned long data) -{ - struct a6xx_gmu *gmu = (struct a6xx_gmu *) data; - struct a6xx_hfi_queue *queue = &gmu->queues[HFI_RESPONSE_QUEUE]; - struct a6xx_hfi_msg_response resp; + /* If the queue is empty our response never made it */ + if (!ret) { + dev_err(gmu->dev, + "The HFI response queue is unexpectedly empty\n"); - for (;;) { - u32 id; - int ret = a6xx_hfi_queue_read(queue, (u32 *) &resp, - sizeof(resp) >> 2); + return -ENOENT; + } + + if (HFI_HEADER_ID(resp.header) == HFI_F2H_MSG_ERROR) { + struct a6xx_hfi_msg_error *error = + (struct a6xx_hfi_msg_error *) &resp; - /* Returns the number of bytes copied or negative on error */ - if (ret <= 0) { - if (ret < 0) - dev_err(gmu->dev, - "Unable to read the HFI message queue\n"); - break; + dev_err(gmu->dev, "GMU firmware error %d\n", + error->code); + continue; + } + + if (seqnum != HFI_HEADER_SEQNUM(resp.ret_header)) { + dev_err(gmu->dev, + "Unexpected message id %d on the response queue\n", + HFI_HEADER_SEQNUM(resp.ret_header)); + continue; + } + + if (resp.error) { + dev_err(gmu->dev, + "Message %s id %d returned error %d\n", + a6xx_hfi_msg_id[id], seqnum, resp.error); + return -EINVAL; } - id = HFI_HEADER_ID(resp.header); + /* All is well, copy over the buffer */ + if (payload && payload_size) + memcpy(payload, resp.payload, + min_t(u32, payload_size, sizeof(resp.payload))); - if (id == HFI_F2H_MSG_ACK) - a6xx_hfi_handle_ack(gmu, &resp); - else if (id == HFI_F2H_MSG_ERROR) - a6xx_hfi_handle_error(gmu, &resp); + return 0; } } @@ -163,7 +152,6 @@ static int a6xx_hfi_send_msg(struct a6xx_gmu *gmu, int id, void *data, u32 size, u32 *payload, u32 payload_size) { struct a6xx_hfi_queue *queue = &gmu->queues[HFI_COMMAND_QUEUE]; - struct a6xx_hfi_response resp = { 0 }; int ret, dwords = size >> 2; u32 seqnum; @@ -173,53 +161,14 @@ static int a6xx_hfi_send_msg(struct a6xx_gmu *gmu, int id, *((u32 *) data) = (seqnum << 20) | (HFI_MSG_CMD << 16) | (dwords << 8) | id; - init_completion(&resp.complete); - resp.id = id; - resp.seqnum = seqnum; - - spin_lock_bh(&hfi_ack_lock); - list_add_tail(&resp.node, &hfi_ack_list); - spin_unlock_bh(&hfi_ack_lock); - ret = a6xx_hfi_queue_write(gmu, queue, data, dwords); if (ret) { dev_err(gmu->dev, "Unable to send message %s id %d\n", a6xx_hfi_msg_id[id], seqnum); - goto out; - } - - /* Wait up to 5 seconds for the response */ - ret = wait_for_completion_timeout(&resp.complete, - msecs_to_jiffies(5000)); - if (!ret) { - dev_err(gmu->dev, - "Message %s id %d timed out waiting for response\n", - a6xx_hfi_msg_id[id], seqnum); - ret = -ETIMEDOUT; - } else - ret = 0; - -out: - spin_lock_bh(&hfi_ack_lock); - list_del(&resp.node); - spin_unlock_bh(&hfi_ack_lock); - - if (ret) return ret; - - if (resp.error) { - dev_err(gmu->dev, "Message %s id %d returned error %d\n", - a6xx_hfi_msg_id[id], seqnum, resp.error); - return -EINVAL; } - if (payload && payload_size) { - int copy = min_t(u32, payload_size, sizeof(resp.payload)); - - memcpy(payload, resp.payload, copy); - } - - return 0; + return a6xx_hfi_wait_for_ack(gmu, id, seqnum, payload, payload_size); } static int a6xx_hfi_send_gmu_init(struct a6xx_gmu *gmu, int boot_state) diff --git a/drivers/gpu/drm/msm/adreno/adreno_common.xml.h b/drivers/gpu/drm/msm/adreno/adreno_common.xml.h index 5dace1350810..1318959d504d 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_common.xml.h +++ b/drivers/gpu/drm/msm/adreno/adreno_common.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c index 7d3e9a129ac7..86abdb2b3a9c 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_device.c +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c @@ -120,6 +120,7 @@ static const struct adreno_info gpulist[] = { [ADRENO_FW_GMU] = "a630_gmu.bin", }, .gmem = SZ_1M, + .inactive_period = DRM_MSM_INACTIVE_PERIOD, .init = a6xx_gpu_init, }, }; diff --git a/drivers/gpu/drm/msm/adreno/adreno_pm4.xml.h b/drivers/gpu/drm/msm/adreno/adreno_pm4.xml.h index 03a91e10b310..15eb03bed984 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_pm4.xml.h +++ b/drivers/gpu/drm/msm/adreno/adreno_pm4.xml.h @@ -12,12 +12,12 @@ The rules-ng-ng source files this header was generated from are: - /home/robclark/src/envytools/rnndb/freedreno_copyright.xml ( 1572 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a2xx.xml ( 36805 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/adreno_common.xml ( 13634 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42393 bytes, from 2018-08-06 18:45:45) +- /home/robclark/src/envytools/rnndb/adreno/adreno_pm4.xml ( 42585 bytes, from 2018-10-04 19:06:37) - /home/robclark/src/envytools/rnndb/adreno/a3xx.xml ( 83840 bytes, from 2018-07-03 19:37:13) - /home/robclark/src/envytools/rnndb/adreno/a4xx.xml ( 112086 bytes, from 2018-07-03 19:37:13) -- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 101627 bytes, from 2018-08-06 18:45:45) -- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-07-03 19:37:13) +- /home/robclark/src/envytools/rnndb/adreno/a5xx.xml ( 147240 bytes, from 2018-10-04 19:06:37) +- /home/robclark/src/envytools/rnndb/adreno/a6xx.xml ( 139581 bytes, from 2018-10-04 19:06:42) +- /home/robclark/src/envytools/rnndb/adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-09-14 13:03:07) - /home/robclark/src/envytools/rnndb/adreno/ocmem.xml ( 1773 bytes, from 2018-07-03 19:37:13) Copyright (C) 2013-2018 by the following authors: @@ -237,7 +237,7 @@ enum adreno_pm4_type3_packets { CP_UNK_A6XX_14 = 20, CP_UNK_A6XX_36 = 54, CP_UNK_A6XX_55 = 85, - UNK_A6XX_6D = 109, + CP_REG_WRITE = 109, }; enum adreno_state_block { @@ -968,19 +968,19 @@ static inline uint32_t CP_SET_BIN_DATA5_4_BIN_SIZE_ADDRESS_HI(uint32_t val) } #define REG_CP_SET_BIN_DATA5_5 0x00000005 -#define CP_SET_BIN_DATA5_5_XXX_ADDRESS_LO__MASK 0xffffffff -#define CP_SET_BIN_DATA5_5_XXX_ADDRESS_LO__SHIFT 0 -static inline uint32_t CP_SET_BIN_DATA5_5_XXX_ADDRESS_LO(uint32_t val) +#define CP_SET_BIN_DATA5_5_BIN_DATA_ADDR2_LO__MASK 0xffffffff +#define CP_SET_BIN_DATA5_5_BIN_DATA_ADDR2_LO__SHIFT 0 +static inline uint32_t CP_SET_BIN_DATA5_5_BIN_DATA_ADDR2_LO(uint32_t val) { - return ((val) << CP_SET_BIN_DATA5_5_XXX_ADDRESS_LO__SHIFT) & CP_SET_BIN_DATA5_5_XXX_ADDRESS_LO__MASK; + return ((val) << CP_SET_BIN_DATA5_5_BIN_DATA_ADDR2_LO__SHIFT) & CP_SET_BIN_DATA5_5_BIN_DATA_ADDR2_LO__MASK; } #define REG_CP_SET_BIN_DATA5_6 0x00000006 -#define CP_SET_BIN_DATA5_6_XXX_ADDRESS_HI__MASK 0xffffffff -#define CP_SET_BIN_DATA5_6_XXX_ADDRESS_HI__SHIFT 0 -static inline uint32_t CP_SET_BIN_DATA5_6_XXX_ADDRESS_HI(uint32_t val) +#define CP_SET_BIN_DATA5_6_BIN_DATA_ADDR2_LO__MASK 0xffffffff +#define CP_SET_BIN_DATA5_6_BIN_DATA_ADDR2_LO__SHIFT 0 +static inline uint32_t CP_SET_BIN_DATA5_6_BIN_DATA_ADDR2_LO(uint32_t val) { - return ((val) << CP_SET_BIN_DATA5_6_XXX_ADDRESS_HI__SHIFT) & CP_SET_BIN_DATA5_6_XXX_ADDRESS_HI__MASK; + return ((val) << CP_SET_BIN_DATA5_6_BIN_DATA_ADDR2_LO__SHIFT) & CP_SET_BIN_DATA5_6_BIN_DATA_ADDR2_LO__MASK; } #define REG_CP_REG_TO_MEM_0 0x00000000 diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c index 80cbf75bc2ff..d4530d60767b 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c @@ -47,237 +47,17 @@ #define LEFT_MIXER 0 #define RIGHT_MIXER 1 -#define MISR_BUFF_SIZE 256 - -static inline struct dpu_kms *_dpu_crtc_get_kms(struct drm_crtc *crtc) -{ - struct msm_drm_private *priv; - - if (!crtc || !crtc->dev || !crtc->dev->dev_private) { - DPU_ERROR("invalid crtc\n"); - return NULL; - } - priv = crtc->dev->dev_private; - if (!priv || !priv->kms) { - DPU_ERROR("invalid kms\n"); - return NULL; - } - - return to_dpu_kms(priv->kms); -} - -static inline int _dpu_crtc_power_enable(struct dpu_crtc *dpu_crtc, bool enable) -{ - struct drm_crtc *crtc; - struct msm_drm_private *priv; - struct dpu_kms *dpu_kms; - - if (!dpu_crtc) { - DPU_ERROR("invalid dpu crtc\n"); - return -EINVAL; - } - - crtc = &dpu_crtc->base; - if (!crtc->dev || !crtc->dev->dev_private) { - DPU_ERROR("invalid drm device\n"); - return -EINVAL; - } - - priv = crtc->dev->dev_private; - if (!priv->kms) { - DPU_ERROR("invalid kms\n"); - return -EINVAL; - } - - dpu_kms = to_dpu_kms(priv->kms); - - if (enable) - pm_runtime_get_sync(&dpu_kms->pdev->dev); - else - pm_runtime_put_sync(&dpu_kms->pdev->dev); - - return 0; -} - -/** - * _dpu_crtc_rp_to_crtc - get crtc from resource pool object - * @rp: Pointer to resource pool - * return: Pointer to drm crtc if success; null otherwise - */ -static struct drm_crtc *_dpu_crtc_rp_to_crtc(struct dpu_crtc_respool *rp) -{ - if (!rp) - return NULL; - - return container_of(rp, struct dpu_crtc_state, rp)->base.crtc; -} - -/** - * _dpu_crtc_rp_reclaim - reclaim unused, or all if forced, resources in pool - * @rp: Pointer to resource pool - * @force: True to reclaim all resources; otherwise, reclaim only unused ones - * return: None - */ -static void _dpu_crtc_rp_reclaim(struct dpu_crtc_respool *rp, bool force) +static inline int _dpu_crtc_get_mixer_width(struct dpu_crtc_state *cstate, + struct drm_display_mode *mode) { - struct dpu_crtc_res *res, *next; - struct drm_crtc *crtc; - - crtc = _dpu_crtc_rp_to_crtc(rp); - if (!crtc) { - DPU_ERROR("invalid crtc\n"); - return; - } - - DPU_DEBUG("crtc%d.%u %s\n", crtc->base.id, rp->sequence_id, - force ? "destroy" : "free_unused"); - - list_for_each_entry_safe(res, next, &rp->res_list, list) { - if (!force && !(res->flags & DPU_CRTC_RES_FLAG_FREE)) - continue; - DPU_DEBUG("crtc%d.%u reclaim res:0x%x/0x%llx/%pK/%d\n", - crtc->base.id, rp->sequence_id, - res->type, res->tag, res->val, - atomic_read(&res->refcount)); - list_del(&res->list); - if (res->ops.put) - res->ops.put(res->val); - kfree(res); - } + return mode->hdisplay / cstate->num_mixers; } -/** - * _dpu_crtc_rp_free_unused - free unused resource in pool - * @rp: Pointer to resource pool - * return: none - */ -static void _dpu_crtc_rp_free_unused(struct dpu_crtc_respool *rp) -{ - mutex_lock(rp->rp_lock); - _dpu_crtc_rp_reclaim(rp, false); - mutex_unlock(rp->rp_lock); -} - -/** - * _dpu_crtc_rp_destroy - destroy resource pool - * @rp: Pointer to resource pool - * return: None - */ -static void _dpu_crtc_rp_destroy(struct dpu_crtc_respool *rp) -{ - mutex_lock(rp->rp_lock); - list_del_init(&rp->rp_list); - _dpu_crtc_rp_reclaim(rp, true); - mutex_unlock(rp->rp_lock); -} - -/** - * _dpu_crtc_hw_blk_get - get callback for hardware block - * @val: Resource handle - * @type: Resource type - * @tag: Search tag for given resource - * return: Resource handle - */ -static void *_dpu_crtc_hw_blk_get(void *val, u32 type, u64 tag) -{ - DPU_DEBUG("res:%d/0x%llx/%pK\n", type, tag, val); - return dpu_hw_blk_get(val, type, tag); -} - -/** - * _dpu_crtc_hw_blk_put - put callback for hardware block - * @val: Resource handle - * return: None - */ -static void _dpu_crtc_hw_blk_put(void *val) -{ - DPU_DEBUG("res://%pK\n", val); - dpu_hw_blk_put(val); -} - -/** - * _dpu_crtc_rp_duplicate - duplicate resource pool and reset reference count - * @rp: Pointer to original resource pool - * @dup_rp: Pointer to duplicated resource pool - * return: None - */ -static void _dpu_crtc_rp_duplicate(struct dpu_crtc_respool *rp, - struct dpu_crtc_respool *dup_rp) -{ - struct dpu_crtc_res *res, *dup_res; - struct drm_crtc *crtc; - - if (!rp || !dup_rp || !rp->rp_head) { - DPU_ERROR("invalid resource pool\n"); - return; - } - - crtc = _dpu_crtc_rp_to_crtc(rp); - if (!crtc) { - DPU_ERROR("invalid crtc\n"); - return; - } - - DPU_DEBUG("crtc%d.%u duplicate\n", crtc->base.id, rp->sequence_id); - - mutex_lock(rp->rp_lock); - dup_rp->sequence_id = rp->sequence_id + 1; - INIT_LIST_HEAD(&dup_rp->res_list); - dup_rp->ops = rp->ops; - list_for_each_entry(res, &rp->res_list, list) { - dup_res = kzalloc(sizeof(struct dpu_crtc_res), GFP_KERNEL); - if (!dup_res) { - mutex_unlock(rp->rp_lock); - return; - } - INIT_LIST_HEAD(&dup_res->list); - atomic_set(&dup_res->refcount, 0); - dup_res->type = res->type; - dup_res->tag = res->tag; - dup_res->val = res->val; - dup_res->ops = res->ops; - dup_res->flags = DPU_CRTC_RES_FLAG_FREE; - DPU_DEBUG("crtc%d.%u dup res:0x%x/0x%llx/%pK/%d\n", - crtc->base.id, dup_rp->sequence_id, - dup_res->type, dup_res->tag, dup_res->val, - atomic_read(&dup_res->refcount)); - list_add_tail(&dup_res->list, &dup_rp->res_list); - if (dup_res->ops.get) - dup_res->ops.get(dup_res->val, 0, -1); - } - - dup_rp->rp_lock = rp->rp_lock; - dup_rp->rp_head = rp->rp_head; - INIT_LIST_HEAD(&dup_rp->rp_list); - list_add_tail(&dup_rp->rp_list, rp->rp_head); - mutex_unlock(rp->rp_lock); -} - -/** - * _dpu_crtc_rp_reset - reset resource pool after allocation - * @rp: Pointer to original resource pool - * @rp_lock: Pointer to serialization resource pool lock - * @rp_head: Pointer to crtc resource pool head - * return: None - */ -static void _dpu_crtc_rp_reset(struct dpu_crtc_respool *rp, - struct mutex *rp_lock, struct list_head *rp_head) +static inline struct dpu_kms *_dpu_crtc_get_kms(struct drm_crtc *crtc) { - if (!rp || !rp_lock || !rp_head) { - DPU_ERROR("invalid resource pool\n"); - return; - } + struct msm_drm_private *priv = crtc->dev->dev_private; - mutex_lock(rp_lock); - rp->rp_lock = rp_lock; - rp->rp_head = rp_head; - INIT_LIST_HEAD(&rp->rp_list); - rp->sequence_id = 0; - INIT_LIST_HEAD(&rp->res_list); - rp->ops.get = _dpu_crtc_hw_blk_get; - rp->ops.put = _dpu_crtc_hw_blk_put; - list_add_tail(&rp->rp_list, rp->rp_head); - mutex_unlock(rp_lock); + return to_dpu_kms(priv->kms); } static void dpu_crtc_destroy(struct drm_crtc *crtc) @@ -297,14 +77,29 @@ static void dpu_crtc_destroy(struct drm_crtc *crtc) } static void _dpu_crtc_setup_blend_cfg(struct dpu_crtc_mixer *mixer, - struct dpu_plane_state *pstate) + struct dpu_plane_state *pstate, struct dpu_format *format) { struct dpu_hw_mixer *lm = mixer->hw_lm; + uint32_t blend_op; + struct drm_format_name_buf format_name; /* default to opaque blending */ - lm->ops.setup_blend_config(lm, pstate->stage, 0XFF, 0, - DPU_BLEND_FG_ALPHA_FG_CONST | - DPU_BLEND_BG_ALPHA_BG_CONST); + blend_op = DPU_BLEND_FG_ALPHA_FG_CONST | + DPU_BLEND_BG_ALPHA_BG_CONST; + + if (format->alpha_enable) { + /* coverage blending */ + blend_op = DPU_BLEND_FG_ALPHA_FG_PIXEL | + DPU_BLEND_BG_ALPHA_FG_PIXEL | + DPU_BLEND_BG_INV_ALPHA; + } + + lm->ops.setup_blend_config(lm, pstate->stage, + 0xFF, 0, blend_op); + + DPU_DEBUG("format:%s, alpha_en:%u blend_op:0x%x\n", + drm_get_format_name(format->base.pixel_format, &format_name), + format->alpha_enable, blend_op); } static void _dpu_crtc_program_lm_output_roi(struct drm_crtc *crtc) @@ -317,9 +112,9 @@ static void _dpu_crtc_program_lm_output_roi(struct drm_crtc *crtc) crtc_state = to_dpu_crtc_state(crtc->state); lm_horiz_position = 0; - for (lm_idx = 0; lm_idx < dpu_crtc->num_mixers; lm_idx++) { + for (lm_idx = 0; lm_idx < crtc_state->num_mixers; lm_idx++) { const struct drm_rect *lm_roi = &crtc_state->lm_bounds[lm_idx]; - struct dpu_hw_mixer *hw_lm = dpu_crtc->mixers[lm_idx].hw_lm; + struct dpu_hw_mixer *hw_lm = crtc_state->mixers[lm_idx].hw_lm; struct dpu_hw_mixer_cfg cfg; if (!lm_roi || !drm_rect_visible(lm_roi)) @@ -339,28 +134,17 @@ static void _dpu_crtc_blend_setup_mixer(struct drm_crtc *crtc, struct drm_plane *plane; struct drm_framebuffer *fb; struct drm_plane_state *state; - struct dpu_crtc_state *cstate; + struct dpu_crtc_state *cstate = to_dpu_crtc_state(crtc->state); struct dpu_plane_state *pstate = NULL; struct dpu_format *format; - struct dpu_hw_ctl *ctl; - struct dpu_hw_mixer *lm; - struct dpu_hw_stage_cfg *stage_cfg; + struct dpu_hw_ctl *ctl = mixer->lm_ctl; + struct dpu_hw_stage_cfg *stage_cfg = &dpu_crtc->stage_cfg; u32 flush_mask; uint32_t stage_idx, lm_idx; int zpos_cnt[DPU_STAGE_MAX + 1] = { 0 }; bool bg_alpha_enable = false; - if (!dpu_crtc || !mixer) { - DPU_ERROR("invalid dpu_crtc or mixer\n"); - return; - } - - ctl = mixer->hw_ctl; - lm = mixer->hw_lm; - stage_cfg = &dpu_crtc->stage_cfg; - cstate = to_dpu_crtc_state(crtc->state); - drm_atomic_crtc_for_each_plane(plane, crtc) { state = plane->state; if (!state) @@ -379,10 +163,6 @@ static void _dpu_crtc_blend_setup_mixer(struct drm_crtc *crtc, state->fb ? state->fb->base.id : -1); format = to_dpu_format(msm_framebuffer_format(pstate->base.fb)); - if (!format) { - DPU_ERROR("invalid format\n"); - return; - } if (pstate->stage == DPU_STAGE_BASE && format->alpha_enable) bg_alpha_enable = true; @@ -400,8 +180,9 @@ static void _dpu_crtc_blend_setup_mixer(struct drm_crtc *crtc, fb ? fb->modifier : 0); /* blend config update */ - for (lm_idx = 0; lm_idx < dpu_crtc->num_mixers; lm_idx++) { - _dpu_crtc_setup_blend_cfg(mixer + lm_idx, pstate); + for (lm_idx = 0; lm_idx < cstate->num_mixers; lm_idx++) { + _dpu_crtc_setup_blend_cfg(mixer + lm_idx, + pstate, format); mixer[lm_idx].flush_mask |= flush_mask; @@ -422,38 +203,25 @@ static void _dpu_crtc_blend_setup_mixer(struct drm_crtc *crtc, */ static void _dpu_crtc_blend_setup(struct drm_crtc *crtc) { - struct dpu_crtc *dpu_crtc; - struct dpu_crtc_state *dpu_crtc_state; - struct dpu_crtc_mixer *mixer; + struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); + struct dpu_crtc_state *cstate = to_dpu_crtc_state(crtc->state); + struct dpu_crtc_mixer *mixer = cstate->mixers; struct dpu_hw_ctl *ctl; struct dpu_hw_mixer *lm; - int i; - if (!crtc) - return; - - dpu_crtc = to_dpu_crtc(crtc); - dpu_crtc_state = to_dpu_crtc_state(crtc->state); - mixer = dpu_crtc->mixers; - DPU_DEBUG("%s\n", dpu_crtc->name); - if (dpu_crtc->num_mixers > CRTC_DUAL_MIXERS) { - DPU_ERROR("invalid number mixers: %d\n", dpu_crtc->num_mixers); - return; - } - - for (i = 0; i < dpu_crtc->num_mixers; i++) { - if (!mixer[i].hw_lm || !mixer[i].hw_ctl) { + for (i = 0; i < cstate->num_mixers; i++) { + if (!mixer[i].hw_lm || !mixer[i].lm_ctl) { DPU_ERROR("invalid lm or ctl assigned to mixer\n"); return; } mixer[i].mixer_op_mode = 0; mixer[i].flush_mask = 0; - if (mixer[i].hw_ctl->ops.clear_all_blendstages) - mixer[i].hw_ctl->ops.clear_all_blendstages( - mixer[i].hw_ctl); + if (mixer[i].lm_ctl->ops.clear_all_blendstages) + mixer[i].lm_ctl->ops.clear_all_blendstages( + mixer[i].lm_ctl); } /* initialize stage cfg */ @@ -461,8 +229,8 @@ static void _dpu_crtc_blend_setup(struct drm_crtc *crtc) _dpu_crtc_blend_setup_mixer(crtc, dpu_crtc, mixer); - for (i = 0; i < dpu_crtc->num_mixers; i++) { - ctl = mixer[i].hw_ctl; + for (i = 0; i < cstate->num_mixers; i++) { + ctl = mixer[i].lm_ctl; lm = mixer[i].hw_lm; lm->ops.setup_alpha_out(lm, mixer[i].mixer_op_mode); @@ -543,34 +311,13 @@ static void dpu_crtc_vblank_cb(void *data) static void dpu_crtc_frame_event_work(struct kthread_work *work) { - struct msm_drm_private *priv; - struct dpu_crtc_frame_event *fevent; - struct drm_crtc *crtc; - struct dpu_crtc *dpu_crtc; - struct dpu_kms *dpu_kms; + struct dpu_crtc_frame_event *fevent = container_of(work, + struct dpu_crtc_frame_event, work); + struct drm_crtc *crtc = fevent->crtc; + struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); unsigned long flags; bool frame_done = false; - if (!work) { - DPU_ERROR("invalid work handle\n"); - return; - } - - fevent = container_of(work, struct dpu_crtc_frame_event, work); - if (!fevent->crtc || !fevent->crtc->state) { - DPU_ERROR("invalid crtc\n"); - return; - } - - crtc = fevent->crtc; - dpu_crtc = to_dpu_crtc(crtc); - - dpu_kms = _dpu_crtc_get_kms(crtc); - if (!dpu_kms) { - DPU_ERROR("invalid kms handle\n"); - return; - } - priv = dpu_kms->dev->dev_private; DPU_ATRACE_BEGIN("crtc_frame_event"); DRM_DEBUG_KMS("crtc%d event:%u ts:%lld\n", crtc->base.id, fevent->event, @@ -636,11 +383,6 @@ static void dpu_crtc_frame_event_cb(void *data, u32 event) unsigned long flags; u32 crtc_id; - if (!crtc || !crtc->dev || !crtc->dev->dev_private) { - DPU_ERROR("invalid parameters\n"); - return; - } - /* Nothing to do on idle event */ if (event & DPU_ENCODER_FRAME_EVENT_IDLE) return; @@ -683,7 +425,7 @@ static void _dpu_crtc_setup_mixer_for_encoder( struct drm_crtc *crtc, struct drm_encoder *enc) { - struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); + struct dpu_crtc_state *cstate = to_dpu_crtc_state(crtc->state); struct dpu_kms *dpu_kms = _dpu_crtc_get_kms(crtc); struct dpu_rm *rm = &dpu_kms->rm; struct dpu_crtc_mixer *mixer; @@ -695,8 +437,8 @@ static void _dpu_crtc_setup_mixer_for_encoder( dpu_rm_init_hw_iter(&ctl_iter, enc->base.id, DPU_HW_BLK_CTL); /* Set up all the mixers and ctls reserved by this encoder */ - for (i = dpu_crtc->num_mixers; i < ARRAY_SIZE(dpu_crtc->mixers); i++) { - mixer = &dpu_crtc->mixers[i]; + for (i = cstate->num_mixers; i < ARRAY_SIZE(cstate->mixers); i++) { + mixer = &cstate->mixers[i]; if (!dpu_rm_get_hw(rm, &lm_iter)) break; @@ -706,14 +448,14 @@ static void _dpu_crtc_setup_mixer_for_encoder( if (!dpu_rm_get_hw(rm, &ctl_iter)) { DPU_DEBUG("no ctl assigned to lm %d, using previous\n", mixer->hw_lm->idx - LM_0); - mixer->hw_ctl = last_valid_ctl; + mixer->lm_ctl = last_valid_ctl; } else { - mixer->hw_ctl = (struct dpu_hw_ctl *)ctl_iter.hw; - last_valid_ctl = mixer->hw_ctl; + mixer->lm_ctl = (struct dpu_hw_ctl *)ctl_iter.hw; + last_valid_ctl = mixer->lm_ctl; } /* Shouldn't happen, mixers are always >= ctls */ - if (!mixer->hw_ctl) { + if (!mixer->lm_ctl) { DPU_ERROR("no valid ctls found for lm %d\n", mixer->hw_lm->idx - LM_0); return; @@ -721,11 +463,11 @@ static void _dpu_crtc_setup_mixer_for_encoder( mixer->encoder = enc; - dpu_crtc->num_mixers++; + cstate->num_mixers++; DPU_DEBUG("setup mixer %d: lm %d\n", i, mixer->hw_lm->idx - LM_0); DPU_DEBUG("setup mixer %d: ctl %d\n", - i, mixer->hw_ctl->idx - CTL_0); + i, mixer->lm_ctl->idx - CTL_0); } } @@ -734,10 +476,6 @@ static void _dpu_crtc_setup_mixers(struct drm_crtc *crtc) struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); struct drm_encoder *enc; - dpu_crtc->num_mixers = 0; - dpu_crtc->mixers_swapped = false; - memset(dpu_crtc->mixers, 0, sizeof(dpu_crtc->mixers)); - mutex_lock(&dpu_crtc->crtc_lock); /* Check for mixers on all encoders attached to this crtc */ list_for_each_entry(enc, &crtc->dev->mode_config.encoder_list, head) { @@ -753,24 +491,13 @@ static void _dpu_crtc_setup_mixers(struct drm_crtc *crtc) static void _dpu_crtc_setup_lm_bounds(struct drm_crtc *crtc, struct drm_crtc_state *state) { - struct dpu_crtc *dpu_crtc; - struct dpu_crtc_state *cstate; - struct drm_display_mode *adj_mode; - u32 crtc_split_width; + struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); + struct dpu_crtc_state *cstate = to_dpu_crtc_state(state); + struct drm_display_mode *adj_mode = &state->adjusted_mode; + u32 crtc_split_width = _dpu_crtc_get_mixer_width(cstate, adj_mode); int i; - if (!crtc || !state) { - DPU_ERROR("invalid args\n"); - return; - } - - dpu_crtc = to_dpu_crtc(crtc); - cstate = to_dpu_crtc_state(state); - - adj_mode = &state->adjusted_mode; - crtc_split_width = dpu_crtc_get_mixer_width(dpu_crtc, cstate, adj_mode); - - for (i = 0; i < dpu_crtc->num_mixers; i++) { + for (i = 0; i < cstate->num_mixers; i++) { struct drm_rect *r = &cstate->lm_bounds[i]; r->x1 = crtc_split_width * i; r->y1 = 0; @@ -787,6 +514,7 @@ static void dpu_crtc_atomic_begin(struct drm_crtc *crtc, struct drm_crtc_state *old_state) { struct dpu_crtc *dpu_crtc; + struct dpu_crtc_state *cstate; struct drm_encoder *encoder; struct drm_device *dev; unsigned long flags; @@ -806,10 +534,11 @@ static void dpu_crtc_atomic_begin(struct drm_crtc *crtc, DPU_DEBUG("crtc%d\n", crtc->base.id); dpu_crtc = to_dpu_crtc(crtc); + cstate = to_dpu_crtc_state(crtc->state); dev = crtc->dev; smmu_state = &dpu_crtc->smmu_state; - if (!dpu_crtc->num_mixers) { + if (!cstate->num_mixers) { _dpu_crtc_setup_mixers(crtc); _dpu_crtc_setup_lm_bounds(crtc, crtc->state); } @@ -836,7 +565,7 @@ static void dpu_crtc_atomic_begin(struct drm_crtc *crtc, * it means we are trying to flush a CRTC whose state is disabled: * nothing else needs to be done. */ - if (unlikely(!dpu_crtc->num_mixers)) + if (unlikely(!cstate->num_mixers)) return; _dpu_crtc_blend_setup(crtc); @@ -861,11 +590,6 @@ static void dpu_crtc_atomic_flush(struct drm_crtc *crtc, unsigned long flags; struct dpu_crtc_state *cstate; - if (!crtc || !crtc->dev || !crtc->dev->dev_private) { - DPU_ERROR("invalid crtc\n"); - return; - } - if (!crtc->state->enable) { DPU_DEBUG("crtc%d -> enable %d, skip atomic_flush\n", crtc->base.id, crtc->state->enable); @@ -900,7 +624,7 @@ static void dpu_crtc_atomic_flush(struct drm_crtc *crtc, * it means we are trying to flush a CRTC whose state is disabled: * nothing else needs to be done. */ - if (unlikely(!dpu_crtc->num_mixers)) + if (unlikely(!cstate->num_mixers)) return; /* @@ -951,8 +675,6 @@ static void dpu_crtc_destroy_state(struct drm_crtc *crtc, DPU_DEBUG("crtc%d\n", crtc->base.id); - _dpu_crtc_rp_destroy(&cstate->rp); - __drm_atomic_helper_crtc_destroy_state(state); kfree(cstate); @@ -960,15 +682,9 @@ static void dpu_crtc_destroy_state(struct drm_crtc *crtc, static int _dpu_crtc_wait_for_frame_done(struct drm_crtc *crtc) { - struct dpu_crtc *dpu_crtc; + struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); int ret, rc = 0; - if (!crtc) { - DPU_ERROR("invalid argument\n"); - return -EINVAL; - } - dpu_crtc = to_dpu_crtc(crtc); - if (!atomic_read(&dpu_crtc->frame_pending)) { DPU_DEBUG("no frames pending\n"); return 0; @@ -989,35 +705,18 @@ static int _dpu_crtc_wait_for_frame_done(struct drm_crtc *crtc) void dpu_crtc_commit_kickoff(struct drm_crtc *crtc) { struct drm_encoder *encoder; - struct drm_device *dev; - struct dpu_crtc *dpu_crtc; - struct msm_drm_private *priv; - struct dpu_kms *dpu_kms; - struct dpu_crtc_state *cstate; + struct drm_device *dev = crtc->dev; + struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); + struct dpu_kms *dpu_kms = _dpu_crtc_get_kms(crtc); + struct dpu_crtc_state *cstate = to_dpu_crtc_state(crtc->state); int ret; - if (!crtc) { - DPU_ERROR("invalid argument\n"); - return; - } - dev = crtc->dev; - dpu_crtc = to_dpu_crtc(crtc); - dpu_kms = _dpu_crtc_get_kms(crtc); - - if (!dpu_kms || !dpu_kms->dev || !dpu_kms->dev->dev_private) { - DPU_ERROR("invalid argument\n"); - return; - } - - priv = dpu_kms->dev->dev_private; - cstate = to_dpu_crtc_state(crtc->state); - /* * If no mixers has been allocated in dpu_crtc_atomic_check(), * it means we are trying to start a CRTC whose state is disabled: * nothing else needs to be done. */ - if (unlikely(!dpu_crtc->num_mixers)) + if (unlikely(!cstate->num_mixers)) return; DPU_ATRACE_BEGIN("crtc_commit"); @@ -1072,33 +771,19 @@ end: * _dpu_crtc_vblank_enable_no_lock - update power resource and vblank request * @dpu_crtc: Pointer to dpu crtc structure * @enable: Whether to enable/disable vblanks - * - * @Return: error code */ -static int _dpu_crtc_vblank_enable_no_lock( +static void _dpu_crtc_vblank_enable_no_lock( struct dpu_crtc *dpu_crtc, bool enable) { - struct drm_device *dev; - struct drm_crtc *crtc; + struct drm_crtc *crtc = &dpu_crtc->base; + struct drm_device *dev = crtc->dev; struct drm_encoder *enc; - if (!dpu_crtc) { - DPU_ERROR("invalid crtc\n"); - return -EINVAL; - } - - crtc = &dpu_crtc->base; - dev = crtc->dev; - if (enable) { - int ret; - /* drop lock since power crtc cb may try to re-acquire lock */ mutex_unlock(&dpu_crtc->crtc_lock); - ret = _dpu_crtc_power_enable(dpu_crtc, true); + pm_runtime_get_sync(dev->dev); mutex_lock(&dpu_crtc->crtc_lock); - if (ret) - return ret; list_for_each_entry(enc, &dev->mode_config.encoder_list, head) { if (enc->crtc != crtc) @@ -1125,11 +810,9 @@ static int _dpu_crtc_vblank_enable_no_lock( /* drop lock since power crtc cb may try to re-acquire lock */ mutex_unlock(&dpu_crtc->crtc_lock); - _dpu_crtc_power_enable(dpu_crtc, false); + pm_runtime_put_sync(dev->dev); mutex_lock(&dpu_crtc->crtc_lock); } - - return 0; } /** @@ -1139,23 +822,7 @@ static int _dpu_crtc_vblank_enable_no_lock( */ static void _dpu_crtc_set_suspend(struct drm_crtc *crtc, bool enable) { - struct dpu_crtc *dpu_crtc; - struct msm_drm_private *priv; - struct dpu_kms *dpu_kms; - int ret = 0; - - if (!crtc || !crtc->dev || !crtc->dev->dev_private) { - DPU_ERROR("invalid crtc\n"); - return; - } - dpu_crtc = to_dpu_crtc(crtc); - priv = crtc->dev->dev_private; - - if (!priv->kms) { - DPU_ERROR("invalid crtc kms\n"); - return; - } - dpu_kms = to_dpu_kms(priv->kms); + struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); DRM_DEBUG_KMS("crtc%d suspend = %d\n", crtc->base.id, enable); @@ -1170,10 +837,7 @@ static void _dpu_crtc_set_suspend(struct drm_crtc *crtc, bool enable) DPU_DEBUG("crtc%d suspend already set to %d, ignoring update\n", crtc->base.id, enable); else if (dpu_crtc->enabled && dpu_crtc->vblank_requested) { - ret = _dpu_crtc_vblank_enable_no_lock(dpu_crtc, !enable); - if (ret) - DPU_ERROR("%s vblank enable failed: %d\n", - dpu_crtc->name, ret); + _dpu_crtc_vblank_enable_no_lock(dpu_crtc, !enable); } dpu_crtc->suspend = enable; @@ -1206,8 +870,6 @@ static struct drm_crtc_state *dpu_crtc_duplicate_state(struct drm_crtc *crtc) /* duplicate base helper */ __drm_atomic_helper_crtc_duplicate_state(crtc, &cstate->base); - _dpu_crtc_rp_duplicate(&old_cstate->rp, &cstate->rp); - return &cstate->base; } @@ -1244,9 +906,6 @@ static void dpu_crtc_reset(struct drm_crtc *crtc) return; } - _dpu_crtc_rp_reset(&cstate->rp, &dpu_crtc->rp_lock, - &dpu_crtc->rp_head); - cstate->base.crtc = crtc; crtc->state = &cstate->base; } @@ -1254,62 +913,19 @@ static void dpu_crtc_reset(struct drm_crtc *crtc) static void dpu_crtc_handle_power_event(u32 event_type, void *arg) { struct drm_crtc *crtc = arg; - struct dpu_crtc *dpu_crtc; + struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); struct drm_encoder *encoder; - struct dpu_crtc_mixer *m; - u32 i, misr_status; - - if (!crtc) { - DPU_ERROR("invalid crtc\n"); - return; - } - dpu_crtc = to_dpu_crtc(crtc); mutex_lock(&dpu_crtc->crtc_lock); trace_dpu_crtc_handle_power_event(DRMID(crtc), event_type); - switch (event_type) { - case DPU_POWER_EVENT_POST_ENABLE: - /* restore encoder; crtc will be programmed during commit */ - drm_for_each_encoder(encoder, crtc->dev) { - if (encoder->crtc != crtc) - continue; - - dpu_encoder_virt_restore(encoder); - } - - for (i = 0; i < dpu_crtc->num_mixers; ++i) { - m = &dpu_crtc->mixers[i]; - if (!m->hw_lm || !m->hw_lm->ops.setup_misr || - !dpu_crtc->misr_enable) - continue; - - m->hw_lm->ops.setup_misr(m->hw_lm, true, - dpu_crtc->misr_frame_count); - } - break; - case DPU_POWER_EVENT_PRE_DISABLE: - for (i = 0; i < dpu_crtc->num_mixers; ++i) { - m = &dpu_crtc->mixers[i]; - if (!m->hw_lm || !m->hw_lm->ops.collect_misr || - !dpu_crtc->misr_enable) - continue; + /* restore encoder; crtc will be programmed during commit */ + drm_for_each_encoder(encoder, crtc->dev) { + if (encoder->crtc != crtc) + continue; - misr_status = m->hw_lm->ops.collect_misr(m->hw_lm); - dpu_crtc->misr_data[i] = misr_status ? misr_status : - dpu_crtc->misr_data[i]; - } - break; - case DPU_POWER_EVENT_POST_DISABLE: - /** - * Nothing to do. All the planes on the CRTC will be - * programmed for every frame - */ - break; - default: - DPU_DEBUG("event:%d not handled\n", event_type); - break; + dpu_encoder_virt_restore(encoder); } mutex_unlock(&dpu_crtc->crtc_lock); @@ -1322,7 +938,6 @@ static void dpu_crtc_disable(struct drm_crtc *crtc) struct drm_display_mode *mode; struct drm_encoder *encoder; struct msm_drm_private *priv; - int ret; unsigned long flags; if (!crtc || !crtc->dev || !crtc->dev->dev_private || !crtc->state) { @@ -1353,10 +968,7 @@ static void dpu_crtc_disable(struct drm_crtc *crtc) trace_dpu_crtc_disable(DRMID(crtc), false, dpu_crtc); if (dpu_crtc->enabled && !dpu_crtc->suspend && dpu_crtc->vblank_requested) { - ret = _dpu_crtc_vblank_enable_no_lock(dpu_crtc, false); - if (ret) - DPU_ERROR("%s vblank enable failed: %d\n", - dpu_crtc->name, ret); + _dpu_crtc_vblank_enable_no_lock(dpu_crtc, false); } dpu_crtc->enabled = false; @@ -1379,9 +991,8 @@ static void dpu_crtc_disable(struct drm_crtc *crtc) dpu_power_handle_unregister_event(dpu_crtc->phandle, dpu_crtc->power_event); - memset(dpu_crtc->mixers, 0, sizeof(dpu_crtc->mixers)); - dpu_crtc->num_mixers = 0; - dpu_crtc->mixers_swapped = false; + memset(cstate->mixers, 0, sizeof(cstate->mixers)); + cstate->num_mixers = 0; /* disable clk & bw control until clk & bw properties are set */ cstate->bw_control = false; @@ -1403,7 +1014,6 @@ static void dpu_crtc_enable(struct drm_crtc *crtc, struct dpu_crtc *dpu_crtc; struct drm_encoder *encoder; struct msm_drm_private *priv; - int ret; if (!crtc || !crtc->dev || !crtc->dev->dev_private) { DPU_ERROR("invalid crtc\n"); @@ -1425,10 +1035,7 @@ static void dpu_crtc_enable(struct drm_crtc *crtc, trace_dpu_crtc_enable(DRMID(crtc), true, dpu_crtc); if (!dpu_crtc->enabled && !dpu_crtc->suspend && dpu_crtc->vblank_requested) { - ret = _dpu_crtc_vblank_enable_no_lock(dpu_crtc, true); - if (ret) - DPU_ERROR("%s vblank enable failed: %d\n", - dpu_crtc->name, ret); + _dpu_crtc_vblank_enable_no_lock(dpu_crtc, true); } dpu_crtc->enabled = true; @@ -1438,9 +1045,7 @@ static void dpu_crtc_enable(struct drm_crtc *crtc, drm_crtc_vblank_on(crtc); dpu_crtc->power_event = dpu_power_handle_register_event( - dpu_crtc->phandle, - DPU_POWER_EVENT_POST_ENABLE | DPU_POWER_EVENT_POST_DISABLE | - DPU_POWER_EVENT_PRE_DISABLE, + dpu_crtc->phandle, DPU_POWER_EVENT_ENABLE, dpu_crtc_handle_power_event, crtc, dpu_crtc->name); } @@ -1496,7 +1101,7 @@ static int dpu_crtc_atomic_check(struct drm_crtc *crtc, memset(pipe_staged, 0, sizeof(pipe_staged)); - mixer_width = dpu_crtc_get_mixer_width(dpu_crtc, cstate, mode); + mixer_width = _dpu_crtc_get_mixer_width(cstate, mode); _dpu_crtc_setup_lm_bounds(crtc, state); @@ -1535,8 +1140,7 @@ static int dpu_crtc_atomic_check(struct drm_crtc *crtc, cnt++; dst = drm_plane_state_dest(pstate); - if (!drm_rect_intersect(&clip, &dst) || - !drm_rect_equals(&clip, &dst)) { + if (!drm_rect_intersect(&clip, &dst)) { DPU_ERROR("invalid vertical/horizontal destination\n"); DPU_ERROR("display: " DRM_RECT_FMT " plane: " DRM_RECT_FMT "\n", DRM_RECT_ARG(&crtc_rect), @@ -1679,7 +1283,6 @@ static int dpu_crtc_atomic_check(struct drm_crtc *crtc, } end: - _dpu_crtc_rp_free_unused(&cstate->rp); kfree(pstates); return rc; } @@ -1687,7 +1290,6 @@ end: int dpu_crtc_vblank(struct drm_crtc *crtc, bool en) { struct dpu_crtc *dpu_crtc; - int ret; if (!crtc) { DPU_ERROR("invalid crtc\n"); @@ -1698,10 +1300,7 @@ int dpu_crtc_vblank(struct drm_crtc *crtc, bool en) mutex_lock(&dpu_crtc->crtc_lock); trace_dpu_crtc_vblank(DRMID(&dpu_crtc->base), en, dpu_crtc); if (dpu_crtc->enabled && !dpu_crtc->suspend) { - ret = _dpu_crtc_vblank_enable_no_lock(dpu_crtc, en); - if (ret) - DPU_ERROR("%s vblank enable failed: %d\n", - dpu_crtc->name, ret); + _dpu_crtc_vblank_enable_no_lock(dpu_crtc, en); } dpu_crtc->vblank_requested = en; mutex_unlock(&dpu_crtc->crtc_lock); @@ -1730,26 +1329,28 @@ static int _dpu_debugfs_status_show(struct seq_file *s, void *data) dpu_crtc = s->private; crtc = &dpu_crtc->base; + + drm_modeset_lock_all(crtc->dev); cstate = to_dpu_crtc_state(crtc->state); mutex_lock(&dpu_crtc->crtc_lock); mode = &crtc->state->adjusted_mode; - out_width = dpu_crtc_get_mixer_width(dpu_crtc, cstate, mode); + out_width = _dpu_crtc_get_mixer_width(cstate, mode); seq_printf(s, "crtc:%d width:%d height:%d\n", crtc->base.id, mode->hdisplay, mode->vdisplay); seq_puts(s, "\n"); - for (i = 0; i < dpu_crtc->num_mixers; ++i) { - m = &dpu_crtc->mixers[i]; + for (i = 0; i < cstate->num_mixers; ++i) { + m = &cstate->mixers[i]; if (!m->hw_lm) seq_printf(s, "\tmixer[%d] has no lm\n", i); - else if (!m->hw_ctl) + else if (!m->lm_ctl) seq_printf(s, "\tmixer[%d] has no ctl\n", i); else seq_printf(s, "\tmixer:%d ctl:%d width:%d height:%d\n", - m->hw_lm->idx - LM_0, m->hw_ctl->idx - CTL_0, + m->hw_lm->idx - LM_0, m->lm_ctl->idx - CTL_0, out_width, mode->vdisplay); } @@ -1822,6 +1423,7 @@ static int _dpu_debugfs_status_show(struct seq_file *s, void *data) seq_printf(s, "vblank_enable:%d\n", dpu_crtc->vblank_requested); mutex_unlock(&dpu_crtc->crtc_lock); + drm_modeset_unlock_all(crtc->dev); return 0; } @@ -1831,113 +1433,6 @@ static int _dpu_debugfs_status_open(struct inode *inode, struct file *file) return single_open(file, _dpu_debugfs_status_show, inode->i_private); } -static ssize_t _dpu_crtc_misr_setup(struct file *file, - const char __user *user_buf, size_t count, loff_t *ppos) -{ - struct dpu_crtc *dpu_crtc; - struct dpu_crtc_mixer *m; - int i = 0, rc; - char buf[MISR_BUFF_SIZE + 1]; - u32 frame_count, enable; - size_t buff_copy; - - if (!file || !file->private_data) - return -EINVAL; - - dpu_crtc = file->private_data; - buff_copy = min_t(size_t, count, MISR_BUFF_SIZE); - if (copy_from_user(buf, user_buf, buff_copy)) { - DPU_ERROR("buffer copy failed\n"); - return -EINVAL; - } - - buf[buff_copy] = 0; /* end of string */ - - if (sscanf(buf, "%u %u", &enable, &frame_count) != 2) - return -EINVAL; - - rc = _dpu_crtc_power_enable(dpu_crtc, true); - if (rc) - return rc; - - mutex_lock(&dpu_crtc->crtc_lock); - dpu_crtc->misr_enable = enable; - dpu_crtc->misr_frame_count = frame_count; - for (i = 0; i < dpu_crtc->num_mixers; ++i) { - dpu_crtc->misr_data[i] = 0; - m = &dpu_crtc->mixers[i]; - if (!m->hw_lm || !m->hw_lm->ops.setup_misr) - continue; - - m->hw_lm->ops.setup_misr(m->hw_lm, enable, frame_count); - } - mutex_unlock(&dpu_crtc->crtc_lock); - _dpu_crtc_power_enable(dpu_crtc, false); - - return count; -} - -static ssize_t _dpu_crtc_misr_read(struct file *file, - char __user *user_buff, size_t count, loff_t *ppos) -{ - struct dpu_crtc *dpu_crtc; - struct dpu_crtc_mixer *m; - int i = 0, rc; - u32 misr_status; - ssize_t len = 0; - char buf[MISR_BUFF_SIZE + 1] = {'\0'}; - - if (*ppos) - return 0; - - if (!file || !file->private_data) - return -EINVAL; - - dpu_crtc = file->private_data; - rc = _dpu_crtc_power_enable(dpu_crtc, true); - if (rc) - return rc; - - mutex_lock(&dpu_crtc->crtc_lock); - if (!dpu_crtc->misr_enable) { - len += snprintf(buf + len, MISR_BUFF_SIZE - len, - "disabled\n"); - goto buff_check; - } - - for (i = 0; i < dpu_crtc->num_mixers; ++i) { - m = &dpu_crtc->mixers[i]; - if (!m->hw_lm || !m->hw_lm->ops.collect_misr) - continue; - - misr_status = m->hw_lm->ops.collect_misr(m->hw_lm); - dpu_crtc->misr_data[i] = misr_status ? misr_status : - dpu_crtc->misr_data[i]; - len += snprintf(buf + len, MISR_BUFF_SIZE - len, "lm idx:%d\n", - m->hw_lm->idx - LM_0); - len += snprintf(buf + len, MISR_BUFF_SIZE - len, "0x%x\n", - dpu_crtc->misr_data[i]); - } - -buff_check: - if (count <= len) { - len = 0; - goto end; - } - - if (copy_to_user(user_buff, buf, len)) { - len = -EFAULT; - goto end; - } - - *ppos += len; /* increase offset */ - -end: - mutex_unlock(&dpu_crtc->crtc_lock); - _dpu_crtc_power_enable(dpu_crtc, false); - return len; -} - #define DEFINE_DPU_DEBUGFS_SEQ_FOPS(__prefix) \ static int __prefix ## _open(struct inode *inode, struct file *file) \ { \ @@ -1955,8 +1450,6 @@ static int dpu_crtc_debugfs_state_show(struct seq_file *s, void *v) { struct drm_crtc *crtc = (struct drm_crtc *) s->private; struct dpu_crtc *dpu_crtc = to_dpu_crtc(crtc); - struct dpu_crtc_res *res; - struct dpu_crtc_respool *rp; int i; seq_printf(s, "client type: %d\n", dpu_crtc_get_client_type(crtc)); @@ -1973,17 +1466,6 @@ static int dpu_crtc_debugfs_state_show(struct seq_file *s, void *v) dpu_crtc->cur_perf.max_per_pipe_ib[i]); } - mutex_lock(&dpu_crtc->rp_lock); - list_for_each_entry(rp, &dpu_crtc->rp_head, rp_list) { - seq_printf(s, "rp.%d: ", rp->sequence_id); - list_for_each_entry(res, &rp->res_list, list) - seq_printf(s, "0x%x/0x%llx/%pK/%d ", - res->type, res->tag, res->val, - atomic_read(&res->refcount)); - seq_puts(s, "\n"); - } - mutex_unlock(&dpu_crtc->rp_lock); - return 0; } DEFINE_DPU_DEBUGFS_SEQ_FOPS(dpu_crtc_debugfs_state); @@ -1999,19 +1481,12 @@ static int _dpu_crtc_init_debugfs(struct drm_crtc *crtc) .llseek = seq_lseek, .release = single_release, }; - static const struct file_operations debugfs_misr_fops = { - .open = simple_open, - .read = _dpu_crtc_misr_read, - .write = _dpu_crtc_misr_setup, - }; if (!crtc) return -EINVAL; dpu_crtc = to_dpu_crtc(crtc); dpu_kms = _dpu_crtc_get_kms(crtc); - if (!dpu_kms) - return -EINVAL; dpu_crtc->debugfs_root = debugfs_create_dir(dpu_crtc->name, crtc->dev->primary->debugfs_root); @@ -2026,8 +1501,6 @@ static int _dpu_crtc_init_debugfs(struct drm_crtc *crtc) dpu_crtc->debugfs_root, &dpu_crtc->base, &dpu_crtc_debugfs_state_fops); - debugfs_create_file("misr_data", 0600, dpu_crtc->debugfs_root, - dpu_crtc, &debugfs_misr_fops); return 0; } @@ -2082,7 +1555,8 @@ static const struct drm_crtc_helper_funcs dpu_crtc_helper_funcs = { }; /* initialize crtc */ -struct drm_crtc *dpu_crtc_init(struct drm_device *dev, struct drm_plane *plane) +struct drm_crtc *dpu_crtc_init(struct drm_device *dev, struct drm_plane *plane, + struct drm_plane *cursor) { struct drm_crtc *crtc = NULL; struct dpu_crtc *dpu_crtc = NULL; @@ -2104,9 +1578,6 @@ struct drm_crtc *dpu_crtc_init(struct drm_device *dev, struct drm_plane *plane) spin_lock_init(&dpu_crtc->spin_lock); atomic_set(&dpu_crtc->frame_pending, 0); - mutex_init(&dpu_crtc->rp_lock); - INIT_LIST_HEAD(&dpu_crtc->rp_head); - init_completion(&dpu_crtc->frame_done_comp); INIT_LIST_HEAD(&dpu_crtc->frame_event_list); @@ -2119,7 +1590,7 @@ struct drm_crtc *dpu_crtc_init(struct drm_device *dev, struct drm_plane *plane) dpu_crtc_frame_event_work); } - drm_crtc_init_with_planes(dev, crtc, plane, NULL, &dpu_crtc_funcs, + drm_crtc_init_with_planes(dev, crtc, plane, cursor, &dpu_crtc_funcs, NULL); drm_crtc_helper_add(crtc, &dpu_crtc_helper_funcs); diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h index e87109e608e9..3723b4830335 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.h @@ -83,14 +83,14 @@ struct dpu_crtc_smmu_state_data { /** * struct dpu_crtc_mixer: stores the map for each virtual pipeline in the CRTC * @hw_lm: LM HW Driver context - * @hw_ctl: CTL Path HW driver context + * @lm_ctl: CTL Path HW driver context * @encoder: Encoder attached to this lm & ctl * @mixer_op_mode: mixer blending operation mode * @flush_mask: mixer flush mask for ctl, mixer and pipe */ struct dpu_crtc_mixer { struct dpu_hw_mixer *hw_lm; - struct dpu_hw_ctl *hw_ctl; + struct dpu_hw_ctl *lm_ctl; struct drm_encoder *encoder; u32 mixer_op_mode; u32 flush_mask; @@ -121,11 +121,6 @@ struct dpu_crtc_frame_event { * struct dpu_crtc - virtualized CRTC data structure * @base : Base drm crtc structure * @name : ASCII description of this crtc - * @num_ctls : Number of ctl paths in use - * @num_mixers : Number of mixers in use - * @mixers_swapped: Whether the mixers have been swapped for left/right update - * especially in the case of DSC Merge. - * @mixers : List of active mixers * @event : Pointer to last received drm vblank event. If there is a * pending vblank event, this will be non-null. * @vsync_count : Running count of received vsync events @@ -156,27 +151,14 @@ struct dpu_crtc_frame_event { * @event_thread : Pointer to event handler thread * @event_worker : Event worker queue * @event_lock : Spinlock around event handling code - * @misr_enable : boolean entry indicates misr enable/disable status. - * @misr_frame_count : misr frame count provided by client - * @misr_data : store misr data before turning off the clocks. * @phandle: Pointer to power handler * @power_event : registered power event handle * @cur_perf : current performance committed to clock/bandwidth driver - * @rp_lock : serialization lock for resource pool - * @rp_head : list of active resource pool - * @scl3_cfg_lut : qseed3 lut config */ struct dpu_crtc { struct drm_crtc base; char name[DPU_CRTC_NAME_SIZE]; - /* HW Resources reserved for the crtc */ - u32 num_ctls; - u32 num_mixers; - bool mixers_swapped; - struct dpu_crtc_mixer mixers[CRTC_DUAL_MIXERS]; - struct dpu_hw_scaler3_lut_cfg *scl3_lut_cfg; - struct drm_pending_vblank_event *event; u32 vsync_count; @@ -206,77 +188,20 @@ struct dpu_crtc { /* for handling internal event thread */ spinlock_t event_lock; - bool misr_enable; - u32 misr_frame_count; - u32 misr_data[CRTC_DUAL_MIXERS]; struct dpu_power_handle *phandle; struct dpu_power_event *power_event; struct dpu_core_perf_params cur_perf; - struct mutex rp_lock; - struct list_head rp_head; - struct dpu_crtc_smmu_state_data smmu_state; }; #define to_dpu_crtc(x) container_of(x, struct dpu_crtc, base) /** - * struct dpu_crtc_res_ops - common operations for crtc resources - * @get: get given resource - * @put: put given resource - */ -struct dpu_crtc_res_ops { - void *(*get)(void *val, u32 type, u64 tag); - void (*put)(void *val); -}; - -#define DPU_CRTC_RES_FLAG_FREE BIT(0) - -/** - * struct dpu_crtc_res - definition of crtc resources - * @list: list of crtc resource - * @type: crtc resource type - * @tag: unique identifier per type - * @refcount: reference/usage count - * @ops: callback operations - * @val: resource handle associated with type/tag - * @flags: customization flags - */ -struct dpu_crtc_res { - struct list_head list; - u32 type; - u64 tag; - atomic_t refcount; - struct dpu_crtc_res_ops ops; - void *val; - u32 flags; -}; - -/** - * dpu_crtc_respool - crtc resource pool - * @rp_lock: pointer to serialization lock - * @rp_head: pointer to head of active resource pools of this crtc - * @rp_list: list of crtc resource pool - * @sequence_id: sequence identifier, incremented per state duplication - * @res_list: list of resource managed by this resource pool - * @ops: resource operations for parent resource pool - */ -struct dpu_crtc_respool { - struct mutex *rp_lock; - struct list_head *rp_head; - struct list_head rp_list; - u32 sequence_id; - struct list_head res_list; - struct dpu_crtc_res_ops ops; -}; - -/** * struct dpu_crtc_state - dpu container for atomic crtc state * @base: Base drm crtc state structure - * @is_ppsplit : Whether current topology requires PPSplit special handling * @bw_control : true if bw/clk controlled by core bw/clk properties * @bw_split_vote : true if bw controlled by llcc/dram bw properties * @lm_bounds : LM boundaries based on current mode full resolution, no ROI. @@ -285,41 +210,41 @@ struct dpu_crtc_respool { * @property_values: Current crtc property values * @input_fence_timeout_ns : Cached input fence timeout, in ns * @new_perf: new performance state being requested + * @num_mixers : Number of mixers in use + * @mixers : List of active mixers + * @num_ctls : Number of ctl paths in use + * @hw_ctls : List of active ctl paths */ struct dpu_crtc_state { struct drm_crtc_state base; bool bw_control; bool bw_split_vote; - - bool is_ppsplit; struct drm_rect lm_bounds[CRTC_DUAL_MIXERS]; uint64_t input_fence_timeout_ns; struct dpu_core_perf_params new_perf; - struct dpu_crtc_respool rp; + + /* HW Resources reserved for the crtc */ + u32 num_mixers; + struct dpu_crtc_mixer mixers[CRTC_DUAL_MIXERS]; + + u32 num_ctls; + struct dpu_hw_ctl *hw_ctls[CRTC_DUAL_MIXERS]; }; #define to_dpu_crtc_state(x) \ container_of(x, struct dpu_crtc_state, base) /** - * dpu_crtc_get_mixer_width - get the mixer width - * Mixer width will be same as panel width(/2 for split) + * dpu_crtc_state_is_stereo - Is crtc virtualized with two mixers? + * @cstate: Pointer to dpu crtc state + * @Return: true - has two mixers, false - has one mixer */ -static inline int dpu_crtc_get_mixer_width(struct dpu_crtc *dpu_crtc, - struct dpu_crtc_state *cstate, struct drm_display_mode *mode) +static inline bool dpu_crtc_state_is_stereo(struct dpu_crtc_state *cstate) { - u32 mixer_width; - - if (!dpu_crtc || !cstate || !mode) - return 0; - - mixer_width = (dpu_crtc->num_mixers == CRTC_DUAL_MIXERS ? - mode->hdisplay / CRTC_DUAL_MIXERS : mode->hdisplay); - - return mixer_width; + return cstate->num_mixers == CRTC_DUAL_MIXERS; } /** @@ -375,9 +300,11 @@ void dpu_crtc_complete_commit(struct drm_crtc *crtc, * dpu_crtc_init - create a new crtc object * @dev: dpu device * @plane: base plane + * @cursor: cursor plane * @Return: new crtc object or error */ -struct drm_crtc *dpu_crtc_init(struct drm_device *dev, struct drm_plane *plane); +struct drm_crtc *dpu_crtc_init(struct drm_device *dev, struct drm_plane *plane, + struct drm_plane *cursor); /** * dpu_crtc_register_custom_event - api for enabling/disabling crtc event diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c index 1b4de3486ef9..96cdf06e7da2 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c @@ -65,8 +65,6 @@ #define MAX_CHANNELS_PER_ENC 2 -#define MISR_BUFF_SIZE 256 - #define IDLE_SHORT_TIMEOUT 1 #define MAX_VDISPLAY_SPLIT 1080 @@ -161,8 +159,6 @@ enum dpu_enc_rc_states { * @frame_done_timer: watchdog timer for frame done event * @vsync_event_timer: vsync timer * @disp_info: local copy of msm_display_info struct - * @misr_enable: misr enable/disable status - * @misr_frame_count: misr frame count before start capturing the data * @idle_pc_supported: indicate if idle power collaps is supported * @rc_lock: resource control mutex lock to protect * virt encoder over various state changes @@ -179,11 +175,10 @@ struct dpu_encoder_virt { spinlock_t enc_spinlock; uint32_t bus_scaling_client; - uint32_t display_num_of_h_tiles; - unsigned int num_phys_encs; struct dpu_encoder_phys *phys_encs[MAX_PHYS_ENCODERS_PER_VIRTUAL]; struct dpu_encoder_phys *cur_master; + struct dpu_encoder_phys *cur_slave; struct dpu_hw_pingpong *hw_pp[MAX_CHANNELS_PER_ENC]; bool intfs_swapped; @@ -202,8 +197,6 @@ struct dpu_encoder_virt { struct timer_list vsync_event_timer; struct msm_display_info disp_info; - bool misr_enable; - u32 misr_frame_count; bool idle_pc_supported; struct mutex rc_lock; @@ -443,30 +436,22 @@ int dpu_encoder_helper_unregister_irq(struct dpu_encoder_phys *phys_enc, } void dpu_encoder_get_hw_resources(struct drm_encoder *drm_enc, - struct dpu_encoder_hw_resources *hw_res, - struct drm_connector_state *conn_state) + struct dpu_encoder_hw_resources *hw_res) { struct dpu_encoder_virt *dpu_enc = NULL; int i = 0; - if (!hw_res || !drm_enc || !conn_state) { - DPU_ERROR("invalid argument(s), drm_enc %d, res %d, state %d\n", - drm_enc != 0, hw_res != 0, conn_state != 0); - return; - } - dpu_enc = to_dpu_encoder_virt(drm_enc); DPU_DEBUG_ENC(dpu_enc, "\n"); /* Query resources used by phys encs, expected to be without overlap */ memset(hw_res, 0, sizeof(*hw_res)); - hw_res->display_num_of_h_tiles = dpu_enc->display_num_of_h_tiles; for (i = 0; i < dpu_enc->num_phys_encs; i++) { struct dpu_encoder_phys *phys = dpu_enc->phys_encs[i]; if (phys && phys->ops.get_hw_resources) - phys->ops.get_hw_resources(phys, hw_res, conn_state); + phys->ops.get_hw_resources(phys, hw_res); } } @@ -525,7 +510,7 @@ void dpu_encoder_helper_split_config( hw_mdptop = phys_enc->hw_mdptop; disp_info = &dpu_enc->disp_info; - if (disp_info->intf_type != DRM_MODE_CONNECTOR_DSI) + if (disp_info->intf_type != DRM_MODE_ENCODER_DSI) return; /** @@ -660,7 +645,7 @@ static int dpu_encoder_virt_atomic_check( if (drm_atomic_crtc_needs_modeset(crtc_state) && dpu_enc->mode_set_complete) { ret = dpu_rm_reserve(&dpu_kms->rm, drm_enc, crtc_state, - conn_state, topology, true); + topology, true); dpu_enc->mode_set_complete = false; } } @@ -1016,9 +1001,9 @@ static void dpu_encoder_virt_mode_set(struct drm_encoder *drm_enc, struct dpu_kms *dpu_kms; struct list_head *connector_list; struct drm_connector *conn = NULL, *conn_iter; - struct dpu_rm_hw_iter pp_iter; + struct dpu_rm_hw_iter pp_iter, ctl_iter; struct msm_display_topology topology; - enum dpu_rm_topology_name topology_name; + struct dpu_hw_ctl *hw_ctl[MAX_CHANNELS_PER_ENC] = { NULL }; int i = 0, ret; if (!drm_enc) { @@ -1051,7 +1036,7 @@ static void dpu_encoder_virt_mode_set(struct drm_encoder *drm_enc, /* Reserve dynamic resources now. Indicating non-AtomicTest phase */ ret = dpu_rm_reserve(&dpu_kms->rm, drm_enc, drm_enc->crtc->state, - conn->state, topology, false); + topology, false); if (ret) { DPU_ERROR_ENC(dpu_enc, "failed to reserve hw resources, %d\n", ret); @@ -1066,19 +1051,33 @@ static void dpu_encoder_virt_mode_set(struct drm_encoder *drm_enc, dpu_enc->hw_pp[i] = (struct dpu_hw_pingpong *) pp_iter.hw; } - topology_name = dpu_rm_get_topology_name(topology); + dpu_rm_init_hw_iter(&ctl_iter, drm_enc->base.id, DPU_HW_BLK_CTL); + for (i = 0; i < MAX_CHANNELS_PER_ENC; i++) { + if (!dpu_rm_get_hw(&dpu_kms->rm, &ctl_iter)) + break; + hw_ctl[i] = (struct dpu_hw_ctl *)ctl_iter.hw; + } + for (i = 0; i < dpu_enc->num_phys_encs; i++) { struct dpu_encoder_phys *phys = dpu_enc->phys_encs[i]; if (phys) { if (!dpu_enc->hw_pp[i]) { - DPU_ERROR_ENC(dpu_enc, - "invalid pingpong block for the encoder\n"); + DPU_ERROR_ENC(dpu_enc, "no pp block assigned" + "at idx: %d\n", i); return; } + + if (!hw_ctl[i]) { + DPU_ERROR_ENC(dpu_enc, "no ctl block assigned" + "at idx: %d\n", i); + return; + } + phys->hw_pp = dpu_enc->hw_pp[i]; + phys->hw_ctl = hw_ctl[i]; + phys->connector = conn->state->connector; - phys->topology_name = topology_name; if (phys->ops.mode_set) phys->ops.mode_set(phys, mode, adj_mode); } @@ -1111,12 +1110,6 @@ static void _dpu_encoder_virt_enable_helper(struct drm_encoder *drm_enc) return; } - if (dpu_enc->disp_info.intf_type == DRM_MODE_CONNECTOR_DisplayPort && - dpu_enc->cur_master->hw_mdptop && - dpu_enc->cur_master->hw_mdptop->ops.intf_audio_select) - dpu_enc->cur_master->hw_mdptop->ops.intf_audio_select( - dpu_enc->cur_master->hw_mdptop); - if (dpu_enc->cur_master->hw_mdptop && dpu_enc->cur_master->hw_mdptop->ops.reset_ubwc) dpu_enc->cur_master->hw_mdptop->ops.reset_ubwc( @@ -1153,7 +1146,7 @@ void dpu_encoder_virt_restore(struct drm_encoder *drm_enc) static void dpu_encoder_virt_enable(struct drm_encoder *drm_enc) { struct dpu_encoder_virt *dpu_enc = NULL; - int i, ret = 0; + int ret = 0; struct drm_display_mode *cur_mode = NULL; if (!drm_enc) { @@ -1166,21 +1159,12 @@ static void dpu_encoder_virt_enable(struct drm_encoder *drm_enc) trace_dpu_enc_enable(DRMID(drm_enc), cur_mode->hdisplay, cur_mode->vdisplay); - dpu_enc->cur_master = NULL; - for (i = 0; i < dpu_enc->num_phys_encs; i++) { - struct dpu_encoder_phys *phys = dpu_enc->phys_encs[i]; + /* always enable slave encoder before master */ + if (dpu_enc->cur_slave && dpu_enc->cur_slave->ops.enable) + dpu_enc->cur_slave->ops.enable(dpu_enc->cur_slave); - if (phys && phys->ops.is_master && phys->ops.is_master(phys)) { - DPU_DEBUG_ENC(dpu_enc, "master is now idx %d\n", i); - dpu_enc->cur_master = phys; - break; - } - } - - if (!dpu_enc->cur_master) { - DPU_ERROR("virt encoder has no master! num_phys %d\n", i); - return; - } + if (dpu_enc->cur_master && dpu_enc->cur_master->ops.enable) + dpu_enc->cur_master->ops.enable(dpu_enc->cur_master); ret = dpu_encoder_resource_control(drm_enc, DPU_ENC_RC_EVENT_KICKOFF); if (ret) { @@ -1189,26 +1173,6 @@ static void dpu_encoder_virt_enable(struct drm_encoder *drm_enc) return; } - for (i = 0; i < dpu_enc->num_phys_encs; i++) { - struct dpu_encoder_phys *phys = dpu_enc->phys_encs[i]; - - if (!phys) - continue; - - if (phys != dpu_enc->cur_master) { - if (phys->ops.enable) - phys->ops.enable(phys); - } - - if (dpu_enc->misr_enable && (dpu_enc->disp_info.capabilities & - MSM_DISPLAY_CAP_VID_MODE) && phys->ops.setup_misr) - phys->ops.setup_misr(phys, true, - dpu_enc->misr_frame_count); - } - - if (dpu_enc->cur_master->ops.enable) - dpu_enc->cur_master->ops.enable(dpu_enc->cur_master); - _dpu_encoder_virt_enable_helper(drm_enc); } @@ -1266,8 +1230,6 @@ static void dpu_encoder_virt_disable(struct drm_encoder *drm_enc) dpu_enc->phys_encs[i]->connector = NULL; } - dpu_enc->cur_master = NULL; - DPU_DEBUG_ENC(dpu_enc, "encoder disabled\n"); dpu_rm_release(&dpu_kms->rm, drm_enc); @@ -1397,9 +1359,9 @@ static void dpu_encoder_frame_done_callback( /* One of the physical encoders has become idle */ for (i = 0; i < dpu_enc->num_phys_encs; i++) { if (dpu_enc->phys_encs[i] == ready_phys) { - clear_bit(i, dpu_enc->frame_busy_mask); trace_dpu_enc_frame_done_cb(DRMID(drm_enc), i, dpu_enc->frame_busy_mask[0]); + clear_bit(i, dpu_enc->frame_busy_mask); } } @@ -1480,7 +1442,8 @@ static inline void _dpu_encoder_trigger_flush(struct drm_encoder *drm_enc, ret = ctl->ops.get_pending_flush(ctl); trace_dpu_enc_trigger_flush(DRMID(drm_enc), phys->intf_idx, - pending_kickoff_cnt, ctl->idx, ret); + pending_kickoff_cnt, ctl->idx, + extra_flush_bits, ret); } /** @@ -1879,7 +1842,7 @@ void dpu_encoder_kickoff(struct drm_encoder *drm_enc) phys->ops.handle_post_kickoff(phys); } - if (dpu_enc->disp_info.intf_type == DRM_MODE_CONNECTOR_DSI && + if (dpu_enc->disp_info.intf_type == DRM_MODE_ENCODER_DSI && !_dpu_encoder_wakeup_time(drm_enc, &wakeup_time)) { trace_dpu_enc_early_kickoff(DRMID(drm_enc), ktime_to_ms(wakeup_time)); @@ -1955,113 +1918,6 @@ static int _dpu_encoder_debugfs_status_open(struct inode *inode, return single_open(file, _dpu_encoder_status_show, inode->i_private); } -static ssize_t _dpu_encoder_misr_setup(struct file *file, - const char __user *user_buf, size_t count, loff_t *ppos) -{ - struct dpu_encoder_virt *dpu_enc; - int i = 0, rc; - char buf[MISR_BUFF_SIZE + 1]; - size_t buff_copy; - u32 frame_count, enable; - - if (!file || !file->private_data) - return -EINVAL; - - dpu_enc = file->private_data; - - buff_copy = min_t(size_t, count, MISR_BUFF_SIZE); - if (copy_from_user(buf, user_buf, buff_copy)) - return -EINVAL; - - buf[buff_copy] = 0; /* end of string */ - - if (sscanf(buf, "%u %u", &enable, &frame_count) != 2) - return -EINVAL; - - rc = _dpu_encoder_power_enable(dpu_enc, true); - if (rc) - return rc; - - mutex_lock(&dpu_enc->enc_lock); - dpu_enc->misr_enable = enable; - dpu_enc->misr_frame_count = frame_count; - for (i = 0; i < dpu_enc->num_phys_encs; i++) { - struct dpu_encoder_phys *phys = dpu_enc->phys_encs[i]; - - if (!phys || !phys->ops.setup_misr) - continue; - - phys->ops.setup_misr(phys, enable, frame_count); - } - mutex_unlock(&dpu_enc->enc_lock); - _dpu_encoder_power_enable(dpu_enc, false); - - return count; -} - -static ssize_t _dpu_encoder_misr_read(struct file *file, - char __user *user_buff, size_t count, loff_t *ppos) -{ - struct dpu_encoder_virt *dpu_enc; - int i = 0, len = 0; - char buf[MISR_BUFF_SIZE + 1] = {'\0'}; - int rc; - - if (*ppos) - return 0; - - if (!file || !file->private_data) - return -EINVAL; - - dpu_enc = file->private_data; - - rc = _dpu_encoder_power_enable(dpu_enc, true); - if (rc) - return rc; - - mutex_lock(&dpu_enc->enc_lock); - if (!dpu_enc->misr_enable) { - len += snprintf(buf + len, MISR_BUFF_SIZE - len, - "disabled\n"); - goto buff_check; - } else if (dpu_enc->disp_info.capabilities & - ~MSM_DISPLAY_CAP_VID_MODE) { - len += snprintf(buf + len, MISR_BUFF_SIZE - len, - "unsupported\n"); - goto buff_check; - } - - for (i = 0; i < dpu_enc->num_phys_encs; i++) { - struct dpu_encoder_phys *phys = dpu_enc->phys_encs[i]; - - if (!phys || !phys->ops.collect_misr) - continue; - - len += snprintf(buf + len, MISR_BUFF_SIZE - len, - "Intf idx:%d\n", phys->intf_idx - INTF_0); - len += snprintf(buf + len, MISR_BUFF_SIZE - len, "0x%x\n", - phys->ops.collect_misr(phys)); - } - -buff_check: - if (count <= len) { - len = 0; - goto end; - } - - if (copy_to_user(user_buff, buf, len)) { - len = -EFAULT; - goto end; - } - - *ppos += len; /* increase offset */ - -end: - mutex_unlock(&dpu_enc->enc_lock); - _dpu_encoder_power_enable(dpu_enc, false); - return len; -} - static int _dpu_encoder_init_debugfs(struct drm_encoder *drm_enc) { struct dpu_encoder_virt *dpu_enc; @@ -2076,12 +1932,6 @@ static int _dpu_encoder_init_debugfs(struct drm_encoder *drm_enc) .release = single_release, }; - static const struct file_operations debugfs_misr_fops = { - .open = simple_open, - .read = _dpu_encoder_misr_read, - .write = _dpu_encoder_misr_setup, - }; - char name[DPU_NAME_SIZE]; if (!drm_enc || !drm_enc->dev || !drm_enc->dev->dev_private) { @@ -2105,9 +1955,6 @@ static int _dpu_encoder_init_debugfs(struct drm_encoder *drm_enc) debugfs_create_file("status", 0600, dpu_enc->debugfs_root, dpu_enc, &debugfs_status_fops); - debugfs_create_file("misr_data", 0600, - dpu_enc->debugfs_root, dpu_enc, &debugfs_misr_fops); - for (i = 0; i < dpu_enc->num_phys_encs; i++) if (dpu_enc->phys_encs[i] && dpu_enc->phys_encs[i]->ops.late_register) @@ -2195,6 +2042,11 @@ static int dpu_encoder_virt_add_phys_encs( ++dpu_enc->num_phys_encs; } + if (params->split_role == ENC_ROLE_SLAVE) + dpu_enc->cur_slave = enc; + else + dpu_enc->cur_master = enc; + return 0; } @@ -2206,8 +2058,7 @@ static const struct dpu_encoder_virt_ops dpu_encoder_parent_ops = { static int dpu_encoder_setup_display(struct dpu_encoder_virt *dpu_enc, struct dpu_kms *dpu_kms, - struct msm_display_info *disp_info, - int *drm_enc_mode) + struct msm_display_info *disp_info) { int ret = 0; int i = 0; @@ -2220,6 +2071,8 @@ static int dpu_encoder_setup_display(struct dpu_encoder_virt *dpu_enc, return -EINVAL; } + dpu_enc->cur_master = NULL; + memset(&phys_params, 0, sizeof(phys_params)); phys_params.dpu_kms = dpu_kms; phys_params.parent = &dpu_enc->base; @@ -2228,24 +2081,17 @@ static int dpu_encoder_setup_display(struct dpu_encoder_virt *dpu_enc, DPU_DEBUG("\n"); - if (disp_info->intf_type == DRM_MODE_CONNECTOR_DSI) { - *drm_enc_mode = DRM_MODE_ENCODER_DSI; + switch (disp_info->intf_type) { + case DRM_MODE_ENCODER_DSI: intf_type = INTF_DSI; - } else if (disp_info->intf_type == DRM_MODE_CONNECTOR_HDMIA) { - *drm_enc_mode = DRM_MODE_ENCODER_TMDS; - intf_type = INTF_HDMI; - } else if (disp_info->intf_type == DRM_MODE_CONNECTOR_DisplayPort) { - *drm_enc_mode = DRM_MODE_ENCODER_TMDS; - intf_type = INTF_DP; - } else { + break; + default: DPU_ERROR_ENC(dpu_enc, "unsupported display interface type\n"); return -EINVAL; } WARN_ON(disp_info->num_of_h_tiles < 1); - dpu_enc->display_num_of_h_tiles = disp_info->num_of_h_tiles; - DPU_DEBUG("dsi_info->num_of_h_tiles %d\n", disp_info->num_of_h_tiles); if ((disp_info->capabilities & MSM_DISPLAY_CAP_CMD_MODE) || @@ -2358,25 +2204,22 @@ int dpu_encoder_setup(struct drm_device *dev, struct drm_encoder *enc, struct dpu_kms *dpu_kms = to_dpu_kms(priv->kms); struct drm_encoder *drm_enc = NULL; struct dpu_encoder_virt *dpu_enc = NULL; - int drm_enc_mode = DRM_MODE_ENCODER_NONE; int ret = 0; dpu_enc = to_dpu_encoder_virt(enc); mutex_init(&dpu_enc->enc_lock); - ret = dpu_encoder_setup_display(dpu_enc, dpu_kms, disp_info, - &drm_enc_mode); + ret = dpu_encoder_setup_display(dpu_enc, dpu_kms, disp_info); if (ret) goto fail; - dpu_enc->cur_master = NULL; spin_lock_init(&dpu_enc->enc_spinlock); atomic_set(&dpu_enc->frame_done_timeout, 0); timer_setup(&dpu_enc->frame_done_timer, dpu_encoder_frame_done_timeout, 0); - if (disp_info->intf_type == DRM_MODE_CONNECTOR_DSI) + if (disp_info->intf_type == DRM_MODE_ENCODER_DSI) timer_setup(&dpu_enc->vsync_event_timer, dpu_encoder_vsync_event_handler, 0); diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.h index 60f809fc7c13..9dbf38f446d9 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.h @@ -32,15 +32,9 @@ /** * Encoder functions and data types * @intfs: Interfaces this encoder is using, INTF_MODE_NONE if unused - * @needs_cdm: Encoder requests a CDM based on pixel format conversion needs - * @display_num_of_h_tiles: Number of horizontal tiles in case of split - * interface - * @topology: Topology of the display */ struct dpu_encoder_hw_resources { enum dpu_intf_mode intfs[INTF_MAX]; - bool needs_cdm; - u32 display_num_of_h_tiles; }; /** @@ -56,11 +50,9 @@ struct dpu_encoder_kickoff_params { * dpu_encoder_get_hw_resources - Populate table of required hardware resources * @encoder: encoder pointer * @hw_res: resource table to populate with encoder required resources - * @conn_state: report hw reqs based on this proposed connector state */ void dpu_encoder_get_hw_resources(struct drm_encoder *encoder, - struct dpu_encoder_hw_resources *hw_res, - struct drm_connector_state *conn_state); + struct dpu_encoder_hw_resources *hw_res); /** * dpu_encoder_register_vblank_callback - provide callback to encoder that diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys.h index c7df8aad6613..964efcc757a4 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys.h @@ -22,8 +22,8 @@ #include "dpu_hw_pingpong.h" #include "dpu_hw_ctl.h" #include "dpu_hw_top.h" -#include "dpu_hw_cdm.h" #include "dpu_encoder.h" +#include "dpu_crtc.h" #define DPU_ENCODER_NAME_MAX 16 @@ -114,8 +114,6 @@ struct dpu_encoder_virt_ops { * @handle_post_kickoff: Do any work necessary post-kickoff work * @trigger_start: Process start event on physical encoder * @needs_single_flush: Whether encoder slaves need to be flushed - * @setup_misr: Sets up MISR, enable and disables based on sysfs - * @collect_misr: Collects MISR data on frame update * @hw_reset: Issue HW recovery such as CTL reset and clear * DPU_ENC_ERR_NEEDS_HW_RESET state * @irq_control: Handler to enable/disable all the encoder IRQs @@ -143,8 +141,7 @@ struct dpu_encoder_phys_ops { struct drm_connector_state *conn_state); void (*destroy)(struct dpu_encoder_phys *encoder); void (*get_hw_resources)(struct dpu_encoder_phys *encoder, - struct dpu_encoder_hw_resources *hw_res, - struct drm_connector_state *conn_state); + struct dpu_encoder_hw_resources *hw_res); int (*control_vblank_irq)(struct dpu_encoder_phys *enc, bool enable); int (*wait_for_commit_done)(struct dpu_encoder_phys *phys_enc); int (*wait_for_tx_complete)(struct dpu_encoder_phys *phys_enc); @@ -154,10 +151,6 @@ struct dpu_encoder_phys_ops { void (*handle_post_kickoff)(struct dpu_encoder_phys *phys_enc); void (*trigger_start)(struct dpu_encoder_phys *phys_enc); bool (*needs_single_flush)(struct dpu_encoder_phys *phys_enc); - - void (*setup_misr)(struct dpu_encoder_phys *phys_encs, - bool enable, u32 frame_count); - u32 (*collect_misr)(struct dpu_encoder_phys *phys_enc); void (*hw_reset)(struct dpu_encoder_phys *phys_enc); void (*irq_control)(struct dpu_encoder_phys *phys, bool enable); void (*prepare_idle_pc)(struct dpu_encoder_phys *phys_enc); @@ -210,8 +203,6 @@ struct dpu_encoder_irq { * @parent_ops: Callbacks exposed by the parent to the phys_enc * @hw_mdptop: Hardware interface to the top registers * @hw_ctl: Hardware interface to the ctl registers - * @hw_cdm: Hardware interface to the cdm registers - * @cdm_cfg: Chroma-down hardware configuration * @hw_pp: Hardware interface to the ping pong registers * @dpu_kms: Pointer to the dpu_kms top level * @cached_mode: DRM mode cached at mode_set time, acted on in enable @@ -219,7 +210,6 @@ struct dpu_encoder_irq { * @split_role: Role to play in a split-panel configuration * @intf_mode: Interface mode * @intf_idx: Interface index on dpu hardware - * @topology_name: topology selected for the display * @enc_spinlock: Virtual-Encoder-Wide Spin Lock for IRQ purposes * @enable_state: Enable state tracking * @vblank_refcount: Reference count of vblank request @@ -241,15 +231,12 @@ struct dpu_encoder_phys { const struct dpu_encoder_virt_ops *parent_ops; struct dpu_hw_mdp *hw_mdptop; struct dpu_hw_ctl *hw_ctl; - struct dpu_hw_cdm *hw_cdm; - struct dpu_hw_cdm_cfg cdm_cfg; struct dpu_hw_pingpong *hw_pp; struct dpu_kms *dpu_kms; struct drm_display_mode cached_mode; enum dpu_enc_split_role split_role; enum dpu_intf_mode intf_mode; enum dpu_intf intf_idx; - enum dpu_rm_topology_name topology_name; spinlock_t *enc_spinlock; enum dpu_enc_enable_state enable_state; atomic_t vblank_refcount; @@ -367,11 +354,15 @@ void dpu_encoder_helper_hw_reset(struct dpu_encoder_phys *phys_enc); static inline enum dpu_3d_blend_mode dpu_encoder_helper_get_3d_blend_mode( struct dpu_encoder_phys *phys_enc) { + struct dpu_crtc_state *dpu_cstate; + if (!phys_enc || phys_enc->enable_state == DPU_ENC_DISABLING) return BLEND_3D_NONE; + dpu_cstate = to_dpu_crtc_state(phys_enc->parent->crtc->state); + if (phys_enc->split_role == ENC_ROLE_SOLO && - phys_enc->topology_name == DPU_RM_TOPOLOGY_DUALPIPE_3DMERGE) + dpu_crtc_state_is_stereo(dpu_cstate)) return BLEND_3D_H_ROW_INT; return BLEND_3D_NONE; diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_cmd.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_cmd.c index 3084675ed425..b2d7f0ded24c 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_cmd.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_cmd.c @@ -196,9 +196,6 @@ static void dpu_encoder_phys_cmd_mode_set( { struct dpu_encoder_phys_cmd *cmd_enc = to_dpu_encoder_phys_cmd(phys_enc); - struct dpu_rm *rm = &phys_enc->dpu_kms->rm; - struct dpu_rm_hw_iter iter; - int i, instance; if (!phys_enc || !mode || !adj_mode) { DPU_ERROR("invalid args\n"); @@ -208,22 +205,6 @@ static void dpu_encoder_phys_cmd_mode_set( DPU_DEBUG_CMDENC(cmd_enc, "caching mode:\n"); drm_mode_debug_printmodeline(adj_mode); - instance = phys_enc->split_role == ENC_ROLE_SLAVE ? 1 : 0; - - /* Retrieve previously allocated HW Resources. Shouldn't fail */ - dpu_rm_init_hw_iter(&iter, phys_enc->parent->base.id, DPU_HW_BLK_CTL); - for (i = 0; i <= instance; i++) { - if (dpu_rm_get_hw(rm, &iter)) - phys_enc->hw_ctl = (struct dpu_hw_ctl *)iter.hw; - } - - if (IS_ERR_OR_NULL(phys_enc->hw_ctl)) { - DPU_ERROR_CMDENC(cmd_enc, "failed to init ctl: %ld\n", - PTR_ERR(phys_enc->hw_ctl)); - phys_enc->hw_ctl = NULL; - return; - } - _dpu_encoder_phys_cmd_setup_irq_hw_idx(phys_enc); } @@ -618,23 +599,8 @@ static void dpu_encoder_phys_cmd_destroy(struct dpu_encoder_phys *phys_enc) static void dpu_encoder_phys_cmd_get_hw_resources( struct dpu_encoder_phys *phys_enc, - struct dpu_encoder_hw_resources *hw_res, - struct drm_connector_state *conn_state) + struct dpu_encoder_hw_resources *hw_res) { - struct dpu_encoder_phys_cmd *cmd_enc = - to_dpu_encoder_phys_cmd(phys_enc); - - if (!phys_enc) { - DPU_ERROR("invalid encoder\n"); - return; - } - - if ((phys_enc->intf_idx - INTF_0) >= INTF_MAX) { - DPU_ERROR("invalid intf idx:%d\n", phys_enc->intf_idx); - return; - } - - DPU_DEBUG_CMDENC(cmd_enc, "\n"); hw_res->intfs[phys_enc->intf_idx - INTF_0] = INTF_MODE_CMD; } @@ -823,7 +789,6 @@ struct dpu_encoder_phys *dpu_encoder_phys_cmd_init( { struct dpu_encoder_phys *phys_enc = NULL; struct dpu_encoder_phys_cmd *cmd_enc = NULL; - struct dpu_hw_mdp *hw_mdp; struct dpu_encoder_irq *irq; int i, ret = 0; @@ -836,14 +801,7 @@ struct dpu_encoder_phys *dpu_encoder_phys_cmd_init( goto fail; } phys_enc = &cmd_enc->base; - - hw_mdp = dpu_rm_get_mdp(&p->dpu_kms->rm); - if (IS_ERR_OR_NULL(hw_mdp)) { - ret = PTR_ERR(hw_mdp); - DPU_ERROR("failed to get mdptop\n"); - goto fail_mdp_init; - } - phys_enc->hw_mdptop = hw_mdp; + phys_enc->hw_mdptop = p->dpu_kms->hw_mdp; phys_enc->intf_idx = p->intf_idx; dpu_encoder_phys_cmd_init_ops(&phys_enc->ops); @@ -898,8 +856,6 @@ struct dpu_encoder_phys *dpu_encoder_phys_cmd_init( return phys_enc; -fail_mdp_init: - kfree(cmd_enc); fail: return ERR_PTR(ret); } diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c index 14fc7c2a6bb7..84de385a9f62 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c @@ -355,13 +355,14 @@ static void dpu_encoder_phys_vid_underrun_irq(void *arg, int irq_idx) static bool _dpu_encoder_phys_is_dual_ctl(struct dpu_encoder_phys *phys_enc) { + struct dpu_crtc_state *dpu_cstate; + if (!phys_enc) return false; - if (phys_enc->topology_name == DPU_RM_TOPOLOGY_DUALPIPE) - return true; + dpu_cstate = to_dpu_crtc_state(phys_enc->parent->crtc->state); - return false; + return dpu_cstate->num_ctls > 1; } static bool dpu_encoder_phys_vid_needs_single_flush( @@ -395,9 +396,6 @@ static void dpu_encoder_phys_vid_mode_set( struct drm_display_mode *mode, struct drm_display_mode *adj_mode) { - struct dpu_rm *rm; - struct dpu_rm_hw_iter iter; - int i, instance; struct dpu_encoder_phys_vid *vid_enc; if (!phys_enc || !phys_enc->dpu_kms) { @@ -405,7 +403,6 @@ static void dpu_encoder_phys_vid_mode_set( return; } - rm = &phys_enc->dpu_kms->rm; vid_enc = to_dpu_encoder_phys_vid(phys_enc); if (adj_mode) { @@ -414,21 +411,6 @@ static void dpu_encoder_phys_vid_mode_set( DPU_DEBUG_VIDENC(vid_enc, "caching mode:\n"); } - instance = phys_enc->split_role == ENC_ROLE_SLAVE ? 1 : 0; - - /* Retrieve previously allocated HW Resources. Shouldn't fail */ - dpu_rm_init_hw_iter(&iter, phys_enc->parent->base.id, DPU_HW_BLK_CTL); - for (i = 0; i <= instance; i++) { - if (dpu_rm_get_hw(rm, &iter)) - phys_enc->hw_ctl = (struct dpu_hw_ctl *)iter.hw; - } - if (IS_ERR_OR_NULL(phys_enc->hw_ctl)) { - DPU_ERROR_VIDENC(vid_enc, "failed to init ctl, %ld\n", - PTR_ERR(phys_enc->hw_ctl)); - phys_enc->hw_ctl = NULL; - return; - } - _dpu_encoder_phys_vid_setup_irq_hw_idx(phys_enc); } @@ -481,7 +463,7 @@ static void dpu_encoder_phys_vid_enable(struct dpu_encoder_phys *phys_enc) { struct msm_drm_private *priv; struct dpu_encoder_phys_vid *vid_enc; - struct dpu_hw_intf *intf; + struct dpu_rm_hw_iter iter; struct dpu_hw_ctl *ctl; u32 flush_mask = 0; @@ -493,11 +475,20 @@ static void dpu_encoder_phys_vid_enable(struct dpu_encoder_phys *phys_enc) priv = phys_enc->parent->dev->dev_private; vid_enc = to_dpu_encoder_phys_vid(phys_enc); - intf = vid_enc->hw_intf; ctl = phys_enc->hw_ctl; - if (!vid_enc->hw_intf || !phys_enc->hw_ctl) { - DPU_ERROR("invalid hw_intf %d hw_ctl %d\n", - vid_enc->hw_intf != 0, phys_enc->hw_ctl != 0); + + dpu_rm_init_hw_iter(&iter, phys_enc->parent->base.id, DPU_HW_BLK_INTF); + while (dpu_rm_get_hw(&phys_enc->dpu_kms->rm, &iter)) { + struct dpu_hw_intf *hw_intf = (struct dpu_hw_intf *)iter.hw; + + if (hw_intf->idx == phys_enc->intf_idx) { + vid_enc->hw_intf = hw_intf; + break; + } + } + + if (!vid_enc->hw_intf) { + DPU_ERROR("hw_intf not assigned\n"); return; } @@ -519,7 +510,7 @@ static void dpu_encoder_phys_vid_enable(struct dpu_encoder_phys *phys_enc) !dpu_encoder_phys_vid_is_master(phys_enc)) goto skip_flush; - ctl->ops.get_bitmask_intf(ctl, &flush_mask, intf->idx); + ctl->ops.get_bitmask_intf(ctl, &flush_mask, vid_enc->hw_intf->idx); ctl->ops.update_pending_flush(ctl, flush_mask); skip_flush: @@ -547,25 +538,9 @@ static void dpu_encoder_phys_vid_destroy(struct dpu_encoder_phys *phys_enc) static void dpu_encoder_phys_vid_get_hw_resources( struct dpu_encoder_phys *phys_enc, - struct dpu_encoder_hw_resources *hw_res, - struct drm_connector_state *conn_state) + struct dpu_encoder_hw_resources *hw_res) { - struct dpu_encoder_phys_vid *vid_enc; - - if (!phys_enc || !hw_res) { - DPU_ERROR("invalid arg(s), enc %d hw_res %d conn_state %d\n", - phys_enc != 0, hw_res != 0, conn_state != 0); - return; - } - - vid_enc = to_dpu_encoder_phys_vid(phys_enc); - if (!vid_enc->hw_intf) { - DPU_ERROR("invalid arg(s), hw_intf\n"); - return; - } - - DPU_DEBUG_VIDENC(vid_enc, "\n"); - hw_res->intfs[vid_enc->hw_intf->idx - INTF_0] = INTF_MODE_VIDEO; + hw_res->intfs[phys_enc->intf_idx - INTF_0] = INTF_MODE_VIDEO; } static int _dpu_encoder_phys_vid_wait_for_vblank( @@ -756,32 +731,6 @@ static void dpu_encoder_phys_vid_irq_control(struct dpu_encoder_phys *phys_enc, } } -static void dpu_encoder_phys_vid_setup_misr(struct dpu_encoder_phys *phys_enc, - bool enable, u32 frame_count) -{ - struct dpu_encoder_phys_vid *vid_enc; - - if (!phys_enc) - return; - vid_enc = to_dpu_encoder_phys_vid(phys_enc); - - if (vid_enc->hw_intf && vid_enc->hw_intf->ops.setup_misr) - vid_enc->hw_intf->ops.setup_misr(vid_enc->hw_intf, - enable, frame_count); -} - -static u32 dpu_encoder_phys_vid_collect_misr(struct dpu_encoder_phys *phys_enc) -{ - struct dpu_encoder_phys_vid *vid_enc; - - if (!phys_enc) - return 0; - vid_enc = to_dpu_encoder_phys_vid(phys_enc); - - return vid_enc->hw_intf && vid_enc->hw_intf->ops.collect_misr ? - vid_enc->hw_intf->ops.collect_misr(vid_enc->hw_intf) : 0; -} - static int dpu_encoder_phys_vid_get_line_count( struct dpu_encoder_phys *phys_enc) { @@ -817,8 +766,6 @@ static void dpu_encoder_phys_vid_init_ops(struct dpu_encoder_phys_ops *ops) ops->prepare_for_kickoff = dpu_encoder_phys_vid_prepare_for_kickoff; ops->handle_post_kickoff = dpu_encoder_phys_vid_handle_post_kickoff; ops->needs_single_flush = dpu_encoder_phys_vid_needs_single_flush; - ops->setup_misr = dpu_encoder_phys_vid_setup_misr; - ops->collect_misr = dpu_encoder_phys_vid_collect_misr; ops->hw_reset = dpu_encoder_helper_hw_reset; ops->get_line_count = dpu_encoder_phys_vid_get_line_count; } @@ -828,8 +775,6 @@ struct dpu_encoder_phys *dpu_encoder_phys_vid_init( { struct dpu_encoder_phys *phys_enc = NULL; struct dpu_encoder_phys_vid *vid_enc = NULL; - struct dpu_rm_hw_iter iter; - struct dpu_hw_mdp *hw_mdp; struct dpu_encoder_irq *irq; int i, ret = 0; @@ -846,35 +791,9 @@ struct dpu_encoder_phys *dpu_encoder_phys_vid_init( phys_enc = &vid_enc->base; - hw_mdp = dpu_rm_get_mdp(&p->dpu_kms->rm); - if (IS_ERR_OR_NULL(hw_mdp)) { - ret = PTR_ERR(hw_mdp); - DPU_ERROR("failed to get mdptop\n"); - goto fail; - } - phys_enc->hw_mdptop = hw_mdp; + phys_enc->hw_mdptop = p->dpu_kms->hw_mdp; phys_enc->intf_idx = p->intf_idx; - /** - * hw_intf resource permanently assigned to this encoder - * Other resources allocated at atomic commit time by use case - */ - dpu_rm_init_hw_iter(&iter, 0, DPU_HW_BLK_INTF); - while (dpu_rm_get_hw(&p->dpu_kms->rm, &iter)) { - struct dpu_hw_intf *hw_intf = (struct dpu_hw_intf *)iter.hw; - - if (hw_intf->idx == p->intf_idx) { - vid_enc->hw_intf = hw_intf; - break; - } - } - - if (!vid_enc->hw_intf) { - ret = -EINVAL; - DPU_ERROR("failed to get hw_intf\n"); - goto fail; - } - DPU_DEBUG_VIDENC(vid_enc, "\n"); dpu_encoder_phys_vid_init_ops(&phys_enc->ops); diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c index 44ee06398b1d..512ac0834d2b 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c @@ -29,6 +29,9 @@ BIT(DPU_SSPP_TS_PREFILL) | BIT(DPU_SSPP_TS_PREFILL_REC1) |\ BIT(DPU_SSPP_CDP) | BIT(DPU_SSPP_EXCL_RECT)) +#define DMA_CURSOR_SDM845_MASK \ + (DMA_SDM845_MASK | BIT(DPU_SSPP_CURSOR)) + #define MIXER_SDM845_MASK \ (BIT(DPU_MIXER_SOURCESPLIT) | BIT(DPU_DIM_LAYER)) @@ -71,7 +74,6 @@ static struct dpu_mdp_cfg sdm845_mdp[] = { .base = 0x0, .len = 0x45C, .features = 0, .highest_bank_bit = 0x2, - .has_dest_scaler = true, .clk_ctrls[DPU_CLK_CTRL_VIG0] = { .reg_off = 0x2AC, .bit_off = 0}, .clk_ctrls[DPU_CLK_CTRL_VIG1] = { @@ -174,45 +176,35 @@ static const struct dpu_sspp_sub_blks sdm845_dma_sblk_1 = _DMA_SBLK("9", 2); static const struct dpu_sspp_sub_blks sdm845_dma_sblk_2 = _DMA_SBLK("10", 3); static const struct dpu_sspp_sub_blks sdm845_dma_sblk_3 = _DMA_SBLK("11", 4); -#define SSPP_VIG_BLK(_name, _id, _base, _sblk, _xinid, _clkctrl) \ - { \ - .name = _name, .id = _id, \ - .base = _base, .len = 0x1c8, \ - .features = VIG_SDM845_MASK, \ - .sblk = &_sblk, \ - .xin_id = _xinid, \ - .type = SSPP_TYPE_VIG, \ - .clk_ctrl = _clkctrl \ - } - -#define SSPP_DMA_BLK(_name, _id, _base, _sblk, _xinid, _clkctrl) \ +#define SSPP_BLK(_name, _id, _base, _features, \ + _sblk, _xinid, _type, _clkctrl) \ { \ .name = _name, .id = _id, \ .base = _base, .len = 0x1c8, \ - .features = DMA_SDM845_MASK, \ + .features = _features, \ .sblk = &_sblk, \ .xin_id = _xinid, \ - .type = SSPP_TYPE_DMA, \ + .type = _type, \ .clk_ctrl = _clkctrl \ } static struct dpu_sspp_cfg sdm845_sspp[] = { - SSPP_VIG_BLK("sspp_0", SSPP_VIG0, 0x4000, - sdm845_vig_sblk_0, 0, DPU_CLK_CTRL_VIG0), - SSPP_VIG_BLK("sspp_1", SSPP_VIG1, 0x6000, - sdm845_vig_sblk_1, 4, DPU_CLK_CTRL_VIG1), - SSPP_VIG_BLK("sspp_2", SSPP_VIG2, 0x8000, - sdm845_vig_sblk_2, 8, DPU_CLK_CTRL_VIG2), - SSPP_VIG_BLK("sspp_3", SSPP_VIG3, 0xa000, - sdm845_vig_sblk_3, 12, DPU_CLK_CTRL_VIG3), - SSPP_DMA_BLK("sspp_8", SSPP_DMA0, 0x24000, - sdm845_dma_sblk_0, 1, DPU_CLK_CTRL_DMA0), - SSPP_DMA_BLK("sspp_9", SSPP_DMA1, 0x26000, - sdm845_dma_sblk_1, 5, DPU_CLK_CTRL_DMA1), - SSPP_DMA_BLK("sspp_10", SSPP_DMA2, 0x28000, - sdm845_dma_sblk_2, 9, DPU_CLK_CTRL_CURSOR0), - SSPP_DMA_BLK("sspp_11", SSPP_DMA3, 0x2a000, - sdm845_dma_sblk_3, 13, DPU_CLK_CTRL_CURSOR1), + SSPP_BLK("sspp_0", SSPP_VIG0, 0x4000, VIG_SDM845_MASK, + sdm845_vig_sblk_0, 0, SSPP_TYPE_VIG, DPU_CLK_CTRL_VIG0), + SSPP_BLK("sspp_1", SSPP_VIG1, 0x6000, VIG_SDM845_MASK, + sdm845_vig_sblk_1, 4, SSPP_TYPE_VIG, DPU_CLK_CTRL_VIG1), + SSPP_BLK("sspp_2", SSPP_VIG2, 0x8000, VIG_SDM845_MASK, + sdm845_vig_sblk_2, 8, SSPP_TYPE_VIG, DPU_CLK_CTRL_VIG2), + SSPP_BLK("sspp_3", SSPP_VIG3, 0xa000, VIG_SDM845_MASK, + sdm845_vig_sblk_3, 12, SSPP_TYPE_VIG, DPU_CLK_CTRL_VIG3), + SSPP_BLK("sspp_8", SSPP_DMA0, 0x24000, DMA_SDM845_MASK, + sdm845_dma_sblk_0, 1, SSPP_TYPE_DMA, DPU_CLK_CTRL_DMA0), + SSPP_BLK("sspp_9", SSPP_DMA1, 0x26000, DMA_SDM845_MASK, + sdm845_dma_sblk_1, 5, SSPP_TYPE_DMA, DPU_CLK_CTRL_DMA1), + SSPP_BLK("sspp_10", SSPP_DMA2, 0x28000, DMA_CURSOR_SDM845_MASK, + sdm845_dma_sblk_2, 9, SSPP_TYPE_DMA, DPU_CLK_CTRL_CURSOR0), + SSPP_BLK("sspp_11", SSPP_DMA3, 0x2a000, DMA_CURSOR_SDM845_MASK, + sdm845_dma_sblk_3, 13, SSPP_TYPE_DMA, DPU_CLK_CTRL_CURSOR1), }; /************************************************************* @@ -227,48 +219,23 @@ static const struct dpu_lm_sub_blks sdm845_lm_sblk = { }, }; -#define LM_BLK(_name, _id, _base, _ds, _pp, _lmpair) \ +#define LM_BLK(_name, _id, _base, _pp, _lmpair) \ { \ .name = _name, .id = _id, \ .base = _base, .len = 0x320, \ .features = MIXER_SDM845_MASK, \ .sblk = &sdm845_lm_sblk, \ - .ds = _ds, \ .pingpong = _pp, \ .lm_pair_mask = (1 << _lmpair) \ } static struct dpu_lm_cfg sdm845_lm[] = { - LM_BLK("lm_0", LM_0, 0x44000, DS_0, PINGPONG_0, LM_1), - LM_BLK("lm_1", LM_1, 0x45000, DS_1, PINGPONG_1, LM_0), - LM_BLK("lm_2", LM_2, 0x46000, DS_MAX, PINGPONG_2, LM_5), - LM_BLK("lm_3", LM_3, 0x0, DS_MAX, PINGPONG_MAX, 0), - LM_BLK("lm_4", LM_4, 0x0, DS_MAX, PINGPONG_MAX, 0), - LM_BLK("lm_5", LM_5, 0x49000, DS_MAX, PINGPONG_3, LM_2), -}; - -/************************************************************* - * DS sub blocks config - *************************************************************/ -static const struct dpu_ds_top_cfg sdm845_ds_top = { - .name = "ds_top_0", .id = DS_TOP, - .base = 0x60000, .len = 0xc, - .maxinputwidth = DEFAULT_DPU_LINE_WIDTH, - .maxoutputwidth = DEFAULT_DPU_OUTPUT_LINE_WIDTH, - .maxupscale = MAX_UPSCALE_RATIO, -}; - -#define DS_BLK(_name, _id, _base) \ - {\ - .name = _name, .id = _id, \ - .base = _base, .len = 0x800, \ - .features = DPU_SSPP_SCALER_QSEED3, \ - .top = &sdm845_ds_top \ - } - -static struct dpu_ds_cfg sdm845_ds[] = { - DS_BLK("ds_0", DS_0, 0x800), - DS_BLK("ds_1", DS_1, 0x1000), + LM_BLK("lm_0", LM_0, 0x44000, PINGPONG_0, LM_1), + LM_BLK("lm_1", LM_1, 0x45000, PINGPONG_1, LM_0), + LM_BLK("lm_2", LM_2, 0x46000, PINGPONG_2, LM_5), + LM_BLK("lm_3", LM_3, 0x0, PINGPONG_MAX, 0), + LM_BLK("lm_4", LM_4, 0x0, PINGPONG_MAX, 0), + LM_BLK("lm_5", LM_5, 0x49000, PINGPONG_3, LM_2), }; /************************************************************* @@ -328,18 +295,6 @@ static struct dpu_intf_cfg sdm845_intf[] = { }; /************************************************************* - * CDM sub blocks config - *************************************************************/ -static struct dpu_cdm_cfg sdm845_cdm[] = { - { - .name = "cdm_0", .id = CDM_0, - .base = 0x79200, .len = 0x224, - .features = 0, - .intf_connect = BIT(INTF_3), - }, -}; - -/************************************************************* * VBIF sub blocks config *************************************************************/ /* VBIF QOS remap */ @@ -461,12 +416,8 @@ static void sdm845_cfg_init(struct dpu_mdss_cfg *dpu_cfg) .sspp = sdm845_sspp, .mixer_count = ARRAY_SIZE(sdm845_lm), .mixer = sdm845_lm, - .ds_count = ARRAY_SIZE(sdm845_ds), - .ds = sdm845_ds, .pingpong_count = ARRAY_SIZE(sdm845_pp), .pingpong = sdm845_pp, - .cdm_count = ARRAY_SIZE(sdm845_cdm), - .cdm = sdm845_cdm, .intf_count = ARRAY_SIZE(sdm845_intf), .intf = sdm845_intf, .vbif_count = ARRAY_SIZE(sdm845_vbif), diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h index f0cb0d4fc80e..dc060e7358e4 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h @@ -428,7 +428,6 @@ struct dpu_clk_ctrl_reg { * @highest_bank_bit: UBWC parameter * @ubwc_static: ubwc static configuration * @ubwc_swizzle: ubwc default swizzle setting - * @has_dest_scaler: indicates support of destination scaler * @clk_ctrls clock control register definition */ struct dpu_mdp_cfg { @@ -436,7 +435,6 @@ struct dpu_mdp_cfg { u32 highest_bank_bit; u32 ubwc_static; u32 ubwc_swizzle; - bool has_dest_scaler; struct dpu_clk_ctrl_reg clk_ctrls[DPU_CLK_CTRL_MAX]; }; @@ -474,50 +472,16 @@ struct dpu_sspp_cfg { * @features bit mask identifying sub-blocks/features * @sblk: LM Sub-blocks information * @pingpong: ID of connected PingPong, PINGPONG_MAX if unsupported - * @ds: ID of connected DS, DS_MAX if unsupported * @lm_pair_mask: Bitmask of LMs that can be controlled by same CTL */ struct dpu_lm_cfg { DPU_HW_BLK_INFO; const struct dpu_lm_sub_blks *sblk; u32 pingpong; - u32 ds; unsigned long lm_pair_mask; }; /** - * struct dpu_ds_top_cfg - information of dest scaler top - * @id enum identifying this block - * @base register offset of this block - * @features bit mask identifying features - * @version hw version of dest scaler - * @maxinputwidth maximum input line width - * @maxoutputwidth maximum output line width - * @maxupscale maximum upscale ratio - */ -struct dpu_ds_top_cfg { - DPU_HW_BLK_INFO; - u32 version; - u32 maxinputwidth; - u32 maxoutputwidth; - u32 maxupscale; -}; - -/** - * struct dpu_ds_cfg - information of dest scaler blocks - * @id enum identifying this block - * @base register offset wrt DS top offset - * @features bit mask identifying features - * @version hw version of the qseed block - * @top DS top information - */ -struct dpu_ds_cfg { - DPU_HW_BLK_INFO; - u32 version; - const struct dpu_ds_top_cfg *top; -}; - -/** * struct dpu_pingpong_cfg - information of PING-PONG blocks * @id enum identifying this block * @base register offset of this block @@ -530,18 +494,6 @@ struct dpu_pingpong_cfg { }; /** - * struct dpu_cdm_cfg - information of chroma down blocks - * @id enum identifying this block - * @base register offset of this block - * @features bit mask identifying sub-blocks/features - * @intf_connect Bitmask of INTF IDs this CDM can connect to - */ -struct dpu_cdm_cfg { - DPU_HW_BLK_INFO; - unsigned long intf_connect; -}; - -/** * struct dpu_intf_cfg - information of timing engine blocks * @id enum identifying this block * @base register offset of this block @@ -728,15 +680,9 @@ struct dpu_mdss_cfg { u32 mixer_count; struct dpu_lm_cfg *mixer; - u32 ds_count; - struct dpu_ds_cfg *ds; - u32 pingpong_count; struct dpu_pingpong_cfg *pingpong; - u32 cdm_count; - struct dpu_cdm_cfg *cdm; - u32 intf_count; struct dpu_intf_cfg *intf; @@ -771,9 +717,7 @@ struct dpu_mdss_hw_cfg_handler { #define BLK_DMA(s) ((s)->dma) #define BLK_CURSOR(s) ((s)->cursor) #define BLK_MIXER(s) ((s)->mixer) -#define BLK_DS(s) ((s)->ds) #define BLK_PINGPONG(s) ((s)->pingpong) -#define BLK_CDM(s) ((s)->cdm) #define BLK_INTF(s) ((s)->intf) #define BLK_AD(s) ((s)->ad) diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_cdm.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_cdm.c deleted file mode 100644 index 554874ba0c3b..000000000000 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_cdm.c +++ /dev/null @@ -1,323 +0,0 @@ -/* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 and - * only version 2 as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - */ - -#include "dpu_hw_mdss.h" -#include "dpu_hwio.h" -#include "dpu_hw_catalog.h" -#include "dpu_hw_cdm.h" -#include "dpu_dbg.h" -#include "dpu_kms.h" - -#define CDM_CSC_10_OPMODE 0x000 -#define CDM_CSC_10_BASE 0x004 - -#define CDM_CDWN2_OP_MODE 0x100 -#define CDM_CDWN2_CLAMP_OUT 0x104 -#define CDM_CDWN2_PARAMS_3D_0 0x108 -#define CDM_CDWN2_PARAMS_3D_1 0x10C -#define CDM_CDWN2_COEFF_COSITE_H_0 0x110 -#define CDM_CDWN2_COEFF_COSITE_H_1 0x114 -#define CDM_CDWN2_COEFF_COSITE_H_2 0x118 -#define CDM_CDWN2_COEFF_OFFSITE_H_0 0x11C -#define CDM_CDWN2_COEFF_OFFSITE_H_1 0x120 -#define CDM_CDWN2_COEFF_OFFSITE_H_2 0x124 -#define CDM_CDWN2_COEFF_COSITE_V 0x128 -#define CDM_CDWN2_COEFF_OFFSITE_V 0x12C -#define CDM_CDWN2_OUT_SIZE 0x130 - -#define CDM_HDMI_PACK_OP_MODE 0x200 -#define CDM_CSC_10_MATRIX_COEFF_0 0x004 - -/** - * Horizontal coefficients for cosite chroma downscale - * s13 representation of coefficients - */ -static u32 cosite_h_coeff[] = {0x00000016, 0x000001cc, 0x0100009e}; - -/** - * Horizontal coefficients for offsite chroma downscale - */ -static u32 offsite_h_coeff[] = {0x000b0005, 0x01db01eb, 0x00e40046}; - -/** - * Vertical coefficients for cosite chroma downscale - */ -static u32 cosite_v_coeff[] = {0x00080004}; -/** - * Vertical coefficients for offsite chroma downscale - */ -static u32 offsite_v_coeff[] = {0x00060002}; - -/* Limited Range rgb2yuv coeff with clamp and bias values for CSC 10 module */ -static struct dpu_csc_cfg rgb2yuv_cfg = { - { - 0x0083, 0x0102, 0x0032, - 0x1fb5, 0x1f6c, 0x00e1, - 0x00e1, 0x1f45, 0x1fdc - }, - { 0x00, 0x00, 0x00 }, - { 0x0040, 0x0200, 0x0200 }, - { 0x000, 0x3ff, 0x000, 0x3ff, 0x000, 0x3ff }, - { 0x040, 0x3ac, 0x040, 0x3c0, 0x040, 0x3c0 }, -}; - -static struct dpu_cdm_cfg *_cdm_offset(enum dpu_cdm cdm, - struct dpu_mdss_cfg *m, - void __iomem *addr, - struct dpu_hw_blk_reg_map *b) -{ - int i; - - for (i = 0; i < m->cdm_count; i++) { - if (cdm == m->cdm[i].id) { - b->base_off = addr; - b->blk_off = m->cdm[i].base; - b->length = m->cdm[i].len; - b->hwversion = m->hwversion; - b->log_mask = DPU_DBG_MASK_CDM; - return &m->cdm[i]; - } - } - - return ERR_PTR(-EINVAL); -} - -static int dpu_hw_cdm_setup_csc_10bit(struct dpu_hw_cdm *ctx, - struct dpu_csc_cfg *data) -{ - dpu_hw_csc_setup(&ctx->hw, CDM_CSC_10_MATRIX_COEFF_0, data, true); - - return 0; -} - -static int dpu_hw_cdm_setup_cdwn(struct dpu_hw_cdm *ctx, - struct dpu_hw_cdm_cfg *cfg) -{ - struct dpu_hw_blk_reg_map *c = &ctx->hw; - u32 opmode = 0; - u32 out_size = 0; - - if (cfg->output_bit_depth == CDM_CDWN_OUTPUT_10BIT) - opmode &= ~BIT(7); - else - opmode |= BIT(7); - - /* ENABLE DWNS_H bit */ - opmode |= BIT(1); - - switch (cfg->h_cdwn_type) { - case CDM_CDWN_DISABLE: - /* CLEAR METHOD_H field */ - opmode &= ~(0x18); - /* CLEAR DWNS_H bit */ - opmode &= ~BIT(1); - break; - case CDM_CDWN_PIXEL_DROP: - /* Clear METHOD_H field (pixel drop is 0) */ - opmode &= ~(0x18); - break; - case CDM_CDWN_AVG: - /* Clear METHOD_H field (Average is 0x1) */ - opmode &= ~(0x18); - opmode |= (0x1 << 0x3); - break; - case CDM_CDWN_COSITE: - /* Clear METHOD_H field (Average is 0x2) */ - opmode &= ~(0x18); - opmode |= (0x2 << 0x3); - /* Co-site horizontal coefficients */ - DPU_REG_WRITE(c, CDM_CDWN2_COEFF_COSITE_H_0, - cosite_h_coeff[0]); - DPU_REG_WRITE(c, CDM_CDWN2_COEFF_COSITE_H_1, - cosite_h_coeff[1]); - DPU_REG_WRITE(c, CDM_CDWN2_COEFF_COSITE_H_2, - cosite_h_coeff[2]); - break; - case CDM_CDWN_OFFSITE: - /* Clear METHOD_H field (Average is 0x3) */ - opmode &= ~(0x18); - opmode |= (0x3 << 0x3); - - /* Off-site horizontal coefficients */ - DPU_REG_WRITE(c, CDM_CDWN2_COEFF_OFFSITE_H_0, - offsite_h_coeff[0]); - DPU_REG_WRITE(c, CDM_CDWN2_COEFF_OFFSITE_H_1, - offsite_h_coeff[1]); - DPU_REG_WRITE(c, CDM_CDWN2_COEFF_OFFSITE_H_2, - offsite_h_coeff[2]); - break; - default: - pr_err("%s invalid horz down sampling type\n", __func__); - return -EINVAL; - } - - /* ENABLE DWNS_V bit */ - opmode |= BIT(2); - - switch (cfg->v_cdwn_type) { - case CDM_CDWN_DISABLE: - /* CLEAR METHOD_V field */ - opmode &= ~(0x60); - /* CLEAR DWNS_V bit */ - opmode &= ~BIT(2); - break; - case CDM_CDWN_PIXEL_DROP: - /* Clear METHOD_V field (pixel drop is 0) */ - opmode &= ~(0x60); - break; - case CDM_CDWN_AVG: - /* Clear METHOD_V field (Average is 0x1) */ - opmode &= ~(0x60); - opmode |= (0x1 << 0x5); - break; - case CDM_CDWN_COSITE: - /* Clear METHOD_V field (Average is 0x2) */ - opmode &= ~(0x60); - opmode |= (0x2 << 0x5); - /* Co-site vertical coefficients */ - DPU_REG_WRITE(c, - CDM_CDWN2_COEFF_COSITE_V, - cosite_v_coeff[0]); - break; - case CDM_CDWN_OFFSITE: - /* Clear METHOD_V field (Average is 0x3) */ - opmode &= ~(0x60); - opmode |= (0x3 << 0x5); - - /* Off-site vertical coefficients */ - DPU_REG_WRITE(c, - CDM_CDWN2_COEFF_OFFSITE_V, - offsite_v_coeff[0]); - break; - default: - return -EINVAL; - } - - if (cfg->v_cdwn_type || cfg->h_cdwn_type) - opmode |= BIT(0); /* EN CDWN module */ - else - opmode &= ~BIT(0); - - out_size = (cfg->output_width & 0xFFFF) | - ((cfg->output_height & 0xFFFF) << 16); - DPU_REG_WRITE(c, CDM_CDWN2_OUT_SIZE, out_size); - DPU_REG_WRITE(c, CDM_CDWN2_OP_MODE, opmode); - DPU_REG_WRITE(c, CDM_CDWN2_CLAMP_OUT, - ((0x3FF << 16) | 0x0)); - - return 0; -} - -static int dpu_hw_cdm_enable(struct dpu_hw_cdm *ctx, - struct dpu_hw_cdm_cfg *cdm) -{ - struct dpu_hw_blk_reg_map *c = &ctx->hw; - const struct dpu_format *fmt = cdm->output_fmt; - struct cdm_output_cfg cdm_cfg = { 0 }; - u32 opmode = 0; - u32 csc = 0; - - if (!DPU_FORMAT_IS_YUV(fmt)) - return -EINVAL; - - if (cdm->output_type == CDM_CDWN_OUTPUT_HDMI) { - if (fmt->chroma_sample != DPU_CHROMA_H1V2) - return -EINVAL; /*unsupported format */ - opmode = BIT(0); - opmode |= (fmt->chroma_sample << 1); - cdm_cfg.intf_en = true; - } - - csc |= BIT(2); - csc &= ~BIT(1); - csc |= BIT(0); - - if (ctx->hw_mdp && ctx->hw_mdp->ops.setup_cdm_output) - ctx->hw_mdp->ops.setup_cdm_output(ctx->hw_mdp, &cdm_cfg); - - DPU_REG_WRITE(c, CDM_CSC_10_OPMODE, csc); - DPU_REG_WRITE(c, CDM_HDMI_PACK_OP_MODE, opmode); - return 0; -} - -static void dpu_hw_cdm_disable(struct dpu_hw_cdm *ctx) -{ - struct cdm_output_cfg cdm_cfg = { 0 }; - - if (ctx->hw_mdp && ctx->hw_mdp->ops.setup_cdm_output) - ctx->hw_mdp->ops.setup_cdm_output(ctx->hw_mdp, &cdm_cfg); -} - -static void _setup_cdm_ops(struct dpu_hw_cdm_ops *ops, - unsigned long features) -{ - ops->setup_csc_data = dpu_hw_cdm_setup_csc_10bit; - ops->setup_cdwn = dpu_hw_cdm_setup_cdwn; - ops->enable = dpu_hw_cdm_enable; - ops->disable = dpu_hw_cdm_disable; -} - -static struct dpu_hw_blk_ops dpu_hw_ops = { - .start = NULL, - .stop = NULL, -}; - -struct dpu_hw_cdm *dpu_hw_cdm_init(enum dpu_cdm idx, - void __iomem *addr, - struct dpu_mdss_cfg *m, - struct dpu_hw_mdp *hw_mdp) -{ - struct dpu_hw_cdm *c; - struct dpu_cdm_cfg *cfg; - int rc; - - c = kzalloc(sizeof(*c), GFP_KERNEL); - if (!c) - return ERR_PTR(-ENOMEM); - - cfg = _cdm_offset(idx, m, addr, &c->hw); - if (IS_ERR_OR_NULL(cfg)) { - kfree(c); - return ERR_PTR(-EINVAL); - } - - c->idx = idx; - c->caps = cfg; - _setup_cdm_ops(&c->ops, c->caps->features); - c->hw_mdp = hw_mdp; - - rc = dpu_hw_blk_init(&c->base, DPU_HW_BLK_CDM, idx, &dpu_hw_ops); - if (rc) { - DPU_ERROR("failed to init hw blk %d\n", rc); - goto blk_init_error; - } - - /* - * Perform any default initialization for the chroma down module - * @setup default csc coefficients - */ - dpu_hw_cdm_setup_csc_10bit(c, &rgb2yuv_cfg); - - return c; - -blk_init_error: - kzfree(c); - - return ERR_PTR(rc); -} - -void dpu_hw_cdm_destroy(struct dpu_hw_cdm *cdm) -{ - if (cdm) - dpu_hw_blk_destroy(&cdm->base); - kfree(cdm); -} diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_cdm.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_cdm.h deleted file mode 100644 index 5cceb1ecb8e0..000000000000 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_cdm.h +++ /dev/null @@ -1,139 +0,0 @@ -/* Copyright (c) 2015-2018, The Linux Foundation. All rights reserved. - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 and - * only version 2 as published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - */ - -#ifndef _DPU_HW_CDM_H -#define _DPU_HW_CDM_H - -#include "dpu_hw_mdss.h" -#include "dpu_hw_top.h" -#include "dpu_hw_blk.h" - -struct dpu_hw_cdm; - -struct dpu_hw_cdm_cfg { - u32 output_width; - u32 output_height; - u32 output_bit_depth; - u32 h_cdwn_type; - u32 v_cdwn_type; - const struct dpu_format *output_fmt; - u32 output_type; - int flags; -}; - -enum dpu_hw_cdwn_type { - CDM_CDWN_DISABLE, - CDM_CDWN_PIXEL_DROP, - CDM_CDWN_AVG, - CDM_CDWN_COSITE, - CDM_CDWN_OFFSITE, -}; - -enum dpu_hw_cdwn_output_type { - CDM_CDWN_OUTPUT_HDMI, - CDM_CDWN_OUTPUT_WB, -}; - -enum dpu_hw_cdwn_output_bit_depth { - CDM_CDWN_OUTPUT_8BIT, - CDM_CDWN_OUTPUT_10BIT, -}; - -/** - * struct dpu_hw_cdm_ops : Interface to the chroma down Hw driver functions - * Assumption is these functions will be called after - * clocks are enabled - * @setup_csc: Programs the csc matrix - * @setup_cdwn: Sets up the chroma down sub module - * @enable: Enables the output to interface and programs the - * output packer - * @disable: Puts the cdm in bypass mode - */ -struct dpu_hw_cdm_ops { - /** - * Programs the CSC matrix for conversion from RGB space to YUV space, - * it is optional to call this function as this matrix is automatically - * set during initialization, user should call this if it wants - * to program a different matrix than default matrix. - * @cdm: Pointer to the chroma down context structure - * @data Pointer to CSC configuration data - * return: 0 if success; error code otherwise - */ - int (*setup_csc_data)(struct dpu_hw_cdm *cdm, - struct dpu_csc_cfg *data); - - /** - * Programs the Chroma downsample part. - * @cdm Pointer to chroma down context - */ - int (*setup_cdwn)(struct dpu_hw_cdm *cdm, - struct dpu_hw_cdm_cfg *cfg); - - /** - * Enable the CDM module - * @cdm Pointer to chroma down context - */ - int (*enable)(struct dpu_hw_cdm *cdm, - struct dpu_hw_cdm_cfg *cfg); - - /** - * Disable the CDM module - * @cdm Pointer to chroma down context - */ - void (*disable)(struct dpu_hw_cdm *cdm); -}; - -struct dpu_hw_cdm { - struct dpu_hw_blk base; - struct dpu_hw_blk_reg_map hw; - - /* chroma down */ - const struct dpu_cdm_cfg *caps; - enum dpu_cdm idx; - - /* mdp top hw driver */ - struct dpu_hw_mdp *hw_mdp; - - /* ops */ - struct dpu_hw_cdm_ops ops; -}; - -/** - * dpu_hw_cdm - convert base object dpu_hw_base to container - * @hw: Pointer to base hardware block - * return: Pointer to hardware block container - */ -static inline struct dpu_hw_cdm *to_dpu_hw_cdm(struct dpu_hw_blk *hw) -{ - return container_of(hw, struct dpu_hw_cdm, base); -} - -/** - * dpu_hw_cdm_init - initializes the cdm hw driver object. - * should be called once before accessing every cdm. - * @idx: cdm index for which driver object is required - * @addr: mapped register io address of MDP - * @m : pointer to mdss catalog data - * @hw_mdp: pointer to mdp top hw driver object - */ -struct dpu_hw_cdm *dpu_hw_cdm_init(enum dpu_cdm idx, - void __iomem *addr, - struct dpu_mdss_cfg *m, - struct dpu_hw_mdp *hw_mdp); - -/** - * dpu_hw_cdm_destroy - destroys CDM driver context - * @cdm: pointer to CDM driver context - */ -void dpu_hw_cdm_destroy(struct dpu_hw_cdm *cdm); - -#endif /*_DPU_HW_CDM_H */ diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c index 06be7cf7ce50..eec1051f2afc 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c @@ -224,19 +224,6 @@ static inline int dpu_hw_ctl_get_bitmask_intf(struct dpu_hw_ctl *ctx, return 0; } -static inline int dpu_hw_ctl_get_bitmask_cdm(struct dpu_hw_ctl *ctx, - u32 *flushbits, enum dpu_cdm cdm) -{ - switch (cdm) { - case CDM_0: - *flushbits |= BIT(26); - break; - default: - return -EINVAL; - } - return 0; -} - static u32 dpu_hw_ctl_poll_reset_status(struct dpu_hw_ctl *ctx, u32 timeout_us) { struct dpu_hw_blk_reg_map *c = &ctx->hw; @@ -310,7 +297,7 @@ static void dpu_hw_ctl_setup_blendstage(struct dpu_hw_ctl *ctx, u32 mixercfg = 0, mixercfg_ext = 0, mix, ext; u32 mixercfg_ext2 = 0, mixercfg_ext3 = 0; int i, j; - u8 stages; + int stages; int pipes_per_stage; stages = _mixer_stages(ctx->mixer_hw_caps, ctx->mixer_count, lm); @@ -485,7 +472,6 @@ static void _setup_ctl_ops(struct dpu_hw_ctl_ops *ops, ops->get_bitmask_sspp = dpu_hw_ctl_get_bitmask_sspp; ops->get_bitmask_mixer = dpu_hw_ctl_get_bitmask_mixer; ops->get_bitmask_intf = dpu_hw_ctl_get_bitmask_intf; - ops->get_bitmask_cdm = dpu_hw_ctl_get_bitmask_cdm; }; static struct dpu_hw_blk_ops dpu_hw_ops = { diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.h index c66a71f8b839..6f313faca43e 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.h @@ -142,10 +142,6 @@ struct dpu_hw_ctl_ops { u32 *flushbits, enum dpu_intf blk); - int (*get_bitmask_cdm)(struct dpu_hw_ctl *ctx, - u32 *flushbits, - enum dpu_cdm blk); - /** * Set all blend stages to disabled * @ctx : ctl path ctx pointer diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.c index d280df5613c9..9c6bba0ac7c3 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.c @@ -65,9 +65,6 @@ #define INTF_FRAME_COUNT 0x0AC #define INTF_LINE_COUNT 0x0B0 -#define INTF_MISR_CTRL 0x180 -#define INTF_MISR_SIGNATURE 0x184 - static struct dpu_intf_cfg *_intf_offset(enum dpu_intf intf, struct dpu_mdss_cfg *m, void __iomem *addr, @@ -246,30 +243,6 @@ static void dpu_hw_intf_get_status( } } -static void dpu_hw_intf_setup_misr(struct dpu_hw_intf *intf, - bool enable, u32 frame_count) -{ - struct dpu_hw_blk_reg_map *c = &intf->hw; - u32 config = 0; - - DPU_REG_WRITE(c, INTF_MISR_CTRL, MISR_CTRL_STATUS_CLEAR); - /* clear misr data */ - wmb(); - - if (enable) - config = (frame_count & MISR_FRAME_COUNT_MASK) | - MISR_CTRL_ENABLE | INTF_MISR_CTRL_FREE_RUN_MASK; - - DPU_REG_WRITE(c, INTF_MISR_CTRL, config); -} - -static u32 dpu_hw_intf_collect_misr(struct dpu_hw_intf *intf) -{ - struct dpu_hw_blk_reg_map *c = &intf->hw; - - return DPU_REG_READ(c, INTF_MISR_SIGNATURE); -} - static u32 dpu_hw_intf_get_line_count(struct dpu_hw_intf *intf) { struct dpu_hw_blk_reg_map *c; @@ -289,8 +262,6 @@ static void _setup_intf_ops(struct dpu_hw_intf_ops *ops, ops->setup_prg_fetch = dpu_hw_intf_setup_prg_fetch; ops->get_status = dpu_hw_intf_get_status; ops->enable_timing = dpu_hw_intf_enable_timing_engine; - ops->setup_misr = dpu_hw_intf_setup_misr; - ops->collect_misr = dpu_hw_intf_collect_misr; ops->get_line_count = dpu_hw_intf_get_line_count; } diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.h index a79d735da68d..3b77df460dea 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.h @@ -59,8 +59,6 @@ struct intf_status { * @ setup_prog_fetch : enables/disables the programmable fetch logic * @ enable_timing: enable/disable timing engine * @ get_status: returns if timing engine is enabled or not - * @ setup_misr: enables/disables MISR in HW register - * @ collect_misr: reads and stores MISR data from HW register * @ get_line_count: reads current vertical line counter */ struct dpu_hw_intf_ops { @@ -77,11 +75,6 @@ struct dpu_hw_intf_ops { void (*get_status)(struct dpu_hw_intf *intf, struct intf_status *status); - void (*setup_misr)(struct dpu_hw_intf *intf, - bool enable, u32 frame_count); - - u32 (*collect_misr)(struct dpu_hw_intf *intf); - u32 (*get_line_count)(struct dpu_hw_intf *intf); }; diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.c index 4ab72b0f07a5..acb8dc8acaa5 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.c @@ -34,9 +34,6 @@ #define LM_BLEND0_FG_ALPHA 0x04 #define LM_BLEND0_BG_ALPHA 0x08 -#define LM_MISR_CTRL 0x310 -#define LM_MISR_SIGNATURE 0x314 - static struct dpu_lm_cfg *_lm_offset(enum dpu_lm mixer, struct dpu_mdss_cfg *m, void __iomem *addr, @@ -171,30 +168,6 @@ static void dpu_hw_lm_gc(struct dpu_hw_mixer *mixer, { } -static void dpu_hw_lm_setup_misr(struct dpu_hw_mixer *ctx, - bool enable, u32 frame_count) -{ - struct dpu_hw_blk_reg_map *c = &ctx->hw; - u32 config = 0; - - DPU_REG_WRITE(c, LM_MISR_CTRL, MISR_CTRL_STATUS_CLEAR); - /* clear misr data */ - wmb(); - - if (enable) - config = (frame_count & MISR_FRAME_COUNT_MASK) | - MISR_CTRL_ENABLE | INTF_MISR_CTRL_FREE_RUN_MASK; - - DPU_REG_WRITE(c, LM_MISR_CTRL, config); -} - -static u32 dpu_hw_lm_collect_misr(struct dpu_hw_mixer *ctx) -{ - struct dpu_hw_blk_reg_map *c = &ctx->hw; - - return DPU_REG_READ(c, LM_MISR_SIGNATURE); -} - static void _setup_mixer_ops(struct dpu_mdss_cfg *m, struct dpu_hw_lm_ops *ops, unsigned long features) @@ -207,8 +180,6 @@ static void _setup_mixer_ops(struct dpu_mdss_cfg *m, ops->setup_alpha_out = dpu_hw_lm_setup_color3; ops->setup_border_color = dpu_hw_lm_setup_border_color; ops->setup_gc = dpu_hw_lm_gc; - ops->setup_misr = dpu_hw_lm_setup_misr; - ops->collect_misr = dpu_hw_lm_collect_misr; }; static struct dpu_hw_blk_ops dpu_hw_ops = { diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.h index e29e5dab31bf..5b036aca8340 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_lm.h @@ -66,13 +66,6 @@ struct dpu_hw_lm_ops { */ void (*setup_gc)(struct dpu_hw_mixer *mixer, void *cfg); - - /* setup_misr: enables/disables MISR in HW register */ - void (*setup_misr)(struct dpu_hw_mixer *ctx, - bool enable, u32 frame_count); - - /* collect_misr: reads and stores MISR data from HW register */ - u32 (*collect_misr)(struct dpu_hw_mixer *ctx); }; struct dpu_hw_mixer { diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_mdss.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_mdss.h index 35e6bf930924..68c54d2c9677 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_mdss.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_mdss.h @@ -100,7 +100,6 @@ enum dpu_hw_blk_type { DPU_HW_BLK_SSPP, DPU_HW_BLK_LM, DPU_HW_BLK_CTL, - DPU_HW_BLK_CDM, DPU_HW_BLK_PINGPONG, DPU_HW_BLK_INTF, DPU_HW_BLK_WB, @@ -173,13 +172,6 @@ enum dpu_dspp { DSPP_MAX }; -enum dpu_ds { - DS_TOP, - DS_0, - DS_1, - DS_MAX -}; - enum dpu_ctl { CTL_0 = 1, CTL_1, @@ -189,12 +181,6 @@ enum dpu_ctl { CTL_MAX }; -enum dpu_cdm { - CDM_0 = 1, - CDM_1, - CDM_MAX -}; - enum dpu_pingpong { PINGPONG_0 = 1, PINGPONG_1, @@ -246,12 +232,6 @@ enum dpu_wb { WB_MAX }; -enum dpu_ad { - AD_0 = 0x1, - AD_1, - AD_MAX -}; - enum dpu_cwb { CWB_0 = 0x1, CWB_1, @@ -451,15 +431,14 @@ struct dpu_mdss_color { * Define bit masks for h/w logging. */ #define DPU_DBG_MASK_NONE (1 << 0) -#define DPU_DBG_MASK_CDM (1 << 1) -#define DPU_DBG_MASK_INTF (1 << 2) -#define DPU_DBG_MASK_LM (1 << 3) -#define DPU_DBG_MASK_CTL (1 << 4) -#define DPU_DBG_MASK_PINGPONG (1 << 5) -#define DPU_DBG_MASK_SSPP (1 << 6) -#define DPU_DBG_MASK_WB (1 << 7) -#define DPU_DBG_MASK_TOP (1 << 8) -#define DPU_DBG_MASK_VBIF (1 << 9) -#define DPU_DBG_MASK_ROT (1 << 10) +#define DPU_DBG_MASK_INTF (1 << 1) +#define DPU_DBG_MASK_LM (1 << 2) +#define DPU_DBG_MASK_CTL (1 << 3) +#define DPU_DBG_MASK_PINGPONG (1 << 4) +#define DPU_DBG_MASK_SSPP (1 << 5) +#define DPU_DBG_MASK_WB (1 << 6) +#define DPU_DBG_MASK_TOP (1 << 7) +#define DPU_DBG_MASK_VBIF (1 << 8) +#define DPU_DBG_MASK_ROT (1 << 9) #endif /* _DPU_HW_MDSS_H */ diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.c index db2798e862fc..b8781256e21b 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.c @@ -98,23 +98,6 @@ static void dpu_hw_setup_split_pipe(struct dpu_hw_mdp *mdp, DPU_REG_WRITE(c, SPLIT_DISPLAY_EN, cfg->en & 0x1); } -static void dpu_hw_setup_cdm_output(struct dpu_hw_mdp *mdp, - struct cdm_output_cfg *cfg) -{ - struct dpu_hw_blk_reg_map *c; - u32 out_ctl = 0; - - if (!mdp || !cfg) - return; - - c = &mdp->hw; - - if (cfg->intf_en) - out_ctl |= BIT(19); - - DPU_REG_WRITE(c, MDP_OUT_CTL_0, out_ctl); -} - static bool dpu_hw_setup_clk_force_ctrl(struct dpu_hw_mdp *mdp, enum dpu_clk_ctrl_type clk_ctrl, bool enable) { @@ -307,7 +290,6 @@ static void _setup_mdp_ops(struct dpu_hw_mdp_ops *ops, unsigned long cap) { ops->setup_split_pipe = dpu_hw_setup_split_pipe; - ops->setup_cdm_output = dpu_hw_setup_cdm_output; ops->setup_clk_force_ctrl = dpu_hw_setup_clk_force_ctrl; ops->get_danger_status = dpu_hw_get_danger_status; ops->setup_vsync_source = dpu_hw_setup_vsync_source; diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.h index 899925aaa6d7..192e338f20bb 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.h @@ -52,14 +52,6 @@ struct split_pipe_cfg { }; /** - * struct cdm_output_cfg: output configuration for cdm - * @intf_en : enable/disable interface output - */ -struct cdm_output_cfg { - bool intf_en; -}; - -/** * struct dpu_danger_safe_status: danger and safe status signals * @mdp: top level status * @sspp: source pipe status @@ -89,7 +81,6 @@ struct dpu_vsync_source_cfg { * Assumption is these functions will be called after clocks are enabled. * @setup_split_pipe : Programs the pipe control registers * @setup_pp_split : Programs the pp split control registers - * @setup_cdm_output : programs cdm control * @setup_traffic_shaper : programs traffic shaper control */ struct dpu_hw_mdp_ops { @@ -102,14 +93,6 @@ struct dpu_hw_mdp_ops { struct split_pipe_cfg *p); /** - * setup_cdm_output() : Setup selection control of the cdm data path - * @mdp : mdp top context driver - * @cfg : cdm output configuration - */ - void (*setup_cdm_output)(struct dpu_hw_mdp *mdp, - struct cdm_output_cfg *cfg); - - /** * setup_traffic_shaper() : Setup traffic shaper control * @mdp : mdp top context driver * @cfg : traffic shaper configuration diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c index 4cabae480a7b..cb5c0170374b 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.c @@ -50,9 +50,6 @@ static u32 dpu_hw_util_log_mask = DPU_DBG_MASK_NONE; #define QSEED3_CLK_CTRL0 0x54 #define QSEED3_CLK_CTRL1 0x58 #define QSEED3_CLK_STATUS 0x5C -#define QSEED3_MISR_CTRL 0x70 -#define QSEED3_MISR_SIGNATURE_0 0x74 -#define QSEED3_MISR_SIGNATURE_1 0x78 #define QSEED3_PHASE_INIT_Y_H 0x90 #define QSEED3_PHASE_INIT_Y_V 0x94 #define QSEED3_PHASE_INIT_UV_H 0x98 diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h index 1240f505ca53..321fc64ddd0e 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_hw_util.h @@ -148,16 +148,6 @@ struct dpu_hw_scaler3_cfg { struct dpu_hw_scaler3_de_cfg de; }; -struct dpu_hw_scaler3_lut_cfg { - bool is_configured; - u32 *dir_lut; - size_t dir_len; - u32 *cir_lut; - size_t cir_len; - u32 *sep_lut; - size_t sep_len; -}; - /** * struct dpu_drm_pix_ext_v1 - version 1 of pixel ext structure * @num_ext_pxls_lr: Number of total horizontal pixels @@ -325,12 +315,6 @@ int dpu_reg_read(struct dpu_hw_blk_reg_map *c, u32 reg_off); #define DPU_REG_WRITE(c, off, val) dpu_reg_write(c, off, val, #off) #define DPU_REG_READ(c, off) dpu_reg_read(c, off) -#define MISR_FRAME_COUNT_MASK 0xFF -#define MISR_CTRL_ENABLE BIT(8) -#define MISR_CTRL_STATUS BIT(9) -#define MISR_CTRL_STATUS_CLEAR BIT(10) -#define INTF_MISR_CTRL_FREE_RUN_MASK BIT(31) - void *dpu_hw_util_get_dir(void); void dpu_hw_setup_scaler3(struct dpu_hw_blk_reg_map *c, diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c index 7dd6bd2d6d37..0a683e65a9f3 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c @@ -450,7 +450,7 @@ static void _dpu_kms_initialize_dsi(struct drm_device *dev, int i, rc; /*TODO: Support two independent DSI connectors */ - encoder = dpu_encoder_init(dev, DRM_MODE_CONNECTOR_DSI); + encoder = dpu_encoder_init(dev, DRM_MODE_ENCODER_DSI); if (IS_ERR_OR_NULL(encoder)) { DPU_ERROR("encoder init failed for dsi display\n"); return; @@ -531,12 +531,13 @@ static int _dpu_kms_drm_obj_init(struct dpu_kms *dpu_kms) { struct drm_device *dev; struct drm_plane *primary_planes[MAX_PLANES], *plane; + struct drm_plane *cursor_planes[MAX_PLANES] = { NULL }; struct drm_crtc *crtc; struct msm_drm_private *priv; struct dpu_mdss_cfg *catalog; - int primary_planes_idx = 0, i, ret; + int primary_planes_idx = 0, cursor_planes_idx = 0, i, ret; int max_crtc_count; if (!dpu_kms || !dpu_kms->dev || !dpu_kms->dev->dev) { @@ -556,16 +557,24 @@ static int _dpu_kms_drm_obj_init(struct dpu_kms *dpu_kms) max_crtc_count = min(catalog->mixer_count, priv->num_encoders); - /* Create the planes */ + /* Create the planes, keeping track of one primary/cursor per crtc */ for (i = 0; i < catalog->sspp_count; i++) { - bool primary = true; - - if (catalog->sspp[i].features & BIT(DPU_SSPP_CURSOR) - || primary_planes_idx >= max_crtc_count) - primary = false; - - plane = dpu_plane_init(dev, catalog->sspp[i].id, primary, - (1UL << max_crtc_count) - 1, 0); + enum drm_plane_type type; + + if ((catalog->sspp[i].features & BIT(DPU_SSPP_CURSOR)) + && cursor_planes_idx < max_crtc_count) + type = DRM_PLANE_TYPE_CURSOR; + else if (primary_planes_idx < max_crtc_count) + type = DRM_PLANE_TYPE_PRIMARY; + else + type = DRM_PLANE_TYPE_OVERLAY; + + DPU_DEBUG("Create plane type %d with features %lx (cur %lx)\n", + type, catalog->sspp[i].features, + catalog->sspp[i].features & BIT(DPU_SSPP_CURSOR)); + + plane = dpu_plane_init(dev, catalog->sspp[i].id, type, + (1UL << max_crtc_count) - 1, 0); if (IS_ERR(plane)) { DPU_ERROR("dpu_plane_init failed\n"); ret = PTR_ERR(plane); @@ -573,7 +582,9 @@ static int _dpu_kms_drm_obj_init(struct dpu_kms *dpu_kms) } priv->planes[priv->num_planes++] = plane; - if (primary) + if (type == DRM_PLANE_TYPE_CURSOR) + cursor_planes[cursor_planes_idx++] = plane; + else if (type == DRM_PLANE_TYPE_PRIMARY) primary_planes[primary_planes_idx++] = plane; } @@ -581,7 +592,7 @@ static int _dpu_kms_drm_obj_init(struct dpu_kms *dpu_kms) /* Create one CRTC per encoder */ for (i = 0; i < max_crtc_count; i++) { - crtc = dpu_crtc_init(dev, primary_planes[i]); + crtc = dpu_crtc_init(dev, primary_planes[i], cursor_planes[i]); if (IS_ERR(crtc)) { ret = PTR_ERR(crtc); goto fail; @@ -956,8 +967,7 @@ static void dpu_kms_handle_power_event(u32 event_type, void *usr) if (!dpu_kms) return; - if (event_type == DPU_POWER_EVENT_POST_ENABLE) - dpu_vbif_init_memtypes(dpu_kms); + dpu_vbif_init_memtypes(dpu_kms); } static int dpu_kms_hw_init(struct msm_kms *kms) @@ -1144,10 +1154,9 @@ static int dpu_kms_hw_init(struct msm_kms *kms) /* * Handle (re)initializations during power enable */ - dpu_kms_handle_power_event(DPU_POWER_EVENT_POST_ENABLE, dpu_kms); + dpu_kms_handle_power_event(DPU_POWER_EVENT_ENABLE, dpu_kms); dpu_kms->power_event = dpu_power_handle_register_event( - &dpu_kms->phandle, - DPU_POWER_EVENT_POST_ENABLE, + &dpu_kms->phandle, DPU_POWER_EVENT_ENABLE, dpu_kms_handle_power_event, dpu_kms, "kms"); pm_runtime_put_sync(&dpu_kms->pdev->dev); diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c index 9e533b86682c..2235ef8129f4 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c @@ -158,6 +158,8 @@ static void dpu_mdss_destroy(struct drm_device *dev) _dpu_mdss_irq_domain_fini(dpu_mdss); + free_irq(platform_get_irq(pdev, 0), dpu_mdss); + msm_dss_put_clk(mp->clk_config, mp->num_clk); devm_kfree(&pdev->dev, mp->clk_config); @@ -215,7 +217,7 @@ int dpu_mdss_init(struct drm_device *dev) if (ret) goto irq_domain_error; - ret = devm_request_irq(dev->dev, platform_get_irq(pdev, 0), + ret = request_irq(platform_get_irq(pdev, 0), dpu_mdss_irq, 0, "dpu_mdss_isr", dpu_mdss); if (ret) { DPU_ERROR("failed to init irq: %d\n", ret); diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c index 015341e2dd4c..f549daf30fe6 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c @@ -125,26 +125,11 @@ struct dpu_plane { static struct dpu_kms *_dpu_plane_get_kms(struct drm_plane *plane) { - struct msm_drm_private *priv; + struct msm_drm_private *priv = plane->dev->dev_private; - if (!plane || !plane->dev) - return NULL; - priv = plane->dev->dev_private; - if (!priv) - return NULL; return to_dpu_kms(priv->kms); } -static bool dpu_plane_enabled(struct drm_plane_state *state) -{ - return state && state->fb && state->crtc; -} - -static bool dpu_plane_sspp_enabled(struct drm_plane_state *state) -{ - return state && state->crtc; -} - /** * _dpu_plane_calc_fill_level - calculate fill level of the given source format * @plane: Pointer to drm plane @@ -160,7 +145,7 @@ static inline int _dpu_plane_calc_fill_level(struct drm_plane *plane, u32 fixed_buff_size; u32 total_fl; - if (!plane || !fmt || !plane->state || !src_width || !fmt->bpp) { + if (!fmt || !plane->state || !src_width || !fmt->bpp) { DPU_ERROR("invalid arguments\n"); return 0; } @@ -170,7 +155,7 @@ static inline int _dpu_plane_calc_fill_level(struct drm_plane *plane, fixed_buff_size = pdpu->pipe_sblk->common->pixel_ram_size; list_for_each_entry(tmp, &pdpu->mplane_list, mplane_list) { - if (!dpu_plane_enabled(tmp->base.state)) + if (!tmp->base.state->visible) continue; DPU_DEBUG("plane%d/%d src_width:%d/%d\n", pdpu->base.base.id, tmp->base.base.id, @@ -241,26 +226,11 @@ static u64 _dpu_plane_get_qos_lut(const struct dpu_qos_lut_tbl *tbl, static void _dpu_plane_set_qos_lut(struct drm_plane *plane, struct drm_framebuffer *fb) { - struct dpu_plane *pdpu; + struct dpu_plane *pdpu = to_dpu_plane(plane); const struct dpu_format *fmt = NULL; u64 qos_lut; u32 total_fl = 0, lut_usage; - if (!plane || !fb) { - DPU_ERROR("invalid arguments plane %d fb %d\n", - plane != 0, fb != 0); - return; - } - - pdpu = to_dpu_plane(plane); - - if (!pdpu->pipe_hw || !pdpu->pipe_sblk || !pdpu->catalog) { - DPU_ERROR("invalid arguments\n"); - return; - } else if (!pdpu->pipe_hw->ops.setup_creq_lut) { - return; - } - if (!pdpu->is_rt_pipe) { lut_usage = DPU_QOS_LUT_USAGE_NRT; } else { @@ -302,24 +272,10 @@ static void _dpu_plane_set_qos_lut(struct drm_plane *plane, static void _dpu_plane_set_danger_lut(struct drm_plane *plane, struct drm_framebuffer *fb) { - struct dpu_plane *pdpu; + struct dpu_plane *pdpu = to_dpu_plane(plane); const struct dpu_format *fmt = NULL; u32 danger_lut, safe_lut; - if (!plane || !fb) { - DPU_ERROR("invalid arguments\n"); - return; - } - - pdpu = to_dpu_plane(plane); - - if (!pdpu->pipe_hw || !pdpu->pipe_sblk || !pdpu->catalog) { - DPU_ERROR("invalid arguments\n"); - return; - } else if (!pdpu->pipe_hw->ops.setup_danger_safe_lut) { - return; - } - if (!pdpu->is_rt_pipe) { danger_lut = pdpu->catalog->perf.danger_lut_tbl [DPU_QOS_LUT_USAGE_NRT]; @@ -373,21 +329,7 @@ static void _dpu_plane_set_danger_lut(struct drm_plane *plane, static void _dpu_plane_set_qos_ctrl(struct drm_plane *plane, bool enable, u32 flags) { - struct dpu_plane *pdpu; - - if (!plane) { - DPU_ERROR("invalid arguments\n"); - return; - } - - pdpu = to_dpu_plane(plane); - - if (!pdpu->pipe_hw || !pdpu->pipe_sblk) { - DPU_ERROR("invalid arguments\n"); - return; - } else if (!pdpu->pipe_hw->ops.setup_qos_ctrl) { - return; - } + struct dpu_plane *pdpu = to_dpu_plane(plane); if (flags & DPU_PLANE_QOS_VBLANK_CTRL) { pdpu->pipe_qos_cfg.creq_vblank = pdpu->pipe_sblk->creq_vblank; @@ -423,35 +365,17 @@ static void _dpu_plane_set_qos_ctrl(struct drm_plane *plane, &pdpu->pipe_qos_cfg); } -int dpu_plane_danger_signal_ctrl(struct drm_plane *plane, bool enable) +static void dpu_plane_danger_signal_ctrl(struct drm_plane *plane, bool enable) { - struct dpu_plane *pdpu; - struct msm_drm_private *priv; - struct dpu_kms *dpu_kms; - - if (!plane || !plane->dev) { - DPU_ERROR("invalid arguments\n"); - return -EINVAL; - } - - priv = plane->dev->dev_private; - if (!priv || !priv->kms) { - DPU_ERROR("invalid KMS reference\n"); - return -EINVAL; - } - - dpu_kms = to_dpu_kms(priv->kms); - pdpu = to_dpu_plane(plane); + struct dpu_plane *pdpu = to_dpu_plane(plane); + struct dpu_kms *dpu_kms = _dpu_plane_get_kms(plane); if (!pdpu->is_rt_pipe) - goto end; + return; pm_runtime_get_sync(&dpu_kms->pdev->dev); _dpu_plane_set_qos_ctrl(plane, enable, DPU_PLANE_QOS_PANIC_CTRL); pm_runtime_put_sync(&dpu_kms->pdev->dev); - -end: - return 0; } /** @@ -462,29 +386,9 @@ end: static void _dpu_plane_set_ot_limit(struct drm_plane *plane, struct drm_crtc *crtc) { - struct dpu_plane *pdpu; + struct dpu_plane *pdpu = to_dpu_plane(plane); struct dpu_vbif_set_ot_params ot_params; - struct msm_drm_private *priv; - struct dpu_kms *dpu_kms; - - if (!plane || !plane->dev || !crtc) { - DPU_ERROR("invalid arguments plane %d crtc %d\n", - plane != 0, crtc != 0); - return; - } - - priv = plane->dev->dev_private; - if (!priv || !priv->kms) { - DPU_ERROR("invalid KMS reference\n"); - return; - } - - dpu_kms = to_dpu_kms(priv->kms); - pdpu = to_dpu_plane(plane); - if (!pdpu->pipe_hw) { - DPU_ERROR("invalid pipe reference\n"); - return; - } + struct dpu_kms *dpu_kms = _dpu_plane_get_kms(plane); memset(&ot_params, 0, sizeof(ot_params)); ot_params.xin_id = pdpu->pipe_hw->cap->xin_id; @@ -506,28 +410,9 @@ static void _dpu_plane_set_ot_limit(struct drm_plane *plane, */ static void _dpu_plane_set_qos_remap(struct drm_plane *plane) { - struct dpu_plane *pdpu; + struct dpu_plane *pdpu = to_dpu_plane(plane); struct dpu_vbif_set_qos_params qos_params; - struct msm_drm_private *priv; - struct dpu_kms *dpu_kms; - - if (!plane || !plane->dev) { - DPU_ERROR("invalid arguments\n"); - return; - } - - priv = plane->dev->dev_private; - if (!priv || !priv->kms) { - DPU_ERROR("invalid KMS reference\n"); - return; - } - - dpu_kms = to_dpu_kms(priv->kms); - pdpu = to_dpu_plane(plane); - if (!pdpu->pipe_hw) { - DPU_ERROR("invalid pipe reference\n"); - return; - } + struct dpu_kms *dpu_kms = _dpu_plane_get_kms(plane); memset(&qos_params, 0, sizeof(qos_params)); qos_params.vbif_idx = VBIF_RT; @@ -548,27 +433,12 @@ static void _dpu_plane_set_qos_remap(struct drm_plane *plane) /** * _dpu_plane_get_aspace: gets the address space */ -static int _dpu_plane_get_aspace( - struct dpu_plane *pdpu, - struct dpu_plane_state *pstate, - struct msm_gem_address_space **aspace) +static inline struct msm_gem_address_space *_dpu_plane_get_aspace( + struct dpu_plane *pdpu) { - struct dpu_kms *kms; + struct dpu_kms *kms = _dpu_plane_get_kms(&pdpu->base); - if (!pdpu || !pstate || !aspace) { - DPU_ERROR("invalid parameters\n"); - return -EINVAL; - } - - kms = _dpu_plane_get_kms(&pdpu->base); - if (!kms) { - DPU_ERROR("invalid kms\n"); - return -EINVAL; - } - - *aspace = kms->base.aspace; - - return 0; + return kms->base.aspace; } static inline void _dpu_plane_set_scanout(struct drm_plane *plane, @@ -576,29 +446,10 @@ static inline void _dpu_plane_set_scanout(struct drm_plane *plane, struct dpu_hw_pipe_cfg *pipe_cfg, struct drm_framebuffer *fb) { - struct dpu_plane *pdpu; - struct msm_gem_address_space *aspace = NULL; + struct dpu_plane *pdpu = to_dpu_plane(plane); + struct msm_gem_address_space *aspace = _dpu_plane_get_aspace(pdpu); int ret; - if (!plane || !pstate || !pipe_cfg || !fb) { - DPU_ERROR( - "invalid arg(s), plane %d state %d cfg %d fb %d\n", - plane != 0, pstate != 0, pipe_cfg != 0, fb != 0); - return; - } - - pdpu = to_dpu_plane(plane); - if (!pdpu->pipe_hw) { - DPU_ERROR_PLANE(pdpu, "invalid pipe_hw\n"); - return; - } - - ret = _dpu_plane_get_aspace(pdpu, pstate, &aspace); - if (ret) { - DPU_ERROR_PLANE(pdpu, "Failed to get aspace %d\n", ret); - return; - } - ret = dpu_format_populate_layout(aspace, fb, &pipe_cfg->layout); if (ret == -EAGAIN) DPU_DEBUG_PLANE(pdpu, "not updating same src addrs\n"); @@ -622,15 +473,6 @@ static void _dpu_plane_setup_scaler3(struct dpu_plane *pdpu, { uint32_t i; - if (!pdpu || !pstate || !scale_cfg || !fmt || !chroma_subsmpl_h || - !chroma_subsmpl_v) { - DPU_ERROR( - "pdpu %d pstate %d scale_cfg %d fmt %d smp_h %d smp_v %d\n", - !!pdpu, !!pstate, !!scale_cfg, !!fmt, chroma_subsmpl_h, - chroma_subsmpl_v); - return; - } - memset(scale_cfg, 0, sizeof(*scale_cfg)); memset(&pstate->pixel_ext, 0, sizeof(struct dpu_hw_pixel_ext)); @@ -734,17 +576,8 @@ static void _dpu_plane_setup_scaler(struct dpu_plane *pdpu, struct dpu_plane_state *pstate, const struct dpu_format *fmt, bool color_fill) { - struct dpu_hw_pixel_ext *pe; uint32_t chroma_subsmpl_h, chroma_subsmpl_v; - if (!pdpu || !fmt || !pstate) { - DPU_ERROR("invalid arg(s), plane %d fmt %d state %d\n", - pdpu != 0, fmt != 0, pstate != 0); - return; - } - - pe = &pstate->pixel_ext; - /* don't chroma subsample if decimating */ chroma_subsmpl_h = drm_format_horz_chroma_subsampling(fmt->base.pixel_format); @@ -772,21 +605,8 @@ static int _dpu_plane_color_fill(struct dpu_plane *pdpu, uint32_t color, uint32_t alpha) { const struct dpu_format *fmt; - const struct drm_plane *plane; - struct dpu_plane_state *pstate; - - if (!pdpu || !pdpu->base.state) { - DPU_ERROR("invalid plane\n"); - return -EINVAL; - } - - if (!pdpu->pipe_hw) { - DPU_ERROR_PLANE(pdpu, "invalid plane h/w pointer\n"); - return -EINVAL; - } - - plane = &pdpu->base; - pstate = to_dpu_plane_state(plane->state); + const struct drm_plane *plane = &pdpu->base; + struct dpu_plane_state *pstate = to_dpu_plane_state(plane->state); DPU_DEBUG_PLANE(pdpu, "\n"); @@ -837,12 +657,7 @@ static int _dpu_plane_color_fill(struct dpu_plane *pdpu, void dpu_plane_clear_multirect(const struct drm_plane_state *drm_state) { - struct dpu_plane_state *pstate; - - if (!drm_state) - return; - - pstate = to_dpu_plane_state(drm_state); + struct dpu_plane_state *pstate = to_dpu_plane_state(drm_state); pstate->multirect_index = DPU_SSPP_RECT_SOLO; pstate->multirect_mode = DPU_SSPP_MULTIRECT_NONE; @@ -973,15 +788,6 @@ done: void dpu_plane_get_ctl_flush(struct drm_plane *plane, struct dpu_hw_ctl *ctl, u32 *flush_sspp) { - struct dpu_plane_state *pstate; - - if (!plane || !flush_sspp) { - DPU_ERROR("invalid parameters\n"); - return; - } - - pstate = to_dpu_plane_state(plane->state); - *flush_sspp = ctl->ops.get_bitmask_sspp(ctl, dpu_plane_pipe(plane)); } @@ -995,7 +801,7 @@ static int dpu_plane_prepare_fb(struct drm_plane *plane, struct drm_gem_object *obj; struct msm_gem_object *msm_obj; struct dma_fence *fence; - struct msm_gem_address_space *aspace; + struct msm_gem_address_space *aspace = _dpu_plane_get_aspace(pdpu); int ret; if (!new_state->fb) @@ -1003,12 +809,6 @@ static int dpu_plane_prepare_fb(struct drm_plane *plane, DPU_DEBUG_PLANE(pdpu, "FB[%u]\n", fb->base.id); - ret = _dpu_plane_get_aspace(pdpu, pstate, &aspace); - if (ret) { - DPU_ERROR_PLANE(pdpu, "Failed to get aspace\n"); - return ret; - } - /* cache aspace */ pstate->aspace = aspace; @@ -1078,33 +878,30 @@ static bool dpu_plane_validate_src(struct drm_rect *src, drm_rect_equals(fb_rect, src); } -static int dpu_plane_sspp_atomic_check(struct drm_plane *plane, - struct drm_plane_state *state) +static int dpu_plane_atomic_check(struct drm_plane *plane, + struct drm_plane_state *state) { - int ret = 0; - struct dpu_plane *pdpu; - struct dpu_plane_state *pstate; + int ret = 0, min_scale; + struct dpu_plane *pdpu = to_dpu_plane(plane); + const struct drm_crtc_state *crtc_state = NULL; const struct dpu_format *fmt; struct drm_rect src, dst, fb_rect = { 0 }; - uint32_t max_upscale = 1, max_downscale = 1; uint32_t min_src_size, max_linewidth; - int hscale = 1, vscale = 1; - if (!plane || !state) { - DPU_ERROR("invalid arg(s), plane %d state %d\n", - plane != 0, state != 0); - ret = -EINVAL; - goto exit; - } - - pdpu = to_dpu_plane(plane); - pstate = to_dpu_plane_state(state); + if (state->crtc) + crtc_state = drm_atomic_get_new_crtc_state(state->state, + state->crtc); - if (!pdpu->pipe_sblk) { - DPU_ERROR_PLANE(pdpu, "invalid catalog\n"); - ret = -EINVAL; - goto exit; + min_scale = FRAC_16_16(1, pdpu->pipe_sblk->maxdwnscale); + ret = drm_atomic_helper_check_plane_state(state, crtc_state, min_scale, + pdpu->pipe_sblk->maxupscale << 16, + true, true); + if (ret) { + DPU_ERROR_PLANE(pdpu, "Check plane state failed (%d)\n", ret); + return ret; } + if (!state->visible) + return 0; src.x1 = state->src_x >> 16; src.y1 = state->src_y >> 16; @@ -1118,25 +915,6 @@ static int dpu_plane_sspp_atomic_check(struct drm_plane *plane, max_linewidth = pdpu->pipe_sblk->common->maxlinewidth; - if (pdpu->features & DPU_SSPP_SCALER) { - max_downscale = pdpu->pipe_sblk->maxdwnscale; - max_upscale = pdpu->pipe_sblk->maxupscale; - } - if (drm_rect_width(&src) < drm_rect_width(&dst)) - hscale = drm_rect_calc_hscale(&src, &dst, 1, max_upscale); - else - hscale = drm_rect_calc_hscale(&dst, &src, 1, max_downscale); - if (drm_rect_height(&src) < drm_rect_height(&dst)) - vscale = drm_rect_calc_vscale(&src, &dst, 1, max_upscale); - else - vscale = drm_rect_calc_vscale(&dst, &src, 1, max_downscale); - - DPU_DEBUG_PLANE(pdpu, "check %d -> %d\n", - dpu_plane_enabled(plane->state), dpu_plane_enabled(state)); - - if (!dpu_plane_enabled(state)) - goto exit; - fmt = to_dpu_format(msm_framebuffer_format(state->fb)); min_src_size = DPU_FORMAT_IS_YUV(fmt) ? 2 : 1; @@ -1147,13 +925,13 @@ static int dpu_plane_sspp_atomic_check(struct drm_plane *plane, | BIT(DPU_SSPP_CSC_10BIT))))) { DPU_ERROR_PLANE(pdpu, "plane doesn't have scaler/csc for yuv\n"); - ret = -EINVAL; + return -EINVAL; /* check src bounds */ } else if (!dpu_plane_validate_src(&src, &fb_rect, min_src_size)) { DPU_ERROR_PLANE(pdpu, "invalid source " DRM_RECT_FMT "\n", DRM_RECT_ARG(&src)); - ret = -E2BIG; + return -E2BIG; /* valid yuv image */ } else if (DPU_FORMAT_IS_YUV(fmt) && @@ -1162,41 +940,22 @@ static int dpu_plane_sspp_atomic_check(struct drm_plane *plane, drm_rect_height(&src) & 0x1)) { DPU_ERROR_PLANE(pdpu, "invalid yuv source " DRM_RECT_FMT "\n", DRM_RECT_ARG(&src)); - ret = -EINVAL; + return -EINVAL; /* min dst support */ } else if (drm_rect_width(&dst) < 0x1 || drm_rect_height(&dst) < 0x1) { DPU_ERROR_PLANE(pdpu, "invalid dest rect " DRM_RECT_FMT "\n", DRM_RECT_ARG(&dst)); - ret = -EINVAL; + return -EINVAL; /* check decimated source width */ } else if (drm_rect_width(&src) > max_linewidth) { DPU_ERROR_PLANE(pdpu, "invalid src " DRM_RECT_FMT " line:%u\n", DRM_RECT_ARG(&src), max_linewidth); - ret = -E2BIG; - - /* check scaler capability */ - } else if (hscale < 0 || vscale < 0) { - DPU_ERROR_PLANE(pdpu, "invalid scaling requested src=" - DRM_RECT_FMT " dst=" DRM_RECT_FMT "\n", - DRM_RECT_ARG(&src), DRM_RECT_ARG(&dst)); - ret = -E2BIG; + return -E2BIG; } -exit: - return ret; -} - -static int dpu_plane_atomic_check(struct drm_plane *plane, - struct drm_plane_state *state) -{ - if (!state->fb) - return 0; - - DPU_DEBUG_PLANE(to_dpu_plane(plane), "\n"); - - return dpu_plane_sspp_atomic_check(plane, state); + return 0; } void dpu_plane_flush(struct drm_plane *plane) @@ -1245,46 +1004,16 @@ void dpu_plane_set_error(struct drm_plane *plane, bool error) pdpu->is_error = error; } -static int dpu_plane_sspp_atomic_update(struct drm_plane *plane, - struct drm_plane_state *old_state) +static void dpu_plane_sspp_atomic_update(struct drm_plane *plane) { - uint32_t nplanes, src_flags; - struct dpu_plane *pdpu; - struct drm_plane_state *state; - struct dpu_plane_state *pstate; - struct dpu_plane_state *old_pstate; - const struct dpu_format *fmt; - struct drm_crtc *crtc; - struct drm_framebuffer *fb; - struct drm_rect src, dst; - - if (!plane) { - DPU_ERROR("invalid plane\n"); - return -EINVAL; - } else if (!plane->state) { - DPU_ERROR("invalid plane state\n"); - return -EINVAL; - } else if (!old_state) { - DPU_ERROR("invalid old state\n"); - return -EINVAL; - } - - pdpu = to_dpu_plane(plane); - state = plane->state; - - pstate = to_dpu_plane_state(state); - - old_pstate = to_dpu_plane_state(old_state); - - crtc = state->crtc; - fb = state->fb; - if (!crtc || !fb) { - DPU_ERROR_PLANE(pdpu, "invalid crtc %d or fb %d\n", - crtc != 0, fb != 0); - return -EINVAL; - } - fmt = to_dpu_format(msm_framebuffer_format(fb)); - nplanes = fmt->num_planes; + uint32_t src_flags; + struct dpu_plane *pdpu = to_dpu_plane(plane); + struct drm_plane_state *state = plane->state; + struct dpu_plane_state *pstate = to_dpu_plane_state(state); + struct drm_crtc *crtc = state->crtc; + struct drm_framebuffer *fb = state->fb; + const struct dpu_format *fmt = + to_dpu_format(msm_framebuffer_format(fb)); memset(&(pdpu->pipe_cfg), 0, sizeof(struct dpu_hw_pipe_cfg)); @@ -1295,28 +1024,27 @@ static int dpu_plane_sspp_atomic_update(struct drm_plane *plane, pdpu->is_rt_pipe = (dpu_crtc_get_client_type(crtc) != NRT_CLIENT); _dpu_plane_set_qos_ctrl(plane, false, DPU_PLANE_QOS_PANIC_CTRL); - src.x1 = state->src_x >> 16; - src.y1 = state->src_y >> 16; - src.x2 = src.x1 + (state->src_w >> 16); - src.y2 = src.y1 + (state->src_h >> 16); + DPU_DEBUG_PLANE(pdpu, "FB[%u] " DRM_RECT_FP_FMT "->crtc%u " DRM_RECT_FMT + ", %4.4s ubwc %d\n", fb->base.id, DRM_RECT_FP_ARG(&state->src), + crtc->base.id, DRM_RECT_ARG(&state->dst), + (char *)&fmt->base.pixel_format, DPU_FORMAT_IS_UBWC(fmt)); - dst = drm_plane_state_dest(state); + pdpu->pipe_cfg.src_rect = state->src; - DPU_DEBUG_PLANE(pdpu, "FB[%u] " DRM_RECT_FMT "->crtc%u " DRM_RECT_FMT - ", %4.4s ubwc %d\n", fb->base.id, DRM_RECT_ARG(&src), - crtc->base.id, DRM_RECT_ARG(&dst), - (char *)&fmt->base.pixel_format, - DPU_FORMAT_IS_UBWC(fmt)); + /* state->src is 16.16, src_rect is not */ + pdpu->pipe_cfg.src_rect.x1 >>= 16; + pdpu->pipe_cfg.src_rect.x2 >>= 16; + pdpu->pipe_cfg.src_rect.y1 >>= 16; + pdpu->pipe_cfg.src_rect.y2 >>= 16; - pdpu->pipe_cfg.src_rect = src; - pdpu->pipe_cfg.dst_rect = dst; + pdpu->pipe_cfg.dst_rect = state->dst; _dpu_plane_setup_scaler(pdpu, pstate, fmt, false); /* override for color fill */ if (pdpu->color_fill & DPU_PLANE_COLOR_FILL_FLAG) { /* skip remaining processing on color fill */ - return 0; + return; } if (pdpu->pipe_hw->ops.setup_rects) { @@ -1387,30 +1115,13 @@ static int dpu_plane_sspp_atomic_update(struct drm_plane *plane, } _dpu_plane_set_qos_remap(plane); - return 0; } -static void _dpu_plane_atomic_disable(struct drm_plane *plane, - struct drm_plane_state *old_state) +static void _dpu_plane_atomic_disable(struct drm_plane *plane) { - struct dpu_plane *pdpu; - struct drm_plane_state *state; - struct dpu_plane_state *pstate; - - if (!plane) { - DPU_ERROR("invalid plane\n"); - return; - } else if (!plane->state) { - DPU_ERROR("invalid plane state\n"); - return; - } else if (!old_state) { - DPU_ERROR("invalid old state\n"); - return; - } - - pdpu = to_dpu_plane(plane); - state = plane->state; - pstate = to_dpu_plane_state(state); + struct dpu_plane *pdpu = to_dpu_plane(plane); + struct drm_plane_state *state = plane->state; + struct dpu_plane_state *pstate = to_dpu_plane_state(state); trace_dpu_plane_disable(DRMID(plane), is_dpu_plane_virtual(plane), pstate->multirect_mode); @@ -1426,31 +1137,17 @@ static void _dpu_plane_atomic_disable(struct drm_plane *plane, static void dpu_plane_atomic_update(struct drm_plane *plane, struct drm_plane_state *old_state) { - struct dpu_plane *pdpu; - struct drm_plane_state *state; - - if (!plane) { - DPU_ERROR("invalid plane\n"); - return; - } else if (!plane->state) { - DPU_ERROR("invalid plane state\n"); - return; - } + struct dpu_plane *pdpu = to_dpu_plane(plane); + struct drm_plane_state *state = plane->state; - pdpu = to_dpu_plane(plane); pdpu->is_error = false; - state = plane->state; DPU_DEBUG_PLANE(pdpu, "\n"); - if (!dpu_plane_sspp_enabled(state)) { - _dpu_plane_atomic_disable(plane, old_state); + if (!state->visible) { + _dpu_plane_atomic_disable(plane); } else { - int ret; - - ret = dpu_plane_sspp_atomic_update(plane, old_state); - /* atomic_check should have ensured that this doesn't fail */ - WARN_ON(ret < 0); + dpu_plane_sspp_atomic_update(plane); } } @@ -1487,8 +1184,7 @@ static void dpu_plane_destroy(struct drm_plane *plane) /* this will destroy the states as well */ drm_plane_cleanup(plane); - if (pdpu->pipe_hw) - dpu_hw_sspp_destroy(pdpu->pipe_hw); + dpu_hw_sspp_destroy(pdpu->pipe_hw); kfree(pdpu); } @@ -1507,9 +1203,7 @@ static void dpu_plane_destroy_state(struct drm_plane *plane, pstate = to_dpu_plane_state(state); - /* remove ref count for frame buffers */ - if (state->fb) - drm_framebuffer_put(state->fb); + __drm_atomic_helper_plane_destroy_state(state); kfree(pstate); } @@ -1829,40 +1523,17 @@ bool is_dpu_plane_virtual(struct drm_plane *plane) /* initialize plane */ struct drm_plane *dpu_plane_init(struct drm_device *dev, - uint32_t pipe, bool primary_plane, + uint32_t pipe, enum drm_plane_type type, unsigned long possible_crtcs, u32 master_plane_id) { struct drm_plane *plane = NULL, *master_plane = NULL; const struct dpu_format_extended *format_list; struct dpu_plane *pdpu; - struct msm_drm_private *priv; - struct dpu_kms *kms; - enum drm_plane_type type; + struct msm_drm_private *priv = dev->dev_private; + struct dpu_kms *kms = to_dpu_kms(priv->kms); int zpos_max = DPU_ZPOS_MAX; int ret = -EINVAL; - if (!dev) { - DPU_ERROR("[%u]device is NULL\n", pipe); - goto exit; - } - - priv = dev->dev_private; - if (!priv) { - DPU_ERROR("[%u]private data is NULL\n", pipe); - goto exit; - } - - if (!priv->kms) { - DPU_ERROR("[%u]invalid KMS reference\n", pipe); - goto exit; - } - kms = to_dpu_kms(priv->kms); - - if (!kms->catalog) { - DPU_ERROR("[%u]invalid catalog reference\n", pipe); - goto exit; - } - /* create and zero local structure */ pdpu = kzalloc(sizeof(*pdpu), GFP_KERNEL); if (!pdpu) { @@ -1918,12 +1589,6 @@ struct drm_plane *dpu_plane_init(struct drm_device *dev, goto clean_sspp; } - if (pdpu->features & BIT(DPU_SSPP_CURSOR)) - type = DRM_PLANE_TYPE_CURSOR; - else if (primary_plane) - type = DRM_PLANE_TYPE_PRIMARY; - else - type = DRM_PLANE_TYPE_OVERLAY; ret = drm_universal_plane_init(dev, plane, 0xff, &dpu_plane_funcs, pdpu->formats, pdpu->nformats, NULL, type, NULL); diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h index f6fe6ddc7a3a..7fed0b627708 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h @@ -122,7 +122,7 @@ void dpu_plane_set_error(struct drm_plane *plane, bool error); * dpu_plane_init - create new dpu plane for the given pipe * @dev: Pointer to DRM device * @pipe: dpu hardware pipe identifier - * @primary_plane: true if this pipe is primary plane for crtc + * @type: Plane type - PRIMARY/OVERLAY/CURSOR * @possible_crtcs: bitmask of crtc that can be attached to the given pipe * @master_plane_id: primary plane id of a multirect pipe. 0 value passed for * a regular plane initialization. A non-zero primary plane @@ -130,7 +130,7 @@ void dpu_plane_set_error(struct drm_plane *plane, bool error); * */ struct drm_plane *dpu_plane_init(struct drm_device *dev, - uint32_t pipe, bool primary_plane, + uint32_t pipe, enum drm_plane_type type, unsigned long possible_crtcs, u32 master_plane_id); /** diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.c index a75eebca2f37..fc14116789f2 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.c @@ -145,6 +145,7 @@ int dpu_power_resource_enable(struct dpu_power_handle *phandle, bool changed = false; u32 max_usecase_ndx = VOTE_INDEX_DISABLE, prev_usecase_ndx; struct dpu_power_client *client; + u32 event_type; if (!phandle || !pclient) { pr_err("invalid input argument\n"); @@ -181,19 +182,9 @@ int dpu_power_resource_enable(struct dpu_power_handle *phandle, if (!changed) goto end; - if (enable) { - dpu_power_event_trigger_locked(phandle, - DPU_POWER_EVENT_PRE_ENABLE); - dpu_power_event_trigger_locked(phandle, - DPU_POWER_EVENT_POST_ENABLE); - - } else { - dpu_power_event_trigger_locked(phandle, - DPU_POWER_EVENT_PRE_DISABLE); - dpu_power_event_trigger_locked(phandle, - DPU_POWER_EVENT_POST_DISABLE); - } + event_type = enable ? DPU_POWER_EVENT_ENABLE : DPU_POWER_EVENT_DISABLE; + dpu_power_event_trigger_locked(phandle, event_type); end: mutex_unlock(&phandle->phandle_lock); return 0; diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.h index 344f74464eca..a65b7a297f21 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_power_handle.h @@ -23,17 +23,9 @@ #include "dpu_io_util.h" -/* event will be triggered before power handler disable */ -#define DPU_POWER_EVENT_PRE_DISABLE 0x1 - -/* event will be triggered after power handler disable */ -#define DPU_POWER_EVENT_POST_DISABLE 0x2 - -/* event will be triggered before power handler enable */ -#define DPU_POWER_EVENT_PRE_ENABLE 0x4 - -/* event will be triggered after power handler enable */ -#define DPU_POWER_EVENT_POST_ENABLE 0x8 +/* events will be triggered on power handler enable/disable */ +#define DPU_POWER_EVENT_DISABLE BIT(0) +#define DPU_POWER_EVENT_ENABLE BIT(1) /** * mdss_bus_vote_type: register bus vote type diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.c b/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.c index 13c0a36d4ef9..bdb117709674 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.c +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.c @@ -16,7 +16,6 @@ #include "dpu_kms.h" #include "dpu_hw_lm.h" #include "dpu_hw_ctl.h" -#include "dpu_hw_cdm.h" #include "dpu_hw_pingpong.h" #include "dpu_hw_intf.h" #include "dpu_encoder.h" @@ -25,38 +24,13 @@ #define RESERVED_BY_OTHER(h, r) \ ((h)->rsvp && ((h)->rsvp->enc_id != (r)->enc_id)) -#define RM_RQ_LOCK(r) ((r)->top_ctrl & BIT(DPU_RM_TOPCTL_RESERVE_LOCK)) -#define RM_RQ_CLEAR(r) ((r)->top_ctrl & BIT(DPU_RM_TOPCTL_RESERVE_CLEAR)) -#define RM_RQ_DS(r) ((r)->top_ctrl & BIT(DPU_RM_TOPCTL_DS)) -#define RM_IS_TOPOLOGY_MATCH(t, r) ((t).num_lm == (r).num_lm && \ - (t).num_comp_enc == (r).num_enc && \ - (t).num_intf == (r).num_intf) - -struct dpu_rm_topology_def { - enum dpu_rm_topology_name top_name; - int num_lm; - int num_comp_enc; - int num_intf; - int num_ctl; - int needs_split_display; -}; - -static const struct dpu_rm_topology_def g_top_table[] = { - { DPU_RM_TOPOLOGY_NONE, 0, 0, 0, 0, false }, - { DPU_RM_TOPOLOGY_SINGLEPIPE, 1, 0, 1, 1, false }, - { DPU_RM_TOPOLOGY_DUALPIPE, 2, 0, 2, 2, true }, - { DPU_RM_TOPOLOGY_DUALPIPE_3DMERGE, 2, 0, 1, 1, false }, -}; - /** * struct dpu_rm_requirements - Reservation requirements parameter bundle - * @top_ctrl: topology control preference from kernel client - * @top: selected topology for the display + * @topology: selected topology for the display * @hw_res: Hardware resources required as reported by the encoders */ struct dpu_rm_requirements { - uint64_t top_ctrl; - const struct dpu_rm_topology_def *topology; + struct msm_display_topology topology; struct dpu_encoder_hw_resources hw_res; }; @@ -72,13 +46,11 @@ struct dpu_rm_requirements { * @enc_id: Reservations are tracked by Encoder DRM object ID. * CRTCs may be connected to multiple Encoders. * An encoder or connector id identifies the display path. - * @topology DRM<->HW topology use case */ struct dpu_rm_rsvp { struct list_head list; uint32_t seq; uint32_t enc_id; - enum dpu_rm_topology_name topology; }; /** @@ -122,8 +94,8 @@ static void _dpu_rm_print_rsvps( DPU_DEBUG("%d\n", stage); list_for_each_entry(rsvp, &rm->rsvps, list) { - DRM_DEBUG_KMS("%d rsvp[s%ue%u] topology %d\n", stage, rsvp->seq, - rsvp->enc_id, rsvp->topology); + DRM_DEBUG_KMS("%d rsvp[s%ue%u]\n", stage, rsvp->seq, + rsvp->enc_id); } for (type = 0; type < DPU_HW_BLK_MAX; type++) { @@ -146,18 +118,6 @@ struct dpu_hw_mdp *dpu_rm_get_mdp(struct dpu_rm *rm) return rm->hw_mdp; } -enum dpu_rm_topology_name -dpu_rm_get_topology_name(struct msm_display_topology topology) -{ - int i; - - for (i = 0; i < DPU_RM_TOPOLOGY_MAX; i++) - if (RM_IS_TOPOLOGY_MATCH(g_top_table[i], topology)) - return g_top_table[i].top_name; - - return DPU_RM_TOPOLOGY_NONE; -} - void dpu_rm_init_hw_iter( struct dpu_rm_hw_iter *iter, uint32_t enc_id, @@ -229,9 +189,6 @@ static void _dpu_rm_hw_destroy(enum dpu_hw_blk_type type, void *hw) case DPU_HW_BLK_CTL: dpu_hw_ctl_destroy(hw); break; - case DPU_HW_BLK_CDM: - dpu_hw_cdm_destroy(hw); - break; case DPU_HW_BLK_PINGPONG: dpu_hw_pingpong_destroy(hw); break; @@ -305,9 +262,6 @@ static int _dpu_rm_hw_blk_create( case DPU_HW_BLK_CTL: hw = dpu_hw_ctl_init(id, mmio, cat); break; - case DPU_HW_BLK_CDM: - hw = dpu_hw_cdm_init(id, mmio, cat, hw_mdp); - break; case DPU_HW_BLK_PINGPONG: hw = dpu_hw_pingpong_init(id, mmio, cat); break; @@ -438,15 +392,6 @@ int dpu_rm_init(struct dpu_rm *rm, } } - for (i = 0; i < cat->cdm_count; i++) { - rc = _dpu_rm_hw_blk_create(rm, cat, mmio, DPU_HW_BLK_CDM, - cat->cdm[i].id, &cat->cdm[i]); - if (rc) { - DPU_ERROR("failed: cdm hw not available\n"); - goto fail; - } - } - return 0; fail: @@ -455,6 +400,11 @@ fail: return rc; } +static bool _dpu_rm_needs_split_display(const struct msm_display_topology *top) +{ + return top->num_intf > 1; +} + /** * _dpu_rm_check_lm_and_get_connected_blks - check if proposed layer mixer meets * proposed use case requirements, incl. hardwired dependent blocks like @@ -538,14 +488,14 @@ static int _dpu_rm_reserve_lms( int lm_count = 0; int i, rc = 0; - if (!reqs->topology->num_lm) { - DPU_ERROR("invalid number of lm: %d\n", reqs->topology->num_lm); + if (!reqs->topology.num_lm) { + DPU_ERROR("invalid number of lm: %d\n", reqs->topology.num_lm); return -EINVAL; } /* Find a primary mixer */ dpu_rm_init_hw_iter(&iter_i, 0, DPU_HW_BLK_LM); - while (lm_count != reqs->topology->num_lm && + while (lm_count != reqs->topology.num_lm && _dpu_rm_get_hw_locked(rm, &iter_i)) { memset(&lm, 0, sizeof(lm)); memset(&pp, 0, sizeof(pp)); @@ -563,7 +513,7 @@ static int _dpu_rm_reserve_lms( /* Valid primary mixer found, find matching peers */ dpu_rm_init_hw_iter(&iter_j, 0, DPU_HW_BLK_LM); - while (lm_count != reqs->topology->num_lm && + while (lm_count != reqs->topology.num_lm && _dpu_rm_get_hw_locked(rm, &iter_j)) { if (iter_i.blk == iter_j.blk) continue; @@ -578,7 +528,7 @@ static int _dpu_rm_reserve_lms( } } - if (lm_count != reqs->topology->num_lm) { + if (lm_count != reqs->topology.num_lm) { DPU_DEBUG("unable to find appropriate mixers\n"); return -ENAVAIL; } @@ -600,14 +550,20 @@ static int _dpu_rm_reserve_lms( static int _dpu_rm_reserve_ctls( struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp, - const struct dpu_rm_topology_def *top) + const struct msm_display_topology *top) { struct dpu_rm_hw_blk *ctls[MAX_BLOCKS]; struct dpu_rm_hw_iter iter; - int i = 0; + int i = 0, num_ctls = 0; + bool needs_split_display = false; memset(&ctls, 0, sizeof(ctls)); + /* each hw_intf needs its own hw_ctrl to program its control path */ + num_ctls = top->num_intf; + + needs_split_display = _dpu_rm_needs_split_display(top); + dpu_rm_init_hw_iter(&iter, 0, DPU_HW_BLK_CTL); while (_dpu_rm_get_hw_locked(rm, &iter)) { const struct dpu_hw_ctl *ctl = to_dpu_hw_ctl(iter.blk->hw); @@ -621,20 +577,20 @@ static int _dpu_rm_reserve_ctls( DPU_DEBUG("ctl %d caps 0x%lX\n", iter.blk->id, features); - if (top->needs_split_display != has_split_display) + if (needs_split_display != has_split_display) continue; ctls[i] = iter.blk; DPU_DEBUG("ctl %d match\n", iter.blk->id); - if (++i == top->num_ctl) + if (++i == num_ctls) break; } - if (i != top->num_ctl) + if (i != num_ctls) return -ENAVAIL; - for (i = 0; i < ARRAY_SIZE(ctls) && i < top->num_ctl; i++) { + for (i = 0; i < ARRAY_SIZE(ctls) && i < num_ctls; i++) { ctls[i]->rsvp_nxt = rsvp; trace_dpu_rm_reserve_ctls(ctls[i]->id, ctls[i]->type, rsvp->enc_id); @@ -643,55 +599,11 @@ static int _dpu_rm_reserve_ctls( return 0; } -static int _dpu_rm_reserve_cdm( - struct dpu_rm *rm, - struct dpu_rm_rsvp *rsvp, - uint32_t id, - enum dpu_hw_blk_type type) -{ - struct dpu_rm_hw_iter iter; - - DRM_DEBUG_KMS("type %d id %d\n", type, id); - - dpu_rm_init_hw_iter(&iter, 0, DPU_HW_BLK_CDM); - while (_dpu_rm_get_hw_locked(rm, &iter)) { - const struct dpu_hw_cdm *cdm = to_dpu_hw_cdm(iter.blk->hw); - const struct dpu_cdm_cfg *caps = cdm->caps; - bool match = false; - - if (RESERVED_BY_OTHER(iter.blk, rsvp)) - continue; - - if (type == DPU_HW_BLK_INTF && id != INTF_MAX) - match = test_bit(id, &caps->intf_connect); - - DRM_DEBUG_KMS("iter: type:%d id:%d enc:%d cdm:%lu match:%d\n", - iter.blk->type, iter.blk->id, rsvp->enc_id, - caps->intf_connect, match); - - if (!match) - continue; - - trace_dpu_rm_reserve_cdm(iter.blk->id, iter.blk->type, - rsvp->enc_id); - iter.blk->rsvp_nxt = rsvp; - break; - } - - if (!iter.hw) { - DPU_ERROR("couldn't reserve cdm for type %d id %d\n", type, id); - return -ENAVAIL; - } - - return 0; -} - static int _dpu_rm_reserve_intf( struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp, uint32_t id, - enum dpu_hw_blk_type type, - bool needs_cdm) + enum dpu_hw_blk_type type) { struct dpu_rm_hw_iter iter; int ret = 0; @@ -719,9 +631,6 @@ static int _dpu_rm_reserve_intf( return -EINVAL; } - if (needs_cdm) - ret = _dpu_rm_reserve_cdm(rm, rsvp, id, type); - return ret; } @@ -738,7 +647,7 @@ static int _dpu_rm_reserve_intf_related_hw( continue; id = i + INTF_0; ret = _dpu_rm_reserve_intf(rm, rsvp, id, - DPU_HW_BLK_INTF, hw_res->needs_cdm); + DPU_HW_BLK_INTF); if (ret) return ret; } @@ -750,17 +659,14 @@ static int _dpu_rm_make_next_rsvp( struct dpu_rm *rm, struct drm_encoder *enc, struct drm_crtc_state *crtc_state, - struct drm_connector_state *conn_state, struct dpu_rm_rsvp *rsvp, struct dpu_rm_requirements *reqs) { int ret; - struct dpu_rm_topology_def topology; /* Create reservation info, tag reserved blocks with it as we go */ rsvp->seq = ++rm->rsvp_next_seq; rsvp->enc_id = enc->base.id; - rsvp->topology = reqs->topology->top_name; list_add_tail(&rsvp->list, &rm->rsvps); ret = _dpu_rm_reserve_lms(rm, rsvp, reqs); @@ -769,23 +675,12 @@ static int _dpu_rm_make_next_rsvp( return ret; } - /* - * Do assignment preferring to give away low-resource CTLs first: - * - Check mixers without Split Display - * - Only then allow to grab from CTLs with split display capability - */ - _dpu_rm_reserve_ctls(rm, rsvp, reqs->topology); - if (ret && !reqs->topology->needs_split_display) { - memcpy(&topology, reqs->topology, sizeof(topology)); - topology.needs_split_display = true; - _dpu_rm_reserve_ctls(rm, rsvp, &topology); - } + ret = _dpu_rm_reserve_ctls(rm, rsvp, &reqs->topology); if (ret) { DPU_ERROR("unable to find appropriate CTL\n"); return ret; } - /* Assign INTFs and blks whose usage is tied to them: CTL & CDM */ ret = _dpu_rm_reserve_intf_related_hw(rm, rsvp, &reqs->hw_res); if (ret) return ret; @@ -797,44 +692,16 @@ static int _dpu_rm_populate_requirements( struct dpu_rm *rm, struct drm_encoder *enc, struct drm_crtc_state *crtc_state, - struct drm_connector_state *conn_state, struct dpu_rm_requirements *reqs, struct msm_display_topology req_topology) { - int i; + dpu_encoder_get_hw_resources(enc, &reqs->hw_res); - memset(reqs, 0, sizeof(*reqs)); + reqs->topology = req_topology; - dpu_encoder_get_hw_resources(enc, &reqs->hw_res, conn_state); - - for (i = 0; i < DPU_RM_TOPOLOGY_MAX; i++) { - if (RM_IS_TOPOLOGY_MATCH(g_top_table[i], - req_topology)) { - reqs->topology = &g_top_table[i]; - break; - } - } - - if (!reqs->topology) { - DPU_ERROR("invalid topology for the display\n"); - return -EINVAL; - } - - /** - * Set the requirement based on caps if not set from user space - * This will ensure to select LM tied with DS blocks - * Currently, DS blocks are tied with LM 0 and LM 1 (primary display) - */ - if (!RM_RQ_DS(reqs) && rm->hw_mdp->caps->has_dest_scaler && - conn_state->connector->connector_type == DRM_MODE_CONNECTOR_DSI) - reqs->top_ctrl |= BIT(DPU_RM_TOPCTL_DS); - - DRM_DEBUG_KMS("top_ctrl: 0x%llX num_h_tiles: %d\n", reqs->top_ctrl, - reqs->hw_res.display_num_of_h_tiles); - DRM_DEBUG_KMS("num_lm: %d num_ctl: %d topology: %d split_display: %d\n", - reqs->topology->num_lm, reqs->topology->num_ctl, - reqs->topology->top_name, - reqs->topology->needs_split_display); + DRM_DEBUG_KMS("num_lm: %d num_enc: %d num_intf: %d\n", + reqs->topology.num_lm, reqs->topology.num_enc, + reqs->topology.num_intf); return 0; } @@ -860,29 +727,12 @@ static struct dpu_rm_rsvp *_dpu_rm_get_rsvp( return NULL; } -static struct drm_connector *_dpu_rm_get_connector( - struct drm_encoder *enc) -{ - struct drm_connector *conn = NULL; - struct list_head *connector_list = - &enc->dev->mode_config.connector_list; - - list_for_each_entry(conn, connector_list, head) - if (conn->encoder == enc) - return conn; - - return NULL; -} - /** * _dpu_rm_release_rsvp - release resources and release a reservation * @rm: KMS handle * @rsvp: RSVP pointer to release and release resources for */ -static void _dpu_rm_release_rsvp( - struct dpu_rm *rm, - struct dpu_rm_rsvp *rsvp, - struct drm_connector *conn) +static void _dpu_rm_release_rsvp(struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp) { struct dpu_rm_rsvp *rsvp_c, *rsvp_n; struct dpu_rm_hw_blk *blk; @@ -923,7 +773,6 @@ static void _dpu_rm_release_rsvp( void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc) { struct dpu_rm_rsvp *rsvp; - struct drm_connector *conn; if (!rm || !enc) { DPU_ERROR("invalid params\n"); @@ -938,25 +787,15 @@ void dpu_rm_release(struct dpu_rm *rm, struct drm_encoder *enc) goto end; } - conn = _dpu_rm_get_connector(enc); - if (!conn) { - DPU_ERROR("failed to get connector for enc %d\n", enc->base.id); - goto end; - } - - _dpu_rm_release_rsvp(rm, rsvp, conn); + _dpu_rm_release_rsvp(rm, rsvp); end: mutex_unlock(&rm->rm_lock); } -static int _dpu_rm_commit_rsvp( - struct dpu_rm *rm, - struct dpu_rm_rsvp *rsvp, - struct drm_connector_state *conn_state) +static void _dpu_rm_commit_rsvp(struct dpu_rm *rm, struct dpu_rm_rsvp *rsvp) { struct dpu_rm_hw_blk *blk; enum dpu_hw_blk_type type; - int ret = 0; /* Swap next rsvp to be the active */ for (type = 0; type < DPU_HW_BLK_MAX; type++) { @@ -967,19 +806,12 @@ static int _dpu_rm_commit_rsvp( } } } - - if (!ret) - DRM_DEBUG_KMS("rsrv enc %d topology %d\n", rsvp->enc_id, - rsvp->topology); - - return ret; } int dpu_rm_reserve( struct dpu_rm *rm, struct drm_encoder *enc, struct drm_crtc_state *crtc_state, - struct drm_connector_state *conn_state, struct msm_display_topology topology, bool test_only) { @@ -987,25 +819,19 @@ int dpu_rm_reserve( struct dpu_rm_requirements reqs; int ret; - if (!rm || !enc || !crtc_state || !conn_state) { - DPU_ERROR("invalid arguments\n"); - return -EINVAL; - } - /* Check if this is just a page-flip */ if (!drm_atomic_crtc_needs_modeset(crtc_state)) return 0; - DRM_DEBUG_KMS("reserving hw for conn %d enc %d crtc %d test_only %d\n", - conn_state->connector->base.id, enc->base.id, - crtc_state->crtc->base.id, test_only); + DRM_DEBUG_KMS("reserving hw for enc %d crtc %d test_only %d\n", + enc->base.id, crtc_state->crtc->base.id, test_only); mutex_lock(&rm->rm_lock); _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_BEGIN); - ret = _dpu_rm_populate_requirements(rm, enc, crtc_state, - conn_state, &reqs, topology); + ret = _dpu_rm_populate_requirements(rm, enc, crtc_state, &reqs, + topology); if (ret) { DPU_ERROR("failed to populate hw requirements\n"); goto end; @@ -1030,28 +856,15 @@ int dpu_rm_reserve( rsvp_cur = _dpu_rm_get_rsvp(rm, enc); - /* - * User can request that we clear out any reservation during the - * atomic_check phase by using this CLEAR bit - */ - if (rsvp_cur && test_only && RM_RQ_CLEAR(&reqs)) { - DPU_DEBUG("test_only & CLEAR: clear rsvp[s%de%d]\n", - rsvp_cur->seq, rsvp_cur->enc_id); - _dpu_rm_release_rsvp(rm, rsvp_cur, conn_state->connector); - rsvp_cur = NULL; - _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_AFTER_CLEAR); - } - /* Check the proposed reservation, store it in hw's "next" field */ - ret = _dpu_rm_make_next_rsvp(rm, enc, crtc_state, conn_state, - rsvp_nxt, &reqs); + ret = _dpu_rm_make_next_rsvp(rm, enc, crtc_state, rsvp_nxt, &reqs); _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_AFTER_RSVPNEXT); if (ret) { DPU_ERROR("failed to reserve hw resources: %d\n", ret); - _dpu_rm_release_rsvp(rm, rsvp_nxt, conn_state->connector); - } else if (test_only && !RM_RQ_LOCK(&reqs)) { + _dpu_rm_release_rsvp(rm, rsvp_nxt); + } else if (test_only) { /* * Normally, if test_only, test the reservation and then undo * However, if the user requests LOCK, then keep the reservation @@ -1059,15 +872,11 @@ int dpu_rm_reserve( */ DPU_DEBUG("test_only: discard test rsvp[s%de%d]\n", rsvp_nxt->seq, rsvp_nxt->enc_id); - _dpu_rm_release_rsvp(rm, rsvp_nxt, conn_state->connector); + _dpu_rm_release_rsvp(rm, rsvp_nxt); } else { - if (test_only && RM_RQ_LOCK(&reqs)) - DPU_DEBUG("test_only & LOCK: lock rsvp[s%de%d]\n", - rsvp_nxt->seq, rsvp_nxt->enc_id); - - _dpu_rm_release_rsvp(rm, rsvp_cur, conn_state->connector); + _dpu_rm_release_rsvp(rm, rsvp_cur); - ret = _dpu_rm_commit_rsvp(rm, rsvp_nxt, conn_state); + _dpu_rm_commit_rsvp(rm, rsvp_nxt); } _dpu_rm_print_rsvps(rm, DPU_RM_STAGE_FINAL); diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.h index ffd1841a6067..b8273bd23801 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_rm.h @@ -21,39 +21,6 @@ #include "dpu_hw_top.h" /** - * enum dpu_rm_topology_name - HW resource use case in use by connector - * @DPU_RM_TOPOLOGY_NONE: No topology in use currently - * @DPU_RM_TOPOLOGY_SINGLEPIPE: 1 LM, 1 PP, 1 INTF/WB - * @DPU_RM_TOPOLOGY_DUALPIPE: 2 LM, 2 PP, 2 INTF/WB - * @DPU_RM_TOPOLOGY_DUALPIPE_3DMERGE: 2 LM, 2 PP, 3DMux, 1 INTF/WB - */ -enum dpu_rm_topology_name { - DPU_RM_TOPOLOGY_NONE = 0, - DPU_RM_TOPOLOGY_SINGLEPIPE, - DPU_RM_TOPOLOGY_DUALPIPE, - DPU_RM_TOPOLOGY_DUALPIPE_3DMERGE, - DPU_RM_TOPOLOGY_MAX, -}; - -/** - * enum dpu_rm_topology_control - HW resource use case in use by connector - * @DPU_RM_TOPCTL_RESERVE_LOCK: If set, in AtomicTest phase, after a successful - * test, reserve the resources for this display. - * Normal behavior would not impact the reservation - * list during the AtomicTest phase. - * @DPU_RM_TOPCTL_RESERVE_CLEAR: If set, in AtomicTest phase, before testing, - * release any reservation held by this display. - * Normal behavior would not impact the - * reservation list during the AtomicTest phase. - * @DPU_RM_TOPCTL_DS : Require layer mixers with DS capabilities - */ -enum dpu_rm_topology_control { - DPU_RM_TOPCTL_RESERVE_LOCK, - DPU_RM_TOPCTL_RESERVE_CLEAR, - DPU_RM_TOPCTL_DS, -}; - -/** * struct dpu_rm - DPU dynamic hardware resource manager * @dev: device handle for event logging purposes * @rsvps: list of hardware reservations by each crtc->encoder->connector @@ -125,7 +92,6 @@ int dpu_rm_destroy(struct dpu_rm *rm); * @rm: DPU Resource Manager handle * @drm_enc: DRM Encoder handle * @crtc_state: Proposed Atomic DRM CRTC State handle - * @conn_state: Proposed Atomic DRM Connector State handle * @topology: Pointer to topology info for the display * @test_only: Atomic-Test phase, discard results (unless property overrides) * @Return: 0 on Success otherwise -ERROR @@ -133,7 +99,6 @@ int dpu_rm_destroy(struct dpu_rm *rm); int dpu_rm_reserve(struct dpu_rm *rm, struct drm_encoder *drm_enc, struct drm_crtc_state *crtc_state, - struct drm_connector_state *conn_state, struct msm_display_topology topology, bool test_only); @@ -187,13 +152,4 @@ bool dpu_rm_get_hw(struct dpu_rm *rm, struct dpu_rm_hw_iter *iter); */ int dpu_rm_check_property_topctl(uint64_t val); -/** - * dpu_rm_get_topology_name - returns the name of the the given topology - * definition - * @topology: topology definition - * @Return: name of the topology - */ -enum dpu_rm_topology_name -dpu_rm_get_topology_name(struct msm_display_topology topology); - #endif /* __DPU_RM_H__ */ diff --git a/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h b/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h index ae0ca5076238..e12c4cefb742 100644 --- a/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h +++ b/drivers/gpu/drm/msm/disp/dpu1/dpu_trace.h @@ -468,14 +468,16 @@ TRACE_EVENT(dpu_enc_frame_done_cb, TRACE_EVENT(dpu_enc_trigger_flush, TP_PROTO(uint32_t drm_id, enum dpu_intf intf_idx, - int pending_kickoff_cnt, int ctl_idx, u32 pending_flush_ret), + int pending_kickoff_cnt, int ctl_idx, u32 extra_flush_bits, + u32 pending_flush_ret), TP_ARGS(drm_id, intf_idx, pending_kickoff_cnt, ctl_idx, - pending_flush_ret), + extra_flush_bits, pending_flush_ret), TP_STRUCT__entry( __field( uint32_t, drm_id ) __field( enum dpu_intf, intf_idx ) __field( int, pending_kickoff_cnt ) __field( int, ctl_idx ) + __field( u32, extra_flush_bits ) __field( u32, pending_flush_ret ) ), TP_fast_assign( @@ -483,12 +485,14 @@ TRACE_EVENT(dpu_enc_trigger_flush, __entry->intf_idx = intf_idx; __entry->pending_kickoff_cnt = pending_kickoff_cnt; __entry->ctl_idx = ctl_idx; + __entry->extra_flush_bits = extra_flush_bits; __entry->pending_flush_ret = pending_flush_ret; ), TP_printk("id=%u, intf_idx=%d, pending_kickoff_cnt=%d ctl_idx=%d " - "pending_flush_ret=%u", __entry->drm_id, - __entry->intf_idx, __entry->pending_kickoff_cnt, - __entry->ctl_idx, __entry->pending_flush_ret) + "extra_flush_bits=0x%x pending_flush_ret=0x%x", + __entry->drm_id, __entry->intf_idx, + __entry->pending_kickoff_cnt, __entry->ctl_idx, + __entry->extra_flush_bits, __entry->pending_flush_ret) ); DECLARE_EVENT_CLASS(dpu_enc_ktime_template, @@ -682,37 +686,41 @@ TRACE_EVENT(dpu_crtc_setup_mixer, TP_STRUCT__entry( __field( uint32_t, crtc_id ) __field( uint32_t, plane_id ) - __field( struct drm_plane_state*,state ) - __field( struct dpu_plane_state*,pstate ) + __field( uint32_t, fb_id ) + __field_struct( struct drm_rect, src_rect ) + __field_struct( struct drm_rect, dst_rect ) __field( uint32_t, stage_idx ) + __field( enum dpu_stage, stage ) __field( enum dpu_sspp, sspp ) + __field( uint32_t, multirect_idx ) + __field( uint32_t, multirect_mode ) __field( uint32_t, pixel_format ) __field( uint64_t, modifier ) ), TP_fast_assign( __entry->crtc_id = crtc_id; __entry->plane_id = plane_id; - __entry->state = state; - __entry->pstate = pstate; + __entry->fb_id = state ? state->fb->base.id : 0; + __entry->src_rect = drm_plane_state_src(state); + __entry->dst_rect = drm_plane_state_dest(state); __entry->stage_idx = stage_idx; + __entry->stage = pstate->stage; __entry->sspp = sspp; + __entry->multirect_idx = pstate->multirect_index; + __entry->multirect_mode = pstate->multirect_mode; __entry->pixel_format = pixel_format; __entry->modifier = modifier; ), - TP_printk("crtc_id:%u plane_id:%u fb_id:%u src:{%ux%u+%ux%u} " - "dst:{%ux%u+%ux%u} stage_idx:%u stage:%d, sspp:%d " + TP_printk("crtc_id:%u plane_id:%u fb_id:%u src:" DRM_RECT_FP_FMT + " dst:" DRM_RECT_FMT " stage_idx:%u stage:%d, sspp:%d " "multirect_index:%d multirect_mode:%u pix_format:%u " "modifier:%llu", - __entry->crtc_id, __entry->plane_id, - __entry->state->fb ? __entry->state->fb->base.id : -1, - __entry->state->src_w >> 16, __entry->state->src_h >> 16, - __entry->state->src_x >> 16, __entry->state->src_y >> 16, - __entry->state->crtc_w, __entry->state->crtc_h, - __entry->state->crtc_x, __entry->state->crtc_y, - __entry->stage_idx, __entry->pstate->stage, __entry->sspp, - __entry->pstate->multirect_index, - __entry->pstate->multirect_mode, __entry->pixel_format, - __entry->modifier) + __entry->crtc_id, __entry->plane_id, __entry->fb_id, + DRM_RECT_FP_ARG(&__entry->src_rect), + DRM_RECT_ARG(&__entry->dst_rect), + __entry->stage_idx, __entry->stage, __entry->sspp, + __entry->multirect_idx, __entry->multirect_mode, + __entry->pixel_format, __entry->modifier) ); TRACE_EVENT(dpu_crtc_setup_lm_bounds, @@ -721,15 +729,15 @@ TRACE_EVENT(dpu_crtc_setup_lm_bounds, TP_STRUCT__entry( __field( uint32_t, drm_id ) __field( int, mixer ) - __field( struct drm_rect *, bounds ) + __field_struct( struct drm_rect, bounds ) ), TP_fast_assign( __entry->drm_id = drm_id; __entry->mixer = mixer; - __entry->bounds = bounds; + __entry->bounds = *bounds; ), TP_printk("id:%u mixer:%d bounds:" DRM_RECT_FMT, __entry->drm_id, - __entry->mixer, DRM_RECT_ARG(__entry->bounds)) + __entry->mixer, DRM_RECT_ARG(&__entry->bounds)) ); TRACE_EVENT(dpu_crtc_vblank_enable, @@ -740,21 +748,25 @@ TRACE_EVENT(dpu_crtc_vblank_enable, __field( uint32_t, drm_id ) __field( uint32_t, enc_id ) __field( bool, enable ) - __field( struct dpu_crtc *, crtc ) + __field( bool, enabled ) + __field( bool, suspend ) + __field( bool, vblank_requested ) ), TP_fast_assign( __entry->drm_id = drm_id; __entry->enc_id = enc_id; __entry->enable = enable; - __entry->crtc = crtc; + __entry->enabled = crtc->enabled; + __entry->suspend = crtc->suspend; + __entry->vblank_requested = crtc->vblank_requested; ), TP_printk("id:%u encoder:%u enable:%s state{enabled:%s suspend:%s " "vblank_req:%s}", __entry->drm_id, __entry->enc_id, __entry->enable ? "true" : "false", - __entry->crtc->enabled ? "true" : "false", - __entry->crtc->suspend ? "true" : "false", - __entry->crtc->vblank_requested ? "true" : "false") + __entry->enabled ? "true" : "false", + __entry->suspend ? "true" : "false", + __entry->vblank_requested ? "true" : "false") ); DECLARE_EVENT_CLASS(dpu_crtc_enable_template, @@ -763,18 +775,22 @@ DECLARE_EVENT_CLASS(dpu_crtc_enable_template, TP_STRUCT__entry( __field( uint32_t, drm_id ) __field( bool, enable ) - __field( struct dpu_crtc *, crtc ) + __field( bool, enabled ) + __field( bool, suspend ) + __field( bool, vblank_requested ) ), TP_fast_assign( __entry->drm_id = drm_id; __entry->enable = enable; - __entry->crtc = crtc; + __entry->enabled = crtc->enabled; + __entry->suspend = crtc->suspend; + __entry->vblank_requested = crtc->vblank_requested; ), TP_printk("id:%u enable:%s state{enabled:%s suspend:%s vblank_req:%s}", __entry->drm_id, __entry->enable ? "true" : "false", - __entry->crtc->enabled ? "true" : "false", - __entry->crtc->suspend ? "true" : "false", - __entry->crtc->vblank_requested ? "true" : "false") + __entry->enabled ? "true" : "false", + __entry->suspend ? "true" : "false", + __entry->vblank_requested ? "true" : "false") ); DEFINE_EVENT(dpu_crtc_enable_template, dpu_crtc_set_suspend, TP_PROTO(uint32_t drm_id, bool enable, struct dpu_crtc *crtc), @@ -814,24 +830,24 @@ TRACE_EVENT(dpu_plane_set_scanout, TP_ARGS(index, layout, multirect_index), TP_STRUCT__entry( __field( enum dpu_sspp, index ) - __field( struct dpu_hw_fmt_layout*, layout ) + __field_struct( struct dpu_hw_fmt_layout, layout ) __field( enum dpu_sspp_multirect_index, multirect_index) ), TP_fast_assign( __entry->index = index; - __entry->layout = layout; + __entry->layout = *layout; __entry->multirect_index = multirect_index; ), TP_printk("index:%d layout:{%ux%u @ [%u/%u, %u/%u, %u/%u, %u/%u]} " - "multirect_index:%d", __entry->index, __entry->layout->width, - __entry->layout->height, __entry->layout->plane_addr[0], - __entry->layout->plane_size[0], - __entry->layout->plane_addr[1], - __entry->layout->plane_size[1], - __entry->layout->plane_addr[2], - __entry->layout->plane_size[2], - __entry->layout->plane_addr[3], - __entry->layout->plane_size[3], __entry->multirect_index) + "multirect_index:%d", __entry->index, __entry->layout.width, + __entry->layout.height, __entry->layout.plane_addr[0], + __entry->layout.plane_size[0], + __entry->layout.plane_addr[1], + __entry->layout.plane_size[1], + __entry->layout.plane_addr[2], + __entry->layout.plane_size[2], + __entry->layout.plane_addr[3], + __entry->layout.plane_size[3], __entry->multirect_index) ); TRACE_EVENT(dpu_plane_disable, @@ -868,10 +884,6 @@ DECLARE_EVENT_CLASS(dpu_rm_iter_template, TP_printk("id:%d type:%d enc_id:%u", __entry->id, __entry->type, __entry->enc_id) ); -DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_cdm, - TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), - TP_ARGS(id, type, enc_id) -); DEFINE_EVENT(dpu_rm_iter_template, dpu_rm_reserve_intf, TP_PROTO(uint32_t id, enum dpu_hw_blk_type type, uint32_t enc_id), TP_ARGS(id, type, enc_id) @@ -979,16 +991,16 @@ TRACE_EVENT(dpu_core_perf_update_clk, TP_PROTO(struct drm_device *dev, bool stop_req, u64 clk_rate), TP_ARGS(dev, stop_req, clk_rate), TP_STRUCT__entry( - __field( struct drm_device *, dev ) + __string( dev_name, dev->unique ) __field( bool, stop_req ) __field( u64, clk_rate ) ), TP_fast_assign( - __entry->dev = dev; + __assign_str(dev_name, dev->unique); __entry->stop_req = stop_req; __entry->clk_rate = clk_rate; ), - TP_printk("dev:%s stop_req:%s clk_rate:%llu", __entry->dev->unique, + TP_printk("dev:%s stop_req:%s clk_rate:%llu", __get_str(dev_name), __entry->stop_req ? "true" : "false", __entry->clk_rate) ); diff --git a/drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c b/drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c index 7d306c5acd09..7f42c3e68a53 100644 --- a/drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c +++ b/drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c @@ -185,7 +185,7 @@ static void mdp5_plane_reset(struct drm_plane *plane) struct mdp5_plane_state *mdp5_state; if (plane->state && plane->state->fb) - drm_framebuffer_unreference(plane->state->fb); + drm_framebuffer_put(plane->state->fb); kfree(to_mdp5_plane_state(plane->state)); mdp5_state = kzalloc(sizeof(*mdp5_state), GFP_KERNEL); @@ -228,7 +228,7 @@ static void mdp5_plane_destroy_state(struct drm_plane *plane, struct mdp5_plane_state *pstate = to_mdp5_plane_state(state); if (state->fb) - drm_framebuffer_unreference(state->fb); + drm_framebuffer_put(state->fb); kfree(pstate); } @@ -259,7 +259,6 @@ static void mdp5_plane_cleanup_fb(struct drm_plane *plane, msm_framebuffer_cleanup(fb, kms->aspace); } -#define FRAC_16_16(mult, div) (((mult) << 16) / (div)) static int mdp5_plane_atomic_check_with_state(struct drm_crtc_state *crtc_state, struct drm_plane_state *state) { diff --git a/drivers/gpu/drm/msm/dsi/dsi.c b/drivers/gpu/drm/msm/dsi/dsi.c index ff8164cc6738..a9768f823290 100644 --- a/drivers/gpu/drm/msm/dsi/dsi.c +++ b/drivers/gpu/drm/msm/dsi/dsi.c @@ -83,6 +83,7 @@ static struct msm_dsi *dsi_init(struct platform_device *pdev) return ERR_PTR(-ENOMEM); DBG("dsi probed=%p", msm_dsi); + msm_dsi->id = -1; msm_dsi->pdev = pdev; platform_set_drvdata(pdev, msm_dsi); @@ -117,8 +118,13 @@ static int dsi_bind(struct device *dev, struct device *master, void *data) DBG(""); msm_dsi = dsi_init(pdev); - if (IS_ERR(msm_dsi)) - return PTR_ERR(msm_dsi); + if (IS_ERR(msm_dsi)) { + /* Don't fail the bind if the dsi port is not connected */ + if (PTR_ERR(msm_dsi) == -ENODEV) + return 0; + else + return PTR_ERR(msm_dsi); + } priv->dsi[msm_dsi->id] = msm_dsi; diff --git a/drivers/gpu/drm/msm/dsi/dsi_host.c b/drivers/gpu/drm/msm/dsi/dsi_host.c index 96fb5f635314..9c6c523eacdc 100644 --- a/drivers/gpu/drm/msm/dsi/dsi_host.c +++ b/drivers/gpu/drm/msm/dsi/dsi_host.c @@ -1750,6 +1750,7 @@ static int dsi_host_parse_dt(struct msm_dsi_host *msm_host) if (ret) { dev_err(dev, "%s: invalid lane configuration %d\n", __func__, ret); + ret = -EINVAL; goto err; } @@ -1757,6 +1758,7 @@ static int dsi_host_parse_dt(struct msm_dsi_host *msm_host) device_node = of_graph_get_remote_node(np, 1, 0); if (!device_node) { dev_dbg(dev, "%s: no valid device\n", __func__); + ret = -ENODEV; goto err; } diff --git a/drivers/gpu/drm/msm/dsi/dsi_manager.c b/drivers/gpu/drm/msm/dsi/dsi_manager.c index 5224010d90e4..80aa6344185e 100644 --- a/drivers/gpu/drm/msm/dsi/dsi_manager.c +++ b/drivers/gpu/drm/msm/dsi/dsi_manager.c @@ -839,6 +839,8 @@ void msm_dsi_manager_unregister(struct msm_dsi *msm_dsi) if (msm_dsi->host) msm_dsi_host_unregister(msm_dsi->host); - msm_dsim->dsi[msm_dsi->id] = NULL; + + if (msm_dsi->id >= 0) + msm_dsim->dsi[msm_dsi->id] = NULL; } diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index c1abad8a8612..4904d0d41409 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -337,7 +337,7 @@ static int msm_drm_uninit(struct device *dev) mdss->funcs->destroy(ddev); ddev->dev_private = NULL; - drm_dev_unref(ddev); + drm_dev_put(ddev); kfree(priv); @@ -452,7 +452,7 @@ static int msm_drm_init(struct device *dev, struct drm_driver *drv) priv = kzalloc(sizeof(*priv), GFP_KERNEL); if (!priv) { ret = -ENOMEM; - goto err_unref_drm_dev; + goto err_put_drm_dev; } ddev->dev_private = priv; @@ -653,8 +653,8 @@ err_destroy_mdss: mdss->funcs->destroy(ddev); err_free_priv: kfree(priv); -err_unref_drm_dev: - drm_dev_unref(ddev); +err_put_drm_dev: + drm_dev_put(ddev); return ret; } diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h index 8e510d5c758a..9d11f321f5a9 100644 --- a/drivers/gpu/drm/msm/msm_drv.h +++ b/drivers/gpu/drm/msm/msm_drv.h @@ -62,6 +62,8 @@ struct msm_gem_vma; #define MAX_BRIDGES 8 #define MAX_CONNECTORS 8 +#define FRAC_16_16(mult, div) (((mult) << 16) / (div)) + struct msm_file_private { rwlock_t queuelock; struct list_head submitqueues; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index 7bd83e0afa97..7a7923e6220d 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -144,7 +144,7 @@ static int submit_lookup_objects(struct msm_gem_submit *submit, goto out_unlock; } - drm_gem_object_reference(obj); + drm_gem_object_get(obj); submit->bos[i].obj = msm_obj; @@ -396,7 +396,7 @@ static void submit_cleanup(struct msm_gem_submit *submit) struct msm_gem_object *msm_obj = submit->bos[i].obj; submit_unlock_unpin_bo(submit, i, false); list_del_init(&msm_obj->submit_entry); - drm_gem_object_unreference(&msm_obj->base); + drm_gem_object_put(&msm_obj->base); } ww_acquire_fini(&submit->ticket); diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 5e808cfec345..11aac8337066 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -41,7 +41,11 @@ static int msm_devfreq_target(struct device *dev, unsigned long *freq, if (IS_ERR(opp)) return PTR_ERR(opp); - clk_set_rate(gpu->core_clk, *freq); + if (gpu->funcs->gpu_set_freq) + gpu->funcs->gpu_set_freq(gpu, (u64)*freq); + else + clk_set_rate(gpu->core_clk, *freq); + dev_pm_opp_put(opp); return 0; @@ -51,16 +55,14 @@ static int msm_devfreq_get_dev_status(struct device *dev, struct devfreq_dev_status *status) { struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev)); - u64 cycles; - u32 freq = ((u32) status->current_frequency) / 1000000; ktime_t time; - status->current_frequency = (unsigned long) clk_get_rate(gpu->core_clk); - gpu->funcs->gpu_busy(gpu, &cycles); - - status->busy_time = ((u32) (cycles - gpu->devfreq.busy_cycles)) / freq; + if (gpu->funcs->gpu_get_freq) + status->current_frequency = gpu->funcs->gpu_get_freq(gpu); + else + status->current_frequency = clk_get_rate(gpu->core_clk); - gpu->devfreq.busy_cycles = cycles; + status->busy_time = gpu->funcs->gpu_busy(gpu); time = ktime_get(); status->total_time = ktime_us_delta(time, gpu->devfreq.time); @@ -73,7 +75,10 @@ static int msm_devfreq_get_cur_freq(struct device *dev, unsigned long *freq) { struct msm_gpu *gpu = platform_get_drvdata(to_platform_device(dev)); - *freq = (unsigned long) clk_get_rate(gpu->core_clk); + if (gpu->funcs->gpu_get_freq) + *freq = gpu->funcs->gpu_get_freq(gpu); + else + *freq = clk_get_rate(gpu->core_clk); return 0; } @@ -88,7 +93,7 @@ static struct devfreq_dev_profile msm_devfreq_profile = { static void msm_devfreq_init(struct msm_gpu *gpu) { /* We need target support to do devfreq */ - if (!gpu->funcs->gpu_busy || !gpu->core_clk) + if (!gpu->funcs->gpu_busy) return; msm_devfreq_profile.initial_freq = gpu->fast_rate; @@ -105,6 +110,8 @@ static void msm_devfreq_init(struct msm_gpu *gpu) dev_err(&gpu->pdev->dev, "Couldn't initialize GPU devfreq\n"); gpu->devfreq.devfreq = NULL; } + + devfreq_suspend_device(gpu->devfreq.devfreq); } static int enable_pwrrail(struct msm_gpu *gpu) @@ -184,6 +191,14 @@ static int disable_axi(struct msm_gpu *gpu) return 0; } +void msm_gpu_resume_devfreq(struct msm_gpu *gpu) +{ + gpu->devfreq.busy_cycles = 0; + gpu->devfreq.time = ktime_get(); + + devfreq_resume_device(gpu->devfreq.devfreq); +} + int msm_gpu_pm_resume(struct msm_gpu *gpu) { int ret; @@ -202,12 +217,7 @@ int msm_gpu_pm_resume(struct msm_gpu *gpu) if (ret) return ret; - if (gpu->devfreq.devfreq) { - gpu->devfreq.busy_cycles = 0; - gpu->devfreq.time = ktime_get(); - - devfreq_resume_device(gpu->devfreq.devfreq); - } + msm_gpu_resume_devfreq(gpu); gpu->needs_hw_init = true; @@ -220,8 +230,7 @@ int msm_gpu_pm_suspend(struct msm_gpu *gpu) DBG("%s", gpu->name); - if (gpu->devfreq.devfreq) - devfreq_suspend_device(gpu->devfreq.devfreq); + devfreq_suspend_device(gpu->devfreq.devfreq); ret = disable_axi(gpu); if (ret) @@ -367,8 +376,8 @@ static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, msm_gpu_devcoredump_read, msm_gpu_devcoredump_free); } #else -static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, char *comm, - char *cmd) +static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, + struct msm_gem_submit *submit, char *comm, char *cmd) { } #endif diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 9122ee6e55e4..f82bac086666 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -70,9 +70,11 @@ struct msm_gpu_funcs { /* for generation specific debugfs: */ int (*debugfs_init)(struct msm_gpu *gpu, struct drm_minor *minor); #endif - int (*gpu_busy)(struct msm_gpu *gpu, uint64_t *value); + unsigned long (*gpu_busy)(struct msm_gpu *gpu); struct msm_gpu_state *(*gpu_state_get)(struct msm_gpu *gpu); int (*gpu_state_put)(struct msm_gpu_state *state); + unsigned long (*gpu_get_freq)(struct msm_gpu *gpu); + void (*gpu_set_freq)(struct msm_gpu *gpu, unsigned long freq); }; struct msm_gpu { @@ -264,6 +266,7 @@ static inline void gpu_write64(struct msm_gpu *gpu, u32 lo, u32 hi, u64 val) int msm_gpu_pm_suspend(struct msm_gpu *gpu); int msm_gpu_pm_resume(struct msm_gpu *gpu); +void msm_gpu_resume_devfreq(struct msm_gpu *gpu); int msm_gpu_hw_init(struct msm_gpu *gpu); diff --git a/drivers/gpu/drm/msm/msm_rd.c b/drivers/gpu/drm/msm/msm_rd.c index 3aa8a8576abe..cca933458439 100644 --- a/drivers/gpu/drm/msm/msm_rd.c +++ b/drivers/gpu/drm/msm/msm_rd.c @@ -366,7 +366,7 @@ void msm_rd_dump_submit(struct msm_rd_state *rd, struct msm_gem_submit *submit, va_list args; va_start(args, fmt); - n = vsnprintf(msg, sizeof(msg), fmt, args); + n = vscnprintf(msg, sizeof(msg), fmt, args); va_end(args); rd_write_section(rd, RD_CMD, msg, ALIGN(n, 4)); @@ -375,11 +375,11 @@ void msm_rd_dump_submit(struct msm_rd_state *rd, struct msm_gem_submit *submit, rcu_read_lock(); task = pid_task(submit->pid, PIDTYPE_PID); if (task) { - n = snprintf(msg, sizeof(msg), "%.*s/%d: fence=%u", + n = scnprintf(msg, sizeof(msg), "%.*s/%d: fence=%u", TASK_COMM_LEN, task->comm, pid_nr(submit->pid), submit->seqno); } else { - n = snprintf(msg, sizeof(msg), "???/%d: fence=%u", + n = scnprintf(msg, sizeof(msg), "???/%d: fence=%u", pid_nr(submit->pid), submit->seqno); } rcu_read_unlock(); diff --git a/drivers/gpu/drm/mxsfb/mxsfb_crtc.c b/drivers/gpu/drm/mxsfb/mxsfb_crtc.c index 0abe77675b76..24b1f0c1432e 100644 --- a/drivers/gpu/drm/mxsfb/mxsfb_crtc.c +++ b/drivers/gpu/drm/mxsfb/mxsfb_crtc.c @@ -129,7 +129,6 @@ static void mxsfb_enable_controller(struct mxsfb_drm_private *mxsfb) if (mxsfb->clk_disp_axi) clk_prepare_enable(mxsfb->clk_disp_axi); clk_prepare_enable(mxsfb->clk); - mxsfb_enable_axi_clk(mxsfb); /* If it was disabled, re-enable the mode again */ writel(CTRL_DOTCLK_MODE, mxsfb->base + LCDC_CTRL + REG_SET); @@ -159,8 +158,6 @@ static void mxsfb_disable_controller(struct mxsfb_drm_private *mxsfb) reg &= ~VDCTRL4_SYNC_SIGNALS_ON; writel(reg, mxsfb->base + LCDC_VDCTRL4); - mxsfb_disable_axi_clk(mxsfb); - clk_disable_unprepare(mxsfb->clk); if (mxsfb->clk_disp_axi) clk_disable_unprepare(mxsfb->clk_disp_axi); @@ -196,6 +193,21 @@ static int mxsfb_reset_block(void __iomem *reset_addr) return clear_poll_bit(reset_addr, MODULE_CLKGATE); } +static dma_addr_t mxsfb_get_fb_paddr(struct mxsfb_drm_private *mxsfb) +{ + struct drm_framebuffer *fb = mxsfb->pipe.plane.state->fb; + struct drm_gem_cma_object *gem; + + if (!fb) + return 0; + + gem = drm_fb_cma_get_gem_obj(fb, 0); + if (!gem) + return 0; + + return gem->paddr; +} + static void mxsfb_crtc_mode_set_nofb(struct mxsfb_drm_private *mxsfb) { struct drm_display_mode *m = &mxsfb->pipe.crtc.state->adjusted_mode; @@ -208,7 +220,6 @@ static void mxsfb_crtc_mode_set_nofb(struct mxsfb_drm_private *mxsfb) * running. This may lead to shifted pictures (FIFO issue?), so * first stop the controller and drain its FIFOs. */ - mxsfb_enable_axi_clk(mxsfb); /* Mandatory eLCDIF reset as per the Reference Manual */ err = mxsfb_reset_block(mxsfb->base); @@ -269,19 +280,29 @@ static void mxsfb_crtc_mode_set_nofb(struct mxsfb_drm_private *mxsfb) writel(SET_DOTCLK_H_VALID_DATA_CNT(m->hdisplay), mxsfb->base + LCDC_VDCTRL4); - - mxsfb_disable_axi_clk(mxsfb); } void mxsfb_crtc_enable(struct mxsfb_drm_private *mxsfb) { + dma_addr_t paddr; + + mxsfb_enable_axi_clk(mxsfb); mxsfb_crtc_mode_set_nofb(mxsfb); + + /* Write cur_buf as well to avoid an initial corrupt frame */ + paddr = mxsfb_get_fb_paddr(mxsfb); + if (paddr) { + writel(paddr, mxsfb->base + mxsfb->devdata->cur_buf); + writel(paddr, mxsfb->base + mxsfb->devdata->next_buf); + } + mxsfb_enable_controller(mxsfb); } void mxsfb_crtc_disable(struct mxsfb_drm_private *mxsfb) { mxsfb_disable_controller(mxsfb); + mxsfb_disable_axi_clk(mxsfb); } void mxsfb_plane_atomic_update(struct mxsfb_drm_private *mxsfb, @@ -289,12 +310,8 @@ void mxsfb_plane_atomic_update(struct mxsfb_drm_private *mxsfb, { struct drm_simple_display_pipe *pipe = &mxsfb->pipe; struct drm_crtc *crtc = &pipe->crtc; - struct drm_framebuffer *fb = pipe->plane.state->fb; struct drm_pending_vblank_event *event; - struct drm_gem_cma_object *gem; - - if (!crtc) - return; + dma_addr_t paddr; spin_lock_irq(&crtc->dev->event_lock); event = crtc->state->event; @@ -309,12 +326,10 @@ void mxsfb_plane_atomic_update(struct mxsfb_drm_private *mxsfb, } spin_unlock_irq(&crtc->dev->event_lock); - if (!fb) - return; - - gem = drm_fb_cma_get_gem_obj(fb, 0); - - mxsfb_enable_axi_clk(mxsfb); - writel(gem->paddr, mxsfb->base + mxsfb->devdata->next_buf); - mxsfb_disable_axi_clk(mxsfb); + paddr = mxsfb_get_fb_paddr(mxsfb); + if (paddr) { + mxsfb_enable_axi_clk(mxsfb); + writel(paddr, mxsfb->base + mxsfb->devdata->next_buf); + mxsfb_disable_axi_clk(mxsfb); + } } diff --git a/drivers/gpu/drm/mxsfb/mxsfb_drv.c b/drivers/gpu/drm/mxsfb/mxsfb_drv.c index ffe5137ccaf8..2393e6d16ffd 100644 --- a/drivers/gpu/drm/mxsfb/mxsfb_drv.c +++ b/drivers/gpu/drm/mxsfb/mxsfb_drv.c @@ -98,12 +98,18 @@ static const struct drm_mode_config_funcs mxsfb_mode_config_funcs = { .atomic_commit = drm_atomic_helper_commit, }; +static const struct drm_mode_config_helper_funcs mxsfb_mode_config_helpers = { + .atomic_commit_tail = drm_atomic_helper_commit_tail_rpm, +}; + static void mxsfb_pipe_enable(struct drm_simple_display_pipe *pipe, struct drm_crtc_state *crtc_state, struct drm_plane_state *plane_state) { struct mxsfb_drm_private *mxsfb = drm_pipe_to_mxsfb_drm_private(pipe); + struct drm_device *drm = pipe->plane.dev; + pm_runtime_get_sync(drm->dev); drm_panel_prepare(mxsfb->panel); mxsfb_crtc_enable(mxsfb); drm_panel_enable(mxsfb->panel); @@ -112,10 +118,22 @@ static void mxsfb_pipe_enable(struct drm_simple_display_pipe *pipe, static void mxsfb_pipe_disable(struct drm_simple_display_pipe *pipe) { struct mxsfb_drm_private *mxsfb = drm_pipe_to_mxsfb_drm_private(pipe); + struct drm_device *drm = pipe->plane.dev; + struct drm_crtc *crtc = &pipe->crtc; + struct drm_pending_vblank_event *event; drm_panel_disable(mxsfb->panel); mxsfb_crtc_disable(mxsfb); drm_panel_unprepare(mxsfb->panel); + pm_runtime_put_sync(drm->dev); + + spin_lock_irq(&drm->event_lock); + event = crtc->state->event; + if (event) { + crtc->state->event = NULL; + drm_crtc_send_vblank_event(crtc, event); + } + spin_unlock_irq(&drm->event_lock); } static void mxsfb_pipe_update(struct drm_simple_display_pipe *pipe, @@ -230,6 +248,7 @@ static int mxsfb_load(struct drm_device *drm, unsigned long flags) drm->mode_config.max_width = MXSFB_MAX_XRES; drm->mode_config.max_height = MXSFB_MAX_YRES; drm->mode_config.funcs = &mxsfb_mode_config_funcs; + drm->mode_config.helper_private = &mxsfb_mode_config_helpers; drm_mode_config_reset(drm); @@ -414,6 +433,26 @@ static int mxsfb_remove(struct platform_device *pdev) return 0; } +#ifdef CONFIG_PM_SLEEP +static int mxsfb_suspend(struct device *dev) +{ + struct drm_device *drm = dev_get_drvdata(dev); + + return drm_mode_config_helper_suspend(drm); +} + +static int mxsfb_resume(struct device *dev) +{ + struct drm_device *drm = dev_get_drvdata(dev); + + return drm_mode_config_helper_resume(drm); +} +#endif + +static const struct dev_pm_ops mxsfb_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(mxsfb_suspend, mxsfb_resume) +}; + static struct platform_driver mxsfb_platform_driver = { .probe = mxsfb_probe, .remove = mxsfb_remove, @@ -421,6 +460,7 @@ static struct platform_driver mxsfb_platform_driver = { .driver = { .name = "mxsfb", .of_match_table = mxsfb_dt_ids, + .pm = &mxsfb_pm_ops, }, }; diff --git a/drivers/gpu/drm/nouveau/dispnv50/disp.c b/drivers/gpu/drm/nouveau/dispnv50/disp.c index 6d576249618a..9d9a18ab95ec 100644 --- a/drivers/gpu/drm/nouveau/dispnv50/disp.c +++ b/drivers/gpu/drm/nouveau/dispnv50/disp.c @@ -36,6 +36,7 @@ #include <drm/drm_dp_helper.h> #include <drm/drm_fb_helper.h> #include <drm/drm_plane_helper.h> +#include <drm/drm_scdc_helper.h> #include <drm/drm_edid.h> #include <nvif/class.h> @@ -531,6 +532,7 @@ nv50_hdmi_disable(struct drm_encoder *encoder, struct nouveau_crtc *nv_crtc) static void nv50_hdmi_enable(struct drm_encoder *encoder, struct drm_display_mode *mode) { + struct nouveau_drm *drm = nouveau_drm(encoder->dev); struct nouveau_encoder *nv_encoder = nouveau_encoder(encoder); struct nouveau_crtc *nv_crtc = nouveau_crtc(encoder->crtc); struct nv50_disp *disp = nv50_disp(encoder->dev); @@ -548,9 +550,12 @@ nv50_hdmi_enable(struct drm_encoder *encoder, struct drm_display_mode *mode) .pwr.rekey = 56, /* binary driver, and tegra, constant */ }; struct nouveau_connector *nv_connector; + struct drm_hdmi_info *hdmi; u32 max_ac_packet; union hdmi_infoframe avi_frame; union hdmi_infoframe vendor_frame; + bool scdc_supported, high_tmds_clock_ratio = false, scrambling = false; + u8 config; int ret; int size; @@ -558,8 +563,11 @@ nv50_hdmi_enable(struct drm_encoder *encoder, struct drm_display_mode *mode) if (!drm_detect_hdmi_monitor(nv_connector->edid)) return; + hdmi = &nv_connector->base.display_info.hdmi; + scdc_supported = hdmi->scdc.supported; + ret = drm_hdmi_avi_infoframe_from_display_mode(&avi_frame.avi, mode, - false); + scdc_supported); if (!ret) { /* We have an AVI InfoFrame, populate it to the display */ args.pwr.avi_infoframe_length @@ -582,12 +590,42 @@ nv50_hdmi_enable(struct drm_encoder *encoder, struct drm_display_mode *mode) max_ac_packet -= 18; /* constant from tegra */ args.pwr.max_ac_packet = max_ac_packet / 32; + if (hdmi->scdc.scrambling.supported) { + high_tmds_clock_ratio = mode->clock > 340000; + scrambling = high_tmds_clock_ratio || + hdmi->scdc.scrambling.low_rates; + } + + args.pwr.scdc = + NV50_DISP_SOR_HDMI_PWR_V0_SCDC_SCRAMBLE * scrambling | + NV50_DISP_SOR_HDMI_PWR_V0_SCDC_DIV_BY_4 * high_tmds_clock_ratio; + size = sizeof(args.base) + sizeof(args.pwr) + args.pwr.avi_infoframe_length + args.pwr.vendor_infoframe_length; nvif_mthd(&disp->disp->object, 0, &args, size); + nv50_audio_enable(encoder, mode); + + /* If SCDC is supported by the downstream monitor, update + * divider / scrambling settings to what we programmed above. + */ + if (!hdmi->scdc.scrambling.supported) + return; + + ret = drm_scdc_readb(nv_encoder->i2c, SCDC_TMDS_CONFIG, &config); + if (ret < 0) { + NV_ERROR(drm, "Failure to read SCDC_TMDS_CONFIG: %d\n", ret); + return; + } + config &= ~(SCDC_TMDS_BIT_CLOCK_RATIO_BY_40 | SCDC_SCRAMBLING_ENABLE); + config |= SCDC_TMDS_BIT_CLOCK_RATIO_BY_40 * high_tmds_clock_ratio; + config |= SCDC_SCRAMBLING_ENABLE * scrambling; + ret = drm_scdc_writeb(nv_encoder->i2c, SCDC_TMDS_CONFIG, config); + if (ret < 0) + NV_ERROR(drm, "Failure to write SCDC_TMDS_CONFIG = 0x%02x: %d\n", + config, ret); } /****************************************************************************** @@ -1117,17 +1155,21 @@ nv50_mstm_enable(struct nv50_mstm *mstm, u8 dpcd, int state) int ret; if (dpcd >= 0x12) { - ret = drm_dp_dpcd_readb(mstm->mgr.aux, DP_MSTM_CTRL, &dpcd); + /* Even if we're enabling MST, start with disabling the + * branching unit to clear any sink-side MST topology state + * that wasn't set by us + */ + ret = drm_dp_dpcd_writeb(mstm->mgr.aux, DP_MSTM_CTRL, 0); if (ret < 0) return ret; - dpcd &= ~DP_MST_EN; - if (state) - dpcd |= DP_MST_EN; - - ret = drm_dp_dpcd_writeb(mstm->mgr.aux, DP_MSTM_CTRL, dpcd); - if (ret < 0) - return ret; + if (state) { + /* Now, start initializing */ + ret = drm_dp_dpcd_writeb(mstm->mgr.aux, DP_MSTM_CTRL, + DP_MST_EN); + if (ret < 0) + return ret; + } } return nvif_mthd(disp, 0, &args, sizeof(args)); @@ -1136,31 +1178,58 @@ nv50_mstm_enable(struct nv50_mstm *mstm, u8 dpcd, int state) int nv50_mstm_detect(struct nv50_mstm *mstm, u8 dpcd[8], int allow) { - int ret, state = 0; + struct drm_dp_aux *aux; + int ret; + bool old_state, new_state; + u8 mstm_ctrl; if (!mstm) return 0; - if (dpcd[0] >= 0x12) { - ret = drm_dp_dpcd_readb(mstm->mgr.aux, DP_MSTM_CAP, &dpcd[1]); + mutex_lock(&mstm->mgr.lock); + + old_state = mstm->mgr.mst_state; + new_state = old_state; + aux = mstm->mgr.aux; + + if (old_state) { + /* Just check that the MST hub is still as we expect it */ + ret = drm_dp_dpcd_readb(aux, DP_MSTM_CTRL, &mstm_ctrl); + if (ret < 0 || !(mstm_ctrl & DP_MST_EN)) { + DRM_DEBUG_KMS("Hub gone, disabling MST topology\n"); + new_state = false; + } + } else if (dpcd[0] >= 0x12) { + ret = drm_dp_dpcd_readb(aux, DP_MSTM_CAP, &dpcd[1]); if (ret < 0) - return ret; + goto probe_error; if (!(dpcd[1] & DP_MST_CAP)) dpcd[0] = 0x11; else - state = allow; + new_state = allow; + } + + if (new_state == old_state) { + mutex_unlock(&mstm->mgr.lock); + return new_state; } - ret = nv50_mstm_enable(mstm, dpcd[0], state); + ret = nv50_mstm_enable(mstm, dpcd[0], new_state); if (ret) - return ret; + goto probe_error; - ret = drm_dp_mst_topology_mgr_set_mst(&mstm->mgr, state); + mutex_unlock(&mstm->mgr.lock); + + ret = drm_dp_mst_topology_mgr_set_mst(&mstm->mgr, new_state); if (ret) return nv50_mstm_enable(mstm, dpcd[0], 0); - return mstm->mgr.mst_state; + return new_state; + +probe_error: + mutex_unlock(&mstm->mgr.lock); + return ret; } static void @@ -2068,7 +2137,7 @@ nv50_disp_atomic_state_alloc(struct drm_device *dev) static const struct drm_mode_config_funcs nv50_disp_func = { .fb_create = nouveau_user_framebuffer_create, - .output_poll_changed = drm_fb_helper_output_poll_changed, + .output_poll_changed = nouveau_fbcon_output_poll_changed, .atomic_check = nv50_disp_atomic_check, .atomic_commit = nv50_disp_atomic_commit, .atomic_state_alloc = nv50_disp_atomic_state_alloc, diff --git a/drivers/gpu/drm/nouveau/include/nvif/cl5070.h b/drivers/gpu/drm/nouveau/include/nvif/cl5070.h index 7cdf53615d7b..bced81987269 100644 --- a/drivers/gpu/drm/nouveau/include/nvif/cl5070.h +++ b/drivers/gpu/drm/nouveau/include/nvif/cl5070.h @@ -69,7 +69,10 @@ struct nv50_disp_sor_hdmi_pwr_v0 { __u8 rekey; __u8 avi_infoframe_length; __u8 vendor_infoframe_length; - __u8 pad06[2]; +#define NV50_DISP_SOR_HDMI_PWR_V0_SCDC_SCRAMBLE (1 << 0) +#define NV50_DISP_SOR_HDMI_PWR_V0_SCDC_DIV_BY_4 (1 << 1) + __u8 scdc; + __u8 pad07[1]; }; struct nv50_disp_sor_lvds_script_v0 { diff --git a/drivers/gpu/drm/nouveau/nouveau_backlight.c b/drivers/gpu/drm/nouveau/nouveau_backlight.c index 408b955e5c39..5f5be6368aed 100644 --- a/drivers/gpu/drm/nouveau/nouveau_backlight.c +++ b/drivers/gpu/drm/nouveau/nouveau_backlight.c @@ -37,18 +37,19 @@ #include "nouveau_drv.h" #include "nouveau_reg.h" #include "nouveau_encoder.h" +#include "nouveau_connector.h" static struct ida bl_ida; #define BL_NAME_SIZE 15 // 12 for name + 2 for digits + 1 for '\0' -struct backlight_connector { - struct list_head head; +struct nouveau_backlight { + struct backlight_device *dev; int id; }; static bool -nouveau_get_backlight_name(char backlight_name[BL_NAME_SIZE], struct backlight_connector - *connector) +nouveau_get_backlight_name(char backlight_name[BL_NAME_SIZE], + struct nouveau_backlight *bl) { const int nb = ida_simple_get(&bl_ida, 0, 0, GFP_KERNEL); if (nb < 0 || nb >= 100) @@ -57,17 +58,18 @@ nouveau_get_backlight_name(char backlight_name[BL_NAME_SIZE], struct backlight_c snprintf(backlight_name, BL_NAME_SIZE, "nv_backlight%d", nb); else snprintf(backlight_name, BL_NAME_SIZE, "nv_backlight"); - connector->id = nb; + bl->id = nb; return true; } static int nv40_get_intensity(struct backlight_device *bd) { - struct nouveau_drm *drm = bl_get_data(bd); + struct nouveau_encoder *nv_encoder = bl_get_data(bd); + struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev); struct nvif_object *device = &drm->client.device.object; int val = (nvif_rd32(device, NV40_PMC_BACKLIGHT) & - NV40_PMC_BACKLIGHT_MASK) >> 16; + NV40_PMC_BACKLIGHT_MASK) >> 16; return val; } @@ -75,13 +77,14 @@ nv40_get_intensity(struct backlight_device *bd) static int nv40_set_intensity(struct backlight_device *bd) { - struct nouveau_drm *drm = bl_get_data(bd); + struct nouveau_encoder *nv_encoder = bl_get_data(bd); + struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev); struct nvif_object *device = &drm->client.device.object; int val = bd->props.brightness; int reg = nvif_rd32(device, NV40_PMC_BACKLIGHT); nvif_wr32(device, NV40_PMC_BACKLIGHT, - (val << 16) | (reg & ~NV40_PMC_BACKLIGHT_MASK)); + (val << 16) | (reg & ~NV40_PMC_BACKLIGHT_MASK)); return 0; } @@ -93,38 +96,19 @@ static const struct backlight_ops nv40_bl_ops = { }; static int -nv40_backlight_init(struct drm_connector *connector) +nv40_backlight_init(struct nouveau_encoder *encoder, + struct backlight_properties *props, + const struct backlight_ops **ops) { - struct nouveau_drm *drm = nouveau_drm(connector->dev); + struct nouveau_drm *drm = nouveau_drm(encoder->base.base.dev); struct nvif_object *device = &drm->client.device.object; - struct backlight_properties props; - struct backlight_device *bd; - struct backlight_connector bl_connector; - char backlight_name[BL_NAME_SIZE]; if (!(nvif_rd32(device, NV40_PMC_BACKLIGHT) & NV40_PMC_BACKLIGHT_MASK)) - return 0; - - memset(&props, 0, sizeof(struct backlight_properties)); - props.type = BACKLIGHT_RAW; - props.max_brightness = 31; - if (!nouveau_get_backlight_name(backlight_name, &bl_connector)) { - NV_ERROR(drm, "Failed to retrieve a unique name for the backlight interface\n"); - return 0; - } - bd = backlight_device_register(backlight_name , connector->kdev, drm, - &nv40_bl_ops, &props); - - if (IS_ERR(bd)) { - if (bl_connector.id > 0) - ida_simple_remove(&bl_ida, bl_connector.id); - return PTR_ERR(bd); - } - list_add(&bl_connector.head, &drm->bl_connectors); - drm->backlight = bd; - bd->props.brightness = nv40_get_intensity(bd); - backlight_update_status(bd); + return -ENODEV; + props->type = BACKLIGHT_RAW; + props->max_brightness = 31; + *ops = &nv40_bl_ops; return 0; } @@ -154,7 +138,7 @@ nv50_set_intensity(struct backlight_device *bd) u32 val = (bd->props.brightness * div) / 100; nvif_wr32(device, NV50_PDISP_SOR_PWM_CTL(or), - NV50_PDISP_SOR_PWM_CTL_NEW | val); + NV50_PDISP_SOR_PWM_CTL_NEW | val); return 0; } @@ -194,9 +178,10 @@ nva3_set_intensity(struct backlight_device *bd) div = nvif_rd32(device, NV50_PDISP_SOR_PWM_DIV(or)); val = (bd->props.brightness * div) / 100; if (div) { - nvif_wr32(device, NV50_PDISP_SOR_PWM_CTL(or), val | - NV50_PDISP_SOR_PWM_CTL_NEW | - NVA3_PDISP_SOR_PWM_CTL_UNK); + nvif_wr32(device, NV50_PDISP_SOR_PWM_CTL(or), + val | + NV50_PDISP_SOR_PWM_CTL_NEW | + NVA3_PDISP_SOR_PWM_CTL_UNK); return 0; } @@ -210,110 +195,119 @@ static const struct backlight_ops nva3_bl_ops = { }; static int -nv50_backlight_init(struct drm_connector *connector) +nv50_backlight_init(struct nouveau_encoder *nv_encoder, + struct backlight_properties *props, + const struct backlight_ops **ops) { - struct nouveau_drm *drm = nouveau_drm(connector->dev); + struct nouveau_drm *drm = nouveau_drm(nv_encoder->base.base.dev); struct nvif_object *device = &drm->client.device.object; - struct nouveau_encoder *nv_encoder; - struct backlight_properties props; - struct backlight_device *bd; - const struct backlight_ops *ops; - struct backlight_connector bl_connector; - char backlight_name[BL_NAME_SIZE]; - - nv_encoder = find_encoder(connector, DCB_OUTPUT_LVDS); - if (!nv_encoder) { - nv_encoder = find_encoder(connector, DCB_OUTPUT_DP); - if (!nv_encoder) - return -ENODEV; - } if (!nvif_rd32(device, NV50_PDISP_SOR_PWM_CTL(ffs(nv_encoder->dcb->or) - 1))) - return 0; + return -ENODEV; if (drm->client.device.info.chipset <= 0xa0 || drm->client.device.info.chipset == 0xaa || drm->client.device.info.chipset == 0xac) - ops = &nv50_bl_ops; + *ops = &nv50_bl_ops; else - ops = &nva3_bl_ops; - - memset(&props, 0, sizeof(struct backlight_properties)); - props.type = BACKLIGHT_RAW; - props.max_brightness = 100; - if (!nouveau_get_backlight_name(backlight_name, &bl_connector)) { - NV_ERROR(drm, "Failed to retrieve a unique name for the backlight interface\n"); - return 0; - } - bd = backlight_device_register(backlight_name , connector->kdev, - nv_encoder, ops, &props); + *ops = &nva3_bl_ops; - if (IS_ERR(bd)) { - if (bl_connector.id > 0) - ida_simple_remove(&bl_ida, bl_connector.id); - return PTR_ERR(bd); - } + props->type = BACKLIGHT_RAW; + props->max_brightness = 100; - list_add(&bl_connector.head, &drm->bl_connectors); - drm->backlight = bd; - bd->props.brightness = bd->ops->get_brightness(bd); - backlight_update_status(bd); return 0; } int -nouveau_backlight_init(struct drm_device *dev) +nouveau_backlight_init(struct drm_connector *connector) { - struct nouveau_drm *drm = nouveau_drm(dev); + struct nouveau_drm *drm = nouveau_drm(connector->dev); + struct nouveau_backlight *bl; + struct nouveau_encoder *nv_encoder = NULL; struct nvif_device *device = &drm->client.device; - struct drm_connector *connector; - struct drm_connector_list_iter conn_iter; - - INIT_LIST_HEAD(&drm->bl_connectors); + char backlight_name[BL_NAME_SIZE]; + struct backlight_properties props = {0}; + const struct backlight_ops *ops; + int ret; if (apple_gmux_present()) { - NV_INFO(drm, "Apple GMUX detected: not registering Nouveau backlight interface\n"); + NV_INFO_ONCE(drm, "Apple GMUX detected: not registering Nouveau backlight interface\n"); return 0; } - drm_connector_list_iter_begin(dev, &conn_iter); - drm_for_each_connector_iter(connector, &conn_iter) { - if (connector->connector_type != DRM_MODE_CONNECTOR_LVDS && - connector->connector_type != DRM_MODE_CONNECTOR_eDP) - continue; - - switch (device->info.family) { - case NV_DEVICE_INFO_V0_CURIE: - return nv40_backlight_init(connector); - case NV_DEVICE_INFO_V0_TESLA: - case NV_DEVICE_INFO_V0_FERMI: - case NV_DEVICE_INFO_V0_KEPLER: - case NV_DEVICE_INFO_V0_MAXWELL: - return nv50_backlight_init(connector); - default: - break; - } + if (connector->connector_type == DRM_MODE_CONNECTOR_LVDS) + nv_encoder = find_encoder(connector, DCB_OUTPUT_LVDS); + else if (connector->connector_type == DRM_MODE_CONNECTOR_eDP) + nv_encoder = find_encoder(connector, DCB_OUTPUT_DP); + else + return 0; + + if (!nv_encoder) + return 0; + + switch (device->info.family) { + case NV_DEVICE_INFO_V0_CURIE: + ret = nv40_backlight_init(nv_encoder, &props, &ops); + break; + case NV_DEVICE_INFO_V0_TESLA: + case NV_DEVICE_INFO_V0_FERMI: + case NV_DEVICE_INFO_V0_KEPLER: + case NV_DEVICE_INFO_V0_MAXWELL: + ret = nv50_backlight_init(nv_encoder, &props, &ops); + break; + default: + return 0; } - drm_connector_list_iter_end(&conn_iter); + + if (ret == -ENODEV) + return 0; + else if (ret) + return ret; + + bl = kzalloc(sizeof(*bl), GFP_KERNEL); + if (!bl) + return -ENOMEM; + + if (!nouveau_get_backlight_name(backlight_name, bl)) { + NV_ERROR(drm, "Failed to retrieve a unique name for the backlight interface\n"); + goto fail_alloc; + } + + bl->dev = backlight_device_register(backlight_name, connector->kdev, + nv_encoder, ops, &props); + if (IS_ERR(bl->dev)) { + if (bl->id >= 0) + ida_simple_remove(&bl_ida, bl->id); + ret = PTR_ERR(bl->dev); + goto fail_alloc; + } + + nouveau_connector(connector)->backlight = bl; + bl->dev->props.brightness = bl->dev->ops->get_brightness(bl->dev); + backlight_update_status(bl->dev); return 0; + +fail_alloc: + kfree(bl); + return ret; } void -nouveau_backlight_exit(struct drm_device *dev) +nouveau_backlight_fini(struct drm_connector *connector) { - struct nouveau_drm *drm = nouveau_drm(dev); - struct backlight_connector *connector; + struct nouveau_connector *nv_conn = nouveau_connector(connector); + struct nouveau_backlight *bl = nv_conn->backlight; - list_for_each_entry(connector, &drm->bl_connectors, head) { - if (connector->id >= 0) - ida_simple_remove(&bl_ida, connector->id); - } + if (!bl) + return; - if (drm->backlight) { - backlight_device_unregister(drm->backlight); - drm->backlight = NULL; - } + if (bl->id >= 0) + ida_simple_remove(&bl_ida, bl->id); + + backlight_device_unregister(bl->dev); + nv_conn->backlight = NULL; + kfree(bl); } void diff --git a/drivers/gpu/drm/nouveau/nouveau_connector.c b/drivers/gpu/drm/nouveau/nouveau_connector.c index eb4f766b5958..fd80661dff92 100644 --- a/drivers/gpu/drm/nouveau/nouveau_connector.c +++ b/drivers/gpu/drm/nouveau/nouveau_connector.c @@ -411,59 +411,45 @@ static struct nouveau_encoder * nouveau_connector_ddc_detect(struct drm_connector *connector) { struct drm_device *dev = connector->dev; - struct nouveau_connector *nv_connector = nouveau_connector(connector); - struct nouveau_drm *drm = nouveau_drm(dev); - struct nvkm_gpio *gpio = nvxx_gpio(&drm->client.device); - struct nouveau_encoder *nv_encoder = NULL; + struct nouveau_encoder *nv_encoder = NULL, *found = NULL; struct drm_encoder *encoder; - int i, panel = -ENODEV; - - /* eDP panels need powering on by us (if the VBIOS doesn't default it - * to on) before doing any AUX channel transactions. LVDS panel power - * is handled by the SOR itself, and not required for LVDS DDC. - */ - if (nv_connector->type == DCB_CONNECTOR_eDP) { - panel = nvkm_gpio_get(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff); - if (panel == 0) { - nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, 1); - msleep(300); - } - } + int i, ret; + bool switcheroo_ddc = false; drm_connector_for_each_possible_encoder(connector, encoder, i) { nv_encoder = nouveau_encoder(encoder); - if (nv_encoder->dcb->type == DCB_OUTPUT_DP) { - int ret = nouveau_dp_detect(nv_encoder); + switch (nv_encoder->dcb->type) { + case DCB_OUTPUT_DP: + ret = nouveau_dp_detect(nv_encoder); if (ret == NOUVEAU_DP_MST) return NULL; - if (ret == NOUVEAU_DP_SST) - break; - } else - if ((vga_switcheroo_handler_flags() & - VGA_SWITCHEROO_CAN_SWITCH_DDC) && - nv_encoder->dcb->type == DCB_OUTPUT_LVDS && - nv_encoder->i2c) { - int ret; - vga_switcheroo_lock_ddc(dev->pdev); - ret = nvkm_probe_i2c(nv_encoder->i2c, 0x50); - vga_switcheroo_unlock_ddc(dev->pdev); - if (ret) + else if (ret == NOUVEAU_DP_SST) + found = nv_encoder; + + break; + case DCB_OUTPUT_LVDS: + switcheroo_ddc = !!(vga_switcheroo_handler_flags() & + VGA_SWITCHEROO_CAN_SWITCH_DDC); + /* fall-through */ + default: + if (!nv_encoder->i2c) break; - } else - if (nv_encoder->i2c) { + + if (switcheroo_ddc) + vga_switcheroo_lock_ddc(dev->pdev); if (nvkm_probe_i2c(nv_encoder->i2c, 0x50)) - break; + found = nv_encoder; + if (switcheroo_ddc) + vga_switcheroo_unlock_ddc(dev->pdev); + + break; } + if (found) + break; } - /* eDP panel not detected, restore panel power GPIO to previous - * state to avoid confusing the SOR for other output types. - */ - if (!nv_encoder && panel == 0) - nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, panel); - - return nv_encoder; + return found; } static struct nouveau_encoder * @@ -557,12 +543,16 @@ nouveau_connector_detect(struct drm_connector *connector, bool force) nv_connector->edid = NULL; } - /* Outputs are only polled while runtime active, so acquiring a - * runtime PM ref here is unnecessary (and would deadlock upon - * runtime suspend because it waits for polling to finish). + /* Outputs are only polled while runtime active, so resuming the + * device here is unnecessary (and would deadlock upon runtime suspend + * because it waits for polling to finish). We do however, want to + * prevent the autosuspend timer from elapsing during this operation + * if possible. */ - if (!drm_kms_helper_is_poll_worker()) { - ret = pm_runtime_get_sync(connector->dev->dev); + if (drm_kms_helper_is_poll_worker()) { + pm_runtime_get_noresume(dev->dev); + } else { + ret = pm_runtime_get_sync(dev->dev); if (ret < 0 && ret != -EACCES) return conn_status; } @@ -641,10 +631,8 @@ detect_analog: out: - if (!drm_kms_helper_is_poll_worker()) { - pm_runtime_mark_last_busy(connector->dev->dev); - pm_runtime_put_autosuspend(connector->dev->dev); - } + pm_runtime_mark_last_busy(dev->dev); + pm_runtime_put_autosuspend(dev->dev); return conn_status; } @@ -898,6 +886,22 @@ nouveau_connector_detect_depth(struct drm_connector *connector) } static int +nouveau_connector_late_register(struct drm_connector *connector) +{ + int ret; + + ret = nouveau_backlight_init(connector); + + return ret; +} + +static void +nouveau_connector_early_unregister(struct drm_connector *connector) +{ + nouveau_backlight_fini(connector); +} + +static int nouveau_connector_get_modes(struct drm_connector *connector) { struct drm_device *dev = connector->dev; @@ -965,18 +969,33 @@ nouveau_connector_get_modes(struct drm_connector *connector) } static unsigned -get_tmds_link_bandwidth(struct drm_connector *connector, bool hdmi) +get_tmds_link_bandwidth(struct drm_connector *connector) { struct nouveau_connector *nv_connector = nouveau_connector(connector); + struct nouveau_encoder *nv_encoder = nv_connector->detected_encoder; struct nouveau_drm *drm = nouveau_drm(connector->dev); struct dcb_output *dcb = nv_connector->detected_encoder->dcb; + struct drm_display_info *info = NULL; + const unsigned duallink_scale = + nouveau_duallink && nv_encoder->dcb->duallink_possible ? 2 : 1; + + if (drm_detect_hdmi_monitor(nv_connector->edid)) + info = &nv_connector->base.display_info; - if (hdmi) { + if (info) { if (nouveau_hdmimhz > 0) return nouveau_hdmimhz * 1000; /* Note: these limits are conservative, some Fermi's * can do 297 MHz. Unclear how this can be determined. */ + if (drm->client.device.info.chipset >= 0x120) { + const int max_tmds_clock = + info->hdmi.scdc.scrambling.supported ? + 594000 : 340000; + return info->max_tmds_clock ? + min(info->max_tmds_clock, max_tmds_clock) : + max_tmds_clock; + } if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_KEPLER) return 297000; if (drm->client.device.info.family >= NV_DEVICE_INFO_V0_FERMI) @@ -984,13 +1003,13 @@ get_tmds_link_bandwidth(struct drm_connector *connector, bool hdmi) } if (dcb->location != DCB_LOC_ON_CHIP || drm->client.device.info.chipset >= 0x46) - return 165000; + return 165000 * duallink_scale; else if (drm->client.device.info.chipset >= 0x40) - return 155000; + return 155000 * duallink_scale; else if (drm->client.device.info.chipset >= 0x18) - return 135000; + return 135000 * duallink_scale; else - return 112000; + return 112000 * duallink_scale; } static enum drm_mode_status @@ -1002,7 +1021,6 @@ nouveau_connector_mode_valid(struct drm_connector *connector, struct drm_encoder *encoder = to_drm_encoder(nv_encoder); unsigned min_clock = 25000, max_clock = min_clock; unsigned clock = mode->clock; - bool hdmi; switch (nv_encoder->dcb->type) { case DCB_OUTPUT_LVDS: @@ -1015,11 +1033,7 @@ nouveau_connector_mode_valid(struct drm_connector *connector, max_clock = 400000; break; case DCB_OUTPUT_TMDS: - hdmi = drm_detect_hdmi_monitor(nv_connector->edid); - max_clock = get_tmds_link_bandwidth(connector, hdmi); - if (!hdmi && nouveau_duallink && - nv_encoder->dcb->duallink_possible) - max_clock *= 2; + max_clock = get_tmds_link_bandwidth(connector); break; case DCB_OUTPUT_ANALOG: max_clock = nv_encoder->dcb->crtconf.maxfreq; @@ -1081,6 +1095,8 @@ nouveau_connector_funcs = { .atomic_destroy_state = nouveau_conn_atomic_destroy_state, .atomic_set_property = nouveau_conn_atomic_set_property, .atomic_get_property = nouveau_conn_atomic_get_property, + .late_register = nouveau_connector_late_register, + .early_unregister = nouveau_connector_early_unregister, }; static const struct drm_connector_funcs @@ -1096,6 +1112,8 @@ nouveau_connector_funcs_lvds = { .atomic_destroy_state = nouveau_conn_atomic_destroy_state, .atomic_set_property = nouveau_conn_atomic_set_property, .atomic_get_property = nouveau_conn_atomic_get_property, + .late_register = nouveau_connector_late_register, + .early_unregister = nouveau_connector_early_unregister, }; static int @@ -1108,6 +1126,26 @@ nouveau_connector_hotplug(struct nvif_notify *notify) const struct nvif_notify_conn_rep_v0 *rep = notify->data; const char *name = connector->name; struct nouveau_encoder *nv_encoder; + int ret; + + ret = pm_runtime_get(drm->dev->dev); + if (ret == 0) { + /* We can't block here if there's a pending PM request + * running, as we'll deadlock nouveau_display_fini() when it + * calls nvif_put() on our nvif_notify struct. So, simply + * defer the hotplug event until the device finishes resuming + */ + NV_DEBUG(drm, "Deferring HPD on %s until runtime resume\n", + name); + schedule_work(&drm->hpd_work); + + pm_runtime_put_noidle(drm->dev->dev); + return NVIF_NOTIFY_KEEP; + } else if (ret != 1 && ret != -EACCES) { + NV_WARN(drm, "HPD on %s dropped due to RPM failure: %d\n", + name, ret); + return NVIF_NOTIFY_DROP; + } if (rep->mask & NVIF_NOTIFY_CONN_V0_IRQ) { NV_DEBUG(drm, "service %s\n", name); @@ -1128,6 +1166,8 @@ nouveau_connector_hotplug(struct nvif_notify *notify) drm_helper_hpd_irq_event(connector->dev); } + pm_runtime_mark_last_busy(drm->dev->dev); + pm_runtime_put_autosuspend(drm->dev->dev); return NVIF_NOTIFY_KEEP; } diff --git a/drivers/gpu/drm/nouveau/nouveau_connector.h b/drivers/gpu/drm/nouveau/nouveau_connector.h index 0acc07555bcd..f57ef35b1e5e 100644 --- a/drivers/gpu/drm/nouveau/nouveau_connector.h +++ b/drivers/gpu/drm/nouveau/nouveau_connector.h @@ -39,6 +39,10 @@ struct nvkm_i2c_port; +#ifdef CONFIG_DRM_NOUVEAU_BACKLIGHT +struct nouveau_backlight; +#endif + struct nouveau_connector { struct drm_connector base; enum dcb_connector_type type; @@ -55,6 +59,9 @@ struct nouveau_connector { struct nouveau_encoder *detected_encoder; struct edid *edid; struct drm_display_mode *native_mode; +#ifdef CONFIG_DRM_NOUVEAU_BACKLIGHT + struct nouveau_backlight *backlight; +#endif }; static inline struct nouveau_connector *nouveau_connector( @@ -181,4 +188,30 @@ int nouveau_conn_atomic_get_property(struct drm_connector *, const struct drm_connector_state *, struct drm_property *, u64 *); struct drm_display_mode *nouveau_conn_native_mode(struct drm_connector *); + +#ifdef CONFIG_DRM_NOUVEAU_BACKLIGHT +extern int nouveau_backlight_init(struct drm_connector *); +extern void nouveau_backlight_fini(struct drm_connector *); +extern void nouveau_backlight_ctor(void); +extern void nouveau_backlight_dtor(void); +#else +static inline int +nouveau_backlight_init(struct drm_connector *connector) +{ + return 0; +} + +static inline void +nouveau_backlight_fini(struct drm_connector *connector) { +} + +static inline void +nouveau_backlight_ctor(void) { +} + +static inline void +nouveau_backlight_dtor(void) { +} +#endif + #endif /* __NOUVEAU_CONNECTOR_H__ */ diff --git a/drivers/gpu/drm/nouveau/nouveau_display.c b/drivers/gpu/drm/nouveau/nouveau_display.c index 139368b31916..f326ffd86766 100644 --- a/drivers/gpu/drm/nouveau/nouveau_display.c +++ b/drivers/gpu/drm/nouveau/nouveau_display.c @@ -293,7 +293,7 @@ nouveau_user_framebuffer_create(struct drm_device *dev, static const struct drm_mode_config_funcs nouveau_mode_config_funcs = { .fb_create = nouveau_user_framebuffer_create, - .output_poll_changed = drm_fb_helper_output_poll_changed, + .output_poll_changed = nouveau_fbcon_output_poll_changed, }; @@ -355,8 +355,6 @@ nouveau_display_hpd_work(struct work_struct *work) pm_runtime_get_sync(drm->dev->dev); drm_helper_hpd_irq_event(drm->dev); - /* enable polling for external displays */ - drm_kms_helper_poll_enable(drm->dev); pm_runtime_mark_last_busy(drm->dev->dev); pm_runtime_put_sync(drm->dev->dev); @@ -379,15 +377,29 @@ nouveau_display_acpi_ntfy(struct notifier_block *nb, unsigned long val, { struct nouveau_drm *drm = container_of(nb, typeof(*drm), acpi_nb); struct acpi_bus_event *info = data; + int ret; if (!strcmp(info->device_class, ACPI_VIDEO_CLASS)) { if (info->type == ACPI_VIDEO_NOTIFY_PROBE) { - /* - * This may be the only indication we receive of a - * connector hotplug on a runtime suspended GPU, - * schedule hpd_work to check. - */ - schedule_work(&drm->hpd_work); + ret = pm_runtime_get(drm->dev->dev); + if (ret == 1 || ret == -EACCES) { + /* If the GPU is already awake, or in a state + * where we can't wake it up, it can handle + * it's own hotplug events. + */ + pm_runtime_put_autosuspend(drm->dev->dev); + } else if (ret == 0) { + /* This may be the only indication we receive + * of a connector hotplug on a runtime + * suspended GPU, schedule hpd_work to check. + */ + NV_DEBUG(drm, "ACPI requested connector reprobe\n"); + schedule_work(&drm->hpd_work); + pm_runtime_put_noidle(drm->dev->dev); + } else { + NV_WARN(drm, "Dropped ACPI reprobe event due to RPM error: %d\n", + ret); + } /* acpi-video should not generate keypresses for this */ return NOTIFY_BAD; @@ -411,6 +423,11 @@ nouveau_display_init(struct drm_device *dev) if (ret) return ret; + /* enable connector detection and polling for connectors without HPD + * support + */ + drm_kms_helper_poll_enable(dev); + /* enable hotplug interrupts */ drm_connector_list_iter_begin(dev, &conn_iter); nouveau_for_each_non_mst_connector_iter(connector, &conn_iter) { @@ -425,7 +442,7 @@ nouveau_display_init(struct drm_device *dev) } void -nouveau_display_fini(struct drm_device *dev, bool suspend) +nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime) { struct nouveau_display *disp = nouveau_display(dev); struct nouveau_drm *drm = nouveau_drm(dev); @@ -450,6 +467,9 @@ nouveau_display_fini(struct drm_device *dev, bool suspend) } drm_connector_list_iter_end(&conn_iter); + if (!runtime) + cancel_work_sync(&drm->hpd_work); + drm_kms_helper_poll_disable(dev); disp->fini(dev); } @@ -562,7 +582,6 @@ nouveau_display_create(struct drm_device *dev) goto vblank_err; } - nouveau_backlight_init(dev); INIT_WORK(&drm->hpd_work, nouveau_display_hpd_work); #ifdef CONFIG_ACPI drm->acpi_nb.notifier_call = nouveau_display_acpi_ntfy; @@ -587,7 +606,6 @@ nouveau_display_destroy(struct drm_device *dev) #ifdef CONFIG_ACPI unregister_acpi_notifier(&nouveau_drm(dev)->acpi_nb); #endif - nouveau_backlight_exit(dev); nouveau_display_vblank_fini(dev); drm_kms_helper_poll_fini(dev); @@ -618,11 +636,11 @@ nouveau_display_suspend(struct drm_device *dev, bool runtime) } } - nouveau_display_fini(dev, true); + nouveau_display_fini(dev, true, runtime); return 0; } - nouveau_display_fini(dev, true); + nouveau_display_fini(dev, true, runtime); list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { struct nouveau_framebuffer *nouveau_fb; diff --git a/drivers/gpu/drm/nouveau/nouveau_display.h b/drivers/gpu/drm/nouveau/nouveau_display.h index 54aa7c3fa42d..eb77e41c2d4e 100644 --- a/drivers/gpu/drm/nouveau/nouveau_display.h +++ b/drivers/gpu/drm/nouveau/nouveau_display.h @@ -62,7 +62,7 @@ nouveau_display(struct drm_device *dev) int nouveau_display_create(struct drm_device *dev); void nouveau_display_destroy(struct drm_device *dev); int nouveau_display_init(struct drm_device *dev); -void nouveau_display_fini(struct drm_device *dev, bool suspend); +void nouveau_display_fini(struct drm_device *dev, bool suspend, bool runtime); int nouveau_display_suspend(struct drm_device *dev, bool runtime); void nouveau_display_resume(struct drm_device *dev, bool runtime); int nouveau_display_vblank_enable(struct drm_device *, unsigned int); @@ -85,31 +85,6 @@ int nouveau_display_dumb_map_offset(struct drm_file *, struct drm_device *, void nouveau_hdmi_mode_set(struct drm_encoder *, struct drm_display_mode *); -#ifdef CONFIG_DRM_NOUVEAU_BACKLIGHT -extern int nouveau_backlight_init(struct drm_device *); -extern void nouveau_backlight_exit(struct drm_device *); -extern void nouveau_backlight_ctor(void); -extern void nouveau_backlight_dtor(void); -#else -static inline int -nouveau_backlight_init(struct drm_device *dev) -{ - return 0; -} - -static inline void -nouveau_backlight_exit(struct drm_device *dev) { -} - -static inline void -nouveau_backlight_ctor(void) { -} - -static inline void -nouveau_backlight_dtor(void) { -} -#endif - struct drm_framebuffer * nouveau_user_framebuffer_create(struct drm_device *, struct drm_file *, const struct drm_mode_fb_cmd2 *); diff --git a/drivers/gpu/drm/nouveau/nouveau_drm.c b/drivers/gpu/drm/nouveau/nouveau_drm.c index c7ec86d6c3c9..2b2baf6e0e0d 100644 --- a/drivers/gpu/drm/nouveau/nouveau_drm.c +++ b/drivers/gpu/drm/nouveau/nouveau_drm.c @@ -230,7 +230,7 @@ nouveau_cli_init(struct nouveau_drm *drm, const char *sname, mutex_unlock(&drm->master.lock); } if (ret) { - NV_ERROR(drm, "Client allocation failed: %d\n", ret); + NV_PRINTK(err, cli, "Client allocation failed: %d\n", ret); goto done; } @@ -240,37 +240,37 @@ nouveau_cli_init(struct nouveau_drm *drm, const char *sname, }, sizeof(struct nv_device_v0), &cli->device); if (ret) { - NV_ERROR(drm, "Device allocation failed: %d\n", ret); + NV_PRINTK(err, cli, "Device allocation failed: %d\n", ret); goto done; } ret = nvif_mclass(&cli->device.object, mmus); if (ret < 0) { - NV_ERROR(drm, "No supported MMU class\n"); + NV_PRINTK(err, cli, "No supported MMU class\n"); goto done; } ret = nvif_mmu_init(&cli->device.object, mmus[ret].oclass, &cli->mmu); if (ret) { - NV_ERROR(drm, "MMU allocation failed: %d\n", ret); + NV_PRINTK(err, cli, "MMU allocation failed: %d\n", ret); goto done; } ret = nvif_mclass(&cli->mmu.object, vmms); if (ret < 0) { - NV_ERROR(drm, "No supported VMM class\n"); + NV_PRINTK(err, cli, "No supported VMM class\n"); goto done; } ret = nouveau_vmm_init(cli, vmms[ret].oclass, &cli->vmm); if (ret) { - NV_ERROR(drm, "VMM allocation failed: %d\n", ret); + NV_PRINTK(err, cli, "VMM allocation failed: %d\n", ret); goto done; } ret = nvif_mclass(&cli->mmu.object, mems); if (ret < 0) { - NV_ERROR(drm, "No supported MEM class\n"); + NV_PRINTK(err, cli, "No supported MEM class\n"); goto done; } @@ -458,75 +458,8 @@ nouveau_accel_init(struct nouveau_drm *drm) nouveau_bo_move_init(drm); } -static int nouveau_drm_probe(struct pci_dev *pdev, - const struct pci_device_id *pent) -{ - struct nvkm_device *device; - struct apertures_struct *aper; - bool boot = false; - int ret; - - if (vga_switcheroo_client_probe_defer(pdev)) - return -EPROBE_DEFER; - - /* We need to check that the chipset is supported before booting - * fbdev off the hardware, as there's no way to put it back. - */ - ret = nvkm_device_pci_new(pdev, NULL, "error", true, false, 0, &device); - if (ret) - return ret; - - nvkm_device_del(&device); - - /* Remove conflicting drivers (vesafb, efifb etc). */ - aper = alloc_apertures(3); - if (!aper) - return -ENOMEM; - - aper->ranges[0].base = pci_resource_start(pdev, 1); - aper->ranges[0].size = pci_resource_len(pdev, 1); - aper->count = 1; - - if (pci_resource_len(pdev, 2)) { - aper->ranges[aper->count].base = pci_resource_start(pdev, 2); - aper->ranges[aper->count].size = pci_resource_len(pdev, 2); - aper->count++; - } - - if (pci_resource_len(pdev, 3)) { - aper->ranges[aper->count].base = pci_resource_start(pdev, 3); - aper->ranges[aper->count].size = pci_resource_len(pdev, 3); - aper->count++; - } - -#ifdef CONFIG_X86 - boot = pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_SHADOW; -#endif - if (nouveau_modeset != 2) - drm_fb_helper_remove_conflicting_framebuffers(aper, "nouveaufb", boot); - kfree(aper); - - ret = nvkm_device_pci_new(pdev, nouveau_config, nouveau_debug, - true, true, ~0ULL, &device); - if (ret) - return ret; - - pci_set_master(pdev); - - if (nouveau_atomic) - driver_pci.driver_features |= DRIVER_ATOMIC; - - ret = drm_get_pci_dev(pdev, pent, &driver_pci); - if (ret) { - nvkm_device_del(&device); - return ret; - } - - return 0; -} - static int -nouveau_drm_load(struct drm_device *dev, unsigned long flags) +nouveau_drm_device_init(struct drm_device *dev) { struct nouveau_drm *drm; int ret; @@ -538,11 +471,11 @@ nouveau_drm_load(struct drm_device *dev, unsigned long flags) ret = nouveau_cli_init(drm, "DRM-master", &drm->master); if (ret) - return ret; + goto fail_alloc; ret = nouveau_cli_init(drm, "DRM", &drm->client); if (ret) - return ret; + goto fail_master; dev->irq_enabled = true; @@ -592,10 +525,8 @@ nouveau_drm_load(struct drm_device *dev, unsigned long flags) pm_runtime_allow(dev->dev); pm_runtime_mark_last_busy(dev->dev); pm_runtime_put(dev->dev); - } else { - /* enable polling for external displays */ - drm_kms_helper_poll_enable(dev); } + return 0; fail_dispinit: @@ -607,13 +538,15 @@ fail_bios: fail_ttm: nouveau_vga_fini(drm); nouveau_cli_fini(&drm->client); +fail_master: nouveau_cli_fini(&drm->master); +fail_alloc: kfree(drm); return ret; } static void -nouveau_drm_unload(struct drm_device *dev) +nouveau_drm_device_fini(struct drm_device *dev) { struct nouveau_drm *drm = nouveau_drm(dev); @@ -629,7 +562,7 @@ nouveau_drm_unload(struct drm_device *dev) nouveau_debugfs_fini(drm); if (dev->mode_config.num_crtc) - nouveau_display_fini(dev, false); + nouveau_display_fini(dev, false, false); nouveau_display_destroy(dev); nouveau_bios_takedown(dev); @@ -642,18 +575,116 @@ nouveau_drm_unload(struct drm_device *dev) kfree(drm); } +static int nouveau_drm_probe(struct pci_dev *pdev, + const struct pci_device_id *pent) +{ + struct nvkm_device *device; + struct drm_device *drm_dev; + struct apertures_struct *aper; + bool boot = false; + int ret; + + if (vga_switcheroo_client_probe_defer(pdev)) + return -EPROBE_DEFER; + + /* We need to check that the chipset is supported before booting + * fbdev off the hardware, as there's no way to put it back. + */ + ret = nvkm_device_pci_new(pdev, NULL, "error", true, false, 0, &device); + if (ret) + return ret; + + nvkm_device_del(&device); + + /* Remove conflicting drivers (vesafb, efifb etc). */ + aper = alloc_apertures(3); + if (!aper) + return -ENOMEM; + + aper->ranges[0].base = pci_resource_start(pdev, 1); + aper->ranges[0].size = pci_resource_len(pdev, 1); + aper->count = 1; + + if (pci_resource_len(pdev, 2)) { + aper->ranges[aper->count].base = pci_resource_start(pdev, 2); + aper->ranges[aper->count].size = pci_resource_len(pdev, 2); + aper->count++; + } + + if (pci_resource_len(pdev, 3)) { + aper->ranges[aper->count].base = pci_resource_start(pdev, 3); + aper->ranges[aper->count].size = pci_resource_len(pdev, 3); + aper->count++; + } + +#ifdef CONFIG_X86 + boot = pdev->resource[PCI_ROM_RESOURCE].flags & IORESOURCE_ROM_SHADOW; +#endif + if (nouveau_modeset != 2) + drm_fb_helper_remove_conflicting_framebuffers(aper, "nouveaufb", boot); + kfree(aper); + + ret = nvkm_device_pci_new(pdev, nouveau_config, nouveau_debug, + true, true, ~0ULL, &device); + if (ret) + return ret; + + pci_set_master(pdev); + + if (nouveau_atomic) + driver_pci.driver_features |= DRIVER_ATOMIC; + + drm_dev = drm_dev_alloc(&driver_pci, &pdev->dev); + if (IS_ERR(drm_dev)) { + ret = PTR_ERR(drm_dev); + goto fail_nvkm; + } + + ret = pci_enable_device(pdev); + if (ret) + goto fail_drm; + + drm_dev->pdev = pdev; + pci_set_drvdata(pdev, drm_dev); + + ret = nouveau_drm_device_init(drm_dev); + if (ret) + goto fail_pci; + + ret = drm_dev_register(drm_dev, pent->driver_data); + if (ret) + goto fail_drm_dev_init; + + return 0; + +fail_drm_dev_init: + nouveau_drm_device_fini(drm_dev); +fail_pci: + pci_disable_device(pdev); +fail_drm: + drm_dev_put(drm_dev); +fail_nvkm: + nvkm_device_del(&device); + return ret; +} + void nouveau_drm_device_remove(struct drm_device *dev) { + struct pci_dev *pdev = dev->pdev; struct nouveau_drm *drm = nouveau_drm(dev); struct nvkm_client *client; struct nvkm_device *device; + drm_dev_unregister(dev); + dev->irq_enabled = false; client = nvxx_client(&drm->client.base); device = nvkm_device_find(client->device); - drm_put_dev(dev); + nouveau_drm_device_fini(dev); + pci_disable_device(pdev); + drm_dev_put(dev); nvkm_device_del(&device); } @@ -835,7 +866,6 @@ nouveau_pmops_runtime_suspend(struct device *dev) return -EBUSY; } - drm_kms_helper_poll_disable(drm_dev); nouveau_switcheroo_optimus_dsm(); ret = nouveau_do_suspend(drm_dev, true); pci_save_state(pdev); @@ -1021,8 +1051,6 @@ driver_stub = { DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME | DRIVER_RENDER | DRIVER_KMS_LEGACY_CONTEXT, - .load = nouveau_drm_load, - .unload = nouveau_drm_unload, .open = nouveau_drm_open, .postclose = nouveau_drm_postclose, .lastclose = nouveau_vga_lastclose, diff --git a/drivers/gpu/drm/nouveau/nouveau_drv.h b/drivers/gpu/drm/nouveau/nouveau_drv.h index 6e1acaec3400..0b2191fa96f7 100644 --- a/drivers/gpu/drm/nouveau/nouveau_drv.h +++ b/drivers/gpu/drm/nouveau/nouveau_drv.h @@ -194,8 +194,6 @@ struct nouveau_drm { /* modesetting */ struct nvbios vbios; struct nouveau_display *display; - struct backlight_device *backlight; - struct list_head bl_connectors; struct work_struct hpd_work; struct work_struct fbcon_work; int fbcon_new_state; @@ -244,10 +242,12 @@ void nouveau_drm_device_remove(struct drm_device *dev); struct nouveau_cli *_cli = (c); \ dev_##l(_cli->drm->dev->dev, "%s: "f, _cli->name, ##a); \ } while(0) + #define NV_FATAL(drm,f,a...) NV_PRINTK(crit, &(drm)->client, f, ##a) #define NV_ERROR(drm,f,a...) NV_PRINTK(err, &(drm)->client, f, ##a) #define NV_WARN(drm,f,a...) NV_PRINTK(warn, &(drm)->client, f, ##a) #define NV_INFO(drm,f,a...) NV_PRINTK(info, &(drm)->client, f, ##a) + #define NV_DEBUG(drm,f,a...) do { \ if (unlikely(drm_debug & DRM_UT_DRIVER)) \ NV_PRINTK(info, &(drm)->client, f, ##a); \ @@ -257,6 +257,12 @@ void nouveau_drm_device_remove(struct drm_device *dev); NV_PRINTK(info, &(drm)->client, f, ##a); \ } while(0) +#define NV_PRINTK_ONCE(l,c,f,a...) NV_PRINTK(l##_once,c,f, ##a) + +#define NV_ERROR_ONCE(drm,f,a...) NV_PRINTK_ONCE(err, &(drm)->client, f, ##a) +#define NV_WARN_ONCE(drm,f,a...) NV_PRINTK_ONCE(warn, &(drm)->client, f, ##a) +#define NV_INFO_ONCE(drm,f,a...) NV_PRINTK_ONCE(info, &(drm)->client, f, ##a) + extern int nouveau_modeset; #endif diff --git a/drivers/gpu/drm/nouveau/nouveau_fbcon.c b/drivers/gpu/drm/nouveau/nouveau_fbcon.c index 20a260887be3..032317c81bf0 100644 --- a/drivers/gpu/drm/nouveau/nouveau_fbcon.c +++ b/drivers/gpu/drm/nouveau/nouveau_fbcon.c @@ -465,6 +465,7 @@ nouveau_fbcon_set_suspend_work(struct work_struct *work) console_unlock(); if (state == FBINFO_STATE_RUNNING) { + nouveau_fbcon_hotplug_resume(drm->fbcon); pm_runtime_mark_last_busy(drm->dev->dev); pm_runtime_put_sync(drm->dev->dev); } @@ -486,6 +487,61 @@ nouveau_fbcon_set_suspend(struct drm_device *dev, int state) schedule_work(&drm->fbcon_work); } +void +nouveau_fbcon_output_poll_changed(struct drm_device *dev) +{ + struct nouveau_drm *drm = nouveau_drm(dev); + struct nouveau_fbdev *fbcon = drm->fbcon; + int ret; + + if (!fbcon) + return; + + mutex_lock(&fbcon->hotplug_lock); + + ret = pm_runtime_get(dev->dev); + if (ret == 1 || ret == -EACCES) { + drm_fb_helper_hotplug_event(&fbcon->helper); + + pm_runtime_mark_last_busy(dev->dev); + pm_runtime_put_autosuspend(dev->dev); + } else if (ret == 0) { + /* If the GPU was already in the process of suspending before + * this event happened, then we can't block here as we'll + * deadlock the runtime pmops since they wait for us to + * finish. So, just defer this event for when we runtime + * resume again. It will be handled by fbcon_work. + */ + NV_DEBUG(drm, "fbcon HPD event deferred until runtime resume\n"); + fbcon->hotplug_waiting = true; + pm_runtime_put_noidle(drm->dev->dev); + } else { + DRM_WARN("fbcon HPD event lost due to RPM failure: %d\n", + ret); + } + + mutex_unlock(&fbcon->hotplug_lock); +} + +void +nouveau_fbcon_hotplug_resume(struct nouveau_fbdev *fbcon) +{ + struct nouveau_drm *drm; + + if (!fbcon) + return; + drm = nouveau_drm(fbcon->helper.dev); + + mutex_lock(&fbcon->hotplug_lock); + if (fbcon->hotplug_waiting) { + fbcon->hotplug_waiting = false; + + NV_DEBUG(drm, "Handling deferred fbcon HPD events\n"); + drm_fb_helper_hotplug_event(&fbcon->helper); + } + mutex_unlock(&fbcon->hotplug_lock); +} + int nouveau_fbcon_init(struct drm_device *dev) { @@ -504,6 +560,7 @@ nouveau_fbcon_init(struct drm_device *dev) drm->fbcon = fbcon; INIT_WORK(&drm->fbcon_work, nouveau_fbcon_set_suspend_work); + mutex_init(&fbcon->hotplug_lock); drm_fb_helper_prepare(dev, &fbcon->helper, &nouveau_fbcon_helper_funcs); diff --git a/drivers/gpu/drm/nouveau/nouveau_fbcon.h b/drivers/gpu/drm/nouveau/nouveau_fbcon.h index a6f192ea3fa6..db9d52047ef8 100644 --- a/drivers/gpu/drm/nouveau/nouveau_fbcon.h +++ b/drivers/gpu/drm/nouveau/nouveau_fbcon.h @@ -41,6 +41,9 @@ struct nouveau_fbdev { struct nvif_object gdi; struct nvif_object blit; struct nvif_object twod; + + struct mutex hotplug_lock; + bool hotplug_waiting; }; void nouveau_fbcon_restore(void); @@ -68,6 +71,8 @@ void nouveau_fbcon_set_suspend(struct drm_device *dev, int state); void nouveau_fbcon_accel_save_disable(struct drm_device *dev); void nouveau_fbcon_accel_restore(struct drm_device *dev); +void nouveau_fbcon_output_poll_changed(struct drm_device *dev); +void nouveau_fbcon_hotplug_resume(struct nouveau_fbdev *fbcon); extern int nouveau_nofbaccel; #endif /* __NV50_FBCON_H__ */ diff --git a/drivers/gpu/drm/nouveau/nouveau_vga.c b/drivers/gpu/drm/nouveau/nouveau_vga.c index 3da5a4305aa4..8f1ce4833230 100644 --- a/drivers/gpu/drm/nouveau/nouveau_vga.c +++ b/drivers/gpu/drm/nouveau/nouveau_vga.c @@ -46,12 +46,10 @@ nouveau_switcheroo_set_state(struct pci_dev *pdev, pr_err("VGA switcheroo: switched nouveau on\n"); dev->switch_power_state = DRM_SWITCH_POWER_CHANGING; nouveau_pmops_resume(&pdev->dev); - drm_kms_helper_poll_enable(dev); dev->switch_power_state = DRM_SWITCH_POWER_ON; } else { pr_err("VGA switcheroo: switched nouveau off\n"); dev->switch_power_state = DRM_SWITCH_POWER_CHANGING; - drm_kms_helper_poll_disable(dev); nouveau_switcheroo_optimus_dsm(); nouveau_pmops_suspend(&pdev->dev); dev->switch_power_state = DRM_SWITCH_POWER_OFF; diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild b/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild index 3d485dbf310a..8089ac9a12e2 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/Kbuild @@ -50,6 +50,7 @@ nvkm-y += nvkm/engine/disp/hdmig84.o nvkm-y += nvkm/engine/disp/hdmigt215.o nvkm-y += nvkm/engine/disp/hdmigf119.o nvkm-y += nvkm/engine/disp/hdmigk104.o +nvkm-y += nvkm/engine/disp/hdmigm200.o nvkm-y += nvkm/engine/disp/hdmigv100.o nvkm-y += nvkm/engine/disp/conn.o diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c index 32fa94a9773f..cbd33e87b799 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/base.c @@ -275,6 +275,7 @@ nvkm_disp_oneinit(struct nvkm_engine *engine) struct nvkm_outp *outp, *outt, *pair; struct nvkm_conn *conn; struct nvkm_head *head; + struct nvkm_ior *ior; struct nvbios_connE connE; struct dcb_output dcbE; u8 hpd = 0, ver, hdr; @@ -399,6 +400,19 @@ nvkm_disp_oneinit(struct nvkm_engine *engine) return ret; } + /* Enforce identity-mapped SOR assignment for panels, which have + * certain bits (ie. backlight controls) wired to a specific SOR. + */ + list_for_each_entry(outp, &disp->outp, head) { + if (outp->conn->info.type == DCB_CONNECTOR_LVDS || + outp->conn->info.type == DCB_CONNECTOR_eDP) { + ior = nvkm_ior_find(disp, SOR, ffs(outp->info.or) - 1); + if (!WARN_ON(!ior)) + ior->identity = true; + outp->identity = true; + } + } + i = 0; list_for_each_entry(head, &disp->head, head) i = max(i, head->id + 1); diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c index 7c5bed29ffef..5f301e632599 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/dp.c @@ -28,6 +28,7 @@ #include <subdev/bios.h> #include <subdev/bios/init.h> +#include <subdev/gpio.h> #include <subdev/i2c.h> #include <nvif/event.h> @@ -412,14 +413,10 @@ nvkm_dp_train(struct nvkm_dp *dp, u32 dataKBps) } static void -nvkm_dp_release(struct nvkm_outp *outp, struct nvkm_ior *ior) +nvkm_dp_disable(struct nvkm_outp *outp, struct nvkm_ior *ior) { struct nvkm_dp *dp = nvkm_dp(outp); - /* Prevent link from being retrained if sink sends an IRQ. */ - atomic_set(&dp->lt.done, 0); - ior->dp.nr = 0; - /* Execute DisableLT script from DP Info Table. */ nvbios_init(&ior->disp->engine.subdev, dp->info.script[4], init.outp = &dp->outp.info; @@ -428,6 +425,16 @@ nvkm_dp_release(struct nvkm_outp *outp, struct nvkm_ior *ior) ); } +static void +nvkm_dp_release(struct nvkm_outp *outp) +{ + struct nvkm_dp *dp = nvkm_dp(outp); + + /* Prevent link from being retrained if sink sends an IRQ. */ + atomic_set(&dp->lt.done, 0); + dp->outp.ior->dp.nr = 0; +} + static int nvkm_dp_acquire(struct nvkm_outp *outp) { @@ -491,7 +498,7 @@ done: return ret; } -static void +static bool nvkm_dp_enable(struct nvkm_dp *dp, bool enable) { struct nvkm_i2c_aux *aux = dp->aux; @@ -505,7 +512,7 @@ nvkm_dp_enable(struct nvkm_dp *dp, bool enable) if (!nvkm_rdaux(aux, DPCD_RC00_DPCD_REV, dp->dpcd, sizeof(dp->dpcd))) - return; + return true; } if (dp->present) { @@ -515,6 +522,7 @@ nvkm_dp_enable(struct nvkm_dp *dp, bool enable) } atomic_set(&dp->lt.done, 0); + return false; } static int @@ -555,9 +563,38 @@ nvkm_dp_fini(struct nvkm_outp *outp) static void nvkm_dp_init(struct nvkm_outp *outp) { + struct nvkm_gpio *gpio = outp->disp->engine.subdev.device->gpio; struct nvkm_dp *dp = nvkm_dp(outp); + nvkm_notify_put(&dp->outp.conn->hpd); - nvkm_dp_enable(dp, true); + + /* eDP panels need powering on by us (if the VBIOS doesn't default it + * to on) before doing any AUX channel transactions. LVDS panel power + * is handled by the SOR itself, and not required for LVDS DDC. + */ + if (dp->outp.conn->info.type == DCB_CONNECTOR_eDP) { + int power = nvkm_gpio_get(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff); + if (power == 0) + nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, 1); + + /* We delay here unconditionally, even if already powered, + * because some laptop panels having a significant resume + * delay before the panel begins responding. + * + * This is likely a bit of a hack, but no better idea for + * handling this at the moment. + */ + msleep(300); + + /* If the eDP panel can't be detected, we need to restore + * the panel power GPIO to avoid breaking another output. + */ + if (!nvkm_dp_enable(dp, true) && power == 0) + nvkm_gpio_set(gpio, 0, DCB_GPIO_PANEL_POWER, 0xff, 0); + } else { + nvkm_dp_enable(dp, true); + } + nvkm_notify_get(&dp->hpd); } @@ -576,6 +613,7 @@ nvkm_dp_func = { .fini = nvkm_dp_fini, .acquire = nvkm_dp_acquire, .release = nvkm_dp_release, + .disable = nvkm_dp_disable, }; static int diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/hdmigm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/hdmigm200.c new file mode 100644 index 000000000000..9b16a08eb4d9 --- /dev/null +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/hdmigm200.c @@ -0,0 +1,36 @@ +/* + * Copyright 2018 Ilia Mirkin + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation + * the rights to use, copy, modify, merge, publish, distribute, sublicense, + * and/or sell copies of the Software, and to permit persons to whom the + * Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR + * OTHER DEALINGS IN THE SOFTWARE. + * + * Authors: Ilia Mirkin + */ +#include "hdmi.h" + +void +gm200_hdmi_scdc(struct nvkm_ior *ior, int head, u8 scdc) +{ + struct nvkm_device *device = ior->disp->engine.subdev.device; + const u32 hoff = head * 0x800; + const u32 ctrl = scdc & 0x3; + + nvkm_mask(device, 0x61c5bc + hoff, 0x00000003, ctrl); + + ior->tmds.high_speed = !!(scdc & 0x2); +} diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h index e0b4e0c5704e..0f0c86c32ec3 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/ior.h @@ -16,6 +16,7 @@ struct nvkm_ior { char name[8]; struct list_head head; + bool identity; struct nvkm_ior_state { struct nvkm_outp *outp; @@ -40,6 +41,11 @@ struct nvkm_ior { u8 nr; u8 bw; } dp; + + /* Armed TMDS state. */ + struct { + bool high_speed; + } tmds; }; struct nvkm_ior_func { @@ -60,6 +66,7 @@ struct nvkm_ior_func { void (*ctrl)(struct nvkm_ior *, int head, bool enable, u8 max_ac_packet, u8 rekey, u8 *avi, u8 avi_size, u8 *vendor, u8 vendor_size); + void (*scdc)(struct nvkm_ior *, int head, u8 scdc); } hdmi; struct { @@ -143,6 +150,8 @@ void gf119_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8, u8 *, u8 , u8 *, u8); void gk104_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8, u8 *, u8 , u8 *, u8); void gv100_hdmi_ctrl(struct nvkm_ior *, int, bool, u8, u8, u8 *, u8 , u8 *, u8); +void gm200_hdmi_scdc(struct nvkm_ior *, int, u8); + void gt215_hda_hpd(struct nvkm_ior *, int, bool); void gt215_hda_eld(struct nvkm_ior *, u8 *, u8); diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c index f89c7b977aa5..def005dd5fda 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/nv50.c @@ -501,11 +501,11 @@ nv50_disp_super_2_0(struct nv50_disp *disp, struct nvkm_head *head) nv50_disp_super_ied_off(head, ior, 2); /* If we're shutting down the OR's only active head, execute - * the output path's release function. + * the output path's disable function. */ if (ior->arm.head == (1 << head->id)) { - if ((outp = ior->arm.outp) && outp->func->release) - outp->func->release(outp, ior); + if ((outp = ior->arm.outp) && outp->func->disable) + outp->func->disable(outp, ior); } } diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.c index be9e7f8c3b23..c62030c96fba 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.c @@ -93,6 +93,8 @@ nvkm_outp_release(struct nvkm_outp *outp, u8 user) if (ior) { outp->acquired &= ~user; if (!outp->acquired) { + if (outp->func->release && outp->ior) + outp->func->release(outp); outp->ior->asy.outp = NULL; outp->ior = NULL; } @@ -127,17 +129,26 @@ nvkm_outp_acquire(struct nvkm_outp *outp, u8 user) if (proto == UNKNOWN) return -ENOSYS; + /* Deal with panels requiring identity-mapped SOR assignment. */ + if (outp->identity) { + ior = nvkm_ior_find(outp->disp, SOR, ffs(outp->info.or) - 1); + if (WARN_ON(!ior)) + return -ENOSPC; + return nvkm_outp_acquire_ior(outp, user, ior); + } + /* First preference is to reuse the OR that is currently armed * on HW, if any, in order to prevent unnecessary switching. */ list_for_each_entry(ior, &outp->disp->ior, head) { - if (!ior->asy.outp && ior->arm.outp == outp) + if (!ior->identity && !ior->asy.outp && ior->arm.outp == outp) return nvkm_outp_acquire_ior(outp, user, ior); } /* Failing that, a completely unused OR is the next best thing. */ list_for_each_entry(ior, &outp->disp->ior, head) { - if (!ior->asy.outp && ior->type == type && !ior->arm.outp && + if (!ior->identity && + !ior->asy.outp && ior->type == type && !ior->arm.outp && (ior->func->route.set || ior->id == __ffs(outp->info.or))) return nvkm_outp_acquire_ior(outp, user, ior); } @@ -146,7 +157,7 @@ nvkm_outp_acquire(struct nvkm_outp *outp, u8 user) * but will be released during the next modeset. */ list_for_each_entry(ior, &outp->disp->ior, head) { - if (!ior->asy.outp && ior->type == type && + if (!ior->identity && !ior->asy.outp && ior->type == type && (ior->func->route.set || ior->id == __ffs(outp->info.or))) return nvkm_outp_acquire_ior(outp, user, ior); } @@ -245,7 +256,6 @@ nvkm_outp_ctor(const struct nvkm_outp_func *func, struct nvkm_disp *disp, outp->index = index; outp->info = *dcbE; outp->i2c = nvkm_i2c_bus_find(i2c, dcbE->i2c_index); - outp->or = ffs(outp->info.or) - 1; OUTP_DBG(outp, "type %02x loc %d or %d link %d con %x " "edid %x bus %d head %x", diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h b/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h index ea84d7d5741a..6c8aa5cfed9d 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/outp.h @@ -13,10 +13,10 @@ struct nvkm_outp { struct dcb_output info; struct nvkm_i2c_bus *i2c; - int or; struct list_head head; struct nvkm_conn *conn; + bool identity; /* Assembly state. */ #define NVKM_OUTP_PRIV 1 @@ -41,7 +41,8 @@ struct nvkm_outp_func { void (*init)(struct nvkm_outp *); void (*fini)(struct nvkm_outp *); int (*acquire)(struct nvkm_outp *); - void (*release)(struct nvkm_outp *, struct nvkm_ior *); + void (*release)(struct nvkm_outp *); + void (*disable)(struct nvkm_outp *, struct nvkm_ior *); }; #define OUTP_MSG(o,l,f,a...) do { \ diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv50.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv50.c index 3aa5a2879239..5f758948d6e1 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv50.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/rootnv50.c @@ -176,9 +176,10 @@ nv50_disp_root_mthd_(struct nvkm_object *object, u32 mthd, void *data, u32 size) nvif_ioctl(object, "disp sor hdmi ctrl size %d\n", size); if (!(ret = nvif_unpack(ret, &data, &size, args->v0, 0, 0, true))) { nvif_ioctl(object, "disp sor hdmi ctrl vers %d state %d " - "max_ac_packet %d rekey %d\n", + "max_ac_packet %d rekey %d scdc %d\n", args->v0.version, args->v0.state, - args->v0.max_ac_packet, args->v0.rekey); + args->v0.max_ac_packet, args->v0.rekey, + args->v0.scdc); if (args->v0.max_ac_packet > 0x1f || args->v0.rekey > 0x7f) return -EINVAL; if ((args->v0.avi_infoframe_length @@ -202,6 +203,11 @@ nv50_disp_root_mthd_(struct nvkm_object *object, u32 mthd, void *data, u32 size) args->v0.max_ac_packet, args->v0.rekey, avi, avi_size, vendor, vendor_size); + + if (outp->ior->func->hdmi.scdc) + outp->ior->func->hdmi.scdc( + outp->ior, hidx, args->v0.scdc); + return 0; } break; diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgf119.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgf119.c index e6e6dfbb1283..456a5a143522 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgf119.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgf119.c @@ -120,13 +120,16 @@ void gf119_sor_clock(struct nvkm_ior *sor) { struct nvkm_device *device = sor->disp->engine.subdev.device; - const int div = sor->asy.link == 3; const u32 soff = nv50_ior_base(sor); + u32 div1 = sor->asy.link == 3; + u32 div2 = sor->asy.link == 3; if (sor->asy.proto == TMDS) { - /* NFI why, but this sets DP_LINK_BW_2_7 when using TMDS. */ - nvkm_mask(device, 0x612300 + soff, 0x007c0000, 0x0a << 18); + const u32 speed = sor->tmds.high_speed ? 0x14 : 0x0a; + nvkm_mask(device, 0x612300 + soff, 0x007c0000, speed << 18); + if (sor->tmds.high_speed) + div2 = 1; } - nvkm_mask(device, 0x612300 + soff, 0x00000707, (div << 8) | div); + nvkm_mask(device, 0x612300 + soff, 0x00000707, (div2 << 8) | div1); } void diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgm200.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgm200.c index d892bdf04034..384f82652bec 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgm200.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgm200.c @@ -99,6 +99,7 @@ gm200_sor = { .clock = gf119_sor_clock, .hdmi = { .ctrl = gk104_hdmi_ctrl, + .scdc = gm200_hdmi_scdc, }, .dp = { .lanes = { 0, 1, 2, 3 }, diff --git a/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgv100.c b/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgv100.c index 040db8a338de..8ba881a729ee 100644 --- a/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgv100.c +++ b/drivers/gpu/drm/nouveau/nvkm/engine/disp/sorgv100.c @@ -88,6 +88,7 @@ gv100_sor = { .clock = gf119_sor_clock, .hdmi = { .ctrl = gv100_hdmi_ctrl, + .scdc = gm200_hdmi_scdc, }, .dp = { .lanes = { 0, 1, 2, 3 }, diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c index b80618e35491..17235e940ca9 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/devinit/gm200.c @@ -86,10 +86,8 @@ pmu_load(struct nv50_devinit *init, u8 type, bool post, struct nvkm_bios *bios = subdev->device->bios; struct nvbios_pmuR pmu; - if (!nvbios_pmuRm(bios, type, &pmu)) { - nvkm_error(subdev, "VBIOS PMU fuc %02x not found\n", type); + if (!nvbios_pmuRm(bios, type, &pmu)) return -EINVAL; - } if (!post) return 0; @@ -124,29 +122,30 @@ gm200_devinit_post(struct nvkm_devinit *base, bool post) return -EINVAL; } + /* Upload DEVINIT application from VBIOS onto PMU. */ ret = pmu_load(init, 0x04, post, &exec, &args); - if (ret) + if (ret) { + nvkm_error(subdev, "VBIOS PMU/DEVINIT not found\n"); return ret; + } - /* upload first chunk of init data */ + /* Upload tables required by opcodes in boot scripts. */ if (post) { - // devinit tables u32 pmu = pmu_args(init, args + 0x08, 0x08); u32 img = nvbios_rd16(bios, bit_I.offset + 0x14); u32 len = nvbios_rd16(bios, bit_I.offset + 0x16); pmu_data(init, pmu, img, len); } - /* upload second chunk of init data */ + /* Upload boot scripts. */ if (post) { - // devinit boot scripts u32 pmu = pmu_args(init, args + 0x08, 0x10); u32 img = nvbios_rd16(bios, bit_I.offset + 0x18); u32 len = nvbios_rd16(bios, bit_I.offset + 0x1a); pmu_data(init, pmu, img, len); } - /* execute init tables */ + /* Execute DEVINIT. */ if (post) { nvkm_wr32(device, 0x10a040, 0x00005000); pmu_exec(init, exec); @@ -157,8 +156,11 @@ gm200_devinit_post(struct nvkm_devinit *base, bool post) return -ETIMEDOUT; } - /* load and execute some other ucode image (bios therm?) */ - return pmu_load(init, 0x01, post, NULL, NULL); + /* Optional: Execute PRE_OS application on PMU, which should at + * least take care of fans until a full PMU has been loaded. + */ + pmu_load(init, 0x01, post, NULL, NULL); + return 0; } static const struct nvkm_devinit_func diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c b/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c index de269eb482dd..7459def78d50 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/mmu/vmm.c @@ -1423,7 +1423,7 @@ nvkm_vmm_get(struct nvkm_vmm *vmm, u8 page, u64 size, struct nvkm_vma **pvma) void nvkm_vmm_part(struct nvkm_vmm *vmm, struct nvkm_memory *inst) { - if (vmm->func->part && inst) { + if (inst && vmm->func->part) { mutex_lock(&vmm->mutex); vmm->func->part(vmm, inst); mutex_unlock(&vmm->mutex); diff --git a/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/acr_r352.c b/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/acr_r352.c index d02e183717dc..5c14d6ac855d 100644 --- a/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/acr_r352.c +++ b/drivers/gpu/drm/nouveau/nvkm/subdev/secboot/acr_r352.c @@ -801,6 +801,7 @@ acr_r352_load(struct nvkm_acr *_acr, struct nvkm_falcon *falcon, bl = acr->hsbl_unload_blob; } else { nvkm_error(_acr->subdev, "invalid secure boot blob!\n"); + kfree(bl_desc); return -EINVAL; } diff --git a/drivers/gpu/drm/omapdrm/dss/dispc.c b/drivers/gpu/drm/omapdrm/dss/dispc.c index e61a9592a650..ba82d916719c 100644 --- a/drivers/gpu/drm/omapdrm/dss/dispc.c +++ b/drivers/gpu/drm/omapdrm/dss/dispc.c @@ -1140,18 +1140,6 @@ static void dispc_ovl_set_color_mode(struct dispc_device *dispc, REG_FLD_MOD(dispc, DISPC_OVL_ATTRIBUTES(plane), m, 4, 1); } -static bool format_is_yuv(u32 fourcc) -{ - switch (fourcc) { - case DRM_FORMAT_YUYV: - case DRM_FORMAT_UYVY: - case DRM_FORMAT_NV12: - return true; - default: - return false; - } -} - static void dispc_ovl_configure_burst_type(struct dispc_device *dispc, enum omap_plane_id plane, enum omap_dss_rotation_type rotation) @@ -1910,11 +1898,14 @@ static void dispc_ovl_set_scaling_uv(struct dispc_device *dispc, int scale_x = out_width != orig_width; int scale_y = out_height != orig_height; bool chroma_upscale = plane != OMAP_DSS_WB; + const struct drm_format_info *info; + + info = drm_format_info(fourcc); if (!dispc_has_feature(dispc, FEAT_HANDLE_UV_SEPARATE)) return; - if (!format_is_yuv(fourcc)) { + if (!info->is_yuv) { /* reset chroma resampling for RGB formats */ if (plane != OMAP_DSS_WB) REG_FLD_MOD(dispc, DISPC_OVL_ATTRIBUTES2(plane), @@ -2624,7 +2615,7 @@ static int dispc_ovl_setup_common(struct dispc_device *dispc, unsigned int offset0, offset1; s32 row_inc; s32 pix_inc; - u16 frame_width, frame_height; + u16 frame_width; unsigned int field_offset = 0; u16 in_height = height; u16 in_width = width; @@ -2632,6 +2623,9 @@ static int dispc_ovl_setup_common(struct dispc_device *dispc, bool ilace = !!(vm->flags & DISPLAY_FLAGS_INTERLACED); unsigned long pclk = dispc_plane_pclk_rate(dispc, plane); unsigned long lclk = dispc_plane_lclk_rate(dispc, plane); + const struct drm_format_info *info; + + info = drm_format_info(fourcc); /* when setting up WB, dispc_plane_pclk_rate() returns 0 */ if (plane == OMAP_DSS_WB) @@ -2640,7 +2634,7 @@ static int dispc_ovl_setup_common(struct dispc_device *dispc, if (paddr == 0 && rotation_type != OMAP_DSS_ROT_TILER) return -EINVAL; - if (format_is_yuv(fourcc) && (in_width & 1)) { + if (info->is_yuv && (in_width & 1)) { DSSERR("input width %d is not even for YUV format\n", in_width); return -EINVAL; } @@ -2680,7 +2674,7 @@ static int dispc_ovl_setup_common(struct dispc_device *dispc, DSSDBG("predecimation %d x %x, new input size %d x %d\n", x_predecim, y_predecim, in_width, in_height); - if (format_is_yuv(fourcc) && (in_width & 1)) { + if (info->is_yuv && (in_width & 1)) { DSSDBG("predecimated input width is not even for YUV format\n"); DSSDBG("adjusting input width %d -> %d\n", in_width, in_width & ~1); @@ -2688,7 +2682,7 @@ static int dispc_ovl_setup_common(struct dispc_device *dispc, in_width &= ~1; } - if (format_is_yuv(fourcc)) + if (info->is_yuv) cconv = 1; if (ilace && !fieldmode) { @@ -2714,13 +2708,10 @@ static int dispc_ovl_setup_common(struct dispc_device *dispc, row_inc = 0; pix_inc = 0; - if (plane == OMAP_DSS_WB) { + if (plane == OMAP_DSS_WB) frame_width = out_width; - frame_height = out_height; - } else { + else frame_width = in_width; - frame_height = height; - } calc_offset(screen_width, frame_width, fourcc, fieldmode, field_offset, diff --git a/drivers/gpu/drm/omapdrm/dss/dss.c b/drivers/gpu/drm/omapdrm/dss/dss.c index 19fc4dfc429e..1aaf260aa9b8 100644 --- a/drivers/gpu/drm/omapdrm/dss/dss.c +++ b/drivers/gpu/drm/omapdrm/dss/dss.c @@ -947,7 +947,7 @@ dss_debugfs_create_file(struct dss_device *dss, const char *name, &dss_debug_fops); if (IS_ERR(d)) { kfree(entry); - return ERR_PTR(PTR_ERR(d)); + return ERR_CAST(d); } entry->dentry = d; diff --git a/drivers/gpu/drm/omapdrm/omap_connector.c b/drivers/gpu/drm/omapdrm/omap_connector.c index 98f5ca29444a..b81302c4bf9e 100644 --- a/drivers/gpu/drm/omapdrm/omap_connector.c +++ b/drivers/gpu/drm/omapdrm/omap_connector.c @@ -164,10 +164,11 @@ static void omap_connector_destroy(struct drm_connector *connector) drm_connector_unregister(connector); drm_connector_cleanup(connector); - kfree(omap_connector); omapdss_device_put(omap_connector->output); omapdss_device_put(omap_connector->display); + + kfree(omap_connector); } #define MAX_EDID 512 diff --git a/drivers/gpu/drm/omapdrm/omap_dmm_priv.h b/drivers/gpu/drm/omapdrm/omap_dmm_priv.h index c2785cc98dc9..60bb3f9297bc 100644 --- a/drivers/gpu/drm/omapdrm/omap_dmm_priv.h +++ b/drivers/gpu/drm/omapdrm/omap_dmm_priv.h @@ -159,6 +159,7 @@ struct dmm_platform_data { struct dmm { struct device *dev; + dma_addr_t phys_base; void __iomem *base; int irq; @@ -189,6 +190,12 @@ struct dmm { struct list_head alloc_head; const struct dmm_platform_data *plat_data; + + bool dmm_workaround; + spinlock_t wa_lock; + u32 *wa_dma_data; + dma_addr_t wa_dma_handle; + struct dma_chan *wa_dma_chan; }; #endif diff --git a/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c b/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c index f92fe205550b..252f5ebb1acc 100644 --- a/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c +++ b/drivers/gpu/drm/omapdrm/omap_dmm_tiler.c @@ -18,6 +18,7 @@ #include <linux/completion.h> #include <linux/delay.h> #include <linux/dma-mapping.h> +#include <linux/dmaengine.h> #include <linux/errno.h> #include <linux/init.h> #include <linux/interrupt.h> @@ -79,14 +80,138 @@ static const u32 reg[][4] = { DMM_PAT_DESCR__2, DMM_PAT_DESCR__3}, }; +static int dmm_dma_copy(struct dmm *dmm, dma_addr_t src, dma_addr_t dst) +{ + struct dma_device *dma_dev = dmm->wa_dma_chan->device; + struct dma_async_tx_descriptor *tx; + enum dma_status status; + dma_cookie_t cookie; + + tx = dma_dev->device_prep_dma_memcpy(dmm->wa_dma_chan, dst, src, 4, 0); + if (!tx) { + dev_err(dmm->dev, "Failed to prepare DMA memcpy\n"); + return -EIO; + } + + cookie = tx->tx_submit(tx); + if (dma_submit_error(cookie)) { + dev_err(dmm->dev, "Failed to do DMA tx_submit\n"); + return -EIO; + } + + dma_async_issue_pending(dmm->wa_dma_chan); + status = dma_sync_wait(dmm->wa_dma_chan, cookie); + if (status != DMA_COMPLETE) + dev_err(dmm->dev, "i878 wa DMA copy failure\n"); + + dmaengine_terminate_all(dmm->wa_dma_chan); + return 0; +} + +static u32 dmm_read_wa(struct dmm *dmm, u32 reg) +{ + dma_addr_t src, dst; + int r; + + src = dmm->phys_base + reg; + dst = dmm->wa_dma_handle; + + r = dmm_dma_copy(dmm, src, dst); + if (r) { + dev_err(dmm->dev, "sDMA read transfer timeout\n"); + return readl(dmm->base + reg); + } + + /* + * As per i878 workaround, the DMA is used to access the DMM registers. + * Make sure that the readl is not moved by the compiler or the CPU + * earlier than the DMA finished writing the value to memory. + */ + rmb(); + return readl(dmm->wa_dma_data); +} + +static void dmm_write_wa(struct dmm *dmm, u32 val, u32 reg) +{ + dma_addr_t src, dst; + int r; + + writel(val, dmm->wa_dma_data); + /* + * As per i878 workaround, the DMA is used to access the DMM registers. + * Make sure that the writel is not moved by the compiler or the CPU, so + * the data will be in place before we start the DMA to do the actual + * register write. + */ + wmb(); + + src = dmm->wa_dma_handle; + dst = dmm->phys_base + reg; + + r = dmm_dma_copy(dmm, src, dst); + if (r) { + dev_err(dmm->dev, "sDMA write transfer timeout\n"); + writel(val, dmm->base + reg); + } +} + static u32 dmm_read(struct dmm *dmm, u32 reg) { - return readl(dmm->base + reg); + if (dmm->dmm_workaround) { + u32 v; + unsigned long flags; + + spin_lock_irqsave(&dmm->wa_lock, flags); + v = dmm_read_wa(dmm, reg); + spin_unlock_irqrestore(&dmm->wa_lock, flags); + + return v; + } else { + return readl(dmm->base + reg); + } } static void dmm_write(struct dmm *dmm, u32 val, u32 reg) { - writel(val, dmm->base + reg); + if (dmm->dmm_workaround) { + unsigned long flags; + + spin_lock_irqsave(&dmm->wa_lock, flags); + dmm_write_wa(dmm, val, reg); + spin_unlock_irqrestore(&dmm->wa_lock, flags); + } else { + writel(val, dmm->base + reg); + } +} + +static int dmm_workaround_init(struct dmm *dmm) +{ + dma_cap_mask_t mask; + + spin_lock_init(&dmm->wa_lock); + + dmm->wa_dma_data = dma_alloc_coherent(dmm->dev, sizeof(u32), + &dmm->wa_dma_handle, GFP_KERNEL); + if (!dmm->wa_dma_data) + return -ENOMEM; + + dma_cap_zero(mask); + dma_cap_set(DMA_MEMCPY, mask); + + dmm->wa_dma_chan = dma_request_channel(mask, NULL, NULL); + if (!dmm->wa_dma_chan) { + dma_free_coherent(dmm->dev, 4, dmm->wa_dma_data, dmm->wa_dma_handle); + return -ENODEV; + } + + return 0; +} + +static void dmm_workaround_uninit(struct dmm *dmm) +{ + dma_release_channel(dmm->wa_dma_chan); + + dma_free_coherent(dmm->dev, 4, dmm->wa_dma_data, dmm->wa_dma_handle); } /* simple allocator to grab next 16 byte aligned memory from txn */ @@ -285,6 +410,17 @@ static int dmm_txn_commit(struct dmm_txn *txn, bool wait) } txn->last_pat->next_pa = 0; + /* ensure that the written descriptors are visible to DMM */ + wmb(); + + /* + * NOTE: the wmb() above should be enough, but there seems to be a bug + * in OMAP's memory barrier implementation, which in some rare cases may + * cause the writes not to be observable after wmb(). + */ + + /* read back to ensure the data is in RAM */ + readl(&txn->last_pat->next_pa); /* write to PAT_DESCR to clear out any pending transaction */ dmm_write(dmm, 0x0, reg[PAT_DESCR][engine->id]); @@ -603,6 +739,10 @@ static int omap_dmm_remove(struct platform_device *dev) unsigned long flags; if (omap_dmm) { + /* Disable all enabled interrupts */ + dmm_write(omap_dmm, 0x7e7e7e7e, DMM_PAT_IRQENABLE_CLR); + free_irq(omap_dmm->irq, omap_dmm); + /* free all area regions */ spin_lock_irqsave(&list_lock, flags); list_for_each_entry_safe(block, _block, &omap_dmm->alloc_head, @@ -625,8 +765,8 @@ static int omap_dmm_remove(struct platform_device *dev) if (omap_dmm->dummy_page) __free_page(omap_dmm->dummy_page); - if (omap_dmm->irq > 0) - free_irq(omap_dmm->irq, omap_dmm); + if (omap_dmm->dmm_workaround) + dmm_workaround_uninit(omap_dmm); iounmap(omap_dmm->base); kfree(omap_dmm); @@ -673,6 +813,7 @@ static int omap_dmm_probe(struct platform_device *dev) goto fail; } + omap_dmm->phys_base = mem->start; omap_dmm->base = ioremap(mem->start, SZ_2K); if (!omap_dmm->base) { @@ -688,6 +829,22 @@ static int omap_dmm_probe(struct platform_device *dev) omap_dmm->dev = &dev->dev; + if (of_machine_is_compatible("ti,dra7")) { + /* + * DRA7 Errata i878 says that MPU should not be used to access + * RAM and DMM at the same time. As it's not possible to prevent + * MPU accessing RAM, we need to access DMM via a proxy. + */ + if (!dmm_workaround_init(omap_dmm)) { + omap_dmm->dmm_workaround = true; + dev_info(&dev->dev, + "workaround for errata i878 in use\n"); + } else { + dev_warn(&dev->dev, + "failed to initialize work-around for i878\n"); + } + } + hwinfo = dmm_read(omap_dmm, DMM_PAT_HWINFO); omap_dmm->num_engines = (hwinfo >> 24) & 0x1F; omap_dmm->num_lut = (hwinfo >> 16) & 0x1F; @@ -714,24 +871,6 @@ static int omap_dmm_probe(struct platform_device *dev) dmm_write(omap_dmm, 0x88888888, DMM_TILER_OR__0); dmm_write(omap_dmm, 0x88888888, DMM_TILER_OR__1); - ret = request_irq(omap_dmm->irq, omap_dmm_irq_handler, IRQF_SHARED, - "omap_dmm_irq_handler", omap_dmm); - - if (ret) { - dev_err(&dev->dev, "couldn't register IRQ %d, error %d\n", - omap_dmm->irq, ret); - omap_dmm->irq = -1; - goto fail; - } - - /* Enable all interrupts for each refill engine except - * ERR_LUT_MISS<n> (which is just advisory, and we don't care - * about because we want to be able to refill live scanout - * buffers for accelerated pan/scroll) and FILL_DSC<n> which - * we just generally don't care about. - */ - dmm_write(omap_dmm, 0x7e7e7e7e, DMM_PAT_IRQENABLE_SET); - omap_dmm->dummy_page = alloc_page(GFP_KERNEL | __GFP_DMA32); if (!omap_dmm->dummy_page) { dev_err(&dev->dev, "could not allocate dummy page\n"); @@ -823,6 +962,24 @@ static int omap_dmm_probe(struct platform_device *dev) .p1.y = omap_dmm->container_height - 1, }; + ret = request_irq(omap_dmm->irq, omap_dmm_irq_handler, IRQF_SHARED, + "omap_dmm_irq_handler", omap_dmm); + + if (ret) { + dev_err(&dev->dev, "couldn't register IRQ %d, error %d\n", + omap_dmm->irq, ret); + omap_dmm->irq = -1; + goto fail; + } + + /* Enable all interrupts for each refill engine except + * ERR_LUT_MISS<n> (which is just advisory, and we don't care + * about because we want to be able to refill live scanout + * buffers for accelerated pan/scroll) and FILL_DSC<n> which + * we just generally don't care about. + */ + dmm_write(omap_dmm, 0x7e7e7e7e, DMM_PAT_IRQENABLE_SET); + /* initialize all LUTs to dummy page entries */ for (i = 0; i < omap_dmm->num_lut; i++) { area.tcm = omap_dmm->tcm[i]; diff --git a/drivers/gpu/drm/omapdrm/omap_drv.c b/drivers/gpu/drm/omapdrm/omap_drv.c index 5f98506ac2c5..5e67d58cbc28 100644 --- a/drivers/gpu/drm/omapdrm/omap_drv.c +++ b/drivers/gpu/drm/omapdrm/omap_drv.c @@ -439,7 +439,7 @@ static int ioctl_gem_info(struct drm_device *dev, void *data, args->size = omap_gem_mmap_size(obj); args->offset = omap_gem_mmap_offset(obj); - drm_gem_object_unreference_unlocked(obj); + drm_gem_object_put_unlocked(obj); return ret; } @@ -614,7 +614,7 @@ err_gem_deinit: omap_disconnect_pipelines(ddev); err_crtc_uninit: omap_crtc_pre_uninit(priv); - drm_dev_unref(ddev); + drm_dev_put(ddev); return ret; } @@ -643,7 +643,7 @@ static void omapdrm_cleanup(struct omap_drm_private *priv) omap_disconnect_pipelines(ddev); omap_crtc_pre_uninit(priv); - drm_dev_unref(ddev); + drm_dev_put(ddev); } static int pdev_probe(struct platform_device *pdev) diff --git a/drivers/gpu/drm/omapdrm/omap_fb.c b/drivers/gpu/drm/omapdrm/omap_fb.c index 9f1e3d8f8488..4d264fd554d8 100644 --- a/drivers/gpu/drm/omapdrm/omap_fb.c +++ b/drivers/gpu/drm/omapdrm/omap_fb.c @@ -319,7 +319,7 @@ struct drm_framebuffer *omap_framebuffer_create(struct drm_device *dev, error: while (--i >= 0) - drm_gem_object_unreference_unlocked(bos[i]); + drm_gem_object_put_unlocked(bos[i]); return fb; } diff --git a/drivers/gpu/drm/omapdrm/omap_fbdev.c b/drivers/gpu/drm/omapdrm/omap_fbdev.c index b445309b0143..aee99194499f 100644 --- a/drivers/gpu/drm/omapdrm/omap_fbdev.c +++ b/drivers/gpu/drm/omapdrm/omap_fbdev.c @@ -150,7 +150,7 @@ static int omap_fbdev_create(struct drm_fb_helper *helper, /* note: if fb creation failed, we can't rely on fb destroy * to unref the bo: */ - drm_gem_object_unreference_unlocked(fbdev->bo); + drm_gem_object_put_unlocked(fbdev->bo); ret = PTR_ERR(fb); goto fail; } diff --git a/drivers/gpu/drm/omapdrm/omap_gem.c b/drivers/gpu/drm/omapdrm/omap_gem.c index 4ba5d035c590..8dcaf9f4aa75 100644 --- a/drivers/gpu/drm/omapdrm/omap_gem.c +++ b/drivers/gpu/drm/omapdrm/omap_gem.c @@ -638,7 +638,7 @@ int omap_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev, *offset = omap_gem_mmap_offset(obj); - drm_gem_object_unreference_unlocked(obj); + drm_gem_object_put_unlocked(obj); fail: return ret; @@ -1312,7 +1312,7 @@ int omap_gem_new_handle(struct drm_device *dev, struct drm_file *file, } /* drop reference from allocate - handle holds it now */ - drm_gem_object_unreference_unlocked(obj); + drm_gem_object_put_unlocked(obj); return 0; } diff --git a/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c b/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c index ec04a69ade46..0f8b597ccd10 100644 --- a/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c +++ b/drivers/gpu/drm/omapdrm/omap_gem_dmabuf.c @@ -168,7 +168,7 @@ struct drm_gem_object *omap_gem_prime_import(struct drm_device *dev, * Importing dmabuf exported from out own gem increases * refcount on gem itself instead of f_count of dmabuf. */ - drm_gem_object_reference(obj); + drm_gem_object_get(obj); return obj; } } diff --git a/drivers/gpu/drm/omapdrm/tcm-sita.h b/drivers/gpu/drm/omapdrm/tcm-sita.h deleted file mode 100644 index 460e63dbf825..000000000000 --- a/drivers/gpu/drm/omapdrm/tcm-sita.h +++ /dev/null @@ -1,93 +0,0 @@ -/* - * SImple Tiler Allocator (SiTA) private structures. - * - * Copyright (C) 2009-2011 Texas Instruments Incorporated - http://www.ti.com/ - * Author: Ravi Ramachandra <r.ramachandra@ti.com> - * - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions - * are met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * * Neither the name of Texas Instruments Incorporated nor the names of - * its contributors may be used to endorse or promote products derived - * from this software without specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, - * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR - * PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR - * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, - * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, - * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; - * OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, - * WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR - * OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, - * EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef _TCM_SITA_H -#define _TCM_SITA_H - -#include "tcm.h" - -/* length between two coordinates */ -#define LEN(a, b) ((a) > (b) ? (a) - (b) + 1 : (b) - (a) + 1) - -enum criteria { - CR_MAX_NEIGHS = 0x01, - CR_FIRST_FOUND = 0x10, - CR_BIAS_HORIZONTAL = 0x20, - CR_BIAS_VERTICAL = 0x40, - CR_DIAGONAL_BALANCE = 0x80 -}; - -/* nearness to the beginning of the search field from 0 to 1000 */ -struct nearness_factor { - s32 x; - s32 y; -}; - -/* - * Statistics on immediately neighboring slots. Edge is the number of - * border segments that are also border segments of the scan field. Busy - * refers to the number of neighbors that are occupied. - */ -struct neighbor_stats { - u16 edge; - u16 busy; -}; - -/* structure to keep the score of a potential allocation */ -struct score { - struct nearness_factor f; - struct neighbor_stats n; - struct tcm_area a; - u16 neighs; /* number of busy neighbors */ -}; - -struct sita_pvt { - spinlock_t lock; /* spinlock to protect access */ - struct tcm_pt div_pt; /* divider point splitting container */ - struct tcm_area ***map; /* pointers to the parent area for each slot */ -}; - -/* assign coordinates to area */ -static inline -void assign(struct tcm_area *a, u16 x0, u16 y0, u16 x1, u16 y1) -{ - a->p0.x = x0; - a->p0.y = y0; - a->p1.x = x1; - a->p1.y = y1; -} - -#endif diff --git a/drivers/gpu/drm/panel/panel-simple.c b/drivers/gpu/drm/panel/panel-simple.c index 97964f7f2ace..a04ffb3b2174 100644 --- a/drivers/gpu/drm/panel/panel-simple.c +++ b/drivers/gpu/drm/panel/panel-simple.c @@ -56,6 +56,8 @@ struct panel_desc { /** * @prepare: the time (in milliseconds) that it takes for the panel to * become ready and start receiving video data + * @hpd_absent_delay: Add this to the prepare delay if we know Hot + * Plug Detect isn't used. * @enable: the time (in milliseconds) that it takes for the panel to * display the first valid frame after starting to receive * video data @@ -66,6 +68,7 @@ struct panel_desc { */ struct { unsigned int prepare; + unsigned int hpd_absent_delay; unsigned int enable; unsigned int disable; unsigned int unprepare; @@ -79,6 +82,7 @@ struct panel_simple { struct drm_panel base; bool prepared; bool enabled; + bool no_hpd; const struct panel_desc *desc; @@ -202,6 +206,7 @@ static int panel_simple_unprepare(struct drm_panel *panel) static int panel_simple_prepare(struct drm_panel *panel) { struct panel_simple *p = to_panel_simple(panel); + unsigned int delay; int err; if (p->prepared) @@ -215,8 +220,11 @@ static int panel_simple_prepare(struct drm_panel *panel) gpiod_set_value_cansleep(p->enable_gpio, 1); - if (p->desc->delay.prepare) - msleep(p->desc->delay.prepare); + delay = p->desc->delay.prepare; + if (p->no_hpd) + delay += p->desc->delay.hpd_absent_delay; + if (delay) + msleep(delay); p->prepared = true; @@ -305,6 +313,8 @@ static int panel_simple_probe(struct device *dev, const struct panel_desc *desc) panel->prepared = false; panel->desc = desc; + panel->no_hpd = of_property_read_bool(dev->of_node, "no-hpd"); + panel->supply = devm_regulator_get(dev, "power"); if (IS_ERR(panel->supply)) return PTR_ERR(panel->supply); @@ -1363,7 +1373,7 @@ static const struct panel_desc innolux_n156bge_l21 = { }, }; -static const struct drm_display_mode innolux_tv123wam_mode = { +static const struct drm_display_mode innolux_p120zdg_bf1_mode = { .clock = 206016, .hdisplay = 2160, .hsync_start = 2160 + 48, @@ -1377,15 +1387,16 @@ static const struct drm_display_mode innolux_tv123wam_mode = { .flags = DRM_MODE_FLAG_PHSYNC | DRM_MODE_FLAG_PVSYNC, }; -static const struct panel_desc innolux_tv123wam = { - .modes = &innolux_tv123wam_mode, +static const struct panel_desc innolux_p120zdg_bf1 = { + .modes = &innolux_p120zdg_bf1_mode, .num_modes = 1, .bpc = 8, .size = { - .width = 259, - .height = 173, + .width = 254, + .height = 169, }, .delay = { + .hpd_absent_delay = 200, .unprepare = 500, }, }; @@ -2445,8 +2456,8 @@ static const struct of_device_id platform_of_match[] = { .compatible = "innolux,n156bge-l21", .data = &innolux_n156bge_l21, }, { - .compatible = "innolux,tv123wam", - .data = &innolux_tv123wam, + .compatible = "innolux,p120zdg-bf1", + .data = &innolux_p120zdg_bf1, }, { .compatible = "innolux,zj070na-01p", .data = &innolux_zj070na_01p, diff --git a/drivers/gpu/drm/pl111/pl111_drv.c b/drivers/gpu/drm/pl111/pl111_drv.c index 47fe30223444..33e0483d62ae 100644 --- a/drivers/gpu/drm/pl111/pl111_drv.c +++ b/drivers/gpu/drm/pl111/pl111_drv.c @@ -194,8 +194,6 @@ static int pl111_modeset_init(struct drm_device *dev) drm_mode_config_reset(dev); - drm_fb_cma_fbdev_init(dev, priv->variant->fb_bpp, 0); - drm_kms_helper_poll_init(dev); goto finish; @@ -232,7 +230,6 @@ DEFINE_DRM_GEM_CMA_FOPS(drm_fops); static struct drm_driver pl111_drm_driver = { .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME | DRIVER_ATOMIC, - .lastclose = drm_fb_helper_lastclose, .ioctls = NULL, .fops = &drm_fops, .name = "pl111", @@ -332,6 +329,8 @@ static int pl111_amba_probe(struct amba_device *amba_dev, if (ret < 0) goto dev_put; + drm_fbdev_generic_setup(drm, priv->variant->fb_bpp); + return 0; dev_put: @@ -348,7 +347,6 @@ static int pl111_amba_remove(struct amba_device *amba_dev) struct pl111_drm_dev_private *priv = drm->dev_private; drm_dev_unregister(drm); - drm_fb_cma_fbdev_fini(drm); if (priv->panel) drm_panel_bridge_remove(priv->bridge); drm_mode_config_cleanup(drm); diff --git a/drivers/gpu/drm/pl111/pl111_vexpress.c b/drivers/gpu/drm/pl111/pl111_vexpress.c index a534b225e31b..5fa0441bb6df 100644 --- a/drivers/gpu/drm/pl111/pl111_vexpress.c +++ b/drivers/gpu/drm/pl111/pl111_vexpress.c @@ -111,7 +111,8 @@ static int vexpress_muxfpga_probe(struct platform_device *pdev) } static const struct of_device_id vexpress_muxfpga_match[] = { - { .compatible = "arm,vexpress-muxfpga", } + { .compatible = "arm,vexpress-muxfpga", }, + {} }; static struct platform_driver vexpress_muxfpga_driver = { diff --git a/drivers/gpu/drm/radeon/evergreen_cs.c b/drivers/gpu/drm/radeon/evergreen_cs.c index 54324330b91f..f471537c852f 100644 --- a/drivers/gpu/drm/radeon/evergreen_cs.c +++ b/drivers/gpu/drm/radeon/evergreen_cs.c @@ -2416,7 +2416,7 @@ static int evergreen_packet3_check(struct radeon_cs_parser *p, size = radeon_get_ib_value(p, idx+1+(i*8)+1); if (p->rdev && (size + offset) > radeon_bo_size(reloc->robj)) { /* force size to size of the buffer */ - dev_warn(p->dev, "vbo resource seems too big for the bo\n"); + dev_warn_ratelimited(p->dev, "vbo resource seems too big for the bo\n"); ib[idx+1+(i*8)+1] = radeon_bo_size(reloc->robj) - offset; } diff --git a/drivers/gpu/drm/radeon/mkregtable.c b/drivers/gpu/drm/radeon/mkregtable.c index ba704633b072..52a7246fed9e 100644 --- a/drivers/gpu/drm/radeon/mkregtable.c +++ b/drivers/gpu/drm/radeon/mkregtable.c @@ -1,4 +1,4 @@ -// SPDX-License-Identifier: GPL-2.0 +// SPDX-License-Identifier: MIT /* utility to create the register check tables * this includes inlined list.h safe for userspace. * diff --git a/drivers/gpu/drm/radeon/r100_track.h b/drivers/gpu/drm/radeon/r100_track.h index ad16a925f8d5..57e2b09784be 100644 --- a/drivers/gpu/drm/radeon/r100_track.h +++ b/drivers/gpu/drm/radeon/r100_track.h @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: GPL-2.0 */ +/* SPDX-License-Identifier: MIT */ #define R100_TRACK_MAX_TEXTURE 3 #define R200_TRACK_MAX_TEXTURE 6 diff --git a/drivers/gpu/drm/radeon/radeon_dp_mst.c b/drivers/gpu/drm/radeon/radeon_dp_mst.c index f920be236cc9..84b3ad2172a3 100644 --- a/drivers/gpu/drm/radeon/radeon_dp_mst.c +++ b/drivers/gpu/drm/radeon/radeon_dp_mst.c @@ -1,4 +1,4 @@ -// SPDX-License-Identifier: GPL-2.0 +// SPDX-License-Identifier: MIT #include <drm/drmP.h> #include <drm/drm_dp_mst_helper.h> diff --git a/drivers/gpu/drm/radeon/radeon_legacy_tv.c b/drivers/gpu/drm/radeon/radeon_legacy_tv.c index 611cf934b211..4278272e3191 100644 --- a/drivers/gpu/drm/radeon/radeon_legacy_tv.c +++ b/drivers/gpu/drm/radeon/radeon_legacy_tv.c @@ -1,4 +1,4 @@ -// SPDX-License-Identifier: GPL-2.0 +// SPDX-License-Identifier: MIT #include <drm/drmP.h> #include <drm/drm_crtc_helper.h> #include "radeon.h" diff --git a/drivers/gpu/drm/radeon/radeon_trace.h b/drivers/gpu/drm/radeon/radeon_trace.h index bc26efd1793e..0d84b8aafab3 100644 --- a/drivers/gpu/drm/radeon/radeon_trace.h +++ b/drivers/gpu/drm/radeon/radeon_trace.h @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: GPL-2.0 */ +/* SPDX-License-Identifier: MIT */ #if !defined(_RADEON_TRACE_H) || defined(TRACE_HEADER_MULTI_READ) #define _RADEON_TRACE_H_ diff --git a/drivers/gpu/drm/radeon/radeon_trace_points.c b/drivers/gpu/drm/radeon/radeon_trace_points.c index 66b3d5084662..65e92302f974 100644 --- a/drivers/gpu/drm/radeon/radeon_trace_points.c +++ b/drivers/gpu/drm/radeon/radeon_trace_points.c @@ -1,4 +1,4 @@ -// SPDX-License-Identifier: GPL-2.0 +// SPDX-License-Identifier: MIT /* Copyright Red Hat Inc 2010. * Author : Dave Airlie <airlied@redhat.com> */ diff --git a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c index 6288b9ad9e24..17741843cf51 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_crtc.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_crtc.c @@ -57,46 +57,12 @@ static void rcar_du_crtc_set(struct rcar_du_crtc *rcrtc, u32 reg, u32 set) rcar_du_read(rcdu, rcrtc->mmio_offset + reg) | set); } -static void rcar_du_crtc_clr_set(struct rcar_du_crtc *rcrtc, u32 reg, - u32 clr, u32 set) +void rcar_du_crtc_dsysr_clr_set(struct rcar_du_crtc *rcrtc, u32 clr, u32 set) { struct rcar_du_device *rcdu = rcrtc->group->dev; - u32 value = rcar_du_read(rcdu, rcrtc->mmio_offset + reg); - rcar_du_write(rcdu, rcrtc->mmio_offset + reg, (value & ~clr) | set); -} - -static int rcar_du_crtc_get(struct rcar_du_crtc *rcrtc) -{ - int ret; - - ret = clk_prepare_enable(rcrtc->clock); - if (ret < 0) - return ret; - - ret = clk_prepare_enable(rcrtc->extclock); - if (ret < 0) - goto error_clock; - - ret = rcar_du_group_get(rcrtc->group); - if (ret < 0) - goto error_group; - - return 0; - -error_group: - clk_disable_unprepare(rcrtc->extclock); -error_clock: - clk_disable_unprepare(rcrtc->clock); - return ret; -} - -static void rcar_du_crtc_put(struct rcar_du_crtc *rcrtc) -{ - rcar_du_group_put(rcrtc->group); - - clk_disable_unprepare(rcrtc->extclock); - clk_disable_unprepare(rcrtc->clock); + rcrtc->dsysr = (rcrtc->dsysr & ~clr) | set; + rcar_du_write(rcdu, rcrtc->mmio_offset + DSYSR, rcrtc->dsysr); } /* ----------------------------------------------------------------------------- @@ -294,6 +260,14 @@ static void rcar_du_crtc_set_display_timing(struct rcar_du_crtc *rcrtc) rcar_du_group_write(rcrtc->group, DPLLCR, dpllcr); escr = ESCR_DCLKSEL_DCLKIN | div; + } else if (rcdu->info->lvds_clk_mask & BIT(rcrtc->index)) { + /* + * Use the LVDS PLL output as the dot clock when outputting to + * the LVDS encoder on an SoC that supports this clock routing + * option. We use the clock directly in that case, without any + * additional divider. + */ + escr = ESCR_DCLKSEL_DCLKIN; } else { struct du_clk_params params = { .diff = (unsigned long)-1 }; @@ -546,6 +520,51 @@ static void rcar_du_crtc_setup(struct rcar_du_crtc *rcrtc) drm_crtc_vblank_on(&rcrtc->crtc); } +static int rcar_du_crtc_get(struct rcar_du_crtc *rcrtc) +{ + int ret; + + /* + * Guard against double-get, as the function is called from both the + * .atomic_enable() and .atomic_begin() handlers. + */ + if (rcrtc->initialized) + return 0; + + ret = clk_prepare_enable(rcrtc->clock); + if (ret < 0) + return ret; + + ret = clk_prepare_enable(rcrtc->extclock); + if (ret < 0) + goto error_clock; + + ret = rcar_du_group_get(rcrtc->group); + if (ret < 0) + goto error_group; + + rcar_du_crtc_setup(rcrtc); + rcrtc->initialized = true; + + return 0; + +error_group: + clk_disable_unprepare(rcrtc->extclock); +error_clock: + clk_disable_unprepare(rcrtc->clock); + return ret; +} + +static void rcar_du_crtc_put(struct rcar_du_crtc *rcrtc) +{ + rcar_du_group_put(rcrtc->group); + + clk_disable_unprepare(rcrtc->extclock); + clk_disable_unprepare(rcrtc->clock); + + rcrtc->initialized = false; +} + static void rcar_du_crtc_start(struct rcar_du_crtc *rcrtc) { bool interlaced; @@ -556,9 +575,9 @@ static void rcar_du_crtc_start(struct rcar_du_crtc *rcrtc) * actively driven). */ interlaced = rcrtc->crtc.mode.flags & DRM_MODE_FLAG_INTERLACE; - rcar_du_crtc_clr_set(rcrtc, DSYSR, DSYSR_TVM_MASK | DSYSR_SCM_MASK, - (interlaced ? DSYSR_SCM_INT_VIDEO : 0) | - DSYSR_TVM_MASTER); + rcar_du_crtc_dsysr_clr_set(rcrtc, DSYSR_TVM_MASK | DSYSR_SCM_MASK, + (interlaced ? DSYSR_SCM_INT_VIDEO : 0) | + DSYSR_TVM_MASTER); rcar_du_group_start_stop(rcrtc->group, true); } @@ -624,8 +643,13 @@ static void rcar_du_crtc_stop(struct rcar_du_crtc *rcrtc) /* * Select switch sync mode. This stops display operation and configures * the HSYNC and VSYNC signals as inputs. + * + * TODO: Find another way to stop the display for DUs that don't support + * TVM sync. */ - rcar_du_crtc_clr_set(rcrtc, DSYSR, DSYSR_TVM_MASK, DSYSR_TVM_SWITCH); + if (rcar_du_has(rcrtc->group->dev, RCAR_DU_FEATURE_TVM_SYNC)) + rcar_du_crtc_dsysr_clr_set(rcrtc, DSYSR_TVM_MASK, + DSYSR_TVM_SWITCH); rcar_du_group_start_stop(rcrtc->group, false); } @@ -639,16 +663,7 @@ static void rcar_du_crtc_atomic_enable(struct drm_crtc *crtc, { struct rcar_du_crtc *rcrtc = to_rcar_crtc(crtc); - /* - * If the CRTC has already been setup by the .atomic_begin() handler we - * can skip the setup stage. - */ - if (!rcrtc->initialized) { - rcar_du_crtc_get(rcrtc); - rcar_du_crtc_setup(rcrtc); - rcrtc->initialized = true; - } - + rcar_du_crtc_get(rcrtc); rcar_du_crtc_start(rcrtc); } @@ -667,7 +682,6 @@ static void rcar_du_crtc_atomic_disable(struct drm_crtc *crtc, } spin_unlock_irq(&crtc->dev->event_lock); - rcrtc->initialized = false; rcrtc->outputs = 0; } @@ -680,14 +694,17 @@ static void rcar_du_crtc_atomic_begin(struct drm_crtc *crtc, /* * If a mode set is in progress we can be called with the CRTC disabled. - * We then need to first setup the CRTC in order to configure planes. - * The .atomic_enable() handler will notice and skip the CRTC setup. + * We thus need to first get and setup the CRTC in order to configure + * planes. We must *not* put the CRTC in .atomic_flush(), as it must be + * kept awake until the .atomic_enable() call that will follow. The get + * operation in .atomic_enable() will in that case be a no-op, and the + * CRTC will be put later in .atomic_disable(). + * + * If a mode set is not in progress the CRTC is enabled, and the + * following get call will be a no-op. There is thus no need to belance + * it in .atomic_flush() either. */ - if (!rcrtc->initialized) { - rcar_du_crtc_get(rcrtc); - rcar_du_crtc_setup(rcrtc); - rcrtc->initialized = true; - } + rcar_du_crtc_get(rcrtc); if (rcar_du_has(rcrtc->group->dev, RCAR_DU_FEATURE_VSP1_SOURCE)) rcar_du_vsp_atomic_begin(rcrtc); @@ -1108,6 +1125,7 @@ int rcar_du_crtc_create(struct rcar_du_group *rgrp, unsigned int swindex, rcrtc->group = rgrp; rcrtc->mmio_offset = mmio_offsets[hwindex]; rcrtc->index = hwindex; + rcrtc->dsysr = (rcrtc->index % 2 ? 0 : DSYSR_DRES) | DSYSR_TVM_TVSYNC; if (rcar_du_has(rcdu, RCAR_DU_FEATURE_VSP1_SOURCE)) primary = &rcrtc->vsp->planes[rcrtc->vsp_pipe].plane; diff --git a/drivers/gpu/drm/rcar-du/rcar_du_crtc.h b/drivers/gpu/drm/rcar-du/rcar_du_crtc.h index 4990bbe9ba26..59ac6e7d22c9 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_crtc.h +++ b/drivers/gpu/drm/rcar-du/rcar_du_crtc.h @@ -30,6 +30,7 @@ struct rcar_du_vsp; * @mmio_offset: offset of the CRTC registers in the DU MMIO block * @index: CRTC software and hardware index * @initialized: whether the CRTC has been initialized and clocks enabled + * @dsysr: cached value of the DSYSR register * @vblank_enable: whether vblank events are enabled on this CRTC * @event: event to post when the pending page flip completes * @flip_wait: wait queue used to signal page flip completion @@ -50,6 +51,8 @@ struct rcar_du_crtc { unsigned int index; bool initialized; + u32 dsysr; + bool vblank_enable; struct drm_pending_vblank_event *event; wait_queue_head_t flip_wait; @@ -103,4 +106,6 @@ void rcar_du_crtc_route_output(struct drm_crtc *crtc, enum rcar_du_output output); void rcar_du_crtc_finish_page_flip(struct rcar_du_crtc *rcrtc); +void rcar_du_crtc_dsysr_clr_set(struct rcar_du_crtc *rcrtc, u32 clr, u32 set); + #endif /* __RCAR_DU_CRTC_H__ */ diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.c b/drivers/gpu/drm/rcar-du/rcar_du_drv.c index 0954ecd2f943..084f58df4a8c 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_drv.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.c @@ -36,7 +36,8 @@ static const struct rcar_du_device_info rzg1_du_r8a7743_info = { .gen = 2, .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(1) | BIT(0), .routes = { /* @@ -58,7 +59,8 @@ static const struct rcar_du_device_info rzg1_du_r8a7745_info = { .gen = 2, .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(1) | BIT(0), .routes = { /* @@ -77,7 +79,8 @@ static const struct rcar_du_device_info rzg1_du_r8a7745_info = { static const struct rcar_du_device_info rcar_du_r8a7779_info = { .gen = 2, - .features = RCAR_DU_FEATURE_INTERLACED, + .features = RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(1) | BIT(0), .routes = { /* @@ -99,7 +102,8 @@ static const struct rcar_du_device_info rcar_du_r8a7790_info = { .gen = 2, .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .quirks = RCAR_DU_QUIRK_ALIGN_128B, .channels_mask = BIT(2) | BIT(1) | BIT(0), .routes = { @@ -128,7 +132,8 @@ static const struct rcar_du_device_info rcar_du_r8a7791_info = { .gen = 2, .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(1) | BIT(0), .routes = { /* @@ -151,7 +156,8 @@ static const struct rcar_du_device_info rcar_du_r8a7792_info = { .gen = 2, .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(1) | BIT(0), .routes = { /* R8A7792 has two RGB outputs. */ @@ -170,7 +176,8 @@ static const struct rcar_du_device_info rcar_du_r8a7794_info = { .gen = 2, .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(1) | BIT(0), .routes = { /* @@ -193,7 +200,8 @@ static const struct rcar_du_device_info rcar_du_r8a7795_info = { .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS | RCAR_DU_FEATURE_VSP1_SOURCE - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(3) | BIT(2) | BIT(1) | BIT(0), .routes = { /* @@ -226,7 +234,8 @@ static const struct rcar_du_device_info rcar_du_r8a7796_info = { .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS | RCAR_DU_FEATURE_VSP1_SOURCE - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(2) | BIT(1) | BIT(0), .routes = { /* @@ -255,7 +264,8 @@ static const struct rcar_du_device_info rcar_du_r8a77965_info = { .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS | RCAR_DU_FEATURE_VSP1_SOURCE - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(3) | BIT(1) | BIT(0), .routes = { /* @@ -284,7 +294,8 @@ static const struct rcar_du_device_info rcar_du_r8a77970_info = { .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK | RCAR_DU_FEATURE_EXT_CTRL_REGS | RCAR_DU_FEATURE_VSP1_SOURCE - | RCAR_DU_FEATURE_INTERLACED, + | RCAR_DU_FEATURE_INTERLACED + | RCAR_DU_FEATURE_TVM_SYNC, .channels_mask = BIT(0), .routes = { /* R8A77970 has one RGB output and one LVDS output. */ @@ -300,6 +311,34 @@ static const struct rcar_du_device_info rcar_du_r8a77970_info = { .num_lvds = 1, }; +static const struct rcar_du_device_info rcar_du_r8a7799x_info = { + .gen = 3, + .features = RCAR_DU_FEATURE_CRTC_IRQ_CLOCK + | RCAR_DU_FEATURE_EXT_CTRL_REGS + | RCAR_DU_FEATURE_VSP1_SOURCE, + .channels_mask = BIT(1) | BIT(0), + .routes = { + /* + * R8A77990 and R8A77995 have one RGB output and two LVDS + * outputs. + */ + [RCAR_DU_OUTPUT_DPAD0] = { + .possible_crtcs = BIT(0) | BIT(1), + .port = 0, + }, + [RCAR_DU_OUTPUT_LVDS0] = { + .possible_crtcs = BIT(0), + .port = 1, + }, + [RCAR_DU_OUTPUT_LVDS1] = { + .possible_crtcs = BIT(1), + .port = 2, + }, + }, + .num_lvds = 2, + .lvds_clk_mask = BIT(1) | BIT(0), +}; + static const struct of_device_id rcar_du_of_table[] = { { .compatible = "renesas,du-r8a7743", .data = &rzg1_du_r8a7743_info }, { .compatible = "renesas,du-r8a7745", .data = &rzg1_du_r8a7745_info }, @@ -313,6 +352,8 @@ static const struct of_device_id rcar_du_of_table[] = { { .compatible = "renesas,du-r8a7796", .data = &rcar_du_r8a7796_info }, { .compatible = "renesas,du-r8a77965", .data = &rcar_du_r8a77965_info }, { .compatible = "renesas,du-r8a77970", .data = &rcar_du_r8a77970_info }, + { .compatible = "renesas,du-r8a77990", .data = &rcar_du_r8a7799x_info }, + { .compatible = "renesas,du-r8a77995", .data = &rcar_du_r8a7799x_info }, { } }; diff --git a/drivers/gpu/drm/rcar-du/rcar_du_drv.h b/drivers/gpu/drm/rcar-du/rcar_du_drv.h index fef9ea5c22f3..143c037e2c0f 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_drv.h +++ b/drivers/gpu/drm/rcar-du/rcar_du_drv.h @@ -27,6 +27,7 @@ struct rcar_du_device; #define RCAR_DU_FEATURE_EXT_CTRL_REGS BIT(1) /* Has extended control registers */ #define RCAR_DU_FEATURE_VSP1_SOURCE BIT(2) /* Has inputs from VSP1 */ #define RCAR_DU_FEATURE_INTERLACED BIT(3) /* HW supports interlaced */ +#define RCAR_DU_FEATURE_TVM_SYNC BIT(4) /* Has TV switch/sync modes */ #define RCAR_DU_QUIRK_ALIGN_128B BIT(0) /* Align pitches to 128 bytes */ @@ -53,6 +54,7 @@ struct rcar_du_output_routing { * @routes: array of CRTC to output routes, indexed by output (RCAR_DU_OUTPUT_*) * @num_lvds: number of internal LVDS encoders * @dpll_mask: bit mask of DU channels equipped with a DPLL + * @lvds_clk_mask: bitmask of channels that can use the LVDS clock as dot clock */ struct rcar_du_device_info { unsigned int gen; @@ -62,6 +64,7 @@ struct rcar_du_device_info { struct rcar_du_output_routing routes[RCAR_DU_OUTPUT_MAX]; unsigned int num_lvds; unsigned int dpll_mask; + unsigned int lvds_clk_mask; }; #define RCAR_DU_MAX_CRTCS 4 diff --git a/drivers/gpu/drm/rcar-du/rcar_du_group.c b/drivers/gpu/drm/rcar-du/rcar_du_group.c index ef2c177afb6d..d85f0a1c1581 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_group.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_group.c @@ -56,8 +56,6 @@ static void rcar_du_group_setup_pins(struct rcar_du_group *rgrp) static void rcar_du_group_setup_defr8(struct rcar_du_group *rgrp) { struct rcar_du_device *rcdu = rgrp->dev; - unsigned int possible_crtcs = - rcdu->info->routes[RCAR_DU_OUTPUT_DPAD0].possible_crtcs; u32 defr8 = DEFR8_CODE; if (rcdu->info->gen < 3) { @@ -69,26 +67,71 @@ static void rcar_du_group_setup_defr8(struct rcar_du_group *rgrp) * DU instances that support it. */ if (rgrp->index == 0) { - if (possible_crtcs > 1) - defr8 |= DEFR8_DRGBS_DU(rcdu->dpad0_source); + defr8 |= DEFR8_DRGBS_DU(rcdu->dpad0_source); if (rgrp->dev->vspd1_sink == 2) defr8 |= DEFR8_VSCS; } } else { /* - * On Gen3 VSPD routing can't be configured, but DPAD routing - * needs to be set despite having a single option available. + * On Gen3 VSPD routing can't be configured, and DPAD routing + * is set in the group corresponding to the DPAD output (no Gen3 + * SoC has multiple DPAD sources belonging to separate groups). */ - unsigned int rgb_crtc = ffs(possible_crtcs) - 1; - struct rcar_du_crtc *crtc = &rcdu->crtcs[rgb_crtc]; - - if (crtc->index / 2 == rgrp->index) - defr8 |= DEFR8_DRGBS_DU(crtc->index); + if (rgrp->index == rcdu->dpad0_source / 2) + defr8 |= DEFR8_DRGBS_DU(rcdu->dpad0_source); } rcar_du_group_write(rgrp, DEFR8, defr8); } +static void rcar_du_group_setup_didsr(struct rcar_du_group *rgrp) +{ + struct rcar_du_device *rcdu = rgrp->dev; + struct rcar_du_crtc *rcrtc; + unsigned int num_crtcs = 0; + unsigned int i; + u32 didsr; + + /* + * Configure input dot clock routing with a hardcoded configuration. If + * the DU channel can use the LVDS encoder output clock as the dot + * clock, do so. Otherwise route DU_DOTCLKINn signal to DUn. + * + * Each channel can then select between the dot clock configured here + * and the clock provided by the CPG through the ESCR register. + */ + if (rcdu->info->gen < 3 && rgrp->index == 0) { + /* + * On Gen2 a single register in the first group controls dot + * clock selection for all channels. + */ + rcrtc = rcdu->crtcs; + num_crtcs = rcdu->num_crtcs; + } else if (rcdu->info->gen == 3 && rgrp->num_crtcs > 1) { + /* + * On Gen3 dot clocks are setup through per-group registers, + * only available when the group has two channels. + */ + rcrtc = &rcdu->crtcs[rgrp->index * 2]; + num_crtcs = rgrp->num_crtcs; + } + + if (!num_crtcs) + return; + + didsr = DIDSR_CODE; + for (i = 0; i < num_crtcs; ++i, ++rcrtc) { + if (rcdu->info->lvds_clk_mask & BIT(rcrtc->index)) + didsr |= DIDSR_LCDS_LVDS0(i) + | DIDSR_PDCS_CLK(i, 0); + else + didsr |= DIDSR_LCDS_DCLKIN(i) + | DIDSR_PDCS_CLK(i, 0); + } + + rcar_du_group_write(rgrp, DIDSR, didsr); +} + static void rcar_du_group_setup(struct rcar_du_group *rgrp) { struct rcar_du_device *rcdu = rgrp->dev; @@ -106,21 +149,7 @@ static void rcar_du_group_setup(struct rcar_du_group *rgrp) if (rcar_du_has(rgrp->dev, RCAR_DU_FEATURE_EXT_CTRL_REGS)) { rcar_du_group_setup_defr8(rgrp); - - /* - * Configure input dot clock routing. We currently hardcode the - * configuration to routing DOTCLKINn to DUn. Register fields - * depend on the DU generation, but the resulting value is 0 in - * all cases. - * - * On Gen2 a single register in the first group controls dot - * clock selection for all channels, while on Gen3 dot clocks - * are setup through per-group registers, only available when - * the group has two channels. - */ - if ((rcdu->info->gen < 3 && rgrp->index == 0) || - (rcdu->info->gen == 3 && rgrp->num_crtcs > 1)) - rcar_du_group_write(rgrp, DIDSR, DIDSR_CODE); + rcar_du_group_setup_didsr(rgrp); } if (rcdu->info->gen >= 3) @@ -173,9 +202,10 @@ void rcar_du_group_put(struct rcar_du_group *rgrp) static void __rcar_du_group_start_stop(struct rcar_du_group *rgrp, bool start) { - rcar_du_group_write(rgrp, DSYSR, - (rcar_du_group_read(rgrp, DSYSR) & ~(DSYSR_DRES | DSYSR_DEN)) | - (start ? DSYSR_DEN : DSYSR_DRES)); + struct rcar_du_crtc *rcrtc = &rgrp->dev->crtcs[rgrp->index * 2]; + + rcar_du_crtc_dsysr_clr_set(rcrtc, DSYSR_DRES | DSYSR_DEN, + start ? DSYSR_DEN : DSYSR_DRES); } void rcar_du_group_start_stop(struct rcar_du_group *rgrp, bool start) diff --git a/drivers/gpu/drm/rcar-du/rcar_du_kms.c b/drivers/gpu/drm/rcar-du/rcar_du_kms.c index b5d79ecd25ea..4ebd61ecbee1 100644 --- a/drivers/gpu/drm/rcar-du/rcar_du_kms.c +++ b/drivers/gpu/drm/rcar-du/rcar_du_kms.c @@ -544,6 +544,7 @@ int rcar_du_modeset_init(struct rcar_du_device *rcdu) struct drm_device *dev = rcdu->ddev; struct drm_encoder *encoder; struct drm_fbdev_cma *fbdev; + unsigned int dpad0_sources; unsigned int num_encoders; unsigned int num_groups; unsigned int swindex; @@ -666,6 +667,17 @@ int rcar_du_modeset_init(struct rcar_du_device *rcdu) encoder->possible_clones = (1 << num_encoders) - 1; } + /* + * Initialize the default DPAD0 source to the index of the first DU + * channel that can be connected to DPAD0. The exact value doesn't + * matter as it should be overwritten by mode setting for the RGB + * output, but it is nonetheless required to ensure a valid initial + * hardware configuration on Gen3 where DU0 can't always be connected to + * DPAD0. + */ + dpad0_sources = rcdu->info->routes[RCAR_DU_OUTPUT_DPAD0].possible_crtcs; + rcdu->dpad0_source = ffs(dpad0_sources) - 1; + drm_mode_config_reset(dev); drm_kms_helper_poll_init(dev); diff --git a/drivers/gpu/drm/rcar-du/rcar_lvds.c b/drivers/gpu/drm/rcar-du/rcar_lvds.c index ce0eb68c3416..173d7ad0b991 100644 --- a/drivers/gpu/drm/rcar-du/rcar_lvds.c +++ b/drivers/gpu/drm/rcar-du/rcar_lvds.c @@ -24,6 +24,8 @@ #include "rcar_lvds_regs.h" +struct rcar_lvds; + /* Keep in sync with the LVDCR0.LVMD hardware register values. */ enum rcar_lvds_mode { RCAR_LVDS_MODE_JEIDA = 0, @@ -31,14 +33,16 @@ enum rcar_lvds_mode { RCAR_LVDS_MODE_VESA = 4, }; -#define RCAR_LVDS_QUIRK_LANES (1 << 0) /* LVDS lanes 1 and 3 inverted */ -#define RCAR_LVDS_QUIRK_GEN2_PLLCR (1 << 1) /* LVDPLLCR has gen2 layout */ -#define RCAR_LVDS_QUIRK_GEN3_LVEN (1 << 2) /* LVEN bit needs to be set */ - /* on R8A77970/R8A7799x */ +#define RCAR_LVDS_QUIRK_LANES BIT(0) /* LVDS lanes 1 and 3 inverted */ +#define RCAR_LVDS_QUIRK_GEN3_LVEN BIT(1) /* LVEN bit needs to be set on R8A77970/R8A7799x */ +#define RCAR_LVDS_QUIRK_PWD BIT(2) /* PWD bit available (all of Gen3 but E3) */ +#define RCAR_LVDS_QUIRK_EXT_PLL BIT(3) /* Has extended PLL */ +#define RCAR_LVDS_QUIRK_DUAL_LINK BIT(4) /* Supports dual-link operation */ struct rcar_lvds_device_info { unsigned int gen; unsigned int quirks; + void (*pll_setup)(struct rcar_lvds *lvds, unsigned int freq); }; struct rcar_lvds { @@ -52,7 +56,11 @@ struct rcar_lvds { struct drm_panel *panel; void __iomem *mmio; - struct clk *clock; + struct { + struct clk *mod; /* CPG module clock */ + struct clk *extal; /* External clock */ + struct clk *dotclkin[2]; /* External DU clocks */ + } clocks; bool enabled; struct drm_display_mode display_mode; @@ -128,33 +136,216 @@ static const struct drm_connector_funcs rcar_lvds_conn_funcs = { }; /* ----------------------------------------------------------------------------- - * Bridge + * PLL Setup */ -static u32 rcar_lvds_lvdpllcr_gen2(unsigned int freq) +static void rcar_lvds_pll_setup_gen2(struct rcar_lvds *lvds, unsigned int freq) { - if (freq < 39000) - return LVDPLLCR_CEEN | LVDPLLCR_COSEL | LVDPLLCR_PLLDLYCNT_38M; - else if (freq < 61000) - return LVDPLLCR_CEEN | LVDPLLCR_COSEL | LVDPLLCR_PLLDLYCNT_60M; - else if (freq < 121000) - return LVDPLLCR_CEEN | LVDPLLCR_COSEL | LVDPLLCR_PLLDLYCNT_121M; + u32 val; + + if (freq < 39000000) + val = LVDPLLCR_CEEN | LVDPLLCR_COSEL | LVDPLLCR_PLLDLYCNT_38M; + else if (freq < 61000000) + val = LVDPLLCR_CEEN | LVDPLLCR_COSEL | LVDPLLCR_PLLDLYCNT_60M; + else if (freq < 121000000) + val = LVDPLLCR_CEEN | LVDPLLCR_COSEL | LVDPLLCR_PLLDLYCNT_121M; else - return LVDPLLCR_PLLDLYCNT_150M; + val = LVDPLLCR_PLLDLYCNT_150M; + + rcar_lvds_write(lvds, LVDPLLCR, val); } -static u32 rcar_lvds_lvdpllcr_gen3(unsigned int freq) +static void rcar_lvds_pll_setup_gen3(struct rcar_lvds *lvds, unsigned int freq) { - if (freq < 42000) - return LVDPLLCR_PLLDIVCNT_42M; - else if (freq < 85000) - return LVDPLLCR_PLLDIVCNT_85M; - else if (freq < 128000) - return LVDPLLCR_PLLDIVCNT_128M; + u32 val; + + if (freq < 42000000) + val = LVDPLLCR_PLLDIVCNT_42M; + else if (freq < 85000000) + val = LVDPLLCR_PLLDIVCNT_85M; + else if (freq < 128000000) + val = LVDPLLCR_PLLDIVCNT_128M; else - return LVDPLLCR_PLLDIVCNT_148M; + val = LVDPLLCR_PLLDIVCNT_148M; + + rcar_lvds_write(lvds, LVDPLLCR, val); } +struct pll_info { + unsigned long diff; + unsigned int pll_m; + unsigned int pll_n; + unsigned int pll_e; + unsigned int div; + u32 clksel; +}; + +static void rcar_lvds_d3_e3_pll_calc(struct rcar_lvds *lvds, struct clk *clk, + unsigned long target, struct pll_info *pll, + u32 clksel) +{ + unsigned long output; + unsigned long fin; + unsigned int m_min; + unsigned int m_max; + unsigned int m; + int error; + + if (!clk) + return; + + /* + * The LVDS PLL is made of a pre-divider and a multiplier (strangely + * enough called M and N respectively), followed by a post-divider E. + * + * ,-----. ,-----. ,-----. ,-----. + * Fin --> | 1/M | -Fpdf-> | PFD | --> | VCO | -Fvco-> | 1/E | --> Fout + * `-----' ,-> | | `-----' | `-----' + * | `-----' | + * | ,-----. | + * `-------- | 1/N | <-------' + * `-----' + * + * The clock output by the PLL is then further divided by a programmable + * divider DIV to achieve the desired target frequency. Finally, an + * optional fixed /7 divider is used to convert the bit clock to a pixel + * clock (as LVDS transmits 7 bits per lane per clock sample). + * + * ,-------. ,-----. |\ + * Fout --> | 1/DIV | --> | 1/7 | --> | | + * `-------' | `-----' | | --> dot clock + * `------------> | | + * |/ + * + * The /7 divider is optional when the LVDS PLL is used to generate a + * dot clock for the DU RGB output, without using the LVDS encoder. We + * don't support this configuration yet. + * + * The PLL allowed input frequency range is 12 MHz to 192 MHz. + */ + + fin = clk_get_rate(clk); + if (fin < 12000000 || fin > 192000000) + return; + + /* + * The comparison frequency range is 12 MHz to 24 MHz, which limits the + * allowed values for the pre-divider M (normal range 1-8). + * + * Fpfd = Fin / M + */ + m_min = max_t(unsigned int, 1, DIV_ROUND_UP(fin, 24000000)); + m_max = min_t(unsigned int, 8, fin / 12000000); + + for (m = m_min; m <= m_max; ++m) { + unsigned long fpfd; + unsigned int n_min; + unsigned int n_max; + unsigned int n; + + /* + * The VCO operating range is 900 Mhz to 1800 MHz, which limits + * the allowed values for the multiplier N (normal range + * 60-120). + * + * Fvco = Fin * N / M + */ + fpfd = fin / m; + n_min = max_t(unsigned int, 60, DIV_ROUND_UP(900000000, fpfd)); + n_max = min_t(unsigned int, 120, 1800000000 / fpfd); + + for (n = n_min; n < n_max; ++n) { + unsigned long fvco; + unsigned int e_min; + unsigned int e; + + /* + * The output frequency is limited to 1039.5 MHz, + * limiting again the allowed values for the + * post-divider E (normal value 1, 2 or 4). + * + * Fout = Fvco / E + */ + fvco = fpfd * n; + e_min = fvco > 1039500000 ? 1 : 0; + + for (e = e_min; e < 3; ++e) { + unsigned long fout; + unsigned long diff; + unsigned int div; + + /* + * Finally we have a programable divider after + * the PLL, followed by a an optional fixed /7 + * divider. + */ + fout = fvco / (1 << e) / 7; + div = DIV_ROUND_CLOSEST(fout, target); + diff = abs(fout / div - target); + + if (diff < pll->diff) { + pll->diff = diff; + pll->pll_m = m; + pll->pll_n = n; + pll->pll_e = e; + pll->div = div; + pll->clksel = clksel; + + if (diff == 0) + goto done; + } + } + } + } + +done: + output = fin * pll->pll_n / pll->pll_m / (1 << pll->pll_e) + / 7 / pll->div; + error = (long)(output - target) * 10000 / (long)target; + + dev_dbg(lvds->dev, + "%pC %lu Hz -> Fout %lu Hz (target %lu Hz, error %d.%02u%%), PLL M/N/E/DIV %u/%u/%u/%u\n", + clk, fin, output, target, error / 100, + error < 0 ? -error % 100 : error % 100, + pll->pll_m, pll->pll_n, pll->pll_e, pll->div); +} + +static void rcar_lvds_pll_setup_d3_e3(struct rcar_lvds *lvds, unsigned int freq) +{ + struct pll_info pll = { .diff = (unsigned long)-1 }; + u32 lvdpllcr; + + rcar_lvds_d3_e3_pll_calc(lvds, lvds->clocks.dotclkin[0], freq, &pll, + LVDPLLCR_CKSEL_DU_DOTCLKIN(0)); + rcar_lvds_d3_e3_pll_calc(lvds, lvds->clocks.dotclkin[1], freq, &pll, + LVDPLLCR_CKSEL_DU_DOTCLKIN(1)); + rcar_lvds_d3_e3_pll_calc(lvds, lvds->clocks.extal, freq, &pll, + LVDPLLCR_CKSEL_EXTAL); + + lvdpllcr = LVDPLLCR_PLLON | pll.clksel | LVDPLLCR_CLKOUT + | LVDPLLCR_PLLN(pll.pll_n - 1) | LVDPLLCR_PLLM(pll.pll_m - 1); + + if (pll.pll_e > 0) + lvdpllcr |= LVDPLLCR_STP_CLKOUTE | LVDPLLCR_OUTCLKSEL + | LVDPLLCR_PLLE(pll.pll_e - 1); + + rcar_lvds_write(lvds, LVDPLLCR, lvdpllcr); + + if (pll.div > 1) + /* + * The DIVRESET bit is a misnomer, setting it to 1 deasserts the + * divisor reset. + */ + rcar_lvds_write(lvds, LVDDIV, LVDDIV_DIVSEL | + LVDDIV_DIVRESET | LVDDIV_DIV(pll.div - 1)); + else + rcar_lvds_write(lvds, LVDDIV, 0); +} + +/* ----------------------------------------------------------------------------- + * Bridge + */ + static void rcar_lvds_enable(struct drm_bridge *bridge) { struct rcar_lvds *lvds = bridge_to_rcar_lvds(bridge); @@ -164,14 +355,13 @@ static void rcar_lvds_enable(struct drm_bridge *bridge) * do we get a state pointer? */ struct drm_crtc *crtc = lvds->bridge.encoder->crtc; - u32 lvdpllcr; u32 lvdhcr; u32 lvdcr0; int ret; WARN_ON(lvds->enabled); - ret = clk_prepare_enable(lvds->clock); + ret = clk_prepare_enable(lvds->clocks.mod); if (ret < 0) return; @@ -196,12 +386,13 @@ static void rcar_lvds_enable(struct drm_bridge *bridge) rcar_lvds_write(lvds, LVDCHCR, lvdhcr); + if (lvds->info->quirks & RCAR_LVDS_QUIRK_DUAL_LINK) { + /* Disable dual-link mode. */ + rcar_lvds_write(lvds, LVDSTRIPE, 0); + } + /* PLL clock configuration. */ - if (lvds->info->quirks & RCAR_LVDS_QUIRK_GEN2_PLLCR) - lvdpllcr = rcar_lvds_lvdpllcr_gen2(mode->clock); - else - lvdpllcr = rcar_lvds_lvdpllcr_gen3(mode->clock); - rcar_lvds_write(lvds, LVDPLLCR, lvdpllcr); + lvds->info->pll_setup(lvds, mode->clock * 1000); /* Set the LVDS mode and select the input. */ lvdcr0 = lvds->mode << LVDCR0_LVMD_SHIFT; @@ -220,11 +411,16 @@ static void rcar_lvds_enable(struct drm_bridge *bridge) rcar_lvds_write(lvds, LVDCR0, lvdcr0); } - /* Turn the PLL on. */ - lvdcr0 |= LVDCR0_PLLON; - rcar_lvds_write(lvds, LVDCR0, lvdcr0); + if (!(lvds->info->quirks & RCAR_LVDS_QUIRK_EXT_PLL)) { + /* + * Turn the PLL on (simple PLL only, extended PLL is fully + * controlled through LVDPLLCR). + */ + lvdcr0 |= LVDCR0_PLLON; + rcar_lvds_write(lvds, LVDCR0, lvdcr0); + } - if (lvds->info->gen > 2) { + if (lvds->info->quirks & RCAR_LVDS_QUIRK_PWD) { /* Set LVDS normal mode. */ lvdcr0 |= LVDCR0_PWD; rcar_lvds_write(lvds, LVDCR0, lvdcr0); @@ -236,8 +432,10 @@ static void rcar_lvds_enable(struct drm_bridge *bridge) rcar_lvds_write(lvds, LVDCR0, lvdcr0); } - /* Wait for the startup delay. */ - usleep_range(100, 150); + if (!(lvds->info->quirks & RCAR_LVDS_QUIRK_EXT_PLL)) { + /* Wait for the PLL startup delay (simple PLL only). */ + usleep_range(100, 150); + } /* Turn the output on. */ lvdcr0 |= LVDCR0_LVRES; @@ -264,8 +462,9 @@ static void rcar_lvds_disable(struct drm_bridge *bridge) rcar_lvds_write(lvds, LVDCR0, 0); rcar_lvds_write(lvds, LVDCR1, 0); + rcar_lvds_write(lvds, LVDPLLCR, 0); - clk_disable_unprepare(lvds->clock); + clk_disable_unprepare(lvds->clocks.mod); lvds->enabled = false; } @@ -446,6 +645,60 @@ done: return ret; } +static struct clk *rcar_lvds_get_clock(struct rcar_lvds *lvds, const char *name, + bool optional) +{ + struct clk *clk; + + clk = devm_clk_get(lvds->dev, name); + if (!IS_ERR(clk)) + return clk; + + if (PTR_ERR(clk) == -ENOENT && optional) + return NULL; + + if (PTR_ERR(clk) != -EPROBE_DEFER) + dev_err(lvds->dev, "failed to get %s clock\n", + name ? name : "module"); + + return clk; +} + +static int rcar_lvds_get_clocks(struct rcar_lvds *lvds) +{ + lvds->clocks.mod = rcar_lvds_get_clock(lvds, NULL, false); + if (IS_ERR(lvds->clocks.mod)) + return PTR_ERR(lvds->clocks.mod); + + /* + * LVDS encoders without an extended PLL have no external clock inputs. + */ + if (!(lvds->info->quirks & RCAR_LVDS_QUIRK_EXT_PLL)) + return 0; + + lvds->clocks.extal = rcar_lvds_get_clock(lvds, "extal", true); + if (IS_ERR(lvds->clocks.extal)) + return PTR_ERR(lvds->clocks.extal); + + lvds->clocks.dotclkin[0] = rcar_lvds_get_clock(lvds, "dclkin.0", true); + if (IS_ERR(lvds->clocks.dotclkin[0])) + return PTR_ERR(lvds->clocks.dotclkin[0]); + + lvds->clocks.dotclkin[1] = rcar_lvds_get_clock(lvds, "dclkin.1", true); + if (IS_ERR(lvds->clocks.dotclkin[1])) + return PTR_ERR(lvds->clocks.dotclkin[1]); + + /* At least one input to the PLL must be available. */ + if (!lvds->clocks.extal && !lvds->clocks.dotclkin[0] && + !lvds->clocks.dotclkin[1]) { + dev_err(lvds->dev, + "no input clock (extal, dclkin.0 or dclkin.1)\n"); + return -EINVAL; + } + + return 0; +} + static int rcar_lvds_probe(struct platform_device *pdev) { struct rcar_lvds *lvds; @@ -475,11 +728,9 @@ static int rcar_lvds_probe(struct platform_device *pdev) if (IS_ERR(lvds->mmio)) return PTR_ERR(lvds->mmio); - lvds->clock = devm_clk_get(&pdev->dev, NULL); - if (IS_ERR(lvds->clock)) { - dev_err(&pdev->dev, "failed to get clock\n"); - return PTR_ERR(lvds->clock); - } + ret = rcar_lvds_get_clocks(lvds); + if (ret < 0) + return ret; drm_bridge_add(&lvds->bridge); @@ -497,21 +748,39 @@ static int rcar_lvds_remove(struct platform_device *pdev) static const struct rcar_lvds_device_info rcar_lvds_gen2_info = { .gen = 2, - .quirks = RCAR_LVDS_QUIRK_GEN2_PLLCR, + .pll_setup = rcar_lvds_pll_setup_gen2, }; static const struct rcar_lvds_device_info rcar_lvds_r8a7790_info = { .gen = 2, - .quirks = RCAR_LVDS_QUIRK_GEN2_PLLCR | RCAR_LVDS_QUIRK_LANES, + .quirks = RCAR_LVDS_QUIRK_LANES, + .pll_setup = rcar_lvds_pll_setup_gen2, }; static const struct rcar_lvds_device_info rcar_lvds_gen3_info = { .gen = 3, + .quirks = RCAR_LVDS_QUIRK_PWD, + .pll_setup = rcar_lvds_pll_setup_gen3, }; static const struct rcar_lvds_device_info rcar_lvds_r8a77970_info = { .gen = 3, - .quirks = RCAR_LVDS_QUIRK_GEN2_PLLCR | RCAR_LVDS_QUIRK_GEN3_LVEN, + .quirks = RCAR_LVDS_QUIRK_PWD | RCAR_LVDS_QUIRK_GEN3_LVEN, + .pll_setup = rcar_lvds_pll_setup_gen2, +}; + +static const struct rcar_lvds_device_info rcar_lvds_r8a77990_info = { + .gen = 3, + .quirks = RCAR_LVDS_QUIRK_GEN3_LVEN | RCAR_LVDS_QUIRK_EXT_PLL + | RCAR_LVDS_QUIRK_DUAL_LINK, + .pll_setup = rcar_lvds_pll_setup_d3_e3, +}; + +static const struct rcar_lvds_device_info rcar_lvds_r8a77995_info = { + .gen = 3, + .quirks = RCAR_LVDS_QUIRK_GEN3_LVEN | RCAR_LVDS_QUIRK_PWD + | RCAR_LVDS_QUIRK_EXT_PLL | RCAR_LVDS_QUIRK_DUAL_LINK, + .pll_setup = rcar_lvds_pll_setup_d3_e3, }; static const struct of_device_id rcar_lvds_of_table[] = { @@ -523,6 +792,8 @@ static const struct of_device_id rcar_lvds_of_table[] = { { .compatible = "renesas,r8a7796-lvds", .data = &rcar_lvds_gen3_info }, { .compatible = "renesas,r8a77970-lvds", .data = &rcar_lvds_r8a77970_info }, { .compatible = "renesas,r8a77980-lvds", .data = &rcar_lvds_gen3_info }, + { .compatible = "renesas,r8a77990-lvds", .data = &rcar_lvds_r8a77990_info }, + { .compatible = "renesas,r8a77995-lvds", .data = &rcar_lvds_r8a77995_info }, { } }; diff --git a/drivers/gpu/drm/rcar-du/rcar_lvds_regs.h b/drivers/gpu/drm/rcar-du/rcar_lvds_regs.h index 4870f50d9bec..87149f2f8056 100644 --- a/drivers/gpu/drm/rcar-du/rcar_lvds_regs.h +++ b/drivers/gpu/drm/rcar-du/rcar_lvds_regs.h @@ -18,7 +18,7 @@ #define LVDCR0_PLLON (1 << 4) #define LVDCR0_PWD (1 << 2) /* Gen3 only */ #define LVDCR0_BEN (1 << 2) /* Gen2 only */ -#define LVDCR0_LVEN (1 << 1) /* Gen2 only */ +#define LVDCR0_LVEN (1 << 1) #define LVDCR0_LVRES (1 << 0) #define LVDCR1 0x0004 @@ -27,21 +27,36 @@ #define LVDCR1_CLKSTBY (3 << 0) #define LVDPLLCR 0x0008 +/* Gen2 & V3M */ #define LVDPLLCR_CEEN (1 << 14) #define LVDPLLCR_FBEN (1 << 13) #define LVDPLLCR_COSEL (1 << 12) -/* Gen2 */ #define LVDPLLCR_PLLDLYCNT_150M (0x1bf << 0) #define LVDPLLCR_PLLDLYCNT_121M (0x22c << 0) #define LVDPLLCR_PLLDLYCNT_60M (0x77b << 0) #define LVDPLLCR_PLLDLYCNT_38M (0x69a << 0) #define LVDPLLCR_PLLDLYCNT_MASK (0x7ff << 0) -/* Gen3 */ +/* Gen3 but V3M,D3 and E3 */ #define LVDPLLCR_PLLDIVCNT_42M (0x014cb << 0) #define LVDPLLCR_PLLDIVCNT_85M (0x00a45 << 0) #define LVDPLLCR_PLLDIVCNT_128M (0x006c3 << 0) #define LVDPLLCR_PLLDIVCNT_148M (0x046c1 << 0) #define LVDPLLCR_PLLDIVCNT_MASK (0x7ffff << 0) +/* D3 and E3 */ +#define LVDPLLCR_PLLON (1 << 22) +#define LVDPLLCR_PLLSEL_PLL0 (0 << 20) +#define LVDPLLCR_PLLSEL_LVX (1 << 20) +#define LVDPLLCR_PLLSEL_PLL1 (2 << 20) +#define LVDPLLCR_CKSEL_LVX (1 << 17) +#define LVDPLLCR_CKSEL_EXTAL (3 << 17) +#define LVDPLLCR_CKSEL_DU_DOTCLKIN(n) ((5 + (n) * 2) << 17) +#define LVDPLLCR_OCKSEL (1 << 16) +#define LVDPLLCR_STP_CLKOUTE (1 << 14) +#define LVDPLLCR_OUTCLKSEL (1 << 12) +#define LVDPLLCR_CLKOUT (1 << 11) +#define LVDPLLCR_PLLE(n) ((n) << 10) +#define LVDPLLCR_PLLN(n) ((n) << 3) +#define LVDPLLCR_PLLM(n) ((n) << 0) #define LVDCTRCR 0x000c #define LVDCTRCR_CTR3SEL_ZERO (0 << 12) @@ -71,4 +86,26 @@ #define LVDCHCR_CHSEL_CH(n, c) ((((c) - (n)) & 3) << ((n) * 4)) #define LVDCHCR_CHSEL_MASK(n) (3 << ((n) * 4)) +/* All registers below are specific to D3 and E3 */ +#define LVDSTRIPE 0x0014 +#define LVDSTRIPE_ST_TRGSEL_DISP (0 << 2) +#define LVDSTRIPE_ST_TRGSEL_HSYNC_R (1 << 2) +#define LVDSTRIPE_ST_TRGSEL_HSYNC_F (2 << 2) +#define LVDSTRIPE_ST_SWAP (1 << 1) +#define LVDSTRIPE_ST_ON (1 << 0) + +#define LVDSCR 0x0018 +#define LVDSCR_DEPTH(n) (((n) - 1) << 29) +#define LVDSCR_BANDSET (1 << 28) +#define LVDSCR_TWGCNT(n) ((((n) - 256) / 16) << 24) +#define LVDSCR_SDIV(n) ((n) << 22) +#define LVDSCR_MODE (1 << 21) +#define LVDSCR_RSTN (1 << 20) + +#define LVDDIV 0x001c +#define LVDDIV_DIVSEL (1 << 8) +#define LVDDIV_DIVRESET (1 << 7) +#define LVDDIV_DIVSTP (1 << 6) +#define LVDDIV_DIV(n) ((n) << 0) + #endif /* __RCAR_LVDS_REGS_H__ */ diff --git a/drivers/gpu/drm/scheduler/sched_entity.c b/drivers/gpu/drm/scheduler/sched_entity.c index 4e5e95c0cab5..3e22a54a99c2 100644 --- a/drivers/gpu/drm/scheduler/sched_entity.c +++ b/drivers/gpu/drm/scheduler/sched_entity.c @@ -467,8 +467,7 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity) struct dma_fence *fence; struct drm_sched_rq *rq; - if (!spsc_queue_count(&entity->job_queue) == 0 || - entity->num_rq_list <= 1) + if (spsc_queue_count(&entity->job_queue) || entity->num_rq_list <= 1) return; fence = READ_ONCE(entity->last_scheduled); diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 9ca741f3a0bc..44fe587aaef9 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -182,6 +182,20 @@ bool drm_sched_dependency_optimized(struct dma_fence* fence, } EXPORT_SYMBOL(drm_sched_dependency_optimized); +/** + * drm_sched_start_timeout - start timeout for reset worker + * + * @sched: scheduler instance to start the worker for + * + * Start the timeout for the given scheduler. + */ +static void drm_sched_start_timeout(struct drm_gpu_scheduler *sched) +{ + if (sched->timeout != MAX_SCHEDULE_TIMEOUT && + !list_empty(&sched->ring_mirror_list)) + schedule_delayed_work(&sched->work_tdr, sched->timeout); +} + /* job_finish is called after hw fence signaled */ static void drm_sched_job_finish(struct work_struct *work) @@ -197,19 +211,13 @@ static void drm_sched_job_finish(struct work_struct *work) * manages to find this job as the next job in the list, the fence * signaled check below will prevent the timeout to be restarted. */ - cancel_delayed_work_sync(&s_job->work_tdr); + cancel_delayed_work_sync(&sched->work_tdr); spin_lock(&sched->job_list_lock); - /* queue TDR for next job */ - if (sched->timeout != MAX_SCHEDULE_TIMEOUT && - !list_is_last(&s_job->node, &sched->ring_mirror_list)) { - struct drm_sched_job *next = list_next_entry(s_job, node); - - if (!dma_fence_is_signaled(&next->s_fence->finished)) - schedule_delayed_work(&next->work_tdr, sched->timeout); - } /* remove job from ring_mirror_list */ list_del(&s_job->node); + /* queue TDR for next job */ + drm_sched_start_timeout(sched); spin_unlock(&sched->job_list_lock); dma_fence_put(&s_job->s_fence->finished); @@ -233,19 +241,49 @@ static void drm_sched_job_begin(struct drm_sched_job *s_job) spin_lock(&sched->job_list_lock); list_add_tail(&s_job->node, &sched->ring_mirror_list); - if (sched->timeout != MAX_SCHEDULE_TIMEOUT && - list_first_entry_or_null(&sched->ring_mirror_list, - struct drm_sched_job, node) == s_job) - schedule_delayed_work(&s_job->work_tdr, sched->timeout); + drm_sched_start_timeout(sched); spin_unlock(&sched->job_list_lock); } static void drm_sched_job_timedout(struct work_struct *work) { - struct drm_sched_job *job = container_of(work, struct drm_sched_job, - work_tdr.work); + struct drm_gpu_scheduler *sched; + struct drm_sched_job *job; + int r; - job->sched->ops->timedout_job(job); + sched = container_of(work, struct drm_gpu_scheduler, work_tdr.work); + + spin_lock(&sched->job_list_lock); + list_for_each_entry_reverse(job, &sched->ring_mirror_list, node) { + struct drm_sched_fence *fence = job->s_fence; + + if (!dma_fence_remove_callback(fence->parent, &fence->cb)) + goto already_signaled; + } + + job = list_first_entry_or_null(&sched->ring_mirror_list, + struct drm_sched_job, node); + spin_unlock(&sched->job_list_lock); + + if (job) + sched->ops->timedout_job(job); + + spin_lock(&sched->job_list_lock); + list_for_each_entry(job, &sched->ring_mirror_list, node) { + struct drm_sched_fence *fence = job->s_fence; + + if (!fence->parent || !list_empty(&fence->cb.node)) + continue; + + r = dma_fence_add_callback(fence->parent, &fence->cb, + drm_sched_process_job); + if (r) + drm_sched_process_job(fence->parent, &fence->cb); + +already_signaled: + ; + } + spin_unlock(&sched->job_list_lock); } /** @@ -312,11 +350,6 @@ void drm_sched_job_recovery(struct drm_gpu_scheduler *sched) int r; spin_lock(&sched->job_list_lock); - s_job = list_first_entry_or_null(&sched->ring_mirror_list, - struct drm_sched_job, node); - if (s_job && sched->timeout != MAX_SCHEDULE_TIMEOUT) - schedule_delayed_work(&s_job->work_tdr, sched->timeout); - list_for_each_entry_safe(s_job, tmp, &sched->ring_mirror_list, node) { struct drm_sched_fence *s_fence = s_job->s_fence; struct dma_fence *fence; @@ -349,6 +382,7 @@ void drm_sched_job_recovery(struct drm_gpu_scheduler *sched) } spin_lock(&sched->job_list_lock); } + drm_sched_start_timeout(sched); spin_unlock(&sched->job_list_lock); } EXPORT_SYMBOL(drm_sched_job_recovery); @@ -384,7 +418,6 @@ int drm_sched_job_init(struct drm_sched_job *job, INIT_WORK(&job->finish_work, drm_sched_job_finish); INIT_LIST_HEAD(&job->node); - INIT_DELAYED_WORK(&job->work_tdr, drm_sched_job_timedout); return 0; } @@ -575,6 +608,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, INIT_LIST_HEAD(&sched->ring_mirror_list); spin_lock_init(&sched->job_list_lock); atomic_set(&sched->hw_rq_count, 0); + INIT_DELAYED_WORK(&sched->work_tdr, drm_sched_job_timedout); atomic_set(&sched->num_jobs, 0); atomic64_set(&sched->job_id_count, 0); diff --git a/drivers/gpu/drm/sti/sti_drv.c b/drivers/gpu/drm/sti/sti_drv.c index 832fc43960ee..6dced8abcf16 100644 --- a/drivers/gpu/drm/sti/sti_drv.c +++ b/drivers/gpu/drm/sti/sti_drv.c @@ -121,7 +121,6 @@ err: static const struct drm_mode_config_funcs sti_mode_config_funcs = { .fb_create = drm_gem_fb_create, - .output_poll_changed = drm_fb_helper_output_poll_changed, .atomic_check = drm_atomic_helper_check, .atomic_commit = drm_atomic_helper_commit, }; @@ -206,7 +205,6 @@ static void sti_cleanup(struct drm_device *ddev) { struct sti_private *private = ddev->dev_private; - drm_fb_cma_fbdev_fini(ddev); drm_kms_helper_poll_fini(ddev); component_unbind_all(ddev->dev, ddev); kfree(private); @@ -236,11 +234,7 @@ static int sti_bind(struct device *dev) drm_mode_config_reset(ddev); - if (ddev->mode_config.num_connector) { - ret = drm_fb_cma_fbdev_init(ddev, 32, 0); - if (ret) - DRM_DEBUG_DRIVER("Warning: fails to create fbdev\n"); - } + drm_fbdev_generic_setup(ddev, 32); return 0; diff --git a/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c b/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c index 856124806353..471993097ced 100644 --- a/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c +++ b/drivers/gpu/drm/sun4i/sun8i_hdmi_phy.c @@ -398,7 +398,6 @@ static struct regmap_config sun8i_hdmi_phy_regmap_config = { static const struct sun8i_hdmi_phy_variant sun50i_a64_hdmi_phy = { .has_phy_clk = true, - .has_second_pll = true, .phy_init = &sun8i_hdmi_phy_init_h3, .phy_disable = &sun8i_hdmi_phy_disable_h3, .phy_config = &sun8i_hdmi_phy_config_h3, diff --git a/drivers/gpu/drm/tegra/dc.c b/drivers/gpu/drm/tegra/dc.c index 965088afcfad..f80e82e16475 100644 --- a/drivers/gpu/drm/tegra/dc.c +++ b/drivers/gpu/drm/tegra/dc.c @@ -1988,6 +1988,28 @@ static int tegra_dc_init(struct host1x_client *client) struct drm_plane *cursor = NULL; int err; + /* + * XXX do not register DCs with no window groups because we cannot + * assign a primary plane to them, which in turn will cause KMS to + * crash. + */ + if (dc->soc->wgrps) { + bool has_wgrps = false; + unsigned int i; + + for (i = 0; i < dc->soc->num_wgrps; i++) { + const struct tegra_windowgroup_soc *wgrp = &dc->soc->wgrps[i]; + + if (wgrp->dc == dc->pipe && wgrp->num_windows > 0) { + has_wgrps = true; + break; + } + } + + if (!has_wgrps) + return 0; + } + dc->syncpt = host1x_syncpt_request(client, flags); if (!dc->syncpt) dev_warn(dc->dev, "failed to allocate syncpoint\n"); @@ -2234,8 +2256,59 @@ static const struct tegra_dc_soc_info tegra186_dc_soc_info = { .num_wgrps = ARRAY_SIZE(tegra186_dc_wgrps), }; +static const struct tegra_windowgroup_soc tegra194_dc_wgrps[] = { + { + .index = 0, + .dc = 0, + .windows = (const unsigned int[]) { 0 }, + .num_windows = 1, + }, { + .index = 1, + .dc = 1, + .windows = (const unsigned int[]) { 1 }, + .num_windows = 1, + }, { + .index = 2, + .dc = 1, + .windows = (const unsigned int[]) { 2 }, + .num_windows = 1, + }, { + .index = 3, + .dc = 2, + .windows = (const unsigned int[]) { 3 }, + .num_windows = 1, + }, { + .index = 4, + .dc = 2, + .windows = (const unsigned int[]) { 4 }, + .num_windows = 1, + }, { + .index = 5, + .dc = 2, + .windows = (const unsigned int[]) { 5 }, + .num_windows = 1, + }, +}; + +static const struct tegra_dc_soc_info tegra194_dc_soc_info = { + .supports_background_color = true, + .supports_interlacing = true, + .supports_cursor = true, + .supports_block_linear = true, + .has_legacy_blending = false, + .pitch_align = 64, + .has_powergate = false, + .coupled_pm = false, + .has_nvdisplay = true, + .wgrps = tegra194_dc_wgrps, + .num_wgrps = ARRAY_SIZE(tegra194_dc_wgrps), +}; + static const struct of_device_id tegra_dc_of_match[] = { { + .compatible = "nvidia,tegra194-dc", + .data = &tegra194_dc_soc_info, + }, { .compatible = "nvidia,tegra186-dc", .data = &tegra186_dc_soc_info, }, { diff --git a/drivers/gpu/drm/tegra/dc.h b/drivers/gpu/drm/tegra/dc.h index e96f582ca692..1256dfb6b2f5 100644 --- a/drivers/gpu/drm/tegra/dc.h +++ b/drivers/gpu/drm/tegra/dc.h @@ -300,7 +300,7 @@ int tegra_dc_rgb_exit(struct tegra_dc *dc); #define SOR1_TIMING_CYA (1 << 27) #define CURSOR_ENABLE (1 << 16) -#define SOR_ENABLE(x) (1 << (25 + (x))) +#define SOR_ENABLE(x) (1 << (25 + (((x) > 1) ? ((x) + 1) : (x)))) #define DC_DISP_DISP_MEM_HIGH_PRIORITY 0x403 #define CURSOR_THRESHOLD(x) (((x) & 0x03) << 24) diff --git a/drivers/gpu/drm/tegra/dpaux.c b/drivers/gpu/drm/tegra/dpaux.c index d84e81ff36ad..ee4180d8db14 100644 --- a/drivers/gpu/drm/tegra/dpaux.c +++ b/drivers/gpu/drm/tegra/dpaux.c @@ -521,7 +521,7 @@ static int tegra_dpaux_probe(struct platform_device *pdev) * is no possibility to perform the I2C mode configuration in the * HDMI path. */ - err = tegra_dpaux_pad_config(dpaux, DPAUX_HYBRID_PADCTL_MODE_I2C); + err = tegra_dpaux_pad_config(dpaux, DPAUX_PADCTL_FUNC_I2C); if (err < 0) return err; @@ -639,6 +639,7 @@ static const struct dev_pm_ops tegra_dpaux_pm_ops = { }; static const struct of_device_id tegra_dpaux_of_match[] = { + { .compatible = "nvidia,tegra194-dpaux", }, { .compatible = "nvidia,tegra186-dpaux", }, { .compatible = "nvidia,tegra210-dpaux", }, { .compatible = "nvidia,tegra124-dpaux", }, diff --git a/drivers/gpu/drm/tegra/drm.c b/drivers/gpu/drm/tegra/drm.c index b424bc911b95..65ea4988b332 100644 --- a/drivers/gpu/drm/tegra/drm.c +++ b/drivers/gpu/drm/tegra/drm.c @@ -15,6 +15,10 @@ #include <drm/drm_atomic.h> #include <drm/drm_atomic_helper.h> +#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU) +#include <asm/dma-iommu.h> +#endif + #include "drm.h" #include "gem.h" @@ -1068,6 +1072,14 @@ struct iommu_group *host1x_client_iommu_attach(struct host1x_client *client, } if (!shared || (shared && (group != tegra->group))) { +#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU) + if (client->dev->archdata.mapping) { + struct dma_iommu_mapping *mapping = + to_dma_iommu_mapping(client->dev); + arm_iommu_detach_device(client->dev); + arm_iommu_release_mapping(mapping); + } +#endif err = iommu_attach_group(tegra->domain, group); if (err < 0) { iommu_group_put(group); @@ -1189,16 +1201,16 @@ static int host1x_drm_probe(struct host1x_device *dev) err = drm_fb_helper_remove_conflicting_framebuffers(NULL, "tegradrmfb", false); if (err < 0) - goto unref; + goto put; err = drm_dev_register(drm, 0); if (err < 0) - goto unref; + goto put; return 0; -unref: - drm_dev_unref(drm); +put: + drm_dev_put(drm); return err; } @@ -1207,7 +1219,7 @@ static int host1x_drm_remove(struct host1x_device *dev) struct drm_device *drm = dev_get_drvdata(&dev->dev); drm_dev_unregister(drm); - drm_dev_unref(drm); + drm_dev_put(drm); return 0; } @@ -1216,31 +1228,15 @@ static int host1x_drm_remove(struct host1x_device *dev) static int host1x_drm_suspend(struct device *dev) { struct drm_device *drm = dev_get_drvdata(dev); - struct tegra_drm *tegra = drm->dev_private; - drm_kms_helper_poll_disable(drm); - tegra_drm_fb_suspend(drm); - - tegra->state = drm_atomic_helper_suspend(drm); - if (IS_ERR(tegra->state)) { - tegra_drm_fb_resume(drm); - drm_kms_helper_poll_enable(drm); - return PTR_ERR(tegra->state); - } - - return 0; + return drm_mode_config_helper_suspend(drm); } static int host1x_drm_resume(struct device *dev) { struct drm_device *drm = dev_get_drvdata(dev); - struct tegra_drm *tegra = drm->dev_private; - - drm_atomic_helper_resume(drm, tegra->state); - tegra_drm_fb_resume(drm); - drm_kms_helper_poll_enable(drm); - return 0; + return drm_mode_config_helper_resume(drm); } #endif @@ -1275,6 +1271,9 @@ static const struct of_device_id host1x_drm_subdevs[] = { { .compatible = "nvidia,tegra186-sor", }, { .compatible = "nvidia,tegra186-sor1", }, { .compatible = "nvidia,tegra186-vic", }, + { .compatible = "nvidia,tegra194-display", }, + { .compatible = "nvidia,tegra194-dc", }, + { .compatible = "nvidia,tegra194-sor", }, { /* sentinel */ } }; diff --git a/drivers/gpu/drm/tegra/drm.h b/drivers/gpu/drm/tegra/drm.h index 92d248784396..1012335bb489 100644 --- a/drivers/gpu/drm/tegra/drm.h +++ b/drivers/gpu/drm/tegra/drm.h @@ -60,8 +60,6 @@ struct tegra_drm { unsigned int pitch_align; struct tegra_display_hub *hub; - - struct drm_atomic_state *state; }; struct tegra_drm_client; @@ -186,8 +184,6 @@ int tegra_drm_fb_prepare(struct drm_device *drm); void tegra_drm_fb_free(struct drm_device *drm); int tegra_drm_fb_init(struct drm_device *drm); void tegra_drm_fb_exit(struct drm_device *drm); -void tegra_drm_fb_suspend(struct drm_device *drm); -void tegra_drm_fb_resume(struct drm_device *drm); extern struct platform_driver tegra_display_hub_driver; extern struct platform_driver tegra_dc_driver; diff --git a/drivers/gpu/drm/tegra/fb.c b/drivers/gpu/drm/tegra/fb.c index 4c22cdded3c2..b947e82bbeb1 100644 --- a/drivers/gpu/drm/tegra/fb.c +++ b/drivers/gpu/drm/tegra/fb.c @@ -356,7 +356,7 @@ static void tegra_fbdev_exit(struct tegra_fbdev *fbdev) /* Undo the special mapping we made in fbdev probe. */ if (bo && bo->pages) { vunmap(bo->vaddr); - bo->vaddr = 0; + bo->vaddr = NULL; } drm_framebuffer_remove(fbdev->fb); @@ -412,25 +412,3 @@ void tegra_drm_fb_exit(struct drm_device *drm) tegra_fbdev_exit(tegra->fbdev); #endif } - -void tegra_drm_fb_suspend(struct drm_device *drm) -{ -#ifdef CONFIG_DRM_FBDEV_EMULATION - struct tegra_drm *tegra = drm->dev_private; - - console_lock(); - drm_fb_helper_set_suspend(&tegra->fbdev->base, 1); - console_unlock(); -#endif -} - -void tegra_drm_fb_resume(struct drm_device *drm) -{ -#ifdef CONFIG_DRM_FBDEV_EMULATION - struct tegra_drm *tegra = drm->dev_private; - - console_lock(); - drm_fb_helper_set_suspend(&tegra->fbdev->base, 0); - console_unlock(); -#endif -} diff --git a/drivers/gpu/drm/tegra/hub.c b/drivers/gpu/drm/tegra/hub.c index 8f4fcbb515fb..6112d9042979 100644 --- a/drivers/gpu/drm/tegra/hub.c +++ b/drivers/gpu/drm/tegra/hub.c @@ -758,10 +758,12 @@ static int tegra_display_hub_probe(struct platform_device *pdev) return err; } - hub->clk_dsc = devm_clk_get(&pdev->dev, "dsc"); - if (IS_ERR(hub->clk_dsc)) { - err = PTR_ERR(hub->clk_dsc); - return err; + if (hub->soc->supports_dsc) { + hub->clk_dsc = devm_clk_get(&pdev->dev, "dsc"); + if (IS_ERR(hub->clk_dsc)) { + err = PTR_ERR(hub->clk_dsc); + return err; + } } hub->clk_hub = devm_clk_get(&pdev->dev, "hub"); @@ -890,10 +892,19 @@ static const struct dev_pm_ops tegra_display_hub_pm_ops = { static const struct tegra_display_hub_soc tegra186_display_hub = { .num_wgrps = 6, + .supports_dsc = true, +}; + +static const struct tegra_display_hub_soc tegra194_display_hub = { + .num_wgrps = 6, + .supports_dsc = false, }; static const struct of_device_id tegra_display_hub_of_match[] = { { + .compatible = "nvidia,tegra194-display", + .data = &tegra194_display_hub + }, { .compatible = "nvidia,tegra186-display", .data = &tegra186_display_hub }, { diff --git a/drivers/gpu/drm/tegra/hub.h b/drivers/gpu/drm/tegra/hub.h index 85b8bf41a395..6696a85fc1f2 100644 --- a/drivers/gpu/drm/tegra/hub.h +++ b/drivers/gpu/drm/tegra/hub.h @@ -38,6 +38,7 @@ to_tegra_shared_plane(struct drm_plane *plane) struct tegra_display_hub_soc { unsigned int num_wgrps; + bool supports_dsc; }; struct tegra_display_hub { diff --git a/drivers/gpu/drm/tegra/sor.c b/drivers/gpu/drm/tegra/sor.c index d7fe9f15def1..b129da2e5afd 100644 --- a/drivers/gpu/drm/tegra/sor.c +++ b/drivers/gpu/drm/tegra/sor.c @@ -282,6 +282,85 @@ static const struct tegra_sor_hdmi_settings tegra186_sor_hdmi_defaults[] = { } }; +static const struct tegra_sor_hdmi_settings tegra194_sor_hdmi_defaults[] = { + { + .frequency = 54000000, + .vcocap = 0, + .filter = 5, + .ichpmp = 5, + .loadadj = 3, + .tmds_termadj = 0xf, + .tx_pu_value = 0, + .bg_temp_coef = 3, + .bg_vref_level = 8, + .avdd10_level = 4, + .avdd14_level = 4, + .sparepll = 0x54, + .drive_current = { 0x3a, 0x3a, 0x3a, 0x33 }, + .preemphasis = { 0x00, 0x00, 0x00, 0x00 }, + }, { + .frequency = 75000000, + .vcocap = 1, + .filter = 5, + .ichpmp = 5, + .loadadj = 3, + .tmds_termadj = 0xf, + .tx_pu_value = 0, + .bg_temp_coef = 3, + .bg_vref_level = 8, + .avdd10_level = 4, + .avdd14_level = 4, + .sparepll = 0x44, + .drive_current = { 0x3a, 0x3a, 0x3a, 0x33 }, + .preemphasis = { 0x00, 0x00, 0x00, 0x00 }, + }, { + .frequency = 150000000, + .vcocap = 3, + .filter = 5, + .ichpmp = 5, + .loadadj = 3, + .tmds_termadj = 15, + .tx_pu_value = 0x66 /* 0 */, + .bg_temp_coef = 3, + .bg_vref_level = 8, + .avdd10_level = 4, + .avdd14_level = 4, + .sparepll = 0x00, /* 0x34 */ + .drive_current = { 0x3a, 0x3a, 0x3a, 0x37 }, + .preemphasis = { 0x00, 0x00, 0x00, 0x00 }, + }, { + .frequency = 300000000, + .vcocap = 3, + .filter = 5, + .ichpmp = 5, + .loadadj = 3, + .tmds_termadj = 15, + .tx_pu_value = 64, + .bg_temp_coef = 3, + .bg_vref_level = 8, + .avdd10_level = 4, + .avdd14_level = 4, + .sparepll = 0x34, + .drive_current = { 0x3d, 0x3d, 0x3d, 0x33 }, + .preemphasis = { 0x00, 0x00, 0x00, 0x00 }, + }, { + .frequency = 600000000, + .vcocap = 3, + .filter = 5, + .ichpmp = 5, + .loadadj = 3, + .tmds_termadj = 12, + .tx_pu_value = 96, + .bg_temp_coef = 3, + .bg_vref_level = 8, + .avdd10_level = 4, + .avdd14_level = 4, + .sparepll = 0x34, + .drive_current = { 0x3d, 0x3d, 0x3d, 0x33 }, + .preemphasis = { 0x00, 0x00, 0x00, 0x00 }, + } +}; + struct tegra_sor_regs { unsigned int head_state0; unsigned int head_state1; @@ -2894,7 +2973,38 @@ static const struct tegra_sor_soc tegra186_sor1 = { .xbar_cfg = tegra124_sor_xbar_cfg, }; +static const struct tegra_sor_regs tegra194_sor_regs = { + .head_state0 = 0x151, + .head_state1 = 0x155, + .head_state2 = 0x159, + .head_state3 = 0x15d, + .head_state4 = 0x161, + .head_state5 = 0x165, + .pll0 = 0x169, + .pll1 = 0x16a, + .pll2 = 0x16b, + .pll3 = 0x16c, + .dp_padctl0 = 0x16e, + .dp_padctl2 = 0x16f, +}; + +static const struct tegra_sor_soc tegra194_sor = { + .supports_edp = true, + .supports_lvds = false, + .supports_hdmi = true, + .supports_dp = true, + + .regs = &tegra194_sor_regs, + .has_nvdisplay = true, + + .num_settings = ARRAY_SIZE(tegra194_sor_hdmi_defaults), + .settings = tegra194_sor_hdmi_defaults, + + .xbar_cfg = tegra210_sor_xbar_cfg, +}; + static const struct of_device_id tegra_sor_of_match[] = { + { .compatible = "nvidia,tegra194-sor", .data = &tegra194_sor }, { .compatible = "nvidia,tegra186-sor1", .data = &tegra186_sor1 }, { .compatible = "nvidia,tegra186-sor", .data = &tegra186_sor }, { .compatible = "nvidia,tegra210-sor1", .data = &tegra210_sor1 }, diff --git a/drivers/gpu/drm/tilcdc/tilcdc_drv.c b/drivers/gpu/drm/tilcdc/tilcdc_drv.c index 0fb300d41a09..33e533268488 100644 --- a/drivers/gpu/drm/tilcdc/tilcdc_drv.c +++ b/drivers/gpu/drm/tilcdc/tilcdc_drv.c @@ -554,29 +554,23 @@ static struct drm_driver tilcdc_driver = { static int tilcdc_pm_suspend(struct device *dev) { struct drm_device *ddev = dev_get_drvdata(dev); - struct tilcdc_drm_private *priv = ddev->dev_private; + int ret = 0; - priv->saved_state = drm_atomic_helper_suspend(ddev); + ret = drm_mode_config_helper_suspend(ddev); /* Select sleep pin state */ pinctrl_pm_select_sleep_state(dev); - return 0; + return ret; } static int tilcdc_pm_resume(struct device *dev) { struct drm_device *ddev = dev_get_drvdata(dev); - struct tilcdc_drm_private *priv = ddev->dev_private; - int ret = 0; /* Select default pin state */ pinctrl_pm_select_default_state(dev); - - if (priv->saved_state) - ret = drm_atomic_helper_resume(ddev, priv->saved_state); - - return ret; + return drm_mode_config_helper_resume(ddev); } #endif diff --git a/drivers/gpu/drm/tilcdc/tilcdc_drv.h b/drivers/gpu/drm/tilcdc/tilcdc_drv.h index ead512216669..62cea5ff5558 100644 --- a/drivers/gpu/drm/tilcdc/tilcdc_drv.h +++ b/drivers/gpu/drm/tilcdc/tilcdc_drv.h @@ -70,9 +70,6 @@ struct tilcdc_drm_private { const uint32_t *pixelformats; uint32_t num_pixelformats; - /* The context for pm susped/resume cycle is stored here */ - struct drm_atomic_state *saved_state; - #ifdef CONFIG_CPU_FREQ struct notifier_block freq_transition; #endif diff --git a/drivers/gpu/drm/ttm/Makefile b/drivers/gpu/drm/ttm/Makefile index a60e560804e0..01fc670ce7a2 100644 --- a/drivers/gpu/drm/ttm/Makefile +++ b/drivers/gpu/drm/ttm/Makefile @@ -4,8 +4,8 @@ ttm-y := ttm_memory.o ttm_tt.o ttm_bo.o \ ttm_bo_util.o ttm_bo_vm.o ttm_module.o \ - ttm_object.o ttm_lock.o ttm_execbuf_util.o ttm_page_alloc.o \ - ttm_bo_manager.o ttm_page_alloc_dma.o + ttm_execbuf_util.o ttm_page_alloc.o ttm_bo_manager.o \ + ttm_page_alloc_dma.o ttm-$(CONFIG_AGP) += ttm_agp_backend.o obj-$(CONFIG_DRM_TTM) += ttm.o diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index b2a33bf1ef10..26b889f86670 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -247,20 +247,6 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_move_to_lru_tail); -static void ttm_list_move_bulk_tail(struct list_head *list, - struct list_head *first, - struct list_head *last) -{ - first->prev->next = last->next; - last->next->prev = first->prev; - - list->prev->next = first; - first->prev = list->prev; - - last->next = list; - list->prev = last; -} - void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) { unsigned i; @@ -276,8 +262,8 @@ void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) reservation_object_assert_held(pos->last->resv); man = &pos->first->bdev->man[TTM_PL_TT]; - ttm_list_move_bulk_tail(&man->lru[i], &pos->first->lru, - &pos->last->lru); + list_bulk_move_tail(&man->lru[i], &pos->first->lru, + &pos->last->lru); } for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { @@ -291,8 +277,8 @@ void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) reservation_object_assert_held(pos->last->resv); man = &pos->first->bdev->man[TTM_PL_VRAM]; - ttm_list_move_bulk_tail(&man->lru[i], &pos->first->lru, - &pos->last->lru); + list_bulk_move_tail(&man->lru[i], &pos->first->lru, + &pos->last->lru); } for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { @@ -306,8 +292,7 @@ void ttm_bo_bulk_move_lru_tail(struct ttm_lru_bulk_move *bulk) reservation_object_assert_held(pos->last->resv); lru = &pos->first->bdev->glob->swap_lru[i]; - ttm_list_move_bulk_tail(lru, &pos->first->swap, - &pos->last->swap); + list_bulk_move_tail(lru, &pos->first->swap, &pos->last->swap); } } EXPORT_SYMBOL(ttm_bo_bulk_move_lru_tail); diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 6fe91c1b692d..a1d977fbade5 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -409,8 +409,7 @@ static struct ttm_buffer_object *ttm_bo_vm_lookup(struct ttm_bo_device *bdev, node = drm_vma_offset_lookup_locked(&bdev->vma_manager, offset, pages); if (likely(node)) { bo = container_of(node, struct ttm_buffer_object, vma_node); - if (!kref_get_unless_zero(&bo->kref)) - bo = NULL; + bo = ttm_bo_get_unless_zero(bo); } drm_vma_offset_unlock_lookup(&bdev->vma_manager); diff --git a/drivers/gpu/drm/tve200/tve200_drv.c b/drivers/gpu/drm/tve200/tve200_drv.c index ac344ddb23bc..72efcecb44f7 100644 --- a/drivers/gpu/drm/tve200/tve200_drv.c +++ b/drivers/gpu/drm/tve200/tve200_drv.c @@ -126,12 +126,6 @@ static int tve200_modeset_init(struct drm_device *dev) } drm_mode_config_reset(dev); - - /* - * Passing in 16 here will make the RGB656 mode the default - * Passing in 32 will use XRGB8888 mode - */ - drm_fb_cma_fbdev_init(dev, 16, 0); drm_kms_helper_poll_init(dev); goto finish; @@ -149,7 +143,6 @@ DEFINE_DRM_GEM_CMA_FOPS(drm_fops); static struct drm_driver tve200_drm_driver = { .driver_features = DRIVER_MODESET | DRIVER_GEM | DRIVER_PRIME | DRIVER_ATOMIC, - .lastclose = drm_fb_helper_lastclose, .ioctls = NULL, .fops = &drm_fops, .name = "tve200", @@ -245,6 +238,12 @@ static int tve200_probe(struct platform_device *pdev) if (ret < 0) goto clk_disable; + /* + * Passing in 16 here will make the RGB565 mode the default + * Passing in 32 will use XRGB8888 mode + */ + drm_fbdev_generic_setup(drm, 16); + return 0; clk_disable: @@ -260,7 +259,6 @@ static int tve200_remove(struct platform_device *pdev) struct tve200_drm_dev_private *priv = drm->dev_private; drm_dev_unregister(drm); - drm_fb_cma_fbdev_fini(drm); if (priv->panel) drm_panel_bridge_remove(priv->bridge); drm_mode_config_cleanup(drm); diff --git a/drivers/gpu/drm/udl/udl_drv.c b/drivers/gpu/drm/udl/udl_drv.c index 9ef515df724b..a63e3011e971 100644 --- a/drivers/gpu/drm/udl/udl_drv.c +++ b/drivers/gpu/drm/udl/udl_drv.c @@ -94,7 +94,7 @@ static int udl_usb_probe(struct usb_interface *interface, return 0; err_free: - drm_dev_unref(dev); + drm_dev_put(dev); return r; } diff --git a/drivers/gpu/drm/udl/udl_fb.c b/drivers/gpu/drm/udl/udl_fb.c index dbb62f6eb48a..dd9ffded223b 100644 --- a/drivers/gpu/drm/udl/udl_fb.c +++ b/drivers/gpu/drm/udl/udl_fb.c @@ -432,9 +432,11 @@ static void udl_fbdev_destroy(struct drm_device *dev, { drm_fb_helper_unregister_fbi(&ufbdev->helper); drm_fb_helper_fini(&ufbdev->helper); - drm_framebuffer_unregister_private(&ufbdev->ufb.base); - drm_framebuffer_cleanup(&ufbdev->ufb.base); - drm_gem_object_put_unlocked(&ufbdev->ufb.obj->base); + if (ufbdev->ufb.obj) { + drm_framebuffer_unregister_private(&ufbdev->ufb.base); + drm_framebuffer_cleanup(&ufbdev->ufb.base); + drm_gem_object_put_unlocked(&ufbdev->ufb.obj->base); + } } int udl_fbdev_init(struct drm_device *dev) diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c index a5501581d96b..9243dea6e6ad 100644 --- a/drivers/gpu/drm/v3d/v3d_sched.c +++ b/drivers/gpu/drm/v3d/v3d_sched.c @@ -168,7 +168,7 @@ v3d_job_timedout(struct drm_sched_job *sched_job) job->timedout_ctca = ctca; job->timedout_ctra = ctra; - schedule_delayed_work(&job->base.work_tdr, + schedule_delayed_work(&job->base.sched->work_tdr, job->base.sched->timeout); return; } diff --git a/drivers/gpu/drm/vc4/vc4_drv.c b/drivers/gpu/drm/vc4/vc4_drv.c index e2a15c63a81f..1f1780ccdbdf 100644 --- a/drivers/gpu/drm/vc4/vc4_drv.c +++ b/drivers/gpu/drm/vc4/vc4_drv.c @@ -178,7 +178,6 @@ static struct drm_driver vc4_drm_driver = { DRIVER_RENDER | DRIVER_PRIME | DRIVER_SYNCOBJ), - .lastclose = drm_fb_helper_lastclose, .open = vc4_open, .postclose = vc4_close, .irq_handler = vc4_irq, @@ -288,6 +287,8 @@ static int vc4_drm_bind(struct device *dev) vc4_kms_load(drm); + drm_fbdev_generic_setup(drm, 32); + return 0; unbind_all: @@ -307,8 +308,6 @@ static void vc4_drm_unbind(struct device *dev) drm_dev_unregister(drm); - drm_fb_cma_fbdev_fini(drm); - drm_mode_config_cleanup(drm); drm_atomic_private_obj_fini(&vc4->ctm_manager); diff --git a/drivers/gpu/drm/vc4/vc4_kms.c b/drivers/gpu/drm/vc4/vc4_kms.c index ca5aa7fba769..127468785f74 100644 --- a/drivers/gpu/drm/vc4/vc4_kms.c +++ b/drivers/gpu/drm/vc4/vc4_kms.c @@ -19,8 +19,6 @@ #include <drm/drm_atomic_helper.h> #include <drm/drm_crtc_helper.h> #include <drm/drm_plane_helper.h> -#include <drm/drm_fb_helper.h> -#include <drm/drm_fb_cma_helper.h> #include <drm/drm_gem_framebuffer_helper.h> #include "vc4_drv.h" #include "vc4_regs.h" @@ -394,7 +392,6 @@ vc4_atomic_check(struct drm_device *dev, struct drm_atomic_state *state) } static const struct drm_mode_config_funcs vc4_mode_funcs = { - .output_poll_changed = drm_fb_helper_output_poll_changed, .atomic_check = vc4_atomic_check, .atomic_commit = vc4_atomic_commit, .fb_create = vc4_fb_create, @@ -434,9 +431,6 @@ int vc4_kms_load(struct drm_device *dev) drm_mode_config_reset(dev); - if (dev->mode_config.num_connector) - drm_fb_cma_fbdev_init(dev, 32, 0); - drm_kms_helper_poll_init(dev); return 0; diff --git a/drivers/gpu/drm/vc4/vc4_plane.c b/drivers/gpu/drm/vc4/vc4_plane.c index f39ee212412d..9dc3fcbd290b 100644 --- a/drivers/gpu/drm/vc4/vc4_plane.c +++ b/drivers/gpu/drm/vc4/vc4_plane.c @@ -296,6 +296,9 @@ static int vc4_plane_setup_clipping_and_scaling(struct drm_plane_state *state) vc4_state->y_scaling[0] = vc4_get_scaling_mode(vc4_state->src_h[0], vc4_state->crtc_h); + vc4_state->is_unity = (vc4_state->x_scaling[0] == VC4_SCALING_NONE && + vc4_state->y_scaling[0] == VC4_SCALING_NONE); + if (num_planes > 1) { vc4_state->is_yuv = true; @@ -311,24 +314,17 @@ static int vc4_plane_setup_clipping_and_scaling(struct drm_plane_state *state) vc4_get_scaling_mode(vc4_state->src_h[1], vc4_state->crtc_h); - /* YUV conversion requires that scaling be enabled, - * even on a plane that's otherwise 1:1. Choose TPZ - * for simplicity. + /* YUV conversion requires that horizontal scaling be enabled, + * even on a plane that's otherwise 1:1. Looks like only PPF + * works in that case, so let's pick that one. */ - if (vc4_state->x_scaling[0] == VC4_SCALING_NONE) - vc4_state->x_scaling[0] = VC4_SCALING_TPZ; - if (vc4_state->y_scaling[0] == VC4_SCALING_NONE) - vc4_state->y_scaling[0] = VC4_SCALING_TPZ; + if (vc4_state->is_unity) + vc4_state->x_scaling[0] = VC4_SCALING_PPF; } else { vc4_state->x_scaling[1] = VC4_SCALING_NONE; vc4_state->y_scaling[1] = VC4_SCALING_NONE; } - vc4_state->is_unity = (vc4_state->x_scaling[0] == VC4_SCALING_NONE && - vc4_state->y_scaling[0] == VC4_SCALING_NONE && - vc4_state->x_scaling[1] == VC4_SCALING_NONE && - vc4_state->y_scaling[1] == VC4_SCALING_NONE); - /* No configuring scaling on the cursor plane, since it gets non-vblank-synced updates, and scaling requires requires LBM changes which have to be vblank-synced. @@ -671,7 +667,10 @@ static int vc4_plane_mode_set(struct drm_plane *plane, vc4_dlist_write(vc4_state, SCALER_CSC2_ITR_R_601_5); } - if (!vc4_state->is_unity) { + if (vc4_state->x_scaling[0] != VC4_SCALING_NONE || + vc4_state->x_scaling[1] != VC4_SCALING_NONE || + vc4_state->y_scaling[0] != VC4_SCALING_NONE || + vc4_state->y_scaling[1] != VC4_SCALING_NONE) { /* LBM Base Address. */ if (vc4_state->y_scaling[0] != VC4_SCALING_NONE || vc4_state->y_scaling[1] != VC4_SCALING_NONE) { diff --git a/drivers/gpu/drm/virtio/virtgpu_display.c b/drivers/gpu/drm/virtio/virtgpu_display.c index 0379d6897659..8f8fed471e34 100644 --- a/drivers/gpu/drm/virtio/virtgpu_display.c +++ b/drivers/gpu/drm/virtio/virtgpu_display.c @@ -307,6 +307,10 @@ virtio_gpu_user_framebuffer_create(struct drm_device *dev, struct virtio_gpu_framebuffer *virtio_gpu_fb; int ret; + if (mode_cmd->pixel_format != DRM_FORMAT_HOST_XRGB8888 && + mode_cmd->pixel_format != DRM_FORMAT_HOST_ARGB8888) + return ERR_PTR(-ENOENT); + /* lookup object associated with res handle */ obj = drm_gem_object_lookup(file_priv, mode_cmd->handles[0]); if (!obj) @@ -355,6 +359,7 @@ int virtio_gpu_modeset_init(struct virtio_gpu_device *vgdev) int i; drm_mode_config_init(vgdev->ddev); + vgdev->ddev->mode_config.quirk_addfb_prefer_host_byte_order = true; vgdev->ddev->mode_config.funcs = &virtio_gpu_mode_funcs; vgdev->ddev->mode_config.helper_private = &virtio_mode_config_helpers; diff --git a/drivers/gpu/drm/virtio/virtgpu_drv.h b/drivers/gpu/drm/virtio/virtgpu_drv.h index a2d79e18bda7..d29f0c7c768c 100644 --- a/drivers/gpu/drm/virtio/virtgpu_drv.h +++ b/drivers/gpu/drm/virtio/virtgpu_drv.h @@ -270,7 +270,8 @@ void virtio_gpu_cmd_create_resource(struct virtio_gpu_device *vgdev, void virtio_gpu_cmd_unref_resource(struct virtio_gpu_device *vgdev, uint32_t resource_id); void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev, - uint32_t resource_id, uint64_t offset, + struct virtio_gpu_object *bo, + uint64_t offset, __le32 width, __le32 height, __le32 x, __le32 y, struct virtio_gpu_fence **fence); @@ -316,7 +317,8 @@ void virtio_gpu_cmd_transfer_from_host_3d(struct virtio_gpu_device *vgdev, struct virtio_gpu_box *box, struct virtio_gpu_fence **fence); void virtio_gpu_cmd_transfer_to_host_3d(struct virtio_gpu_device *vgdev, - uint32_t resource_id, uint32_t ctx_id, + struct virtio_gpu_object *bo, + uint32_t ctx_id, uint64_t offset, uint32_t level, struct virtio_gpu_box *box, struct virtio_gpu_fence **fence); @@ -361,7 +363,8 @@ void virtio_gpu_fence_event_process(struct virtio_gpu_device *vdev, int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, unsigned long size, bool kernel, bool pinned, struct virtio_gpu_object **bo_ptr); -int virtio_gpu_object_kmap(struct virtio_gpu_object *bo, void **ptr); +void virtio_gpu_object_kunmap(struct virtio_gpu_object *bo); +int virtio_gpu_object_kmap(struct virtio_gpu_object *bo); int virtio_gpu_object_get_sg_table(struct virtio_gpu_device *qdev, struct virtio_gpu_object *bo); void virtio_gpu_object_free_sg_table(struct virtio_gpu_object *bo); diff --git a/drivers/gpu/drm/virtio/virtgpu_fb.c b/drivers/gpu/drm/virtio/virtgpu_fb.c index b9678c4082ac..cea749f4ec39 100644 --- a/drivers/gpu/drm/virtio/virtgpu_fb.c +++ b/drivers/gpu/drm/virtio/virtgpu_fb.c @@ -95,7 +95,7 @@ static int virtio_gpu_dirty_update(struct virtio_gpu_framebuffer *fb, offset = (y * fb->base.pitches[0]) + x * bpp; - virtio_gpu_cmd_transfer_to_host_2d(vgdev, obj->hw_res_handle, + virtio_gpu_cmd_transfer_to_host_2d(vgdev, obj, offset, cpu_to_le32(w), cpu_to_le32(h), @@ -203,12 +203,6 @@ static struct fb_ops virtio_gpufb_ops = { .fb_imageblit = virtio_gpu_3d_imageblit, }; -static int virtio_gpu_vmap_fb(struct virtio_gpu_device *vgdev, - struct virtio_gpu_object *obj) -{ - return virtio_gpu_object_kmap(obj, NULL); -} - static int virtio_gpufb_create(struct drm_fb_helper *helper, struct drm_fb_helper_surface_size *sizes) { @@ -226,7 +220,7 @@ static int virtio_gpufb_create(struct drm_fb_helper *helper, mode_cmd.width = sizes->surface_width; mode_cmd.height = sizes->surface_height; mode_cmd.pitches[0] = mode_cmd.width * 4; - mode_cmd.pixel_format = drm_mode_legacy_fb_format(32, 24); + mode_cmd.pixel_format = DRM_FORMAT_HOST_XRGB8888; format = virtio_gpu_translate_format(mode_cmd.pixel_format); if (format == 0) @@ -241,9 +235,9 @@ static int virtio_gpufb_create(struct drm_fb_helper *helper, virtio_gpu_cmd_create_resource(vgdev, resid, format, mode_cmd.width, mode_cmd.height); - ret = virtio_gpu_vmap_fb(vgdev, obj); + ret = virtio_gpu_object_kmap(obj); if (ret) { - DRM_ERROR("failed to vmap fb %d\n", ret); + DRM_ERROR("failed to kmap fb %d\n", ret); goto err_obj_vmap; } diff --git a/drivers/gpu/drm/virtio/virtgpu_gem.c b/drivers/gpu/drm/virtio/virtgpu_gem.c index 0f2768eacaee..82c817f37cf7 100644 --- a/drivers/gpu/drm/virtio/virtgpu_gem.c +++ b/drivers/gpu/drm/virtio/virtgpu_gem.c @@ -90,7 +90,10 @@ int virtio_gpu_mode_dumb_create(struct drm_file *file_priv, uint32_t resid; uint32_t format; - pitch = args->width * ((args->bpp + 1) / 8); + if (args->bpp != 32) + return -EINVAL; + + pitch = args->width * 4; args->size = pitch * args->height; args->size = ALIGN(args->size, PAGE_SIZE); @@ -99,7 +102,7 @@ int virtio_gpu_mode_dumb_create(struct drm_file *file_priv, if (ret) goto fail; - format = virtio_gpu_translate_format(DRM_FORMAT_XRGB8888); + format = virtio_gpu_translate_format(DRM_FORMAT_HOST_XRGB8888); virtio_gpu_resource_id_get(vgdev, &resid); virtio_gpu_cmd_create_resource(vgdev, resid, format, args->width, args->height); diff --git a/drivers/gpu/drm/virtio/virtgpu_ioctl.c b/drivers/gpu/drm/virtio/virtgpu_ioctl.c index 7bdf6f0e58a5..f16b875d6a46 100644 --- a/drivers/gpu/drm/virtio/virtgpu_ioctl.c +++ b/drivers/gpu/drm/virtio/virtgpu_ioctl.c @@ -429,11 +429,11 @@ static int virtio_gpu_transfer_to_host_ioctl(struct drm_device *dev, void *data, convert_to_hw_box(&box, &args->box); if (!vgdev->has_virgl_3d) { virtio_gpu_cmd_transfer_to_host_2d - (vgdev, qobj->hw_res_handle, offset, + (vgdev, qobj, offset, box.w, box.h, box.x, box.y, NULL); } else { virtio_gpu_cmd_transfer_to_host_3d - (vgdev, qobj->hw_res_handle, + (vgdev, qobj, vfpriv ? vfpriv->ctx_id : 0, offset, args->level, &box, &fence); reservation_object_add_excl_fence(qobj->tbo.resv, diff --git a/drivers/gpu/drm/virtio/virtgpu_object.c b/drivers/gpu/drm/virtio/virtgpu_object.c index 9f2f470efd9b..eca765537470 100644 --- a/drivers/gpu/drm/virtio/virtgpu_object.c +++ b/drivers/gpu/drm/virtio/virtgpu_object.c @@ -37,6 +37,8 @@ static void virtio_gpu_ttm_bo_destroy(struct ttm_buffer_object *tbo) virtio_gpu_cmd_unref_resource(vgdev, bo->hw_res_handle); if (bo->pages) virtio_gpu_object_free_sg_table(bo); + if (bo->vmap) + virtio_gpu_object_kunmap(bo); drm_gem_object_release(&bo->gem_base); kfree(bo); } @@ -99,22 +101,23 @@ int virtio_gpu_object_create(struct virtio_gpu_device *vgdev, return 0; } -int virtio_gpu_object_kmap(struct virtio_gpu_object *bo, void **ptr) +void virtio_gpu_object_kunmap(struct virtio_gpu_object *bo) +{ + bo->vmap = NULL; + ttm_bo_kunmap(&bo->kmap); +} + +int virtio_gpu_object_kmap(struct virtio_gpu_object *bo) { bool is_iomem; int r; - if (bo->vmap) { - if (ptr) - *ptr = bo->vmap; - return 0; - } + WARN_ON(bo->vmap); + r = ttm_bo_kmap(&bo->tbo, 0, bo->tbo.num_pages, &bo->kmap); if (r) return r; bo->vmap = ttm_kmap_obj_virtual(&bo->kmap, &is_iomem); - if (ptr) - *ptr = bo->vmap; return 0; } diff --git a/drivers/gpu/drm/virtio/virtgpu_plane.c b/drivers/gpu/drm/virtio/virtgpu_plane.c index 88f2fb8c61c4..a9f4ae7d4483 100644 --- a/drivers/gpu/drm/virtio/virtgpu_plane.c +++ b/drivers/gpu/drm/virtio/virtgpu_plane.c @@ -28,22 +28,11 @@ #include <drm/drm_atomic_helper.h> static const uint32_t virtio_gpu_formats[] = { - DRM_FORMAT_XRGB8888, - DRM_FORMAT_ARGB8888, - DRM_FORMAT_BGRX8888, - DRM_FORMAT_BGRA8888, - DRM_FORMAT_RGBX8888, - DRM_FORMAT_RGBA8888, - DRM_FORMAT_XBGR8888, - DRM_FORMAT_ABGR8888, + DRM_FORMAT_HOST_XRGB8888, }; static const uint32_t virtio_gpu_cursor_formats[] = { -#ifdef __BIG_ENDIAN - DRM_FORMAT_BGRA8888, -#else - DRM_FORMAT_ARGB8888, -#endif + DRM_FORMAT_HOST_ARGB8888, }; uint32_t virtio_gpu_translate_format(uint32_t drm_fourcc) @@ -51,32 +40,6 @@ uint32_t virtio_gpu_translate_format(uint32_t drm_fourcc) uint32_t format; switch (drm_fourcc) { -#ifdef __BIG_ENDIAN - case DRM_FORMAT_XRGB8888: - format = VIRTIO_GPU_FORMAT_X8R8G8B8_UNORM; - break; - case DRM_FORMAT_ARGB8888: - format = VIRTIO_GPU_FORMAT_A8R8G8B8_UNORM; - break; - case DRM_FORMAT_BGRX8888: - format = VIRTIO_GPU_FORMAT_B8G8R8X8_UNORM; - break; - case DRM_FORMAT_BGRA8888: - format = VIRTIO_GPU_FORMAT_B8G8R8A8_UNORM; - break; - case DRM_FORMAT_RGBX8888: - format = VIRTIO_GPU_FORMAT_R8G8B8X8_UNORM; - break; - case DRM_FORMAT_RGBA8888: - format = VIRTIO_GPU_FORMAT_R8G8B8A8_UNORM; - break; - case DRM_FORMAT_XBGR8888: - format = VIRTIO_GPU_FORMAT_X8B8G8R8_UNORM; - break; - case DRM_FORMAT_ABGR8888: - format = VIRTIO_GPU_FORMAT_A8B8G8R8_UNORM; - break; -#else case DRM_FORMAT_XRGB8888: format = VIRTIO_GPU_FORMAT_B8G8R8X8_UNORM; break; @@ -89,19 +52,6 @@ uint32_t virtio_gpu_translate_format(uint32_t drm_fourcc) case DRM_FORMAT_BGRA8888: format = VIRTIO_GPU_FORMAT_A8R8G8B8_UNORM; break; - case DRM_FORMAT_RGBX8888: - format = VIRTIO_GPU_FORMAT_X8B8G8R8_UNORM; - break; - case DRM_FORMAT_RGBA8888: - format = VIRTIO_GPU_FORMAT_A8B8G8R8_UNORM; - break; - case DRM_FORMAT_XBGR8888: - format = VIRTIO_GPU_FORMAT_R8G8B8X8_UNORM; - break; - case DRM_FORMAT_ABGR8888: - format = VIRTIO_GPU_FORMAT_R8G8B8A8_UNORM; - break; -#endif default: /* * This should not happen, we handle everything listed @@ -158,7 +108,7 @@ static void virtio_gpu_primary_plane_update(struct drm_plane *plane, handle = bo->hw_res_handle; if (bo->dumb) { virtio_gpu_cmd_transfer_to_host_2d - (vgdev, handle, 0, + (vgdev, bo, 0, cpu_to_le32(plane->state->src_w >> 16), cpu_to_le32(plane->state->src_h >> 16), cpu_to_le32(plane->state->src_x >> 16), @@ -217,7 +167,7 @@ static void virtio_gpu_cursor_plane_update(struct drm_plane *plane, if (bo && bo->dumb && (plane->state->fb != old_state->fb)) { /* new cursor -- update & wait */ virtio_gpu_cmd_transfer_to_host_2d - (vgdev, handle, 0, + (vgdev, bo, 0, cpu_to_le32(plane->state->crtc_w), cpu_to_le32(plane->state->crtc_h), 0, 0, &fence); diff --git a/drivers/gpu/drm/virtio/virtgpu_prime.c b/drivers/gpu/drm/virtio/virtgpu_prime.c index d27a1688714f..86ce0ae93f59 100644 --- a/drivers/gpu/drm/virtio/virtgpu_prime.c +++ b/drivers/gpu/drm/virtio/virtgpu_prime.c @@ -55,13 +55,18 @@ struct drm_gem_object *virtgpu_gem_prime_import_sg_table( void *virtgpu_gem_prime_vmap(struct drm_gem_object *obj) { - WARN_ONCE(1, "not implemented"); - return ERR_PTR(-ENODEV); + struct virtio_gpu_object *bo = gem_to_virtio_gpu_obj(obj); + int ret; + + ret = virtio_gpu_object_kmap(bo); + if (ret) + return NULL; + return bo->vmap; } void virtgpu_gem_prime_vunmap(struct drm_gem_object *obj, void *vaddr) { - WARN_ONCE(1, "not implemented"); + virtio_gpu_object_kunmap(gem_to_virtio_gpu_obj(obj)); } int virtgpu_gem_prime_mmap(struct drm_gem_object *obj, diff --git a/drivers/gpu/drm/virtio/virtgpu_vq.c b/drivers/gpu/drm/virtio/virtgpu_vq.c index df32811f2c3e..4e2e037aed34 100644 --- a/drivers/gpu/drm/virtio/virtgpu_vq.c +++ b/drivers/gpu/drm/virtio/virtgpu_vq.c @@ -483,28 +483,26 @@ void virtio_gpu_cmd_resource_flush(struct virtio_gpu_device *vgdev, } void virtio_gpu_cmd_transfer_to_host_2d(struct virtio_gpu_device *vgdev, - uint32_t resource_id, uint64_t offset, + struct virtio_gpu_object *bo, + uint64_t offset, __le32 width, __le32 height, __le32 x, __le32 y, struct virtio_gpu_fence **fence) { struct virtio_gpu_transfer_to_host_2d *cmd_p; struct virtio_gpu_vbuffer *vbuf; - struct virtio_gpu_fbdev *vgfbdev = vgdev->vgfbdev; - struct virtio_gpu_framebuffer *fb = &vgfbdev->vgfb; - struct virtio_gpu_object *obj = gem_to_virtio_gpu_obj(fb->base.obj[0]); bool use_dma_api = !virtio_has_iommu_quirk(vgdev->vdev); if (use_dma_api) dma_sync_sg_for_device(vgdev->vdev->dev.parent, - obj->pages->sgl, obj->pages->nents, + bo->pages->sgl, bo->pages->nents, DMA_TO_DEVICE); cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p)); memset(cmd_p, 0, sizeof(*cmd_p)); cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_TRANSFER_TO_HOST_2D); - cmd_p->resource_id = cpu_to_le32(resource_id); + cmd_p->resource_id = cpu_to_le32(bo->hw_res_handle); cmd_p->offset = cpu_to_le64(offset); cmd_p->r.width = width; cmd_p->r.height = height; @@ -791,21 +789,19 @@ virtio_gpu_cmd_resource_create_3d(struct virtio_gpu_device *vgdev, } void virtio_gpu_cmd_transfer_to_host_3d(struct virtio_gpu_device *vgdev, - uint32_t resource_id, uint32_t ctx_id, + struct virtio_gpu_object *bo, + uint32_t ctx_id, uint64_t offset, uint32_t level, struct virtio_gpu_box *box, struct virtio_gpu_fence **fence) { struct virtio_gpu_transfer_host_3d *cmd_p; struct virtio_gpu_vbuffer *vbuf; - struct virtio_gpu_fbdev *vgfbdev = vgdev->vgfbdev; - struct virtio_gpu_framebuffer *fb = &vgfbdev->vgfb; - struct virtio_gpu_object *obj = gem_to_virtio_gpu_obj(fb->base.obj[0]); bool use_dma_api = !virtio_has_iommu_quirk(vgdev->vdev); if (use_dma_api) dma_sync_sg_for_device(vgdev->vdev->dev.parent, - obj->pages->sgl, obj->pages->nents, + bo->pages->sgl, bo->pages->nents, DMA_TO_DEVICE); cmd_p = virtio_gpu_alloc_cmd(vgdev, &vbuf, sizeof(*cmd_p)); @@ -813,7 +809,7 @@ void virtio_gpu_cmd_transfer_to_host_3d(struct virtio_gpu_device *vgdev, cmd_p->hdr.type = cpu_to_le32(VIRTIO_GPU_CMD_TRANSFER_TO_HOST_3D); cmd_p->hdr.ctx_id = cpu_to_le32(ctx_id); - cmd_p->resource_id = cpu_to_le32(resource_id); + cmd_p->resource_id = cpu_to_le32(bo->hw_res_handle); cmd_p->box = *box; cmd_p->offset = cpu_to_le64(offset); cmd_p->level = cpu_to_le32(level); diff --git a/drivers/gpu/drm/vkms/vkms_crc.c b/drivers/gpu/drm/vkms/vkms_crc.c index 0a2745646dfa..9d9e8146db90 100644 --- a/drivers/gpu/drm/vkms/vkms_crc.c +++ b/drivers/gpu/drm/vkms/vkms_crc.c @@ -125,6 +125,7 @@ static uint32_t _vkms_get_crc(struct vkms_crc_data *primary_crc, mutex_lock(&vkms_obj->pages_lock); if (WARN_ON(!vkms_obj->vaddr)) { mutex_unlock(&vkms_obj->pages_lock); + kfree(vaddr_out); return crc; } diff --git a/drivers/gpu/drm/vmwgfx/Makefile b/drivers/gpu/drm/vmwgfx/Makefile index 09b2aa08363e..8841bd30e1e5 100644 --- a/drivers/gpu/drm/vmwgfx/Makefile +++ b/drivers/gpu/drm/vmwgfx/Makefile @@ -7,6 +7,8 @@ vmwgfx-y := vmwgfx_execbuf.o vmwgfx_gmr.o vmwgfx_kms.o vmwgfx_drv.o \ vmwgfx_surface.o vmwgfx_prime.o vmwgfx_mob.o vmwgfx_shader.o \ vmwgfx_cmdbuf_res.o vmwgfx_cmdbuf.o vmwgfx_stdu.o \ vmwgfx_cotable.o vmwgfx_so.o vmwgfx_binding.o vmwgfx_msg.o \ - vmwgfx_simple_resource.o vmwgfx_va.o vmwgfx_blit.o + vmwgfx_simple_resource.o vmwgfx_va.o vmwgfx_blit.o \ + vmwgfx_validation.o \ + ttm_object.o ttm_lock.o obj-$(CONFIG_DRM_VMWGFX) := vmwgfx.o diff --git a/drivers/gpu/drm/ttm/ttm_lock.c b/drivers/gpu/drm/vmwgfx/ttm_lock.c index 20694b8a01ca..16b2083cb9d4 100644 --- a/drivers/gpu/drm/ttm/ttm_lock.c +++ b/drivers/gpu/drm/vmwgfx/ttm_lock.c @@ -29,13 +29,13 @@ * Authors: Thomas Hellstrom <thellstrom-at-vmware-dot-com> */ -#include <drm/ttm/ttm_lock.h> #include <drm/ttm/ttm_module.h> #include <linux/atomic.h> #include <linux/errno.h> #include <linux/wait.h> #include <linux/sched/signal.h> -#include <linux/module.h> +#include "ttm_lock.h" +#include "ttm_object.h" #define TTM_WRITE_LOCK_PENDING (1 << 0) #define TTM_VT_LOCK_PENDING (1 << 1) @@ -52,7 +52,6 @@ void ttm_lock_init(struct ttm_lock *lock) lock->kill_takers = false; lock->signal = SIGKILL; } -EXPORT_SYMBOL(ttm_lock_init); void ttm_read_unlock(struct ttm_lock *lock) { @@ -61,7 +60,6 @@ void ttm_read_unlock(struct ttm_lock *lock) wake_up_all(&lock->queue); spin_unlock(&lock->lock); } -EXPORT_SYMBOL(ttm_read_unlock); static bool __ttm_read_lock(struct ttm_lock *lock) { @@ -92,7 +90,6 @@ int ttm_read_lock(struct ttm_lock *lock, bool interruptible) wait_event(lock->queue, __ttm_read_lock(lock)); return ret; } -EXPORT_SYMBOL(ttm_read_lock); static bool __ttm_read_trylock(struct ttm_lock *lock, bool *locked) { @@ -144,7 +141,6 @@ void ttm_write_unlock(struct ttm_lock *lock) wake_up_all(&lock->queue); spin_unlock(&lock->lock); } -EXPORT_SYMBOL(ttm_write_unlock); static bool __ttm_write_lock(struct ttm_lock *lock) { @@ -185,7 +181,6 @@ int ttm_write_lock(struct ttm_lock *lock, bool interruptible) return ret; } -EXPORT_SYMBOL(ttm_write_lock); static int __ttm_vt_unlock(struct ttm_lock *lock) { @@ -262,14 +257,12 @@ int ttm_vt_lock(struct ttm_lock *lock, return ret; } -EXPORT_SYMBOL(ttm_vt_lock); int ttm_vt_unlock(struct ttm_lock *lock) { return ttm_ref_object_base_unref(lock->vt_holder, - lock->base.hash.key, TTM_REF_USAGE); + lock->base.handle, TTM_REF_USAGE); } -EXPORT_SYMBOL(ttm_vt_unlock); void ttm_suspend_unlock(struct ttm_lock *lock) { @@ -278,7 +271,6 @@ void ttm_suspend_unlock(struct ttm_lock *lock) wake_up_all(&lock->queue); spin_unlock(&lock->lock); } -EXPORT_SYMBOL(ttm_suspend_unlock); static bool __ttm_suspend_lock(struct ttm_lock *lock) { @@ -300,4 +292,3 @@ void ttm_suspend_lock(struct ttm_lock *lock) { wait_event(lock->queue, __ttm_suspend_lock(lock)); } -EXPORT_SYMBOL(ttm_suspend_lock); diff --git a/include/drm/ttm/ttm_lock.h b/drivers/gpu/drm/vmwgfx/ttm_lock.h index 0c3af9836863..0c3af9836863 100644 --- a/include/drm/ttm/ttm_lock.h +++ b/drivers/gpu/drm/vmwgfx/ttm_lock.h diff --git a/drivers/gpu/drm/ttm/ttm_object.c b/drivers/gpu/drm/vmwgfx/ttm_object.c index 74f1b1eb1f8e..36990b80e790 100644 --- a/drivers/gpu/drm/ttm/ttm_object.c +++ b/drivers/gpu/drm/vmwgfx/ttm_object.c @@ -59,13 +59,12 @@ #define pr_fmt(fmt) "[TTM] " fmt -#include <drm/ttm/ttm_object.h> #include <drm/ttm/ttm_module.h> #include <linux/list.h> #include <linux/spinlock.h> #include <linux/slab.h> -#include <linux/module.h> #include <linux/atomic.h> +#include "ttm_object.h" struct ttm_object_file { struct ttm_object_device *tdev; @@ -95,6 +94,7 @@ struct ttm_object_device { struct dma_buf_ops ops; void (*dmabuf_release)(struct dma_buf *dma_buf); size_t dma_buf_size; + struct idr idr; }; /** @@ -172,14 +172,15 @@ int ttm_base_object_init(struct ttm_object_file *tfile, base->ref_obj_release = ref_obj_release; base->object_type = object_type; kref_init(&base->refcount); + idr_preload(GFP_KERNEL); spin_lock(&tdev->object_lock); - ret = drm_ht_just_insert_please_rcu(&tdev->object_hash, - &base->hash, - (unsigned long)base, 31, 0, 0); + ret = idr_alloc(&tdev->idr, base, 0, 0, GFP_NOWAIT); spin_unlock(&tdev->object_lock); - if (unlikely(ret != 0)) - goto out_err0; + idr_preload_end(); + if (ret < 0) + return ret; + base->handle = ret; ret = ttm_ref_object_add(tfile, base, TTM_REF_USAGE, NULL, false); if (unlikely(ret != 0)) goto out_err1; @@ -189,12 +190,10 @@ int ttm_base_object_init(struct ttm_object_file *tfile, return 0; out_err1: spin_lock(&tdev->object_lock); - (void)drm_ht_remove_item_rcu(&tdev->object_hash, &base->hash); + idr_remove(&tdev->idr, base->handle); spin_unlock(&tdev->object_lock); -out_err0: return ret; } -EXPORT_SYMBOL(ttm_base_object_init); static void ttm_release_base(struct kref *kref) { @@ -203,7 +202,7 @@ static void ttm_release_base(struct kref *kref) struct ttm_object_device *tdev = base->tfile->tdev; spin_lock(&tdev->object_lock); - (void)drm_ht_remove_item_rcu(&tdev->object_hash, &base->hash); + idr_remove(&tdev->idr, base->handle); spin_unlock(&tdev->object_lock); /* @@ -225,7 +224,41 @@ void ttm_base_object_unref(struct ttm_base_object **p_base) kref_put(&base->refcount, ttm_release_base); } -EXPORT_SYMBOL(ttm_base_object_unref); + +/** + * ttm_base_object_noref_lookup - look up a base object without reference + * @tfile: The struct ttm_object_file the object is registered with. + * @key: The object handle. + * + * This function looks up a ttm base object and returns a pointer to it + * without refcounting the pointer. The returned pointer is only valid + * until ttm_base_object_noref_release() is called, and the object + * pointed to by the returned pointer may be doomed. Any persistent usage + * of the object requires a refcount to be taken using kref_get_unless_zero(). + * Iff this function returns successfully it needs to be paired with + * ttm_base_object_noref_release() and no sleeping- or scheduling functions + * may be called inbetween these function callse. + * + * Return: A pointer to the object if successful or NULL otherwise. + */ +struct ttm_base_object * +ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint32_t key) +{ + struct drm_hash_item *hash; + struct drm_open_hash *ht = &tfile->ref_hash[TTM_REF_USAGE]; + int ret; + + rcu_read_lock(); + ret = drm_ht_find_item_rcu(ht, key, &hash); + if (ret) { + rcu_read_unlock(); + return NULL; + } + + __release(RCU); + return drm_hash_entry(hash, struct ttm_ref_object, hash)->obj; +} +EXPORT_SYMBOL(ttm_base_object_noref_lookup); struct ttm_base_object *ttm_base_object_lookup(struct ttm_object_file *tfile, uint32_t key) @@ -247,29 +280,21 @@ struct ttm_base_object *ttm_base_object_lookup(struct ttm_object_file *tfile, return base; } -EXPORT_SYMBOL(ttm_base_object_lookup); struct ttm_base_object * ttm_base_object_lookup_for_ref(struct ttm_object_device *tdev, uint32_t key) { - struct ttm_base_object *base = NULL; - struct drm_hash_item *hash; - struct drm_open_hash *ht = &tdev->object_hash; - int ret; + struct ttm_base_object *base; rcu_read_lock(); - ret = drm_ht_find_item_rcu(ht, key, &hash); + base = idr_find(&tdev->idr, key); - if (likely(ret == 0)) { - base = drm_hash_entry(hash, struct ttm_base_object, hash); - if (!kref_get_unless_zero(&base->refcount)) - base = NULL; - } + if (base && !kref_get_unless_zero(&base->refcount)) + base = NULL; rcu_read_unlock(); return base; } -EXPORT_SYMBOL(ttm_base_object_lookup_for_ref); /** * ttm_ref_object_exists - Check whether a caller has a valid ref object @@ -289,7 +314,7 @@ bool ttm_ref_object_exists(struct ttm_object_file *tfile, struct ttm_ref_object *ref; rcu_read_lock(); - if (unlikely(drm_ht_find_item_rcu(ht, base->hash.key, &hash) != 0)) + if (unlikely(drm_ht_find_item_rcu(ht, base->handle, &hash) != 0)) goto out_false; /* @@ -315,7 +340,6 @@ bool ttm_ref_object_exists(struct ttm_object_file *tfile, rcu_read_unlock(); return false; } -EXPORT_SYMBOL(ttm_ref_object_exists); int ttm_ref_object_add(struct ttm_object_file *tfile, struct ttm_base_object *base, @@ -340,7 +364,7 @@ int ttm_ref_object_add(struct ttm_object_file *tfile, while (ret == -EINVAL) { rcu_read_lock(); - ret = drm_ht_find_item_rcu(ht, base->hash.key, &hash); + ret = drm_ht_find_item_rcu(ht, base->handle, &hash); if (ret == 0) { ref = drm_hash_entry(hash, struct ttm_ref_object, hash); @@ -364,7 +388,7 @@ int ttm_ref_object_add(struct ttm_object_file *tfile, return -ENOMEM; } - ref->hash.key = base->hash.key; + ref->hash.key = base->handle; ref->obj = base; ref->tfile = tfile; ref->ref_type = ref_type; @@ -391,9 +415,9 @@ int ttm_ref_object_add(struct ttm_object_file *tfile, return ret; } -EXPORT_SYMBOL(ttm_ref_object_add); -static void ttm_ref_object_release(struct kref *kref) +static void __releases(tfile->lock) __acquires(tfile->lock) +ttm_ref_object_release(struct kref *kref) { struct ttm_ref_object *ref = container_of(kref, struct ttm_ref_object, kref); @@ -435,7 +459,6 @@ int ttm_ref_object_base_unref(struct ttm_object_file *tfile, spin_unlock(&tfile->lock); return 0; } -EXPORT_SYMBOL(ttm_ref_object_base_unref); void ttm_object_file_release(struct ttm_object_file **p_tfile) { @@ -464,7 +487,6 @@ void ttm_object_file_release(struct ttm_object_file **p_tfile) ttm_object_file_unref(&tfile); } -EXPORT_SYMBOL(ttm_object_file_release); struct ttm_object_file *ttm_object_file_init(struct ttm_object_device *tdev, unsigned int hash_order) @@ -499,7 +521,6 @@ out_err: return NULL; } -EXPORT_SYMBOL(ttm_object_file_init); struct ttm_object_device * ttm_object_device_init(struct ttm_mem_global *mem_glob, @@ -519,6 +540,7 @@ ttm_object_device_init(struct ttm_mem_global *mem_glob, if (ret != 0) goto out_no_object_hash; + idr_init(&tdev->idr); tdev->ops = *ops; tdev->dmabuf_release = tdev->ops.release; tdev->ops.release = ttm_prime_dmabuf_release; @@ -530,7 +552,6 @@ out_no_object_hash: kfree(tdev); return NULL; } -EXPORT_SYMBOL(ttm_object_device_init); void ttm_object_device_release(struct ttm_object_device **p_tdev) { @@ -538,11 +559,12 @@ void ttm_object_device_release(struct ttm_object_device **p_tdev) *p_tdev = NULL; + WARN_ON_ONCE(!idr_is_empty(&tdev->idr)); + idr_destroy(&tdev->idr); drm_ht_remove(&tdev->object_hash); kfree(tdev); } -EXPORT_SYMBOL(ttm_object_device_release); /** * get_dma_buf_unless_doomed - get a dma_buf reference if possible. @@ -641,14 +663,13 @@ int ttm_prime_fd_to_handle(struct ttm_object_file *tfile, prime = (struct ttm_prime_object *) dma_buf->priv; base = &prime->base; - *handle = base->hash.key; + *handle = base->handle; ret = ttm_ref_object_add(tfile, base, TTM_REF_USAGE, NULL, false); dma_buf_put(dma_buf); return ret; } -EXPORT_SYMBOL_GPL(ttm_prime_fd_to_handle); /** * ttm_prime_handle_to_fd - Return a dma_buf fd from a ttm prime object @@ -739,7 +760,6 @@ out_unref: ttm_base_object_unref(&base); return ret; } -EXPORT_SYMBOL_GPL(ttm_prime_handle_to_fd); /** * ttm_prime_object_init - Initialize a ttm_prime_object @@ -772,4 +792,3 @@ int ttm_prime_object_init(struct ttm_object_file *tfile, size_t size, ttm_prime_refcount_release, ref_obj_release); } -EXPORT_SYMBOL(ttm_prime_object_init); diff --git a/include/drm/ttm/ttm_object.h b/drivers/gpu/drm/vmwgfx/ttm_object.h index a98bfeb4239e..50d26c7ff42d 100644 --- a/include/drm/ttm/ttm_object.h +++ b/drivers/gpu/drm/vmwgfx/ttm_object.h @@ -42,8 +42,7 @@ #include <linux/kref.h> #include <linux/rcupdate.h> #include <linux/dma-buf.h> - -#include "ttm_memory.h" +#include <drm/ttm/ttm_memory.h> /** * enum ttm_ref_type @@ -125,14 +124,14 @@ struct ttm_object_device; struct ttm_base_object { struct rcu_head rhead; - struct drm_hash_item hash; - enum ttm_object_type object_type; - bool shareable; struct ttm_object_file *tfile; struct kref refcount; void (*refcount_release) (struct ttm_base_object **base); void (*ref_obj_release) (struct ttm_base_object *base, enum ttm_ref_type ref_type); + u32 handle; + enum ttm_object_type object_type; + u32 shareable; }; @@ -351,4 +350,26 @@ extern int ttm_prime_handle_to_fd(struct ttm_object_file *tfile, #define ttm_prime_object_kfree(__obj, __prime) \ kfree_rcu(__obj, __prime.base.rhead) + +/* + * Extra memory required by the base object's idr storage, which is allocated + * separately from the base object itself. We estimate an on-average 128 bytes + * per idr. + */ +#define TTM_OBJ_EXTRA_SIZE 128 + +struct ttm_base_object * +ttm_base_object_noref_lookup(struct ttm_object_file *tfile, uint32_t key); + +/** + * ttm_base_object_noref_release - release a base object pointer looked up + * without reference + * + * Releases a base object pointer looked up with ttm_base_object_noref_lookup(). + */ +static inline void ttm_base_object_noref_release(void) +{ + __acquire(RCU); + rcu_read_unlock(); +} #endif diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c index 2dda03345761..7ce1c2f87d9a 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_bo.c @@ -30,7 +30,7 @@ #include <drm/drmP.h> #include "vmwgfx_drv.h" -#include "drm/ttm/ttm_object.h" +#include "ttm_object.h" /** @@ -441,7 +441,8 @@ static size_t vmw_bo_acc_size(struct vmw_private *dev_priv, size_t size, struct_size = backend_size + ttm_round_pot(sizeof(struct vmw_buffer_object)); user_struct_size = backend_size + - ttm_round_pot(sizeof(struct vmw_user_buffer_object)); + ttm_round_pot(sizeof(struct vmw_user_buffer_object)) + + TTM_OBJ_EXTRA_SIZE; } if (dev_priv->map_mode == vmw_dma_alloc_coherent) @@ -631,7 +632,7 @@ int vmw_user_bo_alloc(struct vmw_private *dev_priv, *p_base = &user_bo->prime.base; kref_get(&(*p_base)->refcount); } - *handle = user_bo->prime.base.hash.key; + *handle = user_bo->prime.base.handle; out_no_base_object: return ret; @@ -920,6 +921,47 @@ int vmw_user_bo_lookup(struct ttm_object_file *tfile, return 0; } +/** + * vmw_user_bo_noref_lookup - Look up a vmw user buffer object without reference + * @tfile: The TTM object file the handle is registered with. + * @handle: The user buffer object handle. + * + * This function looks up a struct vmw_user_bo and returns a pointer to the + * struct vmw_buffer_object it derives from without refcounting the pointer. + * The returned pointer is only valid until vmw_user_bo_noref_release() is + * called, and the object pointed to by the returned pointer may be doomed. + * Any persistent usage of the object requires a refcount to be taken using + * ttm_bo_reference_unless_doomed(). Iff this function returns successfully it + * needs to be paired with vmw_user_bo_noref_release() and no sleeping- + * or scheduling functions may be called inbetween these function calls. + * + * Return: A struct vmw_buffer_object pointer if successful or negative + * error pointer on failure. + */ +struct vmw_buffer_object * +vmw_user_bo_noref_lookup(struct ttm_object_file *tfile, u32 handle) +{ + struct vmw_user_buffer_object *vmw_user_bo; + struct ttm_base_object *base; + + base = ttm_base_object_noref_lookup(tfile, handle); + if (!base) { + DRM_ERROR("Invalid buffer object handle 0x%08lx.\n", + (unsigned long)handle); + return ERR_PTR(-ESRCH); + } + + if (unlikely(ttm_base_object_type(base) != ttm_buffer_type)) { + ttm_base_object_noref_release(); + DRM_ERROR("Invalid buffer object handle 0x%08lx.\n", + (unsigned long)handle); + return ERR_PTR(-EINVAL); + } + + vmw_user_bo = container_of(base, struct vmw_user_buffer_object, + prime.base); + return &vmw_user_bo->vbo; +} /** * vmw_user_bo_reference - Open a handle to a vmw user buffer object. @@ -940,7 +982,7 @@ int vmw_user_bo_reference(struct ttm_object_file *tfile, user_bo = container_of(vbo, struct vmw_user_buffer_object, vbo); - *handle = user_bo->prime.base.hash.key; + *handle = user_bo->prime.base.handle; return ttm_ref_object_add(tfile, &user_bo->prime.base, TTM_REF_USAGE, NULL, false); } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c index e7e4655d3f36..48d1380a952e 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c @@ -660,7 +660,7 @@ static void __vmw_cmdbuf_cur_flush(struct vmw_cmdbuf_man *man) { struct vmw_cmdbuf_header *cur = man->cur; - WARN_ON(!mutex_is_locked(&man->cur_mutex)); + lockdep_assert_held_once(&man->cur_mutex); if (!cur) return; @@ -1045,7 +1045,7 @@ static void vmw_cmdbuf_commit_cur(struct vmw_cmdbuf_man *man, { struct vmw_cmdbuf_header *cur = man->cur; - WARN_ON(!mutex_is_locked(&man->cur_mutex)); + lockdep_assert_held_once(&man->cur_mutex); WARN_ON(size > cur->reserved); man->cur_pos += size; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c index 3b75af9bf85f..4ac55fc2bf97 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c @@ -89,8 +89,7 @@ vmw_cmdbuf_res_lookup(struct vmw_cmdbuf_res_manager *man, if (unlikely(ret != 0)) return ERR_PTR(ret); - return vmw_resource_reference - (drm_hash_entry(hash, struct vmw_cmdbuf_res, hash)->res); + return drm_hash_entry(hash, struct vmw_cmdbuf_res, hash)->res; } /** diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_context.c b/drivers/gpu/drm/vmwgfx/vmwgfx_context.c index 7c3cb8efd11a..14bd760a62fd 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_context.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_context.c @@ -217,9 +217,7 @@ static int vmw_gb_context_init(struct vmw_private *dev_priv, } } - - - vmw_resource_activate(res, vmw_hw_context_destroy); + res->hw_destroy = vmw_hw_context_destroy; return 0; out_cotables: @@ -274,7 +272,7 @@ static int vmw_context_init(struct vmw_private *dev_priv, vmw_fifo_commit(dev_priv, sizeof(*cmd)); vmw_fifo_resource_inc(dev_priv); - vmw_resource_activate(res, vmw_hw_context_destroy); + res->hw_destroy = vmw_hw_context_destroy; return 0; out_early: @@ -757,14 +755,10 @@ static int vmw_context_define(struct drm_device *dev, void *data, return -EINVAL; } - /* - * Approximate idr memory usage with 128 bytes. It will be limited - * by maximum number_of contexts anyway. - */ - if (unlikely(vmw_user_context_size == 0)) - vmw_user_context_size = ttm_round_pot(sizeof(*ctx)) + 128 + - ((dev_priv->has_mob) ? vmw_cmdbuf_res_man_size() : 0); + vmw_user_context_size = ttm_round_pot(sizeof(*ctx)) + + ((dev_priv->has_mob) ? vmw_cmdbuf_res_man_size() : 0) + + + VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE; ret = ttm_read_lock(&dev_priv->reservation_sem, true); if (unlikely(ret != 0)) @@ -809,7 +803,7 @@ static int vmw_context_define(struct drm_device *dev, void *data, goto out_err; } - arg->cid = ctx->base.hash.key; + arg->cid = ctx->base.handle; out_err: vmw_resource_unreference(&res); out_unlock: @@ -867,9 +861,8 @@ struct vmw_resource *vmw_context_cotable(struct vmw_resource *ctx, if (cotable_type >= SVGA_COTABLE_DX10_MAX) return ERR_PTR(-EINVAL); - return vmw_resource_reference - (container_of(ctx, struct vmw_user_context, res)-> - cotables[cotable_type]); + return container_of(ctx, struct vmw_user_context, res)-> + cotables[cotable_type]; } /** diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c index 1d45714e1d5a..44f3f6f107d3 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c @@ -615,7 +615,7 @@ struct vmw_resource *vmw_cotable_alloc(struct vmw_private *dev_priv, vcotbl->type = type; vcotbl->ctx = ctx; - vmw_resource_activate(&vcotbl->res, vmw_hw_cotable_destroy); + vcotbl->res.hw_destroy = vmw_hw_cotable_destroy; return &vcotbl->res; diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c index bb6dbbe18835..61a84b958d67 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c @@ -30,9 +30,9 @@ #include <drm/drmP.h> #include "vmwgfx_drv.h" #include "vmwgfx_binding.h" +#include "ttm_object.h" #include <drm/ttm/ttm_placement.h> #include <drm/ttm/ttm_bo_driver.h> -#include <drm/ttm/ttm_object.h> #include <drm/ttm/ttm_module.h> #include <linux/dma_remapping.h> @@ -667,8 +667,8 @@ static int vmw_driver_load(struct drm_device *dev, unsigned long chipset) mutex_init(&dev_priv->binding_mutex); mutex_init(&dev_priv->requested_layout_mutex); mutex_init(&dev_priv->global_kms_state_mutex); - rwlock_init(&dev_priv->resource_lock); ttm_lock_init(&dev_priv->reservation_sem); + spin_lock_init(&dev_priv->resource_lock); spin_lock_init(&dev_priv->hw_lock); spin_lock_init(&dev_priv->waiter_lock); spin_lock_init(&dev_priv->cap_lock); diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h index 1abe21758b0d..59f614225bcd 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h @@ -28,6 +28,7 @@ #ifndef _VMWGFX_DRV_H_ #define _VMWGFX_DRV_H_ +#include "vmwgfx_validation.h" #include "vmwgfx_reg.h" #include <drm/drmP.h> #include <drm/vmwgfx_drm.h> @@ -35,11 +36,11 @@ #include <drm/drm_auth.h> #include <linux/suspend.h> #include <drm/ttm/ttm_bo_driver.h> -#include <drm/ttm/ttm_object.h> -#include <drm/ttm/ttm_lock.h> #include <drm/ttm/ttm_execbuf_util.h> #include <drm/ttm/ttm_module.h> #include "vmwgfx_fence.h" +#include "ttm_object.h" +#include "ttm_lock.h" #include <linux/sync_file.h> #define VMWGFX_DRIVER_NAME "vmwgfx" @@ -112,21 +113,49 @@ struct vmw_validate_buffer { }; struct vmw_res_func; + + +/** + * struct vmw-resource - base class for hardware resources + * + * @kref: For refcounting. + * @dev_priv: Pointer to the device private for this resource. Immutable. + * @id: Device id. Protected by @dev_priv::resource_lock. + * @backup_size: Backup buffer size. Immutable. + * @res_dirty: Resource contains data not yet in the backup buffer. Protected + * by resource reserved. + * @backup_dirty: Backup buffer contains data not yet in the HW resource. + * Protecte by resource reserved. + * @backup: The backup buffer if any. Protected by resource reserved. + * @backup_offset: Offset into the backup buffer if any. Protected by resource + * reserved. Note that only a few resource types can have a @backup_offset + * different from zero. + * @pin_count: The pin count for this resource. A pinned resource has a + * pin-count greater than zero. It is not on the resource LRU lists and its + * backup buffer is pinned. Hence it can't be evicted. + * @func: Method vtable for this resource. Immutable. + * @lru_head: List head for the LRU list. Protected by @dev_priv::resource_lock. + * @mob_head: List head for the MOB backup list. Protected by @backup reserved. + * @binding_head: List head for the context binding list. Protected by + * the @dev_priv::binding_mutex + * @res_free: The resource destructor. + * @hw_destroy: Callback to destroy the resource on the device, as part of + * resource destruction. + */ struct vmw_resource { struct kref kref; struct vmw_private *dev_priv; int id; - bool avail; unsigned long backup_size; - bool res_dirty; /* Protected by backup buffer reserved */ - bool backup_dirty; /* Protected by backup buffer reserved */ + bool res_dirty; + bool backup_dirty; struct vmw_buffer_object *backup; unsigned long backup_offset; - unsigned long pin_count; /* Protected by resource reserved */ + unsigned long pin_count; const struct vmw_res_func *func; - struct list_head lru_head; /* Protected by the resource lock */ - struct list_head mob_head; /* Protected by @backup reserved */ - struct list_head binding_head; /* Protected by binding_mutex */ + struct list_head lru_head; + struct list_head mob_head; + struct list_head binding_head; void (*res_free) (struct vmw_resource *res); void (*hw_destroy) (struct vmw_resource *res); }; @@ -204,29 +233,24 @@ struct vmw_fifo_state { bool dx; }; -struct vmw_relocation { - SVGAMobId *mob_loc; - SVGAGuestPtr *location; - uint32_t index; -}; - /** * struct vmw_res_cache_entry - resource information cache entry - * + * @handle: User-space handle of a resource. + * @res: Non-ref-counted pointer to the resource. + * @valid_handle: Whether the @handle member is valid. * @valid: Whether the entry is valid, which also implies that the execbuf * code holds a reference to the resource, and it's placed on the * validation list. - * @handle: User-space handle of a resource. - * @res: Non-ref-counted pointer to the resource. * * Used to avoid frequent repeated user-space handle lookups of the * same resource. */ struct vmw_res_cache_entry { - bool valid; uint32_t handle; struct vmw_resource *res; - struct vmw_resource_val_node *node; + void *private; + unsigned short valid_handle; + unsigned short valid; }; /** @@ -291,35 +315,63 @@ enum vmw_display_unit_type { vmw_du_screen_target }; +struct vmw_validation_context; +struct vmw_ctx_validation_info; +/** + * struct vmw_sw_context - Command submission context + * @res_ht: Pointer hash table used to find validation duplicates + * @kernel: Whether the command buffer originates from kernel code rather + * than from user-space + * @fp: If @kernel is false, points to the file of the client. Otherwise + * NULL + * @cmd_bounce: Command bounce buffer used for command validation before + * copying to fifo space + * @cmd_bounce_size: Current command bounce buffer size + * @cur_query_bo: Current buffer object used as query result buffer + * @bo_relocations: List of buffer object relocations + * @res_relocations: List of resource relocations + * @buf_start: Pointer to start of memory where command validation takes + * place + * @res_cache: Cache of recently looked up resources + * @last_query_ctx: Last context that submitted a query + * @needs_post_query_barrier: Whether a query barrier is needed after + * command submission + * @staged_bindings: Cached per-context binding tracker + * @staged_bindings_inuse: Whether the cached per-context binding tracker + * is in use + * @staged_cmd_res: List of staged command buffer managed resources in this + * command buffer + * @ctx_list: List of context resources referenced in this command buffer + * @dx_ctx_node: Validation metadata of the current DX context + * @dx_query_mob: The MOB used for DX queries + * @dx_query_ctx: The DX context used for the last DX query + * @man: Pointer to the command buffer managed resource manager + * @ctx: The validation context + */ struct vmw_sw_context{ struct drm_open_hash res_ht; bool res_ht_initialized; - bool kernel; /**< is the called made from the kernel */ + bool kernel; struct vmw_fpriv *fp; - struct list_head validate_nodes; - struct vmw_relocation relocs[VMWGFX_MAX_RELOCATIONS]; - uint32_t cur_reloc; - struct vmw_validate_buffer val_bufs[VMWGFX_MAX_VALIDATIONS]; - uint32_t cur_val_buf; uint32_t *cmd_bounce; uint32_t cmd_bounce_size; - struct list_head resource_list; - struct list_head ctx_resource_list; /* For contexts and cotables */ struct vmw_buffer_object *cur_query_bo; + struct list_head bo_relocations; struct list_head res_relocations; uint32_t *buf_start; struct vmw_res_cache_entry res_cache[vmw_res_max]; struct vmw_resource *last_query_ctx; bool needs_post_query_barrier; - struct vmw_resource *error_resource; struct vmw_ctx_binding_state *staged_bindings; bool staged_bindings_inuse; struct list_head staged_cmd_res; - struct vmw_resource_val_node *dx_ctx_node; + struct list_head ctx_list; + struct vmw_ctx_validation_info *dx_ctx_node; struct vmw_buffer_object *dx_query_mob; struct vmw_resource *dx_query_ctx; struct vmw_cmdbuf_res_manager *man; + struct vmw_validation_context *ctx; }; struct vmw_legacy_display; @@ -444,7 +496,7 @@ struct vmw_private { * Context and surface management. */ - rwlock_t resource_lock; + spinlock_t resource_lock; struct idr res_idr[vmw_res_max]; /* * Block lastclose from racing with firstopen. @@ -628,7 +680,7 @@ extern void vmw_resource_unreference(struct vmw_resource **p_res); extern struct vmw_resource *vmw_resource_reference(struct vmw_resource *res); extern struct vmw_resource * vmw_resource_reference_unless_doomed(struct vmw_resource *res); -extern int vmw_resource_validate(struct vmw_resource *res); +extern int vmw_resource_validate(struct vmw_resource *res, bool intr); extern int vmw_resource_reserve(struct vmw_resource *res, bool interruptible, bool no_backup); extern bool vmw_resource_needs_backup(const struct vmw_resource *res); @@ -643,6 +695,12 @@ extern int vmw_user_resource_lookup_handle( uint32_t handle, const struct vmw_user_resource_conv *converter, struct vmw_resource **p_res); +extern struct vmw_resource * +vmw_user_resource_noref_lookup_handle(struct vmw_private *dev_priv, + struct ttm_object_file *tfile, + uint32_t handle, + const struct vmw_user_resource_conv * + converter); extern int vmw_stream_claim_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv); extern int vmw_stream_unref_ioctl(struct drm_device *dev, void *data, @@ -662,6 +720,15 @@ extern void vmw_resource_evict_all(struct vmw_private *dev_priv); extern void vmw_resource_unbind_list(struct vmw_buffer_object *vbo); /** + * vmw_user_resource_noref_release - release a user resource pointer looked up + * without reference + */ +static inline void vmw_user_resource_noref_release(void) +{ + ttm_base_object_noref_release(); +} + +/** * Buffer object helper functions - vmwgfx_bo.c */ extern int vmw_bo_pin_in_placement(struct vmw_private *vmw_priv, @@ -717,6 +784,18 @@ extern void vmw_bo_unmap(struct vmw_buffer_object *vbo); extern void vmw_bo_move_notify(struct ttm_buffer_object *bo, struct ttm_mem_reg *mem); extern void vmw_bo_swap_notify(struct ttm_buffer_object *bo); +extern struct vmw_buffer_object * +vmw_user_bo_noref_lookup(struct ttm_object_file *tfile, u32 handle); + +/** + * vmw_user_bo_noref_release - release a buffer object pointer looked up + * without reference + */ +static inline void vmw_user_bo_noref_release(void) +{ + ttm_base_object_noref_release(); +} + /** * Misc Ioctl functionality - vmwgfx_ioctl.c @@ -864,10 +943,6 @@ extern void vmw_execbuf_copy_fence_user(struct vmw_private *dev_priv, uint32_t fence_handle, int32_t out_fence_fd, struct sync_file *sync_file); -extern int vmw_validate_single_buffer(struct vmw_private *dev_priv, - struct ttm_buffer_object *bo, - bool interruptible, - bool validate_as_mob); bool vmw_cmd_describe(const void *buf, u32 *size, char const **cmd); /** diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c index 1f134570b759..5a6b70ba137a 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c @@ -35,6 +35,23 @@ #define VMW_RES_HT_ORDER 12 +/* + * struct vmw_relocation - Buffer object relocation + * + * @head: List head for the command submission context's relocation list + * @vbo: Non ref-counted pointer to buffer object + * @mob_loc: Pointer to location for mob id to be modified + * @location: Pointer to location for guest pointer to be modified + */ +struct vmw_relocation { + struct list_head head; + struct vmw_buffer_object *vbo; + union { + SVGAMobId *mob_loc; + SVGAGuestPtr *location; + }; +}; + /** * enum vmw_resource_relocation_type - Relocation type for resources * @@ -69,35 +86,18 @@ struct vmw_resource_relocation { enum vmw_resource_relocation_type rel_type:3; }; -/** - * struct vmw_resource_val_node - Validation info for resources - * - * @head: List head for the software context's resource list. - * @hash: Hash entry for quick resouce to val_node lookup. - * @res: Ref-counted pointer to the resource. - * @switch_backup: Boolean whether to switch backup buffer on unreserve. - * @new_backup: Refcounted pointer to the new backup buffer. - * @staged_bindings: If @res is a context, tracks bindings set up during - * the command batch. Otherwise NULL. - * @new_backup_offset: New backup buffer offset if @new_backup is non-NUll. - * @first_usage: Set to true the first time the resource is referenced in - * the command stream. - * @switching_backup: The command stream provides a new backup buffer for a - * resource. - * @no_buffer_needed: This means @switching_backup is true on first buffer - * reference. So resource reservation does not need to allocate a backup - * buffer for the resource. +/* + * struct vmw_ctx_validation_info - Extra validation metadata for contexts + * @head: List head of context list + * @ctx: The context resource + * @cur: The context's persistent binding state + * @staged: The binding state changes of this command buffer */ -struct vmw_resource_val_node { +struct vmw_ctx_validation_info { struct list_head head; - struct drm_hash_item hash; - struct vmw_resource *res; - struct vmw_buffer_object *new_backup; - struct vmw_ctx_binding_state *staged_bindings; - unsigned long new_backup_offset; - u32 first_usage : 1; - u32 switching_backup : 1; - u32 no_buffer_needed : 1; + struct vmw_resource *ctx; + struct vmw_ctx_binding_state *cur; + struct vmw_ctx_binding_state *staged; }; /** @@ -127,10 +127,6 @@ static int vmw_translate_mob_ptr(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGAMobId *id, struct vmw_buffer_object **vmw_bo_p); -static int vmw_bo_to_validate_list(struct vmw_sw_context *sw_context, - struct vmw_buffer_object *vbo, - bool validate_as_mob, - uint32_t *p_val_node); /** * vmw_ptr_diff - Compute the offset from a to b in bytes * @@ -145,48 +141,38 @@ static size_t vmw_ptr_diff(void *a, void *b) } /** - * vmw_resources_unreserve - unreserve resources previously reserved for - * command submission. - * - * @sw_context: pointer to the software context - * @backoff: Whether command submission failed. + * vmw_execbuf_bindings_commit - Commit modified binding state + * @sw_context: The command submission context + * @backoff: Whether this is part of the error path and binding state + * changes should be ignored */ -static void vmw_resources_unreserve(struct vmw_sw_context *sw_context, - bool backoff) +static void vmw_execbuf_bindings_commit(struct vmw_sw_context *sw_context, + bool backoff) { - struct vmw_resource_val_node *val; - struct list_head *list = &sw_context->resource_list; + struct vmw_ctx_validation_info *entry; - if (sw_context->dx_query_mob && !backoff) - vmw_context_bind_dx_query(sw_context->dx_query_ctx, - sw_context->dx_query_mob); + list_for_each_entry(entry, &sw_context->ctx_list, head) { + if (!backoff) + vmw_binding_state_commit(entry->cur, entry->staged); + if (entry->staged != sw_context->staged_bindings) + vmw_binding_state_free(entry->staged); + else + sw_context->staged_bindings_inuse = false; + } - list_for_each_entry(val, list, head) { - struct vmw_resource *res = val->res; - bool switch_backup = - (backoff) ? false : val->switching_backup; - - /* - * Transfer staged context bindings to the - * persistent context binding tracker. - */ - if (unlikely(val->staged_bindings)) { - if (!backoff) { - vmw_binding_state_commit - (vmw_context_binding_state(val->res), - val->staged_bindings); - } + /* List entries are freed with the validation context */ + INIT_LIST_HEAD(&sw_context->ctx_list); +} - if (val->staged_bindings != sw_context->staged_bindings) - vmw_binding_state_free(val->staged_bindings); - else - sw_context->staged_bindings_inuse = false; - val->staged_bindings = NULL; - } - vmw_resource_unreserve(res, switch_backup, val->new_backup, - val->new_backup_offset); - vmw_bo_unreference(&val->new_backup); - } +/** + * vmw_bind_dx_query_mob - Bind the DX query MOB if referenced + * @sw_context: The command submission context + */ +static void vmw_bind_dx_query_mob(struct vmw_sw_context *sw_context) +{ + if (sw_context->dx_query_mob) + vmw_context_bind_dx_query(sw_context->dx_query_ctx, + sw_context->dx_query_mob); } /** @@ -194,16 +180,17 @@ static void vmw_resources_unreserve(struct vmw_sw_context *sw_context, * added to the validate list. * * @dev_priv: Pointer to the device private: - * @sw_context: The validation context: - * @node: The validation node holding this context. + * @sw_context: The command submission context + * @node: The validation node holding the context resource metadata */ static int vmw_cmd_ctx_first_setup(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, - struct vmw_resource_val_node *node) + struct vmw_resource *res, + struct vmw_ctx_validation_info *node) { int ret; - ret = vmw_resource_context_res_add(dev_priv, sw_context, node->res); + ret = vmw_resource_context_res_add(dev_priv, sw_context, res); if (unlikely(ret != 0)) goto out_err; @@ -220,91 +207,138 @@ static int vmw_cmd_ctx_first_setup(struct vmw_private *dev_priv, } if (sw_context->staged_bindings_inuse) { - node->staged_bindings = vmw_binding_state_alloc(dev_priv); - if (IS_ERR(node->staged_bindings)) { + node->staged = vmw_binding_state_alloc(dev_priv); + if (IS_ERR(node->staged)) { DRM_ERROR("Failed to allocate context binding " "information.\n"); - ret = PTR_ERR(node->staged_bindings); - node->staged_bindings = NULL; + ret = PTR_ERR(node->staged); + node->staged = NULL; goto out_err; } } else { - node->staged_bindings = sw_context->staged_bindings; + node->staged = sw_context->staged_bindings; sw_context->staged_bindings_inuse = true; } + node->ctx = res; + node->cur = vmw_context_binding_state(res); + list_add_tail(&node->head, &sw_context->ctx_list); + return 0; out_err: return ret; } /** - * vmw_resource_val_add - Add a resource to the software context's - * resource list if it's not already on it. + * vmw_execbuf_res_size - calculate extra size fore the resource validation + * node + * @dev_priv: Pointer to the device private struct. + * @res_type: The resource type. * - * @sw_context: Pointer to the software context. + * Guest-backed contexts and DX contexts require extra size to store + * execbuf private information in the validation node. Typically the + * binding manager associated data structures. + * + * Returns: The extra size requirement based on resource type. + */ +static unsigned int vmw_execbuf_res_size(struct vmw_private *dev_priv, + enum vmw_res_type res_type) +{ + return (res_type == vmw_res_dx_context || + (res_type == vmw_res_context && dev_priv->has_mob)) ? + sizeof(struct vmw_ctx_validation_info) : 0; +} + +/** + * vmw_execbuf_rcache_update - Update a resource-node cache entry + * + * @rcache: Pointer to the entry to update. * @res: Pointer to the resource. - * @p_node On successful return points to a valid pointer to a - * struct vmw_resource_val_node, if non-NULL on entry. + * @private: Pointer to the execbuf-private space in the resource + * validation node. + */ +static void vmw_execbuf_rcache_update(struct vmw_res_cache_entry *rcache, + struct vmw_resource *res, + void *private) +{ + rcache->res = res; + rcache->private = private; + rcache->valid = 1; + rcache->valid_handle = 0; +} + +/** + * vmw_execbuf_res_noref_val_add - Add a resource described by an + * unreferenced rcu-protected pointer to the validation list. + * @sw_context: Pointer to the software context. + * @res: Unreferenced rcu-protected pointer to the resource. + * + * Returns: 0 on success. Negative error code on failure. Typical error + * codes are %-EINVAL on inconsistency and %-ESRCH if the resource was + * doomed. */ -static int vmw_resource_val_add(struct vmw_sw_context *sw_context, - struct vmw_resource *res, - struct vmw_resource_val_node **p_node) +static int vmw_execbuf_res_noref_val_add(struct vmw_sw_context *sw_context, + struct vmw_resource *res) { struct vmw_private *dev_priv = res->dev_priv; - struct vmw_resource_val_node *node; - struct drm_hash_item *hash; int ret; - - if (likely(drm_ht_find_item(&sw_context->res_ht, (unsigned long) res, - &hash) == 0)) { - node = container_of(hash, struct vmw_resource_val_node, hash); - node->first_usage = false; - if (unlikely(p_node != NULL)) - *p_node = node; + enum vmw_res_type res_type = vmw_res_type(res); + struct vmw_res_cache_entry *rcache; + struct vmw_ctx_validation_info *ctx_info; + bool first_usage; + unsigned int priv_size; + + rcache = &sw_context->res_cache[res_type]; + if (likely(rcache->valid && rcache->res == res)) { + vmw_user_resource_noref_release(); return 0; } - node = kzalloc(sizeof(*node), GFP_KERNEL); - if (unlikely(!node)) { - DRM_ERROR("Failed to allocate a resource validation " - "entry.\n"); - return -ENOMEM; - } - - node->hash.key = (unsigned long) res; - ret = drm_ht_insert_item(&sw_context->res_ht, &node->hash); - if (unlikely(ret != 0)) { - DRM_ERROR("Failed to initialize a resource validation " - "entry.\n"); - kfree(node); + priv_size = vmw_execbuf_res_size(dev_priv, res_type); + ret = vmw_validation_add_resource(sw_context->ctx, res, priv_size, + (void **)&ctx_info, &first_usage); + vmw_user_resource_noref_release(); + if (ret) return ret; + + if (priv_size && first_usage) { + ret = vmw_cmd_ctx_first_setup(dev_priv, sw_context, res, + ctx_info); + if (ret) + return ret; } - node->res = vmw_resource_reference(res); - node->first_usage = true; - if (unlikely(p_node != NULL)) - *p_node = node; - if (!dev_priv->has_mob) { - list_add_tail(&node->head, &sw_context->resource_list); + vmw_execbuf_rcache_update(rcache, res, ctx_info); + return 0; +} + +/** + * vmw_execbuf_res_noctx_val_add - Add a non-context resource to the resource + * validation list if it's not already on it + * @sw_context: Pointer to the software context. + * @res: Pointer to the resource. + * + * Returns: Zero on success. Negative error code on failure. + */ +static int vmw_execbuf_res_noctx_val_add(struct vmw_sw_context *sw_context, + struct vmw_resource *res) +{ + struct vmw_res_cache_entry *rcache; + enum vmw_res_type res_type = vmw_res_type(res); + void *ptr; + int ret; + + rcache = &sw_context->res_cache[res_type]; + if (likely(rcache->valid && rcache->res == res)) return 0; - } - switch (vmw_res_type(res)) { - case vmw_res_context: - case vmw_res_dx_context: - list_add(&node->head, &sw_context->ctx_resource_list); - ret = vmw_cmd_ctx_first_setup(dev_priv, sw_context, node); - break; - case vmw_res_cotable: - list_add_tail(&node->head, &sw_context->ctx_resource_list); - break; - default: - list_add_tail(&node->head, &sw_context->resource_list); - break; - } + ret = vmw_validation_add_resource(sw_context->ctx, res, 0, &ptr, NULL); + if (ret) + return ret; - return ret; + vmw_execbuf_rcache_update(rcache, res, ptr); + + return 0; } /** @@ -325,11 +359,11 @@ static int vmw_view_res_val_add(struct vmw_sw_context *sw_context, * First add the resource the view is pointing to, otherwise * it may be swapped out when the view is validated. */ - ret = vmw_resource_val_add(sw_context, vmw_view_srf(view), NULL); + ret = vmw_execbuf_res_noctx_val_add(sw_context, vmw_view_srf(view)); if (ret) return ret; - return vmw_resource_val_add(sw_context, view, NULL); + return vmw_execbuf_res_noctx_val_add(sw_context, view); } /** @@ -342,28 +376,33 @@ static int vmw_view_res_val_add(struct vmw_sw_context *sw_context, * * The view is represented by a view id and the DX context it's created on, * or scheduled for creation on. If there is no DX context set, the function - * will return -EINVAL. Otherwise returns 0 on success and -EINVAL on failure. + * will return an -EINVAL error pointer. + * + * Returns: Unreferenced pointer to the resource on success, negative error + * pointer on failure. */ -static int vmw_view_id_val_add(struct vmw_sw_context *sw_context, - enum vmw_view_type view_type, u32 id) +static struct vmw_resource * +vmw_view_id_val_add(struct vmw_sw_context *sw_context, + enum vmw_view_type view_type, u32 id) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_resource *view; int ret; if (!ctx_node) { DRM_ERROR("DX Context not set.\n"); - return -EINVAL; + return ERR_PTR(-EINVAL); } view = vmw_view_lookup(sw_context->man, view_type, id); if (IS_ERR(view)) - return PTR_ERR(view); + return view; ret = vmw_view_res_val_add(sw_context, view); - vmw_resource_unreference(&view); + if (ret) + return ERR_PTR(ret); - return ret; + return view; } /** @@ -394,8 +433,7 @@ static int vmw_resource_context_res_add(struct vmw_private *dev_priv, if (IS_ERR(res)) continue; - ret = vmw_resource_val_add(sw_context, res, NULL); - vmw_resource_unreference(&res); + ret = vmw_execbuf_res_noctx_val_add(sw_context, res); if (unlikely(ret != 0)) return ret; } @@ -407,17 +445,11 @@ static int vmw_resource_context_res_add(struct vmw_private *dev_priv, binding_list = vmw_context_binding_list(ctx); list_for_each_entry(entry, binding_list, ctx_list) { - /* entry->res is not refcounted */ - res = vmw_resource_reference_unless_doomed(entry->res); - if (unlikely(res == NULL)) - continue; - if (vmw_res_type(entry->res) == vmw_res_view) ret = vmw_view_res_val_add(sw_context, entry->res); else - ret = vmw_resource_val_add(sw_context, entry->res, - NULL); - vmw_resource_unreference(&res); + ret = vmw_execbuf_res_noctx_val_add(sw_context, + entry->res); if (unlikely(ret != 0)) break; } @@ -427,9 +459,8 @@ static int vmw_resource_context_res_add(struct vmw_private *dev_priv, dx_query_mob = vmw_context_get_dx_query_mob(ctx); if (dx_query_mob) - ret = vmw_bo_to_validate_list(sw_context, - dx_query_mob, - true, NULL); + ret = vmw_validation_add_bo(sw_context->ctx, + dx_query_mob, true, false); } mutex_unlock(&dev_priv->binding_mutex); @@ -445,7 +476,7 @@ static int vmw_resource_context_res_add(struct vmw_private *dev_priv, * id that needs fixup is located. Granularity is one byte. * @rel_type: Relocation type. */ -static int vmw_resource_relocation_add(struct list_head *list, +static int vmw_resource_relocation_add(struct vmw_sw_context *sw_context, const struct vmw_resource *res, unsigned long offset, enum vmw_resource_relocation_type @@ -453,7 +484,7 @@ static int vmw_resource_relocation_add(struct list_head *list, { struct vmw_resource_relocation *rel; - rel = kmalloc(sizeof(*rel), GFP_KERNEL); + rel = vmw_validation_mem_alloc(sw_context->ctx, sizeof(*rel)); if (unlikely(!rel)) { DRM_ERROR("Failed to allocate a resource relocation.\n"); return -ENOMEM; @@ -462,7 +493,7 @@ static int vmw_resource_relocation_add(struct list_head *list, rel->res = res; rel->offset = offset; rel->rel_type = rel_type; - list_add_tail(&rel->head, list); + list_add_tail(&rel->head, &sw_context->res_relocations); return 0; } @@ -470,16 +501,13 @@ static int vmw_resource_relocation_add(struct list_head *list, /** * vmw_resource_relocations_free - Free all relocations on a list * - * @list: Pointer to the head of the relocation list. + * @list: Pointer to the head of the relocation list */ static void vmw_resource_relocations_free(struct list_head *list) { - struct vmw_resource_relocation *rel, *n; + /* Memory is validation context memory, so no need to free it */ - list_for_each_entry_safe(rel, n, list, head) { - list_del(&rel->head); - kfree(rel); - } + INIT_LIST_HEAD(list); } /** @@ -532,68 +560,6 @@ static int vmw_cmd_ok(struct vmw_private *dev_priv, } /** - * vmw_bo_to_validate_list - add a bo to a validate list - * - * @sw_context: The software context used for this command submission batch. - * @bo: The buffer object to add. - * @validate_as_mob: Validate this buffer as a MOB. - * @p_val_node: If non-NULL Will be updated with the validate node number - * on return. - * - * Returns -EINVAL if the limit of number of buffer objects per command - * submission is reached. - */ -static int vmw_bo_to_validate_list(struct vmw_sw_context *sw_context, - struct vmw_buffer_object *vbo, - bool validate_as_mob, - uint32_t *p_val_node) -{ - uint32_t val_node; - struct vmw_validate_buffer *vval_buf; - struct ttm_validate_buffer *val_buf; - struct drm_hash_item *hash; - int ret; - - if (likely(drm_ht_find_item(&sw_context->res_ht, (unsigned long) vbo, - &hash) == 0)) { - vval_buf = container_of(hash, struct vmw_validate_buffer, - hash); - if (unlikely(vval_buf->validate_as_mob != validate_as_mob)) { - DRM_ERROR("Inconsistent buffer usage.\n"); - return -EINVAL; - } - val_buf = &vval_buf->base; - val_node = vval_buf - sw_context->val_bufs; - } else { - val_node = sw_context->cur_val_buf; - if (unlikely(val_node >= VMWGFX_MAX_VALIDATIONS)) { - DRM_ERROR("Max number of DMA buffers per submission " - "exceeded.\n"); - return -EINVAL; - } - vval_buf = &sw_context->val_bufs[val_node]; - vval_buf->hash.key = (unsigned long) vbo; - ret = drm_ht_insert_item(&sw_context->res_ht, &vval_buf->hash); - if (unlikely(ret != 0)) { - DRM_ERROR("Failed to initialize a buffer validation " - "entry.\n"); - return ret; - } - ++sw_context->cur_val_buf; - val_buf = &vval_buf->base; - val_buf->bo = ttm_bo_reference(&vbo->base); - val_buf->shared = false; - list_add_tail(&val_buf->head, &sw_context->validate_nodes); - vval_buf->validate_as_mob = validate_as_mob; - } - - if (p_val_node) - *p_val_node = val_node; - - return 0; -} - -/** * vmw_resources_reserve - Reserve all resources on the sw_context's * resource list. * @@ -605,27 +571,11 @@ static int vmw_bo_to_validate_list(struct vmw_sw_context *sw_context, */ static int vmw_resources_reserve(struct vmw_sw_context *sw_context) { - struct vmw_resource_val_node *val; - int ret = 0; - - list_for_each_entry(val, &sw_context->resource_list, head) { - struct vmw_resource *res = val->res; - - ret = vmw_resource_reserve(res, true, val->no_buffer_needed); - if (unlikely(ret != 0)) - return ret; - - if (res->backup) { - struct vmw_buffer_object *vbo = res->backup; - - ret = vmw_bo_to_validate_list - (sw_context, vbo, - vmw_resource_needs_backup(res), NULL); + int ret; - if (unlikely(ret != 0)) - return ret; - } - } + ret = vmw_validation_res_reserve(sw_context->ctx, true); + if (ret) + return ret; if (sw_context->dx_query_mob) { struct vmw_buffer_object *expected_dx_query_mob; @@ -642,87 +592,6 @@ static int vmw_resources_reserve(struct vmw_sw_context *sw_context) } /** - * vmw_resources_validate - Validate all resources on the sw_context's - * resource list. - * - * @sw_context: Pointer to the software context. - * - * Before this function is called, all resource backup buffers must have - * been validated. - */ -static int vmw_resources_validate(struct vmw_sw_context *sw_context) -{ - struct vmw_resource_val_node *val; - int ret; - - list_for_each_entry(val, &sw_context->resource_list, head) { - struct vmw_resource *res = val->res; - struct vmw_buffer_object *backup = res->backup; - - ret = vmw_resource_validate(res); - if (unlikely(ret != 0)) { - if (ret != -ERESTARTSYS) - DRM_ERROR("Failed to validate resource.\n"); - return ret; - } - - /* Check if the resource switched backup buffer */ - if (backup && res->backup && (backup != res->backup)) { - struct vmw_buffer_object *vbo = res->backup; - - ret = vmw_bo_to_validate_list - (sw_context, vbo, - vmw_resource_needs_backup(res), NULL); - if (ret) { - ttm_bo_unreserve(&vbo->base); - return ret; - } - } - } - return 0; -} - -/** - * vmw_cmd_res_reloc_add - Add a resource to a software context's - * relocation- and validation lists. - * - * @dev_priv: Pointer to a struct vmw_private identifying the device. - * @sw_context: Pointer to the software context. - * @id_loc: Pointer to where the id that needs translation is located. - * @res: Valid pointer to a struct vmw_resource. - * @p_val: If non null, a pointer to the struct vmw_resource_validate_node - * used for this resource is returned here. - */ -static int vmw_cmd_res_reloc_add(struct vmw_private *dev_priv, - struct vmw_sw_context *sw_context, - uint32_t *id_loc, - struct vmw_resource *res, - struct vmw_resource_val_node **p_val) -{ - int ret; - struct vmw_resource_val_node *node; - - *p_val = NULL; - ret = vmw_resource_relocation_add(&sw_context->res_relocations, - res, - vmw_ptr_diff(sw_context->buf_start, - id_loc), - vmw_res_rel_normal); - if (unlikely(ret != 0)) - return ret; - - ret = vmw_resource_val_add(sw_context, res, &node); - if (unlikely(ret != 0)) - return ret; - - if (p_val) - *p_val = node; - - return 0; -} - - -/** * vmw_cmd_res_check - Check that a resource is present and if so, put it * on the resource validate list unless it's already there. * @@ -741,17 +610,16 @@ vmw_cmd_res_check(struct vmw_private *dev_priv, enum vmw_res_type res_type, const struct vmw_user_resource_conv *converter, uint32_t *id_loc, - struct vmw_resource_val_node **p_val) + struct vmw_resource **p_res) { - struct vmw_res_cache_entry *rcache = - &sw_context->res_cache[res_type]; + struct vmw_res_cache_entry *rcache = &sw_context->res_cache[res_type]; struct vmw_resource *res; - struct vmw_resource_val_node *node; int ret; + if (p_res) + *p_res = NULL; + if (*id_loc == SVGA3D_INVALID_ID) { - if (p_val) - *p_val = NULL; if (res_type == vmw_res_context) { DRM_ERROR("Illegal context invalid id.\n"); return -EINVAL; @@ -759,56 +627,41 @@ vmw_cmd_res_check(struct vmw_private *dev_priv, return 0; } - /* - * Fastpath in case of repeated commands referencing the same - * resource - */ + if (likely(rcache->valid_handle && *id_loc == rcache->handle)) { + res = rcache->res; + } else { + unsigned int size = vmw_execbuf_res_size(dev_priv, res_type); - if (likely(rcache->valid && *id_loc == rcache->handle)) { - const struct vmw_resource *res = rcache->res; + ret = vmw_validation_preload_res(sw_context->ctx, size); + if (ret) + return ret; - rcache->node->first_usage = false; - if (p_val) - *p_val = rcache->node; + res = vmw_user_resource_noref_lookup_handle + (dev_priv, sw_context->fp->tfile, *id_loc, converter); + if (unlikely(IS_ERR(res))) { + DRM_ERROR("Could not find or use resource 0x%08x.\n", + (unsigned int) *id_loc); + return PTR_ERR(res); + } - return vmw_resource_relocation_add - (&sw_context->res_relocations, res, - vmw_ptr_diff(sw_context->buf_start, id_loc), - vmw_res_rel_normal); - } + ret = vmw_execbuf_res_noref_val_add(sw_context, res); + if (unlikely(ret != 0)) + return ret; - ret = vmw_user_resource_lookup_handle(dev_priv, - sw_context->fp->tfile, - *id_loc, - converter, - &res); - if (unlikely(ret != 0)) { - DRM_ERROR("Could not find or use resource 0x%08x.\n", - (unsigned) *id_loc); - dump_stack(); - return ret; + if (rcache->valid && rcache->res == res) { + rcache->valid_handle = true; + rcache->handle = *id_loc; + } } - rcache->valid = true; - rcache->res = res; - rcache->handle = *id_loc; - - ret = vmw_cmd_res_reloc_add(dev_priv, sw_context, id_loc, - res, &node); - if (unlikely(ret != 0)) - goto out_no_reloc; + ret = vmw_resource_relocation_add(sw_context, res, + vmw_ptr_diff(sw_context->buf_start, + id_loc), + vmw_res_rel_normal); + if (p_res) + *p_res = res; - rcache->node = node; - if (p_val) - *p_val = node; - vmw_resource_unreference(&res); return 0; - -out_no_reloc: - BUG_ON(sw_context->error_resource != NULL); - sw_context->error_resource = res; - - return ret; } /** @@ -861,22 +714,18 @@ static int vmw_rebind_all_dx_query(struct vmw_resource *ctx_res) */ static int vmw_rebind_contexts(struct vmw_sw_context *sw_context) { - struct vmw_resource_val_node *val; + struct vmw_ctx_validation_info *val; int ret; - list_for_each_entry(val, &sw_context->resource_list, head) { - if (unlikely(!val->staged_bindings)) - break; - - ret = vmw_binding_rebind_all - (vmw_context_binding_state(val->res)); + list_for_each_entry(val, &sw_context->ctx_list, head) { + ret = vmw_binding_rebind_all(val->cur); if (unlikely(ret != 0)) { if (ret != -ERESTARTSYS) DRM_ERROR("Failed to rebind context.\n"); return ret; } - ret = vmw_rebind_all_dx_query(val->res); + ret = vmw_rebind_all_dx_query(val->ctx); if (ret != 0) return ret; } @@ -903,45 +752,33 @@ static int vmw_view_bindings_add(struct vmw_sw_context *sw_context, uint32 view_ids[], u32 num_views, u32 first_slot) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; - struct vmw_cmdbuf_res_manager *man; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; u32 i; - int ret; if (!ctx_node) { DRM_ERROR("DX Context not set.\n"); return -EINVAL; } - man = sw_context->man; for (i = 0; i < num_views; ++i) { struct vmw_ctx_bindinfo_view binding; struct vmw_resource *view = NULL; if (view_ids[i] != SVGA3D_INVALID_ID) { - view = vmw_view_lookup(man, view_type, view_ids[i]); + view = vmw_view_id_val_add(sw_context, view_type, + view_ids[i]); if (IS_ERR(view)) { DRM_ERROR("View not found.\n"); return PTR_ERR(view); } - - ret = vmw_view_res_val_add(sw_context, view); - if (ret) { - DRM_ERROR("Could not add view to " - "validation list.\n"); - vmw_resource_unreference(&view); - return ret; - } } - binding.bi.ctx = ctx_node->res; + binding.bi.ctx = ctx_node->ctx; binding.bi.res = view; binding.bi.bt = binding_type; binding.shader_slot = shader_slot; binding.slot = first_slot + i; - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, + vmw_binding_add(ctx_node->staged, &binding.bi, shader_slot, binding.slot); - if (view) - vmw_resource_unreference(&view); } return 0; @@ -971,6 +808,34 @@ static int vmw_cmd_cid_check(struct vmw_private *dev_priv, user_context_converter, &cmd->cid, NULL); } +/** + * vmw_execbuf_info_from_res - Get the private validation metadata for a + * recently validated resource + * @sw_context: Pointer to the command submission context + * @res: The resource + * + * The resource pointed to by @res needs to be present in the command submission + * context's resource cache and hence the last resource of that type to be + * processed by the validation code. + * + * Return: a pointer to the private metadata of the resource, or NULL + * if it wasn't found + */ +static struct vmw_ctx_validation_info * +vmw_execbuf_info_from_res(struct vmw_sw_context *sw_context, + struct vmw_resource *res) +{ + struct vmw_res_cache_entry *rcache = + &sw_context->res_cache[vmw_res_type(res)]; + + if (rcache->valid && rcache->res == res) + return rcache->private; + + WARN_ON_ONCE(true); + return NULL; +} + + static int vmw_cmd_set_render_target_check(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) @@ -979,8 +844,8 @@ static int vmw_cmd_set_render_target_check(struct vmw_private *dev_priv, SVGA3dCmdHeader header; SVGA3dCmdSetRenderTarget body; } *cmd; - struct vmw_resource_val_node *ctx_node; - struct vmw_resource_val_node *res_node; + struct vmw_resource *ctx; + struct vmw_resource *res; int ret; cmd = container_of(header, struct vmw_sid_cmd, header); @@ -993,25 +858,29 @@ static int vmw_cmd_set_render_target_check(struct vmw_private *dev_priv, ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_context, user_context_converter, &cmd->body.cid, - &ctx_node); + &ctx); if (unlikely(ret != 0)) return ret; ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, - user_surface_converter, - &cmd->body.target.sid, &res_node); - if (unlikely(ret != 0)) + user_surface_converter, &cmd->body.target.sid, + &res); + if (unlikely(ret)) return ret; if (dev_priv->has_mob) { struct vmw_ctx_bindinfo_view binding; + struct vmw_ctx_validation_info *node; - binding.bi.ctx = ctx_node->res; - binding.bi.res = res_node ? res_node->res : NULL; + node = vmw_execbuf_info_from_res(sw_context, ctx); + if (!node) + return -EINVAL; + + binding.bi.ctx = ctx; + binding.bi.res = res; binding.bi.bt = vmw_ctx_binding_rt; binding.slot = cmd->body.type; - vmw_binding_add(ctx_node->staged_bindings, - &binding.bi, 0, binding.slot); + vmw_binding_add(node->staged, &binding.bi, 0, binding.slot); } return 0; @@ -1030,8 +899,8 @@ static int vmw_cmd_surface_copy_check(struct vmw_private *dev_priv, cmd = container_of(header, struct vmw_sid_cmd, header); ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, - user_surface_converter, - &cmd->body.src.sid, NULL); + user_surface_converter, + &cmd->body.src.sid, NULL); if (ret) return ret; @@ -1171,17 +1040,17 @@ static int vmw_query_bo_switch_prepare(struct vmw_private *dev_priv, if (unlikely(sw_context->cur_query_bo != NULL)) { sw_context->needs_post_query_barrier = true; - ret = vmw_bo_to_validate_list(sw_context, - sw_context->cur_query_bo, - dev_priv->has_mob, NULL); + ret = vmw_validation_add_bo(sw_context->ctx, + sw_context->cur_query_bo, + dev_priv->has_mob, false); if (unlikely(ret != 0)) return ret; } sw_context->cur_query_bo = new_query_bo; - ret = vmw_bo_to_validate_list(sw_context, - dev_priv->dummy_query_bo, - dev_priv->has_mob, NULL); + ret = vmw_validation_add_bo(sw_context->ctx, + dev_priv->dummy_query_bo, + dev_priv->has_mob, false); if (unlikely(ret != 0)) return ret; @@ -1269,7 +1138,7 @@ static void vmw_query_bo_switch_commit(struct vmw_private *dev_priv, * @sw_context: The software context used for this command batch validation. * @id: Pointer to the user-space handle to be translated. * @vmw_bo_p: Points to a location that, on successful return will carry - * a reference-counted pointer to the DMA buffer identified by the + * a non-reference-counted pointer to the buffer object identified by the * user-space handle in @id. * * This function saves information needed to translate a user-space buffer @@ -1284,40 +1153,34 @@ static int vmw_translate_mob_ptr(struct vmw_private *dev_priv, SVGAMobId *id, struct vmw_buffer_object **vmw_bo_p) { - struct vmw_buffer_object *vmw_bo = NULL; + struct vmw_buffer_object *vmw_bo; uint32_t handle = *id; struct vmw_relocation *reloc; int ret; - ret = vmw_user_bo_lookup(sw_context->fp->tfile, handle, &vmw_bo, NULL); - if (unlikely(ret != 0)) { + vmw_validation_preload_bo(sw_context->ctx); + vmw_bo = vmw_user_bo_noref_lookup(sw_context->fp->tfile, handle); + if (IS_ERR(vmw_bo)) { DRM_ERROR("Could not find or use MOB buffer.\n"); - ret = -EINVAL; - goto out_no_reloc; + return PTR_ERR(vmw_bo); } - if (unlikely(sw_context->cur_reloc >= VMWGFX_MAX_RELOCATIONS)) { - DRM_ERROR("Max number relocations per submission" - " exceeded\n"); - ret = -EINVAL; - goto out_no_reloc; - } + ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo, true, false); + vmw_user_bo_noref_release(); + if (unlikely(ret != 0)) + return ret; - reloc = &sw_context->relocs[sw_context->cur_reloc++]; - reloc->mob_loc = id; - reloc->location = NULL; + reloc = vmw_validation_mem_alloc(sw_context->ctx, sizeof(*reloc)); + if (!reloc) + return -ENOMEM; - ret = vmw_bo_to_validate_list(sw_context, vmw_bo, true, &reloc->index); - if (unlikely(ret != 0)) - goto out_no_reloc; + reloc->mob_loc = id; + reloc->vbo = vmw_bo; *vmw_bo_p = vmw_bo; - return 0; + list_add_tail(&reloc->head, &sw_context->bo_relocations); -out_no_reloc: - vmw_bo_unreference(&vmw_bo); - *vmw_bo_p = NULL; - return ret; + return 0; } /** @@ -1328,7 +1191,7 @@ out_no_reloc: * @sw_context: The software context used for this command batch validation. * @ptr: Pointer to the user-space handle to be translated. * @vmw_bo_p: Points to a location that, on successful return will carry - * a reference-counted pointer to the DMA buffer identified by the + * a non-reference-counted pointer to the DMA buffer identified by the * user-space handle in @id. * * This function saves information needed to translate a user-space buffer @@ -1344,39 +1207,33 @@ static int vmw_translate_guest_ptr(struct vmw_private *dev_priv, SVGAGuestPtr *ptr, struct vmw_buffer_object **vmw_bo_p) { - struct vmw_buffer_object *vmw_bo = NULL; + struct vmw_buffer_object *vmw_bo; uint32_t handle = ptr->gmrId; struct vmw_relocation *reloc; int ret; - ret = vmw_user_bo_lookup(sw_context->fp->tfile, handle, &vmw_bo, NULL); - if (unlikely(ret != 0)) { + vmw_validation_preload_bo(sw_context->ctx); + vmw_bo = vmw_user_bo_noref_lookup(sw_context->fp->tfile, handle); + if (IS_ERR(vmw_bo)) { DRM_ERROR("Could not find or use GMR region.\n"); - ret = -EINVAL; - goto out_no_reloc; - } - - if (unlikely(sw_context->cur_reloc >= VMWGFX_MAX_RELOCATIONS)) { - DRM_ERROR("Max number relocations per submission" - " exceeded\n"); - ret = -EINVAL; - goto out_no_reloc; + return PTR_ERR(vmw_bo); } - reloc = &sw_context->relocs[sw_context->cur_reloc++]; - reloc->location = ptr; - - ret = vmw_bo_to_validate_list(sw_context, vmw_bo, false, &reloc->index); + ret = vmw_validation_add_bo(sw_context->ctx, vmw_bo, false, false); + vmw_user_bo_noref_release(); if (unlikely(ret != 0)) - goto out_no_reloc; + return ret; + + reloc = vmw_validation_mem_alloc(sw_context->ctx, sizeof(*reloc)); + if (!reloc) + return -ENOMEM; + reloc->location = ptr; + reloc->vbo = vmw_bo; *vmw_bo_p = vmw_bo; - return 0; + list_add_tail(&reloc->head, &sw_context->bo_relocations); -out_no_reloc: - vmw_bo_unreference(&vmw_bo); - *vmw_bo_p = NULL; - return ret; + return 0; } @@ -1400,7 +1257,7 @@ static int vmw_cmd_dx_define_query(struct vmw_private *dev_priv, } *cmd; int ret; - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_resource *cotable_res; @@ -1415,9 +1272,8 @@ static int vmw_cmd_dx_define_query(struct vmw_private *dev_priv, cmd->q.type >= SVGA3D_QUERYTYPE_MAX) return -EINVAL; - cotable_res = vmw_context_cotable(ctx_node->res, SVGA_COTABLE_DXQUERY); + cotable_res = vmw_context_cotable(ctx_node->ctx, SVGA_COTABLE_DXQUERY); ret = vmw_cotable_notify(cotable_res, cmd->q.queryId); - vmw_resource_unreference(&cotable_res); return ret; } @@ -1462,11 +1318,8 @@ static int vmw_cmd_dx_bind_query(struct vmw_private *dev_priv, return ret; sw_context->dx_query_mob = vmw_bo; - sw_context->dx_query_ctx = sw_context->dx_ctx_node->res; - - vmw_bo_unreference(&vmw_bo); - - return ret; + sw_context->dx_query_ctx = sw_context->dx_ctx_node->ctx; + return 0; } @@ -1567,7 +1420,6 @@ static int vmw_cmd_end_gb_query(struct vmw_private *dev_priv, ret = vmw_query_bo_switch_prepare(dev_priv, vmw_bo, sw_context); - vmw_bo_unreference(&vmw_bo); return ret; } @@ -1621,7 +1473,6 @@ static int vmw_cmd_end_query(struct vmw_private *dev_priv, ret = vmw_query_bo_switch_prepare(dev_priv, vmw_bo, sw_context); - vmw_bo_unreference(&vmw_bo); return ret; } @@ -1654,7 +1505,6 @@ static int vmw_cmd_wait_gb_query(struct vmw_private *dev_priv, if (unlikely(ret != 0)) return ret; - vmw_bo_unreference(&vmw_bo); return 0; } @@ -1706,7 +1556,6 @@ static int vmw_cmd_wait_query(struct vmw_private *dev_priv, if (unlikely(ret != 0)) return ret; - vmw_bo_unreference(&vmw_bo); return 0; } @@ -1757,7 +1606,7 @@ static int vmw_cmd_dma(struct vmw_private *dev_priv, if (unlikely(ret != 0)) { if (unlikely(ret != -ERESTARTSYS)) DRM_ERROR("could not find surface for DMA.\n"); - goto out_no_surface; + return ret; } srf = vmw_res_to_srf(sw_context->res_cache[vmw_res_surface].res); @@ -1765,9 +1614,7 @@ static int vmw_cmd_dma(struct vmw_private *dev_priv, vmw_kms_cursor_snoop(srf, sw_context->fp->tfile, &vmw_bo->base, header); -out_no_surface: - vmw_bo_unreference(&vmw_bo); - return ret; + return 0; } static int vmw_cmd_draw(struct vmw_private *dev_priv, @@ -1837,8 +1684,8 @@ static int vmw_cmd_tex_state(struct vmw_private *dev_priv, ((unsigned long) header + header->size + sizeof(header)); SVGA3dTextureState *cur_state = (SVGA3dTextureState *) ((unsigned long) header + sizeof(struct vmw_tex_state_cmd)); - struct vmw_resource_val_node *ctx_node; - struct vmw_resource_val_node *res_node; + struct vmw_resource *ctx; + struct vmw_resource *res; int ret; cmd = container_of(header, struct vmw_tex_state_cmd, @@ -1846,7 +1693,7 @@ static int vmw_cmd_tex_state(struct vmw_private *dev_priv, ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_context, user_context_converter, &cmd->state.cid, - &ctx_node); + &ctx); if (unlikely(ret != 0)) return ret; @@ -1862,19 +1709,24 @@ static int vmw_cmd_tex_state(struct vmw_private *dev_priv, ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, user_surface_converter, - &cur_state->value, &res_node); + &cur_state->value, &res); if (unlikely(ret != 0)) return ret; if (dev_priv->has_mob) { struct vmw_ctx_bindinfo_tex binding; + struct vmw_ctx_validation_info *node; + + node = vmw_execbuf_info_from_res(sw_context, ctx); + if (!node) + return -EINVAL; - binding.bi.ctx = ctx_node->res; - binding.bi.res = res_node ? res_node->res : NULL; + binding.bi.ctx = ctx; + binding.bi.res = res; binding.bi.bt = vmw_ctx_binding_tex; binding.texture_stage = cur_state->stage; - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, - 0, binding.texture_stage); + vmw_binding_add(node->staged, &binding.bi, 0, + binding.texture_stage); } } @@ -1893,14 +1745,9 @@ static int vmw_cmd_check_define_gmrfb(struct vmw_private *dev_priv, SVGAFifoCmdDefineGMRFB body; } *cmd = buf; - ret = vmw_translate_guest_ptr(dev_priv, sw_context, - &cmd->body.ptr, - &vmw_bo); - if (unlikely(ret != 0)) - return ret; - - vmw_bo_unreference(&vmw_bo); - + return vmw_translate_guest_ptr(dev_priv, sw_context, + &cmd->body.ptr, + &vmw_bo); return ret; } @@ -1922,25 +1769,24 @@ static int vmw_cmd_check_define_gmrfb(struct vmw_private *dev_priv, */ static int vmw_cmd_res_switch_backup(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, - struct vmw_resource_val_node *val_node, + struct vmw_resource *res, uint32_t *buf_id, unsigned long backup_offset) { - struct vmw_buffer_object *dma_buf; + struct vmw_buffer_object *vbo; + void *info; int ret; - ret = vmw_translate_mob_ptr(dev_priv, sw_context, buf_id, &dma_buf); + info = vmw_execbuf_info_from_res(sw_context, res); + if (!info) + return -EINVAL; + + ret = vmw_translate_mob_ptr(dev_priv, sw_context, buf_id, &vbo); if (ret) return ret; - val_node->switching_backup = true; - if (val_node->first_usage) - val_node->no_buffer_needed = true; - - vmw_bo_unreference(&val_node->new_backup); - val_node->new_backup = dma_buf; - val_node->new_backup_offset = backup_offset; - + vmw_validation_res_switch_backup(sw_context->ctx, info, vbo, + backup_offset); return 0; } @@ -1970,15 +1816,15 @@ static int vmw_cmd_switch_backup(struct vmw_private *dev_priv, uint32_t *buf_id, unsigned long backup_offset) { - struct vmw_resource_val_node *val_node; + struct vmw_resource *res; int ret; ret = vmw_cmd_res_check(dev_priv, sw_context, res_type, - converter, res_id, &val_node); + converter, res_id, &res); if (ret) return ret; - return vmw_cmd_res_switch_backup(dev_priv, sw_context, val_node, + return vmw_cmd_res_switch_backup(dev_priv, sw_context, res, buf_id, backup_offset); } @@ -2170,14 +2016,14 @@ static int vmw_cmd_shader_define(struct vmw_private *dev_priv, } *cmd; int ret; size_t size; - struct vmw_resource_val_node *val; + struct vmw_resource *ctx; cmd = container_of(header, struct vmw_shader_define_cmd, header); ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_context, user_context_converter, &cmd->body.cid, - &val); + &ctx); if (unlikely(ret != 0)) return ret; @@ -2186,14 +2032,14 @@ static int vmw_cmd_shader_define(struct vmw_private *dev_priv, size = cmd->header.size - sizeof(cmd->body); ret = vmw_compat_shader_add(dev_priv, - vmw_context_res_man(val->res), + vmw_context_res_man(ctx), cmd->body.shid, cmd + 1, cmd->body.type, size, &sw_context->staged_cmd_res); if (unlikely(ret != 0)) return ret; - return vmw_resource_relocation_add(&sw_context->res_relocations, + return vmw_resource_relocation_add(sw_context, NULL, vmw_ptr_diff(sw_context->buf_start, &cmd->header.id), @@ -2217,28 +2063,28 @@ static int vmw_cmd_shader_destroy(struct vmw_private *dev_priv, SVGA3dCmdDestroyShader body; } *cmd; int ret; - struct vmw_resource_val_node *val; + struct vmw_resource *ctx; cmd = container_of(header, struct vmw_shader_destroy_cmd, header); ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_context, user_context_converter, &cmd->body.cid, - &val); + &ctx); if (unlikely(ret != 0)) return ret; if (unlikely(!dev_priv->has_mob)) return 0; - ret = vmw_shader_remove(vmw_context_res_man(val->res), + ret = vmw_shader_remove(vmw_context_res_man(ctx), cmd->body.shid, cmd->body.type, &sw_context->staged_cmd_res); if (unlikely(ret != 0)) return ret; - return vmw_resource_relocation_add(&sw_context->res_relocations, + return vmw_resource_relocation_add(sw_context, NULL, vmw_ptr_diff(sw_context->buf_start, &cmd->header.id), @@ -2261,9 +2107,9 @@ static int vmw_cmd_set_shader(struct vmw_private *dev_priv, SVGA3dCmdHeader header; SVGA3dCmdSetShader body; } *cmd; - struct vmw_resource_val_node *ctx_node, *res_node = NULL; struct vmw_ctx_bindinfo_shader binding; - struct vmw_resource *res = NULL; + struct vmw_resource *ctx, *res = NULL; + struct vmw_ctx_validation_info *ctx_info; int ret; cmd = container_of(header, struct vmw_set_shader_cmd, @@ -2277,7 +2123,7 @@ static int vmw_cmd_set_shader(struct vmw_private *dev_priv, ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_context, user_context_converter, &cmd->body.cid, - &ctx_node); + &ctx); if (unlikely(ret != 0)) return ret; @@ -2285,34 +2131,35 @@ static int vmw_cmd_set_shader(struct vmw_private *dev_priv, return 0; if (cmd->body.shid != SVGA3D_INVALID_ID) { - res = vmw_shader_lookup(vmw_context_res_man(ctx_node->res), + res = vmw_shader_lookup(vmw_context_res_man(ctx), cmd->body.shid, cmd->body.type); if (!IS_ERR(res)) { - ret = vmw_cmd_res_reloc_add(dev_priv, sw_context, - &cmd->body.shid, res, - &res_node); - vmw_resource_unreference(&res); + ret = vmw_execbuf_res_noctx_val_add(sw_context, res); if (unlikely(ret != 0)) return ret; } } - if (!res_node) { + if (IS_ERR_OR_NULL(res)) { ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_shader, user_shader_converter, - &cmd->body.shid, &res_node); + &cmd->body.shid, &res); if (unlikely(ret != 0)) return ret; } - binding.bi.ctx = ctx_node->res; - binding.bi.res = res_node ? res_node->res : NULL; + ctx_info = vmw_execbuf_info_from_res(sw_context, ctx); + if (!ctx_info) + return -EINVAL; + + binding.bi.ctx = ctx; + binding.bi.res = res; binding.bi.bt = vmw_ctx_binding_shader; binding.shader_slot = cmd->body.type - SVGA3D_SHADERTYPE_MIN; - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, + vmw_binding_add(ctx_info->staged, &binding.bi, binding.shader_slot, 0); return 0; } @@ -2393,8 +2240,8 @@ vmw_cmd_dx_set_single_constant_buffer(struct vmw_private *dev_priv, SVGA3dCmdHeader header; SVGA3dCmdDXSetSingleConstantBuffer body; } *cmd; - struct vmw_resource_val_node *res_node = NULL; - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_resource *res = NULL; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_ctx_bindinfo_cb binding; int ret; @@ -2406,12 +2253,12 @@ vmw_cmd_dx_set_single_constant_buffer(struct vmw_private *dev_priv, cmd = container_of(header, typeof(*cmd), header); ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, user_surface_converter, - &cmd->body.sid, &res_node); + &cmd->body.sid, &res); if (unlikely(ret != 0)) return ret; - binding.bi.ctx = ctx_node->res; - binding.bi.res = res_node ? res_node->res : NULL; + binding.bi.ctx = ctx_node->ctx; + binding.bi.res = res; binding.bi.bt = vmw_ctx_binding_cb; binding.shader_slot = cmd->body.type - SVGA3D_SHADERTYPE_MIN; binding.offset = cmd->body.offsetInBytes; @@ -2426,7 +2273,7 @@ vmw_cmd_dx_set_single_constant_buffer(struct vmw_private *dev_priv, return -EINVAL; } - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, + vmw_binding_add(ctx_node->staged, &binding.bi, binding.shader_slot, binding.slot); return 0; @@ -2482,7 +2329,7 @@ static int vmw_cmd_dx_set_shader(struct vmw_private *dev_priv, SVGA3dCmdDXSetShader body; } *cmd; struct vmw_resource *res = NULL; - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_ctx_bindinfo_shader binding; int ret = 0; @@ -2506,23 +2353,20 @@ static int vmw_cmd_dx_set_shader(struct vmw_private *dev_priv, return PTR_ERR(res); } - ret = vmw_resource_val_add(sw_context, res, NULL); + ret = vmw_execbuf_res_noctx_val_add(sw_context, res); if (ret) - goto out_unref; + return ret; } - binding.bi.ctx = ctx_node->res; + binding.bi.ctx = ctx_node->ctx; binding.bi.res = res; binding.bi.bt = vmw_ctx_binding_dx_shader; binding.shader_slot = cmd->body.type - SVGA3D_SHADERTYPE_MIN; - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, + vmw_binding_add(ctx_node->staged, &binding.bi, binding.shader_slot, 0); -out_unref: - if (res) - vmw_resource_unreference(&res); - return ret; + return 0; } /** @@ -2537,9 +2381,9 @@ static int vmw_cmd_dx_set_vertex_buffers(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_ctx_bindinfo_vb binding; - struct vmw_resource_val_node *res_node; + struct vmw_resource *res; struct { SVGA3dCmdHeader header; SVGA3dCmdDXSetVertexBuffers body; @@ -2564,18 +2408,18 @@ static int vmw_cmd_dx_set_vertex_buffers(struct vmw_private *dev_priv, for (i = 0; i < num; i++) { ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, user_surface_converter, - &cmd->buf[i].sid, &res_node); + &cmd->buf[i].sid, &res); if (unlikely(ret != 0)) return ret; - binding.bi.ctx = ctx_node->res; + binding.bi.ctx = ctx_node->ctx; binding.bi.bt = vmw_ctx_binding_vb; - binding.bi.res = ((res_node) ? res_node->res : NULL); + binding.bi.res = res; binding.offset = cmd->buf[i].offset; binding.stride = cmd->buf[i].stride; binding.slot = i + cmd->body.startBuffer; - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, + vmw_binding_add(ctx_node->staged, &binding.bi, 0, binding.slot); } @@ -2594,9 +2438,9 @@ static int vmw_cmd_dx_set_index_buffer(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_ctx_bindinfo_ib binding; - struct vmw_resource_val_node *res_node; + struct vmw_resource *res; struct { SVGA3dCmdHeader header; SVGA3dCmdDXSetIndexBuffer body; @@ -2611,17 +2455,17 @@ static int vmw_cmd_dx_set_index_buffer(struct vmw_private *dev_priv, cmd = container_of(header, typeof(*cmd), header); ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, user_surface_converter, - &cmd->body.sid, &res_node); + &cmd->body.sid, &res); if (unlikely(ret != 0)) return ret; - binding.bi.ctx = ctx_node->res; - binding.bi.res = ((res_node) ? res_node->res : NULL); + binding.bi.ctx = ctx_node->ctx; + binding.bi.res = res; binding.bi.bt = vmw_ctx_binding_ib; binding.offset = cmd->body.offset; binding.format = cmd->body.format; - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, 0, 0); + vmw_binding_add(ctx_node->staged, &binding.bi, 0, 0); return 0; } @@ -2679,8 +2523,8 @@ static int vmw_cmd_dx_clear_rendertarget_view(struct vmw_private *dev_priv, SVGA3dCmdDXClearRenderTargetView body; } *cmd = container_of(header, typeof(*cmd), header); - return vmw_view_id_val_add(sw_context, vmw_view_rt, - cmd->body.renderTargetViewId); + return PTR_RET(vmw_view_id_val_add(sw_context, vmw_view_rt, + cmd->body.renderTargetViewId)); } /** @@ -2700,16 +2544,16 @@ static int vmw_cmd_dx_clear_depthstencil_view(struct vmw_private *dev_priv, SVGA3dCmdDXClearDepthStencilView body; } *cmd = container_of(header, typeof(*cmd), header); - return vmw_view_id_val_add(sw_context, vmw_view_ds, - cmd->body.depthStencilViewId); + return PTR_RET(vmw_view_id_val_add(sw_context, vmw_view_ds, + cmd->body.depthStencilViewId)); } static int vmw_cmd_dx_view_define(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; - struct vmw_resource_val_node *srf_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; + struct vmw_resource *srf; struct vmw_resource *res; enum vmw_view_type view_type; int ret; @@ -2734,19 +2578,18 @@ static int vmw_cmd_dx_view_define(struct vmw_private *dev_priv, cmd = container_of(header, typeof(*cmd), header); ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, user_surface_converter, - &cmd->sid, &srf_node); + &cmd->sid, &srf); if (unlikely(ret != 0)) return ret; - res = vmw_context_cotable(ctx_node->res, vmw_view_cotables[view_type]); + res = vmw_context_cotable(ctx_node->ctx, vmw_view_cotables[view_type]); ret = vmw_cotable_notify(res, cmd->defined_id); - vmw_resource_unreference(&res); if (unlikely(ret != 0)) return ret; return vmw_view_add(sw_context->man, - ctx_node->res, - srf_node->res, + ctx_node->ctx, + srf, view_type, cmd->defined_id, header, @@ -2766,9 +2609,9 @@ static int vmw_cmd_dx_set_so_targets(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_ctx_bindinfo_so binding; - struct vmw_resource_val_node *res_node; + struct vmw_resource *res; struct { SVGA3dCmdHeader header; SVGA3dCmdDXSetSOTargets body; @@ -2793,18 +2636,18 @@ static int vmw_cmd_dx_set_so_targets(struct vmw_private *dev_priv, for (i = 0; i < num; i++) { ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_surface, user_surface_converter, - &cmd->targets[i].sid, &res_node); + &cmd->targets[i].sid, &res); if (unlikely(ret != 0)) return ret; - binding.bi.ctx = ctx_node->res; - binding.bi.res = ((res_node) ? res_node->res : NULL); + binding.bi.ctx = ctx_node->ctx; + binding.bi.res = res; binding.bi.bt = vmw_ctx_binding_so, binding.offset = cmd->targets[i].offset; binding.size = cmd->targets[i].sizeInBytes; binding.slot = i; - vmw_binding_add(ctx_node->staged_bindings, &binding.bi, + vmw_binding_add(ctx_node->staged, &binding.bi, 0, binding.slot); } @@ -2815,7 +2658,7 @@ static int vmw_cmd_dx_so_define(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_resource *res; /* * This is based on the fact that all affected define commands have @@ -2834,10 +2677,9 @@ static int vmw_cmd_dx_so_define(struct vmw_private *dev_priv, } so_type = vmw_so_cmd_to_type(header->id); - res = vmw_context_cotable(ctx_node->res, vmw_so_cotables[so_type]); + res = vmw_context_cotable(ctx_node->ctx, vmw_so_cotables[so_type]); cmd = container_of(header, typeof(*cmd), header); ret = vmw_cotable_notify(res, cmd->defined_id); - vmw_resource_unreference(&res); return ret; } @@ -2882,7 +2724,7 @@ static int vmw_cmd_dx_cid_check(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; if (unlikely(ctx_node == NULL)) { DRM_ERROR("DX Context not set.\n"); @@ -2907,7 +2749,7 @@ static int vmw_cmd_dx_view_remove(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct { SVGA3dCmdHeader header; union vmw_view_destroy body; @@ -2934,7 +2776,7 @@ static int vmw_cmd_dx_view_remove(struct vmw_private *dev_priv, * relocation to conditionally make this command a NOP to avoid * device errors. */ - return vmw_resource_relocation_add(&sw_context->res_relocations, + return vmw_resource_relocation_add(sw_context, view, vmw_ptr_diff(sw_context->buf_start, &cmd->header.id), @@ -2953,7 +2795,7 @@ static int vmw_cmd_dx_define_shader(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct vmw_resource *res; struct { SVGA3dCmdHeader header; @@ -2966,13 +2808,12 @@ static int vmw_cmd_dx_define_shader(struct vmw_private *dev_priv, return -EINVAL; } - res = vmw_context_cotable(ctx_node->res, SVGA_COTABLE_DXSHADER); + res = vmw_context_cotable(ctx_node->ctx, SVGA_COTABLE_DXSHADER); ret = vmw_cotable_notify(res, cmd->body.shaderId); - vmw_resource_unreference(&res); if (ret) return ret; - return vmw_dx_shader_add(sw_context->man, ctx_node->res, + return vmw_dx_shader_add(sw_context->man, ctx_node->ctx, cmd->body.shaderId, cmd->body.type, &sw_context->staged_cmd_res); } @@ -2989,7 +2830,7 @@ static int vmw_cmd_dx_destroy_shader(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node = sw_context->dx_ctx_node; + struct vmw_ctx_validation_info *ctx_node = sw_context->dx_ctx_node; struct { SVGA3dCmdHeader header; SVGA3dCmdDXDestroyShader body; @@ -3021,8 +2862,7 @@ static int vmw_cmd_dx_bind_shader(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, SVGA3dCmdHeader *header) { - struct vmw_resource_val_node *ctx_node; - struct vmw_resource_val_node *res_node; + struct vmw_resource *ctx; struct vmw_resource *res; struct { SVGA3dCmdHeader header; @@ -3033,38 +2873,33 @@ static int vmw_cmd_dx_bind_shader(struct vmw_private *dev_priv, if (cmd->body.cid != SVGA3D_INVALID_ID) { ret = vmw_cmd_res_check(dev_priv, sw_context, vmw_res_context, user_context_converter, - &cmd->body.cid, &ctx_node); + &cmd->body.cid, &ctx); if (ret) return ret; } else { - ctx_node = sw_context->dx_ctx_node; - if (!ctx_node) { + if (!sw_context->dx_ctx_node) { DRM_ERROR("DX Context not set.\n"); return -EINVAL; } + ctx = sw_context->dx_ctx_node->ctx; } - res = vmw_shader_lookup(vmw_context_res_man(ctx_node->res), + res = vmw_shader_lookup(vmw_context_res_man(ctx), cmd->body.shid, 0); if (IS_ERR(res)) { DRM_ERROR("Could not find shader to bind.\n"); return PTR_ERR(res); } - ret = vmw_resource_val_add(sw_context, res, &res_node); + ret = vmw_execbuf_res_noctx_val_add(sw_context, res); if (ret) { DRM_ERROR("Error creating resource validation node.\n"); - goto out_unref; + return ret; } - - ret = vmw_cmd_res_switch_backup(dev_priv, sw_context, res_node, - &cmd->body.mobid, - cmd->body.offsetInBytes); -out_unref: - vmw_resource_unreference(&res); - - return ret; + return vmw_cmd_res_switch_backup(dev_priv, sw_context, res, + &cmd->body.mobid, + cmd->body.offsetInBytes); } /** @@ -3083,8 +2918,8 @@ static int vmw_cmd_dx_genmips(struct vmw_private *dev_priv, SVGA3dCmdDXGenMips body; } *cmd = container_of(header, typeof(*cmd), header); - return vmw_view_id_val_add(sw_context, vmw_view_sr, - cmd->body.shaderResourceViewId); + return PTR_RET(vmw_view_id_val_add(sw_context, vmw_view_sr, + cmd->body.shaderResourceViewId)); } /** @@ -3638,20 +3473,18 @@ static int vmw_cmd_check_all(struct vmw_private *dev_priv, static void vmw_free_relocations(struct vmw_sw_context *sw_context) { - sw_context->cur_reloc = 0; + /* Memory is validation context memory, so no need to free it */ + + INIT_LIST_HEAD(&sw_context->bo_relocations); } static void vmw_apply_relocations(struct vmw_sw_context *sw_context) { - uint32_t i; struct vmw_relocation *reloc; - struct ttm_validate_buffer *validate; struct ttm_buffer_object *bo; - for (i = 0; i < sw_context->cur_reloc; ++i) { - reloc = &sw_context->relocs[i]; - validate = &sw_context->val_bufs[reloc->index].base; - bo = validate->bo; + list_for_each_entry(reloc, &sw_context->bo_relocations, head) { + bo = &reloc->vbo->base; switch (bo->mem.mem_type) { case TTM_PL_VRAM: reloc->location->offset += bo->offset; @@ -3670,110 +3503,6 @@ static void vmw_apply_relocations(struct vmw_sw_context *sw_context) vmw_free_relocations(sw_context); } -/** - * vmw_resource_list_unrefererence - Free up a resource list and unreference - * all resources referenced by it. - * - * @list: The resource list. - */ -static void vmw_resource_list_unreference(struct vmw_sw_context *sw_context, - struct list_head *list) -{ - struct vmw_resource_val_node *val, *val_next; - - /* - * Drop references to resources held during command submission. - */ - - list_for_each_entry_safe(val, val_next, list, head) { - list_del_init(&val->head); - vmw_resource_unreference(&val->res); - - if (val->staged_bindings) { - if (val->staged_bindings != sw_context->staged_bindings) - vmw_binding_state_free(val->staged_bindings); - else - sw_context->staged_bindings_inuse = false; - val->staged_bindings = NULL; - } - - kfree(val); - } -} - -static void vmw_clear_validations(struct vmw_sw_context *sw_context) -{ - struct vmw_validate_buffer *entry, *next; - struct vmw_resource_val_node *val; - - /* - * Drop references to DMA buffers held during command submission. - */ - list_for_each_entry_safe(entry, next, &sw_context->validate_nodes, - base.head) { - list_del(&entry->base.head); - ttm_bo_unref(&entry->base.bo); - (void) drm_ht_remove_item(&sw_context->res_ht, &entry->hash); - sw_context->cur_val_buf--; - } - BUG_ON(sw_context->cur_val_buf != 0); - - list_for_each_entry(val, &sw_context->resource_list, head) - (void) drm_ht_remove_item(&sw_context->res_ht, &val->hash); -} - -int vmw_validate_single_buffer(struct vmw_private *dev_priv, - struct ttm_buffer_object *bo, - bool interruptible, - bool validate_as_mob) -{ - struct vmw_buffer_object *vbo = - container_of(bo, struct vmw_buffer_object, base); - struct ttm_operation_ctx ctx = { interruptible, true }; - int ret; - - if (vbo->pin_count > 0) - return 0; - - if (validate_as_mob) - return ttm_bo_validate(bo, &vmw_mob_placement, &ctx); - - /** - * Put BO in VRAM if there is space, otherwise as a GMR. - * If there is no space in VRAM and GMR ids are all used up, - * start evicting GMRs to make room. If the DMA buffer can't be - * used as a GMR, this will return -ENOMEM. - */ - - ret = ttm_bo_validate(bo, &vmw_vram_gmr_placement, &ctx); - if (likely(ret == 0 || ret == -ERESTARTSYS)) - return ret; - - /** - * If that failed, try VRAM again, this time evicting - * previous contents. - */ - - ret = ttm_bo_validate(bo, &vmw_vram_placement, &ctx); - return ret; -} - -static int vmw_validate_buffers(struct vmw_private *dev_priv, - struct vmw_sw_context *sw_context) -{ - struct vmw_validate_buffer *entry; - int ret; - - list_for_each_entry(entry, &sw_context->validate_nodes, base.head) { - ret = vmw_validate_single_buffer(dev_priv, entry->base.bo, - true, - entry->validate_as_mob); - if (unlikely(ret != 0)) - return ret; - } - return 0; -} - static int vmw_resize_cmd_bounce(struct vmw_sw_context *sw_context, uint32_t size) { @@ -3946,7 +3675,7 @@ static int vmw_execbuf_submit_fifo(struct vmw_private *dev_priv, if (sw_context->dx_ctx_node) cmd = vmw_fifo_reserve_dx(dev_priv, command_size, - sw_context->dx_ctx_node->res->id); + sw_context->dx_ctx_node->ctx->id); else cmd = vmw_fifo_reserve(dev_priv, command_size); if (!cmd) { @@ -3980,7 +3709,7 @@ static int vmw_execbuf_submit_cmdbuf(struct vmw_private *dev_priv, u32 command_size, struct vmw_sw_context *sw_context) { - u32 id = ((sw_context->dx_ctx_node) ? sw_context->dx_ctx_node->res->id : + u32 id = ((sw_context->dx_ctx_node) ? sw_context->dx_ctx_node->ctx->id : SVGA3D_INVALID_ID); void *cmd = vmw_cmdbuf_reserve(dev_priv->cman, command_size, id, false, header); @@ -4057,31 +3786,35 @@ static int vmw_execbuf_tie_context(struct vmw_private *dev_priv, struct vmw_sw_context *sw_context, uint32_t handle) { - struct vmw_resource_val_node *ctx_node; struct vmw_resource *res; int ret; + unsigned int size; if (handle == SVGA3D_INVALID_ID) return 0; - ret = vmw_user_resource_lookup_handle(dev_priv, sw_context->fp->tfile, - handle, user_context_converter, - &res); - if (unlikely(ret != 0)) { + size = vmw_execbuf_res_size(dev_priv, vmw_res_dx_context); + ret = vmw_validation_preload_res(sw_context->ctx, size); + if (ret) + return ret; + + res = vmw_user_resource_noref_lookup_handle + (dev_priv, sw_context->fp->tfile, handle, + user_context_converter); + if (unlikely(IS_ERR(res))) { DRM_ERROR("Could not find or user DX context 0x%08x.\n", (unsigned) handle); - return ret; + return PTR_ERR(res); } - ret = vmw_resource_val_add(sw_context, res, &ctx_node); + ret = vmw_execbuf_res_noref_val_add(sw_context, res); if (unlikely(ret != 0)) - goto out_err; + return ret; - sw_context->dx_ctx_node = ctx_node; + sw_context->dx_ctx_node = vmw_execbuf_info_from_res(sw_context, res); sw_context->man = vmw_context_res_man(res); -out_err: - vmw_resource_unreference(&res); - return ret; + + return 0; } int vmw_execbuf_process(struct drm_file *file_priv, @@ -4097,15 +3830,12 @@ int vmw_execbuf_process(struct drm_file *file_priv, { struct vmw_sw_context *sw_context = &dev_priv->ctx; struct vmw_fence_obj *fence = NULL; - struct vmw_resource *error_resource; - struct list_head resource_list; struct vmw_cmdbuf_header *header; - struct ww_acquire_ctx ticket; uint32_t handle; int ret; int32_t out_fence_fd = -1; struct sync_file *sync_file = NULL; - + DECLARE_VAL_CONTEXT(val_ctx, &sw_context->res_ht, 1); if (flags & DRM_VMW_EXECBUF_FLAG_EXPORT_FENCE_FD) { out_fence_fd = get_unused_fd_flags(O_CLOEXEC); @@ -4157,10 +3887,7 @@ int vmw_execbuf_process(struct drm_file *file_priv, sw_context->kernel = true; sw_context->fp = vmw_fpriv(file_priv); - sw_context->cur_reloc = 0; - sw_context->cur_val_buf = 0; - INIT_LIST_HEAD(&sw_context->resource_list); - INIT_LIST_HEAD(&sw_context->ctx_resource_list); + INIT_LIST_HEAD(&sw_context->ctx_list); sw_context->cur_query_bo = dev_priv->pinned_bo; sw_context->last_query_ctx = NULL; sw_context->needs_post_query_barrier = false; @@ -4168,8 +3895,8 @@ int vmw_execbuf_process(struct drm_file *file_priv, sw_context->dx_query_mob = NULL; sw_context->dx_query_ctx = NULL; memset(sw_context->res_cache, 0, sizeof(sw_context->res_cache)); - INIT_LIST_HEAD(&sw_context->validate_nodes); INIT_LIST_HEAD(&sw_context->res_relocations); + INIT_LIST_HEAD(&sw_context->bo_relocations); if (sw_context->staged_bindings) vmw_binding_state_reset(sw_context->staged_bindings); @@ -4180,24 +3907,13 @@ int vmw_execbuf_process(struct drm_file *file_priv, sw_context->res_ht_initialized = true; } INIT_LIST_HEAD(&sw_context->staged_cmd_res); - INIT_LIST_HEAD(&resource_list); + sw_context->ctx = &val_ctx; ret = vmw_execbuf_tie_context(dev_priv, sw_context, dx_context_handle); - if (unlikely(ret != 0)) { - list_splice_init(&sw_context->ctx_resource_list, - &sw_context->resource_list); + if (unlikely(ret != 0)) goto out_err_nores; - } ret = vmw_cmd_check_all(dev_priv, sw_context, kernel_commands, command_size); - /* - * Merge the resource lists before checking the return status - * from vmd_cmd_check_all so that all the open hashtabs will - * be handled properly even if vmw_cmd_check_all fails. - */ - list_splice_init(&sw_context->ctx_resource_list, - &sw_context->resource_list); - if (unlikely(ret != 0)) goto out_err_nores; @@ -4205,18 +3921,18 @@ int vmw_execbuf_process(struct drm_file *file_priv, if (unlikely(ret != 0)) goto out_err_nores; - ret = ttm_eu_reserve_buffers(&ticket, &sw_context->validate_nodes, - true, NULL); + ret = vmw_validation_bo_reserve(&val_ctx, true); if (unlikely(ret != 0)) goto out_err_nores; - ret = vmw_validate_buffers(dev_priv, sw_context); + ret = vmw_validation_bo_validate(&val_ctx, true); if (unlikely(ret != 0)) goto out_err; - ret = vmw_resources_validate(sw_context); + ret = vmw_validation_res_validate(&val_ctx, true); if (unlikely(ret != 0)) goto out_err; + vmw_validation_drop_ht(&val_ctx); ret = mutex_lock_interruptible(&dev_priv->binding_mutex); if (unlikely(ret != 0)) { @@ -4255,17 +3971,16 @@ int vmw_execbuf_process(struct drm_file *file_priv, if (ret != 0) DRM_ERROR("Fence submission error. Syncing.\n"); - vmw_resources_unreserve(sw_context, false); + vmw_execbuf_bindings_commit(sw_context, false); + vmw_bind_dx_query_mob(sw_context); + vmw_validation_res_unreserve(&val_ctx, false); - ttm_eu_fence_buffer_objects(&ticket, &sw_context->validate_nodes, - (void *) fence); + vmw_validation_bo_fence(sw_context->ctx, fence); if (unlikely(dev_priv->pinned_bo != NULL && !dev_priv->query_cid_valid)) __vmw_execbuf_release_pinned_bo(dev_priv, fence); - vmw_clear_validations(sw_context); - /* * If anything fails here, give up trying to export the fence * and do a sync since the user mode will not be able to sync @@ -4300,7 +4015,6 @@ int vmw_execbuf_process(struct drm_file *file_priv, vmw_fence_obj_unreference(&fence); } - list_splice_init(&sw_context->resource_list, &resource_list); vmw_cmdbuf_res_commit(&sw_context->staged_cmd_res); mutex_unlock(&dev_priv->cmdbuf_mutex); @@ -4308,36 +4022,33 @@ int vmw_execbuf_process(struct drm_file *file_priv, * Unreference resources outside of the cmdbuf_mutex to * avoid deadlocks in resource destruction paths. */ - vmw_resource_list_unreference(sw_context, &resource_list); + vmw_validation_unref_lists(&val_ctx); return 0; out_unlock_binding: mutex_unlock(&dev_priv->binding_mutex); out_err: - ttm_eu_backoff_reservation(&ticket, &sw_context->validate_nodes); + vmw_validation_bo_backoff(&val_ctx); out_err_nores: - vmw_resources_unreserve(sw_context, true); + vmw_execbuf_bindings_commit(sw_context, true); + vmw_validation_res_unreserve(&val_ctx, true); vmw_resource_relocations_free(&sw_context->res_relocations); vmw_free_relocations(sw_context); - vmw_clear_validations(sw_context); if (unlikely(dev_priv->pinned_bo != NULL && !dev_priv->query_cid_valid)) __vmw_execbuf_release_pinned_bo(dev_priv, NULL); out_unlock: - list_splice_init(&sw_context->resource_list, &resource_list); - error_resource = sw_context->error_resource; - sw_context->error_resource = NULL; vmw_cmdbuf_res_revert(&sw_context->staged_cmd_res); + vmw_validation_drop_ht(&val_ctx); + WARN_ON(!list_empty(&sw_context->ctx_list)); mutex_unlock(&dev_priv->cmdbuf_mutex); /* * Unreference resources outside of the cmdbuf_mutex to * avoid deadlocks in resource destruction paths. */ - vmw_resource_list_unreference(sw_context, &resource_list); - if (unlikely(error_resource != NULL)) - vmw_resource_unreference(&error_resource); + vmw_validation_unref_lists(&val_ctx); out_free_header: if (header) vmw_cmdbuf_header_free(header); @@ -4398,38 +4109,31 @@ void __vmw_execbuf_release_pinned_bo(struct vmw_private *dev_priv, struct vmw_fence_obj *fence) { int ret = 0; - struct list_head validate_list; - struct ttm_validate_buffer pinned_val, query_val; struct vmw_fence_obj *lfence = NULL; - struct ww_acquire_ctx ticket; + DECLARE_VAL_CONTEXT(val_ctx, NULL, 0); if (dev_priv->pinned_bo == NULL) goto out_unlock; - INIT_LIST_HEAD(&validate_list); - - pinned_val.bo = ttm_bo_reference(&dev_priv->pinned_bo->base); - pinned_val.shared = false; - list_add_tail(&pinned_val.head, &validate_list); + ret = vmw_validation_add_bo(&val_ctx, dev_priv->pinned_bo, false, + false); + if (ret) + goto out_no_reserve; - query_val.bo = ttm_bo_reference(&dev_priv->dummy_query_bo->base); - query_val.shared = false; - list_add_tail(&query_val.head, &validate_list); + ret = vmw_validation_add_bo(&val_ctx, dev_priv->dummy_query_bo, false, + false); + if (ret) + goto out_no_reserve; - ret = ttm_eu_reserve_buffers(&ticket, &validate_list, - false, NULL); - if (unlikely(ret != 0)) { - vmw_execbuf_unpin_panic(dev_priv); + ret = vmw_validation_bo_reserve(&val_ctx, false); + if (ret) goto out_no_reserve; - } if (dev_priv->query_cid_valid) { BUG_ON(fence != NULL); ret = vmw_fifo_emit_dummy_query(dev_priv, dev_priv->query_cid); - if (unlikely(ret != 0)) { - vmw_execbuf_unpin_panic(dev_priv); + if (ret) goto out_no_emit; - } dev_priv->query_cid_valid = false; } @@ -4443,22 +4147,22 @@ void __vmw_execbuf_release_pinned_bo(struct vmw_private *dev_priv, NULL); fence = lfence; } - ttm_eu_fence_buffer_objects(&ticket, &validate_list, (void *) fence); + vmw_validation_bo_fence(&val_ctx, fence); if (lfence != NULL) vmw_fence_obj_unreference(&lfence); - ttm_bo_unref(&query_val.bo); - ttm_bo_unref(&pinned_val.bo); + vmw_validation_unref_lists(&val_ctx); vmw_bo_unreference(&dev_priv->pinned_bo); out_unlock: return; out_no_emit: - ttm_eu_backoff_reservation(&ticket, &validate_list); + vmw_validation_bo_backoff(&val_ctx); out_no_reserve: - ttm_bo_unref(&query_val.bo); - ttm_bo_unref(&pinned_val.bo); + vmw_validation_unref_lists(&val_ctx); + vmw_execbuf_unpin_panic(dev_priv); vmw_bo_unreference(&dev_priv->pinned_bo); + } /** diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c index 3d546d409334..f87261545f2c 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_fence.c @@ -306,7 +306,8 @@ struct vmw_fence_manager *vmw_fence_manager_init(struct vmw_private *dev_priv) INIT_LIST_HEAD(&fman->cleanup_list); INIT_WORK(&fman->work, &vmw_fence_work_func); fman->fifo_down = true; - fman->user_fence_size = ttm_round_pot(sizeof(struct vmw_user_fence)); + fman->user_fence_size = ttm_round_pot(sizeof(struct vmw_user_fence)) + + TTM_OBJ_EXTRA_SIZE; fman->fence_size = ttm_round_pot(sizeof(struct vmw_fence_obj)); fman->event_fence_action_size = ttm_round_pot(sizeof(struct vmw_event_fence_action)); @@ -650,7 +651,7 @@ int vmw_user_fence_create(struct drm_file *file_priv, } *p_fence = &ufence->fence; - *p_handle = ufence->base.hash.key; + *p_handle = ufence->base.handle; return 0; out_err: @@ -1137,7 +1138,7 @@ int vmw_fence_event_ioctl(struct drm_device *dev, void *data, "object.\n"); goto out_no_ref_obj; } - handle = base->hash.key; + handle = base->handle; } ttm_base_object_unref(&base); } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c index 0c25bb8faf80..dca04d4246ea 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.c @@ -1510,21 +1510,19 @@ static int vmw_kms_check_display_memory(struct drm_device *dev, struct drm_rect *rects) { struct vmw_private *dev_priv = vmw_priv(dev); - struct drm_mode_config *mode_config = &dev->mode_config; struct drm_rect bounding_box = {0}; u64 total_pixels = 0, pixel_mem, bb_mem; int i; for (i = 0; i < num_rects; i++) { /* - * Currently this check is limiting the topology within max - * texture/screentarget size. This should change in future when - * user-space support multiple fb with topology. + * For STDU only individual screen (screen target) is limited by + * SCREENTARGET_MAX_WIDTH/HEIGHT registers. */ - if (rects[i].x1 < 0 || rects[i].y1 < 0 || - rects[i].x2 > mode_config->max_width || - rects[i].y2 > mode_config->max_height) { - DRM_ERROR("Invalid GUI layout.\n"); + if (dev_priv->active_display_unit == vmw_du_screen_target && + (drm_rect_width(&rects[i]) > dev_priv->stdu_max_width || + drm_rect_height(&rects[i]) > dev_priv->stdu_max_height)) { + DRM_ERROR("Screen size not supported.\n"); return -EINVAL; } @@ -1613,7 +1611,7 @@ static int vmw_kms_check_topology(struct drm_device *dev, struct drm_connector_state *conn_state; struct vmw_connector_state *vmw_conn_state; - if (!new_crtc_state->enable && old_crtc_state->enable) { + if (!new_crtc_state->enable) { rects[i].x1 = 0; rects[i].y1 = 0; rects[i].x2 = 0; @@ -2214,12 +2212,16 @@ int vmw_du_connector_fill_modes(struct drm_connector *connector, if (dev_priv->assume_16bpp) assumed_bpp = 2; + max_width = min(max_width, dev_priv->texture_max_width); + max_height = min(max_height, dev_priv->texture_max_height); + + /* + * For STDU extra limit for a mode on SVGA_REG_SCREENTARGET_MAX_WIDTH/ + * HEIGHT registers. + */ if (dev_priv->active_display_unit == vmw_du_screen_target) { max_width = min(max_width, dev_priv->stdu_max_width); - max_width = min(max_width, dev_priv->texture_max_width); - max_height = min(max_height, dev_priv->stdu_max_height); - max_height = min(max_height, dev_priv->texture_max_height); } /* Add preferred mode */ @@ -2374,6 +2376,7 @@ int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv) { struct vmw_private *dev_priv = vmw_priv(dev); + struct drm_mode_config *mode_config = &dev->mode_config; struct drm_vmw_update_layout_arg *arg = (struct drm_vmw_update_layout_arg *)data; void __user *user_rects; @@ -2419,6 +2422,21 @@ int vmw_kms_update_layout_ioctl(struct drm_device *dev, void *data, drm_rects[i].y1 = curr_rect.y; drm_rects[i].x2 = curr_rect.x + curr_rect.w; drm_rects[i].y2 = curr_rect.y + curr_rect.h; + + /* + * Currently this check is limiting the topology within + * mode_config->max (which actually is max texture size + * supported by virtual device). This limit is here to address + * window managers that create a big framebuffer for whole + * topology. + */ + if (drm_rects[i].x1 < 0 || drm_rects[i].y1 < 0 || + drm_rects[i].x2 > mode_config->max_width || + drm_rects[i].y2 > mode_config->max_height) { + DRM_ERROR("Invalid GUI layout.\n"); + ret = -EINVAL; + goto out_free; + } } ret = vmw_kms_check_display_memory(dev, arg->num_outputs, drm_rects); @@ -2557,88 +2575,31 @@ int vmw_kms_helper_dirty(struct vmw_private *dev_priv, } /** - * vmw_kms_helper_buffer_prepare - Reserve and validate a buffer object before - * command submission. - * - * @dev_priv. Pointer to a device private structure. - * @buf: The buffer object - * @interruptible: Whether to perform waits as interruptible. - * @validate_as_mob: Whether the buffer should be validated as a MOB. If false, - * The buffer will be validated as a GMR. Already pinned buffers will not be - * validated. - * - * Returns 0 on success, negative error code on failure, -ERESTARTSYS if - * interrupted by a signal. + * vmw_kms_helper_validation_finish - Helper for post KMS command submission + * cleanup and fencing + * @dev_priv: Pointer to the device-private struct + * @file_priv: Pointer identifying the client when user-space fencing is used + * @ctx: Pointer to the validation context + * @out_fence: If non-NULL, returned refcounted fence-pointer + * @user_fence_rep: If non-NULL, pointer to user-space address area + * in which to copy user-space fence info */ -int vmw_kms_helper_buffer_prepare(struct vmw_private *dev_priv, - struct vmw_buffer_object *buf, - bool interruptible, - bool validate_as_mob, - bool for_cpu_blit) -{ - struct ttm_operation_ctx ctx = { - .interruptible = interruptible, - .no_wait_gpu = false}; - struct ttm_buffer_object *bo = &buf->base; - int ret; - - ttm_bo_reserve(bo, false, false, NULL); - if (for_cpu_blit) - ret = ttm_bo_validate(bo, &vmw_nonfixed_placement, &ctx); - else - ret = vmw_validate_single_buffer(dev_priv, bo, interruptible, - validate_as_mob); - if (ret) - ttm_bo_unreserve(bo); - - return ret; -} - -/** - * vmw_kms_helper_buffer_revert - Undo the actions of - * vmw_kms_helper_buffer_prepare. - * - * @res: Pointer to the buffer object. - * - * Helper to be used if an error forces the caller to undo the actions of - * vmw_kms_helper_buffer_prepare. - */ -void vmw_kms_helper_buffer_revert(struct vmw_buffer_object *buf) -{ - if (buf) - ttm_bo_unreserve(&buf->base); -} - -/** - * vmw_kms_helper_buffer_finish - Unreserve and fence a buffer object after - * kms command submission. - * - * @dev_priv: Pointer to a device private structure. - * @file_priv: Pointer to a struct drm_file representing the caller's - * connection. Must be set to NULL if @user_fence_rep is NULL, and conversely - * if non-NULL, @user_fence_rep must be non-NULL. - * @buf: The buffer object. - * @out_fence: Optional pointer to a fence pointer. If non-NULL, a - * ref-counted fence pointer is returned here. - * @user_fence_rep: Optional pointer to a user-space provided struct - * drm_vmw_fence_rep. If provided, @file_priv must also be provided and the - * function copies fence data to user-space in a fail-safe manner. - */ -void vmw_kms_helper_buffer_finish(struct vmw_private *dev_priv, - struct drm_file *file_priv, - struct vmw_buffer_object *buf, - struct vmw_fence_obj **out_fence, - struct drm_vmw_fence_rep __user * - user_fence_rep) -{ - struct vmw_fence_obj *fence; +void vmw_kms_helper_validation_finish(struct vmw_private *dev_priv, + struct drm_file *file_priv, + struct vmw_validation_context *ctx, + struct vmw_fence_obj **out_fence, + struct drm_vmw_fence_rep __user * + user_fence_rep) +{ + struct vmw_fence_obj *fence = NULL; uint32_t handle; int ret; - ret = vmw_execbuf_fence_commands(file_priv, dev_priv, &fence, - file_priv ? &handle : NULL); - if (buf) - vmw_bo_fence_single(&buf->base, fence); + if (file_priv || user_fence_rep || vmw_validation_has_bos(ctx) || + out_fence) + ret = vmw_execbuf_fence_commands(file_priv, dev_priv, &fence, + file_priv ? &handle : NULL); + vmw_validation_done(ctx, fence); if (file_priv) vmw_execbuf_copy_fence_user(dev_priv, vmw_fpriv(file_priv), ret, user_fence_rep, fence, @@ -2647,106 +2608,6 @@ void vmw_kms_helper_buffer_finish(struct vmw_private *dev_priv, *out_fence = fence; else vmw_fence_obj_unreference(&fence); - - vmw_kms_helper_buffer_revert(buf); -} - - -/** - * vmw_kms_helper_resource_revert - Undo the actions of - * vmw_kms_helper_resource_prepare. - * - * @res: Pointer to the resource. Typically a surface. - * - * Helper to be used if an error forces the caller to undo the actions of - * vmw_kms_helper_resource_prepare. - */ -void vmw_kms_helper_resource_revert(struct vmw_validation_ctx *ctx) -{ - struct vmw_resource *res = ctx->res; - - vmw_kms_helper_buffer_revert(ctx->buf); - vmw_bo_unreference(&ctx->buf); - vmw_resource_unreserve(res, false, NULL, 0); - mutex_unlock(&res->dev_priv->cmdbuf_mutex); -} - -/** - * vmw_kms_helper_resource_prepare - Reserve and validate a resource before - * command submission. - * - * @res: Pointer to the resource. Typically a surface. - * @interruptible: Whether to perform waits as interruptible. - * - * Reserves and validates also the backup buffer if a guest-backed resource. - * Returns 0 on success, negative error code on failure. -ERESTARTSYS if - * interrupted by a signal. - */ -int vmw_kms_helper_resource_prepare(struct vmw_resource *res, - bool interruptible, - struct vmw_validation_ctx *ctx) -{ - int ret = 0; - - ctx->buf = NULL; - ctx->res = res; - - if (interruptible) - ret = mutex_lock_interruptible(&res->dev_priv->cmdbuf_mutex); - else - mutex_lock(&res->dev_priv->cmdbuf_mutex); - - if (unlikely(ret != 0)) - return -ERESTARTSYS; - - ret = vmw_resource_reserve(res, interruptible, false); - if (ret) - goto out_unlock; - - if (res->backup) { - ret = vmw_kms_helper_buffer_prepare(res->dev_priv, res->backup, - interruptible, - res->dev_priv->has_mob, - false); - if (ret) - goto out_unreserve; - - ctx->buf = vmw_bo_reference(res->backup); - } - ret = vmw_resource_validate(res); - if (ret) - goto out_revert; - return 0; - -out_revert: - vmw_kms_helper_buffer_revert(ctx->buf); -out_unreserve: - vmw_resource_unreserve(res, false, NULL, 0); -out_unlock: - mutex_unlock(&res->dev_priv->cmdbuf_mutex); - return ret; -} - -/** - * vmw_kms_helper_resource_finish - Unreserve and fence a resource after - * kms command submission. - * - * @res: Pointer to the resource. Typically a surface. - * @out_fence: Optional pointer to a fence pointer. If non-NULL, a - * ref-counted fence pointer is returned here. - */ -void vmw_kms_helper_resource_finish(struct vmw_validation_ctx *ctx, - struct vmw_fence_obj **out_fence) -{ - struct vmw_resource *res = ctx->res; - - if (ctx->buf || out_fence) - vmw_kms_helper_buffer_finish(res->dev_priv, NULL, ctx->buf, - out_fence, NULL); - - vmw_bo_unreference(&ctx->buf); - vmw_resource_unreserve(res, false, NULL, 0); - mutex_unlock(&res->dev_priv->cmdbuf_mutex); } /** diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h index 31311298ec0b..76ec570c0684 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_kms.h @@ -308,24 +308,12 @@ int vmw_kms_helper_dirty(struct vmw_private *dev_priv, int increment, struct vmw_kms_dirty *dirty); -int vmw_kms_helper_buffer_prepare(struct vmw_private *dev_priv, - struct vmw_buffer_object *buf, - bool interruptible, - bool validate_as_mob, - bool for_cpu_blit); -void vmw_kms_helper_buffer_revert(struct vmw_buffer_object *buf); -void vmw_kms_helper_buffer_finish(struct vmw_private *dev_priv, - struct drm_file *file_priv, - struct vmw_buffer_object *buf, - struct vmw_fence_obj **out_fence, - struct drm_vmw_fence_rep __user * - user_fence_rep); -int vmw_kms_helper_resource_prepare(struct vmw_resource *res, - bool interruptible, - struct vmw_validation_ctx *ctx); -void vmw_kms_helper_resource_revert(struct vmw_validation_ctx *ctx); -void vmw_kms_helper_resource_finish(struct vmw_validation_ctx *ctx, - struct vmw_fence_obj **out_fence); +void vmw_kms_helper_validation_finish(struct vmw_private *dev_priv, + struct drm_file *file_priv, + struct vmw_validation_context *ctx, + struct vmw_fence_obj **out_fence, + struct drm_vmw_fence_rep __user * + user_fence_rep); int vmw_kms_readback(struct vmw_private *dev_priv, struct drm_file *file_priv, struct vmw_framebuffer *vfb, diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c b/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c index 0861c821a7fe..e420675e8db3 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_prime.c @@ -31,8 +31,8 @@ */ #include "vmwgfx_drv.h" +#include "ttm_object.h" #include <linux/dma-buf.h> -#include <drm/ttm/ttm_object.h> /* * DMA-BUF attach- and mapping methods. No need to implement diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c index 92003ea5a219..8a029bade32a 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource.c @@ -58,11 +58,11 @@ void vmw_resource_release_id(struct vmw_resource *res) struct vmw_private *dev_priv = res->dev_priv; struct idr *idr = &dev_priv->res_idr[res->func->res_type]; - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); if (res->id != -1) idr_remove(idr, res->id); res->id = -1; - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); } static void vmw_resource_release(struct kref *kref) @@ -73,10 +73,9 @@ static void vmw_resource_release(struct kref *kref) int id; struct idr *idr = &dev_priv->res_idr[res->func->res_type]; - write_lock(&dev_priv->resource_lock); - res->avail = false; + spin_lock(&dev_priv->resource_lock); list_del_init(&res->lru_head); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); if (res->backup) { struct ttm_buffer_object *bo = &res->backup->base; @@ -108,10 +107,10 @@ static void vmw_resource_release(struct kref *kref) else kfree(res); - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); if (id != -1) idr_remove(idr, id); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); } void vmw_resource_unreference(struct vmw_resource **p_res) @@ -140,13 +139,13 @@ int vmw_resource_alloc_id(struct vmw_resource *res) BUG_ON(res->id != -1); idr_preload(GFP_KERNEL); - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); ret = idr_alloc(idr, res, 1, 0, GFP_NOWAIT); if (ret >= 0) res->id = ret; - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); idr_preload_end(); return ret < 0 ? ret : 0; } @@ -170,7 +169,6 @@ int vmw_resource_init(struct vmw_private *dev_priv, struct vmw_resource *res, kref_init(&res->kref); res->hw_destroy = NULL; res->res_free = res_free; - res->avail = false; res->dev_priv = dev_priv; res->func = func; INIT_LIST_HEAD(&res->lru_head); @@ -187,28 +185,6 @@ int vmw_resource_init(struct vmw_private *dev_priv, struct vmw_resource *res, return vmw_resource_alloc_id(res); } -/** - * vmw_resource_activate - * - * @res: Pointer to the newly created resource - * @hw_destroy: Destroy function. NULL if none. - * - * Activate a resource after the hardware has been made aware of it. - * Set tye destroy function to @destroy. Typically this frees the - * resource and destroys the hardware resources associated with it. - * Activate basically means that the function vmw_resource_lookup will - * find it. - */ -void vmw_resource_activate(struct vmw_resource *res, - void (*hw_destroy) (struct vmw_resource *)) -{ - struct vmw_private *dev_priv = res->dev_priv; - - write_lock(&dev_priv->resource_lock); - res->avail = true; - res->hw_destroy = hw_destroy; - write_unlock(&dev_priv->resource_lock); -} /** * vmw_user_resource_lookup_handle - lookup a struct resource from a @@ -243,15 +219,7 @@ int vmw_user_resource_lookup_handle(struct vmw_private *dev_priv, goto out_bad_resource; res = converter->base_obj_to_res(base); - - read_lock(&dev_priv->resource_lock); - if (!res->avail || res->res_free != converter->res_free) { - read_unlock(&dev_priv->resource_lock); - goto out_bad_resource; - } - kref_get(&res->kref); - read_unlock(&dev_priv->resource_lock); *p_res = res; ret = 0; @@ -263,6 +231,41 @@ out_bad_resource: } /** + * vmw_user_resource_lookup_handle - lookup a struct resource from a + * TTM user-space handle and perform basic type checks + * + * @dev_priv: Pointer to a device private struct + * @tfile: Pointer to a struct ttm_object_file identifying the caller + * @handle: The TTM user-space handle + * @converter: Pointer to an object describing the resource type + * @p_res: On successful return the location pointed to will contain + * a pointer to a refcounted struct vmw_resource. + * + * If the handle can't be found or is associated with an incorrect resource + * type, -EINVAL will be returned. + */ +struct vmw_resource * +vmw_user_resource_noref_lookup_handle(struct vmw_private *dev_priv, + struct ttm_object_file *tfile, + uint32_t handle, + const struct vmw_user_resource_conv + *converter) +{ + struct ttm_base_object *base; + + base = ttm_base_object_noref_lookup(tfile, handle); + if (!base) + return ERR_PTR(-ESRCH); + + if (unlikely(ttm_base_object_type(base) != converter->object_type)) { + ttm_base_object_noref_release(); + return ERR_PTR(-EINVAL); + } + + return converter->base_obj_to_res(base); +} + +/** * Helper function that looks either a surface or bo. * * The pointer this pointed at by out_surf and out_buf needs to be null. @@ -422,10 +425,10 @@ void vmw_resource_unreserve(struct vmw_resource *res, if (!res->func->may_evict || res->id == -1 || res->pin_count) return; - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); list_add_tail(&res->lru_head, &res->dev_priv->res_lru[res->func->res_type]); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); } /** @@ -504,9 +507,9 @@ int vmw_resource_reserve(struct vmw_resource *res, bool interruptible, struct vmw_private *dev_priv = res->dev_priv; int ret; - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); list_del_init(&res->lru_head); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); if (res->func->needs_backup && res->backup == NULL && !no_backup) { @@ -587,15 +590,18 @@ out_no_unbind: /** * vmw_resource_validate - Make a resource up-to-date and visible * to the device. - * - * @res: The resource to make visible to the device. + * @res: The resource to make visible to the device. + * @intr: Perform waits interruptible if possible. * * On succesful return, any backup DMA buffer pointed to by @res->backup will * be reserved and validated. * On hardware resource shortage, this function will repeatedly evict * resources of the same type until the validation succeeds. + * + * Return: Zero on success, -ERESTARTSYS if interrupted, negative error code + * on failure. */ -int vmw_resource_validate(struct vmw_resource *res) +int vmw_resource_validate(struct vmw_resource *res, bool intr) { int ret; struct vmw_resource *evict_res; @@ -616,12 +622,12 @@ int vmw_resource_validate(struct vmw_resource *res) if (likely(ret != -EBUSY)) break; - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); if (list_empty(lru_list) || !res->func->may_evict) { DRM_ERROR("Out of device device resources " "for %s.\n", res->func->type_name); ret = -EBUSY; - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); break; } @@ -630,14 +636,14 @@ int vmw_resource_validate(struct vmw_resource *res) lru_head)); list_del_init(&evict_res->lru_head); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); /* Trylock backup buffers with a NULL ticket. */ - ret = vmw_resource_do_evict(NULL, evict_res, true); + ret = vmw_resource_do_evict(NULL, evict_res, intr); if (unlikely(ret != 0)) { - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); list_add_tail(&evict_res->lru_head, lru_list); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); if (ret == -ERESTARTSYS || ++err_count > VMW_RES_EVICT_ERR_COUNT) { vmw_resource_unreference(&evict_res); @@ -819,7 +825,7 @@ static void vmw_resource_evict_type(struct vmw_private *dev_priv, struct ww_acquire_ctx ticket; do { - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); if (list_empty(lru_list)) goto out_unlock; @@ -828,14 +834,14 @@ static void vmw_resource_evict_type(struct vmw_private *dev_priv, list_first_entry(lru_list, struct vmw_resource, lru_head)); list_del_init(&evict_res->lru_head); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); /* Wait lock backup buffers with a ticket. */ ret = vmw_resource_do_evict(&ticket, evict_res, false); if (unlikely(ret != 0)) { - write_lock(&dev_priv->resource_lock); + spin_lock(&dev_priv->resource_lock); list_add_tail(&evict_res->lru_head, lru_list); - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); if (++err_count > VMW_RES_EVICT_ERR_COUNT) { vmw_resource_unreference(&evict_res); return; @@ -846,7 +852,7 @@ static void vmw_resource_evict_type(struct vmw_private *dev_priv, } while (1); out_unlock: - write_unlock(&dev_priv->resource_lock); + spin_unlock(&dev_priv->resource_lock); } /** @@ -914,7 +920,7 @@ int vmw_resource_pin(struct vmw_resource *res, bool interruptible) /* Do we really need to pin the MOB as well? */ vmw_bo_pin_reserved(vbo, true); } - ret = vmw_resource_validate(res); + ret = vmw_resource_validate(res, interruptible); if (vbo) ttm_bo_unreserve(&vbo->base); if (ret) diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_resource_priv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_resource_priv.h index a8c1c5ebd71d..7e19eba0b0b8 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_resource_priv.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_resource_priv.h @@ -30,6 +30,11 @@ #include "vmwgfx_drv.h" +/* + * Extra memory required by the resource id's ida storage, which is allocated + * separately from the base object itself. We estimate an on-average 128 bytes + * per ida. + */ #define VMW_IDA_ACC_SIZE 128 enum vmw_cmdbuf_res_state { @@ -120,8 +125,6 @@ int vmw_resource_init(struct vmw_private *dev_priv, struct vmw_resource *res, bool delay_id, void (*res_free) (struct vmw_resource *res), const struct vmw_res_func *func); -void vmw_resource_activate(struct vmw_resource *res, - void (*hw_destroy) (struct vmw_resource *)); int vmw_simple_resource_create_ioctl(struct drm_device *dev, void *data, diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c index ad0de7f0cd60..53316b1bda3d 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c @@ -946,16 +946,20 @@ int vmw_kms_sou_do_surface_dirty(struct vmw_private *dev_priv, struct vmw_framebuffer_surface *vfbs = container_of(framebuffer, typeof(*vfbs), base); struct vmw_kms_sou_surface_dirty sdirty; - struct vmw_validation_ctx ctx; + DECLARE_VAL_CONTEXT(val_ctx, NULL, 0); int ret; if (!srf) srf = &vfbs->surface->res; - ret = vmw_kms_helper_resource_prepare(srf, true, &ctx); + ret = vmw_validation_add_resource(&val_ctx, srf, 0, NULL, NULL); if (ret) return ret; + ret = vmw_validation_prepare(&val_ctx, &dev_priv->cmdbuf_mutex, true); + if (ret) + goto out_unref; + sdirty.base.fifo_commit = vmw_sou_surface_fifo_commit; sdirty.base.clip = vmw_sou_surface_clip; sdirty.base.dev_priv = dev_priv; @@ -972,9 +976,14 @@ int vmw_kms_sou_do_surface_dirty(struct vmw_private *dev_priv, ret = vmw_kms_helper_dirty(dev_priv, framebuffer, clips, vclips, dest_x, dest_y, num_clips, inc, &sdirty.base); - vmw_kms_helper_resource_finish(&ctx, out_fence); + vmw_kms_helper_validation_finish(dev_priv, NULL, &val_ctx, out_fence, + NULL); return ret; + +out_unref: + vmw_validation_unref_lists(&val_ctx); + return ret; } /** @@ -1051,13 +1060,17 @@ int vmw_kms_sou_do_bo_dirty(struct vmw_private *dev_priv, container_of(framebuffer, struct vmw_framebuffer_bo, base)->buffer; struct vmw_kms_dirty dirty; + DECLARE_VAL_CONTEXT(val_ctx, NULL, 0); int ret; - ret = vmw_kms_helper_buffer_prepare(dev_priv, buf, interruptible, - false, false); + ret = vmw_validation_add_bo(&val_ctx, buf, false, false); if (ret) return ret; + ret = vmw_validation_prepare(&val_ctx, NULL, interruptible); + if (ret) + goto out_unref; + ret = do_bo_define_gmrfb(dev_priv, framebuffer); if (unlikely(ret != 0)) goto out_revert; @@ -1069,12 +1082,15 @@ int vmw_kms_sou_do_bo_dirty(struct vmw_private *dev_priv, num_clips; ret = vmw_kms_helper_dirty(dev_priv, framebuffer, clips, vclips, 0, 0, num_clips, increment, &dirty); - vmw_kms_helper_buffer_finish(dev_priv, NULL, buf, out_fence, NULL); + vmw_kms_helper_validation_finish(dev_priv, NULL, &val_ctx, out_fence, + NULL); return ret; out_revert: - vmw_kms_helper_buffer_revert(buf); + vmw_validation_revert(&val_ctx); +out_unref: + vmw_validation_unref_lists(&val_ctx); return ret; } @@ -1150,13 +1166,17 @@ int vmw_kms_sou_readback(struct vmw_private *dev_priv, struct vmw_buffer_object *buf = container_of(vfb, struct vmw_framebuffer_bo, base)->buffer; struct vmw_kms_dirty dirty; + DECLARE_VAL_CONTEXT(val_ctx, NULL, 0); int ret; - ret = vmw_kms_helper_buffer_prepare(dev_priv, buf, true, false, - false); + ret = vmw_validation_add_bo(&val_ctx, buf, false, false); if (ret) return ret; + ret = vmw_validation_prepare(&val_ctx, NULL, true); + if (ret) + goto out_unref; + ret = do_bo_define_gmrfb(dev_priv, vfb); if (unlikely(ret != 0)) goto out_revert; @@ -1168,13 +1188,15 @@ int vmw_kms_sou_readback(struct vmw_private *dev_priv, num_clips; ret = vmw_kms_helper_dirty(dev_priv, vfb, NULL, vclips, 0, 0, num_clips, 1, &dirty); - vmw_kms_helper_buffer_finish(dev_priv, file_priv, buf, NULL, - user_fence_rep); + vmw_kms_helper_validation_finish(dev_priv, file_priv, &val_ctx, NULL, + user_fence_rep); return ret; out_revert: - vmw_kms_helper_buffer_revert(buf); - + vmw_validation_revert(&val_ctx); +out_unref: + vmw_validation_unref_lists(&val_ctx); + return ret; } diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c index fe4842ca3b6e..bf32fe446219 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_shader.c @@ -186,7 +186,7 @@ static int vmw_gb_shader_init(struct vmw_private *dev_priv, shader->num_input_sig = num_input_sig; shader->num_output_sig = num_output_sig; - vmw_resource_activate(res, vmw_hw_shader_destroy); + res->hw_destroy = vmw_hw_shader_destroy; return 0; } @@ -562,7 +562,7 @@ void vmw_dx_shader_cotable_list_scrub(struct vmw_private *dev_priv, { struct vmw_dx_shader *entry, *next; - WARN_ON_ONCE(!mutex_is_locked(&dev_priv->binding_mutex)); + lockdep_assert_held_once(&dev_priv->binding_mutex); list_for_each_entry_safe(entry, next, list, cotable_head) { WARN_ON(vmw_dx_shader_scrub(&entry->res)); @@ -636,7 +636,8 @@ int vmw_dx_shader_add(struct vmw_cmdbuf_res_manager *man, res = &shader->res; shader->ctx = ctx; - shader->cotable = vmw_context_cotable(ctx, SVGA_COTABLE_DXSHADER); + shader->cotable = vmw_resource_reference + (vmw_context_cotable(ctx, SVGA_COTABLE_DXSHADER)); shader->id = user_key; shader->committed = false; INIT_LIST_HEAD(&shader->cotable_head); @@ -656,7 +657,7 @@ int vmw_dx_shader_add(struct vmw_cmdbuf_res_manager *man, goto out_resource_init; res->id = shader->id; - vmw_resource_activate(res, vmw_hw_shader_destroy); + res->hw_destroy = vmw_hw_shader_destroy; out_resource_init: vmw_resource_unreference(&res); @@ -740,13 +741,10 @@ static int vmw_user_shader_alloc(struct vmw_private *dev_priv, }; int ret; - /* - * Approximate idr memory usage with 128 bytes. It will be limited - * by maximum number_of shaders anyway. - */ if (unlikely(vmw_user_shader_size == 0)) vmw_user_shader_size = - ttm_round_pot(sizeof(struct vmw_user_shader)) + 128; + ttm_round_pot(sizeof(struct vmw_user_shader)) + + VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE; ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv), vmw_user_shader_size, @@ -792,7 +790,7 @@ static int vmw_user_shader_alloc(struct vmw_private *dev_priv, } if (handle) - *handle = ushader->base.hash.key; + *handle = ushader->base.handle; out_err: vmw_resource_unreference(&res); out: @@ -814,13 +812,10 @@ static struct vmw_resource *vmw_shader_alloc(struct vmw_private *dev_priv, }; int ret; - /* - * Approximate idr memory usage with 128 bytes. It will be limited - * by maximum number_of shaders anyway. - */ if (unlikely(vmw_shader_size == 0)) vmw_shader_size = - ttm_round_pot(sizeof(struct vmw_shader)) + 128; + ttm_round_pot(sizeof(struct vmw_shader)) + + VMW_IDA_ACC_SIZE; ret = ttm_mem_global_alloc(vmw_mem_glob(dev_priv), vmw_shader_size, diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c b/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c index 6ebc5affde14..6a6865384e91 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_simple_resource.c @@ -81,7 +81,7 @@ static int vmw_simple_resource_init(struct vmw_private *dev_priv, return ret; } - vmw_resource_activate(&simple->res, simple->func->hw_destroy); + simple->res.hw_destroy = simple->func->hw_destroy; return 0; } @@ -159,7 +159,8 @@ vmw_simple_resource_create_ioctl(struct drm_device *dev, void *data, alloc_size = offsetof(struct vmw_user_simple_resource, simple) + func->size; - account_size = ttm_round_pot(alloc_size) + VMW_IDA_ACC_SIZE; + account_size = ttm_round_pot(alloc_size) + VMW_IDA_ACC_SIZE + + TTM_OBJ_EXTRA_SIZE; ret = ttm_read_lock(&dev_priv->reservation_sem, true); if (ret) @@ -208,7 +209,7 @@ vmw_simple_resource_create_ioctl(struct drm_device *dev, void *data, goto out_err; } - func->set_arg_handle(data, usimple->base.hash.key); + func->set_arg_handle(data, usimple->base.handle); out_err: vmw_resource_unreference(&res); out_ret: diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_so.c b/drivers/gpu/drm/vmwgfx/vmwgfx_so.c index e9b6b7baa009..bc8bb690f1ea 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_so.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_so.c @@ -208,7 +208,7 @@ static int vmw_view_destroy(struct vmw_resource *res) union vmw_view_destroy body; } *cmd; - WARN_ON_ONCE(!mutex_is_locked(&dev_priv->binding_mutex)); + lockdep_assert_held_once(&dev_priv->binding_mutex); vmw_binding_res_list_scrub(&res->binding_head); if (!view->committed || res->id == -1) @@ -366,7 +366,8 @@ int vmw_view_add(struct vmw_cmdbuf_res_manager *man, res = &view->res; view->ctx = ctx; view->srf = vmw_resource_reference(srf); - view->cotable = vmw_context_cotable(ctx, vmw_view_cotables[view_type]); + view->cotable = vmw_resource_reference + (vmw_context_cotable(ctx, vmw_view_cotables[view_type])); view->view_type = view_type; view->view_id = user_key; view->cmd_size = cmd_size; @@ -386,7 +387,7 @@ int vmw_view_add(struct vmw_cmdbuf_res_manager *man, goto out_resource_init; res->id = view->view_id; - vmw_resource_activate(res, vmw_hw_view_destroy); + res->hw_destroy = vmw_hw_view_destroy; out_resource_init: vmw_resource_unreference(&res); @@ -439,7 +440,7 @@ void vmw_view_cotable_list_destroy(struct vmw_private *dev_priv, { struct vmw_view *entry, *next; - WARN_ON_ONCE(!mutex_is_locked(&dev_priv->binding_mutex)); + lockdep_assert_held_once(&dev_priv->binding_mutex); list_for_each_entry_safe(entry, next, list, cotable_head) WARN_ON(vmw_view_destroy(&entry->res)); @@ -459,7 +460,7 @@ void vmw_view_surface_list_destroy(struct vmw_private *dev_priv, { struct vmw_view *entry, *next; - WARN_ON_ONCE(!mutex_is_locked(&dev_priv->binding_mutex)); + lockdep_assert_held_once(&dev_priv->binding_mutex); list_for_each_entry_safe(entry, next, list, srf_head) WARN_ON(vmw_view_destroy(&entry->res)); diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c index 93f6b96ca7bb..e086565c1da6 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c @@ -759,17 +759,21 @@ int vmw_kms_stdu_dma(struct vmw_private *dev_priv, struct vmw_stdu_dirty ddirty; int ret; bool cpu_blit = !(dev_priv->capabilities & SVGA_CAP_3D); + DECLARE_VAL_CONTEXT(val_ctx, NULL, 0); /* * VMs without 3D support don't have the surface DMA command and * we'll be using a CPU blit, and the framebuffer should be moved out * of VRAM. */ - ret = vmw_kms_helper_buffer_prepare(dev_priv, buf, interruptible, - false, cpu_blit); + ret = vmw_validation_add_bo(&val_ctx, buf, false, cpu_blit); if (ret) return ret; + ret = vmw_validation_prepare(&val_ctx, NULL, interruptible); + if (ret) + goto out_unref; + ddirty.transfer = (to_surface) ? SVGA3D_WRITE_HOST_VRAM : SVGA3D_READ_HOST_VRAM; ddirty.left = ddirty.top = S32_MAX; @@ -796,9 +800,13 @@ int vmw_kms_stdu_dma(struct vmw_private *dev_priv, ret = vmw_kms_helper_dirty(dev_priv, vfb, clips, vclips, 0, 0, num_clips, increment, &ddirty.base); - vmw_kms_helper_buffer_finish(dev_priv, file_priv, buf, NULL, - user_fence_rep); + vmw_kms_helper_validation_finish(dev_priv, file_priv, &val_ctx, NULL, + user_fence_rep); + return ret; + +out_unref: + vmw_validation_unref_lists(&val_ctx); return ret; } @@ -924,16 +932,20 @@ int vmw_kms_stdu_surface_dirty(struct vmw_private *dev_priv, struct vmw_framebuffer_surface *vfbs = container_of(framebuffer, typeof(*vfbs), base); struct vmw_stdu_dirty sdirty; - struct vmw_validation_ctx ctx; + DECLARE_VAL_CONTEXT(val_ctx, NULL, 0); int ret; if (!srf) srf = &vfbs->surface->res; - ret = vmw_kms_helper_resource_prepare(srf, true, &ctx); + ret = vmw_validation_add_resource(&val_ctx, srf, 0, NULL, NULL); if (ret) return ret; + ret = vmw_validation_prepare(&val_ctx, &dev_priv->cmdbuf_mutex, true); + if (ret) + goto out_unref; + if (vfbs->is_bo_proxy) { ret = vmw_kms_update_proxy(srf, clips, num_clips, inc); if (ret) @@ -954,8 +966,13 @@ int vmw_kms_stdu_surface_dirty(struct vmw_private *dev_priv, dest_x, dest_y, num_clips, inc, &sdirty.base); out_finish: - vmw_kms_helper_resource_finish(&ctx, out_fence); + vmw_kms_helper_validation_finish(dev_priv, NULL, &val_ctx, out_fence, + NULL); + + return ret; +out_unref: + vmw_validation_unref_lists(&val_ctx); return ret; } @@ -1600,31 +1617,6 @@ int vmw_kms_stdu_init_display(struct vmw_private *dev_priv) dev_priv->active_display_unit = vmw_du_screen_target; - if (dev_priv->capabilities & SVGA_CAP_3D) { - /* - * For 3D VMs, display (scanout) buffer size is the smaller of - * max texture and max STDU - */ - uint32_t max_width, max_height; - - max_width = min(dev_priv->texture_max_width, - dev_priv->stdu_max_width); - max_height = min(dev_priv->texture_max_height, - dev_priv->stdu_max_height); - - dev->mode_config.max_width = max_width; - dev->mode_config.max_height = max_height; - } else { - /* - * Given various display aspect ratios, there's no way to - * estimate these using prim_bb_mem. So just set these to - * something arbitrarily large and we will reject any layout - * that doesn't fit prim_bb_mem later - */ - dev->mode_config.max_width = 8192; - dev->mode_config.max_height = 8192; - } - vmw_kms_create_implicit_placement_property(dev_priv, false); for (i = 0; i < VMWGFX_NUM_DISPLAY_UNITS; ++i) { diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c index e125233e074b..ef09f7edf931 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_surface.c @@ -614,7 +614,7 @@ static int vmw_surface_init(struct vmw_private *dev_priv, */ INIT_LIST_HEAD(&srf->view_list); - vmw_resource_activate(res, vmw_hw_surface_destroy); + res->hw_destroy = vmw_hw_surface_destroy; return ret; } @@ -731,7 +731,7 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data, if (unlikely(vmw_user_surface_size == 0)) vmw_user_surface_size = ttm_round_pot(sizeof(*user_srf)) + - 128; + VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE; num_sizes = 0; for (i = 0; i < DRM_VMW_MAX_SURFACE_FACES; ++i) { @@ -744,7 +744,7 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data, num_sizes == 0) return -EINVAL; - size = vmw_user_surface_size + 128 + + size = vmw_user_surface_size + ttm_round_pot(num_sizes * sizeof(struct drm_vmw_size)) + ttm_round_pot(num_sizes * sizeof(struct vmw_surface_offset)); @@ -886,7 +886,7 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data, goto out_unlock; } - rep->sid = user_srf->prime.base.hash.key; + rep->sid = user_srf->prime.base.handle; vmw_resource_unreference(&res); ttm_read_unlock(&dev_priv->reservation_sem); @@ -1024,7 +1024,7 @@ int vmw_surface_reference_ioctl(struct drm_device *dev, void *data, if (unlikely(ret != 0)) { DRM_ERROR("copy_to_user failed %p %u\n", user_sizes, srf->num_sizes); - ttm_ref_object_base_unref(tfile, base->hash.key, TTM_REF_USAGE); + ttm_ref_object_base_unref(tfile, base->handle, TTM_REF_USAGE); ret = -EFAULT; } @@ -1404,22 +1404,17 @@ int vmw_surface_gb_priv_define(struct drm_device *dev, *srf_out = NULL; if (for_scanout) { - uint32_t max_width, max_height; - if (!svga3dsurface_is_screen_target_format(format)) { DRM_ERROR("Invalid Screen Target surface format."); return -EINVAL; } - max_width = min(dev_priv->texture_max_width, - dev_priv->stdu_max_width); - max_height = min(dev_priv->texture_max_height, - dev_priv->stdu_max_height); - - if (size.width > max_width || size.height > max_height) { + if (size.width > dev_priv->texture_max_width || + size.height > dev_priv->texture_max_height) { DRM_ERROR("%ux%u\n, exceeds max surface size %ux%u", size.width, size.height, - max_width, max_height); + dev_priv->texture_max_width, + dev_priv->texture_max_height); return -EINVAL; } } else { @@ -1495,8 +1490,17 @@ int vmw_surface_gb_priv_define(struct drm_device *dev, if (srf->flags & SVGA3D_SURFACE_BIND_STREAM_OUTPUT) srf->res.backup_size += sizeof(SVGA3dDXSOState); + /* + * Don't set SVGA3D_SURFACE_SCREENTARGET flag for a scanout surface with + * size greater than STDU max width/height. This is really a workaround + * to support creation of big framebuffer requested by some user-space + * for whole topology. That big framebuffer won't really be used for + * binding with screen target as during prepare_fb a separate surface is + * created so it's safe to ignore SVGA3D_SURFACE_SCREENTARGET flag. + */ if (dev_priv->active_display_unit == vmw_du_screen_target && - for_scanout) + for_scanout && size.width <= dev_priv->stdu_max_width && + size.height <= dev_priv->stdu_max_height) srf->flags |= SVGA3D_SURFACE_SCREENTARGET; /* @@ -1609,9 +1613,9 @@ vmw_gb_surface_define_internal(struct drm_device *dev, if (unlikely(vmw_user_surface_size == 0)) vmw_user_surface_size = ttm_round_pot(sizeof(*user_srf)) + - 128; + VMW_IDA_ACC_SIZE + TTM_OBJ_EXTRA_SIZE; - size = vmw_user_surface_size + 128; + size = vmw_user_surface_size; /* Define a surface based on the parameters. */ ret = vmw_surface_gb_priv_define(dev, @@ -1683,7 +1687,7 @@ vmw_gb_surface_define_internal(struct drm_device *dev, goto out_unlock; } - rep->handle = user_srf->prime.base.hash.key; + rep->handle = user_srf->prime.base.handle; rep->backup_size = res->backup_size; if (res->backup) { rep->buffer_map_handle = @@ -1745,7 +1749,7 @@ vmw_gb_surface_reference_internal(struct drm_device *dev, if (unlikely(ret != 0)) { DRM_ERROR("Could not add a reference to a GB surface " "backup buffer.\n"); - (void) ttm_ref_object_base_unref(tfile, base->hash.key, + (void) ttm_ref_object_base_unref(tfile, base->handle, TTM_REF_USAGE); goto out_bad_resource; } @@ -1759,7 +1763,7 @@ vmw_gb_surface_reference_internal(struct drm_device *dev, rep->creq.base.array_size = srf->array_size; rep->creq.base.buffer_handle = backup_handle; rep->creq.base.base_size = srf->base_size; - rep->crep.handle = user_srf->prime.base.hash.key; + rep->crep.handle = user_srf->prime.base.handle; rep->crep.backup_size = srf->res.backup_size; rep->crep.buffer_handle = backup_handle; rep->crep.buffer_map_handle = diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c new file mode 100644 index 000000000000..184025fa938e --- /dev/null +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.c @@ -0,0 +1,770 @@ +// SPDX-License-Identifier: GPL-2.0 OR MIT +/************************************************************************** + * + * Copyright © 2018 VMware, Inc., Palo Alto, CA., USA + * All Rights Reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the + * "Software"), to deal in the Software without restriction, including + * without limitation the rights to use, copy, modify, merge, publish, + * distribute, sub license, and/or sell copies of the Software, and to + * permit persons to whom the Software is furnished to do so, subject to + * the following conditions: + * + * The above copyright notice and this permission notice (including the + * next paragraph) shall be included in all copies or substantial portions + * of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, + * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR + * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE + * USE OR OTHER DEALINGS IN THE SOFTWARE. + * + **************************************************************************/ +#include <linux/slab.h> +#include "vmwgfx_validation.h" +#include "vmwgfx_drv.h" + +/** + * struct vmw_validation_bo_node - Buffer object validation metadata. + * @base: Metadata used for TTM reservation- and validation. + * @hash: A hash entry used for the duplicate detection hash table. + * @as_mob: Validate as mob. + * @cpu_blit: Validate for cpu blit access. + * + * Bit fields are used since these structures are allocated and freed in + * large numbers and space conservation is desired. + */ +struct vmw_validation_bo_node { + struct ttm_validate_buffer base; + struct drm_hash_item hash; + u32 as_mob : 1; + u32 cpu_blit : 1; +}; + +/** + * struct vmw_validation_res_node - Resource validation metadata. + * @head: List head for the resource validation list. + * @hash: A hash entry used for the duplicate detection hash table. + * @res: Reference counted resource pointer. + * @new_backup: Non ref-counted pointer to new backup buffer to be assigned + * to a resource. + * @new_backup_offset: Offset into the new backup mob for resources that can + * share MOBs. + * @no_buffer_needed: Kernel does not need to allocate a MOB during validation, + * the command stream provides a mob bind operation. + * @switching_backup: The validation process is switching backup MOB. + * @first_usage: True iff the resource has been seen only once in the current + * validation batch. + * @reserved: Whether the resource is currently reserved by this process. + * @private: Optionally additional memory for caller-private data. + * + * Bit fields are used since these structures are allocated and freed in + * large numbers and space conservation is desired. + */ +struct vmw_validation_res_node { + struct list_head head; + struct drm_hash_item hash; + struct vmw_resource *res; + struct vmw_buffer_object *new_backup; + unsigned long new_backup_offset; + u32 no_buffer_needed : 1; + u32 switching_backup : 1; + u32 first_usage : 1; + u32 reserved : 1; + unsigned long private[0]; +}; + +/** + * vmw_validation_mem_alloc - Allocate kernel memory from the validation + * context based allocator + * @ctx: The validation context + * @size: The number of bytes to allocated. + * + * The memory allocated may not exceed PAGE_SIZE, and the returned + * address is aligned to sizeof(long). All memory allocated this way is + * reclaimed after validation when calling any of the exported functions: + * vmw_validation_unref_lists() + * vmw_validation_revert() + * vmw_validation_done() + * + * Return: Pointer to the allocated memory on success. NULL on failure. + */ +void *vmw_validation_mem_alloc(struct vmw_validation_context *ctx, + unsigned int size) +{ + void *addr; + + size = vmw_validation_align(size); + if (size > PAGE_SIZE) + return NULL; + + if (ctx->mem_size_left < size) { + struct page *page = alloc_page(GFP_KERNEL | __GFP_ZERO); + + if (!page) + return NULL; + + list_add_tail(&page->lru, &ctx->page_list); + ctx->page_address = page_address(page); + ctx->mem_size_left = PAGE_SIZE; + } + + addr = (void *) (ctx->page_address + (PAGE_SIZE - ctx->mem_size_left)); + ctx->mem_size_left -= size; + + return addr; +} + +/** + * vmw_validation_mem_free - Free all memory allocated using + * vmw_validation_mem_alloc() + * @ctx: The validation context + * + * All memory previously allocated for this context using + * vmw_validation_mem_alloc() is freed. + */ +static void vmw_validation_mem_free(struct vmw_validation_context *ctx) +{ + struct page *entry, *next; + + list_for_each_entry_safe(entry, next, &ctx->page_list, lru) { + list_del_init(&entry->lru); + __free_page(entry); + } + + ctx->mem_size_left = 0; +} + +/** + * vmw_validation_find_bo_dup - Find a duplicate buffer object entry in the + * validation context's lists. + * @ctx: The validation context to search. + * @vbo: The buffer object to search for. + * + * Return: Pointer to the struct vmw_validation_bo_node referencing the + * duplicate, or NULL if none found. + */ +static struct vmw_validation_bo_node * +vmw_validation_find_bo_dup(struct vmw_validation_context *ctx, + struct vmw_buffer_object *vbo) +{ + struct vmw_validation_bo_node *bo_node = NULL; + + if (!ctx->merge_dups) + return NULL; + + if (ctx->ht) { + struct drm_hash_item *hash; + + if (!drm_ht_find_item(ctx->ht, (unsigned long) vbo, &hash)) + bo_node = container_of(hash, typeof(*bo_node), hash); + } else { + struct vmw_validation_bo_node *entry; + + list_for_each_entry(entry, &ctx->bo_list, base.head) { + if (entry->base.bo == &vbo->base) { + bo_node = entry; + break; + } + } + } + + return bo_node; +} + +/** + * vmw_validation_find_res_dup - Find a duplicate resource entry in the + * validation context's lists. + * @ctx: The validation context to search. + * @vbo: The buffer object to search for. + * + * Return: Pointer to the struct vmw_validation_bo_node referencing the + * duplicate, or NULL if none found. + */ +static struct vmw_validation_res_node * +vmw_validation_find_res_dup(struct vmw_validation_context *ctx, + struct vmw_resource *res) +{ + struct vmw_validation_res_node *res_node = NULL; + + if (!ctx->merge_dups) + return NULL; + + if (ctx->ht) { + struct drm_hash_item *hash; + + if (!drm_ht_find_item(ctx->ht, (unsigned long) res, &hash)) + res_node = container_of(hash, typeof(*res_node), hash); + } else { + struct vmw_validation_res_node *entry; + + list_for_each_entry(entry, &ctx->resource_ctx_list, head) { + if (entry->res == res) { + res_node = entry; + goto out; + } + } + + list_for_each_entry(entry, &ctx->resource_list, head) { + if (entry->res == res) { + res_node = entry; + break; + } + } + + } +out: + return res_node; +} + +/** + * vmw_validation_add_bo - Add a buffer object to the validation context. + * @ctx: The validation context. + * @vbo: The buffer object. + * @as_mob: Validate as mob, otherwise suitable for GMR operations. + * @cpu_blit: Validate in a page-mappable location. + * + * Return: Zero on success, negative error code otherwise. + */ +int vmw_validation_add_bo(struct vmw_validation_context *ctx, + struct vmw_buffer_object *vbo, + bool as_mob, + bool cpu_blit) +{ + struct vmw_validation_bo_node *bo_node; + + bo_node = vmw_validation_find_bo_dup(ctx, vbo); + if (bo_node) { + if (bo_node->as_mob != as_mob || + bo_node->cpu_blit != cpu_blit) { + DRM_ERROR("Inconsistent buffer usage.\n"); + return -EINVAL; + } + } else { + struct ttm_validate_buffer *val_buf; + int ret; + + bo_node = vmw_validation_mem_alloc(ctx, sizeof(*bo_node)); + if (!bo_node) + return -ENOMEM; + + if (ctx->ht) { + bo_node->hash.key = (unsigned long) vbo; + ret = drm_ht_insert_item(ctx->ht, &bo_node->hash); + if (ret) { + DRM_ERROR("Failed to initialize a buffer " + "validation entry.\n"); + return ret; + } + } + val_buf = &bo_node->base; + val_buf->bo = ttm_bo_get_unless_zero(&vbo->base); + if (!val_buf->bo) + return -ESRCH; + val_buf->shared = false; + list_add_tail(&val_buf->head, &ctx->bo_list); + bo_node->as_mob = as_mob; + bo_node->cpu_blit = cpu_blit; + } + + return 0; +} + +/** + * vmw_validation_add_resource - Add a resource to the validation context. + * @ctx: The validation context. + * @res: The resource. + * @priv_size: Size of private, additional metadata. + * @p_node: Output pointer of additional metadata address. + * @first_usage: Whether this was the first time this resource was seen. + * + * Return: Zero on success, negative error code otherwise. + */ +int vmw_validation_add_resource(struct vmw_validation_context *ctx, + struct vmw_resource *res, + size_t priv_size, + void **p_node, + bool *first_usage) +{ + struct vmw_validation_res_node *node; + int ret; + + node = vmw_validation_find_res_dup(ctx, res); + if (node) { + node->first_usage = 0; + goto out_fill; + } + + node = vmw_validation_mem_alloc(ctx, sizeof(*node) + priv_size); + if (!node) { + DRM_ERROR("Failed to allocate a resource validation " + "entry.\n"); + return -ENOMEM; + } + + if (ctx->ht) { + node->hash.key = (unsigned long) res; + ret = drm_ht_insert_item(ctx->ht, &node->hash); + if (ret) { + DRM_ERROR("Failed to initialize a resource validation " + "entry.\n"); + return ret; + } + } + node->res = vmw_resource_reference_unless_doomed(res); + if (!node->res) + return -ESRCH; + + node->first_usage = 1; + if (!res->dev_priv->has_mob) { + list_add_tail(&node->head, &ctx->resource_list); + } else { + switch (vmw_res_type(res)) { + case vmw_res_context: + case vmw_res_dx_context: + list_add(&node->head, &ctx->resource_ctx_list); + break; + case vmw_res_cotable: + list_add_tail(&node->head, &ctx->resource_ctx_list); + break; + default: + list_add_tail(&node->head, &ctx->resource_list); + break; + } + } + +out_fill: + if (first_usage) + *first_usage = node->first_usage; + if (p_node) + *p_node = &node->private; + + return 0; +} + +/** + * vmw_validation_res_switch_backup - Register a backup MOB switch during + * validation. + * @ctx: The validation context. + * @val_private: The additional meta-data pointer returned when the + * resource was registered with the validation context. Used to identify + * the resource. + * @vbo: The new backup buffer object MOB. This buffer object needs to have + * already been registered with the validation context. + * @backup_offset: Offset into the new backup MOB. + */ +void vmw_validation_res_switch_backup(struct vmw_validation_context *ctx, + void *val_private, + struct vmw_buffer_object *vbo, + unsigned long backup_offset) +{ + struct vmw_validation_res_node *val; + + val = container_of(val_private, typeof(*val), private); + + val->switching_backup = 1; + if (val->first_usage) + val->no_buffer_needed = 1; + + val->new_backup = vbo; + val->new_backup_offset = backup_offset; +} + +/** + * vmw_validation_res_reserve - Reserve all resources registered with this + * validation context. + * @ctx: The validation context. + * @intr: Use interruptible waits when possible. + * + * Return: Zero on success, -ERESTARTSYS if interrupted. Negative error + * code on failure. + */ +int vmw_validation_res_reserve(struct vmw_validation_context *ctx, + bool intr) +{ + struct vmw_validation_res_node *val; + int ret = 0; + + list_splice_init(&ctx->resource_ctx_list, &ctx->resource_list); + + list_for_each_entry(val, &ctx->resource_list, head) { + struct vmw_resource *res = val->res; + + ret = vmw_resource_reserve(res, intr, val->no_buffer_needed); + if (ret) + goto out_unreserve; + + val->reserved = 1; + if (res->backup) { + struct vmw_buffer_object *vbo = res->backup; + + ret = vmw_validation_add_bo + (ctx, vbo, vmw_resource_needs_backup(res), + false); + if (ret) + goto out_unreserve; + } + } + + return 0; + +out_unreserve: + vmw_validation_res_unreserve(ctx, true); + return ret; +} + +/** + * vmw_validation_res_unreserve - Unreserve all reserved resources + * registered with this validation context. + * @ctx: The validation context. + * @backoff: Whether this is a backoff- of a commit-type operation. This + * is used to determine whether to switch backup MOBs or not. + */ +void vmw_validation_res_unreserve(struct vmw_validation_context *ctx, + bool backoff) +{ + struct vmw_validation_res_node *val; + + list_splice_init(&ctx->resource_ctx_list, &ctx->resource_list); + + list_for_each_entry(val, &ctx->resource_list, head) { + if (val->reserved) + vmw_resource_unreserve(val->res, + !backoff && + val->switching_backup, + val->new_backup, + val->new_backup_offset); + } +} + +/** + * vmw_validation_bo_validate_single - Validate a single buffer object. + * @bo: The TTM buffer object base. + * @interruptible: Whether to perform waits interruptible if possible. + * @validate_as_mob: Whether to validate in MOB memory. + * + * Return: Zero on success, -ERESTARTSYS if interrupted. Negative error + * code on failure. + */ +int vmw_validation_bo_validate_single(struct ttm_buffer_object *bo, + bool interruptible, + bool validate_as_mob) +{ + struct vmw_buffer_object *vbo = + container_of(bo, struct vmw_buffer_object, base); + struct ttm_operation_ctx ctx = { + .interruptible = interruptible, + .no_wait_gpu = false + }; + int ret; + + if (vbo->pin_count > 0) + return 0; + + if (validate_as_mob) + return ttm_bo_validate(bo, &vmw_mob_placement, &ctx); + + /** + * Put BO in VRAM if there is space, otherwise as a GMR. + * If there is no space in VRAM and GMR ids are all used up, + * start evicting GMRs to make room. If the DMA buffer can't be + * used as a GMR, this will return -ENOMEM. + */ + + ret = ttm_bo_validate(bo, &vmw_vram_gmr_placement, &ctx); + if (ret == 0 || ret == -ERESTARTSYS) + return ret; + + /** + * If that failed, try VRAM again, this time evicting + * previous contents. + */ + + ret = ttm_bo_validate(bo, &vmw_vram_placement, &ctx); + return ret; +} + +/** + * vmw_validation_bo_validate - Validate all buffer objects registered with + * the validation context. + * @ctx: The validation context. + * @intr: Whether to perform waits interruptible if possible. + * + * Return: Zero on success, -ERESTARTSYS if interrupted, + * negative error code on failure. + */ +int vmw_validation_bo_validate(struct vmw_validation_context *ctx, bool intr) +{ + struct vmw_validation_bo_node *entry; + int ret; + + list_for_each_entry(entry, &ctx->bo_list, base.head) { + if (entry->cpu_blit) { + struct ttm_operation_ctx ctx = { + .interruptible = intr, + .no_wait_gpu = false + }; + + ret = ttm_bo_validate(entry->base.bo, + &vmw_nonfixed_placement, &ctx); + } else { + ret = vmw_validation_bo_validate_single + (entry->base.bo, intr, entry->as_mob); + } + if (ret) + return ret; + } + return 0; +} + +/** + * vmw_validation_res_validate - Validate all resources registered with the + * validation context. + * @ctx: The validation context. + * @intr: Whether to perform waits interruptible if possible. + * + * Before this function is called, all resource backup buffers must have + * been validated. + * + * Return: Zero on success, -ERESTARTSYS if interrupted, + * negative error code on failure. + */ +int vmw_validation_res_validate(struct vmw_validation_context *ctx, bool intr) +{ + struct vmw_validation_res_node *val; + int ret; + + list_for_each_entry(val, &ctx->resource_list, head) { + struct vmw_resource *res = val->res; + struct vmw_buffer_object *backup = res->backup; + + ret = vmw_resource_validate(res, intr); + if (ret) { + if (ret != -ERESTARTSYS) + DRM_ERROR("Failed to validate resource.\n"); + return ret; + } + + /* Check if the resource switched backup buffer */ + if (backup && res->backup && (backup != res->backup)) { + struct vmw_buffer_object *vbo = res->backup; + + ret = vmw_validation_add_bo + (ctx, vbo, vmw_resource_needs_backup(res), + false); + if (ret) + return ret; + } + } + return 0; +} + +/** + * vmw_validation_drop_ht - Reset the hash table used for duplicate finding + * and unregister it from this validation context. + * @ctx: The validation context. + * + * The hash table used for duplicate finding is an expensive resource and + * may be protected by mutexes that may cause deadlocks during resource + * unreferencing if held. After resource- and buffer object registering, + * there is no longer any use for this hash table, so allow freeing it + * either to shorten any mutex locking time, or before resources- and + * buffer objects are freed during validation context cleanup. + */ +void vmw_validation_drop_ht(struct vmw_validation_context *ctx) +{ + struct vmw_validation_bo_node *entry; + struct vmw_validation_res_node *val; + + if (!ctx->ht) + return; + + list_for_each_entry(entry, &ctx->bo_list, base.head) + (void) drm_ht_remove_item(ctx->ht, &entry->hash); + + list_for_each_entry(val, &ctx->resource_list, head) + (void) drm_ht_remove_item(ctx->ht, &val->hash); + + list_for_each_entry(val, &ctx->resource_ctx_list, head) + (void) drm_ht_remove_item(ctx->ht, &val->hash); + + ctx->ht = NULL; +} + +/** + * vmw_validation_unref_lists - Unregister previously registered buffer + * object and resources. + * @ctx: The validation context. + * + * Note that this function may cause buffer object- and resource destructors + * to be invoked. + */ +void vmw_validation_unref_lists(struct vmw_validation_context *ctx) +{ + struct vmw_validation_bo_node *entry; + struct vmw_validation_res_node *val; + + list_for_each_entry(entry, &ctx->bo_list, base.head) + ttm_bo_unref(&entry->base.bo); + + list_splice_init(&ctx->resource_ctx_list, &ctx->resource_list); + list_for_each_entry(val, &ctx->resource_list, head) + vmw_resource_unreference(&val->res); + + /* + * No need to detach each list entry since they are all freed with + * vmw_validation_free_mem. Just make the inaccessible. + */ + INIT_LIST_HEAD(&ctx->bo_list); + INIT_LIST_HEAD(&ctx->resource_list); + + vmw_validation_mem_free(ctx); +} + +/** + * vmw_validation_prepare - Prepare a validation context for command + * submission. + * @ctx: The validation context. + * @mutex: The mutex used to protect resource reservation. + * @intr: Whether to perform waits interruptible if possible. + * + * Note that the single reservation mutex @mutex is an unfortunate + * construct. Ideally resource reservation should be moved to per-resource + * ww_mutexes. + * If this functions doesn't return Zero to indicate success, all resources + * are left unreserved but still referenced. + * Return: Zero on success, -ERESTARTSYS if interrupted, negative error code + * on error. + */ +int vmw_validation_prepare(struct vmw_validation_context *ctx, + struct mutex *mutex, + bool intr) +{ + int ret = 0; + + if (mutex) { + if (intr) + ret = mutex_lock_interruptible(mutex); + else + mutex_lock(mutex); + if (ret) + return -ERESTARTSYS; + } + + ctx->res_mutex = mutex; + ret = vmw_validation_res_reserve(ctx, intr); + if (ret) + goto out_no_res_reserve; + + ret = vmw_validation_bo_reserve(ctx, intr); + if (ret) + goto out_no_bo_reserve; + + ret = vmw_validation_bo_validate(ctx, intr); + if (ret) + goto out_no_validate; + + ret = vmw_validation_res_validate(ctx, intr); + if (ret) + goto out_no_validate; + + return 0; + +out_no_validate: + vmw_validation_bo_backoff(ctx); +out_no_bo_reserve: + vmw_validation_res_unreserve(ctx, true); +out_no_res_reserve: + if (mutex) + mutex_unlock(mutex); + + return ret; +} + +/** + * vmw_validation_revert - Revert validation actions if command submission + * failed. + * + * @ctx: The validation context. + * + * The caller still needs to unref resources after a call to this function. + */ +void vmw_validation_revert(struct vmw_validation_context *ctx) +{ + vmw_validation_bo_backoff(ctx); + vmw_validation_res_unreserve(ctx, true); + if (ctx->res_mutex) + mutex_unlock(ctx->res_mutex); + vmw_validation_unref_lists(ctx); +} + +/** + * vmw_validation_cone - Commit validation actions after command submission + * success. + * @ctx: The validation context. + * @fence: Fence with which to fence all buffer objects taking part in the + * command submission. + * + * The caller does NOT need to unref resources after a call to this function. + */ +void vmw_validation_done(struct vmw_validation_context *ctx, + struct vmw_fence_obj *fence) +{ + vmw_validation_bo_fence(ctx, fence); + vmw_validation_res_unreserve(ctx, false); + if (ctx->res_mutex) + mutex_unlock(ctx->res_mutex); + vmw_validation_unref_lists(ctx); +} + +/** + * vmw_validation_preload_bo - Preload the validation memory allocator for a + * call to vmw_validation_add_bo(). + * @ctx: Pointer to the validation context. + * + * Iff this function returns successfully, the next call to + * vmw_validation_add_bo() is guaranteed not to sleep. An error is not fatal + * but voids the guarantee. + * + * Returns: Zero if successful, %-EINVAL otherwise. + */ +int vmw_validation_preload_bo(struct vmw_validation_context *ctx) +{ + unsigned int size = sizeof(struct vmw_validation_bo_node); + + if (!vmw_validation_mem_alloc(ctx, size)) + return -ENOMEM; + + ctx->mem_size_left += size; + return 0; +} + +/** + * vmw_validation_preload_res - Preload the validation memory allocator for a + * call to vmw_validation_add_res(). + * @ctx: Pointer to the validation context. + * @size: Size of the validation node extra data. See below. + * + * Iff this function returns successfully, the next call to + * vmw_validation_add_res() with the same or smaller @size is guaranteed not to + * sleep. An error is not fatal but voids the guarantee. + * + * Returns: Zero if successful, %-EINVAL otherwise. + */ +int vmw_validation_preload_res(struct vmw_validation_context *ctx, + unsigned int size) +{ + size = vmw_validation_align(sizeof(struct vmw_validation_res_node) + + size) + + vmw_validation_align(sizeof(struct vmw_validation_bo_node)); + if (!vmw_validation_mem_alloc(ctx, size)) + return -ENOMEM; + + ctx->mem_size_left += size; + return 0; +} diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h new file mode 100644 index 000000000000..b57e3292c386 --- /dev/null +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_validation.h @@ -0,0 +1,227 @@ +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ +/************************************************************************** + * + * Copyright © 2018 VMware, Inc., Palo Alto, CA., USA + * All Rights Reserved. + * + * Permission is hereby granted, free of charge, to any person obtaining a + * copy of this software and associated documentation files (the + * "Software"), to deal in the Software without restriction, including + * without limitation the rights to use, copy, modify, merge, publish, + * distribute, sub license, and/or sell copies of the Software, and to + * permit persons to whom the Software is furnished to do so, subject to + * the following conditions: + * + * The above copyright notice and this permission notice (including the + * next paragraph) shall be included in all copies or substantial portions + * of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL + * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, + * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR + * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE + * USE OR OTHER DEALINGS IN THE SOFTWARE. + * + **************************************************************************/ +#ifndef _VMWGFX_VALIDATION_H_ +#define _VMWGFX_VALIDATION_H_ + +#include <drm/drm_hashtab.h> +#include <linux/list.h> +#include <linux/ww_mutex.h> +#include <drm/ttm/ttm_execbuf_util.h> + +/** + * struct vmw_validation_context - Per command submission validation context + * @ht: Hash table used to find resource- or buffer object duplicates + * @resource_list: List head for resource validation metadata + * @resource_ctx_list: List head for resource validation metadata for + * resources that need to be validated before those in @resource_list + * @bo_list: List head for buffer objects + * @page_list: List of pages used by the memory allocator + * @ticket: Ticked used for ww mutex locking + * @res_mutex: Pointer to mutex used for resource reserving + * @merge_dups: Whether to merge metadata for duplicate resources or + * buffer objects + * @mem_size_left: Free memory left in the last page in @page_list + * @page_address: Kernel virtual address of the last page in @page_list + */ +struct vmw_validation_context { + struct drm_open_hash *ht; + struct list_head resource_list; + struct list_head resource_ctx_list; + struct list_head bo_list; + struct list_head page_list; + struct ww_acquire_ctx ticket; + struct mutex *res_mutex; + unsigned int merge_dups; + unsigned int mem_size_left; + u8 *page_address; +}; + +struct vmw_buffer_object; +struct vmw_resource; +struct vmw_fence_obj; + +#if 0 +/** + * DECLARE_VAL_CONTEXT - Declare a validation context with initialization + * @_name: The name of the variable + * @_ht: The hash table used to find dups or NULL if none + * @_merge_dups: Whether to merge duplicate buffer object- or resource + * entries. If set to true, ideally a hash table pointer should be supplied + * as well unless the number of resources and buffer objects per validation + * is known to be very small + */ +#endif +#define DECLARE_VAL_CONTEXT(_name, _ht, _merge_dups) \ + struct vmw_validation_context _name = \ + { .ht = _ht, \ + .resource_list = LIST_HEAD_INIT((_name).resource_list), \ + .resource_ctx_list = LIST_HEAD_INIT((_name).resource_ctx_list), \ + .bo_list = LIST_HEAD_INIT((_name).bo_list), \ + .page_list = LIST_HEAD_INIT((_name).page_list), \ + .res_mutex = NULL, \ + .merge_dups = _merge_dups, \ + .mem_size_left = 0, \ + } + +/** + * vmw_validation_has_bos - return whether the validation context has + * any buffer objects registered. + * + * @ctx: The validation context + * Returns: Whether any buffer objects are registered + */ +static inline bool +vmw_validation_has_bos(struct vmw_validation_context *ctx) +{ + return !list_empty(&ctx->bo_list); +} + +/** + * vmw_validation_set_ht - Register a hash table for duplicate finding + * @ctx: The validation context + * @ht: Pointer to a hash table to use for duplicate finding + * This function is intended to be used if the hash table wasn't + * available at validation context declaration time + */ +static inline void vmw_validation_set_ht(struct vmw_validation_context *ctx, + struct drm_open_hash *ht) +{ + ctx->ht = ht; +} + +/** + * vmw_validation_bo_reserve - Reserve buffer objects registered with a + * validation context + * @ctx: The validation context + * @intr: Perform waits interruptible + * + * Return: Zero on success, -ERESTARTSYS when interrupted, negative error + * code on failure + */ +static inline int +vmw_validation_bo_reserve(struct vmw_validation_context *ctx, + bool intr) +{ + return ttm_eu_reserve_buffers(&ctx->ticket, &ctx->bo_list, intr, + NULL); +} + +/** + * vmw_validation_bo_backoff - Unreserve buffer objects registered with a + * validation context + * @ctx: The validation context + * + * This function unreserves the buffer objects previously reserved using + * vmw_validation_bo_reserve. It's typically used as part of an error path + */ +static inline void +vmw_validation_bo_backoff(struct vmw_validation_context *ctx) +{ + ttm_eu_backoff_reservation(&ctx->ticket, &ctx->bo_list); +} + +/** + * vmw_validation_bo_fence - Unreserve and fence buffer objects registered + * with a validation context + * @ctx: The validation context + * + * This function unreserves the buffer objects previously reserved using + * vmw_validation_bo_reserve, and fences them with a fence object. + */ +static inline void +vmw_validation_bo_fence(struct vmw_validation_context *ctx, + struct vmw_fence_obj *fence) +{ + ttm_eu_fence_buffer_objects(&ctx->ticket, &ctx->bo_list, + (void *) fence); +} + +/** + * vmw_validation_context_init - Initialize a validation context + * @ctx: Pointer to the validation context to initialize + * + * This function initializes a validation context with @merge_dups set + * to false + */ +static inline void +vmw_validation_context_init(struct vmw_validation_context *ctx) +{ + memset(ctx, 0, sizeof(*ctx)); + INIT_LIST_HEAD(&ctx->resource_list); + INIT_LIST_HEAD(&ctx->resource_ctx_list); + INIT_LIST_HEAD(&ctx->bo_list); +} + +/** + * vmw_validation_align - Align a validation memory allocation + * @val: The size to be aligned + * + * Returns: @val aligned to the granularity used by the validation memory + * allocator. + */ +static inline unsigned int vmw_validation_align(unsigned int val) +{ + return ALIGN(val, sizeof(long)); +} + +int vmw_validation_add_bo(struct vmw_validation_context *ctx, + struct vmw_buffer_object *vbo, + bool as_mob, bool cpu_blit); +int vmw_validation_bo_validate_single(struct ttm_buffer_object *bo, + bool interruptible, + bool validate_as_mob); +int vmw_validation_bo_validate(struct vmw_validation_context *ctx, bool intr); +void vmw_validation_unref_lists(struct vmw_validation_context *ctx); +int vmw_validation_add_resource(struct vmw_validation_context *ctx, + struct vmw_resource *res, + size_t priv_size, + void **p_node, + bool *first_usage); +void vmw_validation_drop_ht(struct vmw_validation_context *ctx); +int vmw_validation_res_reserve(struct vmw_validation_context *ctx, + bool intr); +void vmw_validation_res_unreserve(struct vmw_validation_context *ctx, + bool backoff); +void vmw_validation_res_switch_backup(struct vmw_validation_context *ctx, + void *val_private, + struct vmw_buffer_object *vbo, + unsigned long backup_offset); +int vmw_validation_res_validate(struct vmw_validation_context *ctx, bool intr); + +int vmw_validation_prepare(struct vmw_validation_context *ctx, + struct mutex *mutex, bool intr); +void vmw_validation_revert(struct vmw_validation_context *ctx); +void vmw_validation_done(struct vmw_validation_context *ctx, + struct vmw_fence_obj *fence); + +void *vmw_validation_mem_alloc(struct vmw_validation_context *ctx, + unsigned int size); +int vmw_validation_preload_bo(struct vmw_validation_context *ctx); +int vmw_validation_preload_res(struct vmw_validation_context *ctx, + unsigned int size); +#endif diff --git a/drivers/gpu/drm/zte/zx_drm_drv.c b/drivers/gpu/drm/zte/zx_drm_drv.c index 6f4205e80378..11ef17c2d1c1 100644 --- a/drivers/gpu/drm/zte/zx_drm_drv.c +++ b/drivers/gpu/drm/zte/zx_drm_drv.c @@ -31,7 +31,6 @@ static const struct drm_mode_config_funcs zx_drm_mode_config_funcs = { .fb_create = drm_gem_fb_create, - .output_poll_changed = drm_fb_helper_output_poll_changed, .atomic_check = drm_atomic_helper_check, .atomic_commit = drm_atomic_helper_commit, }; @@ -41,7 +40,6 @@ DEFINE_DRM_GEM_CMA_FOPS(zx_drm_fops); static struct drm_driver zx_drm_driver = { .driver_features = DRIVER_GEM | DRIVER_MODESET | DRIVER_PRIME | DRIVER_ATOMIC, - .lastclose = drm_fb_helper_lastclose, .gem_free_object_unlocked = drm_gem_cma_free_object, .gem_vm_ops = &drm_gem_cma_vm_ops, .dumb_create = drm_gem_cma_dumb_create, @@ -101,20 +99,14 @@ static int zx_drm_bind(struct device *dev) drm_mode_config_reset(drm); drm_kms_helper_poll_init(drm); - ret = drm_fb_cma_fbdev_init(drm, 32, 0); - if (ret) { - DRM_DEV_ERROR(dev, "failed to init cma fbdev: %d\n", ret); - goto out_poll_fini; - } - ret = drm_dev_register(drm, 0); if (ret) - goto out_fbdev_fini; + goto out_poll_fini; + + drm_fbdev_generic_setup(drm, 32); return 0; -out_fbdev_fini: - drm_fb_cma_fbdev_fini(drm); out_poll_fini: drm_kms_helper_poll_fini(drm); drm_mode_config_cleanup(drm); @@ -131,7 +123,6 @@ static void zx_drm_unbind(struct device *dev) struct drm_device *drm = dev_get_drvdata(dev); drm_dev_unregister(drm); - drm_fb_cma_fbdev_fini(drm); drm_kms_helper_poll_fini(drm); drm_mode_config_cleanup(drm); component_unbind_all(dev, drm); @@ -161,10 +152,8 @@ static int zx_drm_probe(struct platform_device *pdev) if (ret) return ret; - for_each_available_child_of_node(parent, child) { + for_each_available_child_of_node(parent, child) component_match_add(dev, &match, compare_of, child); - of_node_put(child); - } return component_master_add_with_match(dev, &zx_drm_master_ops, match); } diff --git a/drivers/gpu/host1x/bus.c b/drivers/gpu/host1x/bus.c index 815bdb42e3f0..b4c385d4a6af 100644 --- a/drivers/gpu/host1x/bus.c +++ b/drivers/gpu/host1x/bus.c @@ -331,7 +331,7 @@ static const struct dev_pm_ops host1x_device_pm_ops = { struct bus_type host1x_bus_type = { .name = "host1x", .match = host1x_device_match, - .dma_configure = host1x_dma_configure, + .dma_configure = host1x_dma_configure, .pm = &host1x_device_pm_ops, }; diff --git a/drivers/gpu/host1x/dev.c b/drivers/gpu/host1x/dev.c index d88073e7d22d..de6bc4e7fa23 100644 --- a/drivers/gpu/host1x/dev.c +++ b/drivers/gpu/host1x/dev.c @@ -29,6 +29,10 @@ #include <trace/events/host1x.h> #undef CREATE_TRACE_POINTS +#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU) +#include <asm/dma-iommu.h> +#endif + #include "bus.h" #include "channel.h" #include "debug.h" @@ -217,7 +221,14 @@ static int host1x_probe(struct platform_device *pdev) dev_err(&pdev->dev, "failed to get reset: %d\n", err); return err; } - +#if IS_ENABLED(CONFIG_ARM_DMA_USE_IOMMU) + if (host->dev->archdata.mapping) { + struct dma_iommu_mapping *mapping = + to_dma_iommu_mapping(host->dev); + arm_iommu_detach_device(host->dev); + arm_iommu_release_mapping(mapping); + } +#endif if (IS_ENABLED(CONFIG_TEGRA_HOST1X_FIREWALL)) goto skip_iommu; diff --git a/drivers/gpu/vga/vga_switcheroo.c b/drivers/gpu/vga/vga_switcheroo.c index a96bf46bc483..cf2a18571d48 100644 --- a/drivers/gpu/vga/vga_switcheroo.c +++ b/drivers/gpu/vga/vga_switcheroo.c @@ -215,6 +215,8 @@ static void vga_switcheroo_enable(void) return; client->id = ret | ID_BIT_AUDIO; + if (client->ops->gpu_bound) + client->ops->gpu_bound(client->pdev, ret); } vga_switcheroo_debugfs_init(&vgasr_priv); diff --git a/drivers/hid/hid-apple.c b/drivers/hid/hid-apple.c index 25b7bd56ae11..1cb41992aaa1 100644 --- a/drivers/hid/hid-apple.c +++ b/drivers/hid/hid-apple.c @@ -335,7 +335,8 @@ static int apple_input_mapping(struct hid_device *hdev, struct hid_input *hi, struct hid_field *field, struct hid_usage *usage, unsigned long **bit, int *max) { - if (usage->hid == (HID_UP_CUSTOM | 0x0003)) { + if (usage->hid == (HID_UP_CUSTOM | 0x0003) || + usage->hid == (HID_UP_MSVENDOR | 0x0003)) { /* The fn key on Apple USB keyboards */ set_bit(EV_REP, hi->input->evbit); hid_map_usage_clear(hi, usage, bit, max, EV_KEY, KEY_FN); @@ -472,6 +473,12 @@ static const struct hid_device_id apple_devices[] = { .driver_data = APPLE_NUMLOCK_EMULATION | APPLE_HAS_FN }, { HID_USB_DEVICE(USB_VENDOR_ID_APPLE, USB_DEVICE_ID_APPLE_MAGIC_KEYBOARD_ANSI), .driver_data = APPLE_HAS_FN }, + { HID_BLUETOOTH_DEVICE(BT_VENDOR_ID_APPLE, USB_DEVICE_ID_APPLE_MAGIC_KEYBOARD_ANSI), + .driver_data = APPLE_HAS_FN }, + { HID_USB_DEVICE(USB_VENDOR_ID_APPLE, USB_DEVICE_ID_APPLE_MAGIC_KEYBOARD_NUMPAD_ANSI), + .driver_data = APPLE_HAS_FN }, + { HID_BLUETOOTH_DEVICE(BT_VENDOR_ID_APPLE, USB_DEVICE_ID_APPLE_MAGIC_KEYBOARD_NUMPAD_ANSI), + .driver_data = APPLE_HAS_FN }, { HID_USB_DEVICE(USB_VENDOR_ID_APPLE, USB_DEVICE_ID_APPLE_WELLSPRING_ANSI), .driver_data = APPLE_HAS_FN }, { HID_USB_DEVICE(USB_VENDOR_ID_APPLE, USB_DEVICE_ID_APPLE_WELLSPRING_ISO), diff --git a/drivers/hid/hid-core.c b/drivers/hid/hid-core.c index 3da354af7a0a..44564f61e9cc 100644 --- a/drivers/hid/hid-core.c +++ b/drivers/hid/hid-core.c @@ -1000,7 +1000,7 @@ int hid_open_report(struct hid_device *device) parser = vzalloc(sizeof(struct hid_parser)); if (!parser) { ret = -ENOMEM; - goto err; + goto alloc_err; } parser->device = device; @@ -1039,6 +1039,7 @@ int hid_open_report(struct hid_device *device) hid_err(device, "unbalanced delimiter at end of report description\n"); goto err; } + kfree(parser->collection_stack); vfree(parser); device->status |= HID_STAT_PARSED; return 0; @@ -1047,6 +1048,8 @@ int hid_open_report(struct hid_device *device) hid_err(device, "item fetching failed at offset %d\n", (int)(end - start)); err: + kfree(parser->collection_stack); +alloc_err: vfree(parser); hid_close_report(device); return ret; diff --git a/drivers/hid/hid-ids.h b/drivers/hid/hid-ids.h index 79bdf0c7e351..5146ee029db4 100644 --- a/drivers/hid/hid-ids.h +++ b/drivers/hid/hid-ids.h @@ -88,6 +88,7 @@ #define USB_DEVICE_ID_ANTON_TOUCH_PAD 0x3101 #define USB_VENDOR_ID_APPLE 0x05ac +#define BT_VENDOR_ID_APPLE 0x004c #define USB_DEVICE_ID_APPLE_MIGHTYMOUSE 0x0304 #define USB_DEVICE_ID_APPLE_MAGICMOUSE 0x030d #define USB_DEVICE_ID_APPLE_MAGICTRACKPAD 0x030e @@ -157,6 +158,7 @@ #define USB_DEVICE_ID_APPLE_ALU_WIRELESS_2011_ISO 0x0256 #define USB_DEVICE_ID_APPLE_ALU_WIRELESS_2011_JIS 0x0257 #define USB_DEVICE_ID_APPLE_MAGIC_KEYBOARD_ANSI 0x0267 +#define USB_DEVICE_ID_APPLE_MAGIC_KEYBOARD_NUMPAD_ANSI 0x026c #define USB_DEVICE_ID_APPLE_WELLSPRING8_ANSI 0x0290 #define USB_DEVICE_ID_APPLE_WELLSPRING8_ISO 0x0291 #define USB_DEVICE_ID_APPLE_WELLSPRING8_JIS 0x0292 @@ -528,9 +530,6 @@ #define I2C_VENDOR_ID_HANTICK 0x0911 #define I2C_PRODUCT_ID_HANTICK_5288 0x5288 -#define I2C_VENDOR_ID_RAYD 0x2386 -#define I2C_PRODUCT_ID_RAYD_3118 0x3118 - #define USB_VENDOR_ID_HANWANG 0x0b57 #define USB_DEVICE_ID_HANWANG_TABLET_FIRST 0x5000 #define USB_DEVICE_ID_HANWANG_TABLET_LAST 0x8fff @@ -950,6 +949,7 @@ #define USB_DEVICE_ID_SAITEK_RUMBLEPAD 0xff17 #define USB_DEVICE_ID_SAITEK_PS1000 0x0621 #define USB_DEVICE_ID_SAITEK_RAT7_OLD 0x0ccb +#define USB_DEVICE_ID_SAITEK_RAT7_CONTAGION 0x0ccd #define USB_DEVICE_ID_SAITEK_RAT7 0x0cd7 #define USB_DEVICE_ID_SAITEK_RAT9 0x0cfa #define USB_DEVICE_ID_SAITEK_MMO7 0x0cd0 diff --git a/drivers/hid/hid-input.c b/drivers/hid/hid-input.c index 4e94ea3e280a..a481eaf39e88 100644 --- a/drivers/hid/hid-input.c +++ b/drivers/hid/hid-input.c @@ -1582,6 +1582,7 @@ static struct hid_input *hidinput_allocate(struct hid_device *hid, input_dev->dev.parent = &hid->dev; hidinput->input = input_dev; + hidinput->application = application; list_add_tail(&hidinput->list, &hid->inputs); INIT_LIST_HEAD(&hidinput->reports); @@ -1677,8 +1678,7 @@ static struct hid_input *hidinput_match_application(struct hid_report *report) struct hid_input *hidinput; list_for_each_entry(hidinput, &hid->inputs, list) { - if (hidinput->report && - hidinput->report->application == report->application) + if (hidinput->application == report->application) return hidinput; } @@ -1815,6 +1815,7 @@ void hidinput_disconnect(struct hid_device *hid) input_unregister_device(hidinput->input); else input_free_device(hidinput->input); + kfree(hidinput->name); kfree(hidinput); } diff --git a/drivers/hid/hid-multitouch.c b/drivers/hid/hid-multitouch.c index 40fbb7c52723..da954f3f4da7 100644 --- a/drivers/hid/hid-multitouch.c +++ b/drivers/hid/hid-multitouch.c @@ -1375,7 +1375,8 @@ static bool mt_need_to_apply_feature(struct hid_device *hdev, struct hid_usage *usage, enum latency_mode latency, bool surface_switch, - bool button_switch) + bool button_switch, + bool *inputmode_found) { struct mt_device *td = hid_get_drvdata(hdev); struct mt_class *cls = &td->mtclass; @@ -1387,6 +1388,14 @@ static bool mt_need_to_apply_feature(struct hid_device *hdev, switch (usage->hid) { case HID_DG_INPUTMODE: + /* + * Some elan panels wrongly declare 2 input mode features, + * and silently ignore when we set the value in the second + * field. Skip the second feature and hope for the best. + */ + if (*inputmode_found) + return false; + if (cls->quirks & MT_QUIRK_FORCE_GET_FEATURE) { report_len = hid_report_len(report); buf = hid_alloc_report_buf(report, GFP_KERNEL); @@ -1402,6 +1411,7 @@ static bool mt_need_to_apply_feature(struct hid_device *hdev, } field->value[index] = td->inputmode_value; + *inputmode_found = true; return true; case HID_DG_CONTACTMAX: @@ -1439,6 +1449,7 @@ static void mt_set_modes(struct hid_device *hdev, enum latency_mode latency, struct hid_usage *usage; int i, j; bool update_report; + bool inputmode_found = false; rep_enum = &hdev->report_enum[HID_FEATURE_REPORT]; list_for_each_entry(rep, &rep_enum->report_list, list) { @@ -1457,7 +1468,8 @@ static void mt_set_modes(struct hid_device *hdev, enum latency_mode latency, usage, latency, surface_switch, - button_switch)) + button_switch, + &inputmode_found)) update_report = true; } } @@ -1685,6 +1697,9 @@ static int mt_probe(struct hid_device *hdev, const struct hid_device_id *id) */ hdev->quirks |= HID_QUIRK_INPUT_PER_APP; + if (id->group != HID_GROUP_MULTITOUCH_WIN_8) + hdev->quirks |= HID_QUIRK_MULTI_INPUT; + timer_setup(&td->release_timer, mt_expired_timeout, 0); ret = hid_parse(hdev); diff --git a/drivers/hid/hid-saitek.c b/drivers/hid/hid-saitek.c index 39e642686ff0..683861f324e3 100644 --- a/drivers/hid/hid-saitek.c +++ b/drivers/hid/hid-saitek.c @@ -183,6 +183,8 @@ static const struct hid_device_id saitek_devices[] = { .driver_data = SAITEK_RELEASE_MODE_RAT7 }, { HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_RAT7), .driver_data = SAITEK_RELEASE_MODE_RAT7 }, + { HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_RAT7_CONTAGION), + .driver_data = SAITEK_RELEASE_MODE_RAT7 }, { HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_RAT9), .driver_data = SAITEK_RELEASE_MODE_RAT7 }, { HID_USB_DEVICE(USB_VENDOR_ID_MADCATZ, USB_DEVICE_ID_MADCATZ_RAT9), diff --git a/drivers/hid/hid-sensor-hub.c b/drivers/hid/hid-sensor-hub.c index 50af72baa5ca..2b63487057c2 100644 --- a/drivers/hid/hid-sensor-hub.c +++ b/drivers/hid/hid-sensor-hub.c @@ -579,6 +579,28 @@ void sensor_hub_device_close(struct hid_sensor_hub_device *hsdev) } EXPORT_SYMBOL_GPL(sensor_hub_device_close); +static __u8 *sensor_hub_report_fixup(struct hid_device *hdev, __u8 *rdesc, + unsigned int *rsize) +{ + /* + * Checks if the report descriptor of Thinkpad Helix 2 has a logical + * minimum for magnetic flux axis greater than the maximum. + */ + if (hdev->product == USB_DEVICE_ID_TEXAS_INSTRUMENTS_LENOVO_YOGA && + *rsize == 2558 && rdesc[913] == 0x17 && rdesc[914] == 0x40 && + rdesc[915] == 0x81 && rdesc[916] == 0x08 && + rdesc[917] == 0x00 && rdesc[918] == 0x27 && + rdesc[921] == 0x07 && rdesc[922] == 0x00) { + /* Sets negative logical minimum for mag x, y and z */ + rdesc[914] = rdesc[935] = rdesc[956] = 0xc0; + rdesc[915] = rdesc[936] = rdesc[957] = 0x7e; + rdesc[916] = rdesc[937] = rdesc[958] = 0xf7; + rdesc[917] = rdesc[938] = rdesc[959] = 0xff; + } + + return rdesc; +} + static int sensor_hub_probe(struct hid_device *hdev, const struct hid_device_id *id) { @@ -743,6 +765,7 @@ static struct hid_driver sensor_hub_driver = { .probe = sensor_hub_probe, .remove = sensor_hub_remove, .raw_event = sensor_hub_raw_event, + .report_fixup = sensor_hub_report_fixup, #ifdef CONFIG_PM .suspend = sensor_hub_suspend, .resume = sensor_hub_resume, diff --git a/drivers/hid/i2c-hid/i2c-hid.c b/drivers/hid/i2c-hid/i2c-hid.c index 2ce194a84868..f3076659361a 100644 --- a/drivers/hid/i2c-hid/i2c-hid.c +++ b/drivers/hid/i2c-hid/i2c-hid.c @@ -170,8 +170,6 @@ static const struct i2c_hid_quirks { I2C_HID_QUIRK_SET_PWR_WAKEUP_DEV }, { I2C_VENDOR_ID_HANTICK, I2C_PRODUCT_ID_HANTICK_5288, I2C_HID_QUIRK_NO_IRQ_AFTER_RESET }, - { I2C_VENDOR_ID_RAYD, I2C_PRODUCT_ID_RAYD_3118, - I2C_HID_QUIRK_RESEND_REPORT_DESCR }, { USB_VENDOR_ID_SIS_TOUCH, USB_DEVICE_ID_SIS10FB_TOUCH, I2C_HID_QUIRK_RESEND_REPORT_DESCR }, { 0, 0 } @@ -1235,11 +1233,16 @@ static int i2c_hid_resume(struct device *dev) pm_runtime_enable(dev); enable_irq(client->irq); - ret = i2c_hid_hwreset(client); + + /* Instead of resetting device, simply powers the device on. This + * solves "incomplete reports" on Raydium devices 2386:3118 and + * 2386:4B33 + */ + ret = i2c_hid_set_power(client, I2C_HID_PWR_ON); if (ret) return ret; - /* RAYDIUM device (2386:3118) need to re-send report descr cmd + /* Some devices need to re-send report descr cmd * after resume, after this it will be back normal. * otherwise it issues too many incomplete reports. */ diff --git a/drivers/hid/intel-ish-hid/ipc/hw-ish.h b/drivers/hid/intel-ish-hid/ipc/hw-ish.h index 97869b7410eb..da133716bed0 100644 --- a/drivers/hid/intel-ish-hid/ipc/hw-ish.h +++ b/drivers/hid/intel-ish-hid/ipc/hw-ish.h @@ -29,6 +29,7 @@ #define CNL_Ax_DEVICE_ID 0x9DFC #define GLK_Ax_DEVICE_ID 0x31A2 #define CNL_H_DEVICE_ID 0xA37C +#define SPT_H_DEVICE_ID 0xA135 #define REVISION_ID_CHT_A0 0x6 #define REVISION_ID_CHT_Ax_SI 0x0 diff --git a/drivers/hid/intel-ish-hid/ipc/pci-ish.c b/drivers/hid/intel-ish-hid/ipc/pci-ish.c index 050f9872f5c0..a1125a5c7965 100644 --- a/drivers/hid/intel-ish-hid/ipc/pci-ish.c +++ b/drivers/hid/intel-ish-hid/ipc/pci-ish.c @@ -38,6 +38,7 @@ static const struct pci_device_id ish_pci_tbl[] = { {PCI_DEVICE(PCI_VENDOR_ID_INTEL, CNL_Ax_DEVICE_ID)}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, GLK_Ax_DEVICE_ID)}, {PCI_DEVICE(PCI_VENDOR_ID_INTEL, CNL_H_DEVICE_ID)}, + {PCI_DEVICE(PCI_VENDOR_ID_INTEL, SPT_H_DEVICE_ID)}, {0, } }; MODULE_DEVICE_TABLE(pci, ish_pci_tbl); diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c index b1b548a21f91..c71cc857b649 100644 --- a/drivers/hv/vmbus_drv.c +++ b/drivers/hv/vmbus_drv.c @@ -1291,6 +1291,9 @@ static ssize_t vmbus_chan_attr_show(struct kobject *kobj, if (!attribute->show) return -EIO; + if (chan->state != CHANNEL_OPENED_STATE) + return -EINVAL; + return attribute->show(chan, buf); } diff --git a/drivers/hwmon/nct6775.c b/drivers/hwmon/nct6775.c index 944f5b63aecd..78603b78cf41 100644 --- a/drivers/hwmon/nct6775.c +++ b/drivers/hwmon/nct6775.c @@ -207,8 +207,6 @@ superio_exit(int ioreg) #define NUM_FAN 7 -#define TEMP_SOURCE_VIRTUAL 0x1f - /* Common and NCT6775 specific data */ /* Voltage min/max registers for nr=7..14 are in bank 5 */ @@ -299,8 +297,9 @@ static const u16 NCT6775_REG_PWM_READ[] = { static const u16 NCT6775_REG_FAN[] = { 0x630, 0x632, 0x634, 0x636, 0x638 }; static const u16 NCT6775_REG_FAN_MIN[] = { 0x3b, 0x3c, 0x3d }; -static const u16 NCT6775_REG_FAN_PULSES[] = { 0x641, 0x642, 0x643, 0x644, 0 }; -static const u16 NCT6775_FAN_PULSE_SHIFT[] = { 0, 0, 0, 0, 0, 0 }; +static const u16 NCT6775_REG_FAN_PULSES[NUM_FAN] = { + 0x641, 0x642, 0x643, 0x644 }; +static const u16 NCT6775_FAN_PULSE_SHIFT[NUM_FAN] = { }; static const u16 NCT6775_REG_TEMP[] = { 0x27, 0x150, 0x250, 0x62b, 0x62c, 0x62d }; @@ -373,6 +372,7 @@ static const char *const nct6775_temp_label[] = { }; #define NCT6775_TEMP_MASK 0x001ffffe +#define NCT6775_VIRT_TEMP_MASK 0x00000000 static const u16 NCT6775_REG_TEMP_ALTERNATE[32] = { [13] = 0x661, @@ -425,8 +425,8 @@ static const u8 NCT6776_PWM_MODE_MASK[] = { 0x01, 0, 0, 0, 0, 0 }; static const u16 NCT6776_REG_FAN_MIN[] = { 0x63a, 0x63c, 0x63e, 0x640, 0x642, 0x64a, 0x64c }; -static const u16 NCT6776_REG_FAN_PULSES[] = { - 0x644, 0x645, 0x646, 0x647, 0x648, 0x649, 0 }; +static const u16 NCT6776_REG_FAN_PULSES[NUM_FAN] = { + 0x644, 0x645, 0x646, 0x647, 0x648, 0x649 }; static const u16 NCT6776_REG_WEIGHT_DUTY_BASE[] = { 0x13e, 0x23e, 0x33e, 0x83e, 0x93e, 0xa3e }; @@ -461,6 +461,7 @@ static const char *const nct6776_temp_label[] = { }; #define NCT6776_TEMP_MASK 0x007ffffe +#define NCT6776_VIRT_TEMP_MASK 0x00000000 static const u16 NCT6776_REG_TEMP_ALTERNATE[32] = { [14] = 0x401, @@ -501,9 +502,9 @@ static const s8 NCT6779_BEEP_BITS[] = { 30, 31 }; /* intrusion0, intrusion1 */ static const u16 NCT6779_REG_FAN[] = { - 0x4b0, 0x4b2, 0x4b4, 0x4b6, 0x4b8, 0x4ba, 0x660 }; -static const u16 NCT6779_REG_FAN_PULSES[] = { - 0x644, 0x645, 0x646, 0x647, 0x648, 0x649, 0 }; + 0x4c0, 0x4c2, 0x4c4, 0x4c6, 0x4c8, 0x4ca, 0x4ce }; +static const u16 NCT6779_REG_FAN_PULSES[NUM_FAN] = { + 0x644, 0x645, 0x646, 0x647, 0x648, 0x649 }; static const u16 NCT6779_REG_CRITICAL_PWM_ENABLE[] = { 0x136, 0x236, 0x336, 0x836, 0x936, 0xa36, 0xb36 }; @@ -559,7 +560,9 @@ static const char *const nct6779_temp_label[] = { }; #define NCT6779_TEMP_MASK 0x07ffff7e +#define NCT6779_VIRT_TEMP_MASK 0x00000000 #define NCT6791_TEMP_MASK 0x87ffff7e +#define NCT6791_VIRT_TEMP_MASK 0x80000000 static const u16 NCT6779_REG_TEMP_ALTERNATE[32] = { 0x490, 0x491, 0x492, 0x493, 0x494, 0x495, 0, 0, @@ -638,6 +641,7 @@ static const char *const nct6792_temp_label[] = { }; #define NCT6792_TEMP_MASK 0x9fffff7e +#define NCT6792_VIRT_TEMP_MASK 0x80000000 static const char *const nct6793_temp_label[] = { "", @@ -675,6 +679,7 @@ static const char *const nct6793_temp_label[] = { }; #define NCT6793_TEMP_MASK 0xbfff037e +#define NCT6793_VIRT_TEMP_MASK 0x80000000 static const char *const nct6795_temp_label[] = { "", @@ -712,6 +717,7 @@ static const char *const nct6795_temp_label[] = { }; #define NCT6795_TEMP_MASK 0xbfffff7e +#define NCT6795_VIRT_TEMP_MASK 0x80000000 static const char *const nct6796_temp_label[] = { "", @@ -724,8 +730,8 @@ static const char *const nct6796_temp_label[] = { "AUXTIN4", "SMBUSMASTER 0", "SMBUSMASTER 1", - "", - "", + "Virtual_TEMP", + "Virtual_TEMP", "", "", "", @@ -748,7 +754,8 @@ static const char *const nct6796_temp_label[] = { "Virtual_TEMP" }; -#define NCT6796_TEMP_MASK 0xbfff03fe +#define NCT6796_TEMP_MASK 0xbfff0ffe +#define NCT6796_VIRT_TEMP_MASK 0x80000c00 /* NCT6102D/NCT6106D specific data */ @@ -779,8 +786,8 @@ static const u16 NCT6106_REG_TEMP_CONFIG[] = { static const u16 NCT6106_REG_FAN[] = { 0x20, 0x22, 0x24 }; static const u16 NCT6106_REG_FAN_MIN[] = { 0xe0, 0xe2, 0xe4 }; -static const u16 NCT6106_REG_FAN_PULSES[] = { 0xf6, 0xf6, 0xf6, 0, 0 }; -static const u16 NCT6106_FAN_PULSE_SHIFT[] = { 0, 2, 4, 0, 0 }; +static const u16 NCT6106_REG_FAN_PULSES[] = { 0xf6, 0xf6, 0xf6 }; +static const u16 NCT6106_FAN_PULSE_SHIFT[] = { 0, 2, 4 }; static const u8 NCT6106_REG_PWM_MODE[] = { 0xf3, 0xf3, 0xf3 }; static const u8 NCT6106_PWM_MODE_MASK[] = { 0x01, 0x02, 0x04 }; @@ -917,6 +924,11 @@ static unsigned int fan_from_reg16(u16 reg, unsigned int divreg) return 1350000U / (reg << divreg); } +static unsigned int fan_from_reg_rpm(u16 reg, unsigned int divreg) +{ + return reg; +} + static u16 fan_to_reg(u32 fan, unsigned int divreg) { if (!fan) @@ -969,6 +981,7 @@ struct nct6775_data { u16 reg_temp_config[NUM_TEMP]; const char * const *temp_label; u32 temp_mask; + u32 virt_temp_mask; u16 REG_CONFIG; u16 REG_VBAT; @@ -1276,11 +1289,11 @@ static bool is_word_sized(struct nct6775_data *data, u16 reg) case nct6795: case nct6796: return reg == 0x150 || reg == 0x153 || reg == 0x155 || - ((reg & 0xfff0) == 0x4b0 && (reg & 0x000f) < 0x0b) || + (reg & 0xfff0) == 0x4c0 || reg == 0x402 || reg == 0x63a || reg == 0x63c || reg == 0x63e || reg == 0x640 || reg == 0x642 || reg == 0x64a || - reg == 0x64c || reg == 0x660 || + reg == 0x64c || reg == 0x73 || reg == 0x75 || reg == 0x77 || reg == 0x79 || reg == 0x7b || reg == 0x7d; } @@ -1558,7 +1571,7 @@ static void nct6775_update_pwm(struct device *dev) reg = nct6775_read_value(data, data->REG_WEIGHT_TEMP_SEL[i]); data->pwm_weight_temp_sel[i] = reg & 0x1f; /* If weight is disabled, report weight source as 0 */ - if (j == 1 && !(reg & 0x80)) + if (!(reg & 0x80)) data->pwm_weight_temp_sel[i] = 0; /* Weight temp data */ @@ -1682,9 +1695,13 @@ static struct nct6775_data *nct6775_update_device(struct device *dev) if (data->has_fan_min & BIT(i)) data->fan_min[i] = nct6775_read_value(data, data->REG_FAN_MIN[i]); - data->fan_pulses[i] = - (nct6775_read_value(data, data->REG_FAN_PULSES[i]) - >> data->FAN_PULSE_SHIFT[i]) & 0x03; + + if (data->REG_FAN_PULSES[i]) { + data->fan_pulses[i] = + (nct6775_read_value(data, + data->REG_FAN_PULSES[i]) + >> data->FAN_PULSE_SHIFT[i]) & 0x03; + } nct6775_select_fan_div(dev, data, i, reg); } @@ -3639,6 +3656,7 @@ static int nct6775_probe(struct platform_device *pdev) data->temp_label = nct6776_temp_label; data->temp_mask = NCT6776_TEMP_MASK; + data->virt_temp_mask = NCT6776_VIRT_TEMP_MASK; data->REG_VBAT = NCT6106_REG_VBAT; data->REG_DIODE = NCT6106_REG_DIODE; @@ -3717,6 +3735,7 @@ static int nct6775_probe(struct platform_device *pdev) data->temp_label = nct6775_temp_label; data->temp_mask = NCT6775_TEMP_MASK; + data->virt_temp_mask = NCT6775_VIRT_TEMP_MASK; data->REG_CONFIG = NCT6775_REG_CONFIG; data->REG_VBAT = NCT6775_REG_VBAT; @@ -3789,6 +3808,7 @@ static int nct6775_probe(struct platform_device *pdev) data->temp_label = nct6776_temp_label; data->temp_mask = NCT6776_TEMP_MASK; + data->virt_temp_mask = NCT6776_VIRT_TEMP_MASK; data->REG_CONFIG = NCT6775_REG_CONFIG; data->REG_VBAT = NCT6775_REG_VBAT; @@ -3853,7 +3873,7 @@ static int nct6775_probe(struct platform_device *pdev) data->ALARM_BITS = NCT6779_ALARM_BITS; data->BEEP_BITS = NCT6779_BEEP_BITS; - data->fan_from_reg = fan_from_reg13; + data->fan_from_reg = fan_from_reg_rpm; data->fan_from_reg_min = fan_from_reg13; data->target_temp_mask = 0xff; data->tolerance_mask = 0x07; @@ -3861,6 +3881,7 @@ static int nct6775_probe(struct platform_device *pdev) data->temp_label = nct6779_temp_label; data->temp_mask = NCT6779_TEMP_MASK; + data->virt_temp_mask = NCT6779_VIRT_TEMP_MASK; data->REG_CONFIG = NCT6775_REG_CONFIG; data->REG_VBAT = NCT6775_REG_VBAT; @@ -3933,7 +3954,7 @@ static int nct6775_probe(struct platform_device *pdev) data->ALARM_BITS = NCT6791_ALARM_BITS; data->BEEP_BITS = NCT6779_BEEP_BITS; - data->fan_from_reg = fan_from_reg13; + data->fan_from_reg = fan_from_reg_rpm; data->fan_from_reg_min = fan_from_reg13; data->target_temp_mask = 0xff; data->tolerance_mask = 0x07; @@ -3944,22 +3965,27 @@ static int nct6775_probe(struct platform_device *pdev) case nct6791: data->temp_label = nct6779_temp_label; data->temp_mask = NCT6791_TEMP_MASK; + data->virt_temp_mask = NCT6791_VIRT_TEMP_MASK; break; case nct6792: data->temp_label = nct6792_temp_label; data->temp_mask = NCT6792_TEMP_MASK; + data->virt_temp_mask = NCT6792_VIRT_TEMP_MASK; break; case nct6793: data->temp_label = nct6793_temp_label; data->temp_mask = NCT6793_TEMP_MASK; + data->virt_temp_mask = NCT6793_VIRT_TEMP_MASK; break; case nct6795: data->temp_label = nct6795_temp_label; data->temp_mask = NCT6795_TEMP_MASK; + data->virt_temp_mask = NCT6795_VIRT_TEMP_MASK; break; case nct6796: data->temp_label = nct6796_temp_label; data->temp_mask = NCT6796_TEMP_MASK; + data->virt_temp_mask = NCT6796_VIRT_TEMP_MASK; break; } @@ -4143,7 +4169,7 @@ static int nct6775_probe(struct platform_device *pdev) * for each fan reflects a different temperature, and there * are no duplicates. */ - if (src != TEMP_SOURCE_VIRTUAL) { + if (!(data->virt_temp_mask & BIT(src))) { if (mask & BIT(src)) continue; mask |= BIT(src); diff --git a/drivers/hwmon/raspberrypi-hwmon.c b/drivers/hwmon/raspberrypi-hwmon.c index fb4e4a6bb1f6..be5ba4690895 100644 --- a/drivers/hwmon/raspberrypi-hwmon.c +++ b/drivers/hwmon/raspberrypi-hwmon.c @@ -164,3 +164,4 @@ module_platform_driver(rpi_hwmon_driver); MODULE_AUTHOR("Stefan Wahren <stefan.wahren@i2se.com>"); MODULE_DESCRIPTION("Raspberry Pi voltage sensor driver"); MODULE_LICENSE("GPL v2"); +MODULE_ALIAS("platform:raspberrypi-hwmon"); diff --git a/drivers/hwtracing/intel_th/core.c b/drivers/hwtracing/intel_th/core.c index da962aa2cef5..fc6b7f8b62fb 100644 --- a/drivers/hwtracing/intel_th/core.c +++ b/drivers/hwtracing/intel_th/core.c @@ -139,7 +139,8 @@ static int intel_th_remove(struct device *dev) th->thdev[i] = NULL; } - th->num_thdevs = lowest; + if (lowest >= 0) + th->num_thdevs = lowest; } if (thdrv->attr_group) @@ -487,7 +488,7 @@ static const struct intel_th_subdevice { .flags = IORESOURCE_MEM, }, { - .start = TH_MMIO_SW, + .start = 1, /* use resource[1] */ .end = 0, .flags = IORESOURCE_MEM, }, @@ -580,6 +581,7 @@ intel_th_subdevice_alloc(struct intel_th *th, struct intel_th_device *thdev; struct resource res[3]; unsigned int req = 0; + bool is64bit = false; int r, err; thdev = intel_th_device_alloc(th, subdev->type, subdev->name, @@ -589,12 +591,18 @@ intel_th_subdevice_alloc(struct intel_th *th, thdev->drvdata = th->drvdata; + for (r = 0; r < th->num_resources; r++) + if (th->resource[r].flags & IORESOURCE_MEM_64) { + is64bit = true; + break; + } + memcpy(res, subdev->res, sizeof(struct resource) * subdev->nres); for (r = 0; r < subdev->nres; r++) { struct resource *devres = th->resource; - int bar = TH_MMIO_CONFIG; + int bar = 0; /* cut subdevices' MMIO from resource[0] */ /* * Take .end == 0 to mean 'take the whole bar', @@ -603,6 +611,8 @@ intel_th_subdevice_alloc(struct intel_th *th, */ if (!res[r].end && res[r].flags == IORESOURCE_MEM) { bar = res[r].start; + if (is64bit) + bar *= 2; res[r].start = 0; res[r].end = resource_size(&devres[bar]) - 1; } diff --git a/drivers/hwtracing/intel_th/pci.c b/drivers/hwtracing/intel_th/pci.c index c2e55e5d97f6..1cf6290d6435 100644 --- a/drivers/hwtracing/intel_th/pci.c +++ b/drivers/hwtracing/intel_th/pci.c @@ -160,6 +160,11 @@ static const struct pci_device_id intel_th_pci_id_table[] = { PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x18e1), .driver_data = (kernel_ulong_t)&intel_th_2x, }, + { + /* Ice Lake PCH */ + PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x34a6), + .driver_data = (kernel_ulong_t)&intel_th_2x, + }, { 0 }, }; diff --git a/drivers/i2c/busses/i2c-i801.c b/drivers/i2c/busses/i2c-i801.c index 04b60a349d7e..c91e145ef5a5 100644 --- a/drivers/i2c/busses/i2c-i801.c +++ b/drivers/i2c/busses/i2c-i801.c @@ -140,6 +140,7 @@ #define SBREG_BAR 0x10 #define SBREG_SMBCTRL 0xc6000c +#define SBREG_SMBCTRL_DNV 0xcf000c /* Host status bits for SMBPCISTS */ #define SMBPCISTS_INTS BIT(3) @@ -1399,7 +1400,11 @@ static void i801_add_tco(struct i801_priv *priv) spin_unlock(&p2sb_spinlock); res = &tco_res[ICH_RES_MEM_OFF]; - res->start = (resource_size_t)base64_addr + SBREG_SMBCTRL; + if (pci_dev->device == PCI_DEVICE_ID_INTEL_DNV_SMBUS) + res->start = (resource_size_t)base64_addr + SBREG_SMBCTRL_DNV; + else + res->start = (resource_size_t)base64_addr + SBREG_SMBCTRL; + res->end = res->start + 3; res->flags = IORESOURCE_MEM; diff --git a/drivers/i2c/busses/i2c-imx-lpi2c.c b/drivers/i2c/busses/i2c-imx-lpi2c.c index 6d975f5221ca..06c4c767af32 100644 --- a/drivers/i2c/busses/i2c-imx-lpi2c.c +++ b/drivers/i2c/busses/i2c-imx-lpi2c.c @@ -538,7 +538,6 @@ static const struct i2c_algorithm lpi2c_imx_algo = { static const struct of_device_id lpi2c_imx_of_match[] = { { .compatible = "fsl,imx7ulp-lpi2c" }, - { .compatible = "fsl,imx8dv-lpi2c" }, { }, }; MODULE_DEVICE_TABLE(of, lpi2c_imx_of_match); diff --git a/drivers/i2c/busses/i2c-uniphier-f.c b/drivers/i2c/busses/i2c-uniphier-f.c index 9918bdd81619..a403e8579b65 100644 --- a/drivers/i2c/busses/i2c-uniphier-f.c +++ b/drivers/i2c/busses/i2c-uniphier-f.c @@ -401,11 +401,8 @@ static int uniphier_fi2c_master_xfer(struct i2c_adapter *adap, return ret; for (msg = msgs; msg < emsg; msg++) { - /* If next message is read, skip the stop condition */ - bool stop = !(msg + 1 < emsg && msg[1].flags & I2C_M_RD); - /* but, force it if I2C_M_STOP is set */ - if (msg->flags & I2C_M_STOP) - stop = true; + /* Emit STOP if it is the last message or I2C_M_STOP is set. */ + bool stop = (msg + 1 == emsg) || (msg->flags & I2C_M_STOP); ret = uniphier_fi2c_master_xfer_one(adap, msg, stop); if (ret) diff --git a/drivers/i2c/busses/i2c-uniphier.c b/drivers/i2c/busses/i2c-uniphier.c index bb181b088291..454f914ae66d 100644 --- a/drivers/i2c/busses/i2c-uniphier.c +++ b/drivers/i2c/busses/i2c-uniphier.c @@ -248,11 +248,8 @@ static int uniphier_i2c_master_xfer(struct i2c_adapter *adap, return ret; for (msg = msgs; msg < emsg; msg++) { - /* If next message is read, skip the stop condition */ - bool stop = !(msg + 1 < emsg && msg[1].flags & I2C_M_RD); - /* but, force it if I2C_M_STOP is set */ - if (msg->flags & I2C_M_STOP) - stop = true; + /* Emit STOP if it is the last message or I2C_M_STOP is set. */ + bool stop = (msg + 1 == emsg) || (msg->flags & I2C_M_STOP); ret = uniphier_i2c_master_xfer_one(adap, msg, stop); if (ret) diff --git a/drivers/i2c/busses/i2c-xiic.c b/drivers/i2c/busses/i2c-xiic.c index 9a71e50d21f1..0c51c0ffdda9 100644 --- a/drivers/i2c/busses/i2c-xiic.c +++ b/drivers/i2c/busses/i2c-xiic.c @@ -532,6 +532,7 @@ static void xiic_start_recv(struct xiic_i2c *i2c) { u8 rx_watermark; struct i2c_msg *msg = i2c->rx_msg = i2c->tx_msg; + unsigned long flags; /* Clear and enable Rx full interrupt. */ xiic_irq_clr_en(i2c, XIIC_INTR_RX_FULL_MASK | XIIC_INTR_TX_ERROR_MASK); @@ -547,6 +548,7 @@ static void xiic_start_recv(struct xiic_i2c *i2c) rx_watermark = IIC_RX_FIFO_DEPTH; xiic_setreg8(i2c, XIIC_RFD_REG_OFFSET, rx_watermark - 1); + local_irq_save(flags); if (!(msg->flags & I2C_M_NOSTART)) /* write the address */ xiic_setreg16(i2c, XIIC_DTR_REG_OFFSET, @@ -556,6 +558,8 @@ static void xiic_start_recv(struct xiic_i2c *i2c) xiic_setreg16(i2c, XIIC_DTR_REG_OFFSET, msg->len | ((i2c->nmsgs == 1) ? XIIC_TX_DYN_STOP_MASK : 0)); + local_irq_restore(flags); + if (i2c->nmsgs == 1) /* very last, enable bus not busy as well */ xiic_irq_clr_en(i2c, XIIC_INTR_BNB_MASK); diff --git a/drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_buffer.c b/drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_buffer.c index 7589f2ad1dae..631360b14ca7 100644 --- a/drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_buffer.c +++ b/drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_buffer.c @@ -187,12 +187,15 @@ static int st_lsm6dsx_set_fifo_odr(struct st_lsm6dsx_sensor *sensor, int st_lsm6dsx_update_watermark(struct st_lsm6dsx_sensor *sensor, u16 watermark) { - u16 fifo_watermark = ~0, cur_watermark, sip = 0, fifo_th_mask; + u16 fifo_watermark = ~0, cur_watermark, fifo_th_mask; struct st_lsm6dsx_hw *hw = sensor->hw; struct st_lsm6dsx_sensor *cur_sensor; int i, err, data; __le16 wdata; + if (!hw->sip) + return 0; + for (i = 0; i < ST_LSM6DSX_ID_MAX; i++) { cur_sensor = iio_priv(hw->iio_devs[i]); @@ -203,14 +206,10 @@ int st_lsm6dsx_update_watermark(struct st_lsm6dsx_sensor *sensor, u16 watermark) : cur_sensor->watermark; fifo_watermark = min_t(u16, fifo_watermark, cur_watermark); - sip += cur_sensor->sip; } - if (!sip) - return 0; - - fifo_watermark = max_t(u16, fifo_watermark, sip); - fifo_watermark = (fifo_watermark / sip) * sip; + fifo_watermark = max_t(u16, fifo_watermark, hw->sip); + fifo_watermark = (fifo_watermark / hw->sip) * hw->sip; fifo_watermark = fifo_watermark * hw->settings->fifo_ops.th_wl; err = regmap_read(hw->regmap, hw->settings->fifo_ops.fifo_th.addr + 1, diff --git a/drivers/iio/temperature/maxim_thermocouple.c b/drivers/iio/temperature/maxim_thermocouple.c index 54e383231d1e..c31b9633f32d 100644 --- a/drivers/iio/temperature/maxim_thermocouple.c +++ b/drivers/iio/temperature/maxim_thermocouple.c @@ -258,7 +258,6 @@ static int maxim_thermocouple_remove(struct spi_device *spi) static const struct spi_device_id maxim_thermocouple_id[] = { {"max6675", MAX6675}, {"max31855", MAX31855}, - {"max31856", MAX31855}, {}, }; MODULE_DEVICE_TABLE(spi, maxim_thermocouple_id); diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c index 0bee1f4b914e..3208ad6ad540 100644 --- a/drivers/infiniband/core/cache.c +++ b/drivers/infiniband/core/cache.c @@ -338,6 +338,39 @@ static int add_roce_gid(struct ib_gid_table_entry *entry) } /** + * del_gid - Delete GID table entry + * + * @ib_dev: IB device whose GID entry to be deleted + * @port: Port number of the IB device + * @table: GID table of the IB device for a port + * @ix: GID entry index to delete + * + */ +static void del_gid(struct ib_device *ib_dev, u8 port, + struct ib_gid_table *table, int ix) +{ + struct ib_gid_table_entry *entry; + + lockdep_assert_held(&table->lock); + + pr_debug("%s device=%s port=%d index=%d gid %pI6\n", __func__, + ib_dev->name, port, ix, + table->data_vec[ix]->attr.gid.raw); + + write_lock_irq(&table->rwlock); + entry = table->data_vec[ix]; + entry->state = GID_TABLE_ENTRY_PENDING_DEL; + /* + * For non RoCE protocol, GID entry slot is ready to use. + */ + if (!rdma_protocol_roce(ib_dev, port)) + table->data_vec[ix] = NULL; + write_unlock_irq(&table->rwlock); + + put_gid_entry_locked(entry); +} + +/** * add_modify_gid - Add or modify GID table entry * * @table: GID table in which GID to be added or modified @@ -358,7 +391,7 @@ static int add_modify_gid(struct ib_gid_table *table, * this index. */ if (is_gid_entry_valid(table->data_vec[attr->index])) - put_gid_entry(table->data_vec[attr->index]); + del_gid(attr->device, attr->port_num, table, attr->index); /* * Some HCA's report multiple GID entries with only one valid GID, and @@ -386,39 +419,6 @@ done: return ret; } -/** - * del_gid - Delete GID table entry - * - * @ib_dev: IB device whose GID entry to be deleted - * @port: Port number of the IB device - * @table: GID table of the IB device for a port - * @ix: GID entry index to delete - * - */ -static void del_gid(struct ib_device *ib_dev, u8 port, - struct ib_gid_table *table, int ix) -{ - struct ib_gid_table_entry *entry; - - lockdep_assert_held(&table->lock); - - pr_debug("%s device=%s port=%d index=%d gid %pI6\n", __func__, - ib_dev->name, port, ix, - table->data_vec[ix]->attr.gid.raw); - - write_lock_irq(&table->rwlock); - entry = table->data_vec[ix]; - entry->state = GID_TABLE_ENTRY_PENDING_DEL; - /* - * For non RoCE protocol, GID entry slot is ready to use. - */ - if (!rdma_protocol_roce(ib_dev, port)) - table->data_vec[ix] = NULL; - write_unlock_irq(&table->rwlock); - - put_gid_entry_locked(entry); -} - /* rwlock should be read locked, or lock should be held */ static int find_gid(struct ib_gid_table *table, const union ib_gid *gid, const struct ib_gid_attr *val, bool default_gid, diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index f72677291b69..a36c94930c31 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -724,6 +724,7 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) dgid = (union ib_gid *) &addr->sib_addr; pkey = ntohs(addr->sib_pkey); + mutex_lock(&lock); list_for_each_entry(cur_dev, &dev_list, list) { for (p = 1; p <= cur_dev->device->phys_port_cnt; ++p) { if (!rdma_cap_af_ib(cur_dev->device, p)) @@ -750,18 +751,19 @@ static int cma_resolve_ib_dev(struct rdma_id_private *id_priv) cma_dev = cur_dev; sgid = gid; id_priv->id.port_num = p; + goto found; } } } } - - if (!cma_dev) - return -ENODEV; + mutex_unlock(&lock); + return -ENODEV; found: cma_attach_to_dev(id_priv, cma_dev); - addr = (struct sockaddr_ib *) cma_src_addr(id_priv); - memcpy(&addr->sib_addr, &sgid, sizeof sgid); + mutex_unlock(&lock); + addr = (struct sockaddr_ib *)cma_src_addr(id_priv); + memcpy(&addr->sib_addr, &sgid, sizeof(sgid)); cma_translate_ib(addr, &id_priv->id.route.addr.dev_addr); return 0; } diff --git a/drivers/infiniband/core/rdma_core.c b/drivers/infiniband/core/rdma_core.c index 6eb64c6f0802..c4118bcd5103 100644 --- a/drivers/infiniband/core/rdma_core.c +++ b/drivers/infiniband/core/rdma_core.c @@ -882,6 +882,8 @@ static int __uverbs_cleanup_ufile(struct ib_uverbs_file *ufile, WARN_ON(uverbs_try_lock_object(obj, UVERBS_LOOKUP_WRITE)); if (!uverbs_destroy_uobject(obj, reason)) ret = 0; + else + atomic_set(&obj->usecnt, 0); } return ret; } diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c index ec8fb289621f..21863ddde63e 100644 --- a/drivers/infiniband/core/ucma.c +++ b/drivers/infiniband/core/ucma.c @@ -124,6 +124,8 @@ static DEFINE_MUTEX(mut); static DEFINE_IDR(ctx_idr); static DEFINE_IDR(multicast_idr); +static const struct file_operations ucma_fops; + static inline struct ucma_context *_ucma_find_context(int id, struct ucma_file *file) { @@ -1581,6 +1583,10 @@ static ssize_t ucma_migrate_id(struct ucma_file *new_file, f = fdget(cmd.fd); if (!f.file) return -ENOENT; + if (f.file->f_op != &ucma_fops) { + ret = -EINVAL; + goto file_put; + } /* Validate current fd and prevent destruction of id. */ ctx = ucma_get_ctx(f.file->private_data, cmd.id); @@ -1753,6 +1759,8 @@ static int ucma_close(struct inode *inode, struct file *filp) mutex_lock(&mut); if (!ctx->closing) { mutex_unlock(&mut); + ucma_put_ctx(ctx); + wait_for_completion(&ctx->comp); /* rdma_destroy_id ensures that no event handlers are * inflight for that id before releasing it. */ diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c index a21d5214afc3..e012ca80f9d1 100644 --- a/drivers/infiniband/core/uverbs_cmd.c +++ b/drivers/infiniband/core/uverbs_cmd.c @@ -2027,33 +2027,55 @@ static int modify_qp(struct ib_uverbs_file *file, if ((cmd->base.attr_mask & IB_QP_CUR_STATE && cmd->base.cur_qp_state > IB_QPS_ERR) || - cmd->base.qp_state > IB_QPS_ERR) { + (cmd->base.attr_mask & IB_QP_STATE && + cmd->base.qp_state > IB_QPS_ERR)) { ret = -EINVAL; goto release_qp; } - attr->qp_state = cmd->base.qp_state; - attr->cur_qp_state = cmd->base.cur_qp_state; - attr->path_mtu = cmd->base.path_mtu; - attr->path_mig_state = cmd->base.path_mig_state; - attr->qkey = cmd->base.qkey; - attr->rq_psn = cmd->base.rq_psn; - attr->sq_psn = cmd->base.sq_psn; - attr->dest_qp_num = cmd->base.dest_qp_num; - attr->qp_access_flags = cmd->base.qp_access_flags; - attr->pkey_index = cmd->base.pkey_index; - attr->alt_pkey_index = cmd->base.alt_pkey_index; - attr->en_sqd_async_notify = cmd->base.en_sqd_async_notify; - attr->max_rd_atomic = cmd->base.max_rd_atomic; - attr->max_dest_rd_atomic = cmd->base.max_dest_rd_atomic; - attr->min_rnr_timer = cmd->base.min_rnr_timer; - attr->port_num = cmd->base.port_num; - attr->timeout = cmd->base.timeout; - attr->retry_cnt = cmd->base.retry_cnt; - attr->rnr_retry = cmd->base.rnr_retry; - attr->alt_port_num = cmd->base.alt_port_num; - attr->alt_timeout = cmd->base.alt_timeout; - attr->rate_limit = cmd->rate_limit; + if (cmd->base.attr_mask & IB_QP_STATE) + attr->qp_state = cmd->base.qp_state; + if (cmd->base.attr_mask & IB_QP_CUR_STATE) + attr->cur_qp_state = cmd->base.cur_qp_state; + if (cmd->base.attr_mask & IB_QP_PATH_MTU) + attr->path_mtu = cmd->base.path_mtu; + if (cmd->base.attr_mask & IB_QP_PATH_MIG_STATE) + attr->path_mig_state = cmd->base.path_mig_state; + if (cmd->base.attr_mask & IB_QP_QKEY) + attr->qkey = cmd->base.qkey; + if (cmd->base.attr_mask & IB_QP_RQ_PSN) + attr->rq_psn = cmd->base.rq_psn; + if (cmd->base.attr_mask & IB_QP_SQ_PSN) + attr->sq_psn = cmd->base.sq_psn; + if (cmd->base.attr_mask & IB_QP_DEST_QPN) + attr->dest_qp_num = cmd->base.dest_qp_num; + if (cmd->base.attr_mask & IB_QP_ACCESS_FLAGS) + attr->qp_access_flags = cmd->base.qp_access_flags; + if (cmd->base.attr_mask & IB_QP_PKEY_INDEX) + attr->pkey_index = cmd->base.pkey_index; + if (cmd->base.attr_mask & IB_QP_EN_SQD_ASYNC_NOTIFY) + attr->en_sqd_async_notify = cmd->base.en_sqd_async_notify; + if (cmd->base.attr_mask & IB_QP_MAX_QP_RD_ATOMIC) + attr->max_rd_atomic = cmd->base.max_rd_atomic; + if (cmd->base.attr_mask & IB_QP_MAX_DEST_RD_ATOMIC) + attr->max_dest_rd_atomic = cmd->base.max_dest_rd_atomic; + if (cmd->base.attr_mask & IB_QP_MIN_RNR_TIMER) + attr->min_rnr_timer = cmd->base.min_rnr_timer; + if (cmd->base.attr_mask & IB_QP_PORT) + attr->port_num = cmd->base.port_num; + if (cmd->base.attr_mask & IB_QP_TIMEOUT) + attr->timeout = cmd->base.timeout; + if (cmd->base.attr_mask & IB_QP_RETRY_CNT) + attr->retry_cnt = cmd->base.retry_cnt; + if (cmd->base.attr_mask & IB_QP_RNR_RETRY) + attr->rnr_retry = cmd->base.rnr_retry; + if (cmd->base.attr_mask & IB_QP_ALT_PATH) { + attr->alt_port_num = cmd->base.alt_port_num; + attr->alt_timeout = cmd->base.alt_timeout; + attr->alt_pkey_index = cmd->base.alt_pkey_index; + } + if (cmd->base.attr_mask & IB_QP_RATE_LIMIT) + attr->rate_limit = cmd->rate_limit; if (cmd->base.attr_mask & IB_QP_AV) copy_ah_attr_from_uverbs(qp->device, &attr->ah_attr, diff --git a/drivers/infiniband/core/uverbs_main.c b/drivers/infiniband/core/uverbs_main.c index 823beca448e1..50152c1b1004 100644 --- a/drivers/infiniband/core/uverbs_main.c +++ b/drivers/infiniband/core/uverbs_main.c @@ -440,6 +440,7 @@ static int ib_uverbs_comp_event_close(struct inode *inode, struct file *filp) list_del(&entry->obj_list); kfree(entry); } + file->ev_queue.is_closed = 1; spin_unlock_irq(&file->ev_queue.lock); uverbs_close_fd(filp); @@ -1050,7 +1051,7 @@ static void ib_uverbs_add_one(struct ib_device *device) uverbs_dev->num_comp_vectors = device->num_comp_vectors; if (ib_uverbs_create_uapi(device, uverbs_dev)) - goto err; + goto err_uapi; cdev_init(&uverbs_dev->cdev, NULL); uverbs_dev->cdev.owner = THIS_MODULE; @@ -1077,11 +1078,10 @@ static void ib_uverbs_add_one(struct ib_device *device) err_class: device_destroy(uverbs_class, uverbs_dev->cdev.dev); - err_cdev: cdev_del(&uverbs_dev->cdev); +err_uapi: clear_bit(devnum, dev_map); - err: if (atomic_dec_and_test(&uverbs_dev->refcount)) ib_uverbs_comp_dev(uverbs_dev); diff --git a/drivers/infiniband/core/uverbs_uapi.c b/drivers/infiniband/core/uverbs_uapi.c index 73ea6f0db88f..be854628a7c6 100644 --- a/drivers/infiniband/core/uverbs_uapi.c +++ b/drivers/infiniband/core/uverbs_uapi.c @@ -248,6 +248,7 @@ void uverbs_destroy_api(struct uverbs_api *uapi) kfree(rcu_dereference_protected(*slot, true)); radix_tree_iter_delete(&uapi->radix, &iter, slot); } + kfree(uapi); } struct uverbs_api *uverbs_alloc_api( diff --git a/drivers/infiniband/hw/bnxt_re/ib_verbs.c b/drivers/infiniband/hw/bnxt_re/ib_verbs.c index bbfb86eb2d24..bc2b9e038439 100644 --- a/drivers/infiniband/hw/bnxt_re/ib_verbs.c +++ b/drivers/infiniband/hw/bnxt_re/ib_verbs.c @@ -833,6 +833,8 @@ int bnxt_re_destroy_qp(struct ib_qp *ib_qp) "Failed to destroy Shadow QP"); return rc; } + bnxt_qplib_free_qp_res(&rdev->qplib_res, + &rdev->qp1_sqp->qplib_qp); mutex_lock(&rdev->qp_lock); list_del(&rdev->qp1_sqp->list); atomic_dec(&rdev->qp_count); diff --git a/drivers/infiniband/hw/bnxt_re/main.c b/drivers/infiniband/hw/bnxt_re/main.c index 20b9f31052bf..85cd1a3593d6 100644 --- a/drivers/infiniband/hw/bnxt_re/main.c +++ b/drivers/infiniband/hw/bnxt_re/main.c @@ -78,7 +78,7 @@ static struct list_head bnxt_re_dev_list = LIST_HEAD_INIT(bnxt_re_dev_list); /* Mutex to protect the list of bnxt_re devices added */ static DEFINE_MUTEX(bnxt_re_dev_lock); static struct workqueue_struct *bnxt_re_wq; -static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait); +static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev); /* SR-IOV helper functions */ @@ -182,7 +182,7 @@ static void bnxt_re_shutdown(void *p) if (!rdev) return; - bnxt_re_ib_unreg(rdev, false); + bnxt_re_ib_unreg(rdev); } static void bnxt_re_stop_irq(void *handle) @@ -251,7 +251,7 @@ static struct bnxt_ulp_ops bnxt_re_ulp_ops = { /* Driver registration routines used to let the networking driver (bnxt_en) * to know that the RoCE driver is now installed */ -static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev, bool lock_wait) +static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev) { struct bnxt_en_dev *en_dev; int rc; @@ -260,14 +260,9 @@ static int bnxt_re_unregister_netdev(struct bnxt_re_dev *rdev, bool lock_wait) return -EINVAL; en_dev = rdev->en_dev; - /* Acquire rtnl lock if it is not invokded from netdev event */ - if (lock_wait) - rtnl_lock(); rc = en_dev->en_ops->bnxt_unregister_device(rdev->en_dev, BNXT_ROCE_ULP); - if (lock_wait) - rtnl_unlock(); return rc; } @@ -281,14 +276,12 @@ static int bnxt_re_register_netdev(struct bnxt_re_dev *rdev) en_dev = rdev->en_dev; - rtnl_lock(); rc = en_dev->en_ops->bnxt_register_device(en_dev, BNXT_ROCE_ULP, &bnxt_re_ulp_ops, rdev); - rtnl_unlock(); return rc; } -static int bnxt_re_free_msix(struct bnxt_re_dev *rdev, bool lock_wait) +static int bnxt_re_free_msix(struct bnxt_re_dev *rdev) { struct bnxt_en_dev *en_dev; int rc; @@ -298,13 +291,9 @@ static int bnxt_re_free_msix(struct bnxt_re_dev *rdev, bool lock_wait) en_dev = rdev->en_dev; - if (lock_wait) - rtnl_lock(); rc = en_dev->en_ops->bnxt_free_msix(rdev->en_dev, BNXT_ROCE_ULP); - if (lock_wait) - rtnl_unlock(); return rc; } @@ -320,7 +309,6 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) num_msix_want = min_t(u32, BNXT_RE_MAX_MSIX, num_online_cpus()); - rtnl_lock(); num_msix_got = en_dev->en_ops->bnxt_request_msix(en_dev, BNXT_ROCE_ULP, rdev->msix_entries, num_msix_want); @@ -335,7 +323,6 @@ static int bnxt_re_request_msix(struct bnxt_re_dev *rdev) } rdev->num_msix = num_msix_got; done: - rtnl_unlock(); return rc; } @@ -358,24 +345,18 @@ static void bnxt_re_fill_fw_msg(struct bnxt_fw_msg *fw_msg, void *msg, fw_msg->timeout = timeout; } -static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id, - bool lock_wait) +static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id) { struct bnxt_en_dev *en_dev = rdev->en_dev; struct hwrm_ring_free_input req = {0}; struct hwrm_ring_free_output resp; struct bnxt_fw_msg fw_msg; - bool do_unlock = false; int rc = -EINVAL; if (!en_dev) return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - if (lock_wait) { - rtnl_lock(); - do_unlock = true; - } bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_RING_FREE, -1, -1); req.ring_type = RING_ALLOC_REQ_RING_TYPE_L2_CMPL; @@ -386,8 +367,6 @@ static int bnxt_re_net_ring_free(struct bnxt_re_dev *rdev, u16 fw_ring_id, if (rc) dev_err(rdev_to_dev(rdev), "Failed to free HW ring:%d :%#x", req.ring_id, rc); - if (do_unlock) - rtnl_unlock(); return rc; } @@ -405,7 +384,6 @@ static int bnxt_re_net_ring_alloc(struct bnxt_re_dev *rdev, dma_addr_t *dma_arr, return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - rtnl_lock(); bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_RING_ALLOC, -1, -1); req.enables = 0; req.page_tbl_addr = cpu_to_le64(dma_arr[0]); @@ -426,27 +404,21 @@ static int bnxt_re_net_ring_alloc(struct bnxt_re_dev *rdev, dma_addr_t *dma_arr, if (!rc) *fw_ring_id = le16_to_cpu(resp.ring_id); - rtnl_unlock(); return rc; } static int bnxt_re_net_stats_ctx_free(struct bnxt_re_dev *rdev, - u32 fw_stats_ctx_id, bool lock_wait) + u32 fw_stats_ctx_id) { struct bnxt_en_dev *en_dev = rdev->en_dev; struct hwrm_stat_ctx_free_input req = {0}; struct bnxt_fw_msg fw_msg; - bool do_unlock = false; int rc = -EINVAL; if (!en_dev) return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - if (lock_wait) { - rtnl_lock(); - do_unlock = true; - } bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_STAT_CTX_FREE, -1, -1); req.stat_ctx_id = cpu_to_le32(fw_stats_ctx_id); @@ -457,8 +429,6 @@ static int bnxt_re_net_stats_ctx_free(struct bnxt_re_dev *rdev, dev_err(rdev_to_dev(rdev), "Failed to free HW stats context %#x", rc); - if (do_unlock) - rtnl_unlock(); return rc; } @@ -478,7 +448,6 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, return rc; memset(&fw_msg, 0, sizeof(fw_msg)); - rtnl_lock(); bnxt_re_init_hwrm_hdr(rdev, (void *)&req, HWRM_STAT_CTX_ALLOC, -1, -1); req.update_period_ms = cpu_to_le32(1000); @@ -490,7 +459,6 @@ static int bnxt_re_net_stats_ctx_alloc(struct bnxt_re_dev *rdev, if (!rc) *fw_stats_ctx_id = le32_to_cpu(resp.stat_ctx_id); - rtnl_unlock(); return rc; } @@ -929,19 +897,19 @@ fail: return rc; } -static void bnxt_re_free_nq_res(struct bnxt_re_dev *rdev, bool lock_wait) +static void bnxt_re_free_nq_res(struct bnxt_re_dev *rdev) { int i; for (i = 0; i < rdev->num_msix - 1; i++) { - bnxt_re_net_ring_free(rdev, rdev->nq[i].ring_id, lock_wait); + bnxt_re_net_ring_free(rdev, rdev->nq[i].ring_id); bnxt_qplib_free_nq(&rdev->nq[i]); } } -static void bnxt_re_free_res(struct bnxt_re_dev *rdev, bool lock_wait) +static void bnxt_re_free_res(struct bnxt_re_dev *rdev) { - bnxt_re_free_nq_res(rdev, lock_wait); + bnxt_re_free_nq_res(rdev); if (rdev->qplib_res.dpi_tbl.max) { bnxt_qplib_dealloc_dpi(&rdev->qplib_res, @@ -1219,7 +1187,7 @@ static int bnxt_re_setup_qos(struct bnxt_re_dev *rdev) return 0; } -static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) +static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev) { int i, rc; @@ -1234,28 +1202,27 @@ static void bnxt_re_ib_unreg(struct bnxt_re_dev *rdev, bool lock_wait) cancel_delayed_work(&rdev->worker); bnxt_re_cleanup_res(rdev); - bnxt_re_free_res(rdev, lock_wait); + bnxt_re_free_res(rdev); if (test_and_clear_bit(BNXT_RE_FLAG_RCFW_CHANNEL_EN, &rdev->flags)) { rc = bnxt_qplib_deinit_rcfw(&rdev->rcfw); if (rc) dev_warn(rdev_to_dev(rdev), "Failed to deinitialize RCFW: %#x", rc); - bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id, - lock_wait); + bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id); bnxt_qplib_free_ctx(rdev->en_dev->pdev, &rdev->qplib_ctx); bnxt_qplib_disable_rcfw_channel(&rdev->rcfw); - bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id, lock_wait); + bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id); bnxt_qplib_free_rcfw_channel(&rdev->rcfw); } if (test_and_clear_bit(BNXT_RE_FLAG_GOT_MSIX, &rdev->flags)) { - rc = bnxt_re_free_msix(rdev, lock_wait); + rc = bnxt_re_free_msix(rdev); if (rc) dev_warn(rdev_to_dev(rdev), "Failed to free MSI-X vectors: %#x", rc); } if (test_and_clear_bit(BNXT_RE_FLAG_NETDEV_REGISTERED, &rdev->flags)) { - rc = bnxt_re_unregister_netdev(rdev, lock_wait); + rc = bnxt_re_unregister_netdev(rdev); if (rc) dev_warn(rdev_to_dev(rdev), "Failed to unregister with netdev: %#x", rc); @@ -1276,6 +1243,12 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) { int i, j, rc; + bool locked; + + /* Acquire rtnl lock through out this function */ + rtnl_lock(); + locked = true; + /* Registered a new RoCE device instance to netdev */ rc = bnxt_re_register_netdev(rdev); if (rc) { @@ -1374,12 +1347,16 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) schedule_delayed_work(&rdev->worker, msecs_to_jiffies(30000)); } + rtnl_unlock(); + locked = false; + /* Register ib dev */ rc = bnxt_re_register_ib(rdev); if (rc) { pr_err("Failed to register with IB: %#x\n", rc); goto fail; } + set_bit(BNXT_RE_FLAG_IBDEV_REGISTERED, &rdev->flags); dev_info(rdev_to_dev(rdev), "Device registered successfully"); for (i = 0; i < ARRAY_SIZE(bnxt_re_attributes); i++) { rc = device_create_file(&rdev->ibdev.dev, @@ -1395,7 +1372,6 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) goto fail; } } - set_bit(BNXT_RE_FLAG_IBDEV_REGISTERED, &rdev->flags); ib_get_eth_speed(&rdev->ibdev, 1, &rdev->active_speed, &rdev->active_width); set_bit(BNXT_RE_FLAG_ISSUE_ROCE_STATS, &rdev->flags); @@ -1404,17 +1380,21 @@ static int bnxt_re_ib_reg(struct bnxt_re_dev *rdev) return 0; free_sctx: - bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id, true); + bnxt_re_net_stats_ctx_free(rdev, rdev->qplib_ctx.stats.fw_id); free_ctx: bnxt_qplib_free_ctx(rdev->en_dev->pdev, &rdev->qplib_ctx); disable_rcfw: bnxt_qplib_disable_rcfw_channel(&rdev->rcfw); free_ring: - bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id, true); + bnxt_re_net_ring_free(rdev, rdev->rcfw.creq_ring_id); free_rcfw: bnxt_qplib_free_rcfw_channel(&rdev->rcfw); fail: - bnxt_re_ib_unreg(rdev, true); + if (!locked) + rtnl_lock(); + bnxt_re_ib_unreg(rdev); + rtnl_unlock(); + return rc; } @@ -1567,7 +1547,7 @@ static int bnxt_re_netdev_event(struct notifier_block *notifier, */ if (atomic_read(&rdev->sched_count) > 0) goto exit; - bnxt_re_ib_unreg(rdev, false); + bnxt_re_ib_unreg(rdev); bnxt_re_remove_one(rdev); bnxt_re_dev_unreg(rdev); break; @@ -1646,7 +1626,10 @@ static void __exit bnxt_re_mod_exit(void) */ flush_workqueue(bnxt_re_wq); bnxt_re_dev_stop(rdev); - bnxt_re_ib_unreg(rdev, true); + /* Acquire the rtnl_lock as the L2 resources are freed here */ + rtnl_lock(); + bnxt_re_ib_unreg(rdev); + rtnl_unlock(); bnxt_re_remove_one(rdev); bnxt_re_dev_unreg(rdev); } diff --git a/drivers/infiniband/hw/bnxt_re/qplib_fp.c b/drivers/infiniband/hw/bnxt_re/qplib_fp.c index e426b990c1dd..6ad0d46ab879 100644 --- a/drivers/infiniband/hw/bnxt_re/qplib_fp.c +++ b/drivers/infiniband/hw/bnxt_re/qplib_fp.c @@ -196,7 +196,7 @@ static int bnxt_qplib_alloc_qp_hdr_buf(struct bnxt_qplib_res *res, struct bnxt_qplib_qp *qp) { struct bnxt_qplib_q *rq = &qp->rq; - struct bnxt_qplib_q *sq = &qp->rq; + struct bnxt_qplib_q *sq = &qp->sq; int rc = 0; if (qp->sq_hdr_buf_size && sq->hwq.max_elements) { diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c index b3203afa3b1d..347fe18b1a41 100644 --- a/drivers/infiniband/hw/cxgb4/qp.c +++ b/drivers/infiniband/hw/cxgb4/qp.c @@ -1685,6 +1685,12 @@ static void flush_qp(struct c4iw_qp *qhp) schp = to_c4iw_cq(qhp->ibqp.send_cq); if (qhp->ibqp.uobject) { + + /* for user qps, qhp->wq.flushed is protected by qhp->mutex */ + if (qhp->wq.flushed) + return; + + qhp->wq.flushed = 1; t4_set_wq_in_error(&qhp->wq, 0); t4_set_cq_in_error(&rchp->cq); spin_lock_irqsave(&rchp->comp_handler_lock, flag); diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c index 2c19bf772451..e1668bcc2d13 100644 --- a/drivers/infiniband/hw/hfi1/chip.c +++ b/drivers/infiniband/hw/hfi1/chip.c @@ -6733,6 +6733,7 @@ void start_freeze_handling(struct hfi1_pportdata *ppd, int flags) struct hfi1_devdata *dd = ppd->dd; struct send_context *sc; int i; + int sc_flags; if (flags & FREEZE_SELF) write_csr(dd, CCE_CTRL, CCE_CTRL_SPC_FREEZE_SMASK); @@ -6743,11 +6744,13 @@ void start_freeze_handling(struct hfi1_pportdata *ppd, int flags) /* notify all SDMA engines that they are going into a freeze */ sdma_freeze_notify(dd, !!(flags & FREEZE_LINK_DOWN)); + sc_flags = SCF_FROZEN | SCF_HALTED | (flags & FREEZE_LINK_DOWN ? + SCF_LINK_DOWN : 0); /* do halt pre-handling on all enabled send contexts */ for (i = 0; i < dd->num_send_contexts; i++) { sc = dd->send_contexts[i].sc; if (sc && (sc->flags & SCF_ENABLED)) - sc_stop(sc, SCF_FROZEN | SCF_HALTED); + sc_stop(sc, sc_flags); } /* Send context are frozen. Notify user space */ @@ -10674,6 +10677,7 @@ int set_link_state(struct hfi1_pportdata *ppd, u32 state) add_rcvctrl(dd, RCV_CTRL_RCV_PORT_ENABLE_SMASK); handle_linkup_change(dd, 1); + pio_kernel_linkup(dd); /* * After link up, a new link width will have been set. diff --git a/drivers/infiniband/hw/hfi1/pcie.c b/drivers/infiniband/hw/hfi1/pcie.c index eec83757d55f..6c967dde58e7 100644 --- a/drivers/infiniband/hw/hfi1/pcie.c +++ b/drivers/infiniband/hw/hfi1/pcie.c @@ -893,14 +893,11 @@ static int trigger_sbr(struct hfi1_devdata *dd) } /* - * A secondary bus reset (SBR) issues a hot reset to our device. - * The following routine does a 1s wait after the reset is dropped - * per PCI Trhfa (recovery time). PCIe 3.0 section 6.6.1 - - * Conventional Reset, paragraph 3, line 35 also says that a 1s - * delay after a reset is required. Per spec requirements, - * the link is either working or not after that point. + * This is an end around to do an SBR during probe time. A new API needs + * to be implemented to have cleaner interface but this fixes the + * current brokenness */ - return pci_reset_bus(dev); + return pci_bridge_secondary_bus_reset(dev->bus->self); } /* diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c index c2c1cba5b23b..752057647f09 100644 --- a/drivers/infiniband/hw/hfi1/pio.c +++ b/drivers/infiniband/hw/hfi1/pio.c @@ -86,6 +86,7 @@ void pio_send_control(struct hfi1_devdata *dd, int op) unsigned long flags; int write = 1; /* write sendctrl back */ int flush = 0; /* re-read sendctrl to make sure it is flushed */ + int i; spin_lock_irqsave(&dd->sendctrl_lock, flags); @@ -95,9 +96,13 @@ void pio_send_control(struct hfi1_devdata *dd, int op) reg |= SEND_CTRL_SEND_ENABLE_SMASK; /* Fall through */ case PSC_DATA_VL_ENABLE: + mask = 0; + for (i = 0; i < ARRAY_SIZE(dd->vld); i++) + if (!dd->vld[i].mtu) + mask |= BIT_ULL(i); /* Disallow sending on VLs not enabled */ - mask = (((~0ull) << num_vls) & SEND_CTRL_UNSUPPORTED_VL_MASK) << - SEND_CTRL_UNSUPPORTED_VL_SHIFT; + mask = (mask & SEND_CTRL_UNSUPPORTED_VL_MASK) << + SEND_CTRL_UNSUPPORTED_VL_SHIFT; reg = (reg & ~SEND_CTRL_UNSUPPORTED_VL_SMASK) | mask; break; case PSC_GLOBAL_DISABLE: @@ -921,20 +926,18 @@ void sc_free(struct send_context *sc) void sc_disable(struct send_context *sc) { u64 reg; - unsigned long flags; struct pio_buf *pbuf; if (!sc) return; /* do all steps, even if already disabled */ - spin_lock_irqsave(&sc->alloc_lock, flags); + spin_lock_irq(&sc->alloc_lock); reg = read_kctxt_csr(sc->dd, sc->hw_context, SC(CTRL)); reg &= ~SC(CTRL_CTXT_ENABLE_SMASK); sc->flags &= ~SCF_ENABLED; sc_wait_for_packet_egress(sc, 1); write_kctxt_csr(sc->dd, sc->hw_context, SC(CTRL), reg); - spin_unlock_irqrestore(&sc->alloc_lock, flags); /* * Flush any waiters. Once the context is disabled, @@ -944,7 +947,7 @@ void sc_disable(struct send_context *sc) * proceed with the flush. */ udelay(1); - spin_lock_irqsave(&sc->release_lock, flags); + spin_lock(&sc->release_lock); if (sc->sr) { /* this context has a shadow ring */ while (sc->sr_tail != sc->sr_head) { pbuf = &sc->sr[sc->sr_tail].pbuf; @@ -955,7 +958,8 @@ void sc_disable(struct send_context *sc) sc->sr_tail = 0; } } - spin_unlock_irqrestore(&sc->release_lock, flags); + spin_unlock(&sc->release_lock); + spin_unlock_irq(&sc->alloc_lock); } /* return SendEgressCtxtStatus.PacketOccupancy */ @@ -1178,11 +1182,39 @@ void pio_kernel_unfreeze(struct hfi1_devdata *dd) sc = dd->send_contexts[i].sc; if (!sc || !(sc->flags & SCF_FROZEN) || sc->type == SC_USER) continue; + if (sc->flags & SCF_LINK_DOWN) + continue; sc_enable(sc); /* will clear the sc frozen flag */ } } +/** + * pio_kernel_linkup() - Re-enable send contexts after linkup event + * @dd: valid devive data + * + * When the link goes down, the freeze path is taken. However, a link down + * event is different from a freeze because if the send context is re-enabled + * whowever is sending data will start sending data again, which will hang + * any QP that is sending data. + * + * The freeze path now looks at the type of event that occurs and takes this + * path for link down event. + */ +void pio_kernel_linkup(struct hfi1_devdata *dd) +{ + struct send_context *sc; + int i; + + for (i = 0; i < dd->num_send_contexts; i++) { + sc = dd->send_contexts[i].sc; + if (!sc || !(sc->flags & SCF_LINK_DOWN) || sc->type == SC_USER) + continue; + + sc_enable(sc); /* will clear the sc link down flag */ + } +} + /* * Wait for the SendPioInitCtxt.PioInitInProgress bit to clear. * Returns: @@ -1382,11 +1414,10 @@ void sc_stop(struct send_context *sc, int flag) { unsigned long flags; - /* mark the context */ - sc->flags |= flag; - /* stop buffer allocations */ spin_lock_irqsave(&sc->alloc_lock, flags); + /* mark the context */ + sc->flags |= flag; sc->flags &= ~SCF_ENABLED; spin_unlock_irqrestore(&sc->alloc_lock, flags); wake_up(&sc->halt_wait); diff --git a/drivers/infiniband/hw/hfi1/pio.h b/drivers/infiniband/hw/hfi1/pio.h index 058b08f459ab..aaf372c3e5d6 100644 --- a/drivers/infiniband/hw/hfi1/pio.h +++ b/drivers/infiniband/hw/hfi1/pio.h @@ -139,6 +139,7 @@ struct send_context { #define SCF_IN_FREE 0x02 #define SCF_HALTED 0x04 #define SCF_FROZEN 0x08 +#define SCF_LINK_DOWN 0x10 struct send_context_info { struct send_context *sc; /* allocated working context */ @@ -306,6 +307,7 @@ void set_pio_integrity(struct send_context *sc); void pio_reset_all(struct hfi1_devdata *dd); void pio_freeze(struct hfi1_devdata *dd); void pio_kernel_unfreeze(struct hfi1_devdata *dd); +void pio_kernel_linkup(struct hfi1_devdata *dd); /* global PIO send control operations */ #define PSC_GLOBAL_ENABLE 0 diff --git a/drivers/infiniband/hw/hfi1/user_sdma.c b/drivers/infiniband/hw/hfi1/user_sdma.c index a3a7b33196d6..5c88706121c1 100644 --- a/drivers/infiniband/hw/hfi1/user_sdma.c +++ b/drivers/infiniband/hw/hfi1/user_sdma.c @@ -828,7 +828,7 @@ static int user_sdma_send_pkts(struct user_sdma_request *req, unsigned maxpkts) if (READ_ONCE(iovec->offset) == iovec->iov.iov_len) { if (++req->iov_idx == req->data_iovs) { ret = -EFAULT; - goto free_txreq; + goto free_tx; } iovec = &req->iovs[req->iov_idx]; WARN_ON(iovec->offset); diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c index 13374c727b14..a7c586a5589d 100644 --- a/drivers/infiniband/hw/hfi1/verbs.c +++ b/drivers/infiniband/hw/hfi1/verbs.c @@ -1582,6 +1582,7 @@ static int hfi1_check_ah(struct ib_device *ibdev, struct rdma_ah_attr *ah_attr) struct hfi1_pportdata *ppd; struct hfi1_devdata *dd; u8 sc5; + u8 sl; if (hfi1_check_mcast(rdma_ah_get_dlid(ah_attr)) && !(rdma_ah_get_ah_flags(ah_attr) & IB_AH_GRH)) @@ -1590,8 +1591,13 @@ static int hfi1_check_ah(struct ib_device *ibdev, struct rdma_ah_attr *ah_attr) /* test the mapping for validity */ ibp = to_iport(ibdev, rdma_ah_get_port_num(ah_attr)); ppd = ppd_from_ibp(ibp); - sc5 = ibp->sl_to_sc[rdma_ah_get_sl(ah_attr)]; dd = dd_from_ppd(ppd); + + sl = rdma_ah_get_sl(ah_attr); + if (sl >= ARRAY_SIZE(ibp->sl_to_sc)) + return -EINVAL; + + sc5 = ibp->sl_to_sc[sl]; if (sc_to_vlt(dd, sc5) > num_vls && sc_to_vlt(dd, sc5) != 0xf) return -EINVAL; return 0; diff --git a/drivers/infiniband/hw/mlx4/main.c b/drivers/infiniband/hw/mlx4/main.c index ca0f1ee26091..0bbeaaae47e0 100644 --- a/drivers/infiniband/hw/mlx4/main.c +++ b/drivers/infiniband/hw/mlx4/main.c @@ -517,9 +517,11 @@ static int mlx4_ib_query_device(struct ib_device *ibdev, props->page_size_cap = dev->dev->caps.page_size_cap; props->max_qp = dev->dev->quotas.qp; props->max_qp_wr = dev->dev->caps.max_wqes - MLX4_IB_SQ_MAX_SPARE; - props->max_send_sge = dev->dev->caps.max_sq_sg; - props->max_recv_sge = dev->dev->caps.max_rq_sg; - props->max_sge_rd = MLX4_MAX_SGE_RD; + props->max_send_sge = + min(dev->dev->caps.max_sq_sg, dev->dev->caps.max_rq_sg); + props->max_recv_sge = + min(dev->dev->caps.max_sq_sg, dev->dev->caps.max_rq_sg); + props->max_sge_rd = MLX4_MAX_SGE_RD; props->max_cq = dev->dev->quotas.cq; props->max_cqe = dev->dev->caps.max_cqes; props->max_mr = dev->dev->quotas.mpt; diff --git a/drivers/infiniband/hw/mlx5/devx.c b/drivers/infiniband/hw/mlx5/devx.c index ac116d63e466..f2f11e652dcd 100644 --- a/drivers/infiniband/hw/mlx5/devx.c +++ b/drivers/infiniband/hw/mlx5/devx.c @@ -723,6 +723,7 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_DEVX_OBJ_CREATE)( attrs, MLX5_IB_ATTR_DEVX_OBJ_CREATE_HANDLE); struct mlx5_ib_ucontext *c = to_mucontext(uobj->context); struct mlx5_ib_dev *dev = to_mdev(c->ibucontext.device); + u32 out[MLX5_ST_SZ_DW(general_obj_out_cmd_hdr)]; struct devx_obj *obj; int err; @@ -754,10 +755,12 @@ static int UVERBS_HANDLER(MLX5_IB_METHOD_DEVX_OBJ_CREATE)( err = uverbs_copy_to(attrs, MLX5_IB_ATTR_DEVX_OBJ_CREATE_CMD_OUT, cmd_out, cmd_out_len); if (err) - goto obj_free; + goto obj_destroy; return 0; +obj_destroy: + mlx5_cmd_exec(obj->mdev, obj->dinbox, obj->dinlen, out, sizeof(out)); obj_free: kfree(obj); return err; diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c index ea01b8dd2be6..3d5424f335cb 100644 --- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c +++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c @@ -1027,12 +1027,14 @@ static int ipoib_cm_rep_handler(struct ib_cm_id *cm_id, skb_queue_head_init(&skqueue); + netif_tx_lock_bh(p->dev); spin_lock_irq(&priv->lock); set_bit(IPOIB_FLAG_OPER_UP, &p->flags); if (p->neigh) while ((skb = __skb_dequeue(&p->neigh->queue))) __skb_queue_tail(&skqueue, skb); spin_unlock_irq(&priv->lock); + netif_tx_unlock_bh(p->dev); while ((skb = __skb_dequeue(&skqueue))) { skb->dev = p->dev; diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c index 444d16520506..0b34e909505f 100644 --- a/drivers/infiniband/ulp/srp/ib_srp.c +++ b/drivers/infiniband/ulp/srp/ib_srp.c @@ -2951,7 +2951,7 @@ static int srp_reset_device(struct scsi_cmnd *scmnd) { struct srp_target_port *target = host_to_target(scmnd->device->host); struct srp_rdma_ch *ch; - int i; + int i, j; u8 status; shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n"); @@ -2965,8 +2965,8 @@ static int srp_reset_device(struct scsi_cmnd *scmnd) for (i = 0; i < target->ch_count; i++) { ch = &target->ch[i]; - for (i = 0; i < target->req_ring_size; ++i) { - struct srp_request *req = &ch->req_ring[i]; + for (j = 0; j < target->req_ring_size; ++j) { + struct srp_request *req = &ch->req_ring[j]; srp_finish_req(ch, req, scmnd->device, DID_RESET << 16); } diff --git a/drivers/input/keyboard/atakbd.c b/drivers/input/keyboard/atakbd.c index 6f62da2909ec..6caee807cafa 100644 --- a/drivers/input/keyboard/atakbd.c +++ b/drivers/input/keyboard/atakbd.c @@ -75,8 +75,7 @@ MODULE_LICENSE("GPL"); */ -static unsigned char atakbd_keycode[0x72] = { /* American layout */ - [0] = KEY_GRAVE, +static unsigned char atakbd_keycode[0x73] = { /* American layout */ [1] = KEY_ESC, [2] = KEY_1, [3] = KEY_2, @@ -117,9 +116,9 @@ static unsigned char atakbd_keycode[0x72] = { /* American layout */ [38] = KEY_L, [39] = KEY_SEMICOLON, [40] = KEY_APOSTROPHE, - [41] = KEY_BACKSLASH, /* FIXME, '#' */ + [41] = KEY_GRAVE, [42] = KEY_LEFTSHIFT, - [43] = KEY_GRAVE, /* FIXME: '~' */ + [43] = KEY_BACKSLASH, [44] = KEY_Z, [45] = KEY_X, [46] = KEY_C, @@ -145,45 +144,34 @@ static unsigned char atakbd_keycode[0x72] = { /* American layout */ [66] = KEY_F8, [67] = KEY_F9, [68] = KEY_F10, - [69] = KEY_ESC, - [70] = KEY_DELETE, - [71] = KEY_KP7, - [72] = KEY_KP8, - [73] = KEY_KP9, + [71] = KEY_HOME, + [72] = KEY_UP, [74] = KEY_KPMINUS, - [75] = KEY_KP4, - [76] = KEY_KP5, - [77] = KEY_KP6, + [75] = KEY_LEFT, + [77] = KEY_RIGHT, [78] = KEY_KPPLUS, - [79] = KEY_KP1, - [80] = KEY_KP2, - [81] = KEY_KP3, - [82] = KEY_KP0, - [83] = KEY_KPDOT, - [90] = KEY_KPLEFTPAREN, - [91] = KEY_KPRIGHTPAREN, - [92] = KEY_KPASTERISK, /* FIXME */ - [93] = KEY_KPASTERISK, - [94] = KEY_KPPLUS, - [95] = KEY_HELP, + [80] = KEY_DOWN, + [82] = KEY_INSERT, + [83] = KEY_DELETE, [96] = KEY_102ND, - [97] = KEY_KPASTERISK, /* FIXME */ - [98] = KEY_KPSLASH, + [97] = KEY_UNDO, + [98] = KEY_HELP, [99] = KEY_KPLEFTPAREN, [100] = KEY_KPRIGHTPAREN, [101] = KEY_KPSLASH, [102] = KEY_KPASTERISK, - [103] = KEY_UP, - [104] = KEY_KPASTERISK, /* FIXME */ - [105] = KEY_LEFT, - [106] = KEY_RIGHT, - [107] = KEY_KPASTERISK, /* FIXME */ - [108] = KEY_DOWN, - [109] = KEY_KPASTERISK, /* FIXME */ - [110] = KEY_KPASTERISK, /* FIXME */ - [111] = KEY_KPASTERISK, /* FIXME */ - [112] = KEY_KPASTERISK, /* FIXME */ - [113] = KEY_KPASTERISK /* FIXME */ + [103] = KEY_KP7, + [104] = KEY_KP8, + [105] = KEY_KP9, + [106] = KEY_KP4, + [107] = KEY_KP5, + [108] = KEY_KP6, + [109] = KEY_KP1, + [110] = KEY_KP2, + [111] = KEY_KP3, + [112] = KEY_KP0, + [113] = KEY_KPDOT, + [114] = KEY_KPENTER, }; static struct input_dev *atakbd_dev; @@ -191,21 +179,15 @@ static struct input_dev *atakbd_dev; static void atakbd_interrupt(unsigned char scancode, char down) { - if (scancode < 0x72) { /* scancodes < 0xf2 are keys */ + if (scancode < 0x73) { /* scancodes < 0xf3 are keys */ // report raw events here? scancode = atakbd_keycode[scancode]; - if (scancode == KEY_CAPSLOCK) { /* CapsLock is a toggle switch key on Amiga */ - input_report_key(atakbd_dev, scancode, 1); - input_report_key(atakbd_dev, scancode, 0); - input_sync(atakbd_dev); - } else { - input_report_key(atakbd_dev, scancode, down); - input_sync(atakbd_dev); - } - } else /* scancodes >= 0xf2 are mouse data, most likely */ + input_report_key(atakbd_dev, scancode, down); + input_sync(atakbd_dev); + } else /* scancodes >= 0xf3 are mouse data, most likely */ printk(KERN_INFO "atakbd: unhandled scancode %x\n", scancode); return; diff --git a/drivers/input/misc/uinput.c b/drivers/input/misc/uinput.c index 96a887f33698..eb14ddf69346 100644 --- a/drivers/input/misc/uinput.c +++ b/drivers/input/misc/uinput.c @@ -410,7 +410,7 @@ static int uinput_validate_absinfo(struct input_dev *dev, unsigned int code, min = abs->minimum; max = abs->maximum; - if ((min != 0 || max != 0) && max <= min) { + if ((min != 0 || max != 0) && max < min) { printk(KERN_DEBUG "%s: invalid abs[%02x] min:%d max:%d\n", UINPUT_NAME, code, min, max); diff --git a/drivers/input/mouse/elantech.c b/drivers/input/mouse/elantech.c index 44f57cf6675b..2d95e8d93cc7 100644 --- a/drivers/input/mouse/elantech.c +++ b/drivers/input/mouse/elantech.c @@ -1178,6 +1178,8 @@ static const struct dmi_system_id elantech_dmi_has_middle_button[] = { static const char * const middle_button_pnp_ids[] = { "LEN2131", /* ThinkPad P52 w/ NFC */ "LEN2132", /* ThinkPad P52 */ + "LEN2133", /* ThinkPad P72 w/ NFC */ + "LEN2134", /* ThinkPad P72 */ NULL }; diff --git a/drivers/input/touchscreen/egalax_ts.c b/drivers/input/touchscreen/egalax_ts.c index 80e69bb8283e..83ac8c128192 100644 --- a/drivers/input/touchscreen/egalax_ts.c +++ b/drivers/input/touchscreen/egalax_ts.c @@ -241,6 +241,9 @@ static int __maybe_unused egalax_ts_suspend(struct device *dev) struct i2c_client *client = to_i2c_client(dev); int ret; + if (device_may_wakeup(dev)) + return enable_irq_wake(client->irq); + ret = i2c_master_send(client, suspend_cmd, MAX_I2C_DATA_LEN); return ret > 0 ? 0 : ret; } @@ -249,6 +252,9 @@ static int __maybe_unused egalax_ts_resume(struct device *dev) { struct i2c_client *client = to_i2c_client(dev); + if (device_may_wakeup(dev)) + return disable_irq_wake(client->irq); + return egalax_wake_up_device(client); } diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 4e04fff23977..73e47d93e7a0 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -246,7 +246,13 @@ static u16 get_alias(struct device *dev) /* The callers make sure that get_device_id() does not fail here */ devid = get_device_id(dev); + + /* For ACPI HID devices, we simply return the devid as such */ + if (!dev_is_pci(dev)) + return devid; + ivrs_alias = amd_iommu_alias_table[devid]; + pci_for_each_dma_alias(pdev, __last_alias, &pci_alias); if (ivrs_alias == pci_alias) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 5f3f10cf9d9d..bedc801b06a0 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -2540,9 +2540,9 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu, if (dev && dev_is_pci(dev) && info->pasid_supported) { ret = intel_pasid_alloc_table(dev); if (ret) { - __dmar_remove_one_dev_info(info); - spin_unlock_irqrestore(&device_domain_lock, flags); - return NULL; + pr_warn("No pasid table for %s, pasid disabled\n", + dev_name(dev)); + info->pasid_supported = 0; } } spin_unlock_irqrestore(&device_domain_lock, flags); diff --git a/drivers/iommu/intel-pasid.h b/drivers/iommu/intel-pasid.h index 1c05ed6fc5a5..1fb5e12b029a 100644 --- a/drivers/iommu/intel-pasid.h +++ b/drivers/iommu/intel-pasid.h @@ -11,7 +11,7 @@ #define __INTEL_PASID_H #define PASID_MIN 0x1 -#define PASID_MAX 0x100000 +#define PASID_MAX 0x20000 struct pasid_entry { u64 val; diff --git a/drivers/iommu/rockchip-iommu.c b/drivers/iommu/rockchip-iommu.c index 258115b10fa9..ad3e2b97469e 100644 --- a/drivers/iommu/rockchip-iommu.c +++ b/drivers/iommu/rockchip-iommu.c @@ -1241,6 +1241,12 @@ err_unprepare_clocks: static void rk_iommu_shutdown(struct platform_device *pdev) { + struct rk_iommu *iommu = platform_get_drvdata(pdev); + int i = 0, irq; + + while ((irq = platform_get_irq(pdev, i++)) != -ENXIO) + devm_free_irq(iommu->dev, irq, iommu); + pm_runtime_force_suspend(&pdev->dev); } diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index 316a57530f6d..c2df341ff6fa 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -1439,6 +1439,7 @@ static struct irq_chip its_irq_chip = { * The consequence of the above is that allocation is cost is low, but * freeing is expensive. We assumes that freeing rarely occurs. */ +#define ITS_MAX_LPI_NRBITS 16 /* 64K LPIs */ static DEFINE_MUTEX(lpi_range_lock); static LIST_HEAD(lpi_range_list); @@ -1625,7 +1626,8 @@ static int __init its_alloc_lpi_tables(void) { phys_addr_t paddr; - lpi_id_bits = GICD_TYPER_ID_BITS(gic_rdists->gicd_typer); + lpi_id_bits = min_t(u32, GICD_TYPER_ID_BITS(gic_rdists->gicd_typer), + ITS_MAX_LPI_NRBITS); gic_rdists->prop_page = its_allocate_prop_table(GFP_NOWAIT); if (!gic_rdists->prop_page) { pr_err("Failed to allocate PROPBASE\n"); diff --git a/drivers/md/bcache/bcache.h b/drivers/md/bcache/bcache.h index 83504dd8100a..954dad29e6e8 100644 --- a/drivers/md/bcache/bcache.h +++ b/drivers/md/bcache/bcache.h @@ -965,6 +965,7 @@ void bch_prio_write(struct cache *ca); void bch_write_bdev_super(struct cached_dev *dc, struct closure *parent); extern struct workqueue_struct *bcache_wq; +extern struct workqueue_struct *bch_journal_wq; extern struct mutex bch_register_lock; extern struct list_head bch_cache_sets; diff --git a/drivers/md/bcache/journal.c b/drivers/md/bcache/journal.c index 6116bbf870d8..522c7426f3a0 100644 --- a/drivers/md/bcache/journal.c +++ b/drivers/md/bcache/journal.c @@ -485,7 +485,7 @@ static void do_journal_discard(struct cache *ca) closure_get(&ca->set->cl); INIT_WORK(&ja->discard_work, journal_discard_work); - schedule_work(&ja->discard_work); + queue_work(bch_journal_wq, &ja->discard_work); } } @@ -592,7 +592,7 @@ static void journal_write_done(struct closure *cl) : &j->w[0]; __closure_wake_up(&w->wait); - continue_at_nobarrier(cl, journal_write, system_wq); + continue_at_nobarrier(cl, journal_write, bch_journal_wq); } static void journal_write_unlock(struct closure *cl) @@ -627,7 +627,7 @@ static void journal_write_unlocked(struct closure *cl) spin_unlock(&c->journal.lock); btree_flush_write(c); - continue_at(cl, journal_write, system_wq); + continue_at(cl, journal_write, bch_journal_wq); return; } diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c index 94c756c66bd7..30ba9aeb5ee8 100644 --- a/drivers/md/bcache/super.c +++ b/drivers/md/bcache/super.c @@ -47,6 +47,7 @@ static int bcache_major; static DEFINE_IDA(bcache_device_idx); static wait_queue_head_t unregister_wait; struct workqueue_struct *bcache_wq; +struct workqueue_struct *bch_journal_wq; #define BTREE_MAX_PAGES (256 * 1024 / PAGE_SIZE) /* limitation of partitions number on single bcache device */ @@ -2341,6 +2342,9 @@ static void bcache_exit(void) kobject_put(bcache_kobj); if (bcache_wq) destroy_workqueue(bcache_wq); + if (bch_journal_wq) + destroy_workqueue(bch_journal_wq); + if (bcache_major) unregister_blkdev(bcache_major, "bcache"); unregister_reboot_notifier(&reboot); @@ -2370,6 +2374,10 @@ static int __init bcache_init(void) if (!bcache_wq) goto err; + bch_journal_wq = alloc_workqueue("bch_journal", WQ_MEM_RECLAIM, 0); + if (!bch_journal_wq) + goto err; + bcache_kobj = kobject_create_and_add("bcache", fs_kobj); if (!bcache_kobj) goto err; diff --git a/drivers/md/dm-crypt.c b/drivers/md/dm-crypt.c index f266c81f396f..0481223b1deb 100644 --- a/drivers/md/dm-crypt.c +++ b/drivers/md/dm-crypt.c @@ -332,7 +332,7 @@ static int crypt_iv_essiv_init(struct crypt_config *cc) int err; desc->tfm = essiv->hash_tfm; - desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP; + desc->flags = 0; err = crypto_shash_digest(desc, cc->key, cc->key_size, essiv->salt); shash_desc_zero(desc); @@ -606,7 +606,7 @@ static int crypt_iv_lmk_one(struct crypt_config *cc, u8 *iv, int i, r; desc->tfm = lmk->hash_tfm; - desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP; + desc->flags = 0; r = crypto_shash_init(desc); if (r) @@ -768,7 +768,7 @@ static int crypt_iv_tcw_whitening(struct crypt_config *cc, /* calculate crc32 for every 32bit part and xor it */ desc->tfm = tcw->crc32_tfm; - desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP; + desc->flags = 0; for (i = 0; i < 4; i++) { r = crypto_shash_init(desc); if (r) @@ -1251,7 +1251,7 @@ static void crypt_alloc_req_skcipher(struct crypt_config *cc, * requests if driver request queue is full. */ skcipher_request_set_callback(ctx->r.req, - CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP, + CRYPTO_TFM_REQ_MAY_BACKLOG, kcryptd_async_done, dmreq_of_req(cc, ctx->r.req)); } @@ -1268,7 +1268,7 @@ static void crypt_alloc_req_aead(struct crypt_config *cc, * requests if driver request queue is full. */ aead_request_set_callback(ctx->r.req_aead, - CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP, + CRYPTO_TFM_REQ_MAY_BACKLOG, kcryptd_async_done, dmreq_of_req(cc, ctx->r.req_aead)); } diff --git a/drivers/md/dm-integrity.c b/drivers/md/dm-integrity.c index 378878599466..89ccb64342de 100644 --- a/drivers/md/dm-integrity.c +++ b/drivers/md/dm-integrity.c @@ -532,7 +532,7 @@ static void section_mac(struct dm_integrity_c *ic, unsigned section, __u8 result unsigned j, size; desc->tfm = ic->journal_mac; - desc->flags = CRYPTO_TFM_REQ_MAY_SLEEP; + desc->flags = 0; r = crypto_shash_init(desc); if (unlikely(r)) { @@ -676,7 +676,7 @@ static void complete_journal_encrypt(struct crypto_async_request *req, int err) static bool do_crypt(bool encrypt, struct skcipher_request *req, struct journal_completion *comp) { int r; - skcipher_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG | CRYPTO_TFM_REQ_MAY_SLEEP, + skcipher_request_set_callback(req, CRYPTO_TFM_REQ_MAY_BACKLOG, complete_journal_encrypt, comp); if (likely(encrypt)) r = crypto_skcipher_encrypt(req); diff --git a/drivers/md/dm-raid.c b/drivers/md/dm-raid.c index cae689de75fd..5ba067fa0c72 100644 --- a/drivers/md/dm-raid.c +++ b/drivers/md/dm-raid.c @@ -1,6 +1,6 @@ /* * Copyright (C) 2010-2011 Neil Brown - * Copyright (C) 2010-2017 Red Hat, Inc. All rights reserved. + * Copyright (C) 2010-2018 Red Hat, Inc. All rights reserved. * * This file is released under the GPL. */ @@ -29,9 +29,6 @@ */ #define MIN_RAID456_JOURNAL_SPACE (4*2048) -/* Global list of all raid sets */ -static LIST_HEAD(raid_sets); - static bool devices_handle_discard_safely = false; /* @@ -227,7 +224,6 @@ struct rs_layout { struct raid_set { struct dm_target *ti; - struct list_head list; uint32_t stripe_cache_entries; unsigned long ctr_flags; @@ -273,19 +269,6 @@ static void rs_config_restore(struct raid_set *rs, struct rs_layout *l) mddev->new_chunk_sectors = l->new_chunk_sectors; } -/* Find any raid_set in active slot for @rs on global list */ -static struct raid_set *rs_find_active(struct raid_set *rs) -{ - struct raid_set *r; - struct mapped_device *md = dm_table_get_md(rs->ti->table); - - list_for_each_entry(r, &raid_sets, list) - if (r != rs && dm_table_get_md(r->ti->table) == md) - return r; - - return NULL; -} - /* raid10 algorithms (i.e. formats) */ #define ALGORITHM_RAID10_DEFAULT 0 #define ALGORITHM_RAID10_NEAR 1 @@ -764,7 +747,6 @@ static struct raid_set *raid_set_alloc(struct dm_target *ti, struct raid_type *r mddev_init(&rs->md); - INIT_LIST_HEAD(&rs->list); rs->raid_disks = raid_devs; rs->delta_disks = 0; @@ -782,9 +764,6 @@ static struct raid_set *raid_set_alloc(struct dm_target *ti, struct raid_type *r for (i = 0; i < raid_devs; i++) md_rdev_init(&rs->dev[i].rdev); - /* Add @rs to global list. */ - list_add(&rs->list, &raid_sets); - /* * Remaining items to be initialized by further RAID params: * rs->md.persistent @@ -797,7 +776,7 @@ static struct raid_set *raid_set_alloc(struct dm_target *ti, struct raid_type *r return rs; } -/* Free all @rs allocations and remove it from global list. */ +/* Free all @rs allocations */ static void raid_set_free(struct raid_set *rs) { int i; @@ -815,8 +794,6 @@ static void raid_set_free(struct raid_set *rs) dm_put_device(rs->ti, rs->dev[i].data_dev); } - list_del(&rs->list); - kfree(rs); } @@ -2649,7 +2626,7 @@ static int rs_adjust_data_offsets(struct raid_set *rs) return 0; } - /* HM FIXME: get InSync raid_dev? */ + /* HM FIXME: get In_Sync raid_dev? */ rdev = &rs->dev[0].rdev; if (rs->delta_disks < 0) { @@ -3149,6 +3126,11 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv) set_bit(RT_FLAG_UPDATE_SBS, &rs->runtime_flags); rs_set_new(rs); } else if (rs_is_recovering(rs)) { + /* Rebuild particular devices */ + if (test_bit(__CTR_FLAG_REBUILD, &rs->ctr_flags)) { + set_bit(RT_FLAG_UPDATE_SBS, &rs->runtime_flags); + rs_setup_recovery(rs, MaxSector); + } /* A recovering raid set may be resized */ ; /* skip setup rs */ } else if (rs_is_reshaping(rs)) { @@ -3242,6 +3224,8 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv) /* Start raid set read-only and assumed clean to change in raid_resume() */ rs->md.ro = 1; rs->md.in_sync = 1; + + /* Keep array frozen */ set_bit(MD_RECOVERY_FROZEN, &rs->md.recovery); /* Has to be held on running the array */ @@ -3265,7 +3249,7 @@ static int raid_ctr(struct dm_target *ti, unsigned int argc, char **argv) rs->callbacks.congested_fn = raid_is_congested; dm_table_add_target_callbacks(ti->table, &rs->callbacks); - /* If raid4/5/6 journal mode explictely requested (only possible with journal dev) -> set it */ + /* If raid4/5/6 journal mode explicitly requested (only possible with journal dev) -> set it */ if (test_bit(__CTR_FLAG_JOURNAL_MODE, &rs->ctr_flags)) { r = r5c_journal_mode_set(&rs->md, rs->journal_dev.mode); if (r) { @@ -3350,32 +3334,53 @@ static int raid_map(struct dm_target *ti, struct bio *bio) return DM_MAPIO_SUBMITTED; } -/* Return string describing the current sync action of @mddev */ -static const char *decipher_sync_action(struct mddev *mddev, unsigned long recovery) +/* Return sync state string for @state */ +enum sync_state { st_frozen, st_reshape, st_resync, st_check, st_repair, st_recover, st_idle }; +static const char *sync_str(enum sync_state state) +{ + /* Has to be in above sync_state order! */ + static const char *sync_strs[] = { + "frozen", + "reshape", + "resync", + "check", + "repair", + "recover", + "idle" + }; + + return __within_range(state, 0, ARRAY_SIZE(sync_strs) - 1) ? sync_strs[state] : "undef"; +}; + +/* Return enum sync_state for @mddev derived from @recovery flags */ +static const enum sync_state decipher_sync_action(struct mddev *mddev, unsigned long recovery) { if (test_bit(MD_RECOVERY_FROZEN, &recovery)) - return "frozen"; + return st_frozen; - /* The MD sync thread can be done with io but still be running */ + /* The MD sync thread can be done with io or be interrupted but still be running */ if (!test_bit(MD_RECOVERY_DONE, &recovery) && (test_bit(MD_RECOVERY_RUNNING, &recovery) || (!mddev->ro && test_bit(MD_RECOVERY_NEEDED, &recovery)))) { if (test_bit(MD_RECOVERY_RESHAPE, &recovery)) - return "reshape"; + return st_reshape; if (test_bit(MD_RECOVERY_SYNC, &recovery)) { if (!test_bit(MD_RECOVERY_REQUESTED, &recovery)) - return "resync"; - else if (test_bit(MD_RECOVERY_CHECK, &recovery)) - return "check"; - return "repair"; + return st_resync; + if (test_bit(MD_RECOVERY_CHECK, &recovery)) + return st_check; + return st_repair; } if (test_bit(MD_RECOVERY_RECOVER, &recovery)) - return "recover"; + return st_recover; + + if (mddev->reshape_position != MaxSector) + return st_reshape; } - return "idle"; + return st_idle; } /* @@ -3409,6 +3414,7 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery, sector_t resync_max_sectors) { sector_t r; + enum sync_state state; struct mddev *mddev = &rs->md; clear_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags); @@ -3419,20 +3425,14 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery, set_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags); } else { - if (!test_bit(__CTR_FLAG_NOSYNC, &rs->ctr_flags) && - !test_bit(MD_RECOVERY_INTR, &recovery) && - (test_bit(MD_RECOVERY_NEEDED, &recovery) || - test_bit(MD_RECOVERY_RESHAPE, &recovery) || - test_bit(MD_RECOVERY_RUNNING, &recovery))) - r = mddev->curr_resync_completed; - else + state = decipher_sync_action(mddev, recovery); + + if (state == st_idle && !test_bit(MD_RECOVERY_INTR, &recovery)) r = mddev->recovery_cp; + else + r = mddev->curr_resync_completed; - if (r >= resync_max_sectors && - (!test_bit(MD_RECOVERY_REQUESTED, &recovery) || - (!test_bit(MD_RECOVERY_FROZEN, &recovery) && - !test_bit(MD_RECOVERY_NEEDED, &recovery) && - !test_bit(MD_RECOVERY_RUNNING, &recovery)))) { + if (state == st_idle && r >= resync_max_sectors) { /* * Sync complete. */ @@ -3440,24 +3440,20 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery, if (test_bit(MD_RECOVERY_RECOVER, &recovery)) set_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags); - } else if (test_bit(MD_RECOVERY_RECOVER, &recovery)) { + } else if (state == st_recover) /* * In case we are recovering, the array is not in sync * and health chars should show the recovering legs. */ ; - - } else if (test_bit(MD_RECOVERY_SYNC, &recovery) && - !test_bit(MD_RECOVERY_REQUESTED, &recovery)) { + else if (state == st_resync) /* * If "resync" is occurring, the raid set * is or may be out of sync hence the health * characters shall be 'a'. */ set_bit(RT_FLAG_RS_RESYNCING, &rs->runtime_flags); - - } else if (test_bit(MD_RECOVERY_RESHAPE, &recovery) && - !test_bit(MD_RECOVERY_REQUESTED, &recovery)) { + else if (state == st_reshape) /* * If "reshape" is occurring, the raid set * is or may be out of sync hence the health @@ -3465,7 +3461,7 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery, */ set_bit(RT_FLAG_RS_RESYNCING, &rs->runtime_flags); - } else if (test_bit(MD_RECOVERY_REQUESTED, &recovery)) { + else if (state == st_check || state == st_repair) /* * If "check" or "repair" is occurring, the raid set has * undergone an initial sync and the health characters @@ -3473,12 +3469,12 @@ static sector_t rs_get_progress(struct raid_set *rs, unsigned long recovery, */ set_bit(RT_FLAG_RS_IN_SYNC, &rs->runtime_flags); - } else { + else { struct md_rdev *rdev; /* * We are idle and recovery is needed, prevent 'A' chars race - * caused by components still set to in-sync by constrcuctor. + * caused by components still set to in-sync by constructor. */ if (test_bit(MD_RECOVERY_NEEDED, &recovery)) set_bit(RT_FLAG_RS_RESYNCING, &rs->runtime_flags); @@ -3542,7 +3538,7 @@ static void raid_status(struct dm_target *ti, status_type_t type, progress = rs_get_progress(rs, recovery, resync_max_sectors); resync_mismatches = (mddev->last_sync_action && !strcasecmp(mddev->last_sync_action, "check")) ? atomic64_read(&mddev->resync_mismatches) : 0; - sync_action = decipher_sync_action(&rs->md, recovery); + sync_action = sync_str(decipher_sync_action(&rs->md, recovery)); /* HM FIXME: do we want another state char for raid0? It shows 'D'/'A'/'-' now */ for (i = 0; i < rs->raid_disks; i++) @@ -3892,14 +3888,13 @@ static int rs_start_reshape(struct raid_set *rs) struct mddev *mddev = &rs->md; struct md_personality *pers = mddev->pers; + /* Don't allow the sync thread to work until the table gets reloaded. */ + set_bit(MD_RECOVERY_WAIT, &mddev->recovery); + r = rs_setup_reshape(rs); if (r) return r; - /* Need to be resumed to be able to start reshape, recovery is frozen until raid_resume() though */ - if (test_and_clear_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags)) - mddev_resume(mddev); - /* * Check any reshape constraints enforced by the personalility * @@ -3923,10 +3918,6 @@ static int rs_start_reshape(struct raid_set *rs) } } - /* Suspend because a resume will happen in raid_resume() */ - set_bit(RT_FLAG_RS_SUSPENDED, &rs->runtime_flags); - mddev_suspend(mddev); - /* * Now reshape got set up, update superblocks to * reflect the fact so that a table reload will @@ -3947,29 +3938,6 @@ static int raid_preresume(struct dm_target *ti) if (test_and_set_bit(RT_FLAG_RS_PRERESUMED, &rs->runtime_flags)) return 0; - if (!test_bit(__CTR_FLAG_REBUILD, &rs->ctr_flags)) { - struct raid_set *rs_active = rs_find_active(rs); - - if (rs_active) { - /* - * In case no rebuilds have been requested - * and an active table slot exists, copy - * current resynchonization completed and - * reshape position pointers across from - * suspended raid set in the active slot. - * - * This resumes the new mapping at current - * offsets to continue recover/reshape without - * necessarily redoing a raid set partially or - * causing data corruption in case of a reshape. - */ - if (rs_active->md.curr_resync_completed != MaxSector) - mddev->curr_resync_completed = rs_active->md.curr_resync_completed; - if (rs_active->md.reshape_position != MaxSector) - mddev->reshape_position = rs_active->md.reshape_position; - } - } - /* * The superblocks need to be updated on disk if the * array is new or new devices got added (thus zeroed @@ -4046,7 +4014,7 @@ static void raid_resume(struct dm_target *ti) static struct target_type raid_target = { .name = "raid", - .version = {1, 13, 2}, + .version = {1, 14, 0}, .module = THIS_MODULE, .ctr = raid_ctr, .dtr = raid_dtr, diff --git a/drivers/md/dm-thin-metadata.c b/drivers/md/dm-thin-metadata.c index 72142021b5c9..74f6770c70b1 100644 --- a/drivers/md/dm-thin-metadata.c +++ b/drivers/md/dm-thin-metadata.c @@ -189,6 +189,12 @@ struct dm_pool_metadata { sector_t data_block_size; /* + * We reserve a section of the metadata for commit overhead. + * All reported space does *not* include this. + */ + dm_block_t metadata_reserve; + + /* * Set if a transaction has to be aborted but the attempt to roll back * to the previous (good) transaction failed. The only pool metadata * operation possible in this state is the closing of the device. @@ -816,6 +822,22 @@ static int __commit_transaction(struct dm_pool_metadata *pmd) return dm_tm_commit(pmd->tm, sblock); } +static void __set_metadata_reserve(struct dm_pool_metadata *pmd) +{ + int r; + dm_block_t total; + dm_block_t max_blocks = 4096; /* 16M */ + + r = dm_sm_get_nr_blocks(pmd->metadata_sm, &total); + if (r) { + DMERR("could not get size of metadata device"); + pmd->metadata_reserve = max_blocks; + } else { + sector_div(total, 10); + pmd->metadata_reserve = min(max_blocks, total); + } +} + struct dm_pool_metadata *dm_pool_metadata_open(struct block_device *bdev, sector_t data_block_size, bool format_device) @@ -849,6 +871,8 @@ struct dm_pool_metadata *dm_pool_metadata_open(struct block_device *bdev, return ERR_PTR(r); } + __set_metadata_reserve(pmd); + return pmd; } @@ -1820,6 +1844,13 @@ int dm_pool_get_free_metadata_block_count(struct dm_pool_metadata *pmd, down_read(&pmd->root_lock); if (!pmd->fail_io) r = dm_sm_get_nr_free(pmd->metadata_sm, result); + + if (!r) { + if (*result < pmd->metadata_reserve) + *result = 0; + else + *result -= pmd->metadata_reserve; + } up_read(&pmd->root_lock); return r; @@ -1932,8 +1963,11 @@ int dm_pool_resize_metadata_dev(struct dm_pool_metadata *pmd, dm_block_t new_cou int r = -EINVAL; down_write(&pmd->root_lock); - if (!pmd->fail_io) + if (!pmd->fail_io) { r = __resize_space_map(pmd->metadata_sm, new_count); + if (!r) + __set_metadata_reserve(pmd); + } up_write(&pmd->root_lock); return r; diff --git a/drivers/md/dm-thin.c b/drivers/md/dm-thin.c index 7bd60a150f8f..aaf1ad481ee8 100644 --- a/drivers/md/dm-thin.c +++ b/drivers/md/dm-thin.c @@ -200,7 +200,13 @@ struct dm_thin_new_mapping; enum pool_mode { PM_WRITE, /* metadata may be changed */ PM_OUT_OF_DATA_SPACE, /* metadata may be changed, though data may not be allocated */ + + /* + * Like READ_ONLY, except may switch back to WRITE on metadata resize. Reported as READ_ONLY. + */ + PM_OUT_OF_METADATA_SPACE, PM_READ_ONLY, /* metadata may not be changed */ + PM_FAIL, /* all I/O fails */ }; @@ -1371,7 +1377,35 @@ static void set_pool_mode(struct pool *pool, enum pool_mode new_mode); static void requeue_bios(struct pool *pool); -static void check_for_space(struct pool *pool) +static bool is_read_only_pool_mode(enum pool_mode mode) +{ + return (mode == PM_OUT_OF_METADATA_SPACE || mode == PM_READ_ONLY); +} + +static bool is_read_only(struct pool *pool) +{ + return is_read_only_pool_mode(get_pool_mode(pool)); +} + +static void check_for_metadata_space(struct pool *pool) +{ + int r; + const char *ooms_reason = NULL; + dm_block_t nr_free; + + r = dm_pool_get_free_metadata_block_count(pool->pmd, &nr_free); + if (r) + ooms_reason = "Could not get free metadata blocks"; + else if (!nr_free) + ooms_reason = "No free metadata blocks"; + + if (ooms_reason && !is_read_only(pool)) { + DMERR("%s", ooms_reason); + set_pool_mode(pool, PM_OUT_OF_METADATA_SPACE); + } +} + +static void check_for_data_space(struct pool *pool) { int r; dm_block_t nr_free; @@ -1397,14 +1431,16 @@ static int commit(struct pool *pool) { int r; - if (get_pool_mode(pool) >= PM_READ_ONLY) + if (get_pool_mode(pool) >= PM_OUT_OF_METADATA_SPACE) return -EINVAL; r = dm_pool_commit_metadata(pool->pmd); if (r) metadata_operation_failed(pool, "dm_pool_commit_metadata", r); - else - check_for_space(pool); + else { + check_for_metadata_space(pool); + check_for_data_space(pool); + } return r; } @@ -1470,6 +1506,19 @@ static int alloc_data_block(struct thin_c *tc, dm_block_t *result) return r; } + r = dm_pool_get_free_metadata_block_count(pool->pmd, &free_blocks); + if (r) { + metadata_operation_failed(pool, "dm_pool_get_free_metadata_block_count", r); + return r; + } + + if (!free_blocks) { + /* Let's commit before we use up the metadata reserve. */ + r = commit(pool); + if (r) + return r; + } + return 0; } @@ -1501,6 +1550,7 @@ static blk_status_t should_error_unserviceable_bio(struct pool *pool) case PM_OUT_OF_DATA_SPACE: return pool->pf.error_if_no_space ? BLK_STS_NOSPC : 0; + case PM_OUT_OF_METADATA_SPACE: case PM_READ_ONLY: case PM_FAIL: return BLK_STS_IOERR; @@ -2464,8 +2514,9 @@ static void set_pool_mode(struct pool *pool, enum pool_mode new_mode) error_retry_list(pool); break; + case PM_OUT_OF_METADATA_SPACE: case PM_READ_ONLY: - if (old_mode != new_mode) + if (!is_read_only_pool_mode(old_mode)) notify_of_pool_mode_change(pool, "read-only"); dm_pool_metadata_read_only(pool->pmd); pool->process_bio = process_bio_read_only; @@ -3403,6 +3454,10 @@ static int maybe_resize_metadata_dev(struct dm_target *ti, bool *need_commit) DMINFO("%s: growing the metadata device from %llu to %llu blocks", dm_device_name(pool->pool_md), sb_metadata_dev_size, metadata_dev_size); + + if (get_pool_mode(pool) == PM_OUT_OF_METADATA_SPACE) + set_pool_mode(pool, PM_WRITE); + r = dm_pool_resize_metadata_dev(pool->pmd, metadata_dev_size); if (r) { metadata_operation_failed(pool, "dm_pool_resize_metadata_dev", r); @@ -3707,7 +3762,7 @@ static int pool_message(struct dm_target *ti, unsigned argc, char **argv, struct pool_c *pt = ti->private; struct pool *pool = pt->pool; - if (get_pool_mode(pool) >= PM_READ_ONLY) { + if (get_pool_mode(pool) >= PM_OUT_OF_METADATA_SPACE) { DMERR("%s: unable to service pool target messages in READ_ONLY or FAIL mode", dm_device_name(pool->pool_md)); return -EOPNOTSUPP; @@ -3781,6 +3836,7 @@ static void pool_status(struct dm_target *ti, status_type_t type, dm_block_t nr_blocks_data; dm_block_t nr_blocks_metadata; dm_block_t held_root; + enum pool_mode mode; char buf[BDEVNAME_SIZE]; char buf2[BDEVNAME_SIZE]; struct pool_c *pt = ti->private; @@ -3851,9 +3907,10 @@ static void pool_status(struct dm_target *ti, status_type_t type, else DMEMIT("- "); - if (pool->pf.mode == PM_OUT_OF_DATA_SPACE) + mode = get_pool_mode(pool); + if (mode == PM_OUT_OF_DATA_SPACE) DMEMIT("out_of_data_space "); - else if (pool->pf.mode == PM_READ_ONLY) + else if (is_read_only_pool_mode(mode)) DMEMIT("ro "); else DMEMIT("rw "); diff --git a/drivers/md/dm-verity-target.c b/drivers/md/dm-verity-target.c index 12decdbd722d..fc65f0dedf7f 100644 --- a/drivers/md/dm-verity-target.c +++ b/drivers/md/dm-verity-target.c @@ -99,10 +99,26 @@ static int verity_hash_update(struct dm_verity *v, struct ahash_request *req, { struct scatterlist sg; - sg_init_one(&sg, data, len); - ahash_request_set_crypt(req, &sg, NULL, len); - - return crypto_wait_req(crypto_ahash_update(req), wait); + if (likely(!is_vmalloc_addr(data))) { + sg_init_one(&sg, data, len); + ahash_request_set_crypt(req, &sg, NULL, len); + return crypto_wait_req(crypto_ahash_update(req), wait); + } else { + do { + int r; + size_t this_step = min_t(size_t, len, PAGE_SIZE - offset_in_page(data)); + flush_kernel_vmap_range((void *)data, this_step); + sg_init_table(&sg, 1); + sg_set_page(&sg, vmalloc_to_page(data), this_step, offset_in_page(data)); + ahash_request_set_crypt(req, &sg, NULL, this_step); + r = crypto_wait_req(crypto_ahash_update(req), wait); + if (unlikely(r)) + return r; + data += this_step; + len -= this_step; + } while (len); + return 0; + } } /* diff --git a/drivers/md/md-cluster.c b/drivers/md/md-cluster.c index 94329e03001e..0b2af6e74fc3 100644 --- a/drivers/md/md-cluster.c +++ b/drivers/md/md-cluster.c @@ -1276,18 +1276,18 @@ static int resync_info_update(struct mddev *mddev, sector_t lo, sector_t hi) static int resync_finish(struct mddev *mddev) { struct md_cluster_info *cinfo = mddev->cluster_info; + int ret = 0; clear_bit(MD_RESYNCING_REMOTE, &mddev->recovery); - dlm_unlock_sync(cinfo->resync_lockres); /* * If resync thread is interrupted so we can't say resync is finished, * another node will launch resync thread to continue. */ - if (test_bit(MD_CLOSING, &mddev->flags)) - return 0; - else - return resync_info_update(mddev, 0, 0); + if (!test_bit(MD_CLOSING, &mddev->flags)) + ret = resync_info_update(mddev, 0, 0); + dlm_unlock_sync(cinfo->resync_lockres); + return ret; } static int area_resyncing(struct mddev *mddev, int direction, diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index 981898049491..d6f7978b4449 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -4529,11 +4529,12 @@ static sector_t reshape_request(struct mddev *mddev, sector_t sector_nr, allow_barrier(conf); } + raise_barrier(conf, 0); read_more: /* Now schedule reads for blocks from sector_nr to last */ r10_bio = raid10_alloc_init_r10buf(conf); r10_bio->state = 0; - raise_barrier(conf, sectors_done != 0); + raise_barrier(conf, 1); atomic_set(&r10_bio->remaining, 0); r10_bio->mddev = mddev; r10_bio->sector = sector_nr; @@ -4629,6 +4630,8 @@ read_more: if (sector_nr <= last) goto read_more; + lower_barrier(conf); + /* Now that we have done the whole section we can * update reshape_progress */ diff --git a/drivers/md/raid5-log.h b/drivers/md/raid5-log.h index a001808a2b77..bfb811407061 100644 --- a/drivers/md/raid5-log.h +++ b/drivers/md/raid5-log.h @@ -46,6 +46,11 @@ extern int ppl_modify_log(struct r5conf *conf, struct md_rdev *rdev, bool add); extern void ppl_quiesce(struct r5conf *conf, int quiesce); extern int ppl_handle_flush_request(struct r5l_log *log, struct bio *bio); +static inline bool raid5_has_log(struct r5conf *conf) +{ + return test_bit(MD_HAS_JOURNAL, &conf->mddev->flags); +} + static inline bool raid5_has_ppl(struct r5conf *conf) { return test_bit(MD_HAS_PPL, &conf->mddev->flags); diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 4ce0d7502fad..e4e98f47865d 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -733,7 +733,7 @@ static bool stripe_can_batch(struct stripe_head *sh) { struct r5conf *conf = sh->raid_conf; - if (conf->log || raid5_has_ppl(conf)) + if (raid5_has_log(conf) || raid5_has_ppl(conf)) return false; return test_bit(STRIPE_BATCH_READY, &sh->state) && !test_bit(STRIPE_BITMAP_PENDING, &sh->state) && @@ -7737,7 +7737,7 @@ static int raid5_resize(struct mddev *mddev, sector_t sectors) sector_t newsize; struct r5conf *conf = mddev->private; - if (conf->log || raid5_has_ppl(conf)) + if (raid5_has_log(conf) || raid5_has_ppl(conf)) return -EINVAL; sectors &= ~((sector_t)conf->chunk_sectors - 1); newsize = raid5_size(mddev, sectors, mddev->raid_disks); @@ -7788,7 +7788,7 @@ static int check_reshape(struct mddev *mddev) { struct r5conf *conf = mddev->private; - if (conf->log || raid5_has_ppl(conf)) + if (raid5_has_log(conf) || raid5_has_ppl(conf)) return -EINVAL; if (mddev->delta_disks == 0 && mddev->new_layout == mddev->layout && diff --git a/drivers/media/i2c/mt9v111.c b/drivers/media/i2c/mt9v111.c index b5410aeb5fe2..bb41bea950ac 100644 --- a/drivers/media/i2c/mt9v111.c +++ b/drivers/media/i2c/mt9v111.c @@ -1159,41 +1159,21 @@ static int mt9v111_probe(struct i2c_client *client) V4L2_CID_AUTO_WHITE_BALANCE, 0, 1, 1, V4L2_WHITE_BALANCE_AUTO); - if (IS_ERR_OR_NULL(mt9v111->auto_awb)) { - ret = PTR_ERR(mt9v111->auto_awb); - goto error_free_ctrls; - } - mt9v111->auto_exp = v4l2_ctrl_new_std_menu(&mt9v111->ctrls, &mt9v111_ctrl_ops, V4L2_CID_EXPOSURE_AUTO, V4L2_EXPOSURE_MANUAL, 0, V4L2_EXPOSURE_AUTO); - if (IS_ERR_OR_NULL(mt9v111->auto_exp)) { - ret = PTR_ERR(mt9v111->auto_exp); - goto error_free_ctrls; - } - - /* Initialize timings */ mt9v111->hblank = v4l2_ctrl_new_std(&mt9v111->ctrls, &mt9v111_ctrl_ops, V4L2_CID_HBLANK, MT9V111_CORE_R05_MIN_HBLANK, MT9V111_CORE_R05_MAX_HBLANK, 1, MT9V111_CORE_R05_DEF_HBLANK); - if (IS_ERR_OR_NULL(mt9v111->hblank)) { - ret = PTR_ERR(mt9v111->hblank); - goto error_free_ctrls; - } - mt9v111->vblank = v4l2_ctrl_new_std(&mt9v111->ctrls, &mt9v111_ctrl_ops, V4L2_CID_VBLANK, MT9V111_CORE_R06_MIN_VBLANK, MT9V111_CORE_R06_MAX_VBLANK, 1, MT9V111_CORE_R06_DEF_VBLANK); - if (IS_ERR_OR_NULL(mt9v111->vblank)) { - ret = PTR_ERR(mt9v111->vblank); - goto error_free_ctrls; - } /* PIXEL_RATE is fixed: just expose it to user space. */ v4l2_ctrl_new_std(&mt9v111->ctrls, &mt9v111_ctrl_ops, @@ -1201,6 +1181,10 @@ static int mt9v111_probe(struct i2c_client *client) DIV_ROUND_CLOSEST(mt9v111->sysclk, 2), 1, DIV_ROUND_CLOSEST(mt9v111->sysclk, 2)); + if (mt9v111->ctrls.error) { + ret = mt9v111->ctrls.error; + goto error_free_ctrls; + } mt9v111->sd.ctrl_handler = &mt9v111->ctrls; /* Start with default configuration: 640x480 UYVY. */ @@ -1226,26 +1210,27 @@ static int mt9v111_probe(struct i2c_client *client) mt9v111->pad.flags = MEDIA_PAD_FL_SOURCE; ret = media_entity_pads_init(&mt9v111->sd.entity, 1, &mt9v111->pad); if (ret) - goto error_free_ctrls; + goto error_free_entity; #endif ret = mt9v111_chip_probe(mt9v111); if (ret) - goto error_free_ctrls; + goto error_free_entity; ret = v4l2_async_register_subdev(&mt9v111->sd); if (ret) - goto error_free_ctrls; + goto error_free_entity; return 0; -error_free_ctrls: - v4l2_ctrl_handler_free(&mt9v111->ctrls); - +error_free_entity: #if IS_ENABLED(CONFIG_MEDIA_CONTROLLER) media_entity_cleanup(&mt9v111->sd.entity); #endif +error_free_ctrls: + v4l2_ctrl_handler_free(&mt9v111->ctrls); + mutex_destroy(&mt9v111->pwr_mutex); mutex_destroy(&mt9v111->stream_mutex); @@ -1259,12 +1244,12 @@ static int mt9v111_remove(struct i2c_client *client) v4l2_async_unregister_subdev(sd); - v4l2_ctrl_handler_free(&mt9v111->ctrls); - #if IS_ENABLED(CONFIG_MEDIA_CONTROLLER) media_entity_cleanup(&sd->entity); #endif + v4l2_ctrl_handler_free(&mt9v111->ctrls); + mutex_destroy(&mt9v111->pwr_mutex); mutex_destroy(&mt9v111->stream_mutex); diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index 94c1fe0e9787..54fe90acb5b2 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -541,6 +541,8 @@ config VIDEO_CROS_EC_CEC depends on MFD_CROS_EC select CEC_CORE select CEC_NOTIFIER + select CHROME_PLATFORMS + select CROS_EC_PROTO ---help--- If you say yes here you will get support for the ChromeOS Embedded Controller's CEC. diff --git a/drivers/media/platform/qcom/camss/camss-csid.c b/drivers/media/platform/qcom/camss/camss-csid.c index 729b31891466..a5ae85674ffb 100644 --- a/drivers/media/platform/qcom/camss/camss-csid.c +++ b/drivers/media/platform/qcom/camss/camss-csid.c @@ -10,6 +10,7 @@ #include <linux/clk.h> #include <linux/completion.h> #include <linux/interrupt.h> +#include <linux/io.h> #include <linux/kernel.h> #include <linux/of.h> #include <linux/platform_device.h> diff --git a/drivers/media/platform/qcom/camss/camss-csiphy-2ph-1-0.c b/drivers/media/platform/qcom/camss/camss-csiphy-2ph-1-0.c index c832539397d7..12bce391d71f 100644 --- a/drivers/media/platform/qcom/camss/camss-csiphy-2ph-1-0.c +++ b/drivers/media/platform/qcom/camss/camss-csiphy-2ph-1-0.c @@ -12,6 +12,7 @@ #include <linux/delay.h> #include <linux/interrupt.h> +#include <linux/io.h> #define CAMSS_CSI_PHY_LNn_CFG2(n) (0x004 + 0x40 * (n)) #define CAMSS_CSI_PHY_LNn_CFG3(n) (0x008 + 0x40 * (n)) diff --git a/drivers/media/platform/qcom/camss/camss-csiphy-3ph-1-0.c b/drivers/media/platform/qcom/camss/camss-csiphy-3ph-1-0.c index bcd0dfd33618..2e65caf1ecae 100644 --- a/drivers/media/platform/qcom/camss/camss-csiphy-3ph-1-0.c +++ b/drivers/media/platform/qcom/camss/camss-csiphy-3ph-1-0.c @@ -12,6 +12,7 @@ #include <linux/delay.h> #include <linux/interrupt.h> +#include <linux/io.h> #define CSIPHY_3PH_LNn_CFG1(n) (0x000 + 0x100 * (n)) #define CSIPHY_3PH_LNn_CFG1_SWI_REC_DLY_PRG (BIT(7) | BIT(6)) diff --git a/drivers/media/platform/qcom/camss/camss-csiphy.c b/drivers/media/platform/qcom/camss/camss-csiphy.c index 4559f3b1b38c..008afb85023b 100644 --- a/drivers/media/platform/qcom/camss/camss-csiphy.c +++ b/drivers/media/platform/qcom/camss/camss-csiphy.c @@ -10,6 +10,7 @@ #include <linux/clk.h> #include <linux/delay.h> #include <linux/interrupt.h> +#include <linux/io.h> #include <linux/kernel.h> #include <linux/of.h> #include <linux/platform_device.h> diff --git a/drivers/media/platform/qcom/camss/camss-ispif.c b/drivers/media/platform/qcom/camss/camss-ispif.c index 7f269021d08c..1f33b4eb198c 100644 --- a/drivers/media/platform/qcom/camss/camss-ispif.c +++ b/drivers/media/platform/qcom/camss/camss-ispif.c @@ -10,6 +10,7 @@ #include <linux/clk.h> #include <linux/completion.h> #include <linux/interrupt.h> +#include <linux/io.h> #include <linux/iopoll.h> #include <linux/kernel.h> #include <linux/mutex.h> @@ -1076,8 +1077,8 @@ int msm_ispif_subdev_init(struct ispif_device *ispif, else return -EINVAL; - ispif->line = kcalloc(ispif->line_num, sizeof(*ispif->line), - GFP_KERNEL); + ispif->line = devm_kcalloc(dev, ispif->line_num, sizeof(*ispif->line), + GFP_KERNEL); if (!ispif->line) return -ENOMEM; diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-1.c b/drivers/media/platform/qcom/camss/camss-vfe-4-1.c index da3a9fed9f2d..174a36be6f5d 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe-4-1.c +++ b/drivers/media/platform/qcom/camss/camss-vfe-4-1.c @@ -9,6 +9,7 @@ */ #include <linux/interrupt.h> +#include <linux/io.h> #include <linux/iopoll.h> #include "camss-vfe.h" diff --git a/drivers/media/platform/qcom/camss/camss-vfe-4-7.c b/drivers/media/platform/qcom/camss/camss-vfe-4-7.c index 4c584bffd179..0dca8bf9281e 100644 --- a/drivers/media/platform/qcom/camss/camss-vfe-4-7.c +++ b/drivers/media/platform/qcom/camss/camss-vfe-4-7.c @@ -9,6 +9,7 @@ */ #include <linux/interrupt.h> +#include <linux/io.h> #include <linux/iopoll.h> #include "camss-vfe.h" diff --git a/drivers/media/platform/qcom/camss/camss.c b/drivers/media/platform/qcom/camss/camss.c index dcc0c30ef1b1..669615fff6a0 100644 --- a/drivers/media/platform/qcom/camss/camss.c +++ b/drivers/media/platform/qcom/camss/camss.c @@ -848,17 +848,18 @@ static int camss_probe(struct platform_device *pdev) return -EINVAL; } - camss->csiphy = kcalloc(camss->csiphy_num, sizeof(*camss->csiphy), - GFP_KERNEL); + camss->csiphy = devm_kcalloc(dev, camss->csiphy_num, + sizeof(*camss->csiphy), GFP_KERNEL); if (!camss->csiphy) return -ENOMEM; - camss->csid = kcalloc(camss->csid_num, sizeof(*camss->csid), - GFP_KERNEL); + camss->csid = devm_kcalloc(dev, camss->csid_num, sizeof(*camss->csid), + GFP_KERNEL); if (!camss->csid) return -ENOMEM; - camss->vfe = kcalloc(camss->vfe_num, sizeof(*camss->vfe), GFP_KERNEL); + camss->vfe = devm_kcalloc(dev, camss->vfe_num, sizeof(*camss->vfe), + GFP_KERNEL); if (!camss->vfe) return -ENOMEM; @@ -993,12 +994,12 @@ static const struct of_device_id camss_dt_match[] = { MODULE_DEVICE_TABLE(of, camss_dt_match); -static int camss_runtime_suspend(struct device *dev) +static int __maybe_unused camss_runtime_suspend(struct device *dev) { return 0; } -static int camss_runtime_resume(struct device *dev) +static int __maybe_unused camss_runtime_resume(struct device *dev) { return 0; } diff --git a/drivers/media/usb/dvb-usb-v2/af9035.c b/drivers/media/usb/dvb-usb-v2/af9035.c index 666d319d3d1a..1f6c1eefe389 100644 --- a/drivers/media/usb/dvb-usb-v2/af9035.c +++ b/drivers/media/usb/dvb-usb-v2/af9035.c @@ -402,8 +402,10 @@ static int af9035_i2c_master_xfer(struct i2c_adapter *adap, if (msg[0].addr == state->af9033_i2c_addr[1]) reg |= 0x100000; - ret = af9035_wr_regs(d, reg, &msg[0].buf[3], - msg[0].len - 3); + ret = (msg[0].len >= 3) ? af9035_wr_regs(d, reg, + &msg[0].buf[3], + msg[0].len - 3) + : -EOPNOTSUPP; } else { /* I2C write */ u8 buf[MAX_XFER_SIZE]; diff --git a/drivers/memory/ti-aemif.c b/drivers/memory/ti-aemif.c index 31112f622b88..475e5b3790ed 100644 --- a/drivers/memory/ti-aemif.c +++ b/drivers/memory/ti-aemif.c @@ -411,7 +411,7 @@ static int aemif_probe(struct platform_device *pdev) if (ret < 0) goto error; } - } else { + } else if (pdata) { for (i = 0; i < pdata->num_sub_devices; i++) { pdata->sub_devices[i].dev.parent = dev; ret = platform_device_register(&pdata->sub_devices[i]); diff --git a/drivers/mfd/omap-usb-host.c b/drivers/mfd/omap-usb-host.c index e11ab12fbdf2..800986a79704 100644 --- a/drivers/mfd/omap-usb-host.c +++ b/drivers/mfd/omap-usb-host.c @@ -528,8 +528,8 @@ static int usbhs_omap_get_dt_pdata(struct device *dev, } static const struct of_device_id usbhs_child_match_table[] = { - { .compatible = "ti,omap-ehci", }, - { .compatible = "ti,omap-ohci", }, + { .compatible = "ti,ehci-omap", }, + { .compatible = "ti,ohci-omap3", }, { } }; @@ -855,6 +855,7 @@ static struct platform_driver usbhs_omap_driver = { .pm = &usbhsomap_dev_pm_ops, .of_match_table = usbhs_omap_dt_ids, }, + .probe = usbhs_omap_probe, .remove = usbhs_omap_remove, }; @@ -864,9 +865,9 @@ MODULE_ALIAS("platform:" USBHS_DRIVER_NAME); MODULE_LICENSE("GPL v2"); MODULE_DESCRIPTION("usb host common core driver for omap EHCI and OHCI"); -static int __init omap_usbhs_drvinit(void) +static int omap_usbhs_drvinit(void) { - return platform_driver_probe(&usbhs_omap_driver, usbhs_omap_probe); + return platform_driver_register(&usbhs_omap_driver); } /* @@ -878,7 +879,7 @@ static int __init omap_usbhs_drvinit(void) */ fs_initcall_sync(omap_usbhs_drvinit); -static void __exit omap_usbhs_drvexit(void) +static void omap_usbhs_drvexit(void) { platform_driver_unregister(&usbhs_omap_driver); } diff --git a/drivers/misc/hmc6352.c b/drivers/misc/hmc6352.c index eeb7eef62174..38f90e179927 100644 --- a/drivers/misc/hmc6352.c +++ b/drivers/misc/hmc6352.c @@ -27,6 +27,7 @@ #include <linux/err.h> #include <linux/delay.h> #include <linux/sysfs.h> +#include <linux/nospec.h> static DEFINE_MUTEX(compass_mutex); @@ -50,6 +51,7 @@ static int compass_store(struct device *dev, const char *buf, size_t count, return ret; if (val >= strlen(map)) return -EINVAL; + val = array_index_nospec(val, strlen(map)); mutex_lock(&compass_mutex); ret = compass_command(c, map[val]); mutex_unlock(&compass_mutex); diff --git a/drivers/misc/ibmvmc.c b/drivers/misc/ibmvmc.c index 8f82bb9d11e2..b8aaa684c397 100644 --- a/drivers/misc/ibmvmc.c +++ b/drivers/misc/ibmvmc.c @@ -2131,7 +2131,7 @@ static int ibmvmc_init_crq_queue(struct crq_server_adapter *adapter) retrc = plpar_hcall_norets(H_REG_CRQ, vdev->unit_address, queue->msg_token, PAGE_SIZE); - retrc = rc; + rc = retrc; if (rc == H_RESOURCE) rc = ibmvmc_reset_crq_queue(adapter); diff --git a/drivers/misc/mei/bus.c b/drivers/misc/mei/bus.c index 7bba62a72921..fc3872fe7b25 100644 --- a/drivers/misc/mei/bus.c +++ b/drivers/misc/mei/bus.c @@ -521,17 +521,15 @@ int mei_cldev_enable(struct mei_cl_device *cldev) cl = cldev->cl; + mutex_lock(&bus->device_lock); if (cl->state == MEI_FILE_UNINITIALIZED) { - mutex_lock(&bus->device_lock); ret = mei_cl_link(cl); - mutex_unlock(&bus->device_lock); if (ret) - return ret; + goto out; /* update pointers */ cl->cldev = cldev; } - mutex_lock(&bus->device_lock); if (mei_cl_is_connected(cl)) { ret = 0; goto out; @@ -616,9 +614,8 @@ int mei_cldev_disable(struct mei_cl_device *cldev) if (err < 0) dev_err(bus->dev, "Could not disconnect from the ME client\n"); -out: mei_cl_bus_module_put(cldev); - +out: /* Flush queues and remove any pending read */ mei_cl_flush_queues(cl, NULL); mei_cl_unlink(cl); @@ -876,12 +873,13 @@ static void mei_cl_bus_dev_release(struct device *dev) mei_me_cl_put(cldev->me_cl); mei_dev_bus_put(cldev->bus); + mei_cl_unlink(cldev->cl); kfree(cldev->cl); kfree(cldev); } static const struct device_type mei_cl_device_type = { - .release = mei_cl_bus_dev_release, + .release = mei_cl_bus_dev_release, }; /** diff --git a/drivers/misc/mei/client.c b/drivers/misc/mei/client.c index 4ab6251d418e..ebdcf0b450e2 100644 --- a/drivers/misc/mei/client.c +++ b/drivers/misc/mei/client.c @@ -1767,7 +1767,7 @@ out: } } - rets = buf->size; + rets = len; err: cl_dbg(dev, cl, "rpm: autosuspend\n"); pm_runtime_mark_last_busy(dev->dev); diff --git a/drivers/misc/mei/hbm.c b/drivers/misc/mei/hbm.c index 09e233d4c0de..e56f3e72d57a 100644 --- a/drivers/misc/mei/hbm.c +++ b/drivers/misc/mei/hbm.c @@ -1161,15 +1161,18 @@ int mei_hbm_dispatch(struct mei_device *dev, struct mei_msg_hdr *hdr) props_res = (struct hbm_props_response *)mei_msg; - if (props_res->status) { + if (props_res->status == MEI_HBMS_CLIENT_NOT_FOUND) { + dev_dbg(dev->dev, "hbm: properties response: %d CLIENT_NOT_FOUND\n", + props_res->me_addr); + } else if (props_res->status) { dev_err(dev->dev, "hbm: properties response: wrong status = %d %s\n", props_res->status, mei_hbm_status_str(props_res->status)); return -EPROTO; + } else { + mei_hbm_me_cl_add(dev, props_res); } - mei_hbm_me_cl_add(dev, props_res); - /* request property for the next client */ if (mei_hbm_prop_req(dev, props_res->me_addr + 1)) return -EIO; diff --git a/drivers/mmc/host/meson-mx-sdio.c b/drivers/mmc/host/meson-mx-sdio.c index 09cb89645d06..2cfec33178c1 100644 --- a/drivers/mmc/host/meson-mx-sdio.c +++ b/drivers/mmc/host/meson-mx-sdio.c @@ -517,19 +517,23 @@ static struct mmc_host_ops meson_mx_mmc_ops = { static struct platform_device *meson_mx_mmc_slot_pdev(struct device *parent) { struct device_node *slot_node; + struct platform_device *pdev; /* * TODO: the MMC core framework currently does not support * controllers with multiple slots properly. So we only register * the first slot for now */ - slot_node = of_find_compatible_node(parent->of_node, NULL, "mmc-slot"); + slot_node = of_get_compatible_child(parent->of_node, "mmc-slot"); if (!slot_node) { dev_warn(parent, "no 'mmc-slot' sub-node found\n"); return ERR_PTR(-ENOENT); } - return of_platform_device_create(slot_node, NULL, parent); + pdev = of_platform_device_create(slot_node, NULL, parent); + of_node_put(slot_node); + + return pdev; } static int meson_mx_mmc_add_host(struct meson_mx_mmc_host *host) diff --git a/drivers/mmc/host/omap_hsmmc.c b/drivers/mmc/host/omap_hsmmc.c index 071693ebfe18..68760d4a5d3d 100644 --- a/drivers/mmc/host/omap_hsmmc.c +++ b/drivers/mmc/host/omap_hsmmc.c @@ -2177,6 +2177,7 @@ static int omap_hsmmc_remove(struct platform_device *pdev) dma_release_channel(host->tx_chan); dma_release_channel(host->rx_chan); + dev_pm_clear_wake_irq(host->dev); pm_runtime_dont_use_autosuspend(host->dev); pm_runtime_put_sync(host->dev); pm_runtime_disable(host->dev); diff --git a/drivers/mtd/devices/m25p80.c b/drivers/mtd/devices/m25p80.c index cbfafc453274..270d3c9580c5 100644 --- a/drivers/mtd/devices/m25p80.c +++ b/drivers/mtd/devices/m25p80.c @@ -39,13 +39,23 @@ static int m25p80_read_reg(struct spi_nor *nor, u8 code, u8 *val, int len) struct spi_mem_op op = SPI_MEM_OP(SPI_MEM_OP_CMD(code, 1), SPI_MEM_OP_NO_ADDR, SPI_MEM_OP_NO_DUMMY, - SPI_MEM_OP_DATA_IN(len, val, 1)); + SPI_MEM_OP_DATA_IN(len, NULL, 1)); + void *scratchbuf; int ret; + scratchbuf = kmalloc(len, GFP_KERNEL); + if (!scratchbuf) + return -ENOMEM; + + op.data.buf.in = scratchbuf; ret = spi_mem_exec_op(flash->spimem, &op); if (ret < 0) dev_err(&flash->spimem->spi->dev, "error %d reading %x\n", ret, code); + else + memcpy(val, scratchbuf, len); + + kfree(scratchbuf); return ret; } @@ -56,9 +66,19 @@ static int m25p80_write_reg(struct spi_nor *nor, u8 opcode, u8 *buf, int len) struct spi_mem_op op = SPI_MEM_OP(SPI_MEM_OP_CMD(opcode, 1), SPI_MEM_OP_NO_ADDR, SPI_MEM_OP_NO_DUMMY, - SPI_MEM_OP_DATA_OUT(len, buf, 1)); + SPI_MEM_OP_DATA_OUT(len, NULL, 1)); + void *scratchbuf; + int ret; - return spi_mem_exec_op(flash->spimem, &op); + scratchbuf = kmemdup(buf, len, GFP_KERNEL); + if (!scratchbuf) + return -ENOMEM; + + op.data.buf.out = scratchbuf; + ret = spi_mem_exec_op(flash->spimem, &op); + kfree(scratchbuf); + + return ret; } static ssize_t m25p80_write(struct spi_nor *nor, loff_t to, size_t len, diff --git a/drivers/mtd/mtdpart.c b/drivers/mtd/mtdpart.c index 52e2cb35fc79..99c460facd5e 100644 --- a/drivers/mtd/mtdpart.c +++ b/drivers/mtd/mtdpart.c @@ -873,8 +873,11 @@ static int mtd_part_of_parse(struct mtd_info *master, int ret, err = 0; np = mtd_get_of_node(master); - if (!mtd_is_partition(master)) + if (mtd_is_partition(master)) + of_node_get(np); + else np = of_get_child_by_name(np, "partitions"); + of_property_for_each_string(np, "compatible", prop, compat) { parser = mtd_part_get_compatible_parser(compat); if (!parser) diff --git a/drivers/mtd/nand/raw/denali.c b/drivers/mtd/nand/raw/denali.c index 67b2065e7a19..b864b93dd289 100644 --- a/drivers/mtd/nand/raw/denali.c +++ b/drivers/mtd/nand/raw/denali.c @@ -596,6 +596,12 @@ static int denali_dma_xfer(struct denali_nand_info *denali, void *buf, } iowrite32(DMA_ENABLE__FLAG, denali->reg + DMA_ENABLE); + /* + * The ->setup_dma() hook kicks DMA by using the data/command + * interface, which belongs to a different AXI port from the + * register interface. Read back the register to avoid a race. + */ + ioread32(denali->reg + DMA_ENABLE); denali_reset_irq(denali); denali->setup_dma(denali, dma_addr, page, write); diff --git a/drivers/mtd/nand/raw/marvell_nand.c b/drivers/mtd/nand/raw/marvell_nand.c index 7af4d6213ee5..bc2ef5209783 100644 --- a/drivers/mtd/nand/raw/marvell_nand.c +++ b/drivers/mtd/nand/raw/marvell_nand.c @@ -1547,7 +1547,7 @@ static void marvell_nfc_parse_instructions(struct nand_chip *chip, for (op_id = 0; op_id < subop->ninstrs; op_id++) { unsigned int offset, naddrs; const u8 *addrs; - int len = nand_subop_get_data_len(subop, op_id); + int len; instr = &subop->instrs[op_id]; @@ -1593,6 +1593,7 @@ static void marvell_nfc_parse_instructions(struct nand_chip *chip, nfc_op->ndcb[0] |= NDCB0_CMD_XTYPE(XTYPE_MONOLITHIC_RW) | NDCB0_LEN_OVRD; + len = nand_subop_get_data_len(subop, op_id); nfc_op->ndcb[3] |= round_up(len, FIFO_DEPTH); } nfc_op->data_delay_ns = instr->delay_ns; @@ -1606,6 +1607,7 @@ static void marvell_nfc_parse_instructions(struct nand_chip *chip, nfc_op->ndcb[0] |= NDCB0_CMD_XTYPE(XTYPE_MONOLITHIC_RW) | NDCB0_LEN_OVRD; + len = nand_subop_get_data_len(subop, op_id); nfc_op->ndcb[3] |= round_up(len, FIFO_DEPTH); } nfc_op->data_delay_ns = instr->delay_ns; diff --git a/drivers/net/appletalk/ipddp.c b/drivers/net/appletalk/ipddp.c index 9375cef22420..3d27616d9c85 100644 --- a/drivers/net/appletalk/ipddp.c +++ b/drivers/net/appletalk/ipddp.c @@ -283,8 +283,12 @@ static int ipddp_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) case SIOCFINDIPDDPRT: spin_lock_bh(&ipddp_route_lock); rp = __ipddp_find_route(&rcp); - if (rp) - memcpy(&rcp2, rp, sizeof(rcp2)); + if (rp) { + memset(&rcp2, 0, sizeof(rcp2)); + rcp2.ip = rp->ip; + rcp2.at = rp->at; + rcp2.flags = rp->flags; + } spin_unlock_bh(&ipddp_route_lock); if (rp) { diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c index a764a83f99da..0d87e11e7f1d 100644 --- a/drivers/net/bonding/bond_main.c +++ b/drivers/net/bonding/bond_main.c @@ -971,16 +971,13 @@ static void bond_poll_controller(struct net_device *bond_dev) struct slave *slave = NULL; struct list_head *iter; struct ad_info ad_info; - struct netpoll_info *ni; - const struct net_device_ops *ops; if (BOND_MODE(bond) == BOND_MODE_8023AD) if (bond_3ad_get_active_agg_info(bond, &ad_info)) return; bond_for_each_slave_rcu(bond, slave, iter) { - ops = slave->dev->netdev_ops; - if (!bond_slave_is_up(slave) || !ops->ndo_poll_controller) + if (!bond_slave_is_up(slave)) continue; if (BOND_MODE(bond) == BOND_MODE_8023AD) { @@ -992,11 +989,7 @@ static void bond_poll_controller(struct net_device *bond_dev) continue; } - ni = rcu_dereference_bh(slave->dev->npinfo); - if (down_trylock(&ni->dev_lock)) - continue; - ops->ndo_poll_controller(slave->dev); - up(&ni->dev_lock); + netpoll_poll_dev(slave->dev); } } diff --git a/drivers/net/dsa/mv88e6xxx/global1.h b/drivers/net/dsa/mv88e6xxx/global1.h index 7c791c1da4b9..bef01331266f 100644 --- a/drivers/net/dsa/mv88e6xxx/global1.h +++ b/drivers/net/dsa/mv88e6xxx/global1.h @@ -128,7 +128,7 @@ #define MV88E6XXX_G1_ATU_OP_GET_CLR_VIOLATION 0x7000 #define MV88E6XXX_G1_ATU_OP_AGE_OUT_VIOLATION BIT(7) #define MV88E6XXX_G1_ATU_OP_MEMBER_VIOLATION BIT(6) -#define MV88E6XXX_G1_ATU_OP_MISS_VIOLTATION BIT(5) +#define MV88E6XXX_G1_ATU_OP_MISS_VIOLATION BIT(5) #define MV88E6XXX_G1_ATU_OP_FULL_VIOLATION BIT(4) /* Offset 0x0C: ATU Data Register */ diff --git a/drivers/net/dsa/mv88e6xxx/global1_atu.c b/drivers/net/dsa/mv88e6xxx/global1_atu.c index 307410898fc9..5200e4bdce93 100644 --- a/drivers/net/dsa/mv88e6xxx/global1_atu.c +++ b/drivers/net/dsa/mv88e6xxx/global1_atu.c @@ -349,7 +349,7 @@ static irqreturn_t mv88e6xxx_g1_atu_prob_irq_thread_fn(int irq, void *dev_id) chip->ports[entry.portvec].atu_member_violation++; } - if (val & MV88E6XXX_G1_ATU_OP_MEMBER_VIOLATION) { + if (val & MV88E6XXX_G1_ATU_OP_MISS_VIOLATION) { dev_err_ratelimited(chip->dev, "ATU miss violation for %pM portvec %x\n", entry.mac, entry.portvec); diff --git a/drivers/net/ethernet/amazon/ena/ena_com.c b/drivers/net/ethernet/amazon/ena/ena_com.c index 17f12c18d225..7635c38e77dd 100644 --- a/drivers/net/ethernet/amazon/ena/ena_com.c +++ b/drivers/net/ethernet/amazon/ena/ena_com.c @@ -459,12 +459,12 @@ static void ena_com_handle_admin_completion(struct ena_com_admin_queue *admin_qu cqe = &admin_queue->cq.entries[head_masked]; /* Go over all the completions */ - while ((cqe->acq_common_descriptor.flags & + while ((READ_ONCE(cqe->acq_common_descriptor.flags) & ENA_ADMIN_ACQ_COMMON_DESC_PHASE_MASK) == phase) { /* Do not read the rest of the completion entry before the * phase bit was validated */ - rmb(); + dma_rmb(); ena_com_handle_single_admin_completion(admin_queue, cqe); head_masked++; @@ -627,17 +627,10 @@ static u32 ena_com_reg_bar_read32(struct ena_com_dev *ena_dev, u16 offset) mmio_read_reg |= mmio_read->seq_num & ENA_REGS_MMIO_REG_READ_REQ_ID_MASK; - /* make sure read_resp->req_id get updated before the hw can write - * there - */ - wmb(); - - writel_relaxed(mmio_read_reg, - ena_dev->reg_bar + ENA_REGS_MMIO_REG_READ_OFF); + writel(mmio_read_reg, ena_dev->reg_bar + ENA_REGS_MMIO_REG_READ_OFF); - mmiowb(); for (i = 0; i < timeout; i++) { - if (read_resp->req_id == mmio_read->seq_num) + if (READ_ONCE(read_resp->req_id) == mmio_read->seq_num) break; udelay(1); @@ -1796,8 +1789,13 @@ void ena_com_aenq_intr_handler(struct ena_com_dev *dev, void *data) aenq_common = &aenq_e->aenq_common_desc; /* Go over all the events */ - while ((aenq_common->flags & ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK) == - phase) { + while ((READ_ONCE(aenq_common->flags) & + ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK) == phase) { + /* Make sure the phase bit (ownership) is as expected before + * reading the rest of the descriptor. + */ + dma_rmb(); + pr_debug("AENQ! Group[%x] Syndrom[%x] timestamp: [%llus]\n", aenq_common->group, aenq_common->syndrom, (u64)aenq_common->timestamp_low + diff --git a/drivers/net/ethernet/amazon/ena/ena_eth_com.c b/drivers/net/ethernet/amazon/ena/ena_eth_com.c index ea149c134e15..1c682b76190f 100644 --- a/drivers/net/ethernet/amazon/ena/ena_eth_com.c +++ b/drivers/net/ethernet/amazon/ena/ena_eth_com.c @@ -51,6 +51,11 @@ static inline struct ena_eth_io_rx_cdesc_base *ena_com_get_next_rx_cdesc( if (desc_phase != expected_phase) return NULL; + /* Make sure we read the rest of the descriptor after the phase bit + * has been read + */ + dma_rmb(); + return cdesc; } @@ -493,6 +498,7 @@ int ena_com_tx_comp_req_id_get(struct ena_com_io_cq *io_cq, u16 *req_id) if (cdesc_phase != expected_phase) return -EAGAIN; + dma_rmb(); if (unlikely(cdesc->req_id >= io_cq->q_depth)) { pr_err("Invalid req id %d\n", cdesc->req_id); return -EINVAL; diff --git a/drivers/net/ethernet/amazon/ena/ena_eth_com.h b/drivers/net/ethernet/amazon/ena/ena_eth_com.h index 6fdc753d9483..2f7657227cfe 100644 --- a/drivers/net/ethernet/amazon/ena/ena_eth_com.h +++ b/drivers/net/ethernet/amazon/ena/ena_eth_com.h @@ -107,8 +107,7 @@ static inline int ena_com_sq_empty_space(struct ena_com_io_sq *io_sq) return io_sq->q_depth - 1 - cnt; } -static inline int ena_com_write_sq_doorbell(struct ena_com_io_sq *io_sq, - bool relaxed) +static inline int ena_com_write_sq_doorbell(struct ena_com_io_sq *io_sq) { u16 tail; @@ -117,10 +116,7 @@ static inline int ena_com_write_sq_doorbell(struct ena_com_io_sq *io_sq, pr_debug("write submission queue doorbell for queue: %d tail: %d\n", io_sq->qid, tail); - if (relaxed) - writel_relaxed(tail, io_sq->db_addr); - else - writel(tail, io_sq->db_addr); + writel(tail, io_sq->db_addr); return 0; } diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c index c673ac2df65b..29b5774dd32d 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c @@ -76,7 +76,7 @@ MODULE_DEVICE_TABLE(pci, ena_pci_tbl); static int ena_rss_init_default(struct ena_adapter *adapter); static void check_for_admin_com_state(struct ena_adapter *adapter); -static void ena_destroy_device(struct ena_adapter *adapter); +static void ena_destroy_device(struct ena_adapter *adapter, bool graceful); static int ena_restore_device(struct ena_adapter *adapter); static void ena_tx_timeout(struct net_device *dev) @@ -461,7 +461,7 @@ static inline int ena_alloc_rx_page(struct ena_ring *rx_ring, return -ENOMEM; } - dma = dma_map_page(rx_ring->dev, page, 0, PAGE_SIZE, + dma = dma_map_page(rx_ring->dev, page, 0, ENA_PAGE_SIZE, DMA_FROM_DEVICE); if (unlikely(dma_mapping_error(rx_ring->dev, dma))) { u64_stats_update_begin(&rx_ring->syncp); @@ -478,7 +478,7 @@ static inline int ena_alloc_rx_page(struct ena_ring *rx_ring, rx_info->page_offset = 0; ena_buf = &rx_info->ena_buf; ena_buf->paddr = dma; - ena_buf->len = PAGE_SIZE; + ena_buf->len = ENA_PAGE_SIZE; return 0; } @@ -495,7 +495,7 @@ static void ena_free_rx_page(struct ena_ring *rx_ring, return; } - dma_unmap_page(rx_ring->dev, ena_buf->paddr, PAGE_SIZE, + dma_unmap_page(rx_ring->dev, ena_buf->paddr, ENA_PAGE_SIZE, DMA_FROM_DEVICE); __free_page(page); @@ -551,14 +551,9 @@ static int ena_refill_rx_bufs(struct ena_ring *rx_ring, u32 num) rx_ring->qid, i, num); } - if (likely(i)) { - /* Add memory barrier to make sure the desc were written before - * issue a doorbell - */ - wmb(); - ena_com_write_sq_doorbell(rx_ring->ena_com_io_sq, true); - mmiowb(); - } + /* ena_com_write_sq_doorbell issues a wmb() */ + if (likely(i)) + ena_com_write_sq_doorbell(rx_ring->ena_com_io_sq); rx_ring->next_to_use = next_to_use; @@ -916,10 +911,10 @@ static struct sk_buff *ena_rx_skb(struct ena_ring *rx_ring, do { dma_unmap_page(rx_ring->dev, dma_unmap_addr(&rx_info->ena_buf, paddr), - PAGE_SIZE, DMA_FROM_DEVICE); + ENA_PAGE_SIZE, DMA_FROM_DEVICE); skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, rx_info->page, - rx_info->page_offset, len, PAGE_SIZE); + rx_info->page_offset, len, ENA_PAGE_SIZE); netif_dbg(rx_ring->adapter, rx_status, rx_ring->netdev, "rx skb updated. len %d. data_len %d\n", @@ -1900,7 +1895,7 @@ static int ena_close(struct net_device *netdev) "Destroy failure, restarting device\n"); ena_dump_stats_to_dmesg(adapter); /* rtnl lock already obtained in dev_ioctl() layer */ - ena_destroy_device(adapter); + ena_destroy_device(adapter, false); ena_restore_device(adapter); } @@ -2112,12 +2107,6 @@ static netdev_tx_t ena_start_xmit(struct sk_buff *skb, struct net_device *dev) tx_ring->next_to_use = ENA_TX_RING_IDX_NEXT(next_to_use, tx_ring->ring_size); - /* This WMB is aimed to: - * 1 - perform smp barrier before reading next_to_completion - * 2 - make sure the desc were written before trigger DB - */ - wmb(); - /* stop the queue when no more space available, the packet can have up * to sgl_size + 2. one for the meta descriptor and one for header * (if the header is larger than tx_max_header_size). @@ -2136,10 +2125,11 @@ static netdev_tx_t ena_start_xmit(struct sk_buff *skb, struct net_device *dev) * stop the queue but meanwhile clean_tx_irq updates * next_to_completion and terminates. * The queue will remain stopped forever. - * To solve this issue this function perform rmb, check - * the wakeup condition and wake up the queue if needed. + * To solve this issue add a mb() to make sure that + * netif_tx_stop_queue() write is vissible before checking if + * there is additional space in the queue. */ - smp_rmb(); + smp_mb(); if (ena_com_sq_empty_space(tx_ring->ena_com_io_sq) > ENA_TX_WAKEUP_THRESH) { @@ -2151,8 +2141,10 @@ static netdev_tx_t ena_start_xmit(struct sk_buff *skb, struct net_device *dev) } if (netif_xmit_stopped(txq) || !skb->xmit_more) { - /* trigger the dma engine */ - ena_com_write_sq_doorbell(tx_ring->ena_com_io_sq, false); + /* trigger the dma engine. ena_com_write_sq_doorbell() + * has a mb + */ + ena_com_write_sq_doorbell(tx_ring->ena_com_io_sq); u64_stats_update_begin(&tx_ring->syncp); tx_ring->tx_stats.doorbells++; u64_stats_update_end(&tx_ring->syncp); @@ -2550,12 +2542,15 @@ err_disable_msix: return rc; } -static void ena_destroy_device(struct ena_adapter *adapter) +static void ena_destroy_device(struct ena_adapter *adapter, bool graceful) { struct net_device *netdev = adapter->netdev; struct ena_com_dev *ena_dev = adapter->ena_dev; bool dev_up; + if (!test_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags)) + return; + netif_carrier_off(netdev); del_timer_sync(&adapter->timer_service); @@ -2563,7 +2558,8 @@ static void ena_destroy_device(struct ena_adapter *adapter) dev_up = test_bit(ENA_FLAG_DEV_UP, &adapter->flags); adapter->dev_up_before_reset = dev_up; - ena_com_set_admin_running_state(ena_dev, false); + if (!graceful) + ena_com_set_admin_running_state(ena_dev, false); if (test_bit(ENA_FLAG_DEV_UP, &adapter->flags)) ena_down(adapter); @@ -2591,6 +2587,7 @@ static void ena_destroy_device(struct ena_adapter *adapter) adapter->reset_reason = ENA_REGS_RESET_NORMAL; clear_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags); + clear_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags); } static int ena_restore_device(struct ena_adapter *adapter) @@ -2635,6 +2632,7 @@ static int ena_restore_device(struct ena_adapter *adapter) } } + set_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags); mod_timer(&adapter->timer_service, round_jiffies(jiffies + HZ)); dev_err(&pdev->dev, "Device reset completed successfully\n"); @@ -2665,7 +2663,7 @@ static void ena_fw_reset_device(struct work_struct *work) return; } rtnl_lock(); - ena_destroy_device(adapter); + ena_destroy_device(adapter, false); ena_restore_device(adapter); rtnl_unlock(); } @@ -3409,30 +3407,24 @@ static void ena_remove(struct pci_dev *pdev) netdev->rx_cpu_rmap = NULL; } #endif /* CONFIG_RFS_ACCEL */ - - unregister_netdev(netdev); del_timer_sync(&adapter->timer_service); cancel_work_sync(&adapter->reset_task); - /* Reset the device only if the device is running. */ - if (test_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags)) - ena_com_dev_reset(ena_dev, adapter->reset_reason); + unregister_netdev(netdev); - ena_free_mgmnt_irq(adapter); + /* If the device is running then we want to make sure the device will be + * reset to make sure no more events will be issued by the device. + */ + if (test_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags)) + set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags); - ena_disable_msix(adapter); + rtnl_lock(); + ena_destroy_device(adapter, true); + rtnl_unlock(); free_netdev(netdev); - ena_com_mmio_reg_read_request_destroy(ena_dev); - - ena_com_abort_admin_commands(ena_dev); - - ena_com_wait_for_abort_completion(ena_dev); - - ena_com_admin_destroy(ena_dev); - ena_com_rss_destroy(ena_dev); ena_com_delete_debug_area(ena_dev); @@ -3467,7 +3459,7 @@ static int ena_suspend(struct pci_dev *pdev, pm_message_t state) "ignoring device reset request as the device is being suspended\n"); clear_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags); } - ena_destroy_device(adapter); + ena_destroy_device(adapter, true); rtnl_unlock(); return 0; } diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.h b/drivers/net/ethernet/amazon/ena/ena_netdev.h index f1972b5ab650..7c7ae56c52cf 100644 --- a/drivers/net/ethernet/amazon/ena/ena_netdev.h +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.h @@ -355,4 +355,15 @@ void ena_dump_stats_to_buf(struct ena_adapter *adapter, u8 *buf); int ena_get_sset_count(struct net_device *netdev, int sset); +/* The ENA buffer length fields is 16 bit long. So when PAGE_SIZE == 64kB the + * driver passas 0. + * Since the max packet size the ENA handles is ~9kB limit the buffer length to + * 16kB. + */ +#if PAGE_SIZE > SZ_16K +#define ENA_PAGE_SIZE SZ_16K +#else +#define ENA_PAGE_SIZE PAGE_SIZE +#endif + #endif /* !(ENA_H) */ diff --git a/drivers/net/ethernet/apple/bmac.c b/drivers/net/ethernet/apple/bmac.c index 024998d6d8c6..6a8e2567f2bd 100644 --- a/drivers/net/ethernet/apple/bmac.c +++ b/drivers/net/ethernet/apple/bmac.c @@ -154,7 +154,7 @@ static irqreturn_t bmac_txdma_intr(int irq, void *dev_id); static irqreturn_t bmac_rxdma_intr(int irq, void *dev_id); static void bmac_set_timeout(struct net_device *dev); static void bmac_tx_timeout(struct timer_list *t); -static int bmac_output(struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t bmac_output(struct sk_buff *skb, struct net_device *dev); static void bmac_start(struct net_device *dev); #define DBDMA_SET(x) ( ((x) | (x) << 16) ) @@ -1456,7 +1456,7 @@ bmac_start(struct net_device *dev) spin_unlock_irqrestore(&bp->lock, flags); } -static int +static netdev_tx_t bmac_output(struct sk_buff *skb, struct net_device *dev) { struct bmac_data *bp = netdev_priv(dev); diff --git a/drivers/net/ethernet/apple/mace.c b/drivers/net/ethernet/apple/mace.c index 0b5429d76bcf..68b9ee489489 100644 --- a/drivers/net/ethernet/apple/mace.c +++ b/drivers/net/ethernet/apple/mace.c @@ -78,7 +78,7 @@ struct mace_data { static int mace_open(struct net_device *dev); static int mace_close(struct net_device *dev); -static int mace_xmit_start(struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t mace_xmit_start(struct sk_buff *skb, struct net_device *dev); static void mace_set_multicast(struct net_device *dev); static void mace_reset(struct net_device *dev); static int mace_set_address(struct net_device *dev, void *addr); @@ -525,7 +525,7 @@ static inline void mace_set_timeout(struct net_device *dev) mp->timeout_active = 1; } -static int mace_xmit_start(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t mace_xmit_start(struct sk_buff *skb, struct net_device *dev) { struct mace_data *mp = netdev_priv(dev); volatile struct dbdma_regs __iomem *td = mp->tx_dma; diff --git a/drivers/net/ethernet/apple/macmace.c b/drivers/net/ethernet/apple/macmace.c index 137cbb470af2..376f2c2613e7 100644 --- a/drivers/net/ethernet/apple/macmace.c +++ b/drivers/net/ethernet/apple/macmace.c @@ -89,7 +89,7 @@ struct mace_frame { static int mace_open(struct net_device *dev); static int mace_close(struct net_device *dev); -static int mace_xmit_start(struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t mace_xmit_start(struct sk_buff *skb, struct net_device *dev); static void mace_set_multicast(struct net_device *dev); static int mace_set_address(struct net_device *dev, void *addr); static void mace_reset(struct net_device *dev); @@ -444,7 +444,7 @@ static int mace_close(struct net_device *dev) * Transmit a frame */ -static int mace_xmit_start(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t mace_xmit_start(struct sk_buff *skb, struct net_device *dev) { struct mace_data *mp = netdev_priv(dev); unsigned long flags; diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c index b5f1f62e8e25..d1e1a0ba8615 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c @@ -225,9 +225,10 @@ int aq_ring_rx_clean(struct aq_ring_s *self, } /* for single fragment packets use build_skb() */ - if (buff->is_eop) { + if (buff->is_eop && + buff->len <= AQ_CFG_RX_FRAME_MAX - AQ_SKB_ALIGN) { skb = build_skb(page_address(buff->page), - buff->len + AQ_SKB_ALIGN); + AQ_CFG_RX_FRAME_MAX); if (unlikely(!skb)) { err = -ENOMEM; goto err_exit; @@ -247,18 +248,21 @@ int aq_ring_rx_clean(struct aq_ring_s *self, buff->len - ETH_HLEN, SKB_TRUESIZE(buff->len - ETH_HLEN)); - for (i = 1U, next_ = buff->next, - buff_ = &self->buff_ring[next_]; true; - next_ = buff_->next, - buff_ = &self->buff_ring[next_], ++i) { - skb_add_rx_frag(skb, i, buff_->page, 0, - buff_->len, - SKB_TRUESIZE(buff->len - - ETH_HLEN)); - buff_->is_cleaned = 1; - - if (buff_->is_eop) - break; + if (!buff->is_eop) { + for (i = 1U, next_ = buff->next, + buff_ = &self->buff_ring[next_]; + true; next_ = buff_->next, + buff_ = &self->buff_ring[next_], ++i) { + skb_add_rx_frag(skb, i, + buff_->page, 0, + buff_->len, + SKB_TRUESIZE(buff->len - + ETH_HLEN)); + buff_->is_cleaned = 1; + + if (buff_->is_eop) + break; + } } } diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c index 71362b7f6040..fcc2328bb0d9 100644 --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c @@ -12894,19 +12894,6 @@ static int bnx2x_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd) } } -#ifdef CONFIG_NET_POLL_CONTROLLER -static void poll_bnx2x(struct net_device *dev) -{ - struct bnx2x *bp = netdev_priv(dev); - int i; - - for_each_eth_queue(bp, i) { - struct bnx2x_fastpath *fp = &bp->fp[i]; - napi_schedule(&bnx2x_fp(bp, fp->index, napi)); - } -} -#endif - static int bnx2x_validate_addr(struct net_device *dev) { struct bnx2x *bp = netdev_priv(dev); @@ -13113,9 +13100,6 @@ static const struct net_device_ops bnx2x_netdev_ops = { .ndo_tx_timeout = bnx2x_tx_timeout, .ndo_vlan_rx_add_vid = bnx2x_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = bnx2x_vlan_rx_kill_vid, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = poll_bnx2x, -#endif .ndo_setup_tc = __bnx2x_setup_tc, #ifdef CONFIG_BNX2X_SRIOV .ndo_set_vf_mac = bnx2x_set_vf_mac, diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 8bb1e38b1681..61957b0bbd8c 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -5913,12 +5913,12 @@ unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp) return bp->hw_resc.max_cp_rings; } -void bnxt_set_max_func_cp_rings(struct bnxt *bp, unsigned int max) +unsigned int bnxt_get_max_func_cp_rings_for_en(struct bnxt *bp) { - bp->hw_resc.max_cp_rings = max; + return bp->hw_resc.max_cp_rings - bnxt_get_ulp_msix_num(bp); } -unsigned int bnxt_get_max_func_irqs(struct bnxt *bp) +static unsigned int bnxt_get_max_func_irqs(struct bnxt *bp) { struct bnxt_hw_resc *hw_resc = &bp->hw_resc; @@ -6684,6 +6684,8 @@ static int bnxt_hwrm_if_change(struct bnxt *bp, bool up) hw_resc->resv_rx_rings = 0; hw_resc->resv_hw_ring_grps = 0; hw_resc->resv_vnics = 0; + bp->tx_nr_rings = 0; + bp->rx_nr_rings = 0; } return rc; } @@ -7670,21 +7672,6 @@ static void bnxt_tx_timeout(struct net_device *dev) bnxt_queue_sp_work(bp); } -#ifdef CONFIG_NET_POLL_CONTROLLER -static void bnxt_poll_controller(struct net_device *dev) -{ - struct bnxt *bp = netdev_priv(dev); - int i; - - /* Only process tx rings/combined rings in netpoll mode. */ - for (i = 0; i < bp->tx_nr_rings; i++) { - struct bnxt_tx_ring_info *txr = &bp->tx_ring[i]; - - napi_schedule(&txr->bnapi->napi); - } -} -#endif - static void bnxt_timer(struct timer_list *t) { struct bnxt *bp = from_timer(bp, t, timer); @@ -8025,7 +8012,7 @@ static int bnxt_change_mac_addr(struct net_device *dev, void *p) if (ether_addr_equal(addr->sa_data, dev->dev_addr)) return 0; - rc = bnxt_approve_mac(bp, addr->sa_data); + rc = bnxt_approve_mac(bp, addr->sa_data, true); if (rc) return rc; @@ -8518,9 +8505,6 @@ static const struct net_device_ops bnxt_netdev_ops = { .ndo_set_vf_spoofchk = bnxt_set_vf_spoofchk, .ndo_set_vf_trust = bnxt_set_vf_trust, #endif -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = bnxt_poll_controller, -#endif .ndo_setup_tc = bnxt_setup_tc, #ifdef CONFIG_RFS_ACCEL .ndo_rx_flow_steer = bnxt_rx_flow_steer, @@ -8629,7 +8613,8 @@ static void _bnxt_get_max_rings(struct bnxt *bp, int *max_rx, int *max_tx, *max_tx = hw_resc->max_tx_rings; *max_rx = hw_resc->max_rx_rings; - *max_cp = min_t(int, hw_resc->max_irqs, hw_resc->max_cp_rings); + *max_cp = min_t(int, bnxt_get_max_func_cp_rings_for_en(bp), + hw_resc->max_irqs); *max_cp = min_t(int, *max_cp, hw_resc->max_stat_ctxs); max_ring_grps = hw_resc->max_hw_ring_grps; if (BNXT_CHIP_TYPE_NITRO_A0(bp) && BNXT_PF(bp)) { @@ -8769,20 +8754,25 @@ static int bnxt_init_dflt_ring_mode(struct bnxt *bp) if (bp->tx_nr_rings) return 0; + bnxt_ulp_irq_stop(bp); + bnxt_clear_int_mode(bp); rc = bnxt_set_dflt_rings(bp, true); if (rc) { netdev_err(bp->dev, "Not enough rings available.\n"); - return rc; + goto init_dflt_ring_err; } rc = bnxt_init_int_mode(bp); if (rc) - return rc; + goto init_dflt_ring_err; + bp->tx_nr_rings_per_tc = bp->tx_nr_rings; if (bnxt_rfs_supported(bp) && bnxt_rfs_capable(bp)) { bp->flags |= BNXT_FLAG_RFS; bp->dev->features |= NETIF_F_NTUPLE; } - return 0; +init_dflt_ring_err: + bnxt_ulp_irq_restart(bp, rc); + return rc; } int bnxt_restore_pf_fw_resources(struct bnxt *bp) @@ -8819,14 +8809,19 @@ static int bnxt_init_mac_addr(struct bnxt *bp) } else { #ifdef CONFIG_BNXT_SRIOV struct bnxt_vf_info *vf = &bp->vf; + bool strict_approval = true; if (is_valid_ether_addr(vf->mac_addr)) { /* overwrite netdev dev_addr with admin VF MAC */ memcpy(bp->dev->dev_addr, vf->mac_addr, ETH_ALEN); + /* Older PF driver or firmware may not approve this + * correctly. + */ + strict_approval = false; } else { eth_hw_addr_random(bp->dev); } - rc = bnxt_approve_mac(bp, bp->dev->dev_addr); + rc = bnxt_approve_mac(bp, bp->dev->dev_addr, strict_approval); #endif } return rc; diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index fefa011320e0..bde384630a75 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1481,8 +1481,7 @@ int bnxt_hwrm_set_coal(struct bnxt *); unsigned int bnxt_get_max_func_stat_ctxs(struct bnxt *bp); void bnxt_set_max_func_stat_ctxs(struct bnxt *bp, unsigned int max); unsigned int bnxt_get_max_func_cp_rings(struct bnxt *bp); -void bnxt_set_max_func_cp_rings(struct bnxt *bp, unsigned int max); -unsigned int bnxt_get_max_func_irqs(struct bnxt *bp); +unsigned int bnxt_get_max_func_cp_rings_for_en(struct bnxt *bp); int bnxt_get_avail_msix(struct bnxt *bp, int num); int bnxt_reserve_rings(struct bnxt *bp); void bnxt_tx_disable(struct bnxt *bp); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c index f3b9fbcc705b..790c684f08ab 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_devlink.c @@ -46,6 +46,9 @@ static int bnxt_hwrm_nvm_req(struct bnxt *bp, u32 param_id, void *msg, } } + if (i == ARRAY_SIZE(nvm_params)) + return -EOPNOTSUPP; + if (nvm_param.dir_type == BNXT_NVM_PORT_CFG) idx = bp->pf.port_id; else if (nvm_param.dir_type == BNXT_NVM_FUNC_CFG) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c index 6d583bcd2a81..3962f6fd543c 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c @@ -451,7 +451,7 @@ static int bnxt_hwrm_func_vf_resc_cfg(struct bnxt *bp, int num_vfs) bnxt_hwrm_cmd_hdr_init(bp, &req, HWRM_FUNC_VF_RESOURCE_CFG, -1, -1); - vf_cp_rings = hw_resc->max_cp_rings - bp->cp_nr_rings; + vf_cp_rings = bnxt_get_max_func_cp_rings_for_en(bp) - bp->cp_nr_rings; vf_stat_ctx = hw_resc->max_stat_ctxs - bp->num_stat_ctxs; if (bp->flags & BNXT_FLAG_AGG_RINGS) vf_rx_rings = hw_resc->max_rx_rings - bp->rx_nr_rings * 2; @@ -549,7 +549,8 @@ static int bnxt_hwrm_func_cfg(struct bnxt *bp, int num_vfs) max_stat_ctxs = hw_resc->max_stat_ctxs; /* Remaining rings are distributed equally amongs VF's for now */ - vf_cp_rings = (hw_resc->max_cp_rings - bp->cp_nr_rings) / num_vfs; + vf_cp_rings = (bnxt_get_max_func_cp_rings_for_en(bp) - + bp->cp_nr_rings) / num_vfs; vf_stat_ctx = (max_stat_ctxs - bp->num_stat_ctxs) / num_vfs; if (bp->flags & BNXT_FLAG_AGG_RINGS) vf_rx_rings = (hw_resc->max_rx_rings - bp->rx_nr_rings * 2) / @@ -643,7 +644,7 @@ static int bnxt_sriov_enable(struct bnxt *bp, int *num_vfs) */ vfs_supported = *num_vfs; - avail_cp = hw_resc->max_cp_rings - bp->cp_nr_rings; + avail_cp = bnxt_get_max_func_cp_rings_for_en(bp) - bp->cp_nr_rings; avail_stat = hw_resc->max_stat_ctxs - bp->num_stat_ctxs; avail_cp = min_t(int, avail_cp, avail_stat); @@ -1103,7 +1104,7 @@ update_vf_mac_exit: mutex_unlock(&bp->hwrm_cmd_lock); } -int bnxt_approve_mac(struct bnxt *bp, u8 *mac) +int bnxt_approve_mac(struct bnxt *bp, u8 *mac, bool strict) { struct hwrm_func_vf_cfg_input req = {0}; int rc = 0; @@ -1121,12 +1122,13 @@ int bnxt_approve_mac(struct bnxt *bp, u8 *mac) memcpy(req.dflt_mac_addr, mac, ETH_ALEN); rc = hwrm_send_message(bp, &req, sizeof(req), HWRM_CMD_TIMEOUT); mac_done: - if (rc) { + if (rc && strict) { rc = -EADDRNOTAVAIL; netdev_warn(bp->dev, "VF MAC address %pM not approved by the PF\n", mac); + return rc; } - return rc; + return 0; } #else @@ -1143,7 +1145,7 @@ void bnxt_update_vf_mac(struct bnxt *bp) { } -int bnxt_approve_mac(struct bnxt *bp, u8 *mac) +int bnxt_approve_mac(struct bnxt *bp, u8 *mac, bool strict) { return 0; } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h index e9b20cd19881..2eed9eda1195 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h @@ -39,5 +39,5 @@ int bnxt_sriov_configure(struct pci_dev *pdev, int num_vfs); void bnxt_sriov_disable(struct bnxt *); void bnxt_hwrm_exec_fwd_req(struct bnxt *); void bnxt_update_vf_mac(struct bnxt *); -int bnxt_approve_mac(struct bnxt *, u8 *); +int bnxt_approve_mac(struct bnxt *, u8 *, bool); #endif diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c index 092c817f8f11..e1594c9df4c6 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_tc.c @@ -75,17 +75,23 @@ static int bnxt_tc_parse_redir(struct bnxt *bp, return 0; } -static void bnxt_tc_parse_vlan(struct bnxt *bp, - struct bnxt_tc_actions *actions, - const struct tc_action *tc_act) +static int bnxt_tc_parse_vlan(struct bnxt *bp, + struct bnxt_tc_actions *actions, + const struct tc_action *tc_act) { - if (tcf_vlan_action(tc_act) == TCA_VLAN_ACT_POP) { + switch (tcf_vlan_action(tc_act)) { + case TCA_VLAN_ACT_POP: actions->flags |= BNXT_TC_ACTION_FLAG_POP_VLAN; - } else if (tcf_vlan_action(tc_act) == TCA_VLAN_ACT_PUSH) { + break; + case TCA_VLAN_ACT_PUSH: actions->flags |= BNXT_TC_ACTION_FLAG_PUSH_VLAN; actions->push_vlan_tci = htons(tcf_vlan_push_vid(tc_act)); actions->push_vlan_tpid = tcf_vlan_push_proto(tc_act); + break; + default: + return -EOPNOTSUPP; } + return 0; } static int bnxt_tc_parse_tunnel_set(struct bnxt *bp, @@ -134,7 +140,9 @@ static int bnxt_tc_parse_actions(struct bnxt *bp, /* Push/pop VLAN */ if (is_tcf_vlan(tc_act)) { - bnxt_tc_parse_vlan(bp, actions, tc_act); + rc = bnxt_tc_parse_vlan(bp, actions, tc_act); + if (rc) + return rc; continue; } diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c index c37b2842f972..beee61292d5e 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c @@ -169,7 +169,6 @@ static int bnxt_req_msix_vecs(struct bnxt_en_dev *edev, int ulp_id, edev->ulp_tbl[ulp_id].msix_requested = avail_msix; } bnxt_fill_msix_vecs(bp, ent); - bnxt_set_max_func_cp_rings(bp, max_cp_rings - avail_msix); edev->flags |= BNXT_EN_FLAG_MSIX_REQUESTED; return avail_msix; } @@ -178,7 +177,6 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, int ulp_id) { struct net_device *dev = edev->net; struct bnxt *bp = netdev_priv(dev); - int max_cp_rings, msix_requested; ASSERT_RTNL(); if (ulp_id != BNXT_ROCE_ULP) @@ -187,9 +185,6 @@ static int bnxt_free_msix_vecs(struct bnxt_en_dev *edev, int ulp_id) if (!(edev->flags & BNXT_EN_FLAG_MSIX_REQUESTED)) return 0; - max_cp_rings = bnxt_get_max_func_cp_rings(bp); - msix_requested = edev->ulp_tbl[ulp_id].msix_requested; - bnxt_set_max_func_cp_rings(bp, max_cp_rings + msix_requested); edev->ulp_tbl[ulp_id].msix_requested = 0; edev->flags &= ~BNXT_EN_FLAG_MSIX_REQUESTED; if (netif_running(dev)) { @@ -220,21 +215,6 @@ int bnxt_get_ulp_msix_base(struct bnxt *bp) return 0; } -void bnxt_subtract_ulp_resources(struct bnxt *bp, int ulp_id) -{ - ASSERT_RTNL(); - if (bnxt_ulp_registered(bp->edev, ulp_id)) { - struct bnxt_en_dev *edev = bp->edev; - unsigned int msix_req, max; - - msix_req = edev->ulp_tbl[ulp_id].msix_requested; - max = bnxt_get_max_func_cp_rings(bp); - bnxt_set_max_func_cp_rings(bp, max - msix_req); - max = bnxt_get_max_func_stat_ctxs(bp); - bnxt_set_max_func_stat_ctxs(bp, max - 1); - } -} - static int bnxt_send_msg(struct bnxt_en_dev *edev, int ulp_id, struct bnxt_fw_msg *fw_msg) { diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h index df48ac71729f..d9bea37cd211 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.h @@ -90,7 +90,6 @@ static inline bool bnxt_ulp_registered(struct bnxt_en_dev *edev, int ulp_id) int bnxt_get_ulp_msix_num(struct bnxt *bp); int bnxt_get_ulp_msix_base(struct bnxt *bp); -void bnxt_subtract_ulp_resources(struct bnxt *bp, int ulp_id); void bnxt_ulp_stop(struct bnxt *bp); void bnxt_ulp_start(struct bnxt *bp); void bnxt_ulp_sriov_cfg(struct bnxt *bp, int num_vfs); diff --git a/drivers/net/ethernet/broadcom/genet/bcmgenet.h b/drivers/net/ethernet/broadcom/genet/bcmgenet.h index b773bc07edf7..14b49612aa86 100644 --- a/drivers/net/ethernet/broadcom/genet/bcmgenet.h +++ b/drivers/net/ethernet/broadcom/genet/bcmgenet.h @@ -186,6 +186,9 @@ struct bcmgenet_mib_counters { #define UMAC_MAC1 0x010 #define UMAC_MAX_FRAME_LEN 0x014 +#define UMAC_MODE 0x44 +#define MODE_LINK_STATUS (1 << 5) + #define UMAC_EEE_CTRL 0x064 #define EN_LPI_RX_PAUSE (1 << 0) #define EN_LPI_TX_PFC (1 << 1) diff --git a/drivers/net/ethernet/broadcom/genet/bcmmii.c b/drivers/net/ethernet/broadcom/genet/bcmmii.c index 5333274a283c..4241ae928d4a 100644 --- a/drivers/net/ethernet/broadcom/genet/bcmmii.c +++ b/drivers/net/ethernet/broadcom/genet/bcmmii.c @@ -115,8 +115,14 @@ void bcmgenet_mii_setup(struct net_device *dev) static int bcmgenet_fixed_phy_link_update(struct net_device *dev, struct fixed_phy_status *status) { - if (dev && dev->phydev && status) - status->link = dev->phydev->link; + struct bcmgenet_priv *priv; + u32 reg; + + if (dev && dev->phydev && status) { + priv = netdev_priv(dev); + reg = bcmgenet_umac_readl(priv, UMAC_MODE); + status->link = !!(reg & MODE_LINK_STATUS); + } return 0; } diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c index c6707ea2d751..f1a86b422617 100644 --- a/drivers/net/ethernet/cadence/macb_main.c +++ b/drivers/net/ethernet/cadence/macb_main.c @@ -649,7 +649,7 @@ static int macb_halt_tx(struct macb *bp) if (!(status & MACB_BIT(TGO))) return 0; - usleep_range(10, 250); + udelay(250); } while (time_before(halt_time, timeout)); return -ETIMEDOUT; @@ -3837,6 +3837,13 @@ static const struct macb_config at91sam9260_config = { .init = macb_init, }; +static const struct macb_config sama5d3macb_config = { + .caps = MACB_CAPS_SG_DISABLED + | MACB_CAPS_USRIO_HAS_CLKEN | MACB_CAPS_USRIO_DEFAULT_IS_MII_GMII, + .clk_init = macb_clk_init, + .init = macb_init, +}; + static const struct macb_config pc302gem_config = { .caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE, .dma_burst_length = 16, @@ -3904,6 +3911,7 @@ static const struct of_device_id macb_dt_ids[] = { { .compatible = "cdns,gem", .data = &pc302gem_config }, { .compatible = "atmel,sama5d2-gem", .data = &sama5d2_config }, { .compatible = "atmel,sama5d3-gem", .data = &sama5d3_config }, + { .compatible = "atmel,sama5d3-macb", .data = &sama5d3macb_config }, { .compatible = "atmel,sama5d4-gem", .data = &sama5d4_config }, { .compatible = "cdns,at91rm9200-emac", .data = &emac_config }, { .compatible = "cdns,emac", .data = &emac_config }, diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h index b8f75a22fb6c..f152da1ce046 100644 --- a/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h +++ b/drivers/net/ethernet/chelsio/cxgb4/t4_msg.h @@ -753,7 +753,6 @@ struct cpl_abort_req_rss { }; struct cpl_abort_req_rss6 { - WR_HDR; union opcode_tid ot; __be32 srqidx_status; }; diff --git a/drivers/net/ethernet/cirrus/ep93xx_eth.c b/drivers/net/ethernet/cirrus/ep93xx_eth.c index e2a702996db4..13dfdfca49fc 100644 --- a/drivers/net/ethernet/cirrus/ep93xx_eth.c +++ b/drivers/net/ethernet/cirrus/ep93xx_eth.c @@ -332,7 +332,7 @@ static int ep93xx_poll(struct napi_struct *napi, int budget) return rx; } -static int ep93xx_xmit(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t ep93xx_xmit(struct sk_buff *skb, struct net_device *dev) { struct ep93xx_priv *ep = netdev_priv(dev); struct ep93xx_tdesc *txd; diff --git a/drivers/net/ethernet/cirrus/mac89x0.c b/drivers/net/ethernet/cirrus/mac89x0.c index 3f8fe8fd79cc..6324e80960c3 100644 --- a/drivers/net/ethernet/cirrus/mac89x0.c +++ b/drivers/net/ethernet/cirrus/mac89x0.c @@ -113,7 +113,7 @@ struct net_local { /* Index to functions, as function prototypes. */ static int net_open(struct net_device *dev); -static int net_send_packet(struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t net_send_packet(struct sk_buff *skb, struct net_device *dev); static irqreturn_t net_interrupt(int irq, void *dev_id); static void set_multicast_list(struct net_device *dev); static void net_rx(struct net_device *dev); @@ -324,7 +324,7 @@ net_open(struct net_device *dev) return 0; } -static int +static netdev_tx_t net_send_packet(struct sk_buff *skb, struct net_device *dev) { struct net_local *lp = netdev_priv(dev); diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.c b/drivers/net/ethernet/emulex/benet/be_cmds.c index ff92ab1daeb8..1e9d882c04ef 100644 --- a/drivers/net/ethernet/emulex/benet/be_cmds.c +++ b/drivers/net/ethernet/emulex/benet/be_cmds.c @@ -4500,7 +4500,7 @@ int be_cmd_get_profile_config(struct be_adapter *adapter, port_res->max_vfs += le16_to_cpu(pcie->num_vfs); } } - return status; + goto err; } pcie = be_get_pcie_desc(resp->func_param, desc_count, diff --git a/drivers/net/ethernet/hisilicon/hns/hnae.h b/drivers/net/ethernet/hisilicon/hns/hnae.h index cad52bd331f7..08a750fb60c4 100644 --- a/drivers/net/ethernet/hisilicon/hns/hnae.h +++ b/drivers/net/ethernet/hisilicon/hns/hnae.h @@ -486,6 +486,8 @@ struct hnae_ae_ops { u8 *auto_neg, u16 *speed, u8 *duplex); void (*toggle_ring_irq)(struct hnae_ring *ring, u32 val); void (*adjust_link)(struct hnae_handle *handle, int speed, int duplex); + bool (*need_adjust_link)(struct hnae_handle *handle, + int speed, int duplex); int (*set_loopback)(struct hnae_handle *handle, enum hnae_loop loop_mode, int en); void (*get_ring_bdnum_limit)(struct hnae_queue *queue, diff --git a/drivers/net/ethernet/hisilicon/hns/hns_ae_adapt.c b/drivers/net/ethernet/hisilicon/hns/hns_ae_adapt.c index e6aad30e7e69..b52029e26d15 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_ae_adapt.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_ae_adapt.c @@ -155,6 +155,41 @@ static void hns_ae_put_handle(struct hnae_handle *handle) hns_ae_get_ring_pair(handle->qs[i])->used_by_vf = 0; } +static int hns_ae_wait_flow_down(struct hnae_handle *handle) +{ + struct dsaf_device *dsaf_dev; + struct hns_ppe_cb *ppe_cb; + struct hnae_vf_cb *vf_cb; + int ret; + int i; + + for (i = 0; i < handle->q_num; i++) { + ret = hns_rcb_wait_tx_ring_clean(handle->qs[i]); + if (ret) + return ret; + } + + ppe_cb = hns_get_ppe_cb(handle); + ret = hns_ppe_wait_tx_fifo_clean(ppe_cb); + if (ret) + return ret; + + dsaf_dev = hns_ae_get_dsaf_dev(handle->dev); + if (!dsaf_dev) + return -EINVAL; + ret = hns_dsaf_wait_pkt_clean(dsaf_dev, handle->dport_id); + if (ret) + return ret; + + vf_cb = hns_ae_get_vf_cb(handle); + ret = hns_mac_wait_fifo_clean(vf_cb->mac_cb); + if (ret) + return ret; + + mdelay(10); + return 0; +} + static void hns_ae_ring_enable_all(struct hnae_handle *handle, int val) { int q_num = handle->q_num; @@ -399,12 +434,41 @@ static int hns_ae_get_mac_info(struct hnae_handle *handle, return hns_mac_get_port_info(mac_cb, auto_neg, speed, duplex); } +static bool hns_ae_need_adjust_link(struct hnae_handle *handle, int speed, + int duplex) +{ + struct hns_mac_cb *mac_cb = hns_get_mac_cb(handle); + + return hns_mac_need_adjust_link(mac_cb, speed, duplex); +} + static void hns_ae_adjust_link(struct hnae_handle *handle, int speed, int duplex) { struct hns_mac_cb *mac_cb = hns_get_mac_cb(handle); - hns_mac_adjust_link(mac_cb, speed, duplex); + switch (mac_cb->dsaf_dev->dsaf_ver) { + case AE_VERSION_1: + hns_mac_adjust_link(mac_cb, speed, duplex); + break; + + case AE_VERSION_2: + /* chip need to clear all pkt inside */ + hns_mac_disable(mac_cb, MAC_COMM_MODE_RX); + if (hns_ae_wait_flow_down(handle)) { + hns_mac_enable(mac_cb, MAC_COMM_MODE_RX); + break; + } + + hns_mac_adjust_link(mac_cb, speed, duplex); + hns_mac_enable(mac_cb, MAC_COMM_MODE_RX); + break; + + default: + break; + } + + return; } static void hns_ae_get_ring_bdnum_limit(struct hnae_queue *queue, @@ -902,6 +966,7 @@ static struct hnae_ae_ops hns_dsaf_ops = { .get_status = hns_ae_get_link_status, .get_info = hns_ae_get_mac_info, .adjust_link = hns_ae_adjust_link, + .need_adjust_link = hns_ae_need_adjust_link, .set_loopback = hns_ae_config_loopback, .get_ring_bdnum_limit = hns_ae_get_ring_bdnum_limit, .get_pauseparam = hns_ae_get_pauseparam, diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_gmac.c b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_gmac.c index 5488c6e89f21..09e4061d1fa6 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_gmac.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_gmac.c @@ -257,6 +257,16 @@ static void hns_gmac_get_pausefrm_cfg(void *mac_drv, u32 *rx_pause_en, *tx_pause_en = dsaf_get_bit(pause_en, GMAC_PAUSE_EN_TX_FDFC_B); } +static bool hns_gmac_need_adjust_link(void *mac_drv, enum mac_speed speed, + int duplex) +{ + struct mac_driver *drv = (struct mac_driver *)mac_drv; + struct hns_mac_cb *mac_cb = drv->mac_cb; + + return (mac_cb->speed != speed) || + (mac_cb->half_duplex == duplex); +} + static int hns_gmac_adjust_link(void *mac_drv, enum mac_speed speed, u32 full_duplex) { @@ -309,6 +319,30 @@ static void hns_gmac_set_promisc(void *mac_drv, u8 en) hns_gmac_set_uc_match(mac_drv, en); } +int hns_gmac_wait_fifo_clean(void *mac_drv) +{ + struct mac_driver *drv = (struct mac_driver *)mac_drv; + int wait_cnt; + u32 val; + + wait_cnt = 0; + while (wait_cnt++ < HNS_MAX_WAIT_CNT) { + val = dsaf_read_dev(drv, GMAC_FIFO_STATE_REG); + /* bit5~bit0 is not send complete pkts */ + if ((val & 0x3f) == 0) + break; + usleep_range(100, 200); + } + + if (wait_cnt >= HNS_MAX_WAIT_CNT) { + dev_err(drv->dev, + "hns ge %d fifo was not idle.\n", drv->mac_id); + return -EBUSY; + } + + return 0; +} + static void hns_gmac_init(void *mac_drv) { u32 port; @@ -690,6 +724,7 @@ void *hns_gmac_config(struct hns_mac_cb *mac_cb, struct mac_params *mac_param) mac_drv->mac_disable = hns_gmac_disable; mac_drv->mac_free = hns_gmac_free; mac_drv->adjust_link = hns_gmac_adjust_link; + mac_drv->need_adjust_link = hns_gmac_need_adjust_link; mac_drv->set_tx_auto_pause_frames = hns_gmac_set_tx_auto_pause_frames; mac_drv->config_max_frame_length = hns_gmac_config_max_frame_length; mac_drv->mac_pausefrm_cfg = hns_gmac_pause_frm_cfg; @@ -717,6 +752,7 @@ void *hns_gmac_config(struct hns_mac_cb *mac_cb, struct mac_params *mac_param) mac_drv->get_strings = hns_gmac_get_strings; mac_drv->update_stats = hns_gmac_update_stats; mac_drv->set_promiscuous = hns_gmac_set_promisc; + mac_drv->wait_fifo_clean = hns_gmac_wait_fifo_clean; return (void *)mac_drv; } diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c index 1c2326bd76e2..6ed6f142427e 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c @@ -114,6 +114,26 @@ int hns_mac_get_port_info(struct hns_mac_cb *mac_cb, return 0; } +/** + *hns_mac_is_adjust_link - check is need change mac speed and duplex register + *@mac_cb: mac device + *@speed: phy device speed + *@duplex:phy device duplex + * + */ +bool hns_mac_need_adjust_link(struct hns_mac_cb *mac_cb, int speed, int duplex) +{ + struct mac_driver *mac_ctrl_drv; + + mac_ctrl_drv = (struct mac_driver *)(mac_cb->priv.mac); + + if (mac_ctrl_drv->need_adjust_link) + return mac_ctrl_drv->need_adjust_link(mac_ctrl_drv, + (enum mac_speed)speed, duplex); + else + return true; +} + void hns_mac_adjust_link(struct hns_mac_cb *mac_cb, int speed, int duplex) { int ret; @@ -430,6 +450,16 @@ int hns_mac_vm_config_bc_en(struct hns_mac_cb *mac_cb, u32 vmid, bool enable) return 0; } +int hns_mac_wait_fifo_clean(struct hns_mac_cb *mac_cb) +{ + struct mac_driver *drv = hns_mac_get_drv(mac_cb); + + if (drv->wait_fifo_clean) + return drv->wait_fifo_clean(drv); + + return 0; +} + void hns_mac_reset(struct hns_mac_cb *mac_cb) { struct mac_driver *drv = hns_mac_get_drv(mac_cb); @@ -998,6 +1028,20 @@ static int hns_mac_get_max_port_num(struct dsaf_device *dsaf_dev) return DSAF_MAX_PORT_NUM; } +void hns_mac_enable(struct hns_mac_cb *mac_cb, enum mac_commom_mode mode) +{ + struct mac_driver *mac_ctrl_drv = hns_mac_get_drv(mac_cb); + + mac_ctrl_drv->mac_enable(mac_cb->priv.mac, mode); +} + +void hns_mac_disable(struct hns_mac_cb *mac_cb, enum mac_commom_mode mode) +{ + struct mac_driver *mac_ctrl_drv = hns_mac_get_drv(mac_cb); + + mac_ctrl_drv->mac_disable(mac_cb->priv.mac, mode); +} + /** * hns_mac_init - init mac * @dsaf_dev: dsa fabric device struct pointer diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.h b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.h index bbc0a98e7ca3..fbc75341bef7 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.h +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.h @@ -356,6 +356,9 @@ struct mac_driver { /*adjust mac mode of port,include speed and duplex*/ int (*adjust_link)(void *mac_drv, enum mac_speed speed, u32 full_duplex); + /* need adjust link */ + bool (*need_adjust_link)(void *mac_drv, enum mac_speed speed, + int duplex); /* config autoegotaite mode of port*/ void (*set_an_mode)(void *mac_drv, u8 enable); /* config loopbank mode */ @@ -394,6 +397,7 @@ struct mac_driver { void (*get_info)(void *mac_drv, struct mac_info *mac_info); void (*update_stats)(void *mac_drv); + int (*wait_fifo_clean)(void *mac_drv); enum mac_mode mac_mode; u8 mac_id; @@ -427,6 +431,7 @@ void *hns_xgmac_config(struct hns_mac_cb *mac_cb, int hns_mac_init(struct dsaf_device *dsaf_dev); void mac_adjust_link(struct net_device *net_dev); +bool hns_mac_need_adjust_link(struct hns_mac_cb *mac_cb, int speed, int duplex); void hns_mac_get_link_status(struct hns_mac_cb *mac_cb, u32 *link_status); int hns_mac_change_vf_addr(struct hns_mac_cb *mac_cb, u32 vmid, char *addr); int hns_mac_set_multi(struct hns_mac_cb *mac_cb, @@ -463,5 +468,8 @@ int hns_mac_add_uc_addr(struct hns_mac_cb *mac_cb, u8 vf_id, int hns_mac_rm_uc_addr(struct hns_mac_cb *mac_cb, u8 vf_id, const unsigned char *addr); int hns_mac_clr_multicast(struct hns_mac_cb *mac_cb, int vfn); +void hns_mac_enable(struct hns_mac_cb *mac_cb, enum mac_commom_mode mode); +void hns_mac_disable(struct hns_mac_cb *mac_cb, enum mac_commom_mode mode); +int hns_mac_wait_fifo_clean(struct hns_mac_cb *mac_cb); #endif /* _HNS_DSAF_MAC_H */ diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c index ca50c2553a9c..e557a4ef5996 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c @@ -2727,6 +2727,35 @@ void hns_dsaf_set_promisc_tcam(struct dsaf_device *dsaf_dev, soft_mac_entry->index = enable ? entry_index : DSAF_INVALID_ENTRY_IDX; } +int hns_dsaf_wait_pkt_clean(struct dsaf_device *dsaf_dev, int port) +{ + u32 val, val_tmp; + int wait_cnt; + + if (port >= DSAF_SERVICE_NW_NUM) + return 0; + + wait_cnt = 0; + while (wait_cnt++ < HNS_MAX_WAIT_CNT) { + val = dsaf_read_dev(dsaf_dev, DSAF_VOQ_IN_PKT_NUM_0_REG + + (port + DSAF_XGE_NUM) * 0x40); + val_tmp = dsaf_read_dev(dsaf_dev, DSAF_VOQ_OUT_PKT_NUM_0_REG + + (port + DSAF_XGE_NUM) * 0x40); + if (val == val_tmp) + break; + + usleep_range(100, 200); + } + + if (wait_cnt >= HNS_MAX_WAIT_CNT) { + dev_err(dsaf_dev->dev, "hns dsaf clean wait timeout(%u - %u).\n", + val, val_tmp); + return -EBUSY; + } + + return 0; +} + /** * dsaf_probe - probo dsaf dev * @pdev: dasf platform device diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.h b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.h index 4507e8222683..0e1cd99831a6 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.h +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.h @@ -44,6 +44,8 @@ struct hns_mac_cb; #define DSAF_ROCE_CREDIT_CHN 8 #define DSAF_ROCE_CHAN_MODE 3 +#define HNS_MAX_WAIT_CNT 10000 + enum dsaf_roce_port_mode { DSAF_ROCE_6PORT_MODE, DSAF_ROCE_4PORT_MODE, @@ -463,5 +465,6 @@ int hns_dsaf_rm_mac_addr( int hns_dsaf_clr_mac_mc_port(struct dsaf_device *dsaf_dev, u8 mac_id, u8 port_num); +int hns_dsaf_wait_pkt_clean(struct dsaf_device *dsaf_dev, int port); #endif /* __HNS_DSAF_MAIN_H__ */ diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.c b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.c index d160d8c9e45b..0942e4916d9d 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.c @@ -275,6 +275,29 @@ static void hns_ppe_exc_irq_en(struct hns_ppe_cb *ppe_cb, int en) dsaf_write_dev(ppe_cb, PPE_INTEN_REG, msk_vlue & vld_msk); } +int hns_ppe_wait_tx_fifo_clean(struct hns_ppe_cb *ppe_cb) +{ + int wait_cnt; + u32 val; + + wait_cnt = 0; + while (wait_cnt++ < HNS_MAX_WAIT_CNT) { + val = dsaf_read_dev(ppe_cb, PPE_CURR_TX_FIFO0_REG) & 0x3ffU; + if (!val) + break; + + usleep_range(100, 200); + } + + if (wait_cnt >= HNS_MAX_WAIT_CNT) { + dev_err(ppe_cb->dev, "hns ppe tx fifo clean wait timeout, still has %u pkt.\n", + val); + return -EBUSY; + } + + return 0; +} + /** * ppe_init_hw - init ppe * @ppe_cb: ppe device diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.h b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.h index 9d8e643e8aa6..f670e63a5a01 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.h +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.h @@ -100,6 +100,7 @@ struct ppe_common_cb { }; +int hns_ppe_wait_tx_fifo_clean(struct hns_ppe_cb *ppe_cb); int hns_ppe_init(struct dsaf_device *dsaf_dev); void hns_ppe_uninit(struct dsaf_device *dsaf_dev); diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.c b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.c index 9d76e2e54f9d..5d64519b9b1d 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.c @@ -66,6 +66,29 @@ void hns_rcb_wait_fbd_clean(struct hnae_queue **qs, int q_num, u32 flag) "queue(%d) wait fbd(%d) clean fail!!\n", i, fbd_num); } +int hns_rcb_wait_tx_ring_clean(struct hnae_queue *qs) +{ + u32 head, tail; + int wait_cnt; + + tail = dsaf_read_dev(&qs->tx_ring, RCB_REG_TAIL); + wait_cnt = 0; + while (wait_cnt++ < HNS_MAX_WAIT_CNT) { + head = dsaf_read_dev(&qs->tx_ring, RCB_REG_HEAD); + if (tail == head) + break; + + usleep_range(100, 200); + } + + if (wait_cnt >= HNS_MAX_WAIT_CNT) { + dev_err(qs->dev->dev, "rcb wait timeout, head not equal to tail.\n"); + return -EBUSY; + } + + return 0; +} + /** *hns_rcb_reset_ring_hw - ring reset *@q: ring struct pointer diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.h b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.h index 602816498c8d..2319b772a271 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.h +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.h @@ -136,6 +136,7 @@ void hns_rcbv2_int_clr_hw(struct hnae_queue *q, u32 flag); void hns_rcb_init_hw(struct ring_pair_cb *ring); void hns_rcb_reset_ring_hw(struct hnae_queue *q); void hns_rcb_wait_fbd_clean(struct hnae_queue **qs, int q_num, u32 flag); +int hns_rcb_wait_tx_ring_clean(struct hnae_queue *qs); u32 hns_rcb_get_rx_coalesced_frames( struct rcb_common_cb *rcb_common, u32 port_idx); u32 hns_rcb_get_tx_coalesced_frames( diff --git a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_reg.h b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_reg.h index 886cbbf25761..74d935d82cbc 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_dsaf_reg.h +++ b/drivers/net/ethernet/hisilicon/hns/hns_dsaf_reg.h @@ -464,6 +464,7 @@ #define RCB_RING_INTMSK_TX_OVERTIME_REG 0x000C4 #define RCB_RING_INTSTS_TX_OVERTIME_REG 0x000C8 +#define GMAC_FIFO_STATE_REG 0x0000UL #define GMAC_DUPLEX_TYPE_REG 0x0008UL #define GMAC_FD_FC_TYPE_REG 0x000CUL #define GMAC_TX_WATER_LINE_REG 0x0010UL diff --git a/drivers/net/ethernet/hisilicon/hns/hns_enet.c b/drivers/net/ethernet/hisilicon/hns/hns_enet.c index 02a0ba20fad5..f56855e63c96 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_enet.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_enet.c @@ -1112,11 +1112,26 @@ static void hns_nic_adjust_link(struct net_device *ndev) struct hnae_handle *h = priv->ae_handle; int state = 1; + /* If there is no phy, do not need adjust link */ if (ndev->phydev) { - h->dev->ops->adjust_link(h, ndev->phydev->speed, - ndev->phydev->duplex); - state = ndev->phydev->link; + /* When phy link down, do nothing */ + if (ndev->phydev->link == 0) + return; + + if (h->dev->ops->need_adjust_link(h, ndev->phydev->speed, + ndev->phydev->duplex)) { + /* because Hi161X chip don't support to change gmac + * speed and duplex with traffic. Delay 200ms to + * make sure there is no more data in chip FIFO. + */ + netif_carrier_off(ndev); + msleep(200); + h->dev->ops->adjust_link(h, ndev->phydev->speed, + ndev->phydev->duplex); + netif_carrier_on(ndev); + } } + state = state && h->dev->ops->get_status(h); if (state != priv->link) { diff --git a/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c b/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c index 08f3c4743f74..774beda040a1 100644 --- a/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c +++ b/drivers/net/ethernet/hisilicon/hns/hns_ethtool.c @@ -243,7 +243,9 @@ static int hns_nic_set_link_ksettings(struct net_device *net_dev, } if (h->dev->ops->adjust_link) { + netif_carrier_off(net_dev); h->dev->ops->adjust_link(h, (int)speed, cmd->base.duplex); + netif_carrier_on(net_dev); return 0; } diff --git a/drivers/net/ethernet/hp/hp100.c b/drivers/net/ethernet/hp/hp100.c index c8c7ad2eff77..9b5a68b65432 100644 --- a/drivers/net/ethernet/hp/hp100.c +++ b/drivers/net/ethernet/hp/hp100.c @@ -2634,7 +2634,7 @@ static int hp100_login_to_vg_hub(struct net_device *dev, u_short force_relogin) /* Wait for link to drop */ time = jiffies + (HZ / 10); do { - if (~(hp100_inb(VG_LAN_CFG_1) & HP100_LINK_UP_ST)) + if (!(hp100_inb(VG_LAN_CFG_1) & HP100_LINK_UP_ST)) break; if (!in_interrupt()) schedule_timeout_interruptible(1); diff --git a/drivers/net/ethernet/i825xx/ether1.c b/drivers/net/ethernet/i825xx/ether1.c index dc983450354b..35f6291a3672 100644 --- a/drivers/net/ethernet/i825xx/ether1.c +++ b/drivers/net/ethernet/i825xx/ether1.c @@ -64,7 +64,8 @@ static unsigned int net_debug = NET_DEBUG; #define RX_AREA_END 0x0fc00 static int ether1_open(struct net_device *dev); -static int ether1_sendpacket(struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t ether1_sendpacket(struct sk_buff *skb, + struct net_device *dev); static irqreturn_t ether1_interrupt(int irq, void *dev_id); static int ether1_close(struct net_device *dev); static void ether1_setmulticastlist(struct net_device *dev); @@ -667,7 +668,7 @@ ether1_timeout(struct net_device *dev) netif_wake_queue(dev); } -static int +static netdev_tx_t ether1_sendpacket (struct sk_buff *skb, struct net_device *dev) { int tmp, tst, nopaddr, txaddr, tbdaddr, dataddr; diff --git a/drivers/net/ethernet/i825xx/lib82596.c b/drivers/net/ethernet/i825xx/lib82596.c index f00a1dc2128c..2f7ae118217f 100644 --- a/drivers/net/ethernet/i825xx/lib82596.c +++ b/drivers/net/ethernet/i825xx/lib82596.c @@ -347,7 +347,7 @@ static const char init_setup[] = 0x7f /* *multi IA */ }; static int i596_open(struct net_device *dev); -static int i596_start_xmit(struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t i596_start_xmit(struct sk_buff *skb, struct net_device *dev); static irqreturn_t i596_interrupt(int irq, void *dev_id); static int i596_close(struct net_device *dev); static void i596_add_cmd(struct net_device *dev, struct i596_cmd *cmd); @@ -966,7 +966,7 @@ static void i596_tx_timeout (struct net_device *dev) } -static int i596_start_xmit(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t i596_start_xmit(struct sk_buff *skb, struct net_device *dev) { struct i596_private *lp = netdev_priv(dev); struct tx_cmd *tx_cmd; diff --git a/drivers/net/ethernet/i825xx/sun3_82586.c b/drivers/net/ethernet/i825xx/sun3_82586.c index 8bb15a8c2a40..1a86184d44c0 100644 --- a/drivers/net/ethernet/i825xx/sun3_82586.c +++ b/drivers/net/ethernet/i825xx/sun3_82586.c @@ -121,7 +121,8 @@ static int sun3_82586_probe1(struct net_device *dev,int ioaddr); static irqreturn_t sun3_82586_interrupt(int irq,void *dev_id); static int sun3_82586_open(struct net_device *dev); static int sun3_82586_close(struct net_device *dev); -static int sun3_82586_send_packet(struct sk_buff *,struct net_device *); +static netdev_tx_t sun3_82586_send_packet(struct sk_buff *, + struct net_device *); static struct net_device_stats *sun3_82586_get_stats(struct net_device *dev); static void set_multicast_list(struct net_device *dev); static void sun3_82586_timeout(struct net_device *dev); @@ -1002,7 +1003,8 @@ static void sun3_82586_timeout(struct net_device *dev) * send frame */ -static int sun3_82586_send_packet(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t +sun3_82586_send_packet(struct sk_buff *skb, struct net_device *dev) { int len,i; #ifndef NO_NOPCOMMANDS diff --git a/drivers/net/ethernet/ibm/emac/core.c b/drivers/net/ethernet/ibm/emac/core.c index 354c0982847b..129f4e9f38da 100644 --- a/drivers/net/ethernet/ibm/emac/core.c +++ b/drivers/net/ethernet/ibm/emac/core.c @@ -494,9 +494,6 @@ static u32 __emac_calc_base_mr1(struct emac_instance *dev, int tx_size, int rx_s case 16384: ret |= EMAC_MR1_RFS_16K; break; - case 8192: - ret |= EMAC4_MR1_RFS_8K; - break; case 4096: ret |= EMAC_MR1_RFS_4K; break; @@ -537,6 +534,9 @@ static u32 __emac4_calc_base_mr1(struct emac_instance *dev, int tx_size, int rx_ case 16384: ret |= EMAC4_MR1_RFS_16K; break; + case 8192: + ret |= EMAC4_MR1_RFS_8K; + break; case 4096: ret |= EMAC4_MR1_RFS_4K; break; @@ -2677,12 +2677,17 @@ static int emac_init_phy(struct emac_instance *dev) if (of_phy_is_fixed_link(np)) { int res = emac_dt_mdio_probe(dev); - if (!res) { - res = of_phy_register_fixed_link(np); - if (res) - mdiobus_unregister(dev->mii_bus); + if (res) + return res; + + res = of_phy_register_fixed_link(np); + dev->phy_dev = of_phy_find_device(np); + if (res || !dev->phy_dev) { + mdiobus_unregister(dev->mii_bus); + return res ? res : -EINVAL; } - return res; + emac_adjust_link(dev->ndev); + put_device(&dev->phy_dev->mdio.dev); } return 0; } diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index dafdd4ade705..4f0daf67b18d 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -1823,11 +1823,17 @@ static int do_reset(struct ibmvnic_adapter *adapter, adapter->map_id = 1; release_rx_pools(adapter); release_tx_pools(adapter); - init_rx_pools(netdev); - init_tx_pools(netdev); + rc = init_rx_pools(netdev); + if (rc) + return rc; + rc = init_tx_pools(netdev); + if (rc) + return rc; release_napi(adapter); - init_napi(adapter); + rc = init_napi(adapter); + if (rc) + return rc; } else { rc = reset_tx_pools(adapter); if (rc) diff --git a/drivers/net/ethernet/intel/fm10k/fm10k.h b/drivers/net/ethernet/intel/fm10k/fm10k.h index a903a0ba45e1..7d42582ed48d 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k.h +++ b/drivers/net/ethernet/intel/fm10k/fm10k.h @@ -504,9 +504,6 @@ void fm10k_update_stats(struct fm10k_intfc *interface); void fm10k_service_event_schedule(struct fm10k_intfc *interface); void fm10k_macvlan_schedule(struct fm10k_intfc *interface); void fm10k_update_rx_drop_en(struct fm10k_intfc *interface); -#ifdef CONFIG_NET_POLL_CONTROLLER -void fm10k_netpoll(struct net_device *netdev); -#endif /* Netdev */ struct net_device *fm10k_alloc_netdev(const struct fm10k_info *info); diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c index 929f538d28bc..538a8467f434 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c +++ b/drivers/net/ethernet/intel/fm10k/fm10k_netdev.c @@ -1648,9 +1648,6 @@ static const struct net_device_ops fm10k_netdev_ops = { .ndo_udp_tunnel_del = fm10k_udp_tunnel_del, .ndo_dfwd_add_station = fm10k_dfwd_add_station, .ndo_dfwd_del_station = fm10k_dfwd_del_station, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = fm10k_netpoll, -#endif .ndo_features_check = fm10k_features_check, }; diff --git a/drivers/net/ethernet/intel/fm10k/fm10k_pci.c b/drivers/net/ethernet/intel/fm10k/fm10k_pci.c index 15071e4adb98..c859ababeed5 100644 --- a/drivers/net/ethernet/intel/fm10k/fm10k_pci.c +++ b/drivers/net/ethernet/intel/fm10k/fm10k_pci.c @@ -1210,28 +1210,6 @@ static irqreturn_t fm10k_msix_mbx_vf(int __always_unused irq, void *data) return IRQ_HANDLED; } -#ifdef CONFIG_NET_POLL_CONTROLLER -/** - * fm10k_netpoll - A Polling 'interrupt' handler - * @netdev: network interface device structure - * - * This is used by netconsole to send skbs without having to re-enable - * interrupts. It's not called while the normal interrupt routine is executing. - **/ -void fm10k_netpoll(struct net_device *netdev) -{ - struct fm10k_intfc *interface = netdev_priv(netdev); - int i; - - /* if interface is down do nothing */ - if (test_bit(__FM10K_DOWN, interface->state)) - return; - - for (i = 0; i < interface->num_q_vectors; i++) - fm10k_msix_clean_rings(0, interface->q_vector[i]); -} - -#endif #define FM10K_ERR_MSG(type) case (type): error = #type; break static void fm10k_handle_fault(struct fm10k_intfc *interface, int type, struct fm10k_fault *fault) diff --git a/drivers/net/ethernet/intel/i40evf/i40evf_main.c b/drivers/net/ethernet/intel/i40evf/i40evf_main.c index 5906c1c1d19d..fef6d892ed4c 100644 --- a/drivers/net/ethernet/intel/i40evf/i40evf_main.c +++ b/drivers/net/ethernet/intel/i40evf/i40evf_main.c @@ -396,29 +396,6 @@ static void i40evf_map_rings_to_vectors(struct i40evf_adapter *adapter) adapter->aq_required |= I40EVF_FLAG_AQ_MAP_VECTORS; } -#ifdef CONFIG_NET_POLL_CONTROLLER -/** - * i40evf_netpoll - A Polling 'interrupt' handler - * @netdev: network interface device structure - * - * This is used by netconsole to send skbs without having to re-enable - * interrupts. It's not called while the normal interrupt routine is executing. - **/ -static void i40evf_netpoll(struct net_device *netdev) -{ - struct i40evf_adapter *adapter = netdev_priv(netdev); - int q_vectors = adapter->num_msix_vectors - NONQ_VECS; - int i; - - /* if interface is down do nothing */ - if (test_bit(__I40E_VSI_DOWN, adapter->vsi.state)) - return; - - for (i = 0; i < q_vectors; i++) - i40evf_msix_clean_rings(0, &adapter->q_vectors[i]); -} - -#endif /** * i40evf_irq_affinity_notify - Callback for affinity changes * @notify: context as to what irq was changed @@ -3229,9 +3206,6 @@ static const struct net_device_ops i40evf_netdev_ops = { .ndo_features_check = i40evf_features_check, .ndo_fix_features = i40evf_fix_features, .ndo_set_features = i40evf_set_features, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = i40evf_netpoll, -#endif .ndo_setup_tc = i40evf_setup_tc, }; diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c index f1e80eed2fd6..3f047bb43348 100644 --- a/drivers/net/ethernet/intel/ice/ice_main.c +++ b/drivers/net/ethernet/intel/ice/ice_main.c @@ -4806,30 +4806,6 @@ void ice_get_stats64(struct net_device *netdev, struct rtnl_link_stats64 *stats) stats->rx_length_errors = vsi_stats->rx_length_errors; } -#ifdef CONFIG_NET_POLL_CONTROLLER -/** - * ice_netpoll - polling "interrupt" handler - * @netdev: network interface device structure - * - * Used by netconsole to send skbs without having to re-enable interrupts. - * This is not called in the normal interrupt path. - */ -static void ice_netpoll(struct net_device *netdev) -{ - struct ice_netdev_priv *np = netdev_priv(netdev); - struct ice_vsi *vsi = np->vsi; - struct ice_pf *pf = vsi->back; - int i; - - if (test_bit(__ICE_DOWN, vsi->state) || - !test_bit(ICE_FLAG_MSIX_ENA, pf->flags)) - return; - - for (i = 0; i < vsi->num_q_vectors; i++) - ice_msix_clean_rings(0, vsi->q_vectors[i]); -} -#endif /* CONFIG_NET_POLL_CONTROLLER */ - /** * ice_napi_disable_all - Disable NAPI for all q_vectors in the VSI * @vsi: VSI having NAPI disabled @@ -5497,9 +5473,6 @@ static const struct net_device_ops ice_netdev_ops = { .ndo_validate_addr = eth_validate_addr, .ndo_change_mtu = ice_change_mtu, .ndo_get_stats64 = ice_get_stats64, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = ice_netpoll, -#endif /* CONFIG_NET_POLL_CONTROLLER */ .ndo_vlan_rx_add_vid = ice_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = ice_vlan_rx_kill_vid, .ndo_set_features = ice_set_features, diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index a32c576c1e65..0796cef96fa3 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -205,10 +205,6 @@ static struct notifier_block dca_notifier = { .priority = 0 }; #endif -#ifdef CONFIG_NET_POLL_CONTROLLER -/* for netdump / net console */ -static void igb_netpoll(struct net_device *); -#endif #ifdef CONFIG_PCI_IOV static unsigned int max_vfs; module_param(max_vfs, uint, 0); @@ -2881,9 +2877,6 @@ static const struct net_device_ops igb_netdev_ops = { .ndo_set_vf_spoofchk = igb_ndo_set_vf_spoofchk, .ndo_set_vf_trust = igb_ndo_set_vf_trust, .ndo_get_vf_config = igb_ndo_get_vf_config, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = igb_netpoll, -#endif .ndo_fix_features = igb_fix_features, .ndo_set_features = igb_set_features, .ndo_fdb_add = igb_ndo_fdb_add, @@ -9053,29 +9046,6 @@ static int igb_pci_sriov_configure(struct pci_dev *dev, int num_vfs) return 0; } -#ifdef CONFIG_NET_POLL_CONTROLLER -/* Polling 'interrupt' - used by things like netconsole to send skbs - * without having to re-enable interrupts. It's not called while - * the interrupt routine is executing. - */ -static void igb_netpoll(struct net_device *netdev) -{ - struct igb_adapter *adapter = netdev_priv(netdev); - struct e1000_hw *hw = &adapter->hw; - struct igb_q_vector *q_vector; - int i; - - for (i = 0; i < adapter->num_q_vectors; i++) { - q_vector = adapter->q_vector[i]; - if (adapter->flags & IGB_FLAG_HAS_MSIX) - wr32(E1000_EIMC, q_vector->eims_value); - else - igb_irq_disable(adapter); - napi_schedule(&q_vector->napi); - } -} -#endif /* CONFIG_NET_POLL_CONTROLLER */ - /** * igb_io_error_detected - called when PCI error is detected * @pdev: Pointer to PCI device diff --git a/drivers/net/ethernet/intel/ixgb/ixgb_main.c b/drivers/net/ethernet/intel/ixgb/ixgb_main.c index d3e72d0f66ef..7722153c4ac2 100644 --- a/drivers/net/ethernet/intel/ixgb/ixgb_main.c +++ b/drivers/net/ethernet/intel/ixgb/ixgb_main.c @@ -81,11 +81,6 @@ static int ixgb_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, u16 vid); static void ixgb_restore_vlan(struct ixgb_adapter *adapter); -#ifdef CONFIG_NET_POLL_CONTROLLER -/* for netdump / net console */ -static void ixgb_netpoll(struct net_device *dev); -#endif - static pci_ers_result_t ixgb_io_error_detected (struct pci_dev *pdev, enum pci_channel_state state); static pci_ers_result_t ixgb_io_slot_reset (struct pci_dev *pdev); @@ -348,9 +343,6 @@ static const struct net_device_ops ixgb_netdev_ops = { .ndo_tx_timeout = ixgb_tx_timeout, .ndo_vlan_rx_add_vid = ixgb_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = ixgb_vlan_rx_kill_vid, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = ixgb_netpoll, -#endif .ndo_fix_features = ixgb_fix_features, .ndo_set_features = ixgb_set_features, }; @@ -2195,23 +2187,6 @@ ixgb_restore_vlan(struct ixgb_adapter *adapter) ixgb_vlan_rx_add_vid(adapter->netdev, htons(ETH_P_8021Q), vid); } -#ifdef CONFIG_NET_POLL_CONTROLLER -/* - * Polling 'interrupt' - used by things like netconsole to send skbs - * without having to re-enable interrupts. It's not called while - * the interrupt routine is executing. - */ - -static void ixgb_netpoll(struct net_device *dev) -{ - struct ixgb_adapter *adapter = netdev_priv(dev); - - disable_irq(adapter->pdev->irq); - ixgb_intr(adapter->pdev->irq, dev); - enable_irq(adapter->pdev->irq); -} -#endif - /** * ixgb_io_error_detected - called when PCI error is detected * @pdev: pointer to pci device with error diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index 9a23d33a47ed..f27d73a7bf16 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -8768,28 +8768,6 @@ static int ixgbe_del_sanmac_netdev(struct net_device *dev) return err; } -#ifdef CONFIG_NET_POLL_CONTROLLER -/* - * Polling 'interrupt' - used by things like netconsole to send skbs - * without having to re-enable interrupts. It's not called while - * the interrupt routine is executing. - */ -static void ixgbe_netpoll(struct net_device *netdev) -{ - struct ixgbe_adapter *adapter = netdev_priv(netdev); - int i; - - /* if interface is down do nothing */ - if (test_bit(__IXGBE_DOWN, &adapter->state)) - return; - - /* loop through and schedule all active queues */ - for (i = 0; i < adapter->num_q_vectors; i++) - ixgbe_msix_clean_rings(0, adapter->q_vector[i]); -} - -#endif - static void ixgbe_get_ring_stats64(struct rtnl_link_stats64 *stats, struct ixgbe_ring *ring) { @@ -10251,9 +10229,6 @@ static const struct net_device_ops ixgbe_netdev_ops = { .ndo_get_vf_config = ixgbe_ndo_get_vf_config, .ndo_get_stats64 = ixgbe_get_stats64, .ndo_setup_tc = __ixgbe_setup_tc, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = ixgbe_netpoll, -#endif #ifdef IXGBE_FCOE .ndo_select_queue = ixgbe_select_queue, .ndo_fcoe_ddp_setup = ixgbe_fcoe_ddp_get, diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index d86446d202d5..5a228582423b 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -4233,24 +4233,6 @@ static int ixgbevf_change_mtu(struct net_device *netdev, int new_mtu) return 0; } -#ifdef CONFIG_NET_POLL_CONTROLLER -/* Polling 'interrupt' - used by things like netconsole to send skbs - * without having to re-enable interrupts. It's not called while - * the interrupt routine is executing. - */ -static void ixgbevf_netpoll(struct net_device *netdev) -{ - struct ixgbevf_adapter *adapter = netdev_priv(netdev); - int i; - - /* if interface is down do nothing */ - if (test_bit(__IXGBEVF_DOWN, &adapter->state)) - return; - for (i = 0; i < adapter->num_rx_queues; i++) - ixgbevf_msix_clean_rings(0, adapter->q_vector[i]); -} -#endif /* CONFIG_NET_POLL_CONTROLLER */ - static int ixgbevf_suspend(struct pci_dev *pdev, pm_message_t state) { struct net_device *netdev = pci_get_drvdata(pdev); @@ -4482,9 +4464,6 @@ static const struct net_device_ops ixgbevf_netdev_ops = { .ndo_tx_timeout = ixgbevf_tx_timeout, .ndo_vlan_rx_add_vid = ixgbevf_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = ixgbevf_vlan_rx_kill_vid, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = ixgbevf_netpoll, -#endif .ndo_features_check = ixgbevf_features_check, .ndo_bpf = ixgbevf_xdp, }; diff --git a/drivers/net/ethernet/lantiq_etop.c b/drivers/net/ethernet/lantiq_etop.c index 7a637b51c7d2..e08301d833e2 100644 --- a/drivers/net/ethernet/lantiq_etop.c +++ b/drivers/net/ethernet/lantiq_etop.c @@ -274,6 +274,7 @@ ltq_etop_hw_init(struct net_device *dev) struct ltq_etop_chan *ch = &priv->ch[i]; ch->idx = ch->dma.nr = i; + ch->dma.dev = &priv->pdev->dev; if (IS_TX(i)) { ltq_dma_alloc_tx(&ch->dma); diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index bc80a678abc3..b4ed7d394d07 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -1890,8 +1890,8 @@ static void mvneta_rxq_drop_pkts(struct mvneta_port *pp, if (!data || !(rx_desc->buf_phys_addr)) continue; - dma_unmap_single(pp->dev->dev.parent, rx_desc->buf_phys_addr, - MVNETA_RX_BUF_SIZE(pp->pkt_size), DMA_FROM_DEVICE); + dma_unmap_page(pp->dev->dev.parent, rx_desc->buf_phys_addr, + PAGE_SIZE, DMA_FROM_DEVICE); __free_page(data); } } @@ -2008,8 +2008,8 @@ static int mvneta_rx_swbm(struct napi_struct *napi, skb_add_rx_frag(rxq->skb, frag_num, page, frag_offset, frag_size, PAGE_SIZE); - dma_unmap_single(dev->dev.parent, phys_addr, - PAGE_SIZE, DMA_FROM_DEVICE); + dma_unmap_page(dev->dev.parent, phys_addr, + PAGE_SIZE, DMA_FROM_DEVICE); rxq->left_size -= frag_size; } } else { @@ -2039,9 +2039,8 @@ static int mvneta_rx_swbm(struct napi_struct *napi, frag_offset, frag_size, PAGE_SIZE); - dma_unmap_single(dev->dev.parent, phys_addr, - PAGE_SIZE, - DMA_FROM_DEVICE); + dma_unmap_page(dev->dev.parent, phys_addr, + PAGE_SIZE, DMA_FROM_DEVICE); rxq->left_size -= frag_size; } diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index 32d785b616e1..38cc01beea79 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -58,6 +58,8 @@ static struct { */ static void mvpp2_mac_config(struct net_device *dev, unsigned int mode, const struct phylink_link_state *state); +static void mvpp2_mac_link_up(struct net_device *dev, unsigned int mode, + phy_interface_t interface, struct phy_device *phy); /* Queue modes */ #define MVPP2_QDIST_SINGLE_MODE 0 @@ -3053,10 +3055,12 @@ static int mvpp2_poll(struct napi_struct *napi, int budget) cause_rx_tx & ~MVPP2_CAUSE_MISC_SUM_MASK); } - cause_tx = cause_rx_tx & MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_MASK; - if (cause_tx) { - cause_tx >>= MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_OFFSET; - mvpp2_tx_done(port, cause_tx, qv->sw_thread_id); + if (port->has_tx_irqs) { + cause_tx = cause_rx_tx & MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_MASK; + if (cause_tx) { + cause_tx >>= MVPP2_CAUSE_TXQ_OCCUP_DESC_ALL_OFFSET; + mvpp2_tx_done(port, cause_tx, qv->sw_thread_id); + } } /* Process RX packets */ @@ -3142,6 +3146,7 @@ static void mvpp2_start_dev(struct mvpp2_port *port) mvpp22_mode_reconfigure(port); if (port->phylink) { + netif_carrier_off(port->dev); phylink_start(port->phylink); } else { /* Phylink isn't used as of now for ACPI, so the MAC has to be @@ -3150,9 +3155,10 @@ static void mvpp2_start_dev(struct mvpp2_port *port) */ struct phylink_link_state state = { .interface = port->phy_interface, - .link = 1, }; mvpp2_mac_config(port->dev, MLO_AN_INBAND, &state); + mvpp2_mac_link_up(port->dev, MLO_AN_INBAND, port->phy_interface, + NULL); } netif_tx_start_all_queues(port->dev); @@ -4495,10 +4501,6 @@ static void mvpp2_mac_config(struct net_device *dev, unsigned int mode, return; } - netif_tx_stop_all_queues(port->dev); - if (!port->has_phy) - netif_carrier_off(port->dev); - /* Make sure the port is disabled when reconfiguring the mode */ mvpp2_port_disable(port); @@ -4523,16 +4525,7 @@ static void mvpp2_mac_config(struct net_device *dev, unsigned int mode, if (port->priv->hw_version == MVPP21 && port->flags & MVPP2_F_LOOPBACK) mvpp2_port_loopback_set(port, state); - /* If the port already was up, make sure it's still in the same state */ - if (state->link || !port->has_phy) { - mvpp2_port_enable(port); - - mvpp2_egress_enable(port); - mvpp2_ingress_enable(port); - if (!port->has_phy) - netif_carrier_on(dev); - netif_tx_wake_all_queues(dev); - } + mvpp2_port_enable(port); } static void mvpp2_mac_link_up(struct net_device *dev, unsigned int mode, @@ -4803,6 +4796,7 @@ static int mvpp2_port_probe(struct platform_device *pdev, dev->min_mtu = ETH_MIN_MTU; /* 9704 == 9728 - 20 and rounding to 8 */ dev->max_mtu = MVPP2_BM_JUMBO_PKT_SIZE; + dev->dev.of_node = port_node; /* Phylink isn't used w/ ACPI as of now */ if (port_node) { diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c index 6785661d1a72..fe49384eba48 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c @@ -1286,20 +1286,6 @@ out: mutex_unlock(&mdev->state_lock); } -#ifdef CONFIG_NET_POLL_CONTROLLER -static void mlx4_en_netpoll(struct net_device *dev) -{ - struct mlx4_en_priv *priv = netdev_priv(dev); - struct mlx4_en_cq *cq; - int i; - - for (i = 0; i < priv->tx_ring_num[TX]; i++) { - cq = priv->tx_cq[TX][i]; - napi_schedule(&cq->napi); - } -} -#endif - static int mlx4_en_set_rss_steer_rules(struct mlx4_en_priv *priv) { u64 reg_id; @@ -2946,9 +2932,6 @@ static const struct net_device_ops mlx4_netdev_ops = { .ndo_tx_timeout = mlx4_en_tx_timeout, .ndo_vlan_rx_add_vid = mlx4_en_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = mlx4_en_vlan_rx_kill_vid, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = mlx4_en_netpoll, -#endif .ndo_set_features = mlx4_en_set_features, .ndo_fix_features = mlx4_en_fix_features, .ndo_setup_tc = __mlx4_en_setup_tc, @@ -2983,9 +2966,6 @@ static const struct net_device_ops mlx4_netdev_ops_master = { .ndo_set_vf_link_state = mlx4_en_set_vf_link_state, .ndo_get_vf_stats = mlx4_en_get_vf_stats, .ndo_get_vf_config = mlx4_en_get_vf_config, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = mlx4_en_netpoll, -#endif .ndo_set_features = mlx4_en_set_features, .ndo_fix_features = mlx4_en_fix_features, .ndo_setup_tc = __mlx4_en_setup_tc, diff --git a/drivers/net/ethernet/mellanox/mlx4/eq.c b/drivers/net/ethernet/mellanox/mlx4/eq.c index 1f3372c1802e..2df92dbd38e1 100644 --- a/drivers/net/ethernet/mellanox/mlx4/eq.c +++ b/drivers/net/ethernet/mellanox/mlx4/eq.c @@ -240,7 +240,8 @@ static void mlx4_set_eq_affinity_hint(struct mlx4_priv *priv, int vec) struct mlx4_dev *dev = &priv->dev; struct mlx4_eq *eq = &priv->eq_table.eq[vec]; - if (!eq->affinity_mask || cpumask_empty(eq->affinity_mask)) + if (!cpumask_available(eq->affinity_mask) || + cpumask_empty(eq->affinity_mask)) return; hint_err = irq_set_affinity_hint(eq->irq, eq->affinity_mask); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c index 3ce14d42ddc8..a53736c26c0c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/cmd.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/cmd.c @@ -206,7 +206,7 @@ static void poll_timeout(struct mlx5_cmd_work_ent *ent) u8 own; do { - own = ent->lay->status_own; + own = READ_ONCE(ent->lay->status_own); if (!(own & CMD_OWNER_HW)) { ent->ret = 0; return; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index b994b80d5714..37ba7c78859d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -132,11 +132,11 @@ void mlx5_add_device(struct mlx5_interface *intf, struct mlx5_priv *priv) delayed_event_start(priv); dev_ctx->context = intf->add(dev); - set_bit(MLX5_INTERFACE_ADDED, &dev_ctx->state); - if (intf->attach) - set_bit(MLX5_INTERFACE_ATTACHED, &dev_ctx->state); - if (dev_ctx->context) { + set_bit(MLX5_INTERFACE_ADDED, &dev_ctx->state); + if (intf->attach) + set_bit(MLX5_INTERFACE_ATTACHED, &dev_ctx->state); + spin_lock_irq(&priv->ctx_lock); list_add_tail(&dev_ctx->list, &priv->ctx_list); @@ -211,12 +211,17 @@ static void mlx5_attach_interface(struct mlx5_interface *intf, struct mlx5_priv if (intf->attach) { if (test_bit(MLX5_INTERFACE_ATTACHED, &dev_ctx->state)) goto out; - intf->attach(dev, dev_ctx->context); + if (intf->attach(dev, dev_ctx->context)) + goto out; + set_bit(MLX5_INTERFACE_ATTACHED, &dev_ctx->state); } else { if (test_bit(MLX5_INTERFACE_ADDED, &dev_ctx->state)) goto out; dev_ctx->context = intf->add(dev); + if (!dev_ctx->context) + goto out; + set_bit(MLX5_INTERFACE_ADDED, &dev_ctx->state); } @@ -391,16 +396,17 @@ void mlx5_remove_dev_by_protocol(struct mlx5_core_dev *dev, int protocol) } } -static u16 mlx5_gen_pci_id(struct mlx5_core_dev *dev) +static u32 mlx5_gen_pci_id(struct mlx5_core_dev *dev) { - return (u16)((dev->pdev->bus->number << 8) | + return (u32)((pci_domain_nr(dev->pdev->bus) << 16) | + (dev->pdev->bus->number << 8) | PCI_SLOT(dev->pdev->devfn)); } /* Must be called with intf_mutex held */ struct mlx5_core_dev *mlx5_get_next_phys_dev(struct mlx5_core_dev *dev) { - u16 pci_id = mlx5_gen_pci_id(dev); + u32 pci_id = mlx5_gen_pci_id(dev); struct mlx5_core_dev *res = NULL; struct mlx5_core_dev *tmp_dev; struct mlx5_priv *priv; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c index eddd7702680b..e88340e196f7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c @@ -183,12 +183,13 @@ static const struct tlsdev_ops mlx5e_tls_ops = { void mlx5e_tls_build_netdev(struct mlx5e_priv *priv) { - u32 caps = mlx5_accel_tls_device_caps(priv->mdev); struct net_device *netdev = priv->netdev; + u32 caps; if (!mlx5_accel_is_tls_device(priv->mdev)) return; + caps = mlx5_accel_tls_device_caps(priv->mdev); if (caps & MLX5_ACCEL_TLS_TX) { netdev->features |= NETIF_F_HW_TLS_TX; netdev->hw_features |= NETIF_F_HW_TLS_TX; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c index 75bb981e00b7..41cde926cdab 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_fs_ethtool.c @@ -191,7 +191,7 @@ set_udp(void *headers_c, void *headers_v, __be16 psrc_m, __be16 psrc_v, { if (psrc_m) { MLX5E_FTE_SET(headers_c, udp_sport, 0xffff); - MLX5E_FTE_SET(headers_c, udp_sport, ntohs(psrc_v)); + MLX5E_FTE_SET(headers_v, udp_sport, ntohs(psrc_v)); } if (pdst_m) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 5a7939e70190..54118b77dc1f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -4315,22 +4315,6 @@ static int mlx5e_xdp(struct net_device *dev, struct netdev_bpf *xdp) } } -#ifdef CONFIG_NET_POLL_CONTROLLER -/* Fake "interrupt" called by netpoll (eg netconsole) to send skbs without - * reenabling interrupts. - */ -static void mlx5e_netpoll(struct net_device *dev) -{ - struct mlx5e_priv *priv = netdev_priv(dev); - struct mlx5e_channels *chs = &priv->channels; - - int i; - - for (i = 0; i < chs->num; i++) - napi_schedule(&chs->c[i]->napi); -} -#endif - static const struct net_device_ops mlx5e_netdev_ops = { .ndo_open = mlx5e_open, .ndo_stop = mlx5e_close, @@ -4356,9 +4340,6 @@ static const struct net_device_ops mlx5e_netdev_ops = { #ifdef CONFIG_MLX5_EN_ARFS .ndo_rx_flow_steer = mlx5e_rx_flow_steer, #endif -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = mlx5e_netpoll, -#endif #ifdef CONFIG_MLX5_ESWITCH /* SRIOV E-Switch NDOs */ .ndo_set_vf_mac = mlx5e_set_vf_mac, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index f72b5c9dcfe9..3028e8d90920 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -663,6 +663,7 @@ static int esw_create_offloads_fdb_tables(struct mlx5_eswitch *esw, int nvports) if (err) goto miss_rule_err; + kvfree(flow_group_in); return 0; miss_rule_err: diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c index f418541af7cf..37d114c668b7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c @@ -1578,6 +1578,33 @@ static u64 matched_fgs_get_version(struct list_head *match_head) return version; } +static struct fs_fte * +lookup_fte_locked(struct mlx5_flow_group *g, + u32 *match_value, + bool take_write) +{ + struct fs_fte *fte_tmp; + + if (take_write) + nested_down_write_ref_node(&g->node, FS_LOCK_PARENT); + else + nested_down_read_ref_node(&g->node, FS_LOCK_PARENT); + fte_tmp = rhashtable_lookup_fast(&g->ftes_hash, match_value, + rhash_fte); + if (!fte_tmp || !tree_get_node(&fte_tmp->node)) { + fte_tmp = NULL; + goto out; + } + + nested_down_write_ref_node(&fte_tmp->node, FS_LOCK_CHILD); +out: + if (take_write) + up_write_ref_node(&g->node); + else + up_read_ref_node(&g->node); + return fte_tmp; +} + static struct mlx5_flow_handle * try_add_to_existing_fg(struct mlx5_flow_table *ft, struct list_head *match_head, @@ -1600,10 +1627,6 @@ try_add_to_existing_fg(struct mlx5_flow_table *ft, if (IS_ERR(fte)) return ERR_PTR(-ENOMEM); - list_for_each_entry(iter, match_head, list) { - nested_down_read_ref_node(&iter->g->node, FS_LOCK_PARENT); - } - search_again_locked: version = matched_fgs_get_version(match_head); /* Try to find a fg that already contains a matching fte */ @@ -1611,20 +1634,9 @@ search_again_locked: struct fs_fte *fte_tmp; g = iter->g; - fte_tmp = rhashtable_lookup_fast(&g->ftes_hash, spec->match_value, - rhash_fte); - if (!fte_tmp || !tree_get_node(&fte_tmp->node)) + fte_tmp = lookup_fte_locked(g, spec->match_value, take_write); + if (!fte_tmp) continue; - - nested_down_write_ref_node(&fte_tmp->node, FS_LOCK_CHILD); - if (!take_write) { - list_for_each_entry(iter, match_head, list) - up_read_ref_node(&iter->g->node); - } else { - list_for_each_entry(iter, match_head, list) - up_write_ref_node(&iter->g->node); - } - rule = add_rule_fg(g, spec->match_value, flow_act, dest, dest_num, fte_tmp); up_write_ref_node(&fte_tmp->node); @@ -1633,19 +1645,6 @@ search_again_locked: return rule; } - /* No group with matching fte found. Try to add a new fte to any - * matching fg. - */ - - if (!take_write) { - list_for_each_entry(iter, match_head, list) - up_read_ref_node(&iter->g->node); - list_for_each_entry(iter, match_head, list) - nested_down_write_ref_node(&iter->g->node, - FS_LOCK_PARENT); - take_write = true; - } - /* Check the ft version, for case that new flow group * was added while the fgs weren't locked */ @@ -1657,27 +1656,30 @@ search_again_locked: /* Check the fgs version, for case the new FTE with the * same values was added while the fgs weren't locked */ - if (version != matched_fgs_get_version(match_head)) + if (version != matched_fgs_get_version(match_head)) { + take_write = true; goto search_again_locked; + } list_for_each_entry(iter, match_head, list) { g = iter->g; if (!g->node.active) continue; + + nested_down_write_ref_node(&g->node, FS_LOCK_PARENT); + err = insert_fte(g, fte); if (err) { + up_write_ref_node(&g->node); if (err == -ENOSPC) continue; - list_for_each_entry(iter, match_head, list) - up_write_ref_node(&iter->g->node); kmem_cache_free(steering->ftes_cache, fte); return ERR_PTR(err); } nested_down_write_ref_node(&fte->node, FS_LOCK_CHILD); - list_for_each_entry(iter, match_head, list) - up_write_ref_node(&iter->g->node); + up_write_ref_node(&g->node); rule = add_rule_fg(g, spec->match_value, flow_act, dest, dest_num, fte); up_write_ref_node(&fte->node); @@ -1686,8 +1688,6 @@ search_again_locked: } rule = ERR_PTR(-ENOENT); out: - list_for_each_entry(iter, match_head, list) - up_write_ref_node(&iter->g->node); kmem_cache_free(steering->ftes_cache, fte); return rule; } @@ -1726,6 +1726,8 @@ search_again_locked: if (err) { if (take_write) up_write_ref_node(&ft->node); + else + up_read_ref_node(&ft->node); return ERR_PTR(err); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c index d39b0b7011b2..9f39aeca863f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c @@ -331,9 +331,17 @@ void mlx5_start_health_poll(struct mlx5_core_dev *dev) add_timer(&health->timer); } -void mlx5_stop_health_poll(struct mlx5_core_dev *dev) +void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health) { struct mlx5_core_health *health = &dev->priv.health; + unsigned long flags; + + if (disable_health) { + spin_lock_irqsave(&health->wq_lock, flags); + set_bit(MLX5_DROP_NEW_HEALTH_WORK, &health->flags); + set_bit(MLX5_DROP_NEW_RECOVERY_WORK, &health->flags); + spin_unlock_irqrestore(&health->wq_lock, flags); + } del_timer_sync(&health->timer); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index cf3e4a659052..b5e9f664fc66 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -878,8 +878,10 @@ static int mlx5_pci_init(struct mlx5_core_dev *dev, struct mlx5_priv *priv) priv->numa_node = dev_to_node(&dev->pdev->dev); priv->dbg_root = debugfs_create_dir(dev_name(&pdev->dev), mlx5_debugfs_root); - if (!priv->dbg_root) + if (!priv->dbg_root) { + dev_err(&pdev->dev, "Cannot create debugfs dir, aborting\n"); return -ENOMEM; + } err = mlx5_pci_enable_device(dev); if (err) { @@ -928,7 +930,7 @@ static void mlx5_pci_close(struct mlx5_core_dev *dev, struct mlx5_priv *priv) pci_clear_master(dev->pdev); release_bar(dev->pdev); mlx5_pci_disable_device(dev); - debugfs_remove(priv->dbg_root); + debugfs_remove_recursive(priv->dbg_root); } static int mlx5_init_once(struct mlx5_core_dev *dev, struct mlx5_priv *priv) @@ -1286,7 +1288,7 @@ err_cleanup_once: mlx5_cleanup_once(dev); err_stop_poll: - mlx5_stop_health_poll(dev); + mlx5_stop_health_poll(dev, boot); if (mlx5_cmd_teardown_hca(dev)) { dev_err(&dev->pdev->dev, "tear_down_hca failed, skip cleanup\n"); goto out_err; @@ -1346,7 +1348,7 @@ static int mlx5_unload_one(struct mlx5_core_dev *dev, struct mlx5_priv *priv, mlx5_free_irq_vectors(dev); if (cleanup) mlx5_cleanup_once(dev); - mlx5_stop_health_poll(dev); + mlx5_stop_health_poll(dev, cleanup); err = mlx5_cmd_teardown_hca(dev); if (err) { dev_err(&dev->pdev->dev, "tear_down_hca failed, skip cleanup\n"); @@ -1608,7 +1610,7 @@ static int mlx5_try_fast_unload(struct mlx5_core_dev *dev) * with the HCA, so the health polll is no longer needed. */ mlx5_drain_health_wq(dev); - mlx5_stop_health_poll(dev); + mlx5_stop_health_poll(dev, false); ret = mlx5_cmd_force_teardown_hca(dev); if (ret) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/transobj.c b/drivers/net/ethernet/mellanox/mlx5/core/transobj.c index dae1c5c5d27c..d2f76070ea7c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/transobj.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/transobj.c @@ -509,7 +509,7 @@ static int mlx5_hairpin_modify_sq(struct mlx5_core_dev *peer_mdev, u32 sqn, sqc = MLX5_ADDR_OF(modify_sq_in, in, ctx); - if (next_state == MLX5_RQC_STATE_RDY) { + if (next_state == MLX5_SQC_STATE_RDY) { MLX5_SET(sqc, sqc, hairpin_peer_rq, peer_rq); MLX5_SET(sqc, sqc, hairpin_peer_vhca, peer_vhca); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/wq.c b/drivers/net/ethernet/mellanox/mlx5/core/wq.c index 86478a6b99c5..68e7f8df2a6d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/wq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/wq.c @@ -39,9 +39,9 @@ u32 mlx5_wq_cyc_get_size(struct mlx5_wq_cyc *wq) return (u32)wq->fbc.sz_m1 + 1; } -u32 mlx5_wq_cyc_get_frag_size(struct mlx5_wq_cyc *wq) +u16 mlx5_wq_cyc_get_frag_size(struct mlx5_wq_cyc *wq) { - return (u32)wq->fbc.frag_sz_m1 + 1; + return wq->fbc.frag_sz_m1 + 1; } u32 mlx5_cqwq_get_size(struct mlx5_cqwq *wq) @@ -138,15 +138,16 @@ int mlx5_wq_qp_create(struct mlx5_core_dev *mdev, struct mlx5_wq_param *param, void *qpc, struct mlx5_wq_qp *wq, struct mlx5_wq_ctrl *wq_ctrl) { - u32 sq_strides_offset; + u16 sq_strides_offset; + u32 rq_pg_remainder; int err; mlx5_fill_fbc(MLX5_GET(qpc, qpc, log_rq_stride) + 4, MLX5_GET(qpc, qpc, log_rq_size), &wq->rq.fbc); - sq_strides_offset = - ((wq->rq.fbc.frag_sz_m1 + 1) % PAGE_SIZE) / MLX5_SEND_WQE_BB; + rq_pg_remainder = mlx5_wq_cyc_get_byte_size(&wq->rq) % PAGE_SIZE; + sq_strides_offset = rq_pg_remainder / MLX5_SEND_WQE_BB; mlx5_fill_fbc_offset(ilog2(MLX5_SEND_WQE_BB), MLX5_GET(qpc, qpc, log_sq_size), diff --git a/drivers/net/ethernet/mellanox/mlx5/core/wq.h b/drivers/net/ethernet/mellanox/mlx5/core/wq.h index 2bd4c3184eba..3a1a170bb2d7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/wq.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/wq.h @@ -80,7 +80,7 @@ int mlx5_wq_cyc_create(struct mlx5_core_dev *mdev, struct mlx5_wq_param *param, void *wqc, struct mlx5_wq_cyc *wq, struct mlx5_wq_ctrl *wq_ctrl); u32 mlx5_wq_cyc_get_size(struct mlx5_wq_cyc *wq); -u32 mlx5_wq_cyc_get_frag_size(struct mlx5_wq_cyc *wq); +u16 mlx5_wq_cyc_get_frag_size(struct mlx5_wq_cyc *wq); int mlx5_wq_qp_create(struct mlx5_core_dev *mdev, struct mlx5_wq_param *param, void *qpc, struct mlx5_wq_qp *wq, diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c index 930700413b1d..b492152c8881 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c @@ -44,8 +44,8 @@ #define MLXSW_SP_FWREV_MINOR_TO_BRANCH(minor) ((minor) / 100) #define MLXSW_SP1_FWREV_MAJOR 13 -#define MLXSW_SP1_FWREV_MINOR 1702 -#define MLXSW_SP1_FWREV_SUBMINOR 6 +#define MLXSW_SP1_FWREV_MINOR 1703 +#define MLXSW_SP1_FWREV_SUBMINOR 4 #define MLXSW_SP1_FWREV_CAN_RESET_MINOR 1702 static const struct mlxsw_fw_rev mlxsw_sp1_fw_rev = { diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c index 4327487553c5..3589432d1643 100644 --- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c +++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_buffers.c @@ -337,14 +337,14 @@ static const struct mlxsw_sp_sb_cm mlxsw_sp_sb_cms_egress[] = { MLXSW_SP_SB_CM(1500, 9, 0), MLXSW_SP_SB_CM(1500, 9, 0), MLXSW_SP_SB_CM(1500, 9, 0), - MLXSW_SP_SB_CM(0, 0, 0), - MLXSW_SP_SB_CM(0, 0, 0), - MLXSW_SP_SB_CM(0, 0, 0), - MLXSW_SP_SB_CM(0, 0, 0), - MLXSW_SP_SB_CM(0, 0, 0), - MLXSW_SP_SB_CM(0, 0, 0), - MLXSW_SP_SB_CM(0, 0, 0), - MLXSW_SP_SB_CM(0, 0, 0), + MLXSW_SP_SB_CM(0, 140000, 15), + MLXSW_SP_SB_CM(0, 140000, 15), + MLXSW_SP_SB_CM(0, 140000, 15), + MLXSW_SP_SB_CM(0, 140000, 15), + MLXSW_SP_SB_CM(0, 140000, 15), + MLXSW_SP_SB_CM(0, 140000, 15), + MLXSW_SP_SB_CM(0, 140000, 15), + MLXSW_SP_SB_CM(0, 140000, 15), MLXSW_SP_SB_CM(1, 0xff, 0), }; diff --git a/drivers/net/ethernet/microchip/lan743x_main.c b/drivers/net/ethernet/microchip/lan743x_main.c index e7dce79ff2c9..001b5f714c1b 100644 --- a/drivers/net/ethernet/microchip/lan743x_main.c +++ b/drivers/net/ethernet/microchip/lan743x_main.c @@ -2850,7 +2850,7 @@ static void lan743x_pcidev_shutdown(struct pci_dev *pdev) lan743x_hardware_cleanup(adapter); } -#ifdef CONFIG_PM +#ifdef CONFIG_PM_SLEEP static u16 lan743x_pm_wakeframe_crc16(const u8 *buf, int len) { return bitrev16(crc16(0xFFFF, buf, len)); @@ -3016,7 +3016,7 @@ static int lan743x_pm_resume(struct device *dev) static const struct dev_pm_ops lan743x_pm_ops = { SET_SYSTEM_SLEEP_PM_OPS(lan743x_pm_suspend, lan743x_pm_resume) }; -#endif /*CONFIG_PM */ +#endif /* CONFIG_PM_SLEEP */ static const struct pci_device_id lan743x_pcidev_tbl[] = { { PCI_DEVICE(PCI_VENDOR_ID_SMSC, PCI_DEVICE_ID_SMSC_LAN7430) }, @@ -3028,7 +3028,7 @@ static struct pci_driver lan743x_pcidev_driver = { .id_table = lan743x_pcidev_tbl, .probe = lan743x_pcidev_probe, .remove = lan743x_pcidev_remove, -#ifdef CONFIG_PM +#ifdef CONFIG_PM_SLEEP .driver.pm = &lan743x_pm_ops, #endif .shutdown = lan743x_pcidev_shutdown, diff --git a/drivers/net/ethernet/mscc/ocelot_board.c b/drivers/net/ethernet/mscc/ocelot_board.c index 26bb3b18f3be..3cdf63e35b53 100644 --- a/drivers/net/ethernet/mscc/ocelot_board.c +++ b/drivers/net/ethernet/mscc/ocelot_board.c @@ -91,7 +91,7 @@ static irqreturn_t ocelot_xtr_irq_handler(int irq, void *arg) struct sk_buff *skb; struct net_device *dev; u32 *buf; - int sz, len; + int sz, len, buf_len; u32 ifh[4]; u32 val; struct frame_info info; @@ -116,14 +116,20 @@ static irqreturn_t ocelot_xtr_irq_handler(int irq, void *arg) err = -ENOMEM; break; } - buf = (u32 *)skb_put(skb, info.len); + buf_len = info.len - ETH_FCS_LEN; + buf = (u32 *)skb_put(skb, buf_len); len = 0; do { sz = ocelot_rx_frame_word(ocelot, grp, false, &val); *buf++ = val; len += sz; - } while ((sz == 4) && (len < info.len)); + } while (len < buf_len); + + /* Read the FCS and discard it */ + sz = ocelot_rx_frame_word(ocelot, grp, false, &val); + /* Update the statistics if part of the FCS was read before */ + len -= ETH_FCS_LEN - sz; if (sz < 0) { err = sz; diff --git a/drivers/net/ethernet/netronome/nfp/flower/action.c b/drivers/net/ethernet/netronome/nfp/flower/action.c index 9044496803e6..46ba0cf257c6 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/action.c +++ b/drivers/net/ethernet/netronome/nfp/flower/action.c @@ -52,6 +52,7 @@ #define NFP_FL_TUNNEL_CSUM cpu_to_be16(0x01) #define NFP_FL_TUNNEL_KEY cpu_to_be16(0x04) #define NFP_FL_TUNNEL_GENEVE_OPT cpu_to_be16(0x0800) +#define NFP_FL_SUPPORTED_TUNNEL_INFO_FLAGS IP_TUNNEL_INFO_TX #define NFP_FL_SUPPORTED_IPV4_UDP_TUN_FLAGS (NFP_FL_TUNNEL_CSUM | \ NFP_FL_TUNNEL_KEY | \ NFP_FL_TUNNEL_GENEVE_OPT) @@ -741,11 +742,16 @@ nfp_flower_loop_action(struct nfp_app *app, const struct tc_action *a, nfp_fl_push_vlan(psh_v, a); *a_len += sizeof(struct nfp_fl_push_vlan); } else if (is_tcf_tunnel_set(a)) { + struct ip_tunnel_info *ip_tun = tcf_tunnel_info(a); struct nfp_repr *repr = netdev_priv(netdev); + *tun_type = nfp_fl_get_tun_from_act_l4_port(repr->app, a); if (*tun_type == NFP_FL_TUNNEL_NONE) return -EOPNOTSUPP; + if (ip_tun->mode & ~NFP_FL_SUPPORTED_TUNNEL_INFO_FLAGS) + return -EOPNOTSUPP; + /* Pre-tunnel action is required for tunnel encap. * This checks for next hop entries on NFP. * If none, the packet falls back before applying other actions. diff --git a/drivers/net/ethernet/netronome/nfp/flower/main.h b/drivers/net/ethernet/netronome/nfp/flower/main.h index 85f8209bf007..81d941ab895c 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/main.h +++ b/drivers/net/ethernet/netronome/nfp/flower/main.h @@ -70,6 +70,7 @@ struct nfp_app; #define NFP_FL_FEATS_GENEVE BIT(0) #define NFP_FL_NBI_MTU_SETTING BIT(1) #define NFP_FL_FEATS_GENEVE_OPT BIT(2) +#define NFP_FL_FEATS_VLAN_PCP BIT(3) #define NFP_FL_FEATS_LAG BIT(31) struct nfp_fl_mask_id { diff --git a/drivers/net/ethernet/netronome/nfp/flower/match.c b/drivers/net/ethernet/netronome/nfp/flower/match.c index a0c72f277faa..17acb8cc6044 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/match.c +++ b/drivers/net/ethernet/netronome/nfp/flower/match.c @@ -56,7 +56,7 @@ nfp_flower_compile_meta_tci(struct nfp_flower_meta_tci *frame, FLOW_DISSECTOR_KEY_VLAN, target); /* Populate the tci field. */ - if (flow_vlan->vlan_id) { + if (flow_vlan->vlan_id || flow_vlan->vlan_priority) { tmp_tci = FIELD_PREP(NFP_FLOWER_MASK_VLAN_PRIO, flow_vlan->vlan_priority) | FIELD_PREP(NFP_FLOWER_MASK_VLAN_VID, diff --git a/drivers/net/ethernet/netronome/nfp/flower/offload.c b/drivers/net/ethernet/netronome/nfp/flower/offload.c index 2edab01c3beb..bd19624f10cf 100644 --- a/drivers/net/ethernet/netronome/nfp/flower/offload.c +++ b/drivers/net/ethernet/netronome/nfp/flower/offload.c @@ -192,6 +192,17 @@ nfp_flower_calculate_key_layers(struct nfp_app *app, key_size += sizeof(struct nfp_flower_mac_mpls); } + if (dissector_uses_key(flow->dissector, FLOW_DISSECTOR_KEY_VLAN)) { + struct flow_dissector_key_vlan *flow_vlan; + + flow_vlan = skb_flow_dissector_target(flow->dissector, + FLOW_DISSECTOR_KEY_VLAN, + flow->mask); + if (!(priv->flower_ext_feats & NFP_FL_FEATS_VLAN_PCP) && + flow_vlan->vlan_priority) + return -EOPNOTSUPP; + } + if (dissector_uses_key(flow->dissector, FLOW_DISSECTOR_KEY_ENC_CONTROL)) { struct flow_dissector_key_ipv4_addrs *mask_ipv4 = NULL; diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index a8b9fbab5f73..8ed38fd5a852 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -229,29 +229,16 @@ done: spin_unlock_bh(&nn->reconfig_lock); } -/** - * nfp_net_reconfig() - Reconfigure the firmware - * @nn: NFP Net device to reconfigure - * @update: The value for the update field in the BAR config - * - * Write the update word to the BAR and ping the reconfig queue. The - * poll until the firmware has acknowledged the update by zeroing the - * update word. - * - * Return: Negative errno on error, 0 on success - */ -int nfp_net_reconfig(struct nfp_net *nn, u32 update) +static void nfp_net_reconfig_sync_enter(struct nfp_net *nn) { bool cancelled_timer = false; u32 pre_posted_requests; - int ret; spin_lock_bh(&nn->reconfig_lock); nn->reconfig_sync_present = true; if (nn->reconfig_timer_active) { - del_timer(&nn->reconfig_timer); nn->reconfig_timer_active = false; cancelled_timer = true; } @@ -260,14 +247,43 @@ int nfp_net_reconfig(struct nfp_net *nn, u32 update) spin_unlock_bh(&nn->reconfig_lock); - if (cancelled_timer) + if (cancelled_timer) { + del_timer_sync(&nn->reconfig_timer); nfp_net_reconfig_wait(nn, nn->reconfig_timer.expires); + } /* Run the posted reconfigs which were issued before we started */ if (pre_posted_requests) { nfp_net_reconfig_start(nn, pre_posted_requests); nfp_net_reconfig_wait(nn, jiffies + HZ * NFP_NET_POLL_TIMEOUT); } +} + +static void nfp_net_reconfig_wait_posted(struct nfp_net *nn) +{ + nfp_net_reconfig_sync_enter(nn); + + spin_lock_bh(&nn->reconfig_lock); + nn->reconfig_sync_present = false; + spin_unlock_bh(&nn->reconfig_lock); +} + +/** + * nfp_net_reconfig() - Reconfigure the firmware + * @nn: NFP Net device to reconfigure + * @update: The value for the update field in the BAR config + * + * Write the update word to the BAR and ping the reconfig queue. The + * poll until the firmware has acknowledged the update by zeroing the + * update word. + * + * Return: Negative errno on error, 0 on success + */ +int nfp_net_reconfig(struct nfp_net *nn, u32 update) +{ + int ret; + + nfp_net_reconfig_sync_enter(nn); nfp_net_reconfig_start(nn, update); ret = nfp_net_reconfig_wait(nn, jiffies + HZ * NFP_NET_POLL_TIMEOUT); @@ -3130,21 +3146,6 @@ nfp_net_vlan_rx_kill_vid(struct net_device *netdev, __be16 proto, u16 vid) return nfp_net_reconfig_mbox(nn, NFP_NET_CFG_MBOX_CMD_CTAG_FILTER_KILL); } -#ifdef CONFIG_NET_POLL_CONTROLLER -static void nfp_net_netpoll(struct net_device *netdev) -{ - struct nfp_net *nn = netdev_priv(netdev); - int i; - - /* nfp_net's NAPIs are statically allocated so even if there is a race - * with reconfig path this will simply try to schedule some disabled - * NAPI instances. - */ - for (i = 0; i < nn->dp.num_stack_tx_rings; i++) - napi_schedule_irqoff(&nn->r_vecs[i].napi); -} -#endif - static void nfp_net_stat64(struct net_device *netdev, struct rtnl_link_stats64 *stats) { @@ -3503,9 +3504,6 @@ const struct net_device_ops nfp_net_netdev_ops = { .ndo_get_stats64 = nfp_net_stat64, .ndo_vlan_rx_add_vid = nfp_net_vlan_rx_add_vid, .ndo_vlan_rx_kill_vid = nfp_net_vlan_rx_kill_vid, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = nfp_net_netpoll, -#endif .ndo_set_vf_mac = nfp_app_set_vf_mac, .ndo_set_vf_vlan = nfp_app_set_vf_vlan, .ndo_set_vf_spoofchk = nfp_app_set_vf_spoofchk, @@ -3633,6 +3631,7 @@ struct nfp_net *nfp_net_alloc(struct pci_dev *pdev, bool needs_netdev, */ void nfp_net_free(struct nfp_net *nn) { + WARN_ON(timer_pending(&nn->reconfig_timer) || nn->reconfig_posted); if (nn->dp.netdev) free_netdev(nn->dp.netdev); else @@ -3920,4 +3919,5 @@ void nfp_net_clean(struct nfp_net *nn) return; unregister_netdev(nn->dp.netdev); + nfp_net_reconfig_wait_posted(nn); } diff --git a/drivers/net/ethernet/qlogic/qed/qed_dcbx.c b/drivers/net/ethernet/qlogic/qed/qed_dcbx.c index 6bb76e6d3c14..f5459de6d60a 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_dcbx.c +++ b/drivers/net/ethernet/qlogic/qed/qed_dcbx.c @@ -190,10 +190,8 @@ qed_dcbx_dp_protocol(struct qed_hwfn *p_hwfn, struct qed_dcbx_results *p_data) static void qed_dcbx_set_params(struct qed_dcbx_results *p_data, - struct qed_hw_info *p_info, - bool enable, - u8 prio, - u8 tc, + struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt, + bool enable, u8 prio, u8 tc, enum dcbx_protocol_type type, enum qed_pci_personality personality) { @@ -206,19 +204,30 @@ qed_dcbx_set_params(struct qed_dcbx_results *p_data, else p_data->arr[type].update = DONT_UPDATE_DCB_DSCP; + /* Do not add vlan tag 0 when DCB is enabled and port in UFP/OV mode */ + if ((test_bit(QED_MF_8021Q_TAGGING, &p_hwfn->cdev->mf_bits) || + test_bit(QED_MF_8021AD_TAGGING, &p_hwfn->cdev->mf_bits))) + p_data->arr[type].dont_add_vlan0 = true; + /* QM reconf data */ - if (p_info->personality == personality) - qed_hw_info_set_offload_tc(p_info, tc); + if (p_hwfn->hw_info.personality == personality) + qed_hw_info_set_offload_tc(&p_hwfn->hw_info, tc); + + /* Configure dcbx vlan priority in doorbell block for roce EDPM */ + if (test_bit(QED_MF_UFP_SPECIFIC, &p_hwfn->cdev->mf_bits) && + type == DCBX_PROTOCOL_ROCE) { + qed_wr(p_hwfn, p_ptt, DORQ_REG_TAG1_OVRD_MODE, 1); + qed_wr(p_hwfn, p_ptt, DORQ_REG_PF_PCP_BB_K2, prio << 1); + } } /* Update app protocol data and hw_info fields with the TLV info */ static void qed_dcbx_update_app_info(struct qed_dcbx_results *p_data, - struct qed_hwfn *p_hwfn, - bool enable, - u8 prio, u8 tc, enum dcbx_protocol_type type) + struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt, + bool enable, u8 prio, u8 tc, + enum dcbx_protocol_type type) { - struct qed_hw_info *p_info = &p_hwfn->hw_info; enum qed_pci_personality personality; enum dcbx_protocol_type id; int i; @@ -231,7 +240,7 @@ qed_dcbx_update_app_info(struct qed_dcbx_results *p_data, personality = qed_dcbx_app_update[i].personality; - qed_dcbx_set_params(p_data, p_info, enable, + qed_dcbx_set_params(p_data, p_hwfn, p_ptt, enable, prio, tc, type, personality); } } @@ -265,7 +274,7 @@ qed_dcbx_get_app_protocol_type(struct qed_hwfn *p_hwfn, * reconfiguring QM. Get protocol specific data for PF update ramrod command. */ static int -qed_dcbx_process_tlv(struct qed_hwfn *p_hwfn, +qed_dcbx_process_tlv(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt, struct qed_dcbx_results *p_data, struct dcbx_app_priority_entry *p_tbl, u32 pri_tc_tbl, int count, u8 dcbx_version) @@ -309,7 +318,7 @@ qed_dcbx_process_tlv(struct qed_hwfn *p_hwfn, enable = true; } - qed_dcbx_update_app_info(p_data, p_hwfn, enable, + qed_dcbx_update_app_info(p_data, p_hwfn, p_ptt, enable, priority, tc, type); } } @@ -331,7 +340,7 @@ qed_dcbx_process_tlv(struct qed_hwfn *p_hwfn, continue; enable = (type == DCBX_PROTOCOL_ETH) ? false : !!dcbx_version; - qed_dcbx_update_app_info(p_data, p_hwfn, enable, + qed_dcbx_update_app_info(p_data, p_hwfn, p_ptt, enable, priority, tc, type); } @@ -341,7 +350,8 @@ qed_dcbx_process_tlv(struct qed_hwfn *p_hwfn, /* Parse app TLV's to update TC information in hw_info structure for * reconfiguring QM. Get protocol specific data for PF update ramrod command. */ -static int qed_dcbx_process_mib_info(struct qed_hwfn *p_hwfn) +static int +qed_dcbx_process_mib_info(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt) { struct dcbx_app_priority_feature *p_app; struct dcbx_app_priority_entry *p_tbl; @@ -365,7 +375,7 @@ static int qed_dcbx_process_mib_info(struct qed_hwfn *p_hwfn) p_info = &p_hwfn->hw_info; num_entries = QED_MFW_GET_FIELD(p_app->flags, DCBX_APP_NUM_ENTRIES); - rc = qed_dcbx_process_tlv(p_hwfn, &data, p_tbl, pri_tc_tbl, + rc = qed_dcbx_process_tlv(p_hwfn, p_ptt, &data, p_tbl, pri_tc_tbl, num_entries, dcbx_version); if (rc) return rc; @@ -891,7 +901,7 @@ qed_dcbx_mib_update_event(struct qed_hwfn *p_hwfn, return rc; if (type == QED_DCBX_OPERATIONAL_MIB) { - rc = qed_dcbx_process_mib_info(p_hwfn); + rc = qed_dcbx_process_mib_info(p_hwfn, p_ptt); if (!rc) { /* reconfigure tcs of QM queues according * to negotiation results @@ -954,6 +964,7 @@ static void qed_dcbx_update_protocol_data(struct protocol_dcb_data *p_data, p_data->dcb_enable_flag = p_src->arr[type].enable; p_data->dcb_priority = p_src->arr[type].priority; p_data->dcb_tc = p_src->arr[type].tc; + p_data->dcb_dont_add_vlan0 = p_src->arr[type].dont_add_vlan0; } /* Set pf update ramrod command params */ diff --git a/drivers/net/ethernet/qlogic/qed/qed_dcbx.h b/drivers/net/ethernet/qlogic/qed/qed_dcbx.h index a4d688c04e18..01f253ea4b22 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_dcbx.h +++ b/drivers/net/ethernet/qlogic/qed/qed_dcbx.h @@ -55,6 +55,7 @@ struct qed_dcbx_app_data { u8 update; /* Update indication */ u8 priority; /* Priority */ u8 tc; /* Traffic Class */ + bool dont_add_vlan0; /* Do not insert a vlan tag with id 0 */ }; #define QED_DCBX_VERSION_DISABLED 0 diff --git a/drivers/net/ethernet/qlogic/qed/qed_dev.c b/drivers/net/ethernet/qlogic/qed/qed_dev.c index 016ca8a7ec8a..97f073fd3725 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_dev.c +++ b/drivers/net/ethernet/qlogic/qed/qed_dev.c @@ -1706,7 +1706,7 @@ static int qed_vf_start(struct qed_hwfn *p_hwfn, int qed_hw_init(struct qed_dev *cdev, struct qed_hw_init_params *p_params) { struct qed_load_req_params load_req_params; - u32 load_code, param, drv_mb_param; + u32 load_code, resp, param, drv_mb_param; bool b_default_mtu = true; struct qed_hwfn *p_hwfn; int rc = 0, mfw_rc, i; @@ -1852,6 +1852,19 @@ int qed_hw_init(struct qed_dev *cdev, struct qed_hw_init_params *p_params) if (IS_PF(cdev)) { p_hwfn = QED_LEADING_HWFN(cdev); + + /* Get pre-negotiated values for stag, bandwidth etc. */ + DP_VERBOSE(p_hwfn, + QED_MSG_SPQ, + "Sending GET_OEM_UPDATES command to trigger stag/bandwidth attention handling\n"); + drv_mb_param = 1 << DRV_MB_PARAM_DUMMY_OEM_UPDATES_OFFSET; + rc = qed_mcp_cmd(p_hwfn, p_hwfn->p_main_ptt, + DRV_MSG_CODE_GET_OEM_UPDATES, + drv_mb_param, &resp, ¶m); + if (rc) + DP_NOTICE(p_hwfn, + "Failed to send GET_OEM_UPDATES attention request\n"); + drv_mb_param = STORM_FW_VERSION; rc = qed_mcp_cmd(p_hwfn, p_hwfn->p_main_ptt, DRV_MSG_CODE_OV_UPDATE_STORM_FW_VER, diff --git a/drivers/net/ethernet/qlogic/qed/qed_hsi.h b/drivers/net/ethernet/qlogic/qed/qed_hsi.h index 8faceb691657..9b3ef00e5782 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_hsi.h +++ b/drivers/net/ethernet/qlogic/qed/qed_hsi.h @@ -12414,6 +12414,7 @@ struct public_drv_mb { #define DRV_MSG_SET_RESOURCE_VALUE_MSG 0x35000000 #define DRV_MSG_CODE_OV_UPDATE_WOL 0x38000000 #define DRV_MSG_CODE_OV_UPDATE_ESWITCH_MODE 0x39000000 +#define DRV_MSG_CODE_GET_OEM_UPDATES 0x41000000 #define DRV_MSG_CODE_BW_UPDATE_ACK 0x32000000 #define DRV_MSG_CODE_NIG_DRAIN 0x30000000 @@ -12541,6 +12542,9 @@ struct public_drv_mb { #define DRV_MB_PARAM_ESWITCH_MODE_VEB 0x1 #define DRV_MB_PARAM_ESWITCH_MODE_VEPA 0x2 +#define DRV_MB_PARAM_DUMMY_OEM_UPDATES_MASK 0x1 +#define DRV_MB_PARAM_DUMMY_OEM_UPDATES_OFFSET 0 + #define DRV_MB_PARAM_SET_LED_MODE_OPER 0x0 #define DRV_MB_PARAM_SET_LED_MODE_ON 0x1 #define DRV_MB_PARAM_SET_LED_MODE_OFF 0x2 diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.c b/drivers/net/ethernet/qlogic/qed/qed_mcp.c index 5d37ec7e9b0b..58c7eb9d8e1b 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_mcp.c +++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.c @@ -1581,13 +1581,29 @@ static void qed_mcp_update_stag(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt) p_hwfn->mcp_info->func_info.ovlan = (u16)shmem_info.ovlan_stag & FUNC_MF_CFG_OV_STAG_MASK; p_hwfn->hw_info.ovlan = p_hwfn->mcp_info->func_info.ovlan; - if ((p_hwfn->hw_info.hw_mode & BIT(MODE_MF_SD)) && - (p_hwfn->hw_info.ovlan != QED_MCP_VLAN_UNSET)) { - qed_wr(p_hwfn, p_ptt, - NIG_REG_LLH_FUNC_TAG_VALUE, p_hwfn->hw_info.ovlan); + if (test_bit(QED_MF_OVLAN_CLSS, &p_hwfn->cdev->mf_bits)) { + if (p_hwfn->hw_info.ovlan != QED_MCP_VLAN_UNSET) { + qed_wr(p_hwfn, p_ptt, NIG_REG_LLH_FUNC_TAG_VALUE, + p_hwfn->hw_info.ovlan); + qed_wr(p_hwfn, p_ptt, NIG_REG_LLH_FUNC_TAG_EN, 1); + + /* Configure DB to add external vlan to EDPM packets */ + qed_wr(p_hwfn, p_ptt, DORQ_REG_TAG1_OVRD_MODE, 1); + qed_wr(p_hwfn, p_ptt, DORQ_REG_PF_EXT_VID_BB_K2, + p_hwfn->hw_info.ovlan); + } else { + qed_wr(p_hwfn, p_ptt, NIG_REG_LLH_FUNC_TAG_EN, 0); + qed_wr(p_hwfn, p_ptt, NIG_REG_LLH_FUNC_TAG_VALUE, 0); + qed_wr(p_hwfn, p_ptt, DORQ_REG_TAG1_OVRD_MODE, 0); + qed_wr(p_hwfn, p_ptt, DORQ_REG_PF_EXT_VID_BB_K2, 0); + } + qed_sp_pf_update_stag(p_hwfn); } + DP_VERBOSE(p_hwfn, QED_MSG_SP, "ovlan = %d hw_mode = 0x%x\n", + p_hwfn->mcp_info->func_info.ovlan, p_hwfn->hw_info.hw_mode); + /* Acknowledge the MFW */ qed_mcp_cmd(p_hwfn, p_ptt, DRV_MSG_CODE_S_TAG_UPDATE_ACK, 0, &resp, ¶m); diff --git a/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h b/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h index f736f70956fd..2440970882c4 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h +++ b/drivers/net/ethernet/qlogic/qed/qed_reg_addr.h @@ -216,6 +216,12 @@ 0x00c000UL #define DORQ_REG_IFEN \ 0x100040UL +#define DORQ_REG_TAG1_OVRD_MODE \ + 0x1008b4UL +#define DORQ_REG_PF_PCP_BB_K2 \ + 0x1008c4UL +#define DORQ_REG_PF_EXT_VID_BB_K2 \ + 0x1008c8UL #define DORQ_REG_DB_DROP_REASON \ 0x100a2cUL #define DORQ_REG_DB_DROP_DETAILS \ diff --git a/drivers/net/ethernet/qualcomm/qca_7k.c b/drivers/net/ethernet/qualcomm/qca_7k.c index ffe7a16bdfc8..6c8543fb90c0 100644 --- a/drivers/net/ethernet/qualcomm/qca_7k.c +++ b/drivers/net/ethernet/qualcomm/qca_7k.c @@ -45,34 +45,33 @@ qcaspi_read_register(struct qcaspi *qca, u16 reg, u16 *result) { __be16 rx_data; __be16 tx_data; - struct spi_transfer *transfer; - struct spi_message *msg; + struct spi_transfer transfer[2]; + struct spi_message msg; int ret; + memset(transfer, 0, sizeof(transfer)); + + spi_message_init(&msg); + tx_data = cpu_to_be16(QCA7K_SPI_READ | QCA7K_SPI_INTERNAL | reg); + *result = 0; + + transfer[0].tx_buf = &tx_data; + transfer[0].len = QCASPI_CMD_LEN; + transfer[1].rx_buf = &rx_data; + transfer[1].len = QCASPI_CMD_LEN; + + spi_message_add_tail(&transfer[0], &msg); if (qca->legacy_mode) { - msg = &qca->spi_msg1; - transfer = &qca->spi_xfer1; - transfer->tx_buf = &tx_data; - transfer->rx_buf = NULL; - transfer->len = QCASPI_CMD_LEN; - spi_sync(qca->spi_dev, msg); - } else { - msg = &qca->spi_msg2; - transfer = &qca->spi_xfer2[0]; - transfer->tx_buf = &tx_data; - transfer->rx_buf = NULL; - transfer->len = QCASPI_CMD_LEN; - transfer = &qca->spi_xfer2[1]; + spi_sync(qca->spi_dev, &msg); + spi_message_init(&msg); } - transfer->tx_buf = NULL; - transfer->rx_buf = &rx_data; - transfer->len = QCASPI_CMD_LEN; - ret = spi_sync(qca->spi_dev, msg); + spi_message_add_tail(&transfer[1], &msg); + ret = spi_sync(qca->spi_dev, &msg); if (!ret) - ret = msg->status; + ret = msg.status; if (ret) qcaspi_spi_error(qca); @@ -86,35 +85,32 @@ int qcaspi_write_register(struct qcaspi *qca, u16 reg, u16 value) { __be16 tx_data[2]; - struct spi_transfer *transfer; - struct spi_message *msg; + struct spi_transfer transfer[2]; + struct spi_message msg; int ret; + memset(&transfer, 0, sizeof(transfer)); + + spi_message_init(&msg); + tx_data[0] = cpu_to_be16(QCA7K_SPI_WRITE | QCA7K_SPI_INTERNAL | reg); tx_data[1] = cpu_to_be16(value); + transfer[0].tx_buf = &tx_data[0]; + transfer[0].len = QCASPI_CMD_LEN; + transfer[1].tx_buf = &tx_data[1]; + transfer[1].len = QCASPI_CMD_LEN; + + spi_message_add_tail(&transfer[0], &msg); if (qca->legacy_mode) { - msg = &qca->spi_msg1; - transfer = &qca->spi_xfer1; - transfer->tx_buf = &tx_data[0]; - transfer->rx_buf = NULL; - transfer->len = QCASPI_CMD_LEN; - spi_sync(qca->spi_dev, msg); - } else { - msg = &qca->spi_msg2; - transfer = &qca->spi_xfer2[0]; - transfer->tx_buf = &tx_data[0]; - transfer->rx_buf = NULL; - transfer->len = QCASPI_CMD_LEN; - transfer = &qca->spi_xfer2[1]; + spi_sync(qca->spi_dev, &msg); + spi_message_init(&msg); } - transfer->tx_buf = &tx_data[1]; - transfer->rx_buf = NULL; - transfer->len = QCASPI_CMD_LEN; - ret = spi_sync(qca->spi_dev, msg); + spi_message_add_tail(&transfer[1], &msg); + ret = spi_sync(qca->spi_dev, &msg); if (!ret) - ret = msg->status; + ret = msg.status; if (ret) qcaspi_spi_error(qca); diff --git a/drivers/net/ethernet/qualcomm/qca_spi.c b/drivers/net/ethernet/qualcomm/qca_spi.c index 206f0266463e..66b775d462fd 100644 --- a/drivers/net/ethernet/qualcomm/qca_spi.c +++ b/drivers/net/ethernet/qualcomm/qca_spi.c @@ -99,22 +99,24 @@ static u32 qcaspi_write_burst(struct qcaspi *qca, u8 *src, u32 len) { __be16 cmd; - struct spi_message *msg = &qca->spi_msg2; - struct spi_transfer *transfer = &qca->spi_xfer2[0]; + struct spi_message msg; + struct spi_transfer transfer[2]; int ret; + memset(&transfer, 0, sizeof(transfer)); + spi_message_init(&msg); + cmd = cpu_to_be16(QCA7K_SPI_WRITE | QCA7K_SPI_EXTERNAL); - transfer->tx_buf = &cmd; - transfer->rx_buf = NULL; - transfer->len = QCASPI_CMD_LEN; - transfer = &qca->spi_xfer2[1]; - transfer->tx_buf = src; - transfer->rx_buf = NULL; - transfer->len = len; + transfer[0].tx_buf = &cmd; + transfer[0].len = QCASPI_CMD_LEN; + transfer[1].tx_buf = src; + transfer[1].len = len; - ret = spi_sync(qca->spi_dev, msg); + spi_message_add_tail(&transfer[0], &msg); + spi_message_add_tail(&transfer[1], &msg); + ret = spi_sync(qca->spi_dev, &msg); - if (ret || (msg->actual_length != QCASPI_CMD_LEN + len)) { + if (ret || (msg.actual_length != QCASPI_CMD_LEN + len)) { qcaspi_spi_error(qca); return 0; } @@ -125,17 +127,20 @@ qcaspi_write_burst(struct qcaspi *qca, u8 *src, u32 len) static u32 qcaspi_write_legacy(struct qcaspi *qca, u8 *src, u32 len) { - struct spi_message *msg = &qca->spi_msg1; - struct spi_transfer *transfer = &qca->spi_xfer1; + struct spi_message msg; + struct spi_transfer transfer; int ret; - transfer->tx_buf = src; - transfer->rx_buf = NULL; - transfer->len = len; + memset(&transfer, 0, sizeof(transfer)); + spi_message_init(&msg); + + transfer.tx_buf = src; + transfer.len = len; - ret = spi_sync(qca->spi_dev, msg); + spi_message_add_tail(&transfer, &msg); + ret = spi_sync(qca->spi_dev, &msg); - if (ret || (msg->actual_length != len)) { + if (ret || (msg.actual_length != len)) { qcaspi_spi_error(qca); return 0; } @@ -146,23 +151,25 @@ qcaspi_write_legacy(struct qcaspi *qca, u8 *src, u32 len) static u32 qcaspi_read_burst(struct qcaspi *qca, u8 *dst, u32 len) { - struct spi_message *msg = &qca->spi_msg2; + struct spi_message msg; __be16 cmd; - struct spi_transfer *transfer = &qca->spi_xfer2[0]; + struct spi_transfer transfer[2]; int ret; + memset(&transfer, 0, sizeof(transfer)); + spi_message_init(&msg); + cmd = cpu_to_be16(QCA7K_SPI_READ | QCA7K_SPI_EXTERNAL); - transfer->tx_buf = &cmd; - transfer->rx_buf = NULL; - transfer->len = QCASPI_CMD_LEN; - transfer = &qca->spi_xfer2[1]; - transfer->tx_buf = NULL; - transfer->rx_buf = dst; - transfer->len = len; + transfer[0].tx_buf = &cmd; + transfer[0].len = QCASPI_CMD_LEN; + transfer[1].rx_buf = dst; + transfer[1].len = len; - ret = spi_sync(qca->spi_dev, msg); + spi_message_add_tail(&transfer[0], &msg); + spi_message_add_tail(&transfer[1], &msg); + ret = spi_sync(qca->spi_dev, &msg); - if (ret || (msg->actual_length != QCASPI_CMD_LEN + len)) { + if (ret || (msg.actual_length != QCASPI_CMD_LEN + len)) { qcaspi_spi_error(qca); return 0; } @@ -173,17 +180,20 @@ qcaspi_read_burst(struct qcaspi *qca, u8 *dst, u32 len) static u32 qcaspi_read_legacy(struct qcaspi *qca, u8 *dst, u32 len) { - struct spi_message *msg = &qca->spi_msg1; - struct spi_transfer *transfer = &qca->spi_xfer1; + struct spi_message msg; + struct spi_transfer transfer; int ret; - transfer->tx_buf = NULL; - transfer->rx_buf = dst; - transfer->len = len; + memset(&transfer, 0, sizeof(transfer)); + spi_message_init(&msg); - ret = spi_sync(qca->spi_dev, msg); + transfer.rx_buf = dst; + transfer.len = len; - if (ret || (msg->actual_length != len)) { + spi_message_add_tail(&transfer, &msg); + ret = spi_sync(qca->spi_dev, &msg); + + if (ret || (msg.actual_length != len)) { qcaspi_spi_error(qca); return 0; } @@ -195,19 +205,23 @@ static int qcaspi_tx_cmd(struct qcaspi *qca, u16 cmd) { __be16 tx_data; - struct spi_message *msg = &qca->spi_msg1; - struct spi_transfer *transfer = &qca->spi_xfer1; + struct spi_message msg; + struct spi_transfer transfer; int ret; + memset(&transfer, 0, sizeof(transfer)); + + spi_message_init(&msg); + tx_data = cpu_to_be16(cmd); - transfer->len = sizeof(tx_data); - transfer->tx_buf = &tx_data; - transfer->rx_buf = NULL; + transfer.len = sizeof(cmd); + transfer.tx_buf = &tx_data; + spi_message_add_tail(&transfer, &msg); - ret = spi_sync(qca->spi_dev, msg); + ret = spi_sync(qca->spi_dev, &msg); if (!ret) - ret = msg->status; + ret = msg.status; if (ret) qcaspi_spi_error(qca); @@ -835,16 +849,6 @@ qcaspi_netdev_setup(struct net_device *dev) qca = netdev_priv(dev); memset(qca, 0, sizeof(struct qcaspi)); - memset(&qca->spi_xfer1, 0, sizeof(struct spi_transfer)); - memset(&qca->spi_xfer2, 0, sizeof(struct spi_transfer) * 2); - - spi_message_init(&qca->spi_msg1); - spi_message_add_tail(&qca->spi_xfer1, &qca->spi_msg1); - - spi_message_init(&qca->spi_msg2); - spi_message_add_tail(&qca->spi_xfer2[0], &qca->spi_msg2); - spi_message_add_tail(&qca->spi_xfer2[1], &qca->spi_msg2); - memset(&qca->txr, 0, sizeof(qca->txr)); qca->txr.count = TX_RING_MAX_LEN; } diff --git a/drivers/net/ethernet/qualcomm/qca_spi.h b/drivers/net/ethernet/qualcomm/qca_spi.h index fc4beb1b32d1..fc0e98726b36 100644 --- a/drivers/net/ethernet/qualcomm/qca_spi.h +++ b/drivers/net/ethernet/qualcomm/qca_spi.h @@ -83,11 +83,6 @@ struct qcaspi { struct tx_ring txr; struct qcaspi_stats stats; - struct spi_message spi_msg1; - struct spi_message spi_msg2; - struct spi_transfer spi_xfer1; - struct spi_transfer spi_xfer2[2]; - u8 *rx_buffer; u32 buffer_size; u8 sync; diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c index 0efa977c422d..ab30aaeac6d3 100644 --- a/drivers/net/ethernet/realtek/r8169.c +++ b/drivers/net/ethernet/realtek/r8169.c @@ -13,6 +13,7 @@ #include <linux/pci.h> #include <linux/netdevice.h> #include <linux/etherdevice.h> +#include <linux/clk.h> #include <linux/delay.h> #include <linux/ethtool.h> #include <linux/phy.h> @@ -218,6 +219,7 @@ static const struct pci_device_id rtl8169_pci_tbl[] = { { PCI_DEVICE(PCI_VENDOR_ID_REALTEK, 0x8161), 0, 0, RTL_CFG_1 }, { PCI_DEVICE(PCI_VENDOR_ID_REALTEK, 0x8167), 0, 0, RTL_CFG_0 }, { PCI_DEVICE(PCI_VENDOR_ID_REALTEK, 0x8168), 0, 0, RTL_CFG_1 }, + { PCI_DEVICE(PCI_VENDOR_ID_NCUBE, 0x8168), 0, 0, RTL_CFG_1 }, { PCI_DEVICE(PCI_VENDOR_ID_REALTEK, 0x8169), 0, 0, RTL_CFG_0 }, { PCI_VENDOR_ID_DLINK, 0x4300, PCI_VENDOR_ID_DLINK, 0x4b10, 0, 0, RTL_CFG_1 }, @@ -630,7 +632,7 @@ struct rtl8169_tc_offsets { }; enum rtl_flag { - RTL_FLAG_TASK_ENABLED, + RTL_FLAG_TASK_ENABLED = 0, RTL_FLAG_TASK_SLOW_PENDING, RTL_FLAG_TASK_RESET_PENDING, RTL_FLAG_MAX @@ -664,6 +666,7 @@ struct rtl8169_private { u16 event_slow; const struct rtl_coalesce_info *coalesce_info; + struct clk *clk; struct mdio_ops { void (*write)(struct rtl8169_private *, int, int); @@ -4068,6 +4071,15 @@ static void rtl8169_init_phy(struct net_device *dev, struct rtl8169_private *tp) phy_speed_up(dev->phydev); genphy_soft_reset(dev->phydev); + + /* It was reported that chip version 33 ends up with 10MBit/Half on a + * 1GBit link after resuming from S3. For whatever reason the PHY on + * this chip doesn't properly start a renegotiation when soft-reset. + * Explicitly requesting a renegotiation fixes this. + */ + if (tp->mac_version == RTL_GIGA_MAC_VER_33 && + dev->phydev->autoneg == AUTONEG_ENABLE) + phy_restart_aneg(dev->phydev); } static void rtl_rar_set(struct rtl8169_private *tp, u8 *addr) @@ -4522,7 +4534,7 @@ static void rtl8169_hw_reset(struct rtl8169_private *tp) rtl_hw_reset(tp); } -static void rtl_set_rx_tx_config_registers(struct rtl8169_private *tp) +static void rtl_set_tx_config_registers(struct rtl8169_private *tp) { /* Set DMA burst size and Interframe Gap Time */ RTL_W32(tp, TxConfig, (TX_DMA_BURST << TxDMAShift) | @@ -4633,12 +4645,14 @@ static void rtl_hw_start(struct rtl8169_private *tp) rtl_set_rx_max_size(tp); rtl_set_rx_tx_desc_registers(tp); - rtl_set_rx_tx_config_registers(tp); RTL_W8(tp, Cfg9346, Cfg9346_Lock); /* Initially a 10 us delay. Turned it into a PCI commit. - FR */ RTL_R8(tp, IntrMask); RTL_W8(tp, ChipCmd, CmdTxEnb | CmdRxEnb); + rtl_init_rxcfg(tp); + rtl_set_tx_config_registers(tp); + rtl_set_rx_mode(tp->dev); /* no early-rx interrupts */ RTL_W16(tp, MultiIntr, RTL_R16(tp, MultiIntr) & 0xf000); @@ -4772,12 +4786,14 @@ static void rtl_pcie_state_l2l3_enable(struct rtl8169_private *tp, bool enable) static void rtl_hw_aspm_clkreq_enable(struct rtl8169_private *tp, bool enable) { if (enable) { - RTL_W8(tp, Config2, RTL_R8(tp, Config2) | ClkReqEn); RTL_W8(tp, Config5, RTL_R8(tp, Config5) | ASPM_en); + RTL_W8(tp, Config2, RTL_R8(tp, Config2) | ClkReqEn); } else { RTL_W8(tp, Config2, RTL_R8(tp, Config2) & ~ClkReqEn); RTL_W8(tp, Config5, RTL_R8(tp, Config5) & ~ASPM_en); } + + udelay(10); } static void rtl_hw_start_8168bb(struct rtl8169_private *tp) @@ -5622,6 +5638,8 @@ static void rtl_hw_start_8402(struct rtl8169_private *tp) static void rtl_hw_start_8106(struct rtl8169_private *tp) { + rtl_hw_aspm_clkreq_enable(tp, false); + /* Force LAN exit from ASPM if Rx/Tx are not idle */ RTL_W32(tp, FuncEvent, RTL_R32(tp, FuncEvent) | 0x002800); @@ -5630,6 +5648,7 @@ static void rtl_hw_start_8106(struct rtl8169_private *tp) RTL_W8(tp, DLLPR, RTL_R8(tp, DLLPR) & ~PFM_EN); rtl_pcie_state_l2l3_enable(tp, false); + rtl_hw_aspm_clkreq_enable(tp, true); } static void rtl_hw_start_8101(struct rtl8169_private *tp) @@ -6652,7 +6671,8 @@ static int rtl8169_close(struct net_device *dev) rtl8169_update_counters(tp); rtl_lock_work(tp); - clear_bit(RTL_FLAG_TASK_ENABLED, tp->wk.flags); + /* Clear all task flags */ + bitmap_zero(tp->wk.flags, RTL_FLAG_MAX); rtl8169_down(dev); rtl_unlock_work(tp); @@ -6835,7 +6855,9 @@ static void rtl8169_net_suspend(struct net_device *dev) rtl_lock_work(tp); napi_disable(&tp->napi); - clear_bit(RTL_FLAG_TASK_ENABLED, tp->wk.flags); + /* Clear all task flags */ + bitmap_zero(tp->wk.flags, RTL_FLAG_MAX); + rtl_unlock_work(tp); rtl_pll_power_down(tp); @@ -7251,6 +7273,11 @@ static int rtl_jumbo_max(struct rtl8169_private *tp) } } +static void rtl_disable_clk(void *data) +{ + clk_disable_unprepare(data); +} + static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) { const struct rtl_cfg_info *cfg = rtl_cfg_infos + ent->driver_data; @@ -7271,6 +7298,32 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) tp->msg_enable = netif_msg_init(debug.msg_enable, R8169_MSG_DEFAULT); tp->supports_gmii = cfg->has_gmii; + /* Get the *optional* external "ether_clk" used on some boards */ + tp->clk = devm_clk_get(&pdev->dev, "ether_clk"); + if (IS_ERR(tp->clk)) { + rc = PTR_ERR(tp->clk); + if (rc == -ENOENT) { + /* clk-core allows NULL (for suspend / resume) */ + tp->clk = NULL; + } else if (rc == -EPROBE_DEFER) { + return rc; + } else { + dev_err(&pdev->dev, "failed to get clk: %d\n", rc); + return rc; + } + } else { + rc = clk_prepare_enable(tp->clk); + if (rc) { + dev_err(&pdev->dev, "failed to enable clk: %d\n", rc); + return rc; + } + + rc = devm_add_action_or_reset(&pdev->dev, rtl_disable_clk, + tp->clk); + if (rc) + return rc; + } + /* enable device (incl. PCI PM wakeup and hotplug setup) */ rc = pcim_enable_device(pdev); if (rc < 0) { diff --git a/drivers/net/ethernet/renesas/Kconfig b/drivers/net/ethernet/renesas/Kconfig index f3f7477043ce..bb0ebdfd4459 100644 --- a/drivers/net/ethernet/renesas/Kconfig +++ b/drivers/net/ethernet/renesas/Kconfig @@ -1,3 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 # # Renesas device configuration # diff --git a/drivers/net/ethernet/renesas/Makefile b/drivers/net/ethernet/renesas/Makefile index a05102a7df02..f21ab8c02af0 100644 --- a/drivers/net/ethernet/renesas/Makefile +++ b/drivers/net/ethernet/renesas/Makefile @@ -1,3 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0 # # Makefile for the Renesas device drivers. # diff --git a/drivers/net/ethernet/renesas/ravb.h b/drivers/net/ethernet/renesas/ravb.h index 1470fc12282b..9b6bf557a2f5 100644 --- a/drivers/net/ethernet/renesas/ravb.h +++ b/drivers/net/ethernet/renesas/ravb.h @@ -428,6 +428,7 @@ enum EIS_BIT { EIS_CULF1 = 0x00000080, EIS_TFFF = 0x00000100, EIS_QFS = 0x00010000, + EIS_RESERVED = (GENMASK(31, 17) | GENMASK(15, 11)), }; /* RIC0 */ @@ -472,6 +473,7 @@ enum RIS0_BIT { RIS0_FRF15 = 0x00008000, RIS0_FRF16 = 0x00010000, RIS0_FRF17 = 0x00020000, + RIS0_RESERVED = GENMASK(31, 18), }; /* RIC1 */ @@ -528,6 +530,7 @@ enum RIS2_BIT { RIS2_QFF16 = 0x00010000, RIS2_QFF17 = 0x00020000, RIS2_RFFF = 0x80000000, + RIS2_RESERVED = GENMASK(30, 18), }; /* TIC */ @@ -544,6 +547,7 @@ enum TIS_BIT { TIS_FTF1 = 0x00000002, /* Undocumented? */ TIS_TFUF = 0x00000100, TIS_TFWF = 0x00000200, + TIS_RESERVED = (GENMASK(31, 20) | GENMASK(15, 12) | GENMASK(7, 4)) }; /* ISS */ @@ -617,6 +621,7 @@ enum GIC_BIT { enum GIS_BIT { GIS_PTCF = 0x00000001, /* Undocumented? */ GIS_PTMF = 0x00000004, + GIS_RESERVED = GENMASK(15, 10), }; /* GIE (R-Car Gen3 only) */ diff --git a/drivers/net/ethernet/renesas/ravb_main.c b/drivers/net/ethernet/renesas/ravb_main.c index aff5516b781e..d6f753925352 100644 --- a/drivers/net/ethernet/renesas/ravb_main.c +++ b/drivers/net/ethernet/renesas/ravb_main.c @@ -739,10 +739,11 @@ static void ravb_error_interrupt(struct net_device *ndev) u32 eis, ris2; eis = ravb_read(ndev, EIS); - ravb_write(ndev, ~EIS_QFS, EIS); + ravb_write(ndev, ~(EIS_QFS | EIS_RESERVED), EIS); if (eis & EIS_QFS) { ris2 = ravb_read(ndev, RIS2); - ravb_write(ndev, ~(RIS2_QFF0 | RIS2_RFFF), RIS2); + ravb_write(ndev, ~(RIS2_QFF0 | RIS2_RFFF | RIS2_RESERVED), + RIS2); /* Receive Descriptor Empty int */ if (ris2 & RIS2_QFF0) @@ -795,7 +796,7 @@ static bool ravb_timestamp_interrupt(struct net_device *ndev) u32 tis = ravb_read(ndev, TIS); if (tis & TIS_TFUF) { - ravb_write(ndev, ~TIS_TFUF, TIS); + ravb_write(ndev, ~(TIS_TFUF | TIS_RESERVED), TIS); ravb_get_tx_tstamp(ndev); return true; } @@ -930,7 +931,7 @@ static int ravb_poll(struct napi_struct *napi, int budget) /* Processing RX Descriptor Ring */ if (ris0 & mask) { /* Clear RX interrupt */ - ravb_write(ndev, ~mask, RIS0); + ravb_write(ndev, ~(mask | RIS0_RESERVED), RIS0); if (ravb_rx(ndev, "a, q)) goto out; } @@ -938,7 +939,7 @@ static int ravb_poll(struct napi_struct *napi, int budget) if (tis & mask) { spin_lock_irqsave(&priv->lock, flags); /* Clear TX interrupt */ - ravb_write(ndev, ~mask, TIS); + ravb_write(ndev, ~(mask | TIS_RESERVED), TIS); ravb_tx_free(ndev, q, true); netif_wake_subqueue(ndev, q); mmiowb(); diff --git a/drivers/net/ethernet/renesas/ravb_ptp.c b/drivers/net/ethernet/renesas/ravb_ptp.c index eede70ec37f8..dce2a40a31e3 100644 --- a/drivers/net/ethernet/renesas/ravb_ptp.c +++ b/drivers/net/ethernet/renesas/ravb_ptp.c @@ -1,13 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* PTP 1588 clock using the Renesas Ethernet AVB * * Copyright (C) 2013-2015 Renesas Electronics Corporation * Copyright (C) 2015 Renesas Solutions Corp. * Copyright (C) 2015-2016 Cogent Embedded, Inc. <source@cogentembedded.com> - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. */ #include "ravb.h" @@ -319,7 +315,7 @@ void ravb_ptp_interrupt(struct net_device *ndev) } } - ravb_write(ndev, ~gis, GIS); + ravb_write(ndev, ~(gis | GIS_RESERVED), GIS); } void ravb_ptp_init(struct net_device *ndev, struct platform_device *pdev) diff --git a/drivers/net/ethernet/renesas/sh_eth.c b/drivers/net/ethernet/renesas/sh_eth.c index ad4433d59237..f27a0dc8c563 100644 --- a/drivers/net/ethernet/renesas/sh_eth.c +++ b/drivers/net/ethernet/renesas/sh_eth.c @@ -798,6 +798,41 @@ static struct sh_eth_cpu_data r8a77980_data = { .magic = 1, .cexcr = 1, }; + +/* R7S9210 */ +static struct sh_eth_cpu_data r7s9210_data = { + .soft_reset = sh_eth_soft_reset, + + .set_duplex = sh_eth_set_duplex, + .set_rate = sh_eth_set_rate_rcar, + + .register_type = SH_ETH_REG_FAST_SH4, + + .edtrr_trns = EDTRR_TRNS_ETHER, + .ecsr_value = ECSR_ICD, + .ecsipr_value = ECSIPR_ICDIP, + .eesipr_value = EESIPR_TWBIP | EESIPR_TABTIP | EESIPR_RABTIP | + EESIPR_RFCOFIP | EESIPR_ECIIP | EESIPR_FTCIP | + EESIPR_TDEIP | EESIPR_TFUFIP | EESIPR_FRIP | + EESIPR_RDEIP | EESIPR_RFOFIP | EESIPR_CNDIP | + EESIPR_DLCIP | EESIPR_CDIP | EESIPR_TROIP | + EESIPR_RMAFIP | EESIPR_RRFIP | EESIPR_RTLFIP | + EESIPR_RTSFIP | EESIPR_PREIP | EESIPR_CERFIP, + + .tx_check = EESR_FTC | EESR_CND | EESR_DLC | EESR_CD | EESR_TRO, + .eesr_err_check = EESR_TWB | EESR_TABT | EESR_RABT | EESR_RFE | + EESR_RDE | EESR_RFRMER | EESR_TFE | EESR_TDE, + + .fdr_value = 0x0000070f, + + .apr = 1, + .mpr = 1, + .tpauser = 1, + .hw_swap = 1, + .rpadir = 1, + .no_ade = 1, + .xdfar_rw = 1, +}; #endif /* CONFIG_OF */ static void sh_eth_set_rate_sh7724(struct net_device *ndev) @@ -3121,6 +3156,7 @@ static const struct of_device_id sh_eth_match_table[] = { { .compatible = "renesas,ether-r8a7794", .data = &rcar_gen2_data }, { .compatible = "renesas,gether-r8a77980", .data = &r8a77980_data }, { .compatible = "renesas,ether-r7s72100", .data = &r7s72100_data }, + { .compatible = "renesas,ether-r7s9210", .data = &r7s9210_data }, { .compatible = "renesas,rcar-gen1-ether", .data = &rcar_gen1_data }, { .compatible = "renesas,rcar-gen2-ether", .data = &rcar_gen2_data }, { } diff --git a/drivers/net/ethernet/seeq/ether3.c b/drivers/net/ethernet/seeq/ether3.c index c5bc124b41a9..d1bb73bf9914 100644 --- a/drivers/net/ethernet/seeq/ether3.c +++ b/drivers/net/ethernet/seeq/ether3.c @@ -77,7 +77,8 @@ static void ether3_setmulticastlist(struct net_device *dev); static int ether3_rx(struct net_device *dev, unsigned int maxcnt); static void ether3_tx(struct net_device *dev); static int ether3_open (struct net_device *dev); -static int ether3_sendpacket (struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t ether3_sendpacket(struct sk_buff *skb, + struct net_device *dev); static irqreturn_t ether3_interrupt (int irq, void *dev_id); static int ether3_close (struct net_device *dev); static void ether3_setmulticastlist (struct net_device *dev); @@ -481,7 +482,7 @@ static void ether3_timeout(struct net_device *dev) /* * Transmit a packet */ -static int +static netdev_tx_t ether3_sendpacket(struct sk_buff *skb, struct net_device *dev) { unsigned long flags; diff --git a/drivers/net/ethernet/seeq/sgiseeq.c b/drivers/net/ethernet/seeq/sgiseeq.c index 573691bc3b71..70cce63a6081 100644 --- a/drivers/net/ethernet/seeq/sgiseeq.c +++ b/drivers/net/ethernet/seeq/sgiseeq.c @@ -578,7 +578,8 @@ static inline int sgiseeq_reset(struct net_device *dev) return 0; } -static int sgiseeq_start_xmit(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t +sgiseeq_start_xmit(struct sk_buff *skb, struct net_device *dev) { struct sgiseeq_private *sp = netdev_priv(dev); struct hpc3_ethregs *hregs = sp->hregs; diff --git a/drivers/net/ethernet/sgi/ioc3-eth.c b/drivers/net/ethernet/sgi/ioc3-eth.c index 18d533fdf14c..3140999642ba 100644 --- a/drivers/net/ethernet/sgi/ioc3-eth.c +++ b/drivers/net/ethernet/sgi/ioc3-eth.c @@ -99,7 +99,7 @@ struct ioc3_private { static int ioc3_ioctl(struct net_device *dev, struct ifreq *rq, int cmd); static void ioc3_set_multicast_list(struct net_device *dev); -static int ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev); +static netdev_tx_t ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev); static void ioc3_timeout(struct net_device *dev); static inline unsigned int ioc3_hash(const unsigned char *addr); static inline void ioc3_stop(struct ioc3_private *ip); @@ -1390,7 +1390,7 @@ static struct pci_driver ioc3_driver = { .remove = ioc3_remove_one, }; -static int ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t ioc3_start_xmit(struct sk_buff *skb, struct net_device *dev) { unsigned long data; struct ioc3_private *ip = netdev_priv(dev); diff --git a/drivers/net/ethernet/sgi/meth.c b/drivers/net/ethernet/sgi/meth.c index ea55abd62ec7..703fbbefea44 100644 --- a/drivers/net/ethernet/sgi/meth.c +++ b/drivers/net/ethernet/sgi/meth.c @@ -697,7 +697,7 @@ static void meth_add_to_tx_ring(struct meth_private *priv, struct sk_buff *skb) /* * Transmit a packet (called by the kernel) */ -static int meth_tx(struct sk_buff *skb, struct net_device *dev) +static netdev_tx_t meth_tx(struct sk_buff *skb, struct net_device *dev) { struct meth_private *priv = netdev_priv(dev); unsigned long flags; diff --git a/drivers/net/ethernet/stmicro/stmmac/Kconfig b/drivers/net/ethernet/stmicro/stmmac/Kconfig index bf4acebb6bcd..324049eebb9b 100644 --- a/drivers/net/ethernet/stmicro/stmmac/Kconfig +++ b/drivers/net/ethernet/stmicro/stmmac/Kconfig @@ -110,7 +110,7 @@ config DWMAC_ROCKCHIP config DWMAC_SOCFPGA tristate "SOCFPGA dwmac support" - default ARCH_SOCFPGA + default (ARCH_SOCFPGA || ARCH_STRATIX10) depends on OF && (ARCH_SOCFPGA || ARCH_STRATIX10 || COMPILE_TEST) select MFD_SYSCON help diff --git a/drivers/net/ethernet/stmicro/stmmac/common.h b/drivers/net/ethernet/stmicro/stmmac/common.h index 1854f270ad66..b1b305f8f414 100644 --- a/drivers/net/ethernet/stmicro/stmmac/common.h +++ b/drivers/net/ethernet/stmicro/stmmac/common.h @@ -258,10 +258,10 @@ struct stmmac_safety_stats { #define MAX_DMA_RIWT 0xff #define MIN_DMA_RIWT 0x20 /* Tx coalesce parameters */ -#define STMMAC_COAL_TX_TIMER 40000 +#define STMMAC_COAL_TX_TIMER 1000 #define STMMAC_MAX_COAL_TX_TICK 100000 #define STMMAC_TX_MAX_FRAMES 256 -#define STMMAC_TX_FRAMES 64 +#define STMMAC_TX_FRAMES 25 /* Packets types */ enum packets_types { diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac.h b/drivers/net/ethernet/stmicro/stmmac/stmmac.h index 76649adf8fb0..63e1064b27a2 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac.h +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac.h @@ -48,6 +48,8 @@ struct stmmac_tx_info { /* Frequently used values are kept adjacent for cache effect */ struct stmmac_tx_queue { + u32 tx_count_frames; + struct timer_list txtimer; u32 queue_index; struct stmmac_priv *priv_data; struct dma_extended_desc *dma_etx ____cacheline_aligned_in_smp; @@ -73,7 +75,14 @@ struct stmmac_rx_queue { u32 rx_zeroc_thresh; dma_addr_t dma_rx_phy; u32 rx_tail_addr; +}; + +struct stmmac_channel { struct napi_struct napi ____cacheline_aligned_in_smp; + struct stmmac_priv *priv_data; + u32 index; + int has_rx; + int has_tx; }; struct stmmac_tc_entry { @@ -109,15 +118,12 @@ struct stmmac_pps_cfg { struct stmmac_priv { /* Frequently used values are kept adjacent for cache effect */ - u32 tx_count_frames; u32 tx_coal_frames; u32 tx_coal_timer; - bool tx_timer_armed; int tx_coalesce; int hwts_tx_en; bool tx_path_in_lpi_mode; - struct timer_list txtimer; bool tso; unsigned int dma_buf_sz; @@ -138,6 +144,9 @@ struct stmmac_priv { /* TX Queue */ struct stmmac_tx_queue tx_queue[MTL_MAX_TX_QUEUES]; + /* Generic channel for NAPI */ + struct stmmac_channel channel[STMMAC_CH_MAX]; + bool oldlink; int speed; int oldduplex; diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index ff1ffb46198a..75896d6ba6e2 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -148,12 +148,14 @@ static void stmmac_verify_args(void) static void stmmac_disable_all_queues(struct stmmac_priv *priv) { u32 rx_queues_cnt = priv->plat->rx_queues_to_use; + u32 tx_queues_cnt = priv->plat->tx_queues_to_use; + u32 maxq = max(rx_queues_cnt, tx_queues_cnt); u32 queue; - for (queue = 0; queue < rx_queues_cnt; queue++) { - struct stmmac_rx_queue *rx_q = &priv->rx_queue[queue]; + for (queue = 0; queue < maxq; queue++) { + struct stmmac_channel *ch = &priv->channel[queue]; - napi_disable(&rx_q->napi); + napi_disable(&ch->napi); } } @@ -164,12 +166,14 @@ static void stmmac_disable_all_queues(struct stmmac_priv *priv) static void stmmac_enable_all_queues(struct stmmac_priv *priv) { u32 rx_queues_cnt = priv->plat->rx_queues_to_use; + u32 tx_queues_cnt = priv->plat->tx_queues_to_use; + u32 maxq = max(rx_queues_cnt, tx_queues_cnt); u32 queue; - for (queue = 0; queue < rx_queues_cnt; queue++) { - struct stmmac_rx_queue *rx_q = &priv->rx_queue[queue]; + for (queue = 0; queue < maxq; queue++) { + struct stmmac_channel *ch = &priv->channel[queue]; - napi_enable(&rx_q->napi); + napi_enable(&ch->napi); } } @@ -1843,18 +1847,18 @@ static void stmmac_dma_operation_mode(struct stmmac_priv *priv) * @queue: TX queue index * Description: it reclaims the transmit resources after transmission completes. */ -static void stmmac_tx_clean(struct stmmac_priv *priv, u32 queue) +static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue) { struct stmmac_tx_queue *tx_q = &priv->tx_queue[queue]; unsigned int bytes_compl = 0, pkts_compl = 0; - unsigned int entry; + unsigned int entry, count = 0; - netif_tx_lock(priv->dev); + __netif_tx_lock_bh(netdev_get_tx_queue(priv->dev, queue)); priv->xstats.tx_clean++; entry = tx_q->dirty_tx; - while (entry != tx_q->cur_tx) { + while ((entry != tx_q->cur_tx) && (count < budget)) { struct sk_buff *skb = tx_q->tx_skbuff[entry]; struct dma_desc *p; int status; @@ -1870,6 +1874,8 @@ static void stmmac_tx_clean(struct stmmac_priv *priv, u32 queue) if (unlikely(status & tx_dma_own)) break; + count++; + /* Make sure descriptor fields are read after reading * the own bit. */ @@ -1937,7 +1943,10 @@ static void stmmac_tx_clean(struct stmmac_priv *priv, u32 queue) stmmac_enable_eee_mode(priv); mod_timer(&priv->eee_ctrl_timer, STMMAC_LPI_T(eee_timer)); } - netif_tx_unlock(priv->dev); + + __netif_tx_unlock_bh(netdev_get_tx_queue(priv->dev, queue)); + + return count; } /** @@ -2020,6 +2029,33 @@ static bool stmmac_safety_feat_interrupt(struct stmmac_priv *priv) return false; } +static int stmmac_napi_check(struct stmmac_priv *priv, u32 chan) +{ + int status = stmmac_dma_interrupt_status(priv, priv->ioaddr, + &priv->xstats, chan); + struct stmmac_channel *ch = &priv->channel[chan]; + bool needs_work = false; + + if ((status & handle_rx) && ch->has_rx) { + needs_work = true; + } else { + status &= ~handle_rx; + } + + if ((status & handle_tx) && ch->has_tx) { + needs_work = true; + } else { + status &= ~handle_tx; + } + + if (needs_work && napi_schedule_prep(&ch->napi)) { + stmmac_disable_dma_irq(priv, priv->ioaddr, chan); + __napi_schedule(&ch->napi); + } + + return status; +} + /** * stmmac_dma_interrupt - DMA ISR * @priv: driver private structure @@ -2034,57 +2070,14 @@ static void stmmac_dma_interrupt(struct stmmac_priv *priv) u32 channels_to_check = tx_channel_count > rx_channel_count ? tx_channel_count : rx_channel_count; u32 chan; - bool poll_scheduled = false; int status[max_t(u32, MTL_MAX_TX_QUEUES, MTL_MAX_RX_QUEUES)]; /* Make sure we never check beyond our status buffer. */ if (WARN_ON_ONCE(channels_to_check > ARRAY_SIZE(status))) channels_to_check = ARRAY_SIZE(status); - /* Each DMA channel can be used for rx and tx simultaneously, yet - * napi_struct is embedded in struct stmmac_rx_queue rather than in a - * stmmac_channel struct. - * Because of this, stmmac_poll currently checks (and possibly wakes) - * all tx queues rather than just a single tx queue. - */ for (chan = 0; chan < channels_to_check; chan++) - status[chan] = stmmac_dma_interrupt_status(priv, priv->ioaddr, - &priv->xstats, chan); - - for (chan = 0; chan < rx_channel_count; chan++) { - if (likely(status[chan] & handle_rx)) { - struct stmmac_rx_queue *rx_q = &priv->rx_queue[chan]; - - if (likely(napi_schedule_prep(&rx_q->napi))) { - stmmac_disable_dma_irq(priv, priv->ioaddr, chan); - __napi_schedule(&rx_q->napi); - poll_scheduled = true; - } - } - } - - /* If we scheduled poll, we already know that tx queues will be checked. - * If we didn't schedule poll, see if any DMA channel (used by tx) has a - * completed transmission, if so, call stmmac_poll (once). - */ - if (!poll_scheduled) { - for (chan = 0; chan < tx_channel_count; chan++) { - if (status[chan] & handle_tx) { - /* It doesn't matter what rx queue we choose - * here. We use 0 since it always exists. - */ - struct stmmac_rx_queue *rx_q = - &priv->rx_queue[0]; - - if (likely(napi_schedule_prep(&rx_q->napi))) { - stmmac_disable_dma_irq(priv, - priv->ioaddr, chan); - __napi_schedule(&rx_q->napi); - } - break; - } - } - } + status[chan] = stmmac_napi_check(priv, chan); for (chan = 0; chan < tx_channel_count; chan++) { if (unlikely(status[chan] & tx_hard_error_bump_tc)) { @@ -2220,8 +2213,7 @@ static int stmmac_init_dma_engine(struct stmmac_priv *priv) stmmac_init_tx_chan(priv, priv->ioaddr, priv->plat->dma_cfg, tx_q->dma_tx_phy, chan); - tx_q->tx_tail_addr = tx_q->dma_tx_phy + - (DMA_TX_SIZE * sizeof(struct dma_desc)); + tx_q->tx_tail_addr = tx_q->dma_tx_phy; stmmac_set_tx_tail_ptr(priv, priv->ioaddr, tx_q->tx_tail_addr, chan); } @@ -2233,6 +2225,13 @@ static int stmmac_init_dma_engine(struct stmmac_priv *priv) return ret; } +static void stmmac_tx_timer_arm(struct stmmac_priv *priv, u32 queue) +{ + struct stmmac_tx_queue *tx_q = &priv->tx_queue[queue]; + + mod_timer(&tx_q->txtimer, STMMAC_COAL_TIMER(priv->tx_coal_timer)); +} + /** * stmmac_tx_timer - mitigation sw timer for tx. * @data: data pointer @@ -2241,13 +2240,14 @@ static int stmmac_init_dma_engine(struct stmmac_priv *priv) */ static void stmmac_tx_timer(struct timer_list *t) { - struct stmmac_priv *priv = from_timer(priv, t, txtimer); - u32 tx_queues_count = priv->plat->tx_queues_to_use; - u32 queue; + struct stmmac_tx_queue *tx_q = from_timer(tx_q, t, txtimer); + struct stmmac_priv *priv = tx_q->priv_data; + struct stmmac_channel *ch; + + ch = &priv->channel[tx_q->queue_index]; - /* let's scan all the tx queues */ - for (queue = 0; queue < tx_queues_count; queue++) - stmmac_tx_clean(priv, queue); + if (likely(napi_schedule_prep(&ch->napi))) + __napi_schedule(&ch->napi); } /** @@ -2260,11 +2260,17 @@ static void stmmac_tx_timer(struct timer_list *t) */ static void stmmac_init_tx_coalesce(struct stmmac_priv *priv) { + u32 tx_channel_count = priv->plat->tx_queues_to_use; + u32 chan; + priv->tx_coal_frames = STMMAC_TX_FRAMES; priv->tx_coal_timer = STMMAC_COAL_TX_TIMER; - timer_setup(&priv->txtimer, stmmac_tx_timer, 0); - priv->txtimer.expires = STMMAC_COAL_TIMER(priv->tx_coal_timer); - add_timer(&priv->txtimer); + + for (chan = 0; chan < tx_channel_count; chan++) { + struct stmmac_tx_queue *tx_q = &priv->tx_queue[chan]; + + timer_setup(&tx_q->txtimer, stmmac_tx_timer, 0); + } } static void stmmac_set_rings_length(struct stmmac_priv *priv) @@ -2592,6 +2598,7 @@ static void stmmac_hw_teardown(struct net_device *dev) static int stmmac_open(struct net_device *dev) { struct stmmac_priv *priv = netdev_priv(dev); + u32 chan; int ret; stmmac_check_ether_addr(priv); @@ -2688,7 +2695,9 @@ irq_error: if (dev->phydev) phy_stop(dev->phydev); - del_timer_sync(&priv->txtimer); + for (chan = 0; chan < priv->plat->tx_queues_to_use; chan++) + del_timer_sync(&priv->tx_queue[chan].txtimer); + stmmac_hw_teardown(dev); init_error: free_dma_desc_resources(priv); @@ -2708,6 +2717,7 @@ dma_desc_error: static int stmmac_release(struct net_device *dev) { struct stmmac_priv *priv = netdev_priv(dev); + u32 chan; if (priv->eee_enabled) del_timer_sync(&priv->eee_ctrl_timer); @@ -2722,7 +2732,8 @@ static int stmmac_release(struct net_device *dev) stmmac_disable_all_queues(priv); - del_timer_sync(&priv->txtimer); + for (chan = 0; chan < priv->plat->tx_queues_to_use; chan++) + del_timer_sync(&priv->tx_queue[chan].txtimer); /* Free the IRQ lines */ free_irq(dev->irq, dev); @@ -2936,14 +2947,13 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev) priv->xstats.tx_tso_nfrags += nfrags; /* Manage tx mitigation */ - priv->tx_count_frames += nfrags + 1; - if (likely(priv->tx_coal_frames > priv->tx_count_frames)) { - mod_timer(&priv->txtimer, - STMMAC_COAL_TIMER(priv->tx_coal_timer)); - } else { - priv->tx_count_frames = 0; + tx_q->tx_count_frames += nfrags + 1; + if (priv->tx_coal_frames <= tx_q->tx_count_frames) { stmmac_set_tx_ic(priv, desc); priv->xstats.tx_set_ic_bit++; + tx_q->tx_count_frames = 0; + } else { + stmmac_tx_timer_arm(priv, queue); } skb_tx_timestamp(skb); @@ -2992,6 +3002,7 @@ static netdev_tx_t stmmac_tso_xmit(struct sk_buff *skb, struct net_device *dev) netdev_tx_sent_queue(netdev_get_tx_queue(dev, queue), skb->len); + tx_q->tx_tail_addr = tx_q->dma_tx_phy + (tx_q->cur_tx * sizeof(*desc)); stmmac_set_tx_tail_ptr(priv, priv->ioaddr, tx_q->tx_tail_addr, queue); return NETDEV_TX_OK; @@ -3146,17 +3157,13 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) * This approach takes care about the fragments: desc is the first * element in case of no SG. */ - priv->tx_count_frames += nfrags + 1; - if (likely(priv->tx_coal_frames > priv->tx_count_frames) && - !priv->tx_timer_armed) { - mod_timer(&priv->txtimer, - STMMAC_COAL_TIMER(priv->tx_coal_timer)); - priv->tx_timer_armed = true; - } else { - priv->tx_count_frames = 0; + tx_q->tx_count_frames += nfrags + 1; + if (priv->tx_coal_frames <= tx_q->tx_count_frames) { stmmac_set_tx_ic(priv, desc); priv->xstats.tx_set_ic_bit++; - priv->tx_timer_armed = false; + tx_q->tx_count_frames = 0; + } else { + stmmac_tx_timer_arm(priv, queue); } skb_tx_timestamp(skb); @@ -3202,6 +3209,8 @@ static netdev_tx_t stmmac_xmit(struct sk_buff *skb, struct net_device *dev) netdev_tx_sent_queue(netdev_get_tx_queue(dev, queue), skb->len); stmmac_enable_dma_transmission(priv, priv->ioaddr); + + tx_q->tx_tail_addr = tx_q->dma_tx_phy + (tx_q->cur_tx * sizeof(*desc)); stmmac_set_tx_tail_ptr(priv, priv->ioaddr, tx_q->tx_tail_addr, queue); return NETDEV_TX_OK; @@ -3322,6 +3331,7 @@ static inline void stmmac_rx_refill(struct stmmac_priv *priv, u32 queue) static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue) { struct stmmac_rx_queue *rx_q = &priv->rx_queue[queue]; + struct stmmac_channel *ch = &priv->channel[queue]; unsigned int entry = rx_q->cur_rx; int coe = priv->hw->rx_csum; unsigned int next_entry; @@ -3494,7 +3504,7 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue) else skb->ip_summed = CHECKSUM_UNNECESSARY; - napi_gro_receive(&rx_q->napi, skb); + napi_gro_receive(&ch->napi, skb); priv->dev->stats.rx_packets++; priv->dev->stats.rx_bytes += frame_len; @@ -3517,27 +3527,33 @@ static int stmmac_rx(struct stmmac_priv *priv, int limit, u32 queue) * Description : * To look at the incoming frames and clear the tx resources. */ -static int stmmac_poll(struct napi_struct *napi, int budget) +static int stmmac_napi_poll(struct napi_struct *napi, int budget) { - struct stmmac_rx_queue *rx_q = - container_of(napi, struct stmmac_rx_queue, napi); - struct stmmac_priv *priv = rx_q->priv_data; - u32 tx_count = priv->plat->tx_queues_to_use; - u32 chan = rx_q->queue_index; - int work_done = 0; - u32 queue; + struct stmmac_channel *ch = + container_of(napi, struct stmmac_channel, napi); + struct stmmac_priv *priv = ch->priv_data; + int work_done = 0, work_rem = budget; + u32 chan = ch->index; priv->xstats.napi_poll++; - /* check all the queues */ - for (queue = 0; queue < tx_count; queue++) - stmmac_tx_clean(priv, queue); + if (ch->has_tx) { + int done = stmmac_tx_clean(priv, work_rem, chan); - work_done = stmmac_rx(priv, budget, rx_q->queue_index); - if (work_done < budget) { - napi_complete_done(napi, work_done); - stmmac_enable_dma_irq(priv, priv->ioaddr, chan); + work_done += done; + work_rem -= done; + } + + if (ch->has_rx) { + int done = stmmac_rx(priv, work_rem, chan); + + work_done += done; + work_rem -= done; } + + if (work_done < budget && napi_complete_done(napi, work_done)) + stmmac_enable_dma_irq(priv, priv->ioaddr, chan); + return work_done; } @@ -4201,8 +4217,8 @@ int stmmac_dvr_probe(struct device *device, { struct net_device *ndev = NULL; struct stmmac_priv *priv; + u32 queue, maxq; int ret = 0; - u32 queue; ndev = alloc_etherdev_mqs(sizeof(struct stmmac_priv), MTL_MAX_TX_QUEUES, @@ -4325,11 +4341,22 @@ int stmmac_dvr_probe(struct device *device, "Enable RX Mitigation via HW Watchdog Timer\n"); } - for (queue = 0; queue < priv->plat->rx_queues_to_use; queue++) { - struct stmmac_rx_queue *rx_q = &priv->rx_queue[queue]; + /* Setup channels NAPI */ + maxq = max(priv->plat->rx_queues_to_use, priv->plat->tx_queues_to_use); - netif_napi_add(ndev, &rx_q->napi, stmmac_poll, - (8 * priv->plat->rx_queues_to_use)); + for (queue = 0; queue < maxq; queue++) { + struct stmmac_channel *ch = &priv->channel[queue]; + + ch->priv_data = priv; + ch->index = queue; + + if (queue < priv->plat->rx_queues_to_use) + ch->has_rx = true; + if (queue < priv->plat->tx_queues_to_use) + ch->has_tx = true; + + netif_napi_add(ndev, &ch->napi, stmmac_napi_poll, + NAPI_POLL_WEIGHT); } mutex_init(&priv->lock); @@ -4375,10 +4402,10 @@ error_netdev_register: priv->hw->pcs != STMMAC_PCS_RTBI) stmmac_mdio_unregister(ndev); error_mdio_register: - for (queue = 0; queue < priv->plat->rx_queues_to_use; queue++) { - struct stmmac_rx_queue *rx_q = &priv->rx_queue[queue]; + for (queue = 0; queue < maxq; queue++) { + struct stmmac_channel *ch = &priv->channel[queue]; - netif_napi_del(&rx_q->napi); + netif_napi_del(&ch->napi); } error_hw_init: destroy_workqueue(priv->wq); diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c index 3609c7b696c7..2b800ce1d5bf 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c @@ -67,7 +67,7 @@ static int dwmac1000_validate_mcast_bins(int mcast_bins) * Description: * This function validates the number of Unicast address entries supported * by a particular Synopsys 10/100/1000 controller. The Synopsys controller - * supports 1, 32, 64, or 128 Unicast filter entries for it's Unicast filter + * supports 1..32, 64, or 128 Unicast filter entries for it's Unicast filter * logic. This function validates a valid, supported configuration is * selected, and defaults to 1 Unicast address if an unsupported * configuration is selected. @@ -77,8 +77,7 @@ static int dwmac1000_validate_ucast_entries(int ucast_entries) int x = ucast_entries; switch (x) { - case 1: - case 32: + case 1 ... 32: case 64: case 128: break; diff --git a/drivers/net/ethernet/ti/Kconfig b/drivers/net/ethernet/ti/Kconfig index 9263d638bd6d..f932923f7d56 100644 --- a/drivers/net/ethernet/ti/Kconfig +++ b/drivers/net/ethernet/ti/Kconfig @@ -41,6 +41,7 @@ config TI_DAVINCI_MDIO config TI_DAVINCI_CPDMA tristate "TI DaVinci CPDMA Support" depends on ARCH_DAVINCI || ARCH_OMAP2PLUS || COMPILE_TEST + select GENERIC_ALLOCATOR ---help--- This driver supports TI's DaVinci CPDMA dma engine. diff --git a/drivers/net/ethernet/ti/cpsw-phy-sel.c b/drivers/net/ethernet/ti/cpsw-phy-sel.c index 0c1adad7415d..396e1cd10667 100644 --- a/drivers/net/ethernet/ti/cpsw-phy-sel.c +++ b/drivers/net/ethernet/ti/cpsw-phy-sel.c @@ -170,10 +170,13 @@ void cpsw_phy_sel(struct device *dev, phy_interface_t phy_mode, int slave) struct device_node *node; struct cpsw_phy_sel_priv *priv; - node = of_get_child_by_name(dev->of_node, "cpsw-phy-sel"); + node = of_parse_phandle(dev->of_node, "cpsw-phy-sel", 0); if (!node) { - dev_err(dev, "Phy mode driver DT not found\n"); - return; + node = of_get_child_by_name(dev->of_node, "cpsw-phy-sel"); + if (!node) { + dev_err(dev, "Phy mode driver DT not found\n"); + return; + } } dev = bus_find_device(&platform_bus_type, NULL, node, match); diff --git a/drivers/net/ethernet/wiznet/w5100.c b/drivers/net/ethernet/wiznet/w5100.c index 2bdfb39215e9..d8ba512f166a 100644 --- a/drivers/net/ethernet/wiznet/w5100.c +++ b/drivers/net/ethernet/wiznet/w5100.c @@ -835,7 +835,7 @@ static void w5100_tx_work(struct work_struct *work) w5100_tx_skb(priv->ndev, skb); } -static int w5100_start_tx(struct sk_buff *skb, struct net_device *ndev) +static netdev_tx_t w5100_start_tx(struct sk_buff *skb, struct net_device *ndev) { struct w5100_priv *priv = netdev_priv(ndev); diff --git a/drivers/net/ethernet/wiznet/w5300.c b/drivers/net/ethernet/wiznet/w5300.c index 56ae573001e8..80fdbff67d82 100644 --- a/drivers/net/ethernet/wiznet/w5300.c +++ b/drivers/net/ethernet/wiznet/w5300.c @@ -365,7 +365,7 @@ static void w5300_tx_timeout(struct net_device *ndev) netif_wake_queue(ndev); } -static int w5300_start_tx(struct sk_buff *skb, struct net_device *ndev) +static netdev_tx_t w5300_start_tx(struct sk_buff *skb, struct net_device *ndev) { struct w5300_priv *priv = netdev_priv(ndev); diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c index 31c3d77b4733..fe01e141c8f8 100644 --- a/drivers/net/hyperv/netvsc.c +++ b/drivers/net/hyperv/netvsc.c @@ -1203,6 +1203,9 @@ static void netvsc_send_vf(struct net_device *ndev, net_device_ctx->vf_alloc = nvmsg->msg.v4_msg.vf_assoc.allocated; net_device_ctx->vf_serial = nvmsg->msg.v4_msg.vf_assoc.serial; + netdev_info(ndev, "VF slot %u %s\n", + net_device_ctx->vf_serial, + net_device_ctx->vf_alloc ? "added" : "removed"); } static void netvsc_receive_inband(struct net_device *ndev, diff --git a/drivers/net/hyperv/netvsc_drv.c b/drivers/net/hyperv/netvsc_drv.c index 1121a1ec407c..3af6d8d15233 100644 --- a/drivers/net/hyperv/netvsc_drv.c +++ b/drivers/net/hyperv/netvsc_drv.c @@ -1894,20 +1894,6 @@ out_unlock: rtnl_unlock(); } -static struct net_device *get_netvsc_bymac(const u8 *mac) -{ - struct net_device_context *ndev_ctx; - - list_for_each_entry(ndev_ctx, &netvsc_dev_list, list) { - struct net_device *dev = hv_get_drvdata(ndev_ctx->device_ctx); - - if (ether_addr_equal(mac, dev->perm_addr)) - return dev; - } - - return NULL; -} - static struct net_device *get_netvsc_byref(struct net_device *vf_netdev) { struct net_device_context *net_device_ctx; @@ -2036,26 +2022,48 @@ static void netvsc_vf_setup(struct work_struct *w) rtnl_unlock(); } +/* Find netvsc by VMBus serial number. + * The PCI hyperv controller records the serial number as the slot. + */ +static struct net_device *get_netvsc_byslot(const struct net_device *vf_netdev) +{ + struct device *parent = vf_netdev->dev.parent; + struct net_device_context *ndev_ctx; + struct pci_dev *pdev; + + if (!parent || !dev_is_pci(parent)) + return NULL; /* not a PCI device */ + + pdev = to_pci_dev(parent); + if (!pdev->slot) { + netdev_notice(vf_netdev, "no PCI slot information\n"); + return NULL; + } + + list_for_each_entry(ndev_ctx, &netvsc_dev_list, list) { + if (!ndev_ctx->vf_alloc) + continue; + + if (ndev_ctx->vf_serial == pdev->slot->number) + return hv_get_drvdata(ndev_ctx->device_ctx); + } + + netdev_notice(vf_netdev, + "no netdev found for slot %u\n", pdev->slot->number); + return NULL; +} + static int netvsc_register_vf(struct net_device *vf_netdev) { - struct net_device *ndev; struct net_device_context *net_device_ctx; - struct device *pdev = vf_netdev->dev.parent; struct netvsc_device *netvsc_dev; + struct net_device *ndev; int ret; if (vf_netdev->addr_len != ETH_ALEN) return NOTIFY_DONE; - if (!pdev || !dev_is_pci(pdev) || dev_is_pf(pdev)) - return NOTIFY_DONE; - - /* - * We will use the MAC address to locate the synthetic interface to - * associate with the VF interface. If we don't find a matching - * synthetic interface, move on. - */ - ndev = get_netvsc_bymac(vf_netdev->perm_addr); + ndev = get_netvsc_byslot(vf_netdev); if (!ndev) return NOTIFY_DONE; @@ -2206,6 +2214,16 @@ static int netvsc_probe(struct hv_device *dev, memcpy(net->dev_addr, device_info.mac_adr, ETH_ALEN); + /* We must get rtnl lock before scheduling nvdev->subchan_work, + * otherwise netvsc_subchan_work() can get rtnl lock first and wait + * all subchannels to show up, but that may not happen because + * netvsc_probe() can't get rtnl lock and as a result vmbus_onoffer() + * -> ... -> device_add() -> ... -> __device_attach() can't get + * the device lock, so all the subchannels can't be processed -- + * finally netvsc_subchan_work() hangs for ever. + */ + rtnl_lock(); + if (nvdev->num_chn > 1) schedule_work(&nvdev->subchan_work); @@ -2224,7 +2242,6 @@ static int netvsc_probe(struct hv_device *dev, else net->max_mtu = ETH_DATA_LEN; - rtnl_lock(); ret = register_netdevice(net); if (ret != 0) { pr_err("Unable to register netdev.\n"); @@ -2263,17 +2280,15 @@ static int netvsc_remove(struct hv_device *dev) cancel_delayed_work_sync(&ndev_ctx->dwork); - rcu_read_lock(); - nvdev = rcu_dereference(ndev_ctx->nvdev); - - if (nvdev) + rtnl_lock(); + nvdev = rtnl_dereference(ndev_ctx->nvdev); + if (nvdev) cancel_work_sync(&nvdev->subchan_work); /* * Call to the vsc driver to let it know that the device is being * removed. Also blocks mtu and channel changes. */ - rtnl_lock(); vf_netdev = rtnl_dereference(ndev_ctx->vf_netdev); if (vf_netdev) netvsc_unregister_vf(vf_netdev); @@ -2285,7 +2300,6 @@ static int netvsc_remove(struct hv_device *dev) list_del(&ndev_ctx->list); rtnl_unlock(); - rcu_read_unlock(); hv_set_drvdata(dev, NULL); diff --git a/drivers/net/phy/sfp-bus.c b/drivers/net/phy/sfp-bus.c index 740655261e5b..83060fb349f4 100644 --- a/drivers/net/phy/sfp-bus.c +++ b/drivers/net/phy/sfp-bus.c @@ -349,6 +349,7 @@ static int sfp_register_bus(struct sfp_bus *bus) } if (bus->started) bus->socket_ops->start(bus->sfp); + bus->netdev->sfp_bus = bus; bus->registered = true; return 0; } @@ -357,6 +358,7 @@ static void sfp_unregister_bus(struct sfp_bus *bus) { const struct sfp_upstream_ops *ops = bus->upstream_ops; + bus->netdev->sfp_bus = NULL; if (bus->registered) { if (bus->started) bus->socket_ops->stop(bus->sfp); @@ -438,7 +440,6 @@ static void sfp_upstream_clear(struct sfp_bus *bus) { bus->upstream_ops = NULL; bus->upstream = NULL; - bus->netdev->sfp_bus = NULL; bus->netdev = NULL; } @@ -467,7 +468,6 @@ struct sfp_bus *sfp_register_upstream(struct fwnode_handle *fwnode, bus->upstream_ops = ops; bus->upstream = upstream; bus->netdev = ndev; - ndev->sfp_bus = bus; if (bus->sfp) { ret = sfp_register_bus(bus); diff --git a/drivers/net/phy/sfp.c b/drivers/net/phy/sfp.c index 4637d980310e..52fffb98fde9 100644 --- a/drivers/net/phy/sfp.c +++ b/drivers/net/phy/sfp.c @@ -398,7 +398,6 @@ static umode_t sfp_hwmon_is_visible(const void *data, switch (type) { case hwmon_temp: switch (attr) { - case hwmon_temp_input: case hwmon_temp_min_alarm: case hwmon_temp_max_alarm: case hwmon_temp_lcrit_alarm: @@ -407,13 +406,16 @@ static umode_t sfp_hwmon_is_visible(const void *data, case hwmon_temp_max: case hwmon_temp_lcrit: case hwmon_temp_crit: + if (!(sfp->id.ext.enhopts & SFP_ENHOPTS_ALARMWARN)) + return 0; + /* fall through */ + case hwmon_temp_input: return 0444; default: return 0; } case hwmon_in: switch (attr) { - case hwmon_in_input: case hwmon_in_min_alarm: case hwmon_in_max_alarm: case hwmon_in_lcrit_alarm: @@ -422,13 +424,16 @@ static umode_t sfp_hwmon_is_visible(const void *data, case hwmon_in_max: case hwmon_in_lcrit: case hwmon_in_crit: + if (!(sfp->id.ext.enhopts & SFP_ENHOPTS_ALARMWARN)) + return 0; + /* fall through */ + case hwmon_in_input: return 0444; default: return 0; } case hwmon_curr: switch (attr) { - case hwmon_curr_input: case hwmon_curr_min_alarm: case hwmon_curr_max_alarm: case hwmon_curr_lcrit_alarm: @@ -437,6 +442,10 @@ static umode_t sfp_hwmon_is_visible(const void *data, case hwmon_curr_max: case hwmon_curr_lcrit: case hwmon_curr_crit: + if (!(sfp->id.ext.enhopts & SFP_ENHOPTS_ALARMWARN)) + return 0; + /* fall through */ + case hwmon_curr_input: return 0444; default: return 0; @@ -452,7 +461,6 @@ static umode_t sfp_hwmon_is_visible(const void *data, channel == 1) return 0; switch (attr) { - case hwmon_power_input: case hwmon_power_min_alarm: case hwmon_power_max_alarm: case hwmon_power_lcrit_alarm: @@ -461,6 +469,10 @@ static umode_t sfp_hwmon_is_visible(const void *data, case hwmon_power_max: case hwmon_power_lcrit: case hwmon_power_crit: + if (!(sfp->id.ext.enhopts & SFP_ENHOPTS_ALARMWARN)) + return 0; + /* fall through */ + case hwmon_power_input: return 0444; default: return 0; diff --git a/drivers/net/ppp/pppoe.c b/drivers/net/ppp/pppoe.c index ce61231e96ea..62dc564b251d 100644 --- a/drivers/net/ppp/pppoe.c +++ b/drivers/net/ppp/pppoe.c @@ -429,6 +429,9 @@ static int pppoe_rcv(struct sk_buff *skb, struct net_device *dev, if (!skb) goto out; + if (skb_mac_header_len(skb) < ETH_HLEN) + goto drop; + if (!pskb_may_pull(skb, sizeof(struct pppoe_hdr))) goto drop; diff --git a/drivers/net/tun.c b/drivers/net/tun.c index ebd07ad82431..e2648b5a3861 100644 --- a/drivers/net/tun.c +++ b/drivers/net/tun.c @@ -1153,43 +1153,6 @@ static netdev_features_t tun_net_fix_features(struct net_device *dev, return (features & tun->set_features) | (features & ~TUN_USER_FEATURES); } -#ifdef CONFIG_NET_POLL_CONTROLLER -static void tun_poll_controller(struct net_device *dev) -{ - /* - * Tun only receives frames when: - * 1) the char device endpoint gets data from user space - * 2) the tun socket gets a sendmsg call from user space - * If NAPI is not enabled, since both of those are synchronous - * operations, we are guaranteed never to have pending data when we poll - * for it so there is nothing to do here but return. - * We need this though so netpoll recognizes us as an interface that - * supports polling, which enables bridge devices in virt setups to - * still use netconsole - * If NAPI is enabled, however, we need to schedule polling for all - * queues unless we are using napi_gro_frags(), which we call in - * process context and not in NAPI context. - */ - struct tun_struct *tun = netdev_priv(dev); - - if (tun->flags & IFF_NAPI) { - struct tun_file *tfile; - int i; - - if (tun_napi_frags_enabled(tun)) - return; - - rcu_read_lock(); - for (i = 0; i < tun->numqueues; i++) { - tfile = rcu_dereference(tun->tfiles[i]); - if (tfile->napi_enabled) - napi_schedule(&tfile->napi); - } - rcu_read_unlock(); - } - return; -} -#endif static void tun_set_headroom(struct net_device *dev, int new_hr) { @@ -1283,9 +1246,6 @@ static const struct net_device_ops tun_netdev_ops = { .ndo_start_xmit = tun_net_xmit, .ndo_fix_features = tun_net_fix_features, .ndo_select_queue = tun_select_queue, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = tun_poll_controller, -#endif .ndo_set_rx_headroom = tun_set_headroom, .ndo_get_stats64 = tun_net_get_stats64, }; @@ -1365,9 +1325,6 @@ static const struct net_device_ops tap_netdev_ops = { .ndo_set_mac_address = eth_mac_addr, .ndo_validate_addr = eth_validate_addr, .ndo_select_queue = tun_select_queue, -#ifdef CONFIG_NET_POLL_CONTROLLER - .ndo_poll_controller = tun_poll_controller, -#endif .ndo_features_check = passthru_features_check, .ndo_set_rx_headroom = tun_set_headroom, .ndo_get_stats64 = tun_net_get_stats64, diff --git a/drivers/net/usb/qmi_wwan.c b/drivers/net/usb/qmi_wwan.c index cb0cc30c3d6a..533b6fb8d923 100644 --- a/drivers/net/usb/qmi_wwan.c +++ b/drivers/net/usb/qmi_wwan.c @@ -967,6 +967,13 @@ static const struct usb_device_id products[] = { USB_DEVICE_AND_INTERFACE_INFO(0x03f0, 0x581d, USB_CLASS_VENDOR_SPEC, 1, 7), .driver_info = (unsigned long)&qmi_wwan_info, }, + { /* Quectel EP06/EG06/EM06 */ + USB_DEVICE_AND_INTERFACE_INFO(0x2c7c, 0x0306, + USB_CLASS_VENDOR_SPEC, + USB_SUBCLASS_VENDOR_SPEC, + 0xff), + .driver_info = (unsigned long)&qmi_wwan_info_quirk_dtr, + }, /* 3. Combined interface devices matching on interface number */ {QMI_FIXED_INTF(0x0408, 0xea42, 4)}, /* Yota / Megafon M100-1 */ @@ -1206,13 +1213,13 @@ static const struct usb_device_id products[] = { {QMI_FIXED_INTF(0x1199, 0x9061, 8)}, /* Sierra Wireless Modem */ {QMI_FIXED_INTF(0x1199, 0x9063, 8)}, /* Sierra Wireless EM7305 */ {QMI_FIXED_INTF(0x1199, 0x9063, 10)}, /* Sierra Wireless EM7305 */ - {QMI_FIXED_INTF(0x1199, 0x9071, 8)}, /* Sierra Wireless MC74xx */ - {QMI_FIXED_INTF(0x1199, 0x9071, 10)}, /* Sierra Wireless MC74xx */ - {QMI_FIXED_INTF(0x1199, 0x9079, 8)}, /* Sierra Wireless EM74xx */ - {QMI_FIXED_INTF(0x1199, 0x9079, 10)}, /* Sierra Wireless EM74xx */ - {QMI_FIXED_INTF(0x1199, 0x907b, 8)}, /* Sierra Wireless EM74xx */ - {QMI_FIXED_INTF(0x1199, 0x907b, 10)}, /* Sierra Wireless EM74xx */ - {QMI_FIXED_INTF(0x1199, 0x9091, 8)}, /* Sierra Wireless EM7565 */ + {QMI_QUIRK_SET_DTR(0x1199, 0x9071, 8)}, /* Sierra Wireless MC74xx */ + {QMI_QUIRK_SET_DTR(0x1199, 0x9071, 10)},/* Sierra Wireless MC74xx */ + {QMI_QUIRK_SET_DTR(0x1199, 0x9079, 8)}, /* Sierra Wireless EM74xx */ + {QMI_QUIRK_SET_DTR(0x1199, 0x9079, 10)},/* Sierra Wireless EM74xx */ + {QMI_QUIRK_SET_DTR(0x1199, 0x907b, 8)}, /* Sierra Wireless EM74xx */ + {QMI_QUIRK_SET_DTR(0x1199, 0x907b, 10)},/* Sierra Wireless EM74xx */ + {QMI_QUIRK_SET_DTR(0x1199, 0x9091, 8)}, /* Sierra Wireless EM7565 */ {QMI_FIXED_INTF(0x1bbb, 0x011e, 4)}, /* Telekom Speedstick LTE II (Alcatel One Touch L100V LTE) */ {QMI_FIXED_INTF(0x1bbb, 0x0203, 2)}, /* Alcatel L800MA */ {QMI_FIXED_INTF(0x2357, 0x0201, 4)}, /* TP-LINK HSUPA Modem MA180 */ @@ -1255,7 +1262,6 @@ static const struct usb_device_id products[] = { {QMI_QUIRK_SET_DTR(0x2c7c, 0x0121, 4)}, /* Quectel EC21 Mini PCIe */ {QMI_QUIRK_SET_DTR(0x2c7c, 0x0191, 4)}, /* Quectel EG91 */ {QMI_FIXED_INTF(0x2c7c, 0x0296, 4)}, /* Quectel BG96 */ - {QMI_QUIRK_SET_DTR(0x2c7c, 0x0306, 4)}, /* Quectel EP06 Mini PCIe */ /* 4. Gobi 1000 devices */ {QMI_GOBI1K_DEVICE(0x05c6, 0x9212)}, /* Acer Gobi Modem Device */ @@ -1331,6 +1337,19 @@ static bool quectel_ec20_detected(struct usb_interface *intf) return false; } +static bool quectel_ep06_diag_detected(struct usb_interface *intf) +{ + struct usb_device *dev = interface_to_usbdev(intf); + struct usb_interface_descriptor intf_desc = intf->cur_altsetting->desc; + + if (le16_to_cpu(dev->descriptor.idVendor) == 0x2c7c && + le16_to_cpu(dev->descriptor.idProduct) == 0x0306 && + intf_desc.bNumEndpoints == 2) + return true; + + return false; +} + static int qmi_wwan_probe(struct usb_interface *intf, const struct usb_device_id *prod) { @@ -1365,6 +1384,15 @@ static int qmi_wwan_probe(struct usb_interface *intf, return -ENODEV; } + /* Quectel EP06/EM06/EG06 supports dynamic interface configuration, so + * we need to match on class/subclass/protocol. These values are + * identical for the diagnostic- and QMI-interface, but bNumEndpoints is + * different. Ignore the current interface if the number of endpoints + * the number for the diag interface (two). + */ + if (quectel_ep06_diag_detected(intf)) + return -ENODEV; + return usbnet_probe(intf, id); } diff --git a/drivers/net/veth.c b/drivers/net/veth.c index 8d679c8b7f25..41a00cd76955 100644 --- a/drivers/net/veth.c +++ b/drivers/net/veth.c @@ -463,6 +463,8 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq, struct sk_buff *skb, int mac_len, delta, off; struct xdp_buff xdp; + skb_orphan(skb); + rcu_read_lock(); xdp_prog = rcu_dereference(rq->xdp_prog); if (unlikely(!xdp_prog)) { @@ -508,8 +510,6 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq, struct sk_buff *skb, skb_copy_header(nskb, skb); head_off = skb_headroom(nskb) - skb_headroom(skb); skb_headers_offset_update(nskb, head_off); - if (skb->sk) - skb_set_owner_w(nskb, skb->sk); consume_skb(skb); skb = nskb; } diff --git a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c index b4c3a957c102..73969dbeb5c5 100644 --- a/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c +++ b/drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c @@ -985,15 +985,12 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg, const u8 *nvm_chan = cfg->nvm_type == IWL_NVM_EXT ? iwl_ext_nvm_channels : iwl_nvm_channels; struct ieee80211_regdomain *regd, *copy_rd; - int size_of_regd, regd_to_copy, wmms_to_copy; - int size_of_wmms = 0; + int size_of_regd, regd_to_copy; struct ieee80211_reg_rule *rule; - struct ieee80211_wmm_rule *wmm_rule, *d_wmm, *s_wmm; struct regdb_ptrs *regdb_ptrs; enum nl80211_band band; int center_freq, prev_center_freq = 0; - int valid_rules = 0, n_wmms = 0; - int i; + int valid_rules = 0; bool new_rule; int max_num_ch = cfg->nvm_type == IWL_NVM_EXT ? IWL_NVM_NUM_CHANNELS_EXT : IWL_NVM_NUM_CHANNELS; @@ -1012,11 +1009,7 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg, sizeof(struct ieee80211_regdomain) + num_of_ch * sizeof(struct ieee80211_reg_rule); - if (geo_info & GEO_WMM_ETSI_5GHZ_INFO) - size_of_wmms = - num_of_ch * sizeof(struct ieee80211_wmm_rule); - - regd = kzalloc(size_of_regd + size_of_wmms, GFP_KERNEL); + regd = kzalloc(size_of_regd, GFP_KERNEL); if (!regd) return ERR_PTR(-ENOMEM); @@ -1030,8 +1023,6 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg, regd->alpha2[0] = fw_mcc >> 8; regd->alpha2[1] = fw_mcc & 0xff; - wmm_rule = (struct ieee80211_wmm_rule *)((u8 *)regd + size_of_regd); - for (ch_idx = 0; ch_idx < num_of_ch; ch_idx++) { ch_flags = (u16)__le32_to_cpup(channels + ch_idx); band = (ch_idx < NUM_2GHZ_CHANNELS) ? @@ -1085,26 +1076,10 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg, band == NL80211_BAND_2GHZ) continue; - if (!reg_query_regdb_wmm(regd->alpha2, center_freq, - ®db_ptrs[n_wmms].token, wmm_rule)) { - /* Add only new rules */ - for (i = 0; i < n_wmms; i++) { - if (regdb_ptrs[i].token == - regdb_ptrs[n_wmms].token) { - rule->wmm_rule = regdb_ptrs[i].rule; - break; - } - } - if (i == n_wmms) { - rule->wmm_rule = wmm_rule; - regdb_ptrs[n_wmms++].rule = wmm_rule; - wmm_rule++; - } - } + reg_query_regdb_wmm(regd->alpha2, center_freq, rule); } regd->n_reg_rules = valid_rules; - regd->n_wmm_rules = n_wmms; /* * Narrow down regdom for unused regulatory rules to prevent hole @@ -1113,28 +1088,13 @@ iwl_parse_nvm_mcc_info(struct device *dev, const struct iwl_cfg *cfg, regd_to_copy = sizeof(struct ieee80211_regdomain) + valid_rules * sizeof(struct ieee80211_reg_rule); - wmms_to_copy = sizeof(struct ieee80211_wmm_rule) * n_wmms; - - copy_rd = kzalloc(regd_to_copy + wmms_to_copy, GFP_KERNEL); + copy_rd = kzalloc(regd_to_copy, GFP_KERNEL); if (!copy_rd) { copy_rd = ERR_PTR(-ENOMEM); goto out; } memcpy(copy_rd, regd, regd_to_copy); - memcpy((u8 *)copy_rd + regd_to_copy, (u8 *)regd + size_of_regd, - wmms_to_copy); - - d_wmm = (struct ieee80211_wmm_rule *)((u8 *)copy_rd + regd_to_copy); - s_wmm = (struct ieee80211_wmm_rule *)((u8 *)regd + size_of_regd); - - for (i = 0; i < regd->n_reg_rules; i++) { - if (!regd->reg_rules[i].wmm_rule) - continue; - - copy_rd->reg_rules[i].wmm_rule = d_wmm + - (regd->reg_rules[i].wmm_rule - s_wmm); - } out: kfree(regdb_ptrs); diff --git a/drivers/net/wireless/mac80211_hwsim.c b/drivers/net/wireless/mac80211_hwsim.c index 998dfac0fcff..1068757ec42e 100644 --- a/drivers/net/wireless/mac80211_hwsim.c +++ b/drivers/net/wireless/mac80211_hwsim.c @@ -34,6 +34,7 @@ #include <net/net_namespace.h> #include <net/netns/generic.h> #include <linux/rhashtable.h> +#include <linux/nospec.h> #include "mac80211_hwsim.h" #define WARN_QUEUE 100 @@ -2820,9 +2821,6 @@ static int mac80211_hwsim_new_radio(struct genl_info *info, IEEE80211_VHT_CAP_SHORT_GI_80 | IEEE80211_VHT_CAP_SHORT_GI_160 | IEEE80211_VHT_CAP_TXSTBC | - IEEE80211_VHT_CAP_RXSTBC_1 | - IEEE80211_VHT_CAP_RXSTBC_2 | - IEEE80211_VHT_CAP_RXSTBC_3 | IEEE80211_VHT_CAP_RXSTBC_4 | IEEE80211_VHT_CAP_MAX_A_MPDU_LENGTH_EXPONENT_MASK; sband->vht_cap.vht_mcs.rx_mcs_map = @@ -3317,6 +3315,11 @@ static int hwsim_new_radio_nl(struct sk_buff *msg, struct genl_info *info) if (info->attrs[HWSIM_ATTR_CHANNELS]) param.channels = nla_get_u32(info->attrs[HWSIM_ATTR_CHANNELS]); + if (param.channels < 1) { + GENL_SET_ERR_MSG(info, "must have at least one channel"); + return -EINVAL; + } + if (param.channels > CFG80211_MAX_NUM_DIFFERENT_CHANNELS) { GENL_SET_ERR_MSG(info, "too many channels specified"); return -EINVAL; @@ -3350,6 +3353,9 @@ static int hwsim_new_radio_nl(struct sk_buff *msg, struct genl_info *info) kfree(hwname); return -EINVAL; } + + idx = array_index_nospec(idx, + ARRAY_SIZE(hwsim_world_regdom_custom)); param.regd = hwsim_world_regdom_custom[idx]; } diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c index 73f596a90c69..f17f602e6171 100644 --- a/drivers/net/xen-netfront.c +++ b/drivers/net/xen-netfront.c @@ -87,8 +87,7 @@ struct netfront_cb { /* IRQ name is queue name with "-tx" or "-rx" appended */ #define IRQ_NAME_SIZE (QUEUE_NAME_SIZE + 3) -static DECLARE_WAIT_QUEUE_HEAD(module_load_q); -static DECLARE_WAIT_QUEUE_HEAD(module_unload_q); +static DECLARE_WAIT_QUEUE_HEAD(module_wq); struct netfront_stats { u64 packets; @@ -909,7 +908,11 @@ static RING_IDX xennet_fill_frags(struct netfront_queue *queue, BUG_ON(pull_to <= skb_headlen(skb)); __pskb_pull_tail(skb, pull_to - skb_headlen(skb)); } - BUG_ON(skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS); + if (unlikely(skb_shinfo(skb)->nr_frags >= MAX_SKB_FRAGS)) { + queue->rx.rsp_cons = ++cons; + kfree_skb(nskb); + return ~0U; + } skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags, skb_frag_page(nfrag), @@ -1046,6 +1049,8 @@ err: skb->len += rx->status; i = xennet_fill_frags(queue, skb, &tmpq); + if (unlikely(i == ~0U)) + goto err; if (rx->flags & XEN_NETRXF_csum_blank) skb->ip_summed = CHECKSUM_PARTIAL; @@ -1332,11 +1337,11 @@ static struct net_device *xennet_create_dev(struct xenbus_device *dev) netif_carrier_off(netdev); xenbus_switch_state(dev, XenbusStateInitialising); - wait_event(module_load_q, - xenbus_read_driver_state(dev->otherend) != - XenbusStateClosed && - xenbus_read_driver_state(dev->otherend) != - XenbusStateUnknown); + wait_event(module_wq, + xenbus_read_driver_state(dev->otherend) != + XenbusStateClosed && + xenbus_read_driver_state(dev->otherend) != + XenbusStateUnknown); return netdev; exit: @@ -2010,15 +2015,14 @@ static void netback_changed(struct xenbus_device *dev, dev_dbg(&dev->dev, "%s\n", xenbus_strstate(backend_state)); + wake_up_all(&module_wq); + switch (backend_state) { case XenbusStateInitialising: case XenbusStateInitialised: case XenbusStateReconfiguring: case XenbusStateReconfigured: - break; - case XenbusStateUnknown: - wake_up_all(&module_unload_q); break; case XenbusStateInitWait: @@ -2034,12 +2038,10 @@ static void netback_changed(struct xenbus_device *dev, break; case XenbusStateClosed: - wake_up_all(&module_unload_q); if (dev->state == XenbusStateClosed) break; /* Missed the backend's CLOSING state -- fallthrough */ case XenbusStateClosing: - wake_up_all(&module_unload_q); xenbus_frontend_closed(dev); break; } @@ -2147,14 +2149,14 @@ static int xennet_remove(struct xenbus_device *dev) if (xenbus_read_driver_state(dev->otherend) != XenbusStateClosed) { xenbus_switch_state(dev, XenbusStateClosing); - wait_event(module_unload_q, + wait_event(module_wq, xenbus_read_driver_state(dev->otherend) == XenbusStateClosing || xenbus_read_driver_state(dev->otherend) == XenbusStateUnknown); xenbus_switch_state(dev, XenbusStateClosed); - wait_event(module_unload_q, + wait_event(module_wq, xenbus_read_driver_state(dev->otherend) == XenbusStateClosed || xenbus_read_driver_state(dev->otherend) == diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c index 5a9562881d4e..9fe3fff818b8 100644 --- a/drivers/nvme/host/multipath.c +++ b/drivers/nvme/host/multipath.c @@ -537,8 +537,10 @@ int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id) INIT_WORK(&ctrl->ana_work, nvme_ana_work); ctrl->ana_log_buf = kmalloc(ctrl->ana_log_size, GFP_KERNEL); - if (!ctrl->ana_log_buf) + if (!ctrl->ana_log_buf) { + error = -ENOMEM; goto out; + } error = nvme_read_ana_log(ctrl, true); if (error) @@ -547,7 +549,7 @@ int nvme_mpath_init(struct nvme_ctrl *ctrl, struct nvme_id_ctrl *id) out_free_ana_log_buf: kfree(ctrl->ana_log_buf); out: - return -ENOMEM; + return error; } void nvme_mpath_uninit(struct nvme_ctrl *ctrl) diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c index a21caea1e080..2008fa62a373 100644 --- a/drivers/nvme/target/admin-cmd.c +++ b/drivers/nvme/target/admin-cmd.c @@ -245,6 +245,10 @@ static void nvmet_execute_get_log_page_ana(struct nvmet_req *req) offset += len; ngrps++; } + for ( ; grpid <= NVMET_MAX_ANAGRPS; grpid++) { + if (nvmet_ana_group_enabled[grpid]) + ngrps++; + } hdr.chgcnt = cpu_to_le64(nvmet_ana_chgcnt); hdr.ngrps = cpu_to_le16(ngrps); diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 3533e918ea37..bfc4da660bb4 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -66,6 +66,7 @@ struct nvmet_rdma_rsp { struct nvmet_req req; + bool allocated; u8 n_rdma; u32 flags; u32 invalidate_rkey; @@ -174,11 +175,19 @@ nvmet_rdma_get_rsp(struct nvmet_rdma_queue *queue) unsigned long flags; spin_lock_irqsave(&queue->rsps_lock, flags); - rsp = list_first_entry(&queue->free_rsps, + rsp = list_first_entry_or_null(&queue->free_rsps, struct nvmet_rdma_rsp, free_list); - list_del(&rsp->free_list); + if (likely(rsp)) + list_del(&rsp->free_list); spin_unlock_irqrestore(&queue->rsps_lock, flags); + if (unlikely(!rsp)) { + rsp = kmalloc(sizeof(*rsp), GFP_KERNEL); + if (unlikely(!rsp)) + return NULL; + rsp->allocated = true; + } + return rsp; } @@ -187,6 +196,11 @@ nvmet_rdma_put_rsp(struct nvmet_rdma_rsp *rsp) { unsigned long flags; + if (rsp->allocated) { + kfree(rsp); + return; + } + spin_lock_irqsave(&rsp->queue->rsps_lock, flags); list_add_tail(&rsp->free_list, &rsp->queue->free_rsps); spin_unlock_irqrestore(&rsp->queue->rsps_lock, flags); @@ -776,6 +790,15 @@ static void nvmet_rdma_recv_done(struct ib_cq *cq, struct ib_wc *wc) cmd->queue = queue; rsp = nvmet_rdma_get_rsp(queue); + if (unlikely(!rsp)) { + /* + * we get here only under memory pressure, + * silently drop and have the host retry + * as we can't even fail it. + */ + nvmet_rdma_post_recv(queue->dev, cmd); + return; + } rsp->queue = queue; rsp->cmd = cmd; rsp->flags = 0; diff --git a/drivers/of/base.c b/drivers/of/base.c index 9095b8290150..74eaedd5b860 100644 --- a/drivers/of/base.c +++ b/drivers/of/base.c @@ -140,6 +140,9 @@ void of_populate_phandle_cache(void) if (np->phandle && np->phandle != OF_PHANDLE_ILLEGAL) phandles++; + if (!phandles) + goto out; + cache_entries = roundup_pow_of_two(phandles); phandle_cache_mask = cache_entries - 1; diff --git a/drivers/of/platform.c b/drivers/of/platform.c index 7ba90c290a42..6c59673933e9 100644 --- a/drivers/of/platform.c +++ b/drivers/of/platform.c @@ -241,6 +241,10 @@ static struct amba_device *of_amba_device_create(struct device_node *node, if (!dev) goto err_clear_flag; + /* AMBA devices only support a single DMA mask */ + dev->dev.coherent_dma_mask = DMA_BIT_MASK(32); + dev->dev.dma_mask = &dev->dev.coherent_dma_mask; + /* setup generic device info */ dev->dev.of_node = of_node_get(node); dev->dev.fwnode = &node->fwnode; diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c index 778c4f76a884..2153956a0b20 100644 --- a/drivers/pci/controller/dwc/pcie-designware.c +++ b/drivers/pci/controller/dwc/pcie-designware.c @@ -135,7 +135,7 @@ static void dw_pcie_prog_outbound_atu_unroll(struct dw_pcie *pci, int index, if (val & PCIE_ATU_ENABLE) return; - usleep_range(LINK_WAIT_IATU_MIN, LINK_WAIT_IATU_MAX); + mdelay(LINK_WAIT_IATU); } dev_err(pci->dev, "Outbound iATU is not being enabled\n"); } @@ -178,7 +178,7 @@ void dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type, if (val & PCIE_ATU_ENABLE) return; - usleep_range(LINK_WAIT_IATU_MIN, LINK_WAIT_IATU_MAX); + mdelay(LINK_WAIT_IATU); } dev_err(pci->dev, "Outbound iATU is not being enabled\n"); } @@ -236,7 +236,7 @@ static int dw_pcie_prog_inbound_atu_unroll(struct dw_pcie *pci, int index, if (val & PCIE_ATU_ENABLE) return 0; - usleep_range(LINK_WAIT_IATU_MIN, LINK_WAIT_IATU_MAX); + mdelay(LINK_WAIT_IATU); } dev_err(pci->dev, "Inbound iATU is not being enabled\n"); @@ -282,7 +282,7 @@ int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int bar, if (val & PCIE_ATU_ENABLE) return 0; - usleep_range(LINK_WAIT_IATU_MIN, LINK_WAIT_IATU_MAX); + mdelay(LINK_WAIT_IATU); } dev_err(pci->dev, "Inbound iATU is not being enabled\n"); diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h index 96126fd8403c..9f1a5e399b70 100644 --- a/drivers/pci/controller/dwc/pcie-designware.h +++ b/drivers/pci/controller/dwc/pcie-designware.h @@ -26,8 +26,7 @@ /* Parameters for the waiting for iATU enabled routine */ #define LINK_WAIT_MAX_IATU_RETRIES 5 -#define LINK_WAIT_IATU_MIN 9000 -#define LINK_WAIT_IATU_MAX 10000 +#define LINK_WAIT_IATU 9 /* Synopsys-specific PCIe configuration registers */ #define PCIE_PORT_LINK_CONTROL 0x710 diff --git a/drivers/pci/controller/pci-hyperv.c b/drivers/pci/controller/pci-hyperv.c index c00f82cc54aa..9ba4d12c179c 100644 --- a/drivers/pci/controller/pci-hyperv.c +++ b/drivers/pci/controller/pci-hyperv.c @@ -89,6 +89,9 @@ static enum pci_protocol_version_t pci_protocol_version; #define STATUS_REVISION_MISMATCH 0xC0000059 +/* space for 32bit serial number as string */ +#define SLOT_NAME_SIZE 11 + /* * Message Types */ @@ -494,6 +497,7 @@ struct hv_pci_dev { struct list_head list_entry; refcount_t refs; enum hv_pcichild_state state; + struct pci_slot *pci_slot; struct pci_function_description desc; bool reported_missing; struct hv_pcibus_device *hbus; @@ -1457,6 +1461,36 @@ static void prepopulate_bars(struct hv_pcibus_device *hbus) spin_unlock_irqrestore(&hbus->device_list_lock, flags); } +/* + * Assign entries in sysfs pci slot directory. + * + * Note that this function does not need to lock the children list + * because it is called from pci_devices_present_work which + * is serialized with hv_eject_device_work because they are on the + * same ordered workqueue. Therefore hbus->children list will not change + * even when pci_create_slot sleeps. + */ +static void hv_pci_assign_slots(struct hv_pcibus_device *hbus) +{ + struct hv_pci_dev *hpdev; + char name[SLOT_NAME_SIZE]; + int slot_nr; + + list_for_each_entry(hpdev, &hbus->children, list_entry) { + if (hpdev->pci_slot) + continue; + + slot_nr = PCI_SLOT(wslot_to_devfn(hpdev->desc.win_slot.slot)); + snprintf(name, SLOT_NAME_SIZE, "%u", hpdev->desc.ser); + hpdev->pci_slot = pci_create_slot(hbus->pci_bus, slot_nr, + name, NULL); + if (IS_ERR(hpdev->pci_slot)) { + pr_warn("pci_create slot %s failed\n", name); + hpdev->pci_slot = NULL; + } + } +} + /** * create_root_hv_pci_bus() - Expose a new root PCI bus * @hbus: Root PCI bus, as understood by this driver @@ -1480,6 +1514,7 @@ static int create_root_hv_pci_bus(struct hv_pcibus_device *hbus) pci_lock_rescan_remove(); pci_scan_child_bus(hbus->pci_bus); pci_bus_assign_resources(hbus->pci_bus); + hv_pci_assign_slots(hbus); pci_bus_add_devices(hbus->pci_bus); pci_unlock_rescan_remove(); hbus->state = hv_pcibus_installed; @@ -1742,6 +1777,7 @@ static void pci_devices_present_work(struct work_struct *work) */ pci_lock_rescan_remove(); pci_scan_child_bus(hbus->pci_bus); + hv_pci_assign_slots(hbus); pci_unlock_rescan_remove(); break; @@ -1858,6 +1894,9 @@ static void hv_eject_device_work(struct work_struct *work) list_del(&hpdev->list_entry); spin_unlock_irqrestore(&hpdev->hbus->device_list_lock, flags); + if (hpdev->pci_slot) + pci_destroy_slot(hpdev->pci_slot); + memset(&ctxt, 0, sizeof(ctxt)); ejct_pkt = (struct pci_eject_response *)&ctxt.pkt.message; ejct_pkt->message_type.type = PCI_EJECTION_COMPLETE; diff --git a/drivers/pci/hotplug/acpiphp_glue.c b/drivers/pci/hotplug/acpiphp_glue.c index ef0b1b6ba86f..12afa7fdf77e 100644 --- a/drivers/pci/hotplug/acpiphp_glue.c +++ b/drivers/pci/hotplug/acpiphp_glue.c @@ -457,17 +457,18 @@ static void acpiphp_native_scan_bridge(struct pci_dev *bridge) /** * enable_slot - enable, configure a slot * @slot: slot to be enabled + * @bridge: true if enable is for the whole bridge (not a single slot) * * This function should be called per *physical slot*, * not per each slot object in ACPI namespace. */ -static void enable_slot(struct acpiphp_slot *slot) +static void enable_slot(struct acpiphp_slot *slot, bool bridge) { struct pci_dev *dev; struct pci_bus *bus = slot->bus; struct acpiphp_func *func; - if (bus->self && hotplug_is_native(bus->self)) { + if (bridge && bus->self && hotplug_is_native(bus->self)) { /* * If native hotplug is used, it will take care of hotplug * slot management and resource allocation for hotplug @@ -701,7 +702,7 @@ static void acpiphp_check_bridge(struct acpiphp_bridge *bridge) trim_stale_devices(dev); /* configure all functions */ - enable_slot(slot); + enable_slot(slot, true); } else { disable_slot(slot); } @@ -785,7 +786,7 @@ static void hotplug_event(u32 type, struct acpiphp_context *context) if (bridge) acpiphp_check_bridge(bridge); else if (!(slot->flags & SLOT_IS_GOING_AWAY)) - enable_slot(slot); + enable_slot(slot, false); break; @@ -973,7 +974,7 @@ int acpiphp_enable_slot(struct acpiphp_slot *slot) /* configure all functions */ if (!(slot->flags & SLOT_ENABLED)) - enable_slot(slot); + enable_slot(slot, false); pci_unlock_rescan_remove(); return 0; diff --git a/drivers/pci/hotplug/pciehp_hpc.c b/drivers/pci/hotplug/pciehp_hpc.c index 7136e3430925..a938abdb41ce 100644 --- a/drivers/pci/hotplug/pciehp_hpc.c +++ b/drivers/pci/hotplug/pciehp_hpc.c @@ -496,7 +496,7 @@ int pciehp_power_on_slot(struct slot *slot) u16 slot_status; int retval; - /* Clear sticky power-fault bit from previous power failures */ + /* Clear power-fault bit from previous power failures */ pcie_capability_read_word(pdev, PCI_EXP_SLTSTA, &slot_status); if (slot_status & PCI_EXP_SLTSTA_PFD) pcie_capability_write_word(pdev, PCI_EXP_SLTSTA, @@ -646,6 +646,14 @@ static irqreturn_t pciehp_ist(int irq, void *dev_id) pciehp_handle_button_press(slot); } + /* Check Power Fault Detected */ + if ((events & PCI_EXP_SLTSTA_PFD) && !ctrl->power_fault_detected) { + ctrl->power_fault_detected = 1; + ctrl_err(ctrl, "Slot(%s): Power fault\n", slot_name(slot)); + pciehp_set_attention_status(slot, 1); + pciehp_green_led_off(slot); + } + /* * Disable requests have higher priority than Presence Detect Changed * or Data Link Layer State Changed events. @@ -657,14 +665,6 @@ static irqreturn_t pciehp_ist(int irq, void *dev_id) pciehp_handle_presence_or_link_change(slot, events); up_read(&ctrl->reset_lock); - /* Check Power Fault Detected */ - if ((events & PCI_EXP_SLTSTA_PFD) && !ctrl->power_fault_detected) { - ctrl->power_fault_detected = 1; - ctrl_err(ctrl, "Slot(%s): Power fault\n", slot_name(slot)); - pciehp_set_attention_status(slot, 1); - pciehp_green_led_off(slot); - } - pci_config_pm_runtime_put(pdev); wake_up(&ctrl->requester); return IRQ_HANDLED; diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 29ff9619b5fa..1835f3a7aa8d 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4547,6 +4547,7 @@ int pci_bridge_secondary_bus_reset(struct pci_dev *dev) return pci_dev_wait(dev, "bus reset", PCIE_RESET_READY_POLL_MS); } +EXPORT_SYMBOL_GPL(pci_bridge_secondary_bus_reset); static int pci_parent_bus_reset(struct pci_dev *dev, int probe) { @@ -5200,7 +5201,7 @@ static int __pci_reset_bus(struct pci_bus *bus) */ int pci_reset_bus(struct pci_dev *pdev) { - return pci_probe_reset_slot(pdev->slot) ? + return (!pci_probe_reset_slot(pdev->slot)) ? __pci_reset_slot(pdev->slot) : __pci_reset_bus(pdev->bus); } EXPORT_SYMBOL_GPL(pci_reset_bus); diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index ec784009a36b..201f9e5ff55c 100644 --- a/drivers/pci/probe.c +++ b/drivers/pci/probe.c @@ -2074,6 +2074,7 @@ static void pci_configure_eetlp_prefix(struct pci_dev *dev) { #ifdef CONFIG_PCI_PASID struct pci_dev *bridge; + int pcie_type; u32 cap; if (!pci_is_pcie(dev)) @@ -2083,7 +2084,9 @@ static void pci_configure_eetlp_prefix(struct pci_dev *dev) if (!(cap & PCI_EXP_DEVCAP2_EE_PREFIX)) return; - if (pci_pcie_type(dev) == PCI_EXP_TYPE_ROOT_PORT) + pcie_type = pci_pcie_type(dev); + if (pcie_type == PCI_EXP_TYPE_ROOT_PORT || + pcie_type == PCI_EXP_TYPE_RC_END) dev->eetlp_prefix_path = 1; else { bridge = pci_upstream_bridge(dev); diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c index ef7143a274e0..6bc27b7fd452 100644 --- a/drivers/pci/quirks.c +++ b/drivers/pci/quirks.c @@ -4355,11 +4355,6 @@ static int pci_quirk_qcom_rp_acs(struct pci_dev *dev, u16 acs_flags) * * 0x9d10-0x9d1b PCI Express Root port #{1-12} * - * The 300 series chipset suffers from the same bug so include those root - * ports here as well. - * - * 0xa32c-0xa343 PCI Express Root port #{0-24} - * * [1] http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html * [2] http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-1.html * [3] http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-spec-update.html @@ -4377,7 +4372,6 @@ static bool pci_quirk_intel_spt_pch_acs_match(struct pci_dev *dev) case 0xa110 ... 0xa11f: case 0xa167 ... 0xa16a: /* Sunrise Point */ case 0xa290 ... 0xa29f: case 0xa2e7 ... 0xa2ee: /* Union Point */ case 0x9d10 ... 0x9d1b: /* 7th & 8th Gen Mobile */ - case 0xa32c ... 0xa343: /* 300 series */ return true; } diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c index 9940cc70f38b..54a8b30dda38 100644 --- a/drivers/pci/switch/switchtec.c +++ b/drivers/pci/switch/switchtec.c @@ -14,6 +14,8 @@ #include <linux/poll.h> #include <linux/wait.h> +#include <linux/nospec.h> + MODULE_DESCRIPTION("Microsemi Switchtec(tm) PCIe Management Driver"); MODULE_VERSION("0.1"); MODULE_LICENSE("GPL"); @@ -909,6 +911,8 @@ static int ioctl_port_to_pff(struct switchtec_dev *stdev, default: if (p.port > ARRAY_SIZE(pcfg->dsp_pff_inst_id)) return -EINVAL; + p.port = array_index_nospec(p.port, + ARRAY_SIZE(pcfg->dsp_pff_inst_id) + 1); p.pff = ioread32(&pcfg->dsp_pff_inst_id[p.port - 1]); break; } diff --git a/drivers/pinctrl/cirrus/pinctrl-madera-core.c b/drivers/pinctrl/cirrus/pinctrl-madera-core.c index ece41fb2848f..c4f4d904e4a6 100644 --- a/drivers/pinctrl/cirrus/pinctrl-madera-core.c +++ b/drivers/pinctrl/cirrus/pinctrl-madera-core.c @@ -1040,7 +1040,7 @@ static int madera_pin_probe(struct platform_device *pdev) } /* if the configuration is provided through pdata, apply it */ - if (pdata) { + if (pdata && pdata->gpio_configs) { ret = pinctrl_register_mappings(pdata->gpio_configs, pdata->n_gpio_configs); if (ret) { diff --git a/drivers/pinctrl/intel/pinctrl-cannonlake.c b/drivers/pinctrl/intel/pinctrl-cannonlake.c index fb1afe55bf53..e7f45d96b0cb 100644 --- a/drivers/pinctrl/intel/pinctrl-cannonlake.c +++ b/drivers/pinctrl/intel/pinctrl-cannonlake.c @@ -15,10 +15,11 @@ #include "pinctrl-intel.h" -#define CNL_PAD_OWN 0x020 -#define CNL_PADCFGLOCK 0x080 -#define CNL_HOSTSW_OWN 0x0b0 -#define CNL_GPI_IE 0x120 +#define CNL_PAD_OWN 0x020 +#define CNL_PADCFGLOCK 0x080 +#define CNL_LP_HOSTSW_OWN 0x0b0 +#define CNL_H_HOSTSW_OWN 0x0c0 +#define CNL_GPI_IE 0x120 #define CNL_GPP(r, s, e, g) \ { \ @@ -30,12 +31,12 @@ #define CNL_NO_GPIO -1 -#define CNL_COMMUNITY(b, s, e, g) \ +#define CNL_COMMUNITY(b, s, e, o, g) \ { \ .barno = (b), \ .padown_offset = CNL_PAD_OWN, \ .padcfglock_offset = CNL_PADCFGLOCK, \ - .hostown_offset = CNL_HOSTSW_OWN, \ + .hostown_offset = (o), \ .ie_offset = CNL_GPI_IE, \ .pin_base = (s), \ .npins = ((e) - (s) + 1), \ @@ -43,6 +44,12 @@ .ngpps = ARRAY_SIZE(g), \ } +#define CNLLP_COMMUNITY(b, s, e, g) \ + CNL_COMMUNITY(b, s, e, CNL_LP_HOSTSW_OWN, g) + +#define CNLH_COMMUNITY(b, s, e, g) \ + CNL_COMMUNITY(b, s, e, CNL_H_HOSTSW_OWN, g) + /* Cannon Lake-H */ static const struct pinctrl_pin_desc cnlh_pins[] = { /* GPP_A */ @@ -379,7 +386,7 @@ static const struct intel_padgroup cnlh_community1_gpps[] = { static const struct intel_padgroup cnlh_community3_gpps[] = { CNL_GPP(0, 155, 178, 192), /* GPP_K */ CNL_GPP(1, 179, 202, 224), /* GPP_H */ - CNL_GPP(2, 203, 215, 258), /* GPP_E */ + CNL_GPP(2, 203, 215, 256), /* GPP_E */ CNL_GPP(3, 216, 239, 288), /* GPP_F */ CNL_GPP(4, 240, 248, CNL_NO_GPIO), /* SPI */ }; @@ -442,10 +449,10 @@ static const struct intel_function cnlh_functions[] = { }; static const struct intel_community cnlh_communities[] = { - CNL_COMMUNITY(0, 0, 50, cnlh_community0_gpps), - CNL_COMMUNITY(1, 51, 154, cnlh_community1_gpps), - CNL_COMMUNITY(2, 155, 248, cnlh_community3_gpps), - CNL_COMMUNITY(3, 249, 298, cnlh_community4_gpps), + CNLH_COMMUNITY(0, 0, 50, cnlh_community0_gpps), + CNLH_COMMUNITY(1, 51, 154, cnlh_community1_gpps), + CNLH_COMMUNITY(2, 155, 248, cnlh_community3_gpps), + CNLH_COMMUNITY(3, 249, 298, cnlh_community4_gpps), }; static const struct intel_pinctrl_soc_data cnlh_soc_data = { @@ -803,9 +810,9 @@ static const struct intel_padgroup cnllp_community4_gpps[] = { }; static const struct intel_community cnllp_communities[] = { - CNL_COMMUNITY(0, 0, 67, cnllp_community0_gpps), - CNL_COMMUNITY(1, 68, 180, cnllp_community1_gpps), - CNL_COMMUNITY(2, 181, 243, cnllp_community4_gpps), + CNLLP_COMMUNITY(0, 0, 67, cnllp_community0_gpps), + CNLLP_COMMUNITY(1, 68, 180, cnllp_community1_gpps), + CNLLP_COMMUNITY(2, 181, 243, cnllp_community4_gpps), }; static const struct intel_pinctrl_soc_data cnllp_soc_data = { diff --git a/drivers/pinctrl/intel/pinctrl-intel.c b/drivers/pinctrl/intel/pinctrl-intel.c index 62b009b27eda..1ea3438ea67e 100644 --- a/drivers/pinctrl/intel/pinctrl-intel.c +++ b/drivers/pinctrl/intel/pinctrl-intel.c @@ -747,13 +747,63 @@ static const struct pinctrl_desc intel_pinctrl_desc = { .owner = THIS_MODULE, }; +/** + * intel_gpio_to_pin() - Translate from GPIO offset to pin number + * @pctrl: Pinctrl structure + * @offset: GPIO offset from gpiolib + * @commmunity: Community is filled here if not %NULL + * @padgrp: Pad group is filled here if not %NULL + * + * When coming through gpiolib irqchip, the GPIO offset is not + * automatically translated to pinctrl pin number. This function can be + * used to find out the corresponding pinctrl pin. + */ +static int intel_gpio_to_pin(struct intel_pinctrl *pctrl, unsigned offset, + const struct intel_community **community, + const struct intel_padgroup **padgrp) +{ + int i; + + for (i = 0; i < pctrl->ncommunities; i++) { + const struct intel_community *comm = &pctrl->communities[i]; + int j; + + for (j = 0; j < comm->ngpps; j++) { + const struct intel_padgroup *pgrp = &comm->gpps[j]; + + if (pgrp->gpio_base < 0) + continue; + + if (offset >= pgrp->gpio_base && + offset < pgrp->gpio_base + pgrp->size) { + int pin; + + pin = pgrp->base + offset - pgrp->gpio_base; + if (community) + *community = comm; + if (padgrp) + *padgrp = pgrp; + + return pin; + } + } + } + + return -EINVAL; +} + static int intel_gpio_get(struct gpio_chip *chip, unsigned offset) { struct intel_pinctrl *pctrl = gpiochip_get_data(chip); void __iomem *reg; u32 padcfg0; + int pin; - reg = intel_get_padcfg(pctrl, offset, PADCFG0); + pin = intel_gpio_to_pin(pctrl, offset, NULL, NULL); + if (pin < 0) + return -EINVAL; + + reg = intel_get_padcfg(pctrl, pin, PADCFG0); if (!reg) return -EINVAL; @@ -770,8 +820,13 @@ static void intel_gpio_set(struct gpio_chip *chip, unsigned offset, int value) unsigned long flags; void __iomem *reg; u32 padcfg0; + int pin; - reg = intel_get_padcfg(pctrl, offset, PADCFG0); + pin = intel_gpio_to_pin(pctrl, offset, NULL, NULL); + if (pin < 0) + return; + + reg = intel_get_padcfg(pctrl, pin, PADCFG0); if (!reg) return; @@ -790,8 +845,13 @@ static int intel_gpio_get_direction(struct gpio_chip *chip, unsigned int offset) struct intel_pinctrl *pctrl = gpiochip_get_data(chip); void __iomem *reg; u32 padcfg0; + int pin; - reg = intel_get_padcfg(pctrl, offset, PADCFG0); + pin = intel_gpio_to_pin(pctrl, offset, NULL, NULL); + if (pin < 0) + return -EINVAL; + + reg = intel_get_padcfg(pctrl, pin, PADCFG0); if (!reg) return -EINVAL; @@ -827,81 +887,6 @@ static const struct gpio_chip intel_gpio_chip = { .set_config = gpiochip_generic_config, }; -/** - * intel_gpio_to_pin() - Translate from GPIO offset to pin number - * @pctrl: Pinctrl structure - * @offset: GPIO offset from gpiolib - * @commmunity: Community is filled here if not %NULL - * @padgrp: Pad group is filled here if not %NULL - * - * When coming through gpiolib irqchip, the GPIO offset is not - * automatically translated to pinctrl pin number. This function can be - * used to find out the corresponding pinctrl pin. - */ -static int intel_gpio_to_pin(struct intel_pinctrl *pctrl, unsigned offset, - const struct intel_community **community, - const struct intel_padgroup **padgrp) -{ - int i; - - for (i = 0; i < pctrl->ncommunities; i++) { - const struct intel_community *comm = &pctrl->communities[i]; - int j; - - for (j = 0; j < comm->ngpps; j++) { - const struct intel_padgroup *pgrp = &comm->gpps[j]; - - if (pgrp->gpio_base < 0) - continue; - - if (offset >= pgrp->gpio_base && - offset < pgrp->gpio_base + pgrp->size) { - int pin; - - pin = pgrp->base + offset - pgrp->gpio_base; - if (community) - *community = comm; - if (padgrp) - *padgrp = pgrp; - - return pin; - } - } - } - - return -EINVAL; -} - -static int intel_gpio_irq_reqres(struct irq_data *d) -{ - struct gpio_chip *gc = irq_data_get_irq_chip_data(d); - struct intel_pinctrl *pctrl = gpiochip_get_data(gc); - int pin; - int ret; - - pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), NULL, NULL); - if (pin >= 0) { - ret = gpiochip_lock_as_irq(gc, pin); - if (ret) { - dev_err(pctrl->dev, "unable to lock HW IRQ %d for IRQ\n", - pin); - return ret; - } - } - return 0; -} - -static void intel_gpio_irq_relres(struct irq_data *d) -{ - struct gpio_chip *gc = irq_data_get_irq_chip_data(d); - struct intel_pinctrl *pctrl = gpiochip_get_data(gc); - int pin; - - pin = intel_gpio_to_pin(pctrl, irqd_to_hwirq(d), NULL, NULL); - if (pin >= 0) - gpiochip_unlock_as_irq(gc, pin); -} - static void intel_gpio_irq_ack(struct irq_data *d) { struct gpio_chip *gc = irq_data_get_irq_chip_data(d); @@ -1117,8 +1102,6 @@ static irqreturn_t intel_gpio_irq(int irq, void *data) static struct irq_chip intel_gpio_irqchip = { .name = "intel-gpio", - .irq_request_resources = intel_gpio_irq_reqres, - .irq_release_resources = intel_gpio_irq_relres, .irq_enable = intel_gpio_irq_enable, .irq_ack = intel_gpio_irq_ack, .irq_mask = intel_gpio_irq_mask, diff --git a/drivers/pinctrl/pinctrl-amd.c b/drivers/pinctrl/pinctrl-amd.c index 41ccc759b8b8..1425c2874d40 100644 --- a/drivers/pinctrl/pinctrl-amd.c +++ b/drivers/pinctrl/pinctrl-amd.c @@ -348,21 +348,12 @@ static void amd_gpio_irq_enable(struct irq_data *d) unsigned long flags; struct gpio_chip *gc = irq_data_get_irq_chip_data(d); struct amd_gpio *gpio_dev = gpiochip_get_data(gc); - u32 mask = BIT(INTERRUPT_ENABLE_OFF) | BIT(INTERRUPT_MASK_OFF); raw_spin_lock_irqsave(&gpio_dev->lock, flags); pin_reg = readl(gpio_dev->base + (d->hwirq)*4); pin_reg |= BIT(INTERRUPT_ENABLE_OFF); pin_reg |= BIT(INTERRUPT_MASK_OFF); writel(pin_reg, gpio_dev->base + (d->hwirq)*4); - /* - * When debounce logic is enabled it takes ~900 us before interrupts - * can be enabled. During this "debounce warm up" period the - * "INTERRUPT_ENABLE" bit will read as 0. Poll the bit here until it - * reads back as 1, signaling that interrupts are now enabled. - */ - while ((readl(gpio_dev->base + (d->hwirq)*4) & mask) != mask) - continue; raw_spin_unlock_irqrestore(&gpio_dev->lock, flags); } @@ -426,7 +417,7 @@ static void amd_gpio_irq_eoi(struct irq_data *d) static int amd_gpio_irq_set_type(struct irq_data *d, unsigned int type) { int ret = 0; - u32 pin_reg; + u32 pin_reg, pin_reg_irq_en, mask; unsigned long flags, irq_flags; struct gpio_chip *gc = irq_data_get_irq_chip_data(d); struct amd_gpio *gpio_dev = gpiochip_get_data(gc); @@ -495,6 +486,28 @@ static int amd_gpio_irq_set_type(struct irq_data *d, unsigned int type) } pin_reg |= CLR_INTR_STAT << INTERRUPT_STS_OFF; + /* + * If WAKE_INT_MASTER_REG.MaskStsEn is set, a software write to the + * debounce registers of any GPIO will block wake/interrupt status + * generation for *all* GPIOs for a lenght of time that depends on + * WAKE_INT_MASTER_REG.MaskStsLength[11:0]. During this period the + * INTERRUPT_ENABLE bit will read as 0. + * + * We temporarily enable irq for the GPIO whose configuration is + * changing, and then wait for it to read back as 1 to know when + * debounce has settled and then disable the irq again. + * We do this polling with the spinlock held to ensure other GPIO + * access routines do not read an incorrect value for the irq enable + * bit of other GPIOs. We keep the GPIO masked while polling to avoid + * spurious irqs, and disable the irq again after polling. + */ + mask = BIT(INTERRUPT_ENABLE_OFF); + pin_reg_irq_en = pin_reg; + pin_reg_irq_en |= mask; + pin_reg_irq_en &= ~BIT(INTERRUPT_MASK_OFF); + writel(pin_reg_irq_en, gpio_dev->base + (d->hwirq)*4); + while ((readl(gpio_dev->base + (d->hwirq)*4) & mask) != mask) + continue; writel(pin_reg, gpio_dev->base + (d->hwirq)*4); raw_spin_unlock_irqrestore(&gpio_dev->lock, flags); diff --git a/drivers/pinctrl/pinctrl-ingenic.c b/drivers/pinctrl/pinctrl-ingenic.c index 6a1b6058b991..628817c40e3b 100644 --- a/drivers/pinctrl/pinctrl-ingenic.c +++ b/drivers/pinctrl/pinctrl-ingenic.c @@ -793,7 +793,7 @@ static int ingenic_pinctrl_probe(struct platform_device *pdev) err = pinctrl_generic_add_group(jzpc->pctl, group->name, group->pins, group->num_pins, group->data); - if (err) { + if (err < 0) { dev_err(dev, "Failed to register group %s\n", group->name); return err; @@ -806,7 +806,7 @@ static int ingenic_pinctrl_probe(struct platform_device *pdev) err = pinmux_generic_add_function(jzpc->pctl, func->name, func->group_names, func->num_group_names, func->data); - if (err) { + if (err < 0) { dev_err(dev, "Failed to register function %s\n", func->name); return err; diff --git a/drivers/pinctrl/qcom/pinctrl-msm.c b/drivers/pinctrl/qcom/pinctrl-msm.c index 2155a30c282b..5d72ffad32c2 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm.c +++ b/drivers/pinctrl/qcom/pinctrl-msm.c @@ -634,6 +634,29 @@ static void msm_gpio_irq_mask(struct irq_data *d) raw_spin_lock_irqsave(&pctrl->lock, flags); val = readl(pctrl->regs + g->intr_cfg_reg); + /* + * There are two bits that control interrupt forwarding to the CPU. The + * RAW_STATUS_EN bit causes the level or edge sensed on the line to be + * latched into the interrupt status register when the hardware detects + * an irq that it's configured for (either edge for edge type or level + * for level type irq). The 'non-raw' status enable bit causes the + * hardware to assert the summary interrupt to the CPU if the latched + * status bit is set. There's a bug though, the edge detection logic + * seems to have a problem where toggling the RAW_STATUS_EN bit may + * cause the status bit to latch spuriously when there isn't any edge + * so we can't touch that bit for edge type irqs and we have to keep + * the bit set anyway so that edges are latched while the line is masked. + * + * To make matters more complicated, leaving the RAW_STATUS_EN bit + * enabled all the time causes level interrupts to re-latch into the + * status register because the level is still present on the line after + * we ack it. We clear the raw status enable bit during mask here and + * set the bit on unmask so the interrupt can't latch into the hardware + * while it's masked. + */ + if (irqd_get_trigger_type(d) & IRQ_TYPE_LEVEL_MASK) + val &= ~BIT(g->intr_raw_status_bit); + val &= ~BIT(g->intr_enable_bit); writel(val, pctrl->regs + g->intr_cfg_reg); @@ -655,6 +678,7 @@ static void msm_gpio_irq_unmask(struct irq_data *d) raw_spin_lock_irqsave(&pctrl->lock, flags); val = readl(pctrl->regs + g->intr_cfg_reg); + val |= BIT(g->intr_raw_status_bit); val |= BIT(g->intr_enable_bit); writel(val, pctrl->regs + g->intr_cfg_reg); diff --git a/drivers/platform/x86/alienware-wmi.c b/drivers/platform/x86/alienware-wmi.c index d975462a4c57..f10af5c383c5 100644 --- a/drivers/platform/x86/alienware-wmi.c +++ b/drivers/platform/x86/alienware-wmi.c @@ -536,6 +536,7 @@ static acpi_status alienware_wmax_command(struct wmax_basic_args *in_args, if (obj && obj->type == ACPI_TYPE_INTEGER) *out_data = (u32) obj->integer.value; } + kfree(output.pointer); return status; } diff --git a/drivers/platform/x86/dell-smbios-wmi.c b/drivers/platform/x86/dell-smbios-wmi.c index 88afe5651d24..cf2229ece9ff 100644 --- a/drivers/platform/x86/dell-smbios-wmi.c +++ b/drivers/platform/x86/dell-smbios-wmi.c @@ -78,6 +78,7 @@ static int run_smbios_call(struct wmi_device *wdev) dev_dbg(&wdev->dev, "result: [%08x,%08x,%08x,%08x]\n", priv->buf->std.output[0], priv->buf->std.output[1], priv->buf->std.output[2], priv->buf->std.output[3]); + kfree(output.pointer); return 0; } diff --git a/drivers/regulator/bd71837-regulator.c b/drivers/regulator/bd71837-regulator.c index 0f8ac8dec3e1..a1bd8aaf4d98 100644 --- a/drivers/regulator/bd71837-regulator.c +++ b/drivers/regulator/bd71837-regulator.c @@ -569,6 +569,25 @@ static int bd71837_probe(struct platform_device *pdev) BD71837_REG_REGLOCK); } + /* + * There is a HW quirk in BD71837. The shutdown sequence timings for + * bucks/LDOs which are controlled via register interface are changed. + * At PMIC poweroff the voltage for BUCK6/7 is cut immediately at the + * beginning of shut-down sequence. As bucks 6 and 7 are parent + * supplies for LDO5 and LDO6 - this causes LDO5/6 voltage + * monitoring to errorneously detect under voltage and force PMIC to + * emergency state instead of poweroff. In order to avoid this we + * disable voltage monitoring for LDO5 and LDO6 + */ + err = regmap_update_bits(pmic->mfd->regmap, BD718XX_REG_MVRFLTMASK2, + BD718XX_LDO5_VRMON80 | BD718XX_LDO6_VRMON80, + BD718XX_LDO5_VRMON80 | BD718XX_LDO6_VRMON80); + if (err) { + dev_err(&pmic->pdev->dev, + "Failed to disable voltage monitoring\n"); + goto err; + } + for (i = 0; i < ARRAY_SIZE(pmic_regulator_inits); i++) { struct regulator_desc *desc; diff --git a/drivers/regulator/core.c b/drivers/regulator/core.c index bb1324f93143..9577d8941846 100644 --- a/drivers/regulator/core.c +++ b/drivers/regulator/core.c @@ -3161,7 +3161,7 @@ static inline int regulator_suspend_toggle(struct regulator_dev *rdev, if (!rstate->changeable) return -EPERM; - rstate->enabled = en; + rstate->enabled = (en) ? ENABLE_IN_SUSPEND : DISABLE_IN_SUSPEND; return 0; } @@ -4395,13 +4395,13 @@ regulator_register(const struct regulator_desc *regulator_desc, !rdev->desc->fixed_uV) rdev->is_switch = true; + dev_set_drvdata(&rdev->dev, rdev); ret = device_register(&rdev->dev); if (ret != 0) { put_device(&rdev->dev); goto unset_supplies; } - dev_set_drvdata(&rdev->dev, rdev); rdev_init_debugfs(rdev); /* try to resolve regulators supply since a new one was registered */ diff --git a/drivers/regulator/of_regulator.c b/drivers/regulator/of_regulator.c index 638f17d4c848..210fc20f7de7 100644 --- a/drivers/regulator/of_regulator.c +++ b/drivers/regulator/of_regulator.c @@ -213,8 +213,6 @@ static void of_get_regulation_constraints(struct device_node *np, else if (of_property_read_bool(suspend_np, "regulator-off-in-suspend")) suspend_state->enabled = DISABLE_IN_SUSPEND; - else - suspend_state->enabled = DO_NOTHING_IN_SUSPEND; if (!of_property_read_u32(np, "regulator-suspend-min-microvolt", &pval)) diff --git a/drivers/s390/crypto/ap_bus.c b/drivers/s390/crypto/ap_bus.c index ec891bc7d10a..f039266b275d 100644 --- a/drivers/s390/crypto/ap_bus.c +++ b/drivers/s390/crypto/ap_bus.c @@ -872,8 +872,6 @@ static int hex2bitmap(const char *str, unsigned long *bitmap, int bits) if (bits & 0x07) return -EINVAL; - memset(bitmap, 0, bits / 8); - if (str[0] == '0' && str[1] == 'x') str++; if (*str == 'x') @@ -895,25 +893,23 @@ static int hex2bitmap(const char *str, unsigned long *bitmap, int bits) } /* - * str2clrsetmasks() - parse bitmask argument and set the clear and - * the set bitmap mask. A concatenation (done with ',') of these terms - * is recognized: + * modify_bitmap() - parse bitmask argument and modify an existing + * bit mask accordingly. A concatenation (done with ',') of these + * terms is recognized: * +<bitnr>[-<bitnr>] or -<bitnr>[-<bitnr>] * <bitnr> may be any valid number (hex, decimal or octal) in the range * 0...bits-1; the leading + or - is required. Here are some examples: * +0-15,+32,-128,-0xFF * -0-255,+1-16,+0x128 * +1,+2,+3,+4,-5,-7-10 - * Returns a clear and a set bitmask. Every positive value in the string - * results in a bit set in the set mask and every negative value in the - * string results in a bit SET in the clear mask. As a bit may be touched - * more than once, the last 'operation' wins: +0-255,-128 = all but bit - * 128 set in the set mask, only bit 128 set in the clear mask. + * Returns the new bitmap after all changes have been applied. Every + * positive value in the string will set a bit and every negative value + * in the string will clear a bit. As a bit may be touched more than once, + * the last 'operation' wins: + * +0-255,-128 = first bits 0-255 will be set, then bit 128 will be + * cleared again. All other bits are unmodified. */ -static int str2clrsetmasks(const char *str, - unsigned long *clrmap, - unsigned long *setmap, - int bits) +static int modify_bitmap(const char *str, unsigned long *bitmap, int bits) { int a, i, z; char *np, sign; @@ -922,9 +918,6 @@ static int str2clrsetmasks(const char *str, if (bits & 0x07) return -EINVAL; - memset(clrmap, 0, bits / 8); - memset(setmap, 0, bits / 8); - while (*str) { sign = *str++; if (sign != '+' && sign != '-') @@ -940,13 +933,10 @@ static int str2clrsetmasks(const char *str, str = np; } for (i = a; i <= z; i++) - if (sign == '+') { - set_bit_inv(i, setmap); - clear_bit_inv(i, clrmap); - } else { - clear_bit_inv(i, setmap); - set_bit_inv(i, clrmap); - } + if (sign == '+') + set_bit_inv(i, bitmap); + else + clear_bit_inv(i, bitmap); while (*str == ',' || *str == '\n') str++; } @@ -970,44 +960,34 @@ static int process_mask_arg(const char *str, unsigned long *bitmap, int bits, struct mutex *lock) { - int i; + unsigned long *newmap, size; + int rc; /* bits needs to be a multiple of 8 */ if (bits & 0x07) return -EINVAL; + size = BITS_TO_LONGS(bits)*sizeof(unsigned long); + newmap = kmalloc(size, GFP_KERNEL); + if (!newmap) + return -ENOMEM; + if (mutex_lock_interruptible(lock)) { + kfree(newmap); + return -ERESTARTSYS; + } + if (*str == '+' || *str == '-') { - DECLARE_BITMAP(clrm, bits); - DECLARE_BITMAP(setm, bits); - - i = str2clrsetmasks(str, clrm, setm, bits); - if (i) - return i; - if (mutex_lock_interruptible(lock)) - return -ERESTARTSYS; - for (i = 0; i < bits; i++) { - if (test_bit_inv(i, clrm)) - clear_bit_inv(i, bitmap); - if (test_bit_inv(i, setm)) - set_bit_inv(i, bitmap); - } + memcpy(newmap, bitmap, size); + rc = modify_bitmap(str, newmap, bits); } else { - DECLARE_BITMAP(setm, bits); - - i = hex2bitmap(str, setm, bits); - if (i) - return i; - if (mutex_lock_interruptible(lock)) - return -ERESTARTSYS; - for (i = 0; i < bits; i++) - if (test_bit_inv(i, setm)) - set_bit_inv(i, bitmap); - else - clear_bit_inv(i, bitmap); + memset(newmap, 0, size); + rc = hex2bitmap(str, newmap, bits); } + if (rc == 0) + memcpy(bitmap, newmap, size); mutex_unlock(lock); - - return 0; + kfree(newmap); + return rc; } /* diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c index 49f64eb3eab0..de8282420f96 100644 --- a/drivers/s390/net/qeth_core_main.c +++ b/drivers/s390/net/qeth_core_main.c @@ -25,6 +25,7 @@ #include <linux/netdevice.h> #include <linux/netdev_features.h> #include <linux/skbuff.h> +#include <linux/vmalloc.h> #include <net/iucv/af_iucv.h> #include <net/dsfield.h> @@ -4699,7 +4700,7 @@ static int qeth_query_oat_command(struct qeth_card *card, char __user *udata) priv.buffer_len = oat_data.buffer_len; priv.response_len = 0; - priv.buffer = kzalloc(oat_data.buffer_len, GFP_KERNEL); + priv.buffer = vzalloc(oat_data.buffer_len); if (!priv.buffer) { rc = -ENOMEM; goto out; @@ -4740,7 +4741,7 @@ static int qeth_query_oat_command(struct qeth_card *card, char __user *udata) rc = -EFAULT; out_free: - kfree(priv.buffer); + vfree(priv.buffer); out: return rc; } @@ -5706,6 +5707,8 @@ static struct net_device *qeth_alloc_netdev(struct qeth_card *card) dev->priv_flags &= ~IFF_TX_SKB_SHARING; dev->hw_features |= NETIF_F_SG; dev->vlan_features |= NETIF_F_SG; + if (IS_IQD(card)) + dev->features |= NETIF_F_SG; } return dev; @@ -5768,8 +5771,10 @@ static int qeth_core_probe_device(struct ccwgroup_device *gdev) qeth_update_from_chp_desc(card); card->dev = qeth_alloc_netdev(card); - if (!card->dev) + if (!card->dev) { + rc = -ENOMEM; goto err_card; + } qeth_determine_capabilities(card); enforced_disc = qeth_enforce_discipline(card); diff --git a/drivers/s390/net/qeth_l2_main.c b/drivers/s390/net/qeth_l2_main.c index 710fa74892ae..b5e38531733f 100644 --- a/drivers/s390/net/qeth_l2_main.c +++ b/drivers/s390/net/qeth_l2_main.c @@ -423,7 +423,7 @@ static int qeth_l2_process_inbound_buffer(struct qeth_card *card, default: dev_kfree_skb_any(skb); QETH_CARD_TEXT(card, 3, "inbunkno"); - QETH_DBF_HEX(CTRL, 3, hdr, QETH_DBF_CTRL_LEN); + QETH_DBF_HEX(CTRL, 3, hdr, sizeof(*hdr)); continue; } work_done++; diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c index 7175086677fb..ada258c01a08 100644 --- a/drivers/s390/net/qeth_l3_main.c +++ b/drivers/s390/net/qeth_l3_main.c @@ -1390,7 +1390,7 @@ static int qeth_l3_process_inbound_buffer(struct qeth_card *card, default: dev_kfree_skb_any(skb); QETH_CARD_TEXT(card, 3, "inbunkno"); - QETH_DBF_HEX(CTRL, 3, hdr, QETH_DBF_CTRL_LEN); + QETH_DBF_HEX(CTRL, 3, hdr, sizeof(*hdr)); continue; } work_done++; diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig index 8fc851a9e116..7c097006c54d 100644 --- a/drivers/scsi/Kconfig +++ b/drivers/scsi/Kconfig @@ -52,12 +52,12 @@ config SCSI_MQ_DEFAULT default y depends on SCSI ---help--- - This option enables the new blk-mq based I/O path for SCSI - devices by default. With the option the scsi_mod.use_blk_mq - module/boot option defaults to Y, without it to N, but it can - still be overridden either way. + This option enables the blk-mq based I/O path for SCSI devices by + default. With this option the scsi_mod.use_blk_mq module/boot + option defaults to Y, without it to N, but it can still be + overridden either way. - If unsure say N. + If unsure say Y. config SCSI_PROC_FS bool "legacy /proc/scsi/ support" diff --git a/drivers/scsi/aacraid/aacraid.h b/drivers/scsi/aacraid/aacraid.h index 29bf1e60f542..39eb415987fc 100644 --- a/drivers/scsi/aacraid/aacraid.h +++ b/drivers/scsi/aacraid/aacraid.h @@ -1346,7 +1346,7 @@ struct fib { struct aac_hba_map_info { __le32 rmw_nexus; /* nexus for native HBA devices */ u8 devtype; /* device type */ - u8 reset_state; /* 0 - no reset, 1..x - */ + s8 reset_state; /* 0 - no reset, 1..x - */ /* after xth TM LUN reset */ u16 qd_limit; u32 scan_counter; diff --git a/drivers/scsi/csiostor/csio_hw.c b/drivers/scsi/csiostor/csio_hw.c index 23d07e9f87d0..e51923886475 100644 --- a/drivers/scsi/csiostor/csio_hw.c +++ b/drivers/scsi/csiostor/csio_hw.c @@ -1602,6 +1602,46 @@ fw_port_cap32_t fwcaps16_to_caps32(fw_port_cap16_t caps16) } /** + * fwcaps32_to_caps16 - convert 32-bit Port Capabilities to 16-bits + * @caps32: a 32-bit Port Capabilities value + * + * Returns the equivalent 16-bit Port Capabilities value. Note that + * not all 32-bit Port Capabilities can be represented in the 16-bit + * Port Capabilities and some fields/values may not make it. + */ +fw_port_cap16_t fwcaps32_to_caps16(fw_port_cap32_t caps32) +{ + fw_port_cap16_t caps16 = 0; + + #define CAP32_TO_CAP16(__cap) \ + do { \ + if (caps32 & FW_PORT_CAP32_##__cap) \ + caps16 |= FW_PORT_CAP_##__cap; \ + } while (0) + + CAP32_TO_CAP16(SPEED_100M); + CAP32_TO_CAP16(SPEED_1G); + CAP32_TO_CAP16(SPEED_10G); + CAP32_TO_CAP16(SPEED_25G); + CAP32_TO_CAP16(SPEED_40G); + CAP32_TO_CAP16(SPEED_100G); + CAP32_TO_CAP16(FC_RX); + CAP32_TO_CAP16(FC_TX); + CAP32_TO_CAP16(802_3_PAUSE); + CAP32_TO_CAP16(802_3_ASM_DIR); + CAP32_TO_CAP16(ANEG); + CAP32_TO_CAP16(FORCE_PAUSE); + CAP32_TO_CAP16(MDIAUTO); + CAP32_TO_CAP16(MDISTRAIGHT); + CAP32_TO_CAP16(FEC_RS); + CAP32_TO_CAP16(FEC_BASER_RS); + + #undef CAP32_TO_CAP16 + + return caps16; +} + +/** * lstatus_to_fwcap - translate old lstatus to 32-bit Port Capabilities * @lstatus: old FW_PORT_ACTION_GET_PORT_INFO lstatus value * @@ -1759,7 +1799,7 @@ csio_enable_ports(struct csio_hw *hw) val = 1; csio_mb_params(hw, mbp, CSIO_MB_DEFAULT_TMO, - hw->pfn, 0, 1, ¶m, &val, false, + hw->pfn, 0, 1, ¶m, &val, true, NULL); if (csio_mb_issue(hw, mbp)) { @@ -1769,16 +1809,9 @@ csio_enable_ports(struct csio_hw *hw) return -EINVAL; } - csio_mb_process_read_params_rsp(hw, mbp, &retval, 1, - &val); - if (retval != FW_SUCCESS) { - csio_err(hw, "FW_PARAMS_CMD(r) port:%d failed: 0x%x\n", - portid, retval); - mempool_free(mbp, hw->mb_mempool); - return -EINVAL; - } - - fw_caps = val; + csio_mb_process_read_params_rsp(hw, mbp, &retval, + 0, NULL); + fw_caps = retval ? FW_CAPS16 : FW_CAPS32; } /* Read PORT information */ @@ -2364,8 +2397,8 @@ bye: } /* - * Returns -EINVAL if attempts to flash the firmware failed - * else returns 0, + * Returns -EINVAL if attempts to flash the firmware failed, + * -ENOMEM if memory allocation failed else returns 0, * if flashing was not attempted because the card had the * latest firmware ECANCELED is returned */ @@ -2393,6 +2426,13 @@ csio_hw_flash_fw(struct csio_hw *hw, int *reset) return -EINVAL; } + /* allocate memory to read the header of the firmware on the + * card + */ + card_fw = kmalloc(sizeof(*card_fw), GFP_KERNEL); + if (!card_fw) + return -ENOMEM; + if (csio_is_t5(pci_dev->device & CSIO_HW_CHIP_MASK)) fw_bin_file = FW_FNAME_T5; else @@ -2406,11 +2446,6 @@ csio_hw_flash_fw(struct csio_hw *hw, int *reset) fw_size = fw->size; } - /* allocate memory to read the header of the firmware on the - * card - */ - card_fw = kmalloc(sizeof(*card_fw), GFP_KERNEL); - /* upgrade FW logic */ ret = csio_hw_prep_fw(hw, fw_info, fw_data, fw_size, card_fw, hw->fw_state, reset); diff --git a/drivers/scsi/csiostor/csio_hw.h b/drivers/scsi/csiostor/csio_hw.h index 9e73ef771eb7..e351af6e7c81 100644 --- a/drivers/scsi/csiostor/csio_hw.h +++ b/drivers/scsi/csiostor/csio_hw.h @@ -639,6 +639,7 @@ int csio_handle_intr_status(struct csio_hw *, unsigned int, fw_port_cap32_t fwcap_to_fwspeed(fw_port_cap32_t acaps); fw_port_cap32_t fwcaps16_to_caps32(fw_port_cap16_t caps16); +fw_port_cap16_t fwcaps32_to_caps16(fw_port_cap32_t caps32); fw_port_cap32_t lstatus_to_fwcap(u32 lstatus); int csio_hw_start(struct csio_hw *); diff --git a/drivers/scsi/csiostor/csio_mb.c b/drivers/scsi/csiostor/csio_mb.c index c026417269c3..6f13673d6aa0 100644 --- a/drivers/scsi/csiostor/csio_mb.c +++ b/drivers/scsi/csiostor/csio_mb.c @@ -368,7 +368,7 @@ csio_mb_port(struct csio_hw *hw, struct csio_mb *mbp, uint32_t tmo, FW_CMD_LEN16_V(sizeof(*cmdp) / 16)); if (fw_caps == FW_CAPS16) - cmdp->u.l1cfg.rcap = cpu_to_be32(fc); + cmdp->u.l1cfg.rcap = cpu_to_be32(fwcaps32_to_caps16(fc)); else cmdp->u.l1cfg32.rcap32 = cpu_to_be32(fc); } @@ -395,8 +395,8 @@ csio_mb_process_read_port_rsp(struct csio_hw *hw, struct csio_mb *mbp, *pcaps = fwcaps16_to_caps32(ntohs(rsp->u.info.pcap)); *acaps = fwcaps16_to_caps32(ntohs(rsp->u.info.acap)); } else { - *pcaps = ntohs(rsp->u.info32.pcaps32); - *acaps = ntohs(rsp->u.info32.acaps32); + *pcaps = be32_to_cpu(rsp->u.info32.pcaps32); + *acaps = be32_to_cpu(rsp->u.info32.acaps32); } } } diff --git a/drivers/scsi/hosts.c b/drivers/scsi/hosts.c index f02dcc875a09..ea4b0bb0c1cd 100644 --- a/drivers/scsi/hosts.c +++ b/drivers/scsi/hosts.c @@ -563,35 +563,13 @@ struct Scsi_Host *scsi_host_get(struct Scsi_Host *shost) } EXPORT_SYMBOL(scsi_host_get); -struct scsi_host_mq_in_flight { - int cnt; -}; - -static void scsi_host_check_in_flight(struct request *rq, void *data, - bool reserved) -{ - struct scsi_host_mq_in_flight *in_flight = data; - - if (blk_mq_request_started(rq)) - in_flight->cnt++; -} - /** * scsi_host_busy - Return the host busy counter * @shost: Pointer to Scsi_Host to inc. **/ int scsi_host_busy(struct Scsi_Host *shost) { - struct scsi_host_mq_in_flight in_flight = { - .cnt = 0, - }; - - if (!shost->use_blk_mq) - return atomic_read(&shost->host_busy); - - blk_mq_tagset_busy_iter(&shost->tag_set, scsi_host_check_in_flight, - &in_flight); - return in_flight.cnt; + return atomic_read(&shost->host_busy); } EXPORT_SYMBOL(scsi_host_busy); diff --git a/drivers/scsi/hpsa.c b/drivers/scsi/hpsa.c index 58bb70b886d7..c120929d4ffe 100644 --- a/drivers/scsi/hpsa.c +++ b/drivers/scsi/hpsa.c @@ -976,7 +976,7 @@ static struct scsi_host_template hpsa_driver_template = { #endif .sdev_attrs = hpsa_sdev_attrs, .shost_attrs = hpsa_shost_attrs, - .max_sectors = 1024, + .max_sectors = 2048, .no_write_same = 1, }; diff --git a/drivers/scsi/ibmvscsi_tgt/ibmvscsi_tgt.c b/drivers/scsi/ibmvscsi_tgt/ibmvscsi_tgt.c index fac377320158..f42a619198c4 100644 --- a/drivers/scsi/ibmvscsi_tgt/ibmvscsi_tgt.c +++ b/drivers/scsi/ibmvscsi_tgt/ibmvscsi_tgt.c @@ -3474,11 +3474,10 @@ static int ibmvscsis_probe(struct vio_dev *vdev, vscsi->dds.window[LOCAL].liobn, vscsi->dds.window[REMOTE].liobn); - strcpy(vscsi->eye, "VSCSI "); - strncat(vscsi->eye, vdev->name, MAX_EYE); + snprintf(vscsi->eye, sizeof(vscsi->eye), "VSCSI %s", vdev->name); vscsi->dds.unit_id = vdev->unit_address; - strncpy(vscsi->dds.partition_name, partition_name, + strscpy(vscsi->dds.partition_name, partition_name, sizeof(vscsi->dds.partition_name)); vscsi->dds.partition_num = partition_number; diff --git a/drivers/scsi/ipr.c b/drivers/scsi/ipr.c index f2ec80b0ffc0..271990bc065b 100644 --- a/drivers/scsi/ipr.c +++ b/drivers/scsi/ipr.c @@ -3335,6 +3335,65 @@ static void ipr_release_dump(struct kref *kref) LEAVE; } +static void ipr_add_remove_thread(struct work_struct *work) +{ + unsigned long lock_flags; + struct ipr_resource_entry *res; + struct scsi_device *sdev; + struct ipr_ioa_cfg *ioa_cfg = + container_of(work, struct ipr_ioa_cfg, scsi_add_work_q); + u8 bus, target, lun; + int did_work; + + ENTER; + spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags); + +restart: + do { + did_work = 0; + if (!ioa_cfg->hrrq[IPR_INIT_HRRQ].allow_cmds) { + spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); + return; + } + + list_for_each_entry(res, &ioa_cfg->used_res_q, queue) { + if (res->del_from_ml && res->sdev) { + did_work = 1; + sdev = res->sdev; + if (!scsi_device_get(sdev)) { + if (!res->add_to_ml) + list_move_tail(&res->queue, &ioa_cfg->free_res_q); + else + res->del_from_ml = 0; + spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); + scsi_remove_device(sdev); + scsi_device_put(sdev); + spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags); + } + break; + } + } + } while (did_work); + + list_for_each_entry(res, &ioa_cfg->used_res_q, queue) { + if (res->add_to_ml) { + bus = res->bus; + target = res->target; + lun = res->lun; + res->add_to_ml = 0; + spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); + scsi_add_device(ioa_cfg->host, bus, target, lun); + spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags); + goto restart; + } + } + + ioa_cfg->scan_done = 1; + spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); + kobject_uevent(&ioa_cfg->host->shost_dev.kobj, KOBJ_CHANGE); + LEAVE; +} + /** * ipr_worker_thread - Worker thread * @work: ioa config struct @@ -3349,13 +3408,9 @@ static void ipr_release_dump(struct kref *kref) static void ipr_worker_thread(struct work_struct *work) { unsigned long lock_flags; - struct ipr_resource_entry *res; - struct scsi_device *sdev; struct ipr_dump *dump; struct ipr_ioa_cfg *ioa_cfg = container_of(work, struct ipr_ioa_cfg, work_q); - u8 bus, target, lun; - int did_work; ENTER; spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags); @@ -3393,49 +3448,9 @@ static void ipr_worker_thread(struct work_struct *work) return; } -restart: - do { - did_work = 0; - if (!ioa_cfg->hrrq[IPR_INIT_HRRQ].allow_cmds) { - spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); - return; - } + schedule_work(&ioa_cfg->scsi_add_work_q); - list_for_each_entry(res, &ioa_cfg->used_res_q, queue) { - if (res->del_from_ml && res->sdev) { - did_work = 1; - sdev = res->sdev; - if (!scsi_device_get(sdev)) { - if (!res->add_to_ml) - list_move_tail(&res->queue, &ioa_cfg->free_res_q); - else - res->del_from_ml = 0; - spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); - scsi_remove_device(sdev); - scsi_device_put(sdev); - spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags); - } - break; - } - } - } while (did_work); - - list_for_each_entry(res, &ioa_cfg->used_res_q, queue) { - if (res->add_to_ml) { - bus = res->bus; - target = res->target; - lun = res->lun; - res->add_to_ml = 0; - spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); - scsi_add_device(ioa_cfg->host, bus, target, lun); - spin_lock_irqsave(ioa_cfg->host->host_lock, lock_flags); - goto restart; - } - } - - ioa_cfg->scan_done = 1; spin_unlock_irqrestore(ioa_cfg->host->host_lock, lock_flags); - kobject_uevent(&ioa_cfg->host->shost_dev.kobj, KOBJ_CHANGE); LEAVE; } @@ -9933,6 +9948,7 @@ static void ipr_init_ioa_cfg(struct ipr_ioa_cfg *ioa_cfg, INIT_LIST_HEAD(&ioa_cfg->free_res_q); INIT_LIST_HEAD(&ioa_cfg->used_res_q); INIT_WORK(&ioa_cfg->work_q, ipr_worker_thread); + INIT_WORK(&ioa_cfg->scsi_add_work_q, ipr_add_remove_thread); init_waitqueue_head(&ioa_cfg->reset_wait_q); init_waitqueue_head(&ioa_cfg->msi_wait_q); init_waitqueue_head(&ioa_cfg->eeh_wait_q); diff --git a/drivers/scsi/ipr.h b/drivers/scsi/ipr.h index 68afbbde54d3..f6baa2351313 100644 --- a/drivers/scsi/ipr.h +++ b/drivers/scsi/ipr.h @@ -1575,6 +1575,7 @@ struct ipr_ioa_cfg { u8 saved_mode_page_len; struct work_struct work_q; + struct work_struct scsi_add_work_q; struct workqueue_struct *reset_work_q; wait_queue_head_t reset_wait_q; diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index e0d0da5f43d6..43732e8d1347 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -672,7 +672,7 @@ struct lpfc_hba { #define LS_NPIV_FAB_SUPPORTED 0x2 /* Fabric supports NPIV */ #define LS_IGNORE_ERATT 0x4 /* intr handler should ignore ERATT */ #define LS_MDS_LINK_DOWN 0x8 /* MDS Diagnostics Link Down */ -#define LS_MDS_LOOPBACK 0x16 /* MDS Diagnostics Link Up (Loopback) */ +#define LS_MDS_LOOPBACK 0x10 /* MDS Diagnostics Link Up (Loopback) */ uint32_t hba_flag; /* hba generic flags */ #define HBA_ERATT_HANDLED 0x1 /* This flag is set when eratt handled */ diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index 5a25553415f8..1a6ed9b0a249 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -360,12 +360,12 @@ lpfc_nvme_info_show(struct device *dev, struct device_attribute *attr, goto buffer_done; list_for_each_entry(ndlp, &vport->fc_nodes, nlp_listp) { + nrport = NULL; + spin_lock(&vport->phba->hbalock); rport = lpfc_ndlp_get_nrport(ndlp); - if (!rport) - continue; - - /* local short-hand pointer. */ - nrport = rport->remoteport; + if (rport) + nrport = rport->remoteport; + spin_unlock(&vport->phba->hbalock); if (!nrport) continue; @@ -3386,6 +3386,7 @@ lpfc_update_rport_devloss_tmo(struct lpfc_vport *vport) struct lpfc_nodelist *ndlp; #if (IS_ENABLED(CONFIG_NVME_FC)) struct lpfc_nvme_rport *rport; + struct nvme_fc_remote_port *remoteport = NULL; #endif shost = lpfc_shost_from_vport(vport); @@ -3396,8 +3397,12 @@ lpfc_update_rport_devloss_tmo(struct lpfc_vport *vport) if (ndlp->rport) ndlp->rport->dev_loss_tmo = vport->cfg_devloss_tmo; #if (IS_ENABLED(CONFIG_NVME_FC)) + spin_lock(&vport->phba->hbalock); rport = lpfc_ndlp_get_nrport(ndlp); if (rport) + remoteport = rport->remoteport; + spin_unlock(&vport->phba->hbalock); + if (remoteport) nvme_fc_set_remoteport_devloss(rport->remoteport, vport->cfg_devloss_tmo); #endif @@ -5122,16 +5127,16 @@ LPFC_ATTR_R(enable_SmartSAN, 0, 0, 1, "Enable SmartSAN functionality"); /* # lpfc_fdmi_on: Controls FDMI support. -# 0 No FDMI support (default) -# 1 Traditional FDMI support +# 0 No FDMI support +# 1 Traditional FDMI support (default) # Traditional FDMI support means the driver will assume FDMI-2 support; # however, if that fails, it will fallback to FDMI-1. # If lpfc_enable_SmartSAN is set to 1, the driver ignores lpfc_fdmi_on. # If lpfc_enable_SmartSAN is set 0, the driver uses the current value of # lpfc_fdmi_on. -# Value range [0,1]. Default value is 0. +# Value range [0,1]. Default value is 1. */ -LPFC_ATTR_R(fdmi_on, 0, 0, 1, "Enable FDMI support"); +LPFC_ATTR_R(fdmi_on, 1, 0, 1, "Enable FDMI support"); /* # Specifies the maximum number of ELS cmds we can have outstanding (for diff --git a/drivers/scsi/lpfc/lpfc_debugfs.c b/drivers/scsi/lpfc/lpfc_debugfs.c index 9df0c051349f..aec5b10a8c85 100644 --- a/drivers/scsi/lpfc/lpfc_debugfs.c +++ b/drivers/scsi/lpfc/lpfc_debugfs.c @@ -551,7 +551,7 @@ lpfc_debugfs_nodelist_data(struct lpfc_vport *vport, char *buf, int size) unsigned char *statep; struct nvme_fc_local_port *localport; struct lpfc_nvmet_tgtport *tgtp; - struct nvme_fc_remote_port *nrport; + struct nvme_fc_remote_port *nrport = NULL; struct lpfc_nvme_rport *rport; cnt = (LPFC_NODELIST_SIZE / LPFC_NODELIST_ENTRY_SIZE); @@ -696,11 +696,11 @@ lpfc_debugfs_nodelist_data(struct lpfc_vport *vport, char *buf, int size) len += snprintf(buf + len, size - len, "\tRport List:\n"); list_for_each_entry(ndlp, &vport->fc_nodes, nlp_listp) { /* local short-hand pointer. */ + spin_lock(&phba->hbalock); rport = lpfc_ndlp_get_nrport(ndlp); - if (!rport) - continue; - - nrport = rport->remoteport; + if (rport) + nrport = rport->remoteport; + spin_unlock(&phba->hbalock); if (!nrport) continue; diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 028462e5994d..918ae18ef8a8 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -2725,7 +2725,9 @@ lpfc_nvme_register_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) rpinfo.port_name = wwn_to_u64(ndlp->nlp_portname.u.wwn); rpinfo.node_name = wwn_to_u64(ndlp->nlp_nodename.u.wwn); + spin_lock_irq(&vport->phba->hbalock); oldrport = lpfc_ndlp_get_nrport(ndlp); + spin_unlock_irq(&vport->phba->hbalock); if (!oldrport) lpfc_nlp_get(ndlp); @@ -2840,7 +2842,7 @@ lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) struct nvme_fc_local_port *localport; struct lpfc_nvme_lport *lport; struct lpfc_nvme_rport *rport; - struct nvme_fc_remote_port *remoteport; + struct nvme_fc_remote_port *remoteport = NULL; localport = vport->localport; @@ -2854,11 +2856,14 @@ lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) if (!lport) goto input_err; + spin_lock_irq(&vport->phba->hbalock); rport = lpfc_ndlp_get_nrport(ndlp); - if (!rport) + if (rport) + remoteport = rport->remoteport; + spin_unlock_irq(&vport->phba->hbalock); + if (!remoteport) goto input_err; - remoteport = rport->remoteport; lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME_DISC, "6033 Unreg nvme remoteport %p, portname x%llx, " "port_id x%06x, portstate x%x port type x%x\n", diff --git a/drivers/scsi/qedi/qedi.h b/drivers/scsi/qedi/qedi.h index fc3babc15fa3..a6f96b35e971 100644 --- a/drivers/scsi/qedi/qedi.h +++ b/drivers/scsi/qedi/qedi.h @@ -77,6 +77,11 @@ enum qedi_nvm_tgts { QEDI_NVM_TGT_SEC, }; +struct qedi_nvm_iscsi_image { + struct nvm_iscsi_cfg iscsi_cfg; + u32 crc; +}; + struct qedi_uio_ctrl { /* meta data */ u32 uio_hsi_version; @@ -294,7 +299,7 @@ struct qedi_ctx { void *bdq_pbl_list; dma_addr_t bdq_pbl_list_dma; u8 bdq_pbl_list_num_entries; - struct nvm_iscsi_cfg *iscsi_cfg; + struct qedi_nvm_iscsi_image *iscsi_image; dma_addr_t nvm_buf_dma; void __iomem *bdq_primary_prod; void __iomem *bdq_secondary_prod; diff --git a/drivers/scsi/qedi/qedi_main.c b/drivers/scsi/qedi/qedi_main.c index aa96bccb5a96..cc8e64dc65ad 100644 --- a/drivers/scsi/qedi/qedi_main.c +++ b/drivers/scsi/qedi/qedi_main.c @@ -1346,23 +1346,26 @@ exit_setup_int: static void qedi_free_nvm_iscsi_cfg(struct qedi_ctx *qedi) { - if (qedi->iscsi_cfg) + if (qedi->iscsi_image) dma_free_coherent(&qedi->pdev->dev, - sizeof(struct nvm_iscsi_cfg), - qedi->iscsi_cfg, qedi->nvm_buf_dma); + sizeof(struct qedi_nvm_iscsi_image), + qedi->iscsi_image, qedi->nvm_buf_dma); } static int qedi_alloc_nvm_iscsi_cfg(struct qedi_ctx *qedi) { - qedi->iscsi_cfg = dma_zalloc_coherent(&qedi->pdev->dev, - sizeof(struct nvm_iscsi_cfg), - &qedi->nvm_buf_dma, GFP_KERNEL); - if (!qedi->iscsi_cfg) { + struct qedi_nvm_iscsi_image nvm_image; + + qedi->iscsi_image = dma_zalloc_coherent(&qedi->pdev->dev, + sizeof(nvm_image), + &qedi->nvm_buf_dma, + GFP_KERNEL); + if (!qedi->iscsi_image) { QEDI_ERR(&qedi->dbg_ctx, "Could not allocate NVM BUF.\n"); return -ENOMEM; } QEDI_INFO(&qedi->dbg_ctx, QEDI_LOG_INFO, - "NVM BUF addr=0x%p dma=0x%llx.\n", qedi->iscsi_cfg, + "NVM BUF addr=0x%p dma=0x%llx.\n", qedi->iscsi_image, qedi->nvm_buf_dma); return 0; @@ -1905,7 +1908,7 @@ qedi_get_nvram_block(struct qedi_ctx *qedi) struct nvm_iscsi_block *block; pf = qedi->dev_info.common.abs_pf_id; - block = &qedi->iscsi_cfg->block[0]; + block = &qedi->iscsi_image->iscsi_cfg.block[0]; for (i = 0; i < NUM_OF_ISCSI_PF_SUPPORTED; i++, block++) { flags = ((block->id) & NVM_ISCSI_CFG_BLK_CTRL_FLAG_MASK) >> NVM_ISCSI_CFG_BLK_CTRL_FLAG_OFFSET; @@ -2194,15 +2197,14 @@ static void qedi_boot_release(void *data) static int qedi_get_boot_info(struct qedi_ctx *qedi) { int ret = 1; - u16 len; - - len = sizeof(struct nvm_iscsi_cfg); + struct qedi_nvm_iscsi_image nvm_image; QEDI_INFO(&qedi->dbg_ctx, QEDI_LOG_INFO, "Get NVM iSCSI CFG image\n"); ret = qedi_ops->common->nvm_get_image(qedi->cdev, QED_NVM_IMAGE_ISCSI_CFG, - (char *)qedi->iscsi_cfg, len); + (char *)qedi->iscsi_image, + sizeof(nvm_image)); if (ret) QEDI_ERR(&qedi->dbg_ctx, "Could not get NVM image. ret = %d\n", ret); diff --git a/drivers/scsi/qla2xxx/qla_target.h b/drivers/scsi/qla2xxx/qla_target.h index fecf96f0225c..199d3ba1916d 100644 --- a/drivers/scsi/qla2xxx/qla_target.h +++ b/drivers/scsi/qla2xxx/qla_target.h @@ -374,8 +374,8 @@ struct atio_from_isp { static inline int fcpcmd_is_corrupted(struct atio *atio) { if (atio->entry_type == ATIO_TYPE7 && - (le16_to_cpu(atio->attr_n_length & FCP_CMD_LENGTH_MASK) < - FCP_CMD_LENGTH_MIN)) + ((le16_to_cpu(atio->attr_n_length) & FCP_CMD_LENGTH_MASK) < + FCP_CMD_LENGTH_MIN)) return 1; else return 0; diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 0adfb3bce0fd..eb97d2dd3651 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -345,8 +345,7 @@ static void scsi_dec_host_busy(struct Scsi_Host *shost) unsigned long flags; rcu_read_lock(); - if (!shost->use_blk_mq) - atomic_dec(&shost->host_busy); + atomic_dec(&shost->host_busy); if (unlikely(scsi_host_in_recovery(shost))) { spin_lock_irqsave(shost->host_lock, flags); if (shost->host_failed || shost->host_eh_scheduled) @@ -445,12 +444,7 @@ static inline bool scsi_target_is_busy(struct scsi_target *starget) static inline bool scsi_host_is_busy(struct Scsi_Host *shost) { - /* - * blk-mq can handle host queue busy efficiently via host-wide driver - * tag allocation - */ - - if (!shost->use_blk_mq && shost->can_queue > 0 && + if (shost->can_queue > 0 && atomic_read(&shost->host_busy) >= shost->can_queue) return true; if (atomic_read(&shost->host_blocked) > 0) @@ -1606,10 +1600,7 @@ static inline int scsi_host_queue_ready(struct request_queue *q, if (scsi_host_in_recovery(shost)) return 0; - if (!shost->use_blk_mq) - busy = atomic_inc_return(&shost->host_busy) - 1; - else - busy = 0; + busy = atomic_inc_return(&shost->host_busy) - 1; if (atomic_read(&shost->host_blocked) > 0) { if (busy) goto starved; @@ -1625,7 +1616,7 @@ static inline int scsi_host_queue_ready(struct request_queue *q, "unblocking host at zero depth\n")); } - if (!shost->use_blk_mq && shost->can_queue > 0 && busy >= shost->can_queue) + if (shost->can_queue > 0 && busy >= shost->can_queue) goto starved; if (shost->host_self_blocked) goto starved; @@ -1711,9 +1702,7 @@ static void scsi_kill_request(struct request *req, struct request_queue *q) * with the locks as normal issue path does. */ atomic_inc(&sdev->device_busy); - - if (!shost->use_blk_mq) - atomic_inc(&shost->host_busy); + atomic_inc(&shost->host_busy); if (starget->can_queue > 0) atomic_inc(&starget->target_busy); diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c index b79b366a94f7..4a57ffecc7e6 100644 --- a/drivers/scsi/sd.c +++ b/drivers/scsi/sd.c @@ -1276,7 +1276,8 @@ static int sd_init_command(struct scsi_cmnd *cmd) case REQ_OP_ZONE_RESET: return sd_zbc_setup_reset_cmnd(cmd); default: - BUG(); + WARN_ON_ONCE(1); + return BLKPREP_KILL; } } @@ -2959,6 +2960,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp) if (rot == 1) { blk_queue_flag_set(QUEUE_FLAG_NONROT, q); blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q); + } else { + blk_queue_flag_clear(QUEUE_FLAG_NONROT, q); + blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q); } if (sdkp->device->type == TYPE_ZBC) { diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c index 9d5d2ca7fc4f..c55f38ec391c 100644 --- a/drivers/scsi/ufs/ufshcd.c +++ b/drivers/scsi/ufs/ufshcd.c @@ -7940,6 +7940,13 @@ int ufshcd_alloc_host(struct device *dev, struct ufs_hba **hba_handle) err = -ENOMEM; goto out_error; } + + /* + * Do not use blk-mq at this time because blk-mq does not support + * runtime pm. + */ + host->use_blk_mq = false; + hba = shost_priv(host); hba->host = host; hba->dev = dev; diff --git a/drivers/soundwire/stream.c b/drivers/soundwire/stream.c index 4b5e250e8615..e5c7e1ef6318 100644 --- a/drivers/soundwire/stream.c +++ b/drivers/soundwire/stream.c @@ -899,9 +899,10 @@ static void sdw_release_master_stream(struct sdw_stream_runtime *stream) struct sdw_master_runtime *m_rt = stream->m_rt; struct sdw_slave_runtime *s_rt, *_s_rt; - list_for_each_entry_safe(s_rt, _s_rt, - &m_rt->slave_rt_list, m_rt_node) - sdw_stream_remove_slave(s_rt->slave, stream); + list_for_each_entry_safe(s_rt, _s_rt, &m_rt->slave_rt_list, m_rt_node) { + sdw_slave_port_release(s_rt->slave->bus, s_rt->slave, stream); + sdw_release_slave_stream(s_rt->slave, stream); + } list_del(&m_rt->bus_node); } @@ -1112,7 +1113,7 @@ int sdw_stream_add_master(struct sdw_bus *bus, "Master runtime config failed for stream:%s", stream->name); ret = -ENOMEM; - goto error; + goto unlock; } ret = sdw_config_stream(bus->dev, stream, stream_config, false); @@ -1123,11 +1124,11 @@ int sdw_stream_add_master(struct sdw_bus *bus, if (ret) goto stream_error; - stream->state = SDW_STREAM_CONFIGURED; + goto unlock; stream_error: sdw_release_master_stream(stream); -error: +unlock: mutex_unlock(&bus->bus_lock); return ret; } @@ -1141,6 +1142,10 @@ EXPORT_SYMBOL(sdw_stream_add_master); * @stream: SoundWire stream * @port_config: Port configuration for audio stream * @num_ports: Number of ports + * + * It is expected that Slave is added before adding Master + * to the Stream. + * */ int sdw_stream_add_slave(struct sdw_slave *slave, struct sdw_stream_config *stream_config, @@ -1186,6 +1191,12 @@ int sdw_stream_add_slave(struct sdw_slave *slave, if (ret) goto stream_error; + /* + * Change stream state to CONFIGURED on first Slave add. + * Bus is not aware of number of Slave(s) in a stream at this + * point so cannot depend on all Slave(s) to be added in order to + * change stream state to CONFIGURED. + */ stream->state = SDW_STREAM_CONFIGURED; goto error; diff --git a/drivers/spi/spi-fsl-dspi.c b/drivers/spi/spi-fsl-dspi.c index 7cb3ab0a35a0..3082e72e4f6c 100644 --- a/drivers/spi/spi-fsl-dspi.c +++ b/drivers/spi/spi-fsl-dspi.c @@ -30,7 +30,11 @@ #define DRIVER_NAME "fsl-dspi" +#ifdef CONFIG_M5441x +#define DSPI_FIFO_SIZE 16 +#else #define DSPI_FIFO_SIZE 4 +#endif #define DSPI_DMA_BUFSIZE (DSPI_FIFO_SIZE * 1024) #define SPI_MCR 0x00 @@ -623,9 +627,11 @@ static void dspi_tcfq_read(struct fsl_dspi *dspi) static void dspi_eoq_write(struct fsl_dspi *dspi) { int fifo_size = DSPI_FIFO_SIZE; + u16 xfer_cmd = dspi->tx_cmd; /* Fill TX FIFO with as many transfers as possible */ while (dspi->len && fifo_size--) { + dspi->tx_cmd = xfer_cmd; /* Request EOQF for last transfer in FIFO */ if (dspi->len == dspi->bytes_per_word || fifo_size == 0) dspi->tx_cmd |= SPI_PUSHR_CMD_EOQ; diff --git a/drivers/spi/spi-gpio.c b/drivers/spi/spi-gpio.c index 0626e6e3ea0c..421bfc7dda67 100644 --- a/drivers/spi/spi-gpio.c +++ b/drivers/spi/spi-gpio.c @@ -300,8 +300,8 @@ static int spi_gpio_request(struct device *dev, *mflags |= SPI_MASTER_NO_RX; spi_gpio->sck = devm_gpiod_get(dev, "sck", GPIOD_OUT_LOW); - if (IS_ERR(spi_gpio->mosi)) - return PTR_ERR(spi_gpio->mosi); + if (IS_ERR(spi_gpio->sck)) + return PTR_ERR(spi_gpio->sck); for (i = 0; i < num_chipselects; i++) { spi_gpio->cs_gpios[i] = devm_gpiod_get_index(dev, "cs", diff --git a/drivers/spi/spi-rspi.c b/drivers/spi/spi-rspi.c index 95dc4d78618d..b37de1d991d6 100644 --- a/drivers/spi/spi-rspi.c +++ b/drivers/spi/spi-rspi.c @@ -598,11 +598,13 @@ static int rspi_dma_transfer(struct rspi_data *rspi, struct sg_table *tx, ret = wait_event_interruptible_timeout(rspi->wait, rspi->dma_callbacked, HZ); - if (ret > 0 && rspi->dma_callbacked) + if (ret > 0 && rspi->dma_callbacked) { ret = 0; - else if (!ret) { - dev_err(&rspi->master->dev, "DMA timeout\n"); - ret = -ETIMEDOUT; + } else { + if (!ret) { + dev_err(&rspi->master->dev, "DMA timeout\n"); + ret = -ETIMEDOUT; + } if (tx) dmaengine_terminate_all(rspi->master->dma_tx); if (rx) @@ -1350,12 +1352,36 @@ static const struct platform_device_id spi_driver_ids[] = { MODULE_DEVICE_TABLE(platform, spi_driver_ids); +#ifdef CONFIG_PM_SLEEP +static int rspi_suspend(struct device *dev) +{ + struct platform_device *pdev = to_platform_device(dev); + struct rspi_data *rspi = platform_get_drvdata(pdev); + + return spi_master_suspend(rspi->master); +} + +static int rspi_resume(struct device *dev) +{ + struct platform_device *pdev = to_platform_device(dev); + struct rspi_data *rspi = platform_get_drvdata(pdev); + + return spi_master_resume(rspi->master); +} + +static SIMPLE_DEV_PM_OPS(rspi_pm_ops, rspi_suspend, rspi_resume); +#define DEV_PM_OPS &rspi_pm_ops +#else +#define DEV_PM_OPS NULL +#endif /* CONFIG_PM_SLEEP */ + static struct platform_driver rspi_driver = { .probe = rspi_probe, .remove = rspi_remove, .id_table = spi_driver_ids, .driver = { .name = "renesas_spi", + .pm = DEV_PM_OPS, .of_match_table = of_match_ptr(rspi_of_match), }, }; diff --git a/drivers/spi/spi-sh-msiof.c b/drivers/spi/spi-sh-msiof.c index 539d6d1a277a..101cd6aae2ea 100644 --- a/drivers/spi/spi-sh-msiof.c +++ b/drivers/spi/spi-sh-msiof.c @@ -397,7 +397,8 @@ static void sh_msiof_spi_set_mode_regs(struct sh_msiof_spi_priv *p, static void sh_msiof_reset_str(struct sh_msiof_spi_priv *p) { - sh_msiof_write(p, STR, sh_msiof_read(p, STR)); + sh_msiof_write(p, STR, + sh_msiof_read(p, STR) & ~(STR_TDREQ | STR_RDREQ)); } static void sh_msiof_spi_write_fifo_8(struct sh_msiof_spi_priv *p, @@ -1426,12 +1427,37 @@ static const struct platform_device_id spi_driver_ids[] = { }; MODULE_DEVICE_TABLE(platform, spi_driver_ids); +#ifdef CONFIG_PM_SLEEP +static int sh_msiof_spi_suspend(struct device *dev) +{ + struct platform_device *pdev = to_platform_device(dev); + struct sh_msiof_spi_priv *p = platform_get_drvdata(pdev); + + return spi_master_suspend(p->master); +} + +static int sh_msiof_spi_resume(struct device *dev) +{ + struct platform_device *pdev = to_platform_device(dev); + struct sh_msiof_spi_priv *p = platform_get_drvdata(pdev); + + return spi_master_resume(p->master); +} + +static SIMPLE_DEV_PM_OPS(sh_msiof_spi_pm_ops, sh_msiof_spi_suspend, + sh_msiof_spi_resume); +#define DEV_PM_OPS &sh_msiof_spi_pm_ops +#else +#define DEV_PM_OPS NULL +#endif /* CONFIG_PM_SLEEP */ + static struct platform_driver sh_msiof_spi_drv = { .probe = sh_msiof_spi_probe, .remove = sh_msiof_spi_remove, .id_table = spi_driver_ids, .driver = { .name = "spi_sh_msiof", + .pm = DEV_PM_OPS, .of_match_table = of_match_ptr(sh_msiof_match), }, }; diff --git a/drivers/spi/spi-tegra20-slink.c b/drivers/spi/spi-tegra20-slink.c index 6f7b946b5ced..1427f343b39a 100644 --- a/drivers/spi/spi-tegra20-slink.c +++ b/drivers/spi/spi-tegra20-slink.c @@ -1063,6 +1063,24 @@ static int tegra_slink_probe(struct platform_device *pdev) goto exit_free_master; } + /* disabled clock may cause interrupt storm upon request */ + tspi->clk = devm_clk_get(&pdev->dev, NULL); + if (IS_ERR(tspi->clk)) { + ret = PTR_ERR(tspi->clk); + dev_err(&pdev->dev, "Can not get clock %d\n", ret); + goto exit_free_master; + } + ret = clk_prepare(tspi->clk); + if (ret < 0) { + dev_err(&pdev->dev, "Clock prepare failed %d\n", ret); + goto exit_free_master; + } + ret = clk_enable(tspi->clk); + if (ret < 0) { + dev_err(&pdev->dev, "Clock enable failed %d\n", ret); + goto exit_free_master; + } + spi_irq = platform_get_irq(pdev, 0); tspi->irq = spi_irq; ret = request_threaded_irq(tspi->irq, tegra_slink_isr, @@ -1071,14 +1089,7 @@ static int tegra_slink_probe(struct platform_device *pdev) if (ret < 0) { dev_err(&pdev->dev, "Failed to register ISR for IRQ %d\n", tspi->irq); - goto exit_free_master; - } - - tspi->clk = devm_clk_get(&pdev->dev, NULL); - if (IS_ERR(tspi->clk)) { - dev_err(&pdev->dev, "can not get clock\n"); - ret = PTR_ERR(tspi->clk); - goto exit_free_irq; + goto exit_clk_disable; } tspi->rst = devm_reset_control_get_exclusive(&pdev->dev, "spi"); @@ -1138,6 +1149,8 @@ exit_rx_dma_free: tegra_slink_deinit_dma_param(tspi, true); exit_free_irq: free_irq(spi_irq, tspi); +exit_clk_disable: + clk_disable(tspi->clk); exit_free_master: spi_master_put(master); return ret; @@ -1150,6 +1163,8 @@ static int tegra_slink_remove(struct platform_device *pdev) free_irq(tspi->irq, tspi); + clk_disable(tspi->clk); + if (tspi->tx_dma_chan) tegra_slink_deinit_dma_param(tspi, false); diff --git a/drivers/spi/spi.c b/drivers/spi/spi.c index ec395a6baf9c..9da0bc5a036c 100644 --- a/drivers/spi/spi.c +++ b/drivers/spi/spi.c @@ -2143,8 +2143,17 @@ int spi_register_controller(struct spi_controller *ctlr) */ if (ctlr->num_chipselect == 0) return -EINVAL; - /* allocate dynamic bus number using Linux idr */ - if ((ctlr->bus_num < 0) && ctlr->dev.of_node) { + if (ctlr->bus_num >= 0) { + /* devices with a fixed bus num must check-in with the num */ + mutex_lock(&board_lock); + id = idr_alloc(&spi_master_idr, ctlr, ctlr->bus_num, + ctlr->bus_num + 1, GFP_KERNEL); + mutex_unlock(&board_lock); + if (WARN(id < 0, "couldn't get idr")) + return id == -ENOSPC ? -EBUSY : id; + ctlr->bus_num = id; + } else if (ctlr->dev.of_node) { + /* allocate dynamic bus number using Linux idr */ id = of_alias_get_id(ctlr->dev.of_node, "spi"); if (id >= 0) { ctlr->bus_num = id; diff --git a/drivers/staging/erofs/Kconfig b/drivers/staging/erofs/Kconfig index 96f614934df1..663b755bf2fb 100644 --- a/drivers/staging/erofs/Kconfig +++ b/drivers/staging/erofs/Kconfig @@ -2,7 +2,7 @@ config EROFS_FS tristate "EROFS filesystem support" - depends on BROKEN + depends on BLOCK help EROFS(Enhanced Read-Only File System) is a lightweight read-only file system with modern designs (eg. page-sized diff --git a/drivers/staging/erofs/super.c b/drivers/staging/erofs/super.c index 1aec509c805f..2df9768edac9 100644 --- a/drivers/staging/erofs/super.c +++ b/drivers/staging/erofs/super.c @@ -340,7 +340,7 @@ static int erofs_read_super(struct super_block *sb, goto err_sbread; sb->s_magic = EROFS_SUPER_MAGIC; - sb->s_flags |= MS_RDONLY | MS_NOATIME; + sb->s_flags |= SB_RDONLY | SB_NOATIME; sb->s_maxbytes = MAX_LFS_FILESIZE; sb->s_time_gran = 1; @@ -627,7 +627,7 @@ static int erofs_remount(struct super_block *sb, int *flags, char *data) { BUG_ON(!sb_rdonly(sb)); - *flags |= MS_RDONLY; + *flags |= SB_RDONLY; return 0; } diff --git a/drivers/staging/fbtft/TODO b/drivers/staging/fbtft/TODO index 7e64c7e438f0..a9f4802bb6be 100644 --- a/drivers/staging/fbtft/TODO +++ b/drivers/staging/fbtft/TODO @@ -2,3 +2,7 @@ GPIO descriptor API in <linux/gpio/consumer.h> and look up GPIO lines from device tree, ACPI or board files, board files should use <linux/gpio/machine.h> + +* convert all these over to drm_simple_display_pipe and submit for inclusion + into the DRM subsystem under drivers/gpu/drm - fbdev doesn't take any new + drivers anymore. diff --git a/drivers/staging/gasket/TODO b/drivers/staging/gasket/TODO index 6ff8e01b04cc..5b1865f8af2d 100644 --- a/drivers/staging/gasket/TODO +++ b/drivers/staging/gasket/TODO @@ -1,9 +1,22 @@ This is a list of things that need to be done to get this driver out of the staging directory. + +- Implement the gasket framework's functionality through UIO instead of + introducing a new user-space drivers framework that is quite similar. + + UIO provides the necessary bits to implement user-space drivers. Meanwhile + the gasket APIs adds some extra conveniences like PCI BAR mapping, and + MSI interrupts. Add these features to the UIO subsystem, then re-implement + the Apex driver as a basic UIO driver instead (include/linux/uio_driver.h) + - Document sysfs files with Documentation/ABI/ entries. + - Use misc interface instead of major number for driver version description. + - Add descriptions of module_param's + - apex_get_status() should actually check status. + - "drivers" should never be dealing with "raw" sysfs calls or mess around with kobjects at all. The driver core should handle all of this for you automaically. There should not be a need for raw attribute macros. diff --git a/drivers/staging/media/mt9t031/Kconfig b/drivers/staging/media/mt9t031/Kconfig index f48e06a03cdb..9a58aaf72edd 100644 --- a/drivers/staging/media/mt9t031/Kconfig +++ b/drivers/staging/media/mt9t031/Kconfig @@ -1,9 +1,3 @@ -config SOC_CAMERA_IMX074 - tristate "imx074 support (DEPRECATED)" - depends on SOC_CAMERA && I2C - help - This driver supports IMX074 cameras from Sony - config SOC_CAMERA_MT9T031 tristate "mt9t031 support (DEPRECATED)" depends on SOC_CAMERA && I2C diff --git a/drivers/staging/vboxvideo/vbox_drv.c b/drivers/staging/vboxvideo/vbox_drv.c index da92c493f157..69cc508af1bc 100644 --- a/drivers/staging/vboxvideo/vbox_drv.c +++ b/drivers/staging/vboxvideo/vbox_drv.c @@ -59,6 +59,11 @@ static int vbox_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) ret = PTR_ERR(dev); goto err_drv_alloc; } + + ret = pci_enable_device(pdev); + if (ret) + goto err_pci_enable; + dev->pdev = pdev; pci_set_drvdata(pdev, dev); @@ -75,6 +80,8 @@ static int vbox_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) err_drv_dev_register: vbox_driver_unload(dev); err_vbox_driver_load: + pci_disable_device(pdev); + err_pci_enable: drm_dev_put(dev); err_drv_alloc: return ret; diff --git a/drivers/staging/vboxvideo/vbox_mode.c b/drivers/staging/vboxvideo/vbox_mode.c index a83eac8668d0..79836c8fb909 100644 --- a/drivers/staging/vboxvideo/vbox_mode.c +++ b/drivers/staging/vboxvideo/vbox_mode.c @@ -323,6 +323,11 @@ static int vbox_crtc_page_flip(struct drm_crtc *crtc, if (rc) return rc; + mutex_lock(&vbox->hw_mutex); + vbox_set_view(crtc); + vbox_do_modeset(crtc, &crtc->mode); + mutex_unlock(&vbox->hw_mutex); + spin_lock_irqsave(&drm->event_lock, flags); if (event) diff --git a/drivers/staging/wilc1000/Makefile b/drivers/staging/wilc1000/Makefile index f7b07c0b5ce2..ee7e26b886a5 100644 --- a/drivers/staging/wilc1000/Makefile +++ b/drivers/staging/wilc1000/Makefile @@ -1,4 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_WILC1000) += wilc1000.o ccflags-y += -DFIRMWARE_1002=\"atmel/wilc1002_firmware.bin\" \ -DFIRMWARE_1003=\"atmel/wilc1003_firmware.bin\" @@ -11,9 +12,7 @@ wilc1000-objs := wilc_wfi_cfgoperations.o linux_wlan.o linux_mon.o \ wilc_wlan.o obj-$(CONFIG_WILC1000_SDIO) += wilc1000-sdio.o -wilc1000-sdio-objs += $(wilc1000-objs) wilc1000-sdio-objs += wilc_sdio.o obj-$(CONFIG_WILC1000_SPI) += wilc1000-spi.o -wilc1000-spi-objs += $(wilc1000-objs) wilc1000-spi-objs += wilc_spi.o diff --git a/drivers/staging/wilc1000/linux_wlan.c b/drivers/staging/wilc1000/linux_wlan.c index 01cf4bd2e192..3b8d237decbf 100644 --- a/drivers/staging/wilc1000/linux_wlan.c +++ b/drivers/staging/wilc1000/linux_wlan.c @@ -1038,8 +1038,8 @@ void wilc_netdev_cleanup(struct wilc *wilc) } kfree(wilc); - wilc_debugfs_remove(); } +EXPORT_SYMBOL_GPL(wilc_netdev_cleanup); static const struct net_device_ops wilc_netdev_ops = { .ndo_init = mac_init_fn, @@ -1062,7 +1062,6 @@ int wilc_netdev_init(struct wilc **wilc, struct device *dev, int io_type, if (!wl) return -ENOMEM; - wilc_debugfs_init(); *wilc = wl; wl->io_type = io_type; wl->hif_func = ops; @@ -1124,3 +1123,6 @@ int wilc_netdev_init(struct wilc **wilc, struct device *dev, int io_type, return 0; } +EXPORT_SYMBOL_GPL(wilc_netdev_init); + +MODULE_LICENSE("GPL"); diff --git a/drivers/staging/wilc1000/wilc_debugfs.c b/drivers/staging/wilc1000/wilc_debugfs.c index edc72876458d..8001df66b8c2 100644 --- a/drivers/staging/wilc1000/wilc_debugfs.c +++ b/drivers/staging/wilc1000/wilc_debugfs.c @@ -19,6 +19,7 @@ static struct dentry *wilc_dir; #define DBG_LEVEL_ALL (DEBUG | INFO | WRN | ERR) static atomic_t WILC_DEBUG_LEVEL = ATOMIC_INIT(ERR); +EXPORT_SYMBOL_GPL(WILC_DEBUG_LEVEL); static ssize_t wilc_debug_level_read(struct file *file, char __user *userbuf, size_t count, loff_t *ppos) @@ -87,7 +88,7 @@ static struct wilc_debugfs_info_t debugfs_info[] = { }, }; -int wilc_debugfs_init(void) +static int __init wilc_debugfs_init(void) { int i; struct wilc_debugfs_info_t *info; @@ -103,10 +104,12 @@ int wilc_debugfs_init(void) } return 0; } +module_init(wilc_debugfs_init); -void wilc_debugfs_remove(void) +static void __exit wilc_debugfs_remove(void) { debugfs_remove_recursive(wilc_dir); } +module_exit(wilc_debugfs_remove); #endif diff --git a/drivers/staging/wilc1000/wilc_wlan.c b/drivers/staging/wilc1000/wilc_wlan.c index 6787b6e9f124..8b184aa30d25 100644 --- a/drivers/staging/wilc1000/wilc_wlan.c +++ b/drivers/staging/wilc1000/wilc_wlan.c @@ -417,6 +417,7 @@ void chip_allow_sleep(struct wilc *wilc) wilc->hif_func->hif_write_reg(wilc, 0xf0, reg & ~BIT(0)); wilc->hif_func->hif_write_reg(wilc, 0xfa, 0); } +EXPORT_SYMBOL_GPL(chip_allow_sleep); void chip_wakeup(struct wilc *wilc) { @@ -471,6 +472,7 @@ void chip_wakeup(struct wilc *wilc) } chip_ps_state = CHIP_WAKEDUP; } +EXPORT_SYMBOL_GPL(chip_wakeup); void wilc_chip_sleep_manually(struct wilc *wilc) { @@ -484,6 +486,7 @@ void wilc_chip_sleep_manually(struct wilc *wilc) chip_ps_state = CHIP_SLEEPING_MANUAL; release_bus(wilc, RELEASE_ONLY); } +EXPORT_SYMBOL_GPL(wilc_chip_sleep_manually); void host_wakeup_notify(struct wilc *wilc) { @@ -491,6 +494,7 @@ void host_wakeup_notify(struct wilc *wilc) wilc->hif_func->hif_write_reg(wilc, 0x10b0, 1); release_bus(wilc, RELEASE_ONLY); } +EXPORT_SYMBOL_GPL(host_wakeup_notify); void host_sleep_notify(struct wilc *wilc) { @@ -498,6 +502,7 @@ void host_sleep_notify(struct wilc *wilc) wilc->hif_func->hif_write_reg(wilc, 0x10ac, 1); release_bus(wilc, RELEASE_ONLY); } +EXPORT_SYMBOL_GPL(host_sleep_notify); int wilc_wlan_handle_txq(struct net_device *dev, u32 *txq_count) { @@ -871,6 +876,7 @@ void wilc_handle_isr(struct wilc *wilc) release_bus(wilc, RELEASE_ALLOW_SLEEP); } +EXPORT_SYMBOL_GPL(wilc_handle_isr); int wilc_wlan_firmware_download(struct wilc *wilc, const u8 *buffer, u32 buffer_size) diff --git a/drivers/staging/wilc1000/wilc_wlan_if.h b/drivers/staging/wilc1000/wilc_wlan_if.h index 00d13b153f80..b81a73b9bd67 100644 --- a/drivers/staging/wilc1000/wilc_wlan_if.h +++ b/drivers/staging/wilc1000/wilc_wlan_if.h @@ -831,6 +831,4 @@ struct wilc; int wilc_wlan_init(struct net_device *dev); u32 wilc_get_chipid(struct wilc *wilc, bool update); -int wilc_debugfs_init(void); -void wilc_debugfs_remove(void); #endif diff --git a/drivers/target/iscsi/cxgbit/cxgbit_ddp.c b/drivers/target/iscsi/cxgbit/cxgbit_ddp.c index 768cce0ccb80..76a262674c8d 100644 --- a/drivers/target/iscsi/cxgbit/cxgbit_ddp.c +++ b/drivers/target/iscsi/cxgbit/cxgbit_ddp.c @@ -207,8 +207,8 @@ cxgbit_ddp_reserve(struct cxgbit_sock *csk, struct cxgbi_task_tag_info *ttinfo, ret = dma_map_sg(&ppm->pdev->dev, sgl, sgcnt, DMA_FROM_DEVICE); sgl->offset = sg_offset; if (!ret) { - pr_info("%s: 0x%x, xfer %u, sgl %u dma mapping err.\n", - __func__, 0, xferlen, sgcnt); + pr_debug("%s: 0x%x, xfer %u, sgl %u dma mapping err.\n", + __func__, 0, xferlen, sgcnt); goto rel_ppods; } @@ -250,8 +250,8 @@ cxgbit_get_r2t_ttt(struct iscsi_conn *conn, struct iscsi_cmd *cmd, ret = cxgbit_ddp_reserve(csk, ttinfo, cmd->se_cmd.data_length); if (ret < 0) { - pr_info("csk 0x%p, cmd 0x%p, xfer len %u, sgcnt %u no ddp.\n", - csk, cmd, cmd->se_cmd.data_length, ttinfo->nents); + pr_debug("csk 0x%p, cmd 0x%p, xfer len %u, sgcnt %u no ddp.\n", + csk, cmd, cmd->se_cmd.data_length, ttinfo->nents); ttinfo->sgl = NULL; ttinfo->nents = 0; diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c index 94bad43c41ff..cc756a123fd8 100644 --- a/drivers/target/iscsi/iscsi_target.c +++ b/drivers/target/iscsi/iscsi_target.c @@ -1416,7 +1416,8 @@ static void iscsit_do_crypto_hash_buf(struct ahash_request *hash, sg_init_table(sg, ARRAY_SIZE(sg)); sg_set_buf(sg, buf, payload_length); - sg_set_buf(sg + 1, pad_bytes, padding); + if (padding) + sg_set_buf(sg + 1, pad_bytes, padding); ahash_request_set_crypt(hash, sg, data_crc, payload_length + padding); @@ -3910,10 +3911,14 @@ static bool iscsi_target_check_conn_state(struct iscsi_conn *conn) static void iscsit_get_rx_pdu(struct iscsi_conn *conn) { int ret; - u8 buffer[ISCSI_HDR_LEN], opcode; + u8 *buffer, opcode; u32 checksum = 0, digest = 0; struct kvec iov; + buffer = kcalloc(ISCSI_HDR_LEN, sizeof(*buffer), GFP_KERNEL); + if (!buffer) + return; + while (!kthread_should_stop()) { /* * Ensure that both TX and RX per connection kthreads @@ -3921,7 +3926,6 @@ static void iscsit_get_rx_pdu(struct iscsi_conn *conn) */ iscsit_thread_check_cpumask(conn, current, 0); - memset(buffer, 0, ISCSI_HDR_LEN); memset(&iov, 0, sizeof(struct kvec)); iov.iov_base = buffer; @@ -3930,7 +3934,7 @@ static void iscsit_get_rx_pdu(struct iscsi_conn *conn) ret = rx_data(conn, &iov, 1, ISCSI_HDR_LEN); if (ret != ISCSI_HDR_LEN) { iscsit_rx_thread_wait_for_tcp(conn); - return; + break; } if (conn->conn_ops->HeaderDigest) { @@ -3940,7 +3944,7 @@ static void iscsit_get_rx_pdu(struct iscsi_conn *conn) ret = rx_data(conn, &iov, 1, ISCSI_CRC_LEN); if (ret != ISCSI_CRC_LEN) { iscsit_rx_thread_wait_for_tcp(conn); - return; + break; } iscsit_do_crypto_hash_buf(conn->conn_rx_hash, buffer, @@ -3964,7 +3968,7 @@ static void iscsit_get_rx_pdu(struct iscsi_conn *conn) } if (conn->conn_state == TARG_CONN_STATE_IN_LOGOUT) - return; + break; opcode = buffer[0] & ISCSI_OPCODE_MASK; @@ -3975,13 +3979,15 @@ static void iscsit_get_rx_pdu(struct iscsi_conn *conn) " while in Discovery Session, rejecting.\n", opcode); iscsit_add_reject(conn, ISCSI_REASON_PROTOCOL_ERROR, buffer); - return; + break; } ret = iscsi_target_rx_opcode(conn, buffer); if (ret < 0) - return; + break; } + + kfree(buffer); } int iscsi_target_rx_thread(void *arg) @@ -4208,22 +4214,15 @@ int iscsit_close_connection( crypto_free_ahash(tfm); } - free_cpumask_var(conn->conn_cpumask); - - kfree(conn->conn_ops); - conn->conn_ops = NULL; - if (conn->sock) sock_release(conn->sock); if (conn->conn_transport->iscsit_free_conn) conn->conn_transport->iscsit_free_conn(conn); - iscsit_put_transport(conn->conn_transport); - pr_debug("Moving to TARG_CONN_STATE_FREE.\n"); conn->conn_state = TARG_CONN_STATE_FREE; - kfree(conn); + iscsit_free_conn(conn); spin_lock_bh(&sess->conn_lock); atomic_dec(&sess->nconn); diff --git a/drivers/target/iscsi/iscsi_target_auth.c b/drivers/target/iscsi/iscsi_target_auth.c index 9518ffd8b8ba..4e680d753941 100644 --- a/drivers/target/iscsi/iscsi_target_auth.c +++ b/drivers/target/iscsi/iscsi_target_auth.c @@ -26,27 +26,6 @@ #include "iscsi_target_nego.h" #include "iscsi_target_auth.h" -static int chap_string_to_hex(unsigned char *dst, unsigned char *src, int len) -{ - int j = DIV_ROUND_UP(len, 2), rc; - - rc = hex2bin(dst, src, j); - if (rc < 0) - pr_debug("CHAP string contains non hex digit symbols\n"); - - dst[j] = '\0'; - return j; -} - -static void chap_binaryhex_to_asciihex(char *dst, char *src, int src_len) -{ - int i; - - for (i = 0; i < src_len; i++) { - sprintf(&dst[i*2], "%02x", (int) src[i] & 0xff); - } -} - static int chap_gen_challenge( struct iscsi_conn *conn, int caller, @@ -62,7 +41,7 @@ static int chap_gen_challenge( ret = get_random_bytes_wait(chap->challenge, CHAP_CHALLENGE_LENGTH); if (unlikely(ret)) return ret; - chap_binaryhex_to_asciihex(challenge_asciihex, chap->challenge, + bin2hex(challenge_asciihex, chap->challenge, CHAP_CHALLENGE_LENGTH); /* * Set CHAP_C, and copy the generated challenge into c_str. @@ -248,9 +227,16 @@ static int chap_server_compute_md5( pr_err("Could not find CHAP_R.\n"); goto out; } + if (strlen(chap_r) != MD5_SIGNATURE_SIZE * 2) { + pr_err("Malformed CHAP_R\n"); + goto out; + } + if (hex2bin(client_digest, chap_r, MD5_SIGNATURE_SIZE) < 0) { + pr_err("Malformed CHAP_R\n"); + goto out; + } pr_debug("[server] Got CHAP_R=%s\n", chap_r); - chap_string_to_hex(client_digest, chap_r, strlen(chap_r)); tfm = crypto_alloc_shash("md5", 0, 0); if (IS_ERR(tfm)) { @@ -294,7 +280,7 @@ static int chap_server_compute_md5( goto out; } - chap_binaryhex_to_asciihex(response, server_digest, MD5_SIGNATURE_SIZE); + bin2hex(response, server_digest, MD5_SIGNATURE_SIZE); pr_debug("[server] MD5 Server Digest: %s\n", response); if (memcmp(server_digest, client_digest, MD5_SIGNATURE_SIZE) != 0) { @@ -349,9 +335,7 @@ static int chap_server_compute_md5( pr_err("Could not find CHAP_C.\n"); goto out; } - pr_debug("[server] Got CHAP_C=%s\n", challenge); - challenge_len = chap_string_to_hex(challenge_binhex, challenge, - strlen(challenge)); + challenge_len = DIV_ROUND_UP(strlen(challenge), 2); if (!challenge_len) { pr_err("Unable to convert incoming challenge\n"); goto out; @@ -360,6 +344,11 @@ static int chap_server_compute_md5( pr_err("CHAP_C exceeds maximum binary size of 1024 bytes\n"); goto out; } + if (hex2bin(challenge_binhex, challenge, challenge_len) < 0) { + pr_err("Malformed CHAP_C\n"); + goto out; + } + pr_debug("[server] Got CHAP_C=%s\n", challenge); /* * During mutual authentication, the CHAP_C generated by the * initiator must not match the original CHAP_C generated by @@ -413,7 +402,7 @@ static int chap_server_compute_md5( /* * Convert response from binary hex to ascii hext. */ - chap_binaryhex_to_asciihex(response, digest, MD5_SIGNATURE_SIZE); + bin2hex(response, digest, MD5_SIGNATURE_SIZE); *nr_out_len += sprintf(nr_out_ptr + *nr_out_len, "CHAP_R=0x%s", response); *nr_out_len += 1; diff --git a/drivers/target/iscsi/iscsi_target_login.c b/drivers/target/iscsi/iscsi_target_login.c index 9e74f8bc2963..bb90c80ff388 100644 --- a/drivers/target/iscsi/iscsi_target_login.c +++ b/drivers/target/iscsi/iscsi_target_login.c @@ -67,45 +67,10 @@ static struct iscsi_login *iscsi_login_init_conn(struct iscsi_conn *conn) goto out_req_buf; } - conn->conn_ops = kzalloc(sizeof(struct iscsi_conn_ops), GFP_KERNEL); - if (!conn->conn_ops) { - pr_err("Unable to allocate memory for" - " struct iscsi_conn_ops.\n"); - goto out_rsp_buf; - } - - init_waitqueue_head(&conn->queues_wq); - INIT_LIST_HEAD(&conn->conn_list); - INIT_LIST_HEAD(&conn->conn_cmd_list); - INIT_LIST_HEAD(&conn->immed_queue_list); - INIT_LIST_HEAD(&conn->response_queue_list); - init_completion(&conn->conn_post_wait_comp); - init_completion(&conn->conn_wait_comp); - init_completion(&conn->conn_wait_rcfr_comp); - init_completion(&conn->conn_waiting_on_uc_comp); - init_completion(&conn->conn_logout_comp); - init_completion(&conn->rx_half_close_comp); - init_completion(&conn->tx_half_close_comp); - init_completion(&conn->rx_login_comp); - spin_lock_init(&conn->cmd_lock); - spin_lock_init(&conn->conn_usage_lock); - spin_lock_init(&conn->immed_queue_lock); - spin_lock_init(&conn->nopin_timer_lock); - spin_lock_init(&conn->response_queue_lock); - spin_lock_init(&conn->state_lock); - - if (!zalloc_cpumask_var(&conn->conn_cpumask, GFP_KERNEL)) { - pr_err("Unable to allocate conn->conn_cpumask\n"); - goto out_conn_ops; - } conn->conn_login = login; return login; -out_conn_ops: - kfree(conn->conn_ops); -out_rsp_buf: - kfree(login->rsp_buf); out_req_buf: kfree(login->req_buf); out_login: @@ -310,11 +275,9 @@ static int iscsi_login_zero_tsih_s1( return -ENOMEM; } - ret = iscsi_login_set_conn_values(sess, conn, pdu->cid); - if (unlikely(ret)) { - kfree(sess); - return ret; - } + if (iscsi_login_set_conn_values(sess, conn, pdu->cid)) + goto free_sess; + sess->init_task_tag = pdu->itt; memcpy(&sess->isid, pdu->isid, 6); sess->exp_cmd_sn = be32_to_cpu(pdu->cmdsn); @@ -1149,6 +1112,75 @@ iscsit_conn_set_transport(struct iscsi_conn *conn, struct iscsit_transport *t) return 0; } +static struct iscsi_conn *iscsit_alloc_conn(struct iscsi_np *np) +{ + struct iscsi_conn *conn; + + conn = kzalloc(sizeof(struct iscsi_conn), GFP_KERNEL); + if (!conn) { + pr_err("Could not allocate memory for new connection\n"); + return NULL; + } + pr_debug("Moving to TARG_CONN_STATE_FREE.\n"); + conn->conn_state = TARG_CONN_STATE_FREE; + + init_waitqueue_head(&conn->queues_wq); + INIT_LIST_HEAD(&conn->conn_list); + INIT_LIST_HEAD(&conn->conn_cmd_list); + INIT_LIST_HEAD(&conn->immed_queue_list); + INIT_LIST_HEAD(&conn->response_queue_list); + init_completion(&conn->conn_post_wait_comp); + init_completion(&conn->conn_wait_comp); + init_completion(&conn->conn_wait_rcfr_comp); + init_completion(&conn->conn_waiting_on_uc_comp); + init_completion(&conn->conn_logout_comp); + init_completion(&conn->rx_half_close_comp); + init_completion(&conn->tx_half_close_comp); + init_completion(&conn->rx_login_comp); + spin_lock_init(&conn->cmd_lock); + spin_lock_init(&conn->conn_usage_lock); + spin_lock_init(&conn->immed_queue_lock); + spin_lock_init(&conn->nopin_timer_lock); + spin_lock_init(&conn->response_queue_lock); + spin_lock_init(&conn->state_lock); + + timer_setup(&conn->nopin_response_timer, + iscsit_handle_nopin_response_timeout, 0); + timer_setup(&conn->nopin_timer, iscsit_handle_nopin_timeout, 0); + + if (iscsit_conn_set_transport(conn, np->np_transport) < 0) + goto free_conn; + + conn->conn_ops = kzalloc(sizeof(struct iscsi_conn_ops), GFP_KERNEL); + if (!conn->conn_ops) { + pr_err("Unable to allocate memory for struct iscsi_conn_ops.\n"); + goto put_transport; + } + + if (!zalloc_cpumask_var(&conn->conn_cpumask, GFP_KERNEL)) { + pr_err("Unable to allocate conn->conn_cpumask\n"); + goto free_mask; + } + + return conn; + +free_mask: + free_cpumask_var(conn->conn_cpumask); +put_transport: + iscsit_put_transport(conn->conn_transport); +free_conn: + kfree(conn); + return NULL; +} + +void iscsit_free_conn(struct iscsi_conn *conn) +{ + free_cpumask_var(conn->conn_cpumask); + kfree(conn->conn_ops); + iscsit_put_transport(conn->conn_transport); + kfree(conn); +} + void iscsi_target_login_sess_out(struct iscsi_conn *conn, struct iscsi_np *np, bool zero_tsih, bool new_sess) { @@ -1198,10 +1230,6 @@ old_sess_out: crypto_free_ahash(tfm); } - free_cpumask_var(conn->conn_cpumask); - - kfree(conn->conn_ops); - if (conn->param_list) { iscsi_release_param_list(conn->param_list); conn->param_list = NULL; @@ -1219,8 +1247,7 @@ old_sess_out: if (conn->conn_transport->iscsit_free_conn) conn->conn_transport->iscsit_free_conn(conn); - iscsit_put_transport(conn->conn_transport); - kfree(conn); + iscsit_free_conn(conn); } static int __iscsi_target_login_thread(struct iscsi_np *np) @@ -1250,31 +1277,16 @@ static int __iscsi_target_login_thread(struct iscsi_np *np) } spin_unlock_bh(&np->np_thread_lock); - conn = kzalloc(sizeof(struct iscsi_conn), GFP_KERNEL); + conn = iscsit_alloc_conn(np); if (!conn) { - pr_err("Could not allocate memory for" - " new connection\n"); /* Get another socket */ return 1; } - pr_debug("Moving to TARG_CONN_STATE_FREE.\n"); - conn->conn_state = TARG_CONN_STATE_FREE; - - timer_setup(&conn->nopin_response_timer, - iscsit_handle_nopin_response_timeout, 0); - timer_setup(&conn->nopin_timer, iscsit_handle_nopin_timeout, 0); - - if (iscsit_conn_set_transport(conn, np->np_transport) < 0) { - kfree(conn); - return 1; - } rc = np->np_transport->iscsit_accept_np(np, conn); if (rc == -ENOSYS) { complete(&np->np_restart_comp); - iscsit_put_transport(conn->conn_transport); - kfree(conn); - conn = NULL; + iscsit_free_conn(conn); goto exit; } else if (rc < 0) { spin_lock_bh(&np->np_thread_lock); @@ -1282,17 +1294,13 @@ static int __iscsi_target_login_thread(struct iscsi_np *np) np->np_thread_state = ISCSI_NP_THREAD_ACTIVE; spin_unlock_bh(&np->np_thread_lock); complete(&np->np_restart_comp); - iscsit_put_transport(conn->conn_transport); - kfree(conn); - conn = NULL; + iscsit_free_conn(conn); /* Get another socket */ return 1; } spin_unlock_bh(&np->np_thread_lock); - iscsit_put_transport(conn->conn_transport); - kfree(conn); - conn = NULL; - goto out; + iscsit_free_conn(conn); + return 1; } /* * Perform the remaining iSCSI connection initialization items.. @@ -1442,7 +1450,6 @@ old_sess_out: tpg_np = NULL; } -out: return 1; exit: diff --git a/drivers/target/iscsi/iscsi_target_login.h b/drivers/target/iscsi/iscsi_target_login.h index 74ac3abc44a0..3b8e3639ff5d 100644 --- a/drivers/target/iscsi/iscsi_target_login.h +++ b/drivers/target/iscsi/iscsi_target_login.h @@ -19,7 +19,7 @@ extern int iscsi_target_setup_login_socket(struct iscsi_np *, extern int iscsit_accept_np(struct iscsi_np *, struct iscsi_conn *); extern int iscsit_get_login_rx(struct iscsi_conn *, struct iscsi_login *); extern int iscsit_put_login_tx(struct iscsi_conn *, struct iscsi_login *, u32); -extern void iscsit_free_conn(struct iscsi_np *, struct iscsi_conn *); +extern void iscsit_free_conn(struct iscsi_conn *); extern int iscsit_start_kthreads(struct iscsi_conn *); extern void iscsi_post_login_handler(struct iscsi_np *, struct iscsi_conn *, u8); extern void iscsi_target_login_sess_out(struct iscsi_conn *, struct iscsi_np *, diff --git a/drivers/tty/hvc/hvc_console.c b/drivers/tty/hvc/hvc_console.c index 5414c4a87bea..27284a2dcd2b 100644 --- a/drivers/tty/hvc/hvc_console.c +++ b/drivers/tty/hvc/hvc_console.c @@ -522,6 +522,8 @@ static int hvc_write(struct tty_struct *tty, const unsigned char *buf, int count return -EIO; while (count > 0) { + int ret = 0; + spin_lock_irqsave(&hp->lock, flags); rsize = hp->outbuf_size - hp->n_outbuf; @@ -537,10 +539,13 @@ static int hvc_write(struct tty_struct *tty, const unsigned char *buf, int count } if (hp->n_outbuf > 0) - hvc_push(hp); + ret = hvc_push(hp); spin_unlock_irqrestore(&hp->lock, flags); + if (!ret) + break; + if (count) { if (hp->n_outbuf > 0) hvc_flush(hp); @@ -623,6 +628,15 @@ static int hvc_chars_in_buffer(struct tty_struct *tty) #define MAX_TIMEOUT (2000) static u32 timeout = MIN_TIMEOUT; +/* + * Maximum number of bytes to get from the console driver if hvc_poll is + * called from driver (and can't sleep). Any more than this and we break + * and start polling with khvcd. This value was derived from from an OpenBMC + * console with the OPAL driver that results in about 0.25ms interrupts off + * latency. + */ +#define HVC_ATOMIC_READ_MAX 128 + #define HVC_POLL_READ 0x00000001 #define HVC_POLL_WRITE 0x00000002 @@ -669,8 +683,8 @@ static int __hvc_poll(struct hvc_struct *hp, bool may_sleep) if (!hp->irq_requested) poll_mask |= HVC_POLL_READ; + read_again: /* Read data if any */ - count = tty_buffer_request_room(&hp->port, N_INBUF); /* If flip is full, just reschedule a later read */ @@ -717,9 +731,23 @@ static int __hvc_poll(struct hvc_struct *hp, bool may_sleep) #endif /* CONFIG_MAGIC_SYSRQ */ tty_insert_flip_char(&hp->port, buf[i], 0); } - if (n == count) - poll_mask |= HVC_POLL_READ; - read_total = n; + read_total += n; + + if (may_sleep) { + /* Keep going until the flip is full */ + spin_unlock_irqrestore(&hp->lock, flags); + cond_resched(); + spin_lock_irqsave(&hp->lock, flags); + goto read_again; + } else if (read_total < HVC_ATOMIC_READ_MAX) { + /* Break and defer if it's a large read in atomic */ + goto read_again; + } + + /* + * Latency break, schedule another poll immediately. + */ + poll_mask |= HVC_POLL_READ; out: /* Wakeup write queue if necessary */ diff --git a/drivers/tty/serial/cpm_uart/cpm_uart_core.c b/drivers/tty/serial/cpm_uart/cpm_uart_core.c index 24a5f05e769b..e5389591bb4f 100644 --- a/drivers/tty/serial/cpm_uart/cpm_uart_core.c +++ b/drivers/tty/serial/cpm_uart/cpm_uart_core.c @@ -1054,8 +1054,8 @@ static int poll_wait_key(char *obuf, struct uart_cpm_port *pinfo) /* Get the address of the host memory buffer. */ bdp = pinfo->rx_cur; - while (bdp->cbd_sc & BD_SC_EMPTY) - ; + if (bdp->cbd_sc & BD_SC_EMPTY) + return NO_POLL_CHAR; /* If the buffer address is in the CPM DPRAM, don't * convert it. @@ -1090,7 +1090,11 @@ static int cpm_get_poll_char(struct uart_port *port) poll_chars = 0; } if (poll_chars <= 0) { - poll_chars = poll_wait_key(poll_buf, pinfo); + int ret = poll_wait_key(poll_buf, pinfo); + + if (ret == NO_POLL_CHAR) + return ret; + poll_chars = ret; pollp = poll_buf; } poll_chars--; diff --git a/drivers/tty/serial/fsl_lpuart.c b/drivers/tty/serial/fsl_lpuart.c index 51e47a63d61a..3f8d1274fc85 100644 --- a/drivers/tty/serial/fsl_lpuart.c +++ b/drivers/tty/serial/fsl_lpuart.c @@ -979,7 +979,8 @@ static inline int lpuart_start_rx_dma(struct lpuart_port *sport) struct circ_buf *ring = &sport->rx_ring; int ret, nent; int bits, baud; - struct tty_struct *tty = tty_port_tty_get(&sport->port.state->port); + struct tty_port *port = &sport->port.state->port; + struct tty_struct *tty = port->tty; struct ktermios *termios = &tty->termios; baud = tty_get_baud_rate(tty); diff --git a/drivers/tty/serial/imx.c b/drivers/tty/serial/imx.c index 239c0fa2e981..0f67197a3783 100644 --- a/drivers/tty/serial/imx.c +++ b/drivers/tty/serial/imx.c @@ -2351,6 +2351,14 @@ static int imx_uart_probe(struct platform_device *pdev) ret); return ret; } + + ret = devm_request_irq(&pdev->dev, rtsirq, imx_uart_rtsint, 0, + dev_name(&pdev->dev), sport); + if (ret) { + dev_err(&pdev->dev, "failed to request rts irq: %d\n", + ret); + return ret; + } } else { ret = devm_request_irq(&pdev->dev, rxirq, imx_uart_int, 0, dev_name(&pdev->dev), sport); diff --git a/drivers/tty/serial/mvebu-uart.c b/drivers/tty/serial/mvebu-uart.c index d04b5eeea3c6..170e446a2f62 100644 --- a/drivers/tty/serial/mvebu-uart.c +++ b/drivers/tty/serial/mvebu-uart.c @@ -511,6 +511,7 @@ static void mvebu_uart_set_termios(struct uart_port *port, termios->c_iflag |= old->c_iflag & ~(INPCK | IGNPAR); termios->c_cflag &= CREAD | CBAUD; termios->c_cflag |= old->c_cflag & ~(CREAD | CBAUD); + termios->c_cflag |= CS8; } spin_unlock_irqrestore(&port->lock, flags); diff --git a/drivers/tty/tty_io.c b/drivers/tty/tty_io.c index 32bc3e3fe4d3..5e5da9acaf0a 100644 --- a/drivers/tty/tty_io.c +++ b/drivers/tty/tty_io.c @@ -1255,6 +1255,7 @@ static void tty_driver_remove_tty(struct tty_driver *driver, struct tty_struct * static int tty_reopen(struct tty_struct *tty) { struct tty_driver *driver = tty->driver; + int retval; if (driver->type == TTY_DRIVER_TYPE_PTY && driver->subtype == PTY_TYPE_MASTER) @@ -1268,10 +1269,14 @@ static int tty_reopen(struct tty_struct *tty) tty->count++; - if (!tty->ldisc) - return tty_ldisc_reinit(tty, tty->termios.c_line); + if (tty->ldisc) + return 0; - return 0; + retval = tty_ldisc_reinit(tty, tty->termios.c_line); + if (retval) + tty->count--; + + return retval; } /** diff --git a/drivers/tty/vt/vt_ioctl.c b/drivers/tty/vt/vt_ioctl.c index a78ad10a119b..73cdc0d633dd 100644 --- a/drivers/tty/vt/vt_ioctl.c +++ b/drivers/tty/vt/vt_ioctl.c @@ -32,6 +32,8 @@ #include <asm/io.h> #include <linux/uaccess.h> +#include <linux/nospec.h> + #include <linux/kbd_kern.h> #include <linux/vt_kern.h> #include <linux/kbd_diacr.h> @@ -700,6 +702,8 @@ int vt_ioctl(struct tty_struct *tty, if (vsa.console == 0 || vsa.console > MAX_NR_CONSOLES) ret = -ENXIO; else { + vsa.console = array_index_nospec(vsa.console, + MAX_NR_CONSOLES + 1); vsa.console--; console_lock(); ret = vc_allocate(vsa.console); diff --git a/drivers/usb/class/cdc-acm.c b/drivers/usb/class/cdc-acm.c index 27346d69f393..f9b40a9dc4d3 100644 --- a/drivers/usb/class/cdc-acm.c +++ b/drivers/usb/class/cdc-acm.c @@ -780,20 +780,9 @@ static int acm_tty_write(struct tty_struct *tty, } if (acm->susp_count) { - if (acm->putbuffer) { - /* now to preserve order */ - usb_anchor_urb(acm->putbuffer->urb, &acm->delayed); - acm->putbuffer = NULL; - } usb_anchor_urb(wb->urb, &acm->delayed); spin_unlock_irqrestore(&acm->write_lock, flags); return count; - } else { - if (acm->putbuffer) { - /* at this point there is no good way to handle errors */ - acm_start_wb(acm, acm->putbuffer); - acm->putbuffer = NULL; - } } stat = acm_start_wb(acm, wb); @@ -804,66 +793,6 @@ static int acm_tty_write(struct tty_struct *tty, return count; } -static void acm_tty_flush_chars(struct tty_struct *tty) -{ - struct acm *acm = tty->driver_data; - struct acm_wb *cur; - int err; - unsigned long flags; - - spin_lock_irqsave(&acm->write_lock, flags); - - cur = acm->putbuffer; - if (!cur) /* nothing to do */ - goto out; - - acm->putbuffer = NULL; - err = usb_autopm_get_interface_async(acm->control); - if (err < 0) { - cur->use = 0; - acm->putbuffer = cur; - goto out; - } - - if (acm->susp_count) - usb_anchor_urb(cur->urb, &acm->delayed); - else - acm_start_wb(acm, cur); -out: - spin_unlock_irqrestore(&acm->write_lock, flags); - return; -} - -static int acm_tty_put_char(struct tty_struct *tty, unsigned char ch) -{ - struct acm *acm = tty->driver_data; - struct acm_wb *cur; - int wbn; - unsigned long flags; - -overflow: - cur = acm->putbuffer; - if (!cur) { - spin_lock_irqsave(&acm->write_lock, flags); - wbn = acm_wb_alloc(acm); - if (wbn >= 0) { - cur = &acm->wb[wbn]; - acm->putbuffer = cur; - } - spin_unlock_irqrestore(&acm->write_lock, flags); - if (!cur) - return 0; - } - - if (cur->len == acm->writesize) { - acm_tty_flush_chars(tty); - goto overflow; - } - - cur->buf[cur->len++] = ch; - return 1; -} - static int acm_tty_write_room(struct tty_struct *tty) { struct acm *acm = tty->driver_data; @@ -1987,8 +1916,6 @@ static const struct tty_operations acm_ops = { .cleanup = acm_tty_cleanup, .hangup = acm_tty_hangup, .write = acm_tty_write, - .put_char = acm_tty_put_char, - .flush_chars = acm_tty_flush_chars, .write_room = acm_tty_write_room, .ioctl = acm_tty_ioctl, .throttle = acm_tty_throttle, diff --git a/drivers/usb/class/cdc-acm.h b/drivers/usb/class/cdc-acm.h index eacc116e83da..ca06b20d7af9 100644 --- a/drivers/usb/class/cdc-acm.h +++ b/drivers/usb/class/cdc-acm.h @@ -96,7 +96,6 @@ struct acm { unsigned long read_urbs_free; struct urb *read_urbs[ACM_NR]; struct acm_rb read_buffers[ACM_NR]; - struct acm_wb *putbuffer; /* for acm_tty_put_char() */ int rx_buflimit; spinlock_t read_lock; u8 *notification_buffer; /* to reassemble fragmented notifications */ diff --git a/drivers/usb/common/common.c b/drivers/usb/common/common.c index 50a2362ed3ea..48277bbc15e4 100644 --- a/drivers/usb/common/common.c +++ b/drivers/usb/common/common.c @@ -246,6 +246,31 @@ int of_usb_update_otg_caps(struct device_node *np, } EXPORT_SYMBOL_GPL(of_usb_update_otg_caps); +/** + * usb_of_get_companion_dev - Find the companion device + * @dev: the device pointer to find a companion + * + * Find the companion device from platform bus. + * + * Takes a reference to the returned struct device which needs to be dropped + * after use. + * + * Return: On success, a pointer to the companion device, %NULL on failure. + */ +struct device *usb_of_get_companion_dev(struct device *dev) +{ + struct device_node *node; + struct platform_device *pdev = NULL; + + node = of_parse_phandle(dev->of_node, "companion", 0); + if (node) + pdev = of_find_device_by_node(node); + + of_node_put(node); + + return pdev ? &pdev->dev : NULL; +} +EXPORT_SYMBOL_GPL(usb_of_get_companion_dev); #endif MODULE_LICENSE("GPL"); diff --git a/drivers/usb/common/roles.c b/drivers/usb/common/roles.c index 15cc76e22123..99116af07f1d 100644 --- a/drivers/usb/common/roles.c +++ b/drivers/usb/common/roles.c @@ -109,8 +109,15 @@ static void *usb_role_switch_match(struct device_connection *con, int ep, */ struct usb_role_switch *usb_role_switch_get(struct device *dev) { - return device_connection_find_match(dev, "usb-role-switch", NULL, - usb_role_switch_match); + struct usb_role_switch *sw; + + sw = device_connection_find_match(dev, "usb-role-switch", NULL, + usb_role_switch_match); + + if (!IS_ERR_OR_NULL(sw)) + WARN_ON(!try_module_get(sw->dev.parent->driver->owner)); + + return sw; } EXPORT_SYMBOL_GPL(usb_role_switch_get); @@ -122,8 +129,10 @@ EXPORT_SYMBOL_GPL(usb_role_switch_get); */ void usb_role_switch_put(struct usb_role_switch *sw) { - if (!IS_ERR_OR_NULL(sw)) + if (!IS_ERR_OR_NULL(sw)) { put_device(&sw->dev); + module_put(sw->dev.parent->driver->owner); + } } EXPORT_SYMBOL_GPL(usb_role_switch_put); diff --git a/drivers/usb/core/devio.c b/drivers/usb/core/devio.c index 6ce77b33da61..244417d0dfd1 100644 --- a/drivers/usb/core/devio.c +++ b/drivers/usb/core/devio.c @@ -1434,10 +1434,13 @@ static int proc_do_submiturb(struct usb_dev_state *ps, struct usbdevfs_urb *uurb struct async *as = NULL; struct usb_ctrlrequest *dr = NULL; unsigned int u, totlen, isofrmlen; - int i, ret, is_in, num_sgs = 0, ifnum = -1; + int i, ret, num_sgs = 0, ifnum = -1; int number_of_packets = 0; unsigned int stream_id = 0; void *buf; + bool is_in; + bool allow_short = false; + bool allow_zero = false; unsigned long mask = USBDEVFS_URB_SHORT_NOT_OK | USBDEVFS_URB_BULK_CONTINUATION | USBDEVFS_URB_NO_FSBR | @@ -1471,6 +1474,8 @@ static int proc_do_submiturb(struct usb_dev_state *ps, struct usbdevfs_urb *uurb u = 0; switch (uurb->type) { case USBDEVFS_URB_TYPE_CONTROL: + if (is_in) + allow_short = true; if (!usb_endpoint_xfer_control(&ep->desc)) return -EINVAL; /* min 8 byte setup packet */ @@ -1511,6 +1516,10 @@ static int proc_do_submiturb(struct usb_dev_state *ps, struct usbdevfs_urb *uurb break; case USBDEVFS_URB_TYPE_BULK: + if (!is_in) + allow_zero = true; + else + allow_short = true; switch (usb_endpoint_type(&ep->desc)) { case USB_ENDPOINT_XFER_CONTROL: case USB_ENDPOINT_XFER_ISOC: @@ -1531,6 +1540,10 @@ static int proc_do_submiturb(struct usb_dev_state *ps, struct usbdevfs_urb *uurb if (!usb_endpoint_xfer_int(&ep->desc)) return -EINVAL; interrupt_urb: + if (!is_in) + allow_zero = true; + else + allow_short = true; break; case USBDEVFS_URB_TYPE_ISO: @@ -1676,14 +1689,19 @@ static int proc_do_submiturb(struct usb_dev_state *ps, struct usbdevfs_urb *uurb u = (is_in ? URB_DIR_IN : URB_DIR_OUT); if (uurb->flags & USBDEVFS_URB_ISO_ASAP) u |= URB_ISO_ASAP; - if (uurb->flags & USBDEVFS_URB_SHORT_NOT_OK && is_in) + if (allow_short && uurb->flags & USBDEVFS_URB_SHORT_NOT_OK) u |= URB_SHORT_NOT_OK; - if (uurb->flags & USBDEVFS_URB_ZERO_PACKET) + if (allow_zero && uurb->flags & USBDEVFS_URB_ZERO_PACKET) u |= URB_ZERO_PACKET; if (uurb->flags & USBDEVFS_URB_NO_INTERRUPT) u |= URB_NO_INTERRUPT; as->urb->transfer_flags = u; + if (!allow_short && uurb->flags & USBDEVFS_URB_SHORT_NOT_OK) + dev_warn(&ps->dev->dev, "Requested nonsensical USBDEVFS_URB_SHORT_NOT_OK.\n"); + if (!allow_zero && uurb->flags & USBDEVFS_URB_ZERO_PACKET) + dev_warn(&ps->dev->dev, "Requested nonsensical USBDEVFS_URB_ZERO_PACKET.\n"); + as->urb->transfer_buffer_length = uurb->buffer_length; as->urb->setup_packet = (unsigned char *)dr; dr = NULL; diff --git a/drivers/usb/core/driver.c b/drivers/usb/core/driver.c index e76e95f62f76..a1f225f077cd 100644 --- a/drivers/usb/core/driver.c +++ b/drivers/usb/core/driver.c @@ -512,7 +512,6 @@ int usb_driver_claim_interface(struct usb_driver *driver, struct device *dev; struct usb_device *udev; int retval = 0; - int lpm_disable_error = -ENODEV; if (!iface) return -ENODEV; @@ -533,16 +532,6 @@ int usb_driver_claim_interface(struct usb_driver *driver, iface->condition = USB_INTERFACE_BOUND; - /* See the comment about disabling LPM in usb_probe_interface(). */ - if (driver->disable_hub_initiated_lpm) { - lpm_disable_error = usb_unlocked_disable_lpm(udev); - if (lpm_disable_error) { - dev_err(&iface->dev, "%s Failed to disable LPM for driver %s\n", - __func__, driver->name); - return -ENOMEM; - } - } - /* Claimed interfaces are initially inactive (suspended) and * runtime-PM-enabled, but only if the driver has autosuspend * support. Otherwise they are marked active, to prevent the @@ -561,9 +550,20 @@ int usb_driver_claim_interface(struct usb_driver *driver, if (device_is_registered(dev)) retval = device_bind_driver(dev); - /* Attempt to re-enable USB3 LPM, if the disable was successful. */ - if (!lpm_disable_error) - usb_unlocked_enable_lpm(udev); + if (retval) { + dev->driver = NULL; + usb_set_intfdata(iface, NULL); + iface->needs_remote_wakeup = 0; + iface->condition = USB_INTERFACE_UNBOUND; + + /* + * Unbound interfaces are always runtime-PM-disabled + * and runtime-PM-suspended + */ + if (driver->supports_autosuspend) + pm_runtime_disable(dev); + pm_runtime_set_suspended(dev); + } return retval; } diff --git a/drivers/usb/core/hcd-pci.c b/drivers/usb/core/hcd-pci.c index 66fe1b78d952..03432467b05f 100644 --- a/drivers/usb/core/hcd-pci.c +++ b/drivers/usb/core/hcd-pci.c @@ -515,8 +515,6 @@ static int resume_common(struct device *dev, int event) event == PM_EVENT_RESTORE); if (retval) { dev_err(dev, "PCI post-resume error %d!\n", retval); - if (hcd->shared_hcd) - usb_hc_died(hcd->shared_hcd); usb_hc_died(hcd); } } diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c index 228672f2c4a1..bfa5eda0cc26 100644 --- a/drivers/usb/core/message.c +++ b/drivers/usb/core/message.c @@ -1341,6 +1341,11 @@ void usb_enable_interface(struct usb_device *dev, * is submitted that needs that bandwidth. Some other operating systems * allocate bandwidth early, when a configuration is chosen. * + * xHCI reserves bandwidth and configures the alternate setting in + * usb_hcd_alloc_bandwidth(). If it fails the original interface altsetting + * may be disabled. Drivers cannot rely on any particular alternate + * setting being in effect after a failure. + * * This call is synchronous, and may not be used in an interrupt context. * Also, drivers must not change altsettings while urbs are scheduled for * endpoints in that interface; all such urbs must first be completed @@ -1376,6 +1381,12 @@ int usb_set_interface(struct usb_device *dev, int interface, int alternate) alternate); return -EINVAL; } + /* + * usb3 hosts configure the interface in usb_hcd_alloc_bandwidth, + * including freeing dropped endpoint ring buffers. + * Make sure the interface endpoints are flushed before that + */ + usb_disable_interface(dev, iface, false); /* Make sure we have enough bandwidth for this alternate interface. * Remove the current alt setting and add the new alt setting. diff --git a/drivers/usb/core/of.c b/drivers/usb/core/of.c index fd77442c2d12..651708d8c908 100644 --- a/drivers/usb/core/of.c +++ b/drivers/usb/core/of.c @@ -105,29 +105,3 @@ usb_of_get_interface_node(struct usb_device *udev, u8 config, u8 ifnum) return NULL; } EXPORT_SYMBOL_GPL(usb_of_get_interface_node); - -/** - * usb_of_get_companion_dev - Find the companion device - * @dev: the device pointer to find a companion - * - * Find the companion device from platform bus. - * - * Takes a reference to the returned struct device which needs to be dropped - * after use. - * - * Return: On success, a pointer to the companion device, %NULL on failure. - */ -struct device *usb_of_get_companion_dev(struct device *dev) -{ - struct device_node *node; - struct platform_device *pdev = NULL; - - node = of_parse_phandle(dev->of_node, "companion", 0); - if (node) - pdev = of_find_device_by_node(node); - - of_node_put(node); - - return pdev ? &pdev->dev : NULL; -} -EXPORT_SYMBOL_GPL(usb_of_get_companion_dev); diff --git a/drivers/usb/core/quirks.c b/drivers/usb/core/quirks.c index 097057d2eacf..178d6c6063c0 100644 --- a/drivers/usb/core/quirks.c +++ b/drivers/usb/core/quirks.c @@ -58,6 +58,7 @@ static int quirks_param_set(const char *val, const struct kernel_param *kp) quirk_list = kcalloc(quirk_count, sizeof(struct quirk_entry), GFP_KERNEL); if (!quirk_list) { + quirk_count = 0; mutex_unlock(&quirk_mutex); return -ENOMEM; } @@ -154,7 +155,7 @@ static struct kparam_string quirks_param_string = { .string = quirks_param, }; -module_param_cb(quirks, &quirks_param_ops, &quirks_param_string, 0644); +device_param_cb(quirks, &quirks_param_ops, &quirks_param_string, 0644); MODULE_PARM_DESC(quirks, "Add/modify USB quirks by specifying quirks=vendorID:productID:quirks"); /* Lists of quirky USB devices, split in device quirks and interface quirks. @@ -178,6 +179,10 @@ static const struct usb_device_id usb_quirk_list[] = { /* CBM - Flash disk */ { USB_DEVICE(0x0204, 0x6025), .driver_info = USB_QUIRK_RESET_RESUME }, + /* WORLDE Controller KS49 or Prodipe MIDI 49C USB controller */ + { USB_DEVICE(0x0218, 0x0201), .driver_info = + USB_QUIRK_CONFIG_INTF_STRINGS }, + /* WORLDE easy key (easykey.25) MIDI controller */ { USB_DEVICE(0x0218, 0x0401), .driver_info = USB_QUIRK_CONFIG_INTF_STRINGS }, @@ -406,6 +411,9 @@ static const struct usb_device_id usb_quirk_list[] = { { USB_DEVICE(0x2040, 0x7200), .driver_info = USB_QUIRK_CONFIG_INTF_STRINGS }, + /* DJI CineSSD */ + { USB_DEVICE(0x2ca3, 0x0031), .driver_info = USB_QUIRK_NO_LPM }, + /* INTEL VALUE SSD */ { USB_DEVICE(0x8086, 0xf1a5), .driver_info = USB_QUIRK_RESET_RESUME }, diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c index 623be3174fb3..79d8bd7a612e 100644 --- a/drivers/usb/core/usb.c +++ b/drivers/usb/core/usb.c @@ -228,6 +228,8 @@ struct usb_host_interface *usb_find_alt_setting( struct usb_interface_cache *intf_cache = NULL; int i; + if (!config) + return NULL; for (i = 0; i < config->desc.bNumInterfaces; i++) { if (config->intf_cache[i]->altsetting[0].desc.bInterfaceNumber == iface_num) { diff --git a/drivers/usb/dwc2/platform.c b/drivers/usb/dwc2/platform.c index 9a53a58e676e..577642895b57 100644 --- a/drivers/usb/dwc2/platform.c +++ b/drivers/usb/dwc2/platform.c @@ -412,8 +412,6 @@ static int dwc2_driver_probe(struct platform_device *dev) dev_dbg(&dev->dev, "mapped PA %08lx to VA %p\n", (unsigned long)res->start, hsotg->regs); - hsotg->needs_byte_swap = dwc2_check_core_endianness(hsotg); - retval = dwc2_lowlevel_hw_init(hsotg); if (retval) return retval; @@ -438,6 +436,8 @@ static int dwc2_driver_probe(struct platform_device *dev) if (retval) return retval; + hsotg->needs_byte_swap = dwc2_check_core_endianness(hsotg); + retval = dwc2_get_dr_mode(hsotg); if (retval) goto error; diff --git a/drivers/usb/dwc3/dwc3-of-simple.c b/drivers/usb/dwc3/dwc3-of-simple.c index 40bf9e0bbc59..4c2771c5e727 100644 --- a/drivers/usb/dwc3/dwc3-of-simple.c +++ b/drivers/usb/dwc3/dwc3-of-simple.c @@ -180,8 +180,7 @@ static int dwc3_of_simple_remove(struct platform_device *pdev) return 0; } -#ifdef CONFIG_PM -static int dwc3_of_simple_runtime_suspend(struct device *dev) +static int __maybe_unused dwc3_of_simple_runtime_suspend(struct device *dev) { struct dwc3_of_simple *simple = dev_get_drvdata(dev); int i; @@ -192,7 +191,7 @@ static int dwc3_of_simple_runtime_suspend(struct device *dev) return 0; } -static int dwc3_of_simple_runtime_resume(struct device *dev) +static int __maybe_unused dwc3_of_simple_runtime_resume(struct device *dev) { struct dwc3_of_simple *simple = dev_get_drvdata(dev); int ret; @@ -210,7 +209,7 @@ static int dwc3_of_simple_runtime_resume(struct device *dev) return 0; } -static int dwc3_of_simple_suspend(struct device *dev) +static int __maybe_unused dwc3_of_simple_suspend(struct device *dev) { struct dwc3_of_simple *simple = dev_get_drvdata(dev); @@ -220,7 +219,7 @@ static int dwc3_of_simple_suspend(struct device *dev) return 0; } -static int dwc3_of_simple_resume(struct device *dev) +static int __maybe_unused dwc3_of_simple_resume(struct device *dev) { struct dwc3_of_simple *simple = dev_get_drvdata(dev); @@ -229,7 +228,6 @@ static int dwc3_of_simple_resume(struct device *dev) return 0; } -#endif static const struct dev_pm_ops dwc3_of_simple_dev_pm_ops = { SET_SYSTEM_SLEEP_PM_OPS(dwc3_of_simple_suspend, dwc3_of_simple_resume) diff --git a/drivers/usb/dwc3/dwc3-pci.c b/drivers/usb/dwc3/dwc3-pci.c index 5edd79470368..1286076a8890 100644 --- a/drivers/usb/dwc3/dwc3-pci.c +++ b/drivers/usb/dwc3/dwc3-pci.c @@ -85,8 +85,8 @@ static int dwc3_byt_enable_ulpi_refclock(struct pci_dev *pci) u32 value; reg = pcim_iomap(pci, GP_RWBAR, 0); - if (IS_ERR(reg)) - return PTR_ERR(reg); + if (!reg) + return -ENOMEM; value = readl(reg + GP_RWREG1); if (!(value & GP_RWREG1_ULPI_REFCLK_DISABLE)) diff --git a/drivers/usb/dwc3/gadget.c b/drivers/usb/dwc3/gadget.c index 032ea7d709ba..2b53194081ba 100644 --- a/drivers/usb/dwc3/gadget.c +++ b/drivers/usb/dwc3/gadget.c @@ -473,7 +473,6 @@ static int dwc3_gadget_set_xfer_resource(struct dwc3_ep *dep) /** * dwc3_gadget_start_config - configure ep resources - * @dwc: pointer to our controller context structure * @dep: endpoint that is being enabled * * Issue a %DWC3_DEPCMD_DEPSTARTCFG command to @dep. After the command's diff --git a/drivers/usb/gadget/udc/fotg210-udc.c b/drivers/usb/gadget/udc/fotg210-udc.c index 53a48f561458..587c5037ff07 100644 --- a/drivers/usb/gadget/udc/fotg210-udc.c +++ b/drivers/usb/gadget/udc/fotg210-udc.c @@ -1063,12 +1063,15 @@ static const struct usb_gadget_ops fotg210_gadget_ops = { static int fotg210_udc_remove(struct platform_device *pdev) { struct fotg210_udc *fotg210 = platform_get_drvdata(pdev); + int i; usb_del_gadget_udc(&fotg210->gadget); iounmap(fotg210->reg); free_irq(platform_get_irq(pdev, 0), fotg210); fotg210_ep_free_request(&fotg210->ep[0]->ep, fotg210->ep0_req); + for (i = 0; i < FOTG210_MAX_NUM_EP; i++) + kfree(fotg210->ep[i]); kfree(fotg210); return 0; @@ -1099,7 +1102,7 @@ static int fotg210_udc_probe(struct platform_device *pdev) /* initialize udc */ fotg210 = kzalloc(sizeof(struct fotg210_udc), GFP_KERNEL); if (fotg210 == NULL) - goto err_alloc; + goto err; for (i = 0; i < FOTG210_MAX_NUM_EP; i++) { _ep[i] = kzalloc(sizeof(struct fotg210_ep), GFP_KERNEL); @@ -1111,7 +1114,7 @@ static int fotg210_udc_probe(struct platform_device *pdev) fotg210->reg = ioremap(res->start, resource_size(res)); if (fotg210->reg == NULL) { pr_err("ioremap error.\n"); - goto err_map; + goto err_alloc; } spin_lock_init(&fotg210->lock); @@ -1159,7 +1162,7 @@ static int fotg210_udc_probe(struct platform_device *pdev) fotg210->ep0_req = fotg210_ep_alloc_request(&fotg210->ep[0]->ep, GFP_KERNEL); if (fotg210->ep0_req == NULL) - goto err_req; + goto err_map; fotg210_init(fotg210); @@ -1187,12 +1190,14 @@ err_req: fotg210_ep_free_request(&fotg210->ep[0]->ep, fotg210->ep0_req); err_map: - if (fotg210->reg) - iounmap(fotg210->reg); + iounmap(fotg210->reg); err_alloc: + for (i = 0; i < FOTG210_MAX_NUM_EP; i++) + kfree(fotg210->ep[i]); kfree(fotg210); +err: return ret; } diff --git a/drivers/usb/gadget/udc/net2280.c b/drivers/usb/gadget/udc/net2280.c index 318246d8b2e2..b02ab2a8d927 100644 --- a/drivers/usb/gadget/udc/net2280.c +++ b/drivers/usb/gadget/udc/net2280.c @@ -1545,11 +1545,14 @@ static int net2280_pullup(struct usb_gadget *_gadget, int is_on) writel(tmp | BIT(USB_DETECT_ENABLE), &dev->usb->usbctl); } else { writel(tmp & ~BIT(USB_DETECT_ENABLE), &dev->usb->usbctl); - stop_activity(dev, dev->driver); + stop_activity(dev, NULL); } spin_unlock_irqrestore(&dev->lock, flags); + if (!is_on && dev->driver) + dev->driver->disconnect(&dev->gadget); + return 0; } @@ -2466,8 +2469,11 @@ static void stop_activity(struct net2280 *dev, struct usb_gadget_driver *driver) nuke(&dev->ep[i]); /* report disconnect; the driver is already quiesced */ - if (driver) + if (driver) { + spin_unlock(&dev->lock); driver->disconnect(&dev->gadget); + spin_lock(&dev->lock); + } usb_reinit(dev); } @@ -3341,6 +3347,8 @@ next_endpoints: BIT(PCI_RETRY_ABORT_INTERRUPT)) static void handle_stat1_irqs(struct net2280 *dev, u32 stat) +__releases(dev->lock) +__acquires(dev->lock) { struct net2280_ep *ep; u32 tmp, num, mask, scratch; @@ -3381,12 +3389,14 @@ static void handle_stat1_irqs(struct net2280 *dev, u32 stat) if (disconnect || reset) { stop_activity(dev, dev->driver); ep0_start(dev); + spin_unlock(&dev->lock); if (reset) usb_gadget_udc_reset (&dev->gadget, dev->driver); else (dev->driver->disconnect) (&dev->gadget); + spin_lock(&dev->lock); return; } } @@ -3405,6 +3415,7 @@ static void handle_stat1_irqs(struct net2280 *dev, u32 stat) tmp = BIT(SUSPEND_REQUEST_CHANGE_INTERRUPT); if (stat & tmp) { writel(tmp, &dev->regs->irqstat1); + spin_unlock(&dev->lock); if (stat & BIT(SUSPEND_REQUEST_INTERRUPT)) { if (dev->driver->suspend) dev->driver->suspend(&dev->gadget); @@ -3415,6 +3426,7 @@ static void handle_stat1_irqs(struct net2280 *dev, u32 stat) dev->driver->resume(&dev->gadget); /* at high speed, note erratum 0133 */ } + spin_lock(&dev->lock); stat &= ~tmp; } diff --git a/drivers/usb/gadget/udc/renesas_usb3.c b/drivers/usb/gadget/udc/renesas_usb3.c index 1f879b3f2c96..e1656f361e08 100644 --- a/drivers/usb/gadget/udc/renesas_usb3.c +++ b/drivers/usb/gadget/udc/renesas_usb3.c @@ -812,12 +812,15 @@ static void usb3_irq_epc_int_1_speed(struct renesas_usb3 *usb3) switch (speed) { case USB_STA_SPEED_SS: usb3->gadget.speed = USB_SPEED_SUPER; + usb3->gadget.ep0->maxpacket = USB3_EP0_SS_MAX_PACKET_SIZE; break; case USB_STA_SPEED_HS: usb3->gadget.speed = USB_SPEED_HIGH; + usb3->gadget.ep0->maxpacket = USB3_EP0_HSFS_MAX_PACKET_SIZE; break; case USB_STA_SPEED_FS: usb3->gadget.speed = USB_SPEED_FULL; + usb3->gadget.ep0->maxpacket = USB3_EP0_HSFS_MAX_PACKET_SIZE; break; default: usb3->gadget.speed = USB_SPEED_UNKNOWN; @@ -2513,7 +2516,7 @@ static int renesas_usb3_init_ep(struct renesas_usb3 *usb3, struct device *dev, /* for control pipe */ usb3->gadget.ep0 = &usb3_ep->ep; usb_ep_set_maxpacket_limit(&usb3_ep->ep, - USB3_EP0_HSFS_MAX_PACKET_SIZE); + USB3_EP0_SS_MAX_PACKET_SIZE); usb3_ep->ep.caps.type_control = true; usb3_ep->ep.caps.dir_in = true; usb3_ep->ep.caps.dir_out = true; diff --git a/drivers/usb/host/u132-hcd.c b/drivers/usb/host/u132-hcd.c index 072bd5d5738e..5b8a3d9530c4 100644 --- a/drivers/usb/host/u132-hcd.c +++ b/drivers/usb/host/u132-hcd.c @@ -2555,7 +2555,7 @@ static int u132_get_frame(struct usb_hcd *hcd) } else { int frame = 0; dev_err(&u132->platform_dev->dev, "TODO: u132_get_frame\n"); - msleep(100); + mdelay(100); return frame; } } diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c index ef350c33dc4a..b1f27aa38b10 100644 --- a/drivers/usb/host/xhci-mem.c +++ b/drivers/usb/host/xhci-mem.c @@ -1613,6 +1613,10 @@ void xhci_endpoint_copy(struct xhci_hcd *xhci, in_ep_ctx->ep_info2 = out_ep_ctx->ep_info2; in_ep_ctx->deq = out_ep_ctx->deq; in_ep_ctx->tx_info = out_ep_ctx->tx_info; + if (xhci->quirks & XHCI_MTK_HOST) { + in_ep_ctx->reserved[0] = out_ep_ctx->reserved[0]; + in_ep_ctx->reserved[1] = out_ep_ctx->reserved[1]; + } } /* Copy output xhci_slot_ctx to the input xhci_slot_ctx. diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c index 8dc77e34a859..94e939249b2b 100644 --- a/drivers/usb/host/xhci-plat.c +++ b/drivers/usb/host/xhci-plat.c @@ -153,7 +153,7 @@ static int xhci_plat_probe(struct platform_device *pdev) { const struct xhci_plat_priv *priv_match; const struct hc_driver *driver; - struct device *sysdev; + struct device *sysdev, *tmpdev; struct xhci_hcd *xhci; struct resource *res; struct usb_hcd *hcd; @@ -273,19 +273,24 @@ static int xhci_plat_probe(struct platform_device *pdev) goto disable_clk; } - if (device_property_read_bool(sysdev, "usb2-lpm-disable")) - xhci->quirks |= XHCI_HW_LPM_DISABLE; + /* imod_interval is the interrupt moderation value in nanoseconds. */ + xhci->imod_interval = 40000; - if (device_property_read_bool(sysdev, "usb3-lpm-capable")) - xhci->quirks |= XHCI_LPM_SUPPORT; + /* Iterate over all parent nodes for finding quirks */ + for (tmpdev = &pdev->dev; tmpdev; tmpdev = tmpdev->parent) { - if (device_property_read_bool(&pdev->dev, "quirk-broken-port-ped")) - xhci->quirks |= XHCI_BROKEN_PORT_PED; + if (device_property_read_bool(tmpdev, "usb2-lpm-disable")) + xhci->quirks |= XHCI_HW_LPM_DISABLE; - /* imod_interval is the interrupt moderation value in nanoseconds. */ - xhci->imod_interval = 40000; - device_property_read_u32(sysdev, "imod-interval-ns", - &xhci->imod_interval); + if (device_property_read_bool(tmpdev, "usb3-lpm-capable")) + xhci->quirks |= XHCI_LPM_SUPPORT; + + if (device_property_read_bool(tmpdev, "quirk-broken-port-ped")) + xhci->quirks |= XHCI_BROKEN_PORT_PED; + + device_property_read_u32(tmpdev, "imod-interval-ns", + &xhci->imod_interval); + } hcd->usb_phy = devm_usb_get_phy_by_phandle(sysdev, "usb-phy", 0); if (IS_ERR(hcd->usb_phy)) { diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c index 61f48b17e57b..0420eefa647a 100644 --- a/drivers/usb/host/xhci.c +++ b/drivers/usb/host/xhci.c @@ -37,6 +37,21 @@ static unsigned long long quirks; module_param(quirks, ullong, S_IRUGO); MODULE_PARM_DESC(quirks, "Bit flags for quirks to be enabled as default"); +static bool td_on_ring(struct xhci_td *td, struct xhci_ring *ring) +{ + struct xhci_segment *seg = ring->first_seg; + + if (!td || !td->start_seg) + return false; + do { + if (seg == td->start_seg) + return true; + seg = seg->next; + } while (seg && seg != ring->first_seg); + + return false; +} + /* TODO: copied from ehci-hcd.c - can this be refactored? */ /* * xhci_handshake - spin reading hc until handshake completes or fails @@ -1571,6 +1586,21 @@ static int xhci_urb_dequeue(struct usb_hcd *hcd, struct urb *urb, int status) goto done; } + /* + * check ring is not re-allocated since URB was enqueued. If it is, then + * make sure none of the ring related pointers in this URB private data + * are touched, such as td_list, otherwise we overwrite freed data + */ + if (!td_on_ring(&urb_priv->td[0], ep_ring)) { + xhci_err(xhci, "Canceled URB td not found on endpoint ring"); + for (i = urb_priv->num_tds_done; i < urb_priv->num_tds; i++) { + td = &urb_priv->td[i]; + if (!list_empty(&td->cancelled_td_list)) + list_del_init(&td->cancelled_td_list); + } + goto err_giveback; + } + if (xhci->xhc_state & XHCI_STATE_HALTED) { xhci_dbg_trace(xhci, trace_xhci_dbg_cancel_urb, "HC halted, freeing TD manually."); diff --git a/drivers/usb/misc/uss720.c b/drivers/usb/misc/uss720.c index 82f220631bd7..b5d661644263 100644 --- a/drivers/usb/misc/uss720.c +++ b/drivers/usb/misc/uss720.c @@ -369,7 +369,7 @@ static unsigned char parport_uss720_frob_control(struct parport *pp, unsigned ch mask &= 0x0f; val &= 0x0f; d = (priv->reg[1] & (~mask)) ^ val; - if (set_1284_register(pp, 2, d, GFP_KERNEL)) + if (set_1284_register(pp, 2, d, GFP_ATOMIC)) return 0; priv->reg[1] = d; return d & 0xf; @@ -379,7 +379,7 @@ static unsigned char parport_uss720_read_status(struct parport *pp) { unsigned char ret; - if (get_1284_register(pp, 1, &ret, GFP_KERNEL)) + if (get_1284_register(pp, 1, &ret, GFP_ATOMIC)) return 0; return ret & 0xf8; } diff --git a/drivers/usb/misc/yurex.c b/drivers/usb/misc/yurex.c index 3be40eaa1ac9..6d9fd5f64903 100644 --- a/drivers/usb/misc/yurex.c +++ b/drivers/usb/misc/yurex.c @@ -413,6 +413,9 @@ static ssize_t yurex_read(struct file *file, char __user *buffer, size_t count, spin_unlock_irqrestore(&dev->lock, flags); mutex_unlock(&dev->io_mutex); + if (WARN_ON_ONCE(len >= sizeof(in_buffer))) + return -EIO; + return simple_read_from_buffer(buffer, count, ppos, in_buffer, len); } @@ -421,13 +424,13 @@ static ssize_t yurex_write(struct file *file, const char __user *user_buffer, { struct usb_yurex *dev; int i, set = 0, retval = 0; - char buffer[16]; + char buffer[16 + 1]; char *data = buffer; unsigned long long c, c2 = 0; signed long timeout = 0; DEFINE_WAIT(wait); - count = min(sizeof(buffer), count); + count = min(sizeof(buffer) - 1, count); dev = file->private_data; /* verify that we actually have some data to write */ @@ -446,6 +449,7 @@ static ssize_t yurex_write(struct file *file, const char __user *user_buffer, retval = -EFAULT; goto error; } + buffer[count] = 0; memset(dev->cntl_buffer, CMD_PADDING, YUREX_BUF_SIZE); switch (buffer[0]) { diff --git a/drivers/usb/mtu3/mtu3_core.c b/drivers/usb/mtu3/mtu3_core.c index eecfd0671362..d045d8458f81 100644 --- a/drivers/usb/mtu3/mtu3_core.c +++ b/drivers/usb/mtu3/mtu3_core.c @@ -107,8 +107,12 @@ static int mtu3_device_enable(struct mtu3 *mtu) (SSUSB_U2_PORT_DIS | SSUSB_U2_PORT_PDN | SSUSB_U2_PORT_HOST_SEL)); - if (mtu->ssusb->dr_mode == USB_DR_MODE_OTG) + if (mtu->ssusb->dr_mode == USB_DR_MODE_OTG) { mtu3_setbits(ibase, SSUSB_U2_CTRL(0), SSUSB_U2_PORT_OTG_SEL); + if (mtu->is_u3_ip) + mtu3_setbits(ibase, SSUSB_U3_CTRL(0), + SSUSB_U3_PORT_DUAL_MODE); + } return ssusb_check_clocks(mtu->ssusb, check_clk); } diff --git a/drivers/usb/mtu3/mtu3_hw_regs.h b/drivers/usb/mtu3/mtu3_hw_regs.h index 6ee371478d89..a45bb253939f 100644 --- a/drivers/usb/mtu3/mtu3_hw_regs.h +++ b/drivers/usb/mtu3/mtu3_hw_regs.h @@ -459,6 +459,7 @@ /* U3D_SSUSB_U3_CTRL_0P */ #define SSUSB_U3_PORT_SSP_SPEED BIT(9) +#define SSUSB_U3_PORT_DUAL_MODE BIT(7) #define SSUSB_U3_PORT_HOST_SEL BIT(2) #define SSUSB_U3_PORT_PDN BIT(1) #define SSUSB_U3_PORT_DIS BIT(0) diff --git a/drivers/usb/musb/musb_dsps.c b/drivers/usb/musb/musb_dsps.c index df827ff57b0d..23a0df79ef21 100644 --- a/drivers/usb/musb/musb_dsps.c +++ b/drivers/usb/musb/musb_dsps.c @@ -658,16 +658,6 @@ dsps_dma_controller_create(struct musb *musb, void __iomem *base) return controller; } -static void dsps_dma_controller_destroy(struct dma_controller *c) -{ - struct musb *musb = c->musb; - struct dsps_glue *glue = dev_get_drvdata(musb->controller->parent); - void __iomem *usbss_base = glue->usbss_base; - - musb_writel(usbss_base, USBSS_IRQ_CLEARR, USBSS_IRQ_PD_COMP); - cppi41_dma_controller_destroy(c); -} - #ifdef CONFIG_PM_SLEEP static void dsps_dma_controller_suspend(struct dsps_glue *glue) { @@ -697,7 +687,7 @@ static struct musb_platform_ops dsps_ops = { #ifdef CONFIG_USB_TI_CPPI41_DMA .dma_init = dsps_dma_controller_create, - .dma_exit = dsps_dma_controller_destroy, + .dma_exit = cppi41_dma_controller_destroy, #endif .enable = dsps_musb_enable, .disable = dsps_musb_disable, diff --git a/drivers/usb/serial/io_ti.h b/drivers/usb/serial/io_ti.h index e53c68261017..9bbcee37524e 100644 --- a/drivers/usb/serial/io_ti.h +++ b/drivers/usb/serial/io_ti.h @@ -173,7 +173,7 @@ struct ump_interrupt { } __attribute__((packed)); -#define TIUMP_GET_PORT_FROM_CODE(c) (((c) >> 4) - 3) +#define TIUMP_GET_PORT_FROM_CODE(c) (((c) >> 6) & 0x01) #define TIUMP_GET_FUNC_FROM_CODE(c) ((c) & 0x0f) #define TIUMP_INTERRUPT_CODE_LSR 0x03 #define TIUMP_INTERRUPT_CODE_MSR 0x04 diff --git a/drivers/usb/serial/ti_usb_3410_5052.c b/drivers/usb/serial/ti_usb_3410_5052.c index 3010878f7f8e..e3c5832337e0 100644 --- a/drivers/usb/serial/ti_usb_3410_5052.c +++ b/drivers/usb/serial/ti_usb_3410_5052.c @@ -1119,7 +1119,7 @@ static void ti_break(struct tty_struct *tty, int break_state) static int ti_get_port_from_code(unsigned char code) { - return (code >> 4) - 3; + return (code >> 6) & 0x01; } static int ti_get_func_from_code(unsigned char code) diff --git a/drivers/usb/storage/scsiglue.c b/drivers/usb/storage/scsiglue.c index c267f2812a04..e227bb5b794f 100644 --- a/drivers/usb/storage/scsiglue.c +++ b/drivers/usb/storage/scsiglue.c @@ -376,6 +376,15 @@ static int queuecommand_lck(struct scsi_cmnd *srb, return 0; } + if ((us->fflags & US_FL_NO_ATA_1X) && + (srb->cmnd[0] == ATA_12 || srb->cmnd[0] == ATA_16)) { + memcpy(srb->sense_buffer, usb_stor_sense_invalidCDB, + sizeof(usb_stor_sense_invalidCDB)); + srb->result = SAM_STAT_CHECK_CONDITION; + done(srb); + return 0; + } + /* enqueue the command and wake up the control thread */ srb->scsi_done = done; us->srb = srb; diff --git a/drivers/usb/storage/uas.c b/drivers/usb/storage/uas.c index 9e9de5452860..1f7b401c4d04 100644 --- a/drivers/usb/storage/uas.c +++ b/drivers/usb/storage/uas.c @@ -842,6 +842,27 @@ static int uas_slave_configure(struct scsi_device *sdev) sdev->skip_ms_page_8 = 1; sdev->wce_default_on = 1; } + + /* + * Some disks return the total number of blocks in response + * to READ CAPACITY rather than the highest block number. + * If this device makes that mistake, tell the sd driver. + */ + if (devinfo->flags & US_FL_FIX_CAPACITY) + sdev->fix_capacity = 1; + + /* + * Some devices don't like MODE SENSE with page=0x3f, + * which is the command used for checking if a device + * is write-protected. Now that we tell the sd driver + * to do a 192-byte transfer with this command the + * majority of devices work fine, but a few still can't + * handle it. The sd driver will simply assume those + * devices are write-enabled. + */ + if (devinfo->flags & US_FL_NO_WP_DETECT) + sdev->skip_ms_page_3f = 1; + scsi_change_queue_depth(sdev, devinfo->qdepth - 2); return 0; } diff --git a/drivers/usb/storage/unusual_devs.h b/drivers/usb/storage/unusual_devs.h index 22fcfccf453a..f7f83b21dc74 100644 --- a/drivers/usb/storage/unusual_devs.h +++ b/drivers/usb/storage/unusual_devs.h @@ -2288,6 +2288,13 @@ UNUSUAL_DEV( 0x2735, 0x100b, 0x0000, 0x9999, USB_SC_DEVICE, USB_PR_DEVICE, NULL, US_FL_GO_SLOW ), +/* Reported-by: Tim Anderson <tsa@biglakesoftware.com> */ +UNUSUAL_DEV( 0x2ca3, 0x0031, 0x0000, 0x9999, + "DJI", + "CineSSD", + USB_SC_DEVICE, USB_PR_DEVICE, NULL, + US_FL_NO_ATA_1X), + /* * Reported by Frederic Marchal <frederic.marchal@wowcompany.com> * Mio Moov 330 diff --git a/drivers/usb/typec/bus.c b/drivers/usb/typec/bus.c index 95a2b10127db..76299b6ff06d 100644 --- a/drivers/usb/typec/bus.c +++ b/drivers/usb/typec/bus.c @@ -255,12 +255,13 @@ EXPORT_SYMBOL_GPL(typec_altmode_unregister_driver); /* API for the port drivers */ /** - * typec_match_altmode - Match SVID to an array of alternate modes + * typec_match_altmode - Match SVID and mode to an array of alternate modes * @altmodes: Array of alternate modes - * @n: Number of elements in the array, or -1 for NULL termiated arrays + * @n: Number of elements in the array, or -1 for NULL terminated arrays * @svid: Standard or Vendor ID to match with + * @mode: Mode to match with * - * Return pointer to an alternate mode with SVID mathing @svid, or NULL when no + * Return pointer to an alternate mode with SVID matching @svid, or NULL when no * match is found. */ struct typec_altmode *typec_match_altmode(struct typec_altmode **altmodes, diff --git a/drivers/usb/typec/class.c b/drivers/usb/typec/class.c index c202975f8097..e61dffb27a0c 100644 --- a/drivers/usb/typec/class.c +++ b/drivers/usb/typec/class.c @@ -1484,7 +1484,6 @@ EXPORT_SYMBOL_GPL(typec_set_mode); * typec_port_register_altmode - Register USB Type-C Port Alternate Mode * @port: USB Type-C Port that supports the alternate mode * @desc: Description of the alternate mode - * @drvdata: Private pointer to driver specific info * * This routine is used to register an alternate mode that @port is capable of * supporting. diff --git a/drivers/usb/typec/mux.c b/drivers/usb/typec/mux.c index ddaac63ecf12..d990aa510fab 100644 --- a/drivers/usb/typec/mux.c +++ b/drivers/usb/typec/mux.c @@ -9,6 +9,7 @@ #include <linux/device.h> #include <linux/list.h> +#include <linux/module.h> #include <linux/mutex.h> #include <linux/usb/typec_mux.h> @@ -49,8 +50,10 @@ struct typec_switch *typec_switch_get(struct device *dev) mutex_lock(&switch_lock); sw = device_connection_find_match(dev, "typec-switch", NULL, typec_switch_match); - if (!IS_ERR_OR_NULL(sw)) + if (!IS_ERR_OR_NULL(sw)) { + WARN_ON(!try_module_get(sw->dev->driver->owner)); get_device(sw->dev); + } mutex_unlock(&switch_lock); return sw; @@ -65,8 +68,10 @@ EXPORT_SYMBOL_GPL(typec_switch_get); */ void typec_switch_put(struct typec_switch *sw) { - if (!IS_ERR_OR_NULL(sw)) + if (!IS_ERR_OR_NULL(sw)) { + module_put(sw->dev->driver->owner); put_device(sw->dev); + } } EXPORT_SYMBOL_GPL(typec_switch_put); @@ -136,8 +141,10 @@ struct typec_mux *typec_mux_get(struct device *dev, const char *name) mutex_lock(&mux_lock); mux = device_connection_find_match(dev, name, NULL, typec_mux_match); - if (!IS_ERR_OR_NULL(mux)) + if (!IS_ERR_OR_NULL(mux)) { + WARN_ON(!try_module_get(mux->dev->driver->owner)); get_device(mux->dev); + } mutex_unlock(&mux_lock); return mux; @@ -152,8 +159,10 @@ EXPORT_SYMBOL_GPL(typec_mux_get); */ void typec_mux_put(struct typec_mux *mux) { - if (!IS_ERR_OR_NULL(mux)) + if (!IS_ERR_OR_NULL(mux)) { + module_put(mux->dev->driver->owner); put_device(mux->dev); + } } EXPORT_SYMBOL_GPL(typec_mux_put); diff --git a/drivers/xen/Kconfig b/drivers/xen/Kconfig index b459edfacff3..90d387b50ab7 100644 --- a/drivers/xen/Kconfig +++ b/drivers/xen/Kconfig @@ -79,15 +79,19 @@ config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT This value is used to allocate enough space in internal tables needed for physical memory administration. -config XEN_SCRUB_PAGES - bool "Scrub pages before returning them to system" +config XEN_SCRUB_PAGES_DEFAULT + bool "Scrub pages before returning them to system by default" depends on XEN_BALLOON default y help Scrub pages before returning them to the system for reuse by other domains. This makes sure that any confidential data is not accidentally visible to other domains. Is it more - secure, but slightly less efficient. + secure, but slightly less efficient. This can be controlled with + xen_scrub_pages=0 parameter and + /sys/devices/system/xen_memory/xen_memory0/scrub_pages. + This option only sets the default value. + If in doubt, say yes. config XEN_DEV_EVTCHN diff --git a/drivers/xen/cpu_hotplug.c b/drivers/xen/cpu_hotplug.c index d4265c8ebb22..b1357aa4bc55 100644 --- a/drivers/xen/cpu_hotplug.c +++ b/drivers/xen/cpu_hotplug.c @@ -19,15 +19,16 @@ static void enable_hotplug_cpu(int cpu) static void disable_hotplug_cpu(int cpu) { - if (cpu_online(cpu)) { - lock_device_hotplug(); + if (!cpu_is_hotpluggable(cpu)) + return; + lock_device_hotplug(); + if (cpu_online(cpu)) device_offline(get_cpu_device(cpu)); - unlock_device_hotplug(); - } - if (cpu_present(cpu)) + if (!cpu_online(cpu) && cpu_present(cpu)) { xen_arch_unregister_cpu(cpu); - - set_cpu_present(cpu, false); + set_cpu_present(cpu, false); + } + unlock_device_hotplug(); } static int vcpu_online(unsigned int cpu) diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c index 08e4af04d6f2..e6c1934734b7 100644 --- a/drivers/xen/events/events_base.c +++ b/drivers/xen/events/events_base.c @@ -138,7 +138,7 @@ static int set_evtchn_to_irq(unsigned evtchn, unsigned irq) clear_evtchn_to_irq_row(row); } - evtchn_to_irq[EVTCHN_ROW(evtchn)][EVTCHN_COL(evtchn)] = irq; + evtchn_to_irq[row][col] = irq; return 0; } diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c index 57390c7666e5..b0b02a501167 100644 --- a/drivers/xen/gntdev.c +++ b/drivers/xen/gntdev.c @@ -492,12 +492,19 @@ static bool in_range(struct gntdev_grant_map *map, return true; } -static void unmap_if_in_range(struct gntdev_grant_map *map, - unsigned long start, unsigned long end) +static int unmap_if_in_range(struct gntdev_grant_map *map, + unsigned long start, unsigned long end, + bool blockable) { unsigned long mstart, mend; int err; + if (!in_range(map, start, end)) + return 0; + + if (!blockable) + return -EAGAIN; + mstart = max(start, map->vma->vm_start); mend = min(end, map->vma->vm_end); pr_debug("map %d+%d (%lx %lx), range %lx %lx, mrange %lx %lx\n", @@ -508,6 +515,8 @@ static void unmap_if_in_range(struct gntdev_grant_map *map, (mstart - map->vma->vm_start) >> PAGE_SHIFT, (mend - mstart) >> PAGE_SHIFT); WARN_ON(err); + + return 0; } static int mn_invl_range_start(struct mmu_notifier *mn, @@ -519,25 +528,20 @@ static int mn_invl_range_start(struct mmu_notifier *mn, struct gntdev_grant_map *map; int ret = 0; - /* TODO do we really need a mutex here? */ if (blockable) mutex_lock(&priv->lock); else if (!mutex_trylock(&priv->lock)) return -EAGAIN; list_for_each_entry(map, &priv->maps, next) { - if (in_range(map, start, end)) { - ret = -EAGAIN; + ret = unmap_if_in_range(map, start, end, blockable); + if (ret) goto out_unlock; - } - unmap_if_in_range(map, start, end); } list_for_each_entry(map, &priv->freeable_maps, next) { - if (in_range(map, start, end)) { - ret = -EAGAIN; + ret = unmap_if_in_range(map, start, end, blockable); + if (ret) goto out_unlock; - } - unmap_if_in_range(map, start, end); } out_unlock: diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c index 7bafa703a992..84575baceebc 100644 --- a/drivers/xen/grant-table.c +++ b/drivers/xen/grant-table.c @@ -1040,18 +1040,33 @@ int gnttab_map_refs(struct gnttab_map_grant_ref *map_ops, return ret; for (i = 0; i < count; i++) { - /* Retry eagain maps */ - if (map_ops[i].status == GNTST_eagain) - gnttab_retry_eagain_gop(GNTTABOP_map_grant_ref, map_ops + i, - &map_ops[i].status, __func__); - - if (map_ops[i].status == GNTST_okay) { + switch (map_ops[i].status) { + case GNTST_okay: + { struct xen_page_foreign *foreign; SetPageForeign(pages[i]); foreign = xen_page_foreign(pages[i]); foreign->domid = map_ops[i].dom; foreign->gref = map_ops[i].ref; + break; + } + + case GNTST_no_device_space: + pr_warn_ratelimited("maptrack limit reached, can't map all guest pages\n"); + break; + + case GNTST_eagain: + /* Retry eagain maps */ + gnttab_retry_eagain_gop(GNTTABOP_map_grant_ref, + map_ops + i, + &map_ops[i].status, __func__); + /* Test status in next loop iteration. */ + i--; + break; + + default: + break; } } diff --git a/drivers/xen/manage.c b/drivers/xen/manage.c index c93d8ef8df34..5bb01a62f214 100644 --- a/drivers/xen/manage.c +++ b/drivers/xen/manage.c @@ -280,9 +280,11 @@ static void sysrq_handler(struct xenbus_watch *watch, const char *path, /* * The Xenstore watch fires directly after registering it and * after a suspend/resume cycle. So ENOENT is no error but - * might happen in those cases. + * might happen in those cases. ERANGE is observed when we get + * an empty value (''), this happens when we acknowledge the + * request by writing '\0' below. */ - if (err != -ENOENT) + if (err != -ENOENT && err != -ERANGE) pr_err("Error %d reading sysrq code in control/sysrq\n", err); xenbus_transaction_end(xbt, 1); diff --git a/drivers/xen/mem-reservation.c b/drivers/xen/mem-reservation.c index 084799c6180e..3782cf070338 100644 --- a/drivers/xen/mem-reservation.c +++ b/drivers/xen/mem-reservation.c @@ -14,6 +14,10 @@ #include <xen/interface/memory.h> #include <xen/mem-reservation.h> +#include <linux/moduleparam.h> + +bool __read_mostly xen_scrub_pages = IS_ENABLED(CONFIG_XEN_SCRUB_PAGES_DEFAULT); +core_param(xen_scrub_pages, xen_scrub_pages, bool, 0); /* * Use one extent per PAGE_SIZE to avoid to break down the page into diff --git a/drivers/xen/xen-balloon.c b/drivers/xen/xen-balloon.c index 294f35ce9e46..63c1494a8d73 100644 --- a/drivers/xen/xen-balloon.c +++ b/drivers/xen/xen-balloon.c @@ -44,6 +44,7 @@ #include <xen/xenbus.h> #include <xen/features.h> #include <xen/page.h> +#include <xen/mem-reservation.h> #define PAGES2KB(_p) ((_p)<<(PAGE_SHIFT-10)) @@ -137,6 +138,7 @@ static DEVICE_ULONG_ATTR(schedule_delay, 0444, balloon_stats.schedule_delay); static DEVICE_ULONG_ATTR(max_schedule_delay, 0644, balloon_stats.max_schedule_delay); static DEVICE_ULONG_ATTR(retry_count, 0444, balloon_stats.retry_count); static DEVICE_ULONG_ATTR(max_retry_count, 0644, balloon_stats.max_retry_count); +static DEVICE_BOOL_ATTR(scrub_pages, 0644, xen_scrub_pages); static ssize_t show_target_kb(struct device *dev, struct device_attribute *attr, char *buf) @@ -203,6 +205,7 @@ static struct attribute *balloon_attrs[] = { &dev_attr_max_schedule_delay.attr.attr, &dev_attr_retry_count.attr.attr, &dev_attr_max_retry_count.attr.attr, + &dev_attr_scrub_pages.attr.attr, NULL }; diff --git a/fs/afs/proc.c b/fs/afs/proc.c index 0c3285c8db95..476dcbb79713 100644 --- a/fs/afs/proc.c +++ b/fs/afs/proc.c @@ -98,13 +98,13 @@ static int afs_proc_cells_write(struct file *file, char *buf, size_t size) goto inval; args = strchr(name, ' '); - if (!args) - goto inval; - do { - *args++ = 0; - } while(*args == ' '); - if (!*args) - goto inval; + if (args) { + do { + *args++ = 0; + } while(*args == ' '); + if (!*args) + goto inval; + } /* determine command to perform */ _debug("cmd=%s name=%s args=%s", buf, name, args); @@ -120,7 +120,6 @@ static int afs_proc_cells_write(struct file *file, char *buf, size_t size) if (test_and_set_bit(AFS_CELL_FL_NO_GC, &cell->flags)) afs_put_cell(net, cell); - printk("kAFS: Added new cell '%s'\n", name); } else { goto inval; } diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 53af9f5253f4..2cddfe7806a4 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h @@ -1280,6 +1280,7 @@ struct btrfs_root { int send_in_progress; struct btrfs_subvolume_writers *subv_writers; atomic_t will_be_snapshotted; + atomic_t snapshot_force_cow; /* For qgroup metadata reserved space */ spinlock_t qgroup_meta_rsv_lock; @@ -3390,9 +3391,9 @@ do { \ #define btrfs_debug(fs_info, fmt, args...) \ btrfs_no_printk(fs_info, KERN_DEBUG fmt, ##args) #define btrfs_debug_in_rcu(fs_info, fmt, args...) \ - btrfs_no_printk(fs_info, KERN_DEBUG fmt, ##args) + btrfs_no_printk_in_rcu(fs_info, KERN_DEBUG fmt, ##args) #define btrfs_debug_rl_in_rcu(fs_info, fmt, args...) \ - btrfs_no_printk(fs_info, KERN_DEBUG fmt, ##args) + btrfs_no_printk_in_rcu(fs_info, KERN_DEBUG fmt, ##args) #define btrfs_debug_rl(fs_info, fmt, args...) \ btrfs_no_printk(fs_info, KERN_DEBUG fmt, ##args) #endif @@ -3404,6 +3405,13 @@ do { \ rcu_read_unlock(); \ } while (0) +#define btrfs_no_printk_in_rcu(fs_info, fmt, args...) \ +do { \ + rcu_read_lock(); \ + btrfs_no_printk(fs_info, fmt, ##args); \ + rcu_read_unlock(); \ +} while (0) + #define btrfs_printk_ratelimited(fs_info, fmt, args...) \ do { \ static DEFINE_RATELIMIT_STATE(_rs, \ diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 5124c15705ce..05dc3c17cb62 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -1187,6 +1187,7 @@ static void __setup_root(struct btrfs_root *root, struct btrfs_fs_info *fs_info, atomic_set(&root->log_batch, 0); refcount_set(&root->refs, 1); atomic_set(&root->will_be_snapshotted, 0); + atomic_set(&root->snapshot_force_cow, 0); root->log_transid = 0; root->log_transid_committed = -1; root->last_log_commit = 0; diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index de6f75f5547b..2d9074295d7f 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5800,7 +5800,7 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans) * root: the root of the parent directory * rsv: block reservation * items: the number of items that we need do reservation - * qgroup_reserved: used to return the reserved size in qgroup + * use_global_rsv: allow fallback to the global block reservation * * This function is used to reserve the space for snapshot/subvolume * creation and deletion. Those operations are different with the @@ -5810,10 +5810,10 @@ void btrfs_trans_release_chunk_metadata(struct btrfs_trans_handle *trans) * the space reservation mechanism in start_transaction(). */ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root, - struct btrfs_block_rsv *rsv, - int items, + struct btrfs_block_rsv *rsv, int items, bool use_global_rsv) { + u64 qgroup_num_bytes = 0; u64 num_bytes; int ret; struct btrfs_fs_info *fs_info = root->fs_info; @@ -5821,12 +5821,11 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root, if (test_bit(BTRFS_FS_QUOTA_ENABLED, &fs_info->flags)) { /* One for parent inode, two for dir entries */ - num_bytes = 3 * fs_info->nodesize; - ret = btrfs_qgroup_reserve_meta_prealloc(root, num_bytes, true); + qgroup_num_bytes = 3 * fs_info->nodesize; + ret = btrfs_qgroup_reserve_meta_prealloc(root, + qgroup_num_bytes, true); if (ret) return ret; - } else { - num_bytes = 0; } num_bytes = btrfs_calc_trans_metadata_size(fs_info, items); @@ -5838,8 +5837,8 @@ int btrfs_subvolume_reserve_metadata(struct btrfs_root *root, if (ret == -ENOSPC && use_global_rsv) ret = btrfs_block_rsv_migrate(global_rsv, rsv, num_bytes, 1); - if (ret && num_bytes) - btrfs_qgroup_free_meta_prealloc(root, num_bytes); + if (ret && qgroup_num_bytes) + btrfs_qgroup_free_meta_prealloc(root, qgroup_num_bytes); return ret; } diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 9357a19d2bff..3ea5339603cf 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -1271,7 +1271,7 @@ static noinline int run_delalloc_nocow(struct inode *inode, u64 disk_num_bytes; u64 ram_bytes; int extent_type; - int ret, err; + int ret; int type; int nocow; int check_prev = 1; @@ -1403,11 +1403,8 @@ next_slot: * if there are pending snapshots for this root, * we fall into common COW way. */ - if (!nolock) { - err = btrfs_start_write_no_snapshotting(root); - if (!err) - goto out_check; - } + if (!nolock && atomic_read(&root->snapshot_force_cow)) + goto out_check; /* * force cow if csum exists in the range. * this ensure that csum for a given extent are @@ -1416,9 +1413,6 @@ next_slot: ret = csum_exist_in_range(fs_info, disk_bytenr, num_bytes); if (ret) { - if (!nolock) - btrfs_end_write_no_snapshotting(root); - /* * ret could be -EIO if the above fails to read * metadata. @@ -1431,11 +1425,8 @@ next_slot: WARN_ON_ONCE(nolock); goto out_check; } - if (!btrfs_inc_nocow_writers(fs_info, disk_bytenr)) { - if (!nolock) - btrfs_end_write_no_snapshotting(root); + if (!btrfs_inc_nocow_writers(fs_info, disk_bytenr)) goto out_check; - } nocow = 1; } else if (extent_type == BTRFS_FILE_EXTENT_INLINE) { extent_end = found_key.offset + @@ -1448,8 +1439,6 @@ next_slot: out_check: if (extent_end <= start) { path->slots[0]++; - if (!nolock && nocow) - btrfs_end_write_no_snapshotting(root); if (nocow) btrfs_dec_nocow_writers(fs_info, disk_bytenr); goto next_slot; @@ -1471,8 +1460,6 @@ out_check: end, page_started, nr_written, 1, NULL); if (ret) { - if (!nolock && nocow) - btrfs_end_write_no_snapshotting(root); if (nocow) btrfs_dec_nocow_writers(fs_info, disk_bytenr); @@ -1492,8 +1479,6 @@ out_check: ram_bytes, BTRFS_COMPRESS_NONE, BTRFS_ORDERED_PREALLOC); if (IS_ERR(em)) { - if (!nolock && nocow) - btrfs_end_write_no_snapshotting(root); if (nocow) btrfs_dec_nocow_writers(fs_info, disk_bytenr); @@ -1532,8 +1517,6 @@ out_check: EXTENT_CLEAR_DATA_RESV, PAGE_UNLOCK | PAGE_SET_PRIVATE2); - if (!nolock && nocow) - btrfs_end_write_no_snapshotting(root); cur_offset = extent_end; /* @@ -6639,6 +6622,8 @@ static int btrfs_link(struct dentry *old_dentry, struct inode *dir, drop_inode = 1; } else { struct dentry *parent = dentry->d_parent; + int ret; + err = btrfs_update_inode(trans, root, inode); if (err) goto fail; @@ -6652,7 +6637,12 @@ static int btrfs_link(struct dentry *old_dentry, struct inode *dir, goto fail; } d_instantiate(dentry, inode); - btrfs_log_new_name(trans, BTRFS_I(inode), NULL, parent); + ret = btrfs_log_new_name(trans, BTRFS_I(inode), NULL, parent, + true, NULL); + if (ret == BTRFS_NEED_TRANS_COMMIT) { + err = btrfs_commit_transaction(trans); + trans = NULL; + } } fail: @@ -9388,14 +9378,21 @@ static int btrfs_rename_exchange(struct inode *old_dir, u64 new_idx = 0; u64 root_objectid; int ret; - int ret2; bool root_log_pinned = false; bool dest_log_pinned = false; + struct btrfs_log_ctx ctx_root; + struct btrfs_log_ctx ctx_dest; + bool sync_log_root = false; + bool sync_log_dest = false; + bool commit_transaction = false; /* we only allow rename subvolume link between subvolumes */ if (old_ino != BTRFS_FIRST_FREE_OBJECTID && root != dest) return -EXDEV; + btrfs_init_log_ctx(&ctx_root, old_inode); + btrfs_init_log_ctx(&ctx_dest, new_inode); + /* close the race window with snapshot create/destroy ioctl */ if (old_ino == BTRFS_FIRST_FREE_OBJECTID) down_read(&fs_info->subvol_sem); @@ -9542,15 +9539,29 @@ static int btrfs_rename_exchange(struct inode *old_dir, if (root_log_pinned) { parent = new_dentry->d_parent; - btrfs_log_new_name(trans, BTRFS_I(old_inode), BTRFS_I(old_dir), - parent); + ret = btrfs_log_new_name(trans, BTRFS_I(old_inode), + BTRFS_I(old_dir), parent, + false, &ctx_root); + if (ret == BTRFS_NEED_LOG_SYNC) + sync_log_root = true; + else if (ret == BTRFS_NEED_TRANS_COMMIT) + commit_transaction = true; + ret = 0; btrfs_end_log_trans(root); root_log_pinned = false; } if (dest_log_pinned) { - parent = old_dentry->d_parent; - btrfs_log_new_name(trans, BTRFS_I(new_inode), BTRFS_I(new_dir), - parent); + if (!commit_transaction) { + parent = old_dentry->d_parent; + ret = btrfs_log_new_name(trans, BTRFS_I(new_inode), + BTRFS_I(new_dir), parent, + false, &ctx_dest); + if (ret == BTRFS_NEED_LOG_SYNC) + sync_log_dest = true; + else if (ret == BTRFS_NEED_TRANS_COMMIT) + commit_transaction = true; + ret = 0; + } btrfs_end_log_trans(dest); dest_log_pinned = false; } @@ -9583,8 +9594,26 @@ out_fail: dest_log_pinned = false; } } - ret2 = btrfs_end_transaction(trans); - ret = ret ? ret : ret2; + if (!ret && sync_log_root && !commit_transaction) { + ret = btrfs_sync_log(trans, BTRFS_I(old_inode)->root, + &ctx_root); + if (ret) + commit_transaction = true; + } + if (!ret && sync_log_dest && !commit_transaction) { + ret = btrfs_sync_log(trans, BTRFS_I(new_inode)->root, + &ctx_dest); + if (ret) + commit_transaction = true; + } + if (commit_transaction) { + ret = btrfs_commit_transaction(trans); + } else { + int ret2; + + ret2 = btrfs_end_transaction(trans); + ret = ret ? ret : ret2; + } out_notrans: if (new_ino == BTRFS_FIRST_FREE_OBJECTID) up_read(&fs_info->subvol_sem); @@ -9661,6 +9690,9 @@ static int btrfs_rename(struct inode *old_dir, struct dentry *old_dentry, int ret; u64 old_ino = btrfs_ino(BTRFS_I(old_inode)); bool log_pinned = false; + struct btrfs_log_ctx ctx; + bool sync_log = false; + bool commit_transaction = false; if (btrfs_ino(BTRFS_I(new_dir)) == BTRFS_EMPTY_SUBVOL_DIR_OBJECTID) return -EPERM; @@ -9818,8 +9850,15 @@ static int btrfs_rename(struct inode *old_dir, struct dentry *old_dentry, if (log_pinned) { struct dentry *parent = new_dentry->d_parent; - btrfs_log_new_name(trans, BTRFS_I(old_inode), BTRFS_I(old_dir), - parent); + btrfs_init_log_ctx(&ctx, old_inode); + ret = btrfs_log_new_name(trans, BTRFS_I(old_inode), + BTRFS_I(old_dir), parent, + false, &ctx); + if (ret == BTRFS_NEED_LOG_SYNC) + sync_log = true; + else if (ret == BTRFS_NEED_TRANS_COMMIT) + commit_transaction = true; + ret = 0; btrfs_end_log_trans(root); log_pinned = false; } @@ -9856,7 +9895,19 @@ out_fail: btrfs_end_log_trans(root); log_pinned = false; } - btrfs_end_transaction(trans); + if (!ret && sync_log) { + ret = btrfs_sync_log(trans, BTRFS_I(old_inode)->root, &ctx); + if (ret) + commit_transaction = true; + } + if (commit_transaction) { + ret = btrfs_commit_transaction(trans); + } else { + int ret2; + + ret2 = btrfs_end_transaction(trans); + ret = ret ? ret : ret2; + } out_notrans: if (old_ino == BTRFS_FIRST_FREE_OBJECTID) up_read(&fs_info->subvol_sem); diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 63600dc2ac4c..d60b6caf09e8 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -747,6 +747,7 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir, struct btrfs_pending_snapshot *pending_snapshot; struct btrfs_trans_handle *trans; int ret; + bool snapshot_force_cow = false; if (!test_bit(BTRFS_ROOT_REF_COWS, &root->state)) return -EINVAL; @@ -763,6 +764,11 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir, goto free_pending; } + /* + * Force new buffered writes to reserve space even when NOCOW is + * possible. This is to avoid later writeback (running dealloc) to + * fallback to COW mode and unexpectedly fail with ENOSPC. + */ atomic_inc(&root->will_be_snapshotted); smp_mb__after_atomic(); /* wait for no snapshot writes */ @@ -773,6 +779,14 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir, if (ret) goto dec_and_free; + /* + * All previous writes have started writeback in NOCOW mode, so now + * we force future writes to fallback to COW mode during snapshot + * creation. + */ + atomic_inc(&root->snapshot_force_cow); + snapshot_force_cow = true; + btrfs_wait_ordered_extents(root, U64_MAX, 0, (u64)-1); btrfs_init_block_rsv(&pending_snapshot->block_rsv, @@ -837,6 +851,8 @@ static int create_snapshot(struct btrfs_root *root, struct inode *dir, fail: btrfs_subvolume_release_metadata(fs_info, &pending_snapshot->block_rsv); dec_and_free: + if (snapshot_force_cow) + atomic_dec(&root->snapshot_force_cow); if (atomic_dec_and_test(&root->will_be_snapshotted)) wake_up_var(&root->will_be_snapshotted); free_pending: @@ -3453,6 +3469,25 @@ static int btrfs_extent_same_range(struct inode *src, u64 loff, u64 olen, same_lock_start = min_t(u64, loff, dst_loff); same_lock_len = max_t(u64, loff, dst_loff) + len - same_lock_start; + } else { + /* + * If the source and destination inodes are different, the + * source's range end offset matches the source's i_size, that + * i_size is not a multiple of the sector size, and the + * destination range does not go past the destination's i_size, + * we must round down the length to the nearest sector size + * multiple. If we don't do this adjustment we end replacing + * with zeroes the bytes in the range that starts at the + * deduplication range's end offset and ends at the next sector + * size multiple. + */ + if (loff + olen == i_size_read(src) && + dst_loff + len < i_size_read(dst)) { + const u64 sz = BTRFS_I(src)->root->fs_info->sectorsize; + + len = round_down(i_size_read(src), sz) - loff; + olen = len; + } } again: diff --git a/fs/btrfs/qgroup.c b/fs/btrfs/qgroup.c index 4353bb69bb86..d4917c0cddf5 100644 --- a/fs/btrfs/qgroup.c +++ b/fs/btrfs/qgroup.c @@ -1019,10 +1019,9 @@ out_add_root: spin_unlock(&fs_info->qgroup_lock); ret = btrfs_commit_transaction(trans); - if (ret) { - trans = NULL; + trans = NULL; + if (ret) goto out_free_path; - } ret = qgroup_rescan_init(fs_info, 0, 1); if (!ret) { diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 1650dc44a5e3..3c2ae0e4f25a 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -6025,14 +6025,25 @@ void btrfs_record_snapshot_destroy(struct btrfs_trans_handle *trans, * Call this after adding a new name for a file and it will properly * update the log to reflect the new name. * - * It will return zero if all goes well, and it will return 1 if a - * full transaction commit is required. + * @ctx can not be NULL when @sync_log is false, and should be NULL when it's + * true (because it's not used). + * + * Return value depends on whether @sync_log is true or false. + * When true: returns BTRFS_NEED_TRANS_COMMIT if the transaction needs to be + * committed by the caller, and BTRFS_DONT_NEED_TRANS_COMMIT + * otherwise. + * When false: returns BTRFS_DONT_NEED_LOG_SYNC if the caller does not need to + * to sync the log, BTRFS_NEED_LOG_SYNC if it needs to sync the log, + * or BTRFS_NEED_TRANS_COMMIT if the transaction needs to be + * committed (without attempting to sync the log). */ int btrfs_log_new_name(struct btrfs_trans_handle *trans, struct btrfs_inode *inode, struct btrfs_inode *old_dir, - struct dentry *parent) + struct dentry *parent, + bool sync_log, struct btrfs_log_ctx *ctx) { struct btrfs_fs_info *fs_info = trans->fs_info; + int ret; /* * this will force the logging code to walk the dentry chain @@ -6047,9 +6058,34 @@ int btrfs_log_new_name(struct btrfs_trans_handle *trans, */ if (inode->logged_trans <= fs_info->last_trans_committed && (!old_dir || old_dir->logged_trans <= fs_info->last_trans_committed)) - return 0; + return sync_log ? BTRFS_DONT_NEED_TRANS_COMMIT : + BTRFS_DONT_NEED_LOG_SYNC; + + if (sync_log) { + struct btrfs_log_ctx ctx2; + + btrfs_init_log_ctx(&ctx2, &inode->vfs_inode); + ret = btrfs_log_inode_parent(trans, inode, parent, 0, LLONG_MAX, + LOG_INODE_EXISTS, &ctx2); + if (ret == BTRFS_NO_LOG_SYNC) + return BTRFS_DONT_NEED_TRANS_COMMIT; + else if (ret) + return BTRFS_NEED_TRANS_COMMIT; + + ret = btrfs_sync_log(trans, inode->root, &ctx2); + if (ret) + return BTRFS_NEED_TRANS_COMMIT; + return BTRFS_DONT_NEED_TRANS_COMMIT; + } + + ASSERT(ctx); + ret = btrfs_log_inode_parent(trans, inode, parent, 0, LLONG_MAX, + LOG_INODE_EXISTS, ctx); + if (ret == BTRFS_NO_LOG_SYNC) + return BTRFS_DONT_NEED_LOG_SYNC; + else if (ret) + return BTRFS_NEED_TRANS_COMMIT; - return btrfs_log_inode_parent(trans, inode, parent, 0, LLONG_MAX, - LOG_INODE_EXISTS, NULL); + return BTRFS_NEED_LOG_SYNC; } diff --git a/fs/btrfs/tree-log.h b/fs/btrfs/tree-log.h index 122e68b89a5a..7ab9bb88a639 100644 --- a/fs/btrfs/tree-log.h +++ b/fs/btrfs/tree-log.h @@ -71,8 +71,16 @@ void btrfs_record_unlink_dir(struct btrfs_trans_handle *trans, int for_rename); void btrfs_record_snapshot_destroy(struct btrfs_trans_handle *trans, struct btrfs_inode *dir); +/* Return values for btrfs_log_new_name() */ +enum { + BTRFS_DONT_NEED_TRANS_COMMIT, + BTRFS_NEED_TRANS_COMMIT, + BTRFS_DONT_NEED_LOG_SYNC, + BTRFS_NEED_LOG_SYNC, +}; int btrfs_log_new_name(struct btrfs_trans_handle *trans, struct btrfs_inode *inode, struct btrfs_inode *old_dir, - struct dentry *parent); + struct dentry *parent, + bool sync_log, struct btrfs_log_ctx *ctx); #endif diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index da86706123ff..f4405e430da6 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -4491,7 +4491,12 @@ again: /* Now btrfs_update_device() will change the on-disk size. */ ret = btrfs_update_device(trans, device); - btrfs_end_transaction(trans); + if (ret < 0) { + btrfs_abort_transaction(trans, ret); + btrfs_end_transaction(trans); + } else { + ret = btrfs_commit_transaction(trans); + } done: btrfs_free_path(path); if (ret) { diff --git a/fs/ceph/super.c b/fs/ceph/super.c index 43ca3b763875..eab1359d0553 100644 --- a/fs/ceph/super.c +++ b/fs/ceph/super.c @@ -602,6 +602,8 @@ static int extra_mon_dispatch(struct ceph_client *client, struct ceph_msg *msg) /* * create a new fs client + * + * Success or not, this function consumes @fsopt and @opt. */ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, struct ceph_options *opt) @@ -609,17 +611,20 @@ static struct ceph_fs_client *create_fs_client(struct ceph_mount_options *fsopt, struct ceph_fs_client *fsc; int page_count; size_t size; - int err = -ENOMEM; + int err; fsc = kzalloc(sizeof(*fsc), GFP_KERNEL); - if (!fsc) - return ERR_PTR(-ENOMEM); + if (!fsc) { + err = -ENOMEM; + goto fail; + } fsc->client = ceph_create_client(opt, fsc); if (IS_ERR(fsc->client)) { err = PTR_ERR(fsc->client); goto fail; } + opt = NULL; /* fsc->client now owns this */ fsc->client->extra_mon_dispatch = extra_mon_dispatch; fsc->client->osdc.abort_on_full = true; @@ -677,6 +682,9 @@ fail_client: ceph_destroy_client(fsc->client); fail: kfree(fsc); + if (opt) + ceph_destroy_options(opt); + destroy_mount_options(fsopt); return ERR_PTR(err); } @@ -1042,8 +1050,6 @@ static struct dentry *ceph_mount(struct file_system_type *fs_type, fsc = create_fs_client(fsopt, opt); if (IS_ERR(fsc)) { res = ERR_CAST(fsc); - destroy_mount_options(fsopt); - ceph_destroy_options(opt); goto out_final; } diff --git a/fs/cifs/Kconfig b/fs/cifs/Kconfig index 35c83fe7dba0..abcd78e332fe 100644 --- a/fs/cifs/Kconfig +++ b/fs/cifs/Kconfig @@ -6,6 +6,7 @@ config CIFS select CRYPTO_MD4 select CRYPTO_MD5 select CRYPTO_SHA256 + select CRYPTO_SHA512 select CRYPTO_CMAC select CRYPTO_HMAC select CRYPTO_ARC4 diff --git a/fs/cifs/cifs_unicode.c b/fs/cifs/cifs_unicode.c index b380e0871372..a2b2355e7f01 100644 --- a/fs/cifs/cifs_unicode.c +++ b/fs/cifs/cifs_unicode.c @@ -105,9 +105,6 @@ convert_sfm_char(const __u16 src_char, char *target) case SFM_LESSTHAN: *target = '<'; break; - case SFM_SLASH: - *target = '\\'; - break; case SFM_SPACE: *target = ' '; break; diff --git a/fs/cifs/cifssmb.c b/fs/cifs/cifssmb.c index dc2f4cf08fe9..5657b79dbc99 100644 --- a/fs/cifs/cifssmb.c +++ b/fs/cifs/cifssmb.c @@ -601,10 +601,15 @@ CIFSSMBNegotiate(const unsigned int xid, struct cifs_ses *ses) } count = 0; + /* + * We know that all the name entries in the protocols array + * are short (< 16 bytes anyway) and are NUL terminated. + */ for (i = 0; i < CIFS_NUM_PROT; i++) { - strncpy(pSMB->DialectsArray+count, protocols[i].name, 16); - count += strlen(protocols[i].name) + 1; - /* null at end of source and target buffers anyway */ + size_t len = strlen(protocols[i].name) + 1; + + memcpy(pSMB->DialectsArray+count, protocols[i].name, len); + count += len; } inc_rfc1001_len(pSMB, count); pSMB->ByteCount = cpu_to_le16(count); diff --git a/fs/cifs/connect.c b/fs/cifs/connect.c index c832a8a1970a..7aa08dba4719 100644 --- a/fs/cifs/connect.c +++ b/fs/cifs/connect.c @@ -2547,7 +2547,7 @@ cifs_setup_ipc(struct cifs_ses *ses, struct smb_vol *volume_info) if (tcon == NULL) return -ENOMEM; - snprintf(unc, sizeof(unc), "\\\\%s\\IPC$", ses->serverName); + snprintf(unc, sizeof(unc), "\\\\%s\\IPC$", ses->server->hostname); /* cannot fail */ nls_codepage = load_nls_default(); diff --git a/fs/cifs/inode.c b/fs/cifs/inode.c index d32eaa4b2437..6e8765f44508 100644 --- a/fs/cifs/inode.c +++ b/fs/cifs/inode.c @@ -467,6 +467,8 @@ cifs_sfu_type(struct cifs_fattr *fattr, const char *path, oparms.cifs_sb = cifs_sb; oparms.desired_access = GENERIC_READ; oparms.create_options = CREATE_NOT_DIR; + if (backup_cred(cifs_sb)) + oparms.create_options |= CREATE_OPEN_BACKUP_INTENT; oparms.disposition = FILE_OPEN; oparms.path = path; oparms.fid = &fid; diff --git a/fs/cifs/misc.c b/fs/cifs/misc.c index dacb2c05674c..6926685e513c 100644 --- a/fs/cifs/misc.c +++ b/fs/cifs/misc.c @@ -402,9 +402,17 @@ is_valid_oplock_break(char *buffer, struct TCP_Server_Info *srv) (struct smb_com_transaction_change_notify_rsp *)buf; struct file_notify_information *pnotify; __u32 data_offset = 0; + size_t len = srv->total_read - sizeof(pSMBr->hdr.smb_buf_length); + if (get_bcc(buf) > sizeof(struct file_notify_information)) { data_offset = le32_to_cpu(pSMBr->DataOffset); + if (data_offset > + len - sizeof(struct file_notify_information)) { + cifs_dbg(FYI, "invalid data_offset %u\n", + data_offset); + return true; + } pnotify = (struct file_notify_information *) ((char *)&pSMBr->hdr.Protocol + data_offset); cifs_dbg(FYI, "dnotify on %s Action: 0x%x\n", diff --git a/fs/cifs/readdir.c b/fs/cifs/readdir.c index eeab81c9452f..e169e1a5fd35 100644 --- a/fs/cifs/readdir.c +++ b/fs/cifs/readdir.c @@ -376,8 +376,15 @@ static char *nxt_dir_entry(char *old_entry, char *end_of_smb, int level) new_entry = old_entry + sizeof(FIND_FILE_STANDARD_INFO) + pfData->FileNameLength; - } else - new_entry = old_entry + le32_to_cpu(pDirInfo->NextEntryOffset); + } else { + u32 next_offset = le32_to_cpu(pDirInfo->NextEntryOffset); + + if (old_entry + next_offset < old_entry) { + cifs_dbg(VFS, "invalid offset %u\n", next_offset); + return NULL; + } + new_entry = old_entry + next_offset; + } cifs_dbg(FYI, "new entry %p old entry %p\n", new_entry, old_entry); /* validate that new_entry is not past end of SMB */ if (new_entry >= end_of_smb) { diff --git a/fs/cifs/smb2misc.c b/fs/cifs/smb2misc.c index db0453660ff6..6a9c47541c53 100644 --- a/fs/cifs/smb2misc.c +++ b/fs/cifs/smb2misc.c @@ -248,16 +248,20 @@ smb2_check_message(char *buf, unsigned int len, struct TCP_Server_Info *srvr) * MacOS server pads after SMB2.1 write response with 3 bytes * of junk. Other servers match RFC1001 len to actual * SMB2/SMB3 frame length (header + smb2 response specific data) - * Some windows servers do too when compounding is used. - * Log the server error (once), but allow it and continue + * Some windows servers also pad up to 8 bytes when compounding. + * If pad is longer than eight bytes, log the server behavior + * (once), since may indicate a problem but allow it and continue * since the frame is parseable. */ if (clc_len < len) { - printk_once(KERN_WARNING - "SMB2 server sent bad RFC1001 len %d not %d\n", - len, clc_len); + pr_warn_once( + "srv rsp padded more than expected. Length %d not %d for cmd:%d mid:%llu\n", + len, clc_len, command, mid); return 0; } + pr_warn_once( + "srv rsp too short, len %d not %d. cmd:%d mid:%llu\n", + len, clc_len, command, mid); return 1; } diff --git a/fs/cifs/smb2ops.c b/fs/cifs/smb2ops.c index 247a98e6c856..d954ce36b473 100644 --- a/fs/cifs/smb2ops.c +++ b/fs/cifs/smb2ops.c @@ -630,7 +630,10 @@ smb2_is_path_accessible(const unsigned int xid, struct cifs_tcon *tcon, oparms.tcon = tcon; oparms.desired_access = FILE_READ_ATTRIBUTES; oparms.disposition = FILE_OPEN; - oparms.create_options = 0; + if (backup_cred(cifs_sb)) + oparms.create_options = CREATE_OPEN_BACKUP_INTENT; + else + oparms.create_options = 0; oparms.fid = &fid; oparms.reconnect = false; @@ -779,7 +782,10 @@ smb2_query_eas(const unsigned int xid, struct cifs_tcon *tcon, oparms.tcon = tcon; oparms.desired_access = FILE_READ_EA; oparms.disposition = FILE_OPEN; - oparms.create_options = 0; + if (backup_cred(cifs_sb)) + oparms.create_options = CREATE_OPEN_BACKUP_INTENT; + else + oparms.create_options = 0; oparms.fid = &fid; oparms.reconnect = false; @@ -858,7 +864,10 @@ smb2_set_ea(const unsigned int xid, struct cifs_tcon *tcon, oparms.tcon = tcon; oparms.desired_access = FILE_WRITE_EA; oparms.disposition = FILE_OPEN; - oparms.create_options = 0; + if (backup_cred(cifs_sb)) + oparms.create_options = CREATE_OPEN_BACKUP_INTENT; + else + oparms.create_options = 0; oparms.fid = &fid; oparms.reconnect = false; @@ -1453,7 +1462,10 @@ smb2_query_dir_first(const unsigned int xid, struct cifs_tcon *tcon, oparms.tcon = tcon; oparms.desired_access = FILE_READ_ATTRIBUTES | FILE_READ_DATA; oparms.disposition = FILE_OPEN; - oparms.create_options = 0; + if (backup_cred(cifs_sb)) + oparms.create_options = CREATE_OPEN_BACKUP_INTENT; + else + oparms.create_options = 0; oparms.fid = fid; oparms.reconnect = false; @@ -1857,7 +1869,10 @@ smb2_query_symlink(const unsigned int xid, struct cifs_tcon *tcon, oparms.tcon = tcon; oparms.desired_access = FILE_READ_ATTRIBUTES; oparms.disposition = FILE_OPEN; - oparms.create_options = 0; + if (backup_cred(cifs_sb)) + oparms.create_options = CREATE_OPEN_BACKUP_INTENT; + else + oparms.create_options = 0; oparms.fid = &fid; oparms.reconnect = false; @@ -3639,7 +3654,7 @@ struct smb_version_values smb21_values = { struct smb_version_values smb3any_values = { .version_string = SMB3ANY_VERSION_STRING, .protocol_id = SMB302_PROT_ID, /* doesn't matter, send protocol array */ - .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION, + .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION | SMB2_GLOBAL_CAP_DIRECTORY_LEASING, .large_lock_type = 0, .exclusive_lock_type = SMB2_LOCKFLAG_EXCLUSIVE_LOCK, .shared_lock_type = SMB2_LOCKFLAG_SHARED_LOCK, @@ -3660,7 +3675,7 @@ struct smb_version_values smb3any_values = { struct smb_version_values smbdefault_values = { .version_string = SMBDEFAULT_VERSION_STRING, .protocol_id = SMB302_PROT_ID, /* doesn't matter, send protocol array */ - .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION, + .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION | SMB2_GLOBAL_CAP_DIRECTORY_LEASING, .large_lock_type = 0, .exclusive_lock_type = SMB2_LOCKFLAG_EXCLUSIVE_LOCK, .shared_lock_type = SMB2_LOCKFLAG_SHARED_LOCK, @@ -3681,7 +3696,7 @@ struct smb_version_values smbdefault_values = { struct smb_version_values smb30_values = { .version_string = SMB30_VERSION_STRING, .protocol_id = SMB30_PROT_ID, - .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION, + .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION | SMB2_GLOBAL_CAP_DIRECTORY_LEASING, .large_lock_type = 0, .exclusive_lock_type = SMB2_LOCKFLAG_EXCLUSIVE_LOCK, .shared_lock_type = SMB2_LOCKFLAG_SHARED_LOCK, @@ -3702,7 +3717,7 @@ struct smb_version_values smb30_values = { struct smb_version_values smb302_values = { .version_string = SMB302_VERSION_STRING, .protocol_id = SMB302_PROT_ID, - .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION, + .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION | SMB2_GLOBAL_CAP_DIRECTORY_LEASING, .large_lock_type = 0, .exclusive_lock_type = SMB2_LOCKFLAG_EXCLUSIVE_LOCK, .shared_lock_type = SMB2_LOCKFLAG_SHARED_LOCK, @@ -3723,7 +3738,7 @@ struct smb_version_values smb302_values = { struct smb_version_values smb311_values = { .version_string = SMB311_VERSION_STRING, .protocol_id = SMB311_PROT_ID, - .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION, + .req_capabilities = SMB2_GLOBAL_CAP_DFS | SMB2_GLOBAL_CAP_LEASING | SMB2_GLOBAL_CAP_LARGE_MTU | SMB2_GLOBAL_CAP_PERSISTENT_HANDLES | SMB2_GLOBAL_CAP_ENCRYPTION | SMB2_GLOBAL_CAP_DIRECTORY_LEASING, .large_lock_type = 0, .exclusive_lock_type = SMB2_LOCKFLAG_EXCLUSIVE_LOCK, .shared_lock_type = SMB2_LOCKFLAG_SHARED_LOCK, diff --git a/fs/cifs/smb2pdu.c b/fs/cifs/smb2pdu.c index 5740aa809be6..f54d07bda067 100644 --- a/fs/cifs/smb2pdu.c +++ b/fs/cifs/smb2pdu.c @@ -2178,6 +2178,9 @@ SMB2_open_init(struct cifs_tcon *tcon, struct smb_rqst *rqst, __u8 *oplock, if (!(server->capabilities & SMB2_GLOBAL_CAP_LEASING) || *oplock == SMB2_OPLOCK_LEVEL_NONE) req->RequestedOplockLevel = *oplock; + else if (!(server->capabilities & SMB2_GLOBAL_CAP_DIRECTORY_LEASING) && + (oparms->create_options & CREATE_NOT_FILE)) + req->RequestedOplockLevel = *oplock; /* no srv lease support */ else { rc = add_lease_context(server, iov, &n_iov, oparms->fid->lease_key, oplock); @@ -2456,14 +2459,14 @@ SMB2_ioctl(const unsigned int xid, struct cifs_tcon *tcon, u64 persistent_fid, /* We check for obvious errors in the output buffer length and offset */ if (*plen == 0) goto ioctl_exit; /* server returned no data */ - else if (*plen > 0xFF00) { + else if (*plen > rsp_iov.iov_len || *plen > 0xFF00) { cifs_dbg(VFS, "srv returned invalid ioctl length: %d\n", *plen); *plen = 0; rc = -EIO; goto ioctl_exit; } - if (rsp_iov.iov_len < le32_to_cpu(rsp->OutputOffset) + *plen) { + if (rsp_iov.iov_len - *plen < le32_to_cpu(rsp->OutputOffset)) { cifs_dbg(VFS, "Malformed ioctl resp: len %d offset %d\n", *plen, le32_to_cpu(rsp->OutputOffset)); *plen = 0; @@ -3574,33 +3577,38 @@ num_entries(char *bufstart, char *end_of_buf, char **lastentry, size_t size) int len; unsigned int entrycount = 0; unsigned int next_offset = 0; - FILE_DIRECTORY_INFO *entryptr; + char *entryptr; + FILE_DIRECTORY_INFO *dir_info; if (bufstart == NULL) return 0; - entryptr = (FILE_DIRECTORY_INFO *)bufstart; + entryptr = bufstart; while (1) { - entryptr = (FILE_DIRECTORY_INFO *) - ((char *)entryptr + next_offset); - - if ((char *)entryptr + size > end_of_buf) { + if (entryptr + next_offset < entryptr || + entryptr + next_offset > end_of_buf || + entryptr + next_offset + size > end_of_buf) { cifs_dbg(VFS, "malformed search entry would overflow\n"); break; } - len = le32_to_cpu(entryptr->FileNameLength); - if ((char *)entryptr + len + size > end_of_buf) { + entryptr = entryptr + next_offset; + dir_info = (FILE_DIRECTORY_INFO *)entryptr; + + len = le32_to_cpu(dir_info->FileNameLength); + if (entryptr + len < entryptr || + entryptr + len > end_of_buf || + entryptr + len + size > end_of_buf) { cifs_dbg(VFS, "directory entry name would overflow frame end of buf %p\n", end_of_buf); break; } - *lastentry = (char *)entryptr; + *lastentry = entryptr; entrycount++; - next_offset = le32_to_cpu(entryptr->NextEntryOffset); + next_offset = le32_to_cpu(dir_info->NextEntryOffset); if (!next_offset) break; } @@ -447,6 +447,7 @@ bool dax_lock_mapping_entry(struct page *page) xa_unlock_irq(&mapping->i_pages); break; } else if (IS_ERR(entry)) { + xa_unlock_irq(&mapping->i_pages); WARN_ON_ONCE(PTR_ERR(entry) != -EAGAIN); continue; } @@ -1120,21 +1121,12 @@ static vm_fault_t dax_load_hole(struct address_space *mapping, void *entry, { struct inode *inode = mapping->host; unsigned long vaddr = vmf->address; - vm_fault_t ret = VM_FAULT_NOPAGE; - struct page *zero_page; - pfn_t pfn; - - zero_page = ZERO_PAGE(0); - if (unlikely(!zero_page)) { - ret = VM_FAULT_OOM; - goto out; - } + pfn_t pfn = pfn_to_pfn_t(my_zero_pfn(vaddr)); + vm_fault_t ret; - pfn = page_to_pfn_t(zero_page); dax_insert_mapping_entry(mapping, vmf, entry, pfn, RADIX_DAX_ZERO_PAGE, false); ret = vmf_insert_mixed(vmf->vma, vaddr, pfn); -out: trace_dax_load_hole(inode, vmf, ret); return ret; } diff --git a/fs/ext2/inode.c b/fs/ext2/inode.c index 7f7ee18fe179..e4bb9386c045 100644 --- a/fs/ext2/inode.c +++ b/fs/ext2/inode.c @@ -1448,6 +1448,7 @@ struct inode *ext2_iget (struct super_block *sb, unsigned long ino) } inode->i_blocks = le32_to_cpu(raw_inode->i_blocks); ei->i_flags = le32_to_cpu(raw_inode->i_flags); + ext2_set_inode_flags(inode); ei->i_faddr = le32_to_cpu(raw_inode->i_faddr); ei->i_frag_no = raw_inode->i_frag; ei->i_frag_size = raw_inode->i_fsize; @@ -1517,7 +1518,6 @@ struct inode *ext2_iget (struct super_block *sb, unsigned long ino) new_decode_dev(le32_to_cpu(raw_inode->i_block[1]))); } brelse (bh); - ext2_set_inode_flags(inode); unlock_new_inode(inode); return inode; diff --git a/fs/ext4/dir.c b/fs/ext4/dir.c index e2902d394f1b..f93f9881ec18 100644 --- a/fs/ext4/dir.c +++ b/fs/ext4/dir.c @@ -76,7 +76,7 @@ int __ext4_check_dir_entry(const char *function, unsigned int line, else if (unlikely(rlen < EXT4_DIR_REC_LEN(de->name_len))) error_msg = "rec_len is too small for name_len"; else if (unlikely(((char *) de - buf) + rlen > size)) - error_msg = "directory entry across range"; + error_msg = "directory entry overrun"; else if (unlikely(le32_to_cpu(de->inode) > le32_to_cpu(EXT4_SB(dir->i_sb)->s_es->s_inodes_count))) error_msg = "inode out of bounds"; @@ -85,18 +85,16 @@ int __ext4_check_dir_entry(const char *function, unsigned int line, if (filp) ext4_error_file(filp, function, line, bh->b_blocknr, - "bad entry in directory: %s - offset=%u(%u), " - "inode=%u, rec_len=%d, name_len=%d", - error_msg, (unsigned) (offset % size), - offset, le32_to_cpu(de->inode), - rlen, de->name_len); + "bad entry in directory: %s - offset=%u, " + "inode=%u, rec_len=%d, name_len=%d, size=%d", + error_msg, offset, le32_to_cpu(de->inode), + rlen, de->name_len, size); else ext4_error_inode(dir, function, line, bh->b_blocknr, - "bad entry in directory: %s - offset=%u(%u), " - "inode=%u, rec_len=%d, name_len=%d", - error_msg, (unsigned) (offset % size), - offset, le32_to_cpu(de->inode), - rlen, de->name_len); + "bad entry in directory: %s - offset=%u, " + "inode=%u, rec_len=%d, name_len=%d, size=%d", + error_msg, offset, le32_to_cpu(de->inode), + rlen, de->name_len, size); return 1; } diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 0f0edd1cd0cd..caff935fbeb8 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h @@ -43,6 +43,17 @@ #define __FS_HAS_ENCRYPTION IS_ENABLED(CONFIG_EXT4_FS_ENCRYPTION) #include <linux/fscrypt.h> +#include <linux/compiler.h> + +/* Until this gets included into linux/compiler-gcc.h */ +#ifndef __nonstring +#if defined(GCC_VERSION) && (GCC_VERSION >= 80000) +#define __nonstring __attribute__((nonstring)) +#else +#define __nonstring +#endif +#endif + /* * The fourth extended filesystem constants/structures */ @@ -675,6 +686,9 @@ enum { /* Max physical block we can address w/o extents */ #define EXT4_MAX_BLOCK_FILE_PHYS 0xFFFFFFFF +/* Max logical block we can support */ +#define EXT4_MAX_LOGICAL_BLOCK 0xFFFFFFFF + /* * Structure of an inode on the disk */ @@ -1226,7 +1240,7 @@ struct ext4_super_block { __le32 s_feature_ro_compat; /* readonly-compatible feature set */ /*68*/ __u8 s_uuid[16]; /* 128-bit uuid for volume */ /*78*/ char s_volume_name[16]; /* volume name */ -/*88*/ char s_last_mounted[64]; /* directory where last mounted */ +/*88*/ char s_last_mounted[64] __nonstring; /* directory where last mounted */ /*C8*/ __le32 s_algorithm_usage_bitmap; /* For compression */ /* * Performance hints. Directory preallocation should only @@ -1277,13 +1291,13 @@ struct ext4_super_block { __le32 s_first_error_time; /* first time an error happened */ __le32 s_first_error_ino; /* inode involved in first error */ __le64 s_first_error_block; /* block involved of first error */ - __u8 s_first_error_func[32]; /* function where the error happened */ + __u8 s_first_error_func[32] __nonstring; /* function where the error happened */ __le32 s_first_error_line; /* line number where error happened */ __le32 s_last_error_time; /* most recent time of an error */ __le32 s_last_error_ino; /* inode involved in last error */ __le32 s_last_error_line; /* line number where error happened */ __le64 s_last_error_block; /* block involved of last error */ - __u8 s_last_error_func[32]; /* function where the error happened */ + __u8 s_last_error_func[32] __nonstring; /* function where the error happened */ #define EXT4_S_ERR_END offsetof(struct ext4_super_block, s_mount_opts) __u8 s_mount_opts[64]; __le32 s_usr_quota_inum; /* inode for tracking user quota */ diff --git a/fs/ext4/inline.c b/fs/ext4/inline.c index 3543fe80a3c4..7b4736022761 100644 --- a/fs/ext4/inline.c +++ b/fs/ext4/inline.c @@ -1753,6 +1753,7 @@ bool empty_inline_dir(struct inode *dir, int *has_inline_data) { int err, inline_size; struct ext4_iloc iloc; + size_t inline_len; void *inline_pos; unsigned int offset; struct ext4_dir_entry_2 *de; @@ -1780,8 +1781,9 @@ bool empty_inline_dir(struct inode *dir, int *has_inline_data) goto out; } + inline_len = ext4_get_inline_size(dir); offset = EXT4_INLINE_DOTDOT_SIZE; - while (offset < dir->i_size) { + while (offset < inline_len) { de = ext4_get_inline_entry(dir, &iloc, offset, &inline_pos, &inline_size); if (ext4_check_dir_entry(dir, NULL, de, diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index d0dd585add6a..d767e993591d 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3413,12 +3413,16 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length, { struct ext4_sb_info *sbi = EXT4_SB(inode->i_sb); unsigned int blkbits = inode->i_blkbits; - unsigned long first_block = offset >> blkbits; - unsigned long last_block = (offset + length - 1) >> blkbits; + unsigned long first_block, last_block; struct ext4_map_blocks map; bool delalloc = false; int ret; + if ((offset >> blkbits) > EXT4_MAX_LOGICAL_BLOCK) + return -EINVAL; + first_block = offset >> blkbits; + last_block = min_t(loff_t, (offset + length - 1) >> blkbits, + EXT4_MAX_LOGICAL_BLOCK); if (flags & IOMAP_REPORT) { if (ext4_has_inline_data(inode)) { @@ -3948,6 +3952,7 @@ static const struct address_space_operations ext4_dax_aops = { .writepages = ext4_dax_writepages, .direct_IO = noop_direct_IO, .set_page_dirty = noop_set_page_dirty, + .bmap = ext4_bmap, .invalidatepage = noop_invalidatepage, }; @@ -4192,9 +4197,8 @@ int ext4_update_disksize_before_punch(struct inode *inode, loff_t offset, return 0; } -static void ext4_wait_dax_page(struct ext4_inode_info *ei, bool *did_unlock) +static void ext4_wait_dax_page(struct ext4_inode_info *ei) { - *did_unlock = true; up_write(&ei->i_mmap_sem); schedule(); down_write(&ei->i_mmap_sem); @@ -4204,14 +4208,12 @@ int ext4_break_layouts(struct inode *inode) { struct ext4_inode_info *ei = EXT4_I(inode); struct page *page; - bool retry; int error; if (WARN_ON_ONCE(!rwsem_is_locked(&ei->i_mmap_sem))) return -EINVAL; do { - retry = false; page = dax_layout_busy_page(inode->i_mapping); if (!page) return 0; @@ -4219,8 +4221,8 @@ int ext4_break_layouts(struct inode *inode) error = ___wait_var_event(&page->_refcount, atomic_read(&page->_refcount) == 1, TASK_INTERRUPTIBLE, 0, 0, - ext4_wait_dax_page(ei, &retry)); - } while (error == 0 && retry); + ext4_wait_dax_page(ei)); + } while (error == 0); return error; } @@ -4895,6 +4897,7 @@ struct inode *ext4_iget(struct super_block *sb, unsigned long ino) * not initialized on a new filesystem. */ } ei->i_flags = le32_to_cpu(raw_inode->i_flags); + ext4_set_inode_flags(inode); inode->i_blocks = ext4_inode_blocks(raw_inode, ei); ei->i_file_acl = le32_to_cpu(raw_inode->i_file_acl_lo); if (ext4_has_feature_64bit(sb)) @@ -5041,7 +5044,6 @@ struct inode *ext4_iget(struct super_block *sb, unsigned long ino) goto bad_inode; } brelse(iloc.bh); - ext4_set_inode_flags(inode); unlock_new_inode(inode); return inode; diff --git a/fs/ext4/mmp.c b/fs/ext4/mmp.c index 39b07c2d3384..2305b4374fd3 100644 --- a/fs/ext4/mmp.c +++ b/fs/ext4/mmp.c @@ -49,7 +49,6 @@ static int write_mmp_block(struct super_block *sb, struct buffer_head *bh) */ sb_start_write(sb); ext4_mmp_csum_set(sb, mmp); - mark_buffer_dirty(bh); lock_buffer(bh); bh->b_end_io = end_buffer_write_sync; get_bh(bh); diff --git a/fs/ext4/namei.c b/fs/ext4/namei.c index 116ff68c5bd4..377d516c475f 100644 --- a/fs/ext4/namei.c +++ b/fs/ext4/namei.c @@ -3478,6 +3478,12 @@ static int ext4_rename(struct inode *old_dir, struct dentry *old_dentry, int credits; u8 old_file_type; + if (new.inode && new.inode->i_nlink == 0) { + EXT4_ERROR_INODE(new.inode, + "target of rename is already freed"); + return -EFSCORRUPTED; + } + if ((ext4_test_inode_flag(new_dir, EXT4_INODE_PROJINHERIT)) && (!projid_eq(EXT4_I(new_dir)->i_projid, EXT4_I(old_dentry->d_inode)->i_projid))) diff --git a/fs/ext4/resize.c b/fs/ext4/resize.c index e5fb38451a73..ebbc663d0798 100644 --- a/fs/ext4/resize.c +++ b/fs/ext4/resize.c @@ -19,6 +19,7 @@ int ext4_resize_begin(struct super_block *sb) { + struct ext4_sb_info *sbi = EXT4_SB(sb); int ret = 0; if (!capable(CAP_SYS_RESOURCE)) @@ -29,7 +30,7 @@ int ext4_resize_begin(struct super_block *sb) * because the user tools have no way of handling this. Probably a * bad time to do it anyways. */ - if (EXT4_SB(sb)->s_sbh->b_blocknr != + if (EXT4_B2C(sbi, sbi->s_sbh->b_blocknr) != le32_to_cpu(EXT4_SB(sb)->s_es->s_first_data_block)) { ext4_warning(sb, "won't resize using backup superblock at %llu", (unsigned long long)EXT4_SB(sb)->s_sbh->b_blocknr); @@ -1986,6 +1987,26 @@ retry: } } + /* + * Make sure the last group has enough space so that it's + * guaranteed to have enough space for all metadata blocks + * that it might need to hold. (We might not need to store + * the inode table blocks in the last block group, but there + * will be cases where this might be needed.) + */ + if ((ext4_group_first_block_no(sb, n_group) + + ext4_group_overhead_blocks(sb, n_group) + 2 + + sbi->s_itb_per_group + sbi->s_cluster_ratio) >= n_blocks_count) { + n_blocks_count = ext4_group_first_block_no(sb, n_group); + n_group--; + n_blocks_count_retry = 0; + if (resize_inode) { + iput(resize_inode); + resize_inode = NULL; + } + goto retry; + } + /* extend the last group */ if (n_group == o_group) add = n_blocks_count - o_blocks_count; diff --git a/fs/ext4/super.c b/fs/ext4/super.c index 5863fd22e90b..1145109968ef 100644 --- a/fs/ext4/super.c +++ b/fs/ext4/super.c @@ -2145,6 +2145,8 @@ static int _ext4_show_options(struct seq_file *seq, struct super_block *sb, SEQ_OPTS_PRINT("max_dir_size_kb=%u", sbi->s_max_dir_size_kb); if (test_opt(sb, DATA_ERR_ABORT)) SEQ_OPTS_PUTS("data_err=abort"); + if (DUMMY_ENCRYPTION_ENABLED(sbi)) + SEQ_OPTS_PUTS("test_dummy_encryption"); ext4_show_quota_options(seq, sb); return 0; @@ -4378,11 +4380,13 @@ no_journal: block = ext4_count_free_clusters(sb); ext4_free_blocks_count_set(sbi->s_es, EXT4_C2B(sbi, block)); + ext4_superblock_csum_set(sb); err = percpu_counter_init(&sbi->s_freeclusters_counter, block, GFP_KERNEL); if (!err) { unsigned long freei = ext4_count_free_inodes(sb); sbi->s_es->s_free_inodes_count = cpu_to_le32(freei); + ext4_superblock_csum_set(sb); err = percpu_counter_init(&sbi->s_freeinodes_counter, freei, GFP_KERNEL); } diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 34830f6457ea..8220a168282e 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -1637,6 +1637,14 @@ static void nfs_state_set_delegation(struct nfs4_state *state, write_sequnlock(&state->seqlock); } +static void nfs_state_clear_delegation(struct nfs4_state *state) +{ + write_seqlock(&state->seqlock); + nfs4_stateid_copy(&state->stateid, &state->open_stateid); + clear_bit(NFS_DELEGATED_STATE, &state->flags); + write_sequnlock(&state->seqlock); +} + static int update_open_stateid(struct nfs4_state *state, const nfs4_stateid *open_stateid, const nfs4_stateid *delegation, @@ -2145,10 +2153,7 @@ int nfs4_open_delegation_recall(struct nfs_open_context *ctx, if (IS_ERR(opendata)) return PTR_ERR(opendata); nfs4_stateid_copy(&opendata->o_arg.u.delegation, stateid); - write_seqlock(&state->seqlock); - nfs4_stateid_copy(&state->stateid, &state->open_stateid); - write_sequnlock(&state->seqlock); - clear_bit(NFS_DELEGATED_STATE, &state->flags); + nfs_state_clear_delegation(state); switch (type & (FMODE_READ|FMODE_WRITE)) { case FMODE_READ|FMODE_WRITE: case FMODE_WRITE: @@ -2601,10 +2606,7 @@ static void nfs_finish_clear_delegation_stateid(struct nfs4_state *state, const nfs4_stateid *stateid) { nfs_remove_bad_delegation(state->inode, stateid); - write_seqlock(&state->seqlock); - nfs4_stateid_copy(&state->stateid, &state->open_stateid); - write_sequnlock(&state->seqlock); - clear_bit(NFS_DELEGATED_STATE, &state->flags); + nfs_state_clear_delegation(state); } static void nfs40_clear_delegation_stateid(struct nfs4_state *state) @@ -2672,15 +2674,20 @@ static void nfs41_check_delegation_stateid(struct nfs4_state *state) delegation = rcu_dereference(NFS_I(state->inode)->delegation); if (delegation == NULL) { rcu_read_unlock(); + nfs_state_clear_delegation(state); return; } nfs4_stateid_copy(&stateid, &delegation->stateid); - if (test_bit(NFS_DELEGATION_REVOKED, &delegation->flags) || - !test_and_clear_bit(NFS_DELEGATION_TEST_EXPIRED, - &delegation->flags)) { + if (test_bit(NFS_DELEGATION_REVOKED, &delegation->flags)) { + rcu_read_unlock(); + nfs_state_clear_delegation(state); + return; + } + + if (!test_and_clear_bit(NFS_DELEGATION_TEST_EXPIRED, + &delegation->flags)) { rcu_read_unlock(); - nfs_finish_clear_delegation_stateid(state, &stateid); return; } diff --git a/fs/nfs/nfs4state.c b/fs/nfs/nfs4state.c index 3df0eb52da1c..40a08cd483f0 100644 --- a/fs/nfs/nfs4state.c +++ b/fs/nfs/nfs4state.c @@ -1390,6 +1390,8 @@ int nfs4_schedule_stateid_recovery(const struct nfs_server *server, struct nfs4_ if (!nfs4_state_mark_reclaim_nograce(clp, state)) return -EBADF; + nfs_inode_find_delegation_state_and_recover(state->inode, + &state->stateid); dprintk("%s: scheduling stateid recovery for server %s\n", __func__, clp->cl_hostname); nfs4_schedule_state_manager(clp); diff --git a/fs/nfs/nfs4trace.h b/fs/nfs/nfs4trace.h index a275fba93170..b1483b303e0b 100644 --- a/fs/nfs/nfs4trace.h +++ b/fs/nfs/nfs4trace.h @@ -1137,7 +1137,7 @@ DECLARE_EVENT_CLASS(nfs4_inode_callback_event, TP_fast_assign( __entry->error = error; __entry->fhandle = nfs_fhandle_hash(fhandle); - if (inode != NULL) { + if (!IS_ERR_OR_NULL(inode)) { __entry->fileid = NFS_FILEID(inode); __entry->dev = inode->i_sb->s_dev; } else { @@ -1194,7 +1194,7 @@ DECLARE_EVENT_CLASS(nfs4_inode_stateid_callback_event, TP_fast_assign( __entry->error = error; __entry->fhandle = nfs_fhandle_hash(fhandle); - if (inode != NULL) { + if (!IS_ERR_OR_NULL(inode)) { __entry->fileid = NFS_FILEID(inode); __entry->dev = inode->i_sb->s_dev; } else { diff --git a/fs/nfs/pnfs.c b/fs/nfs/pnfs.c index e8f232de484f..7d9a51e6b847 100644 --- a/fs/nfs/pnfs.c +++ b/fs/nfs/pnfs.c @@ -1740,16 +1740,16 @@ static bool pnfs_within_mdsthreshold(struct nfs_open_context *ctx, return ret; } -static bool pnfs_prepare_to_retry_layoutget(struct pnfs_layout_hdr *lo) +static int pnfs_prepare_to_retry_layoutget(struct pnfs_layout_hdr *lo) { /* * send layoutcommit as it can hold up layoutreturn due to lseg * reference */ pnfs_layoutcommit_inode(lo->plh_inode, false); - return !wait_on_bit_action(&lo->plh_flags, NFS_LAYOUT_RETURN, + return wait_on_bit_action(&lo->plh_flags, NFS_LAYOUT_RETURN, nfs_wait_bit_killable, - TASK_UNINTERRUPTIBLE); + TASK_KILLABLE); } static void nfs_layoutget_begin(struct pnfs_layout_hdr *lo) @@ -1830,7 +1830,9 @@ pnfs_update_layout(struct inode *ino, } lookup_again: - nfs4_client_recover_expired_lease(clp); + lseg = ERR_PTR(nfs4_client_recover_expired_lease(clp)); + if (IS_ERR(lseg)) + goto out; first = false; spin_lock(&ino->i_lock); lo = pnfs_find_alloc_layout(ino, ctx, gfp_flags); @@ -1863,9 +1865,9 @@ lookup_again: if (list_empty(&lo->plh_segs) && atomic_read(&lo->plh_outstanding) != 0) { spin_unlock(&ino->i_lock); - if (wait_var_event_killable(&lo->plh_outstanding, - atomic_read(&lo->plh_outstanding) == 0 - || !list_empty(&lo->plh_segs))) + lseg = ERR_PTR(wait_var_event_killable(&lo->plh_outstanding, + atomic_read(&lo->plh_outstanding))); + if (IS_ERR(lseg) || !list_empty(&lo->plh_segs)) goto out_put_layout_hdr; pnfs_put_layout_hdr(lo); goto lookup_again; @@ -1898,8 +1900,11 @@ lookup_again: if (test_and_set_bit(NFS_LAYOUT_FIRST_LAYOUTGET, &lo->plh_flags)) { spin_unlock(&ino->i_lock); - wait_on_bit(&lo->plh_flags, NFS_LAYOUT_FIRST_LAYOUTGET, - TASK_UNINTERRUPTIBLE); + lseg = ERR_PTR(wait_on_bit(&lo->plh_flags, + NFS_LAYOUT_FIRST_LAYOUTGET, + TASK_KILLABLE)); + if (IS_ERR(lseg)) + goto out_put_layout_hdr; pnfs_put_layout_hdr(lo); dprintk("%s retrying\n", __func__); goto lookup_again; @@ -1925,7 +1930,8 @@ lookup_again: if (test_bit(NFS_LAYOUT_RETURN, &lo->plh_flags)) { spin_unlock(&ino->i_lock); dprintk("%s wait for layoutreturn\n", __func__); - if (pnfs_prepare_to_retry_layoutget(lo)) { + lseg = ERR_PTR(pnfs_prepare_to_retry_layoutget(lo)); + if (!IS_ERR(lseg)) { if (first) pnfs_clear_first_layoutget(lo); pnfs_put_layout_hdr(lo); diff --git a/fs/nilfs2/alloc.c b/fs/nilfs2/alloc.c index 03b8ba933eb2..235b959fc2b3 100644 --- a/fs/nilfs2/alloc.c +++ b/fs/nilfs2/alloc.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * alloc.c - NILFS dat/inode allocator * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Originally written by Koji Sato. * Two allocators were unified by Ryusuke Konishi and Amagai Yoshiji. */ diff --git a/fs/nilfs2/alloc.h b/fs/nilfs2/alloc.h index 05149e606a78..0303c3968cee 100644 --- a/fs/nilfs2/alloc.h +++ b/fs/nilfs2/alloc.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * alloc.h - persistent object (dat entry/disk inode) allocator/deallocator * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Originally written by Koji Sato. * Two allocators were unified by Ryusuke Konishi and Amagai Yoshiji. */ diff --git a/fs/nilfs2/bmap.c b/fs/nilfs2/bmap.c index 01fb1831ca25..fb5a9a8a13cf 100644 --- a/fs/nilfs2/bmap.c +++ b/fs/nilfs2/bmap.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * bmap.c - NILFS block mapping. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/bmap.h b/fs/nilfs2/bmap.h index 2b6ffbe5997a..2c63858e81c9 100644 --- a/fs/nilfs2/bmap.h +++ b/fs/nilfs2/bmap.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * bmap.h - NILFS block mapping. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/btnode.c b/fs/nilfs2/btnode.c index dec98cab729d..ebb24a314f43 100644 --- a/fs/nilfs2/btnode.c +++ b/fs/nilfs2/btnode.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * btnode.c - NILFS B-tree node cache * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Originally written by Seiji Kihara. * Fully revised by Ryusuke Konishi for stabilization and simplification. * diff --git a/fs/nilfs2/btnode.h b/fs/nilfs2/btnode.h index 4e8aaa1aeb65..0f88dbc9bcb3 100644 --- a/fs/nilfs2/btnode.h +++ b/fs/nilfs2/btnode.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * btnode.h - NILFS B-tree node cache * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Seiji Kihara. * Revised by Ryusuke Konishi. */ diff --git a/fs/nilfs2/btree.c b/fs/nilfs2/btree.c index 16a7a67a11c9..23e043eca237 100644 --- a/fs/nilfs2/btree.c +++ b/fs/nilfs2/btree.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * btree.c - NILFS B-tree. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/btree.h b/fs/nilfs2/btree.h index 2184e47fa4bf..d1421b646ce4 100644 --- a/fs/nilfs2/btree.h +++ b/fs/nilfs2/btree.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * btree.h - NILFS B-tree. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/cpfile.c b/fs/nilfs2/cpfile.c index a15a1601e931..8d41311b5db4 100644 --- a/fs/nilfs2/cpfile.c +++ b/fs/nilfs2/cpfile.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * cpfile.c - NILFS checkpoint file. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/cpfile.h b/fs/nilfs2/cpfile.h index 6eca972f9673..6336222df24a 100644 --- a/fs/nilfs2/cpfile.h +++ b/fs/nilfs2/cpfile.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * cpfile.h - NILFS checkpoint file. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/dat.c b/fs/nilfs2/dat.c index dffedb2f8817..6f4066636be9 100644 --- a/fs/nilfs2/dat.c +++ b/fs/nilfs2/dat.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * dat.c - NILFS disk address translation. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/dat.h b/fs/nilfs2/dat.h index 57dc6cf466d0..b17ee34580ae 100644 --- a/fs/nilfs2/dat.h +++ b/fs/nilfs2/dat.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * dat.h - NILFS disk address translation. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/dir.c b/fs/nilfs2/dir.c index 582831ab3eb9..81394e22d0a0 100644 --- a/fs/nilfs2/dir.c +++ b/fs/nilfs2/dir.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * dir.c - NILFS directory entry operations * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Modified for NILFS by Amagai Yoshiji. */ /* diff --git a/fs/nilfs2/direct.c b/fs/nilfs2/direct.c index 96e3ed0d9652..533e24ea3a88 100644 --- a/fs/nilfs2/direct.c +++ b/fs/nilfs2/direct.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * direct.c - NILFS direct block pointer. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/direct.h b/fs/nilfs2/direct.h index cfe85e848bba..ec9a23c77994 100644 --- a/fs/nilfs2/direct.h +++ b/fs/nilfs2/direct.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * direct.h - NILFS direct block pointer. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/file.c b/fs/nilfs2/file.c index 7da0fac71dc2..64bc81363c6c 100644 --- a/fs/nilfs2/file.c +++ b/fs/nilfs2/file.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * file.c - NILFS regular file handling primitives including fsync(). * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Amagai Yoshiji and Ryusuke Konishi. */ diff --git a/fs/nilfs2/gcinode.c b/fs/nilfs2/gcinode.c index 853a831dcde0..aa3c328ee189 100644 --- a/fs/nilfs2/gcinode.c +++ b/fs/nilfs2/gcinode.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * gcinode.c - dummy inodes to buffer blocks for garbage collection * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Seiji Kihara, Amagai Yoshiji, and Ryusuke Konishi. * Revised by Ryusuke Konishi. * diff --git a/fs/nilfs2/ifile.c b/fs/nilfs2/ifile.c index b8fa45c20c63..4140d232cadc 100644 --- a/fs/nilfs2/ifile.c +++ b/fs/nilfs2/ifile.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * ifile.c - NILFS inode file * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Amagai Yoshiji. * Revised by Ryusuke Konishi. * diff --git a/fs/nilfs2/ifile.h b/fs/nilfs2/ifile.h index 188b94fe0ec5..a1e1e5711a05 100644 --- a/fs/nilfs2/ifile.h +++ b/fs/nilfs2/ifile.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * ifile.h - NILFS inode file * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Amagai Yoshiji. * Revised by Ryusuke Konishi. * diff --git a/fs/nilfs2/inode.c b/fs/nilfs2/inode.c index 6a612d832e7d..671085512e0f 100644 --- a/fs/nilfs2/inode.c +++ b/fs/nilfs2/inode.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * inode.c - NILFS inode operations. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. * */ diff --git a/fs/nilfs2/ioctl.c b/fs/nilfs2/ioctl.c index 1d2c3d7711fe..9b96d79eea6c 100644 --- a/fs/nilfs2/ioctl.c +++ b/fs/nilfs2/ioctl.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * ioctl.c - NILFS ioctl operations. * * Copyright (C) 2007, 2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/mdt.c b/fs/nilfs2/mdt.c index c6bc1033e7d2..700870a92bc4 100644 --- a/fs/nilfs2/mdt.c +++ b/fs/nilfs2/mdt.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * mdt.c - meta data file for NILFS * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. */ diff --git a/fs/nilfs2/mdt.h b/fs/nilfs2/mdt.h index 3f67f3932097..e77aea4bb921 100644 --- a/fs/nilfs2/mdt.h +++ b/fs/nilfs2/mdt.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * mdt.h - NILFS meta data file prototype and definitions * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. */ diff --git a/fs/nilfs2/namei.c b/fs/nilfs2/namei.c index dd52d3f82e8d..9fe6d4ab74f0 100644 --- a/fs/nilfs2/namei.c +++ b/fs/nilfs2/namei.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * namei.c - NILFS pathname lookup operations. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Modified for NILFS by Amagai Yoshiji and Ryusuke Konishi. */ /* diff --git a/fs/nilfs2/nilfs.h b/fs/nilfs2/nilfs.h index 33f8c8fc96e8..a2f247b6a209 100644 --- a/fs/nilfs2/nilfs.h +++ b/fs/nilfs2/nilfs.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * nilfs.h - NILFS local header file. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato and Ryusuke Konishi. */ diff --git a/fs/nilfs2/page.c b/fs/nilfs2/page.c index 4cb850a6f1c2..329a056b73b1 100644 --- a/fs/nilfs2/page.c +++ b/fs/nilfs2/page.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * page.c - buffer/page management specific to NILFS * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi and Seiji Kihara. */ diff --git a/fs/nilfs2/page.h b/fs/nilfs2/page.h index f3687c958fa8..62b9bb469e92 100644 --- a/fs/nilfs2/page.h +++ b/fs/nilfs2/page.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * page.h - buffer/page management specific to NILFS * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi and Seiji Kihara. */ diff --git a/fs/nilfs2/recovery.c b/fs/nilfs2/recovery.c index 5139efed1888..140b663e91c7 100644 --- a/fs/nilfs2/recovery.c +++ b/fs/nilfs2/recovery.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * recovery.c - NILFS recovery logic * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. */ diff --git a/fs/nilfs2/segbuf.c b/fs/nilfs2/segbuf.c index 68cb9e4740b4..20c479b5e41b 100644 --- a/fs/nilfs2/segbuf.c +++ b/fs/nilfs2/segbuf.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * segbuf.c - NILFS segment buffer * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. * */ diff --git a/fs/nilfs2/segbuf.h b/fs/nilfs2/segbuf.h index 10e16935fff6..9bea1bd59041 100644 --- a/fs/nilfs2/segbuf.h +++ b/fs/nilfs2/segbuf.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * segbuf.h - NILFS Segment buffer prototypes and definitions * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. * */ diff --git a/fs/nilfs2/segment.c b/fs/nilfs2/segment.c index 0953635e7d48..445eef41bfaf 100644 --- a/fs/nilfs2/segment.c +++ b/fs/nilfs2/segment.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * segment.c - NILFS segment constructor. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. * */ diff --git a/fs/nilfs2/segment.h b/fs/nilfs2/segment.h index 04634e3e3d58..f5cf5308f3fc 100644 --- a/fs/nilfs2/segment.h +++ b/fs/nilfs2/segment.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * segment.h - NILFS Segment constructor prototypes and definitions * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. * */ diff --git a/fs/nilfs2/sufile.c b/fs/nilfs2/sufile.c index c7fa139d50e8..bf3f8f05c89b 100644 --- a/fs/nilfs2/sufile.c +++ b/fs/nilfs2/sufile.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * sufile.c - NILFS segment usage file. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. * Revised by Ryusuke Konishi. */ diff --git a/fs/nilfs2/sufile.h b/fs/nilfs2/sufile.h index 673a891350f4..c4e2c7a7add1 100644 --- a/fs/nilfs2/sufile.h +++ b/fs/nilfs2/sufile.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * sufile.h - NILFS segment usage file. * * Copyright (C) 2006-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Koji Sato. */ diff --git a/fs/nilfs2/super.c b/fs/nilfs2/super.c index 1b9067cf4511..26290aa1023f 100644 --- a/fs/nilfs2/super.c +++ b/fs/nilfs2/super.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * super.c - NILFS module and super block management. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. */ /* diff --git a/fs/nilfs2/sysfs.c b/fs/nilfs2/sysfs.c index 4b25837e7724..e60be7bb55b0 100644 --- a/fs/nilfs2/sysfs.c +++ b/fs/nilfs2/sysfs.c @@ -1,19 +1,10 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * sysfs.c - sysfs support implementation. * * Copyright (C) 2005-2014 Nippon Telegraph and Telephone Corporation. * Copyright (C) 2014 HGST, Inc., a Western Digital Company. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Vyacheslav Dubeyko <Vyacheslav.Dubeyko@hgst.com> */ diff --git a/fs/nilfs2/sysfs.h b/fs/nilfs2/sysfs.h index 648cedf9c06e..d001eb862dae 100644 --- a/fs/nilfs2/sysfs.h +++ b/fs/nilfs2/sysfs.h @@ -1,19 +1,10 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * sysfs.h - sysfs support declarations. * * Copyright (C) 2005-2014 Nippon Telegraph and Telephone Corporation. * Copyright (C) 2014 HGST, Inc., a Western Digital Company. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Vyacheslav Dubeyko <Vyacheslav.Dubeyko@hgst.com> */ diff --git a/fs/nilfs2/the_nilfs.c b/fs/nilfs2/the_nilfs.c index 1a85317e83f0..484785cdf96e 100644 --- a/fs/nilfs2/the_nilfs.c +++ b/fs/nilfs2/the_nilfs.c @@ -1,18 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0+ /* * the_nilfs.c - the_nilfs shared structure. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. * */ diff --git a/fs/nilfs2/the_nilfs.h b/fs/nilfs2/the_nilfs.h index 36da1779f976..380a543c5b19 100644 --- a/fs/nilfs2/the_nilfs.h +++ b/fs/nilfs2/the_nilfs.h @@ -1,18 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ /* * the_nilfs.h - the_nilfs shared structure. * * Copyright (C) 2005-2008 Nippon Telegraph and Telephone Corporation. * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * * Written by Ryusuke Konishi. * */ diff --git a/fs/notify/fsnotify.c b/fs/notify/fsnotify.c index f174397b63a0..ababdbfab537 100644 --- a/fs/notify/fsnotify.c +++ b/fs/notify/fsnotify.c @@ -351,16 +351,9 @@ int fsnotify(struct inode *to_tell, __u32 mask, const void *data, int data_is, iter_info.srcu_idx = srcu_read_lock(&fsnotify_mark_srcu); - if ((mask & FS_MODIFY) || - (test_mask & to_tell->i_fsnotify_mask)) { - iter_info.marks[FSNOTIFY_OBJ_TYPE_INODE] = - fsnotify_first_mark(&to_tell->i_fsnotify_marks); - } - - if (mnt && ((mask & FS_MODIFY) || - (test_mask & mnt->mnt_fsnotify_mask))) { - iter_info.marks[FSNOTIFY_OBJ_TYPE_INODE] = - fsnotify_first_mark(&to_tell->i_fsnotify_marks); + iter_info.marks[FSNOTIFY_OBJ_TYPE_INODE] = + fsnotify_first_mark(&to_tell->i_fsnotify_marks); + if (mnt) { iter_info.marks[FSNOTIFY_OBJ_TYPE_VFSMOUNT] = fsnotify_first_mark(&mnt->mnt_fsnotify_marks); } diff --git a/fs/ocfs2/buffer_head_io.c b/fs/ocfs2/buffer_head_io.c index d9ebe11c8990..1d098c3c00e0 100644 --- a/fs/ocfs2/buffer_head_io.c +++ b/fs/ocfs2/buffer_head_io.c @@ -342,6 +342,7 @@ int ocfs2_read_blocks(struct ocfs2_caching_info *ci, u64 block, int nr, * for this bh as it's not marked locally * uptodate. */ status = -EIO; + clear_buffer_needs_validate(bh); put_bh(bh); bhs[i] = NULL; continue; diff --git a/fs/overlayfs/file.c b/fs/overlayfs/file.c index 32e9282893c9..aeaefd2a551b 100644 --- a/fs/overlayfs/file.c +++ b/fs/overlayfs/file.c @@ -131,9 +131,6 @@ static int ovl_open(struct inode *inode, struct file *file) if (IS_ERR(realfile)) return PTR_ERR(realfile); - /* For O_DIRECT dentry_open() checks f_mapping->a_ops->direct_IO */ - file->f_mapping = realfile->f_mapping; - file->private_data = realfile; return 0; @@ -334,6 +331,25 @@ static long ovl_fallocate(struct file *file, int mode, loff_t offset, loff_t len return ret; } +static int ovl_fadvise(struct file *file, loff_t offset, loff_t len, int advice) +{ + struct fd real; + const struct cred *old_cred; + int ret; + + ret = ovl_real_fdget(file, &real); + if (ret) + return ret; + + old_cred = ovl_override_creds(file_inode(file)->i_sb); + ret = vfs_fadvise(real.file, offset, len, advice); + revert_creds(old_cred); + + fdput(real); + + return ret; +} + static long ovl_real_ioctl(struct file *file, unsigned int cmd, unsigned long arg) { @@ -502,6 +518,7 @@ const struct file_operations ovl_file_operations = { .fsync = ovl_fsync, .mmap = ovl_mmap, .fallocate = ovl_fallocate, + .fadvise = ovl_fadvise, .unlocked_ioctl = ovl_ioctl, .compat_ioctl = ovl_compat_ioctl, diff --git a/fs/overlayfs/inode.c b/fs/overlayfs/inode.c index e0bb217c01e2..b6ac545b5a32 100644 --- a/fs/overlayfs/inode.c +++ b/fs/overlayfs/inode.c @@ -467,6 +467,10 @@ static int ovl_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, return -EOPNOTSUPP; old_cred = ovl_override_creds(inode->i_sb); + + if (fieinfo->fi_flags & FIEMAP_FLAG_SYNC) + filemap_write_and_wait(realinode->i_mapping); + err = realinode->i_op->fiemap(realinode, fieinfo, start, len); revert_creds(old_cred); @@ -500,6 +504,11 @@ static const struct inode_operations ovl_special_inode_operations = { .update_time = ovl_update_time, }; +const struct address_space_operations ovl_aops = { + /* For O_DIRECT dentry_open() checks f_mapping->a_ops->direct_IO */ + .direct_IO = noop_direct_IO, +}; + /* * It is possible to stack overlayfs instance on top of another * overlayfs instance as lower layer. We need to annonate the @@ -571,6 +580,7 @@ static void ovl_fill_inode(struct inode *inode, umode_t mode, dev_t rdev, case S_IFREG: inode->i_op = &ovl_file_inode_operations; inode->i_fop = &ovl_file_operations; + inode->i_mapping->a_ops = &ovl_aops; break; case S_IFDIR: diff --git a/fs/overlayfs/super.c b/fs/overlayfs/super.c index 2e0fc93c2c06..30adc9d408a0 100644 --- a/fs/overlayfs/super.c +++ b/fs/overlayfs/super.c @@ -982,16 +982,6 @@ static int ovl_get_upper(struct ovl_fs *ofs, struct path *upperpath) if (err) goto out; - err = -EBUSY; - if (ovl_inuse_trylock(upperpath->dentry)) { - ofs->upperdir_locked = true; - } else if (ofs->config.index) { - pr_err("overlayfs: upperdir is in-use by another mount, mount with '-o index=off' to override exclusive upperdir protection.\n"); - goto out; - } else { - pr_warn("overlayfs: upperdir is in-use by another mount, accessing files from both mounts will result in undefined behavior.\n"); - } - upper_mnt = clone_private_mount(upperpath); err = PTR_ERR(upper_mnt); if (IS_ERR(upper_mnt)) { @@ -1002,6 +992,17 @@ static int ovl_get_upper(struct ovl_fs *ofs, struct path *upperpath) /* Don't inherit atime flags */ upper_mnt->mnt_flags &= ~(MNT_NOATIME | MNT_NODIRATIME | MNT_RELATIME); ofs->upper_mnt = upper_mnt; + + err = -EBUSY; + if (ovl_inuse_trylock(ofs->upper_mnt->mnt_root)) { + ofs->upperdir_locked = true; + } else if (ofs->config.index) { + pr_err("overlayfs: upperdir is in-use by another mount, mount with '-o index=off' to override exclusive upperdir protection.\n"); + goto out; + } else { + pr_warn("overlayfs: upperdir is in-use by another mount, accessing files from both mounts will result in undefined behavior.\n"); + } + err = 0; out: return err; @@ -1101,8 +1102,10 @@ static int ovl_get_workdir(struct ovl_fs *ofs, struct path *upperpath) goto out; } + ofs->workbasedir = dget(workpath.dentry); + err = -EBUSY; - if (ovl_inuse_trylock(workpath.dentry)) { + if (ovl_inuse_trylock(ofs->workbasedir)) { ofs->workdir_locked = true; } else if (ofs->config.index) { pr_err("overlayfs: workdir is in-use by another mount, mount with '-o index=off' to override exclusive workdir protection.\n"); @@ -1111,7 +1114,6 @@ static int ovl_get_workdir(struct ovl_fs *ofs, struct path *upperpath) pr_warn("overlayfs: workdir is in-use by another mount, accessing files from both mounts will result in undefined behavior.\n"); } - ofs->workbasedir = dget(workpath.dentry); err = ovl_make_workdir(ofs, &workpath); if (err) goto out; diff --git a/fs/proc/kcore.c b/fs/proc/kcore.c index ad72261ee3fe..d297fe4472a9 100644 --- a/fs/proc/kcore.c +++ b/fs/proc/kcore.c @@ -464,6 +464,7 @@ read_kcore(struct file *file, char __user *buffer, size_t buflen, loff_t *fpos) ret = -EFAULT; goto out; } + m = NULL; /* skip the list anchor */ } else if (m->type == KCORE_VMALLOC) { vread(buf, (char *)start, tsz); /* we have to zero-fill user buffer even if no read */ diff --git a/fs/pstore/ram_core.c b/fs/pstore/ram_core.c index 951a14edcf51..0792595ebcfb 100644 --- a/fs/pstore/ram_core.c +++ b/fs/pstore/ram_core.c @@ -429,7 +429,12 @@ static void *persistent_ram_vmap(phys_addr_t start, size_t size, vaddr = vmap(pages, page_count, VM_MAP, prot); kfree(pages); - return vaddr; + /* + * Since vmap() uses page granularity, we must add the offset + * into the page here, to get the byte granularity address + * into the mapping to represent the actual "start" location. + */ + return vaddr + offset_in_page(start); } static void *persistent_ram_iomap(phys_addr_t start, size_t size, @@ -448,6 +453,11 @@ static void *persistent_ram_iomap(phys_addr_t start, size_t size, else va = ioremap_wc(start, size); + /* + * Since request_mem_region() and ioremap() are byte-granularity + * there is no need handle anything special like we do when the + * vmap() case in persistent_ram_vmap() above. + */ return va; } @@ -468,7 +478,7 @@ static int persistent_ram_buffer_map(phys_addr_t start, phys_addr_t size, return -ENOMEM; } - prz->buffer = prz->vaddr + offset_in_page(start); + prz->buffer = prz->vaddr; prz->buffer_size = size - sizeof(struct persistent_ram_buffer); return 0; @@ -515,7 +525,8 @@ void persistent_ram_free(struct persistent_ram_zone *prz) if (prz->vaddr) { if (pfn_valid(prz->paddr >> PAGE_SHIFT)) { - vunmap(prz->vaddr); + /* We must vunmap() at page-granularity. */ + vunmap(prz->vaddr - offset_in_page(prz->paddr)); } else { iounmap(prz->vaddr); release_mem_region(prz->paddr, prz->size); diff --git a/fs/ubifs/super.c b/fs/ubifs/super.c index 23e7042666a7..bf000c8aeffb 100644 --- a/fs/ubifs/super.c +++ b/fs/ubifs/super.c @@ -1912,7 +1912,9 @@ static int ubifs_remount_fs(struct super_block *sb, int *flags, char *data) mutex_unlock(&c->bu_mutex); } - ubifs_assert(c, c->lst.taken_empty_lebs > 0); + if (!c->need_recovery) + ubifs_assert(c, c->lst.taken_empty_lebs > 0); + return 0; } @@ -1954,6 +1956,9 @@ static struct ubi_volume_desc *open_ubi(const char *name, int mode) int dev, vol; char *endptr; + if (!name || !*name) + return ERR_PTR(-EINVAL); + /* First, try to open using the device node path method */ ubi = ubi_open_volume_path(name, mode); if (!IS_ERR(ubi)) diff --git a/fs/ubifs/xattr.c b/fs/ubifs/xattr.c index 61afdfee4b28..f5ad1ede7990 100644 --- a/fs/ubifs/xattr.c +++ b/fs/ubifs/xattr.c @@ -152,12 +152,6 @@ static int create_xattr(struct ubifs_info *c, struct inode *host, ui->data_len = size; mutex_lock(&host_ui->ui_mutex); - - if (!host->i_nlink) { - err = -ENOENT; - goto out_noent; - } - host->i_ctime = current_time(host); host_ui->xattr_cnt += 1; host_ui->xattr_size += CALC_DENT_SIZE(fname_len(nm)); @@ -190,7 +184,6 @@ out_cancel: host_ui->xattr_size -= CALC_XATTR_BYTES(size); host_ui->xattr_names -= fname_len(nm); host_ui->flags &= ~UBIFS_CRYPT_FL; -out_noent: mutex_unlock(&host_ui->ui_mutex); out_free: make_bad_inode(inode); @@ -242,12 +235,6 @@ static int change_xattr(struct ubifs_info *c, struct inode *host, mutex_unlock(&ui->ui_mutex); mutex_lock(&host_ui->ui_mutex); - - if (!host->i_nlink) { - err = -ENOENT; - goto out_noent; - } - host->i_ctime = current_time(host); host_ui->xattr_size -= CALC_XATTR_BYTES(old_size); host_ui->xattr_size += CALC_XATTR_BYTES(size); @@ -269,7 +256,6 @@ static int change_xattr(struct ubifs_info *c, struct inode *host, out_cancel: host_ui->xattr_size -= CALC_XATTR_BYTES(size); host_ui->xattr_size += CALC_XATTR_BYTES(old_size); -out_noent: mutex_unlock(&host_ui->ui_mutex); make_bad_inode(inode); out_free: @@ -496,12 +482,6 @@ static int remove_xattr(struct ubifs_info *c, struct inode *host, return err; mutex_lock(&host_ui->ui_mutex); - - if (!host->i_nlink) { - err = -ENOENT; - goto out_noent; - } - host->i_ctime = current_time(host); host_ui->xattr_cnt -= 1; host_ui->xattr_size -= CALC_DENT_SIZE(fname_len(nm)); @@ -521,7 +501,6 @@ out_cancel: host_ui->xattr_size += CALC_DENT_SIZE(fname_len(nm)); host_ui->xattr_size += CALC_XATTR_BYTES(ui->data_len); host_ui->xattr_names += fname_len(nm); -out_noent: mutex_unlock(&host_ui->ui_mutex); ubifs_release_budget(c, &req); make_bad_inode(inode); @@ -561,9 +540,6 @@ static int ubifs_xattr_remove(struct inode *host, const char *name) ubifs_assert(c, inode_is_locked(host)); - if (!host->i_nlink) - return -ENOENT; - if (fname_len(&nm) > UBIFS_MAX_NLEN) return -ENAMETOOLONG; diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h index 66d1d45fa2e1..d356f802945a 100644 --- a/include/asm-generic/io.h +++ b/include/asm-generic/io.h @@ -1026,7 +1026,8 @@ static inline void __iomem *ioremap_wt(phys_addr_t offset, size_t size) #define ioport_map ioport_map static inline void __iomem *ioport_map(unsigned long port, unsigned int nr) { - return PCI_IOBASE + (port & MMIO_UPPER_LIMIT); + port &= IO_SPACE_LIMIT; + return (port > MMIO_UPPER_LIMIT) ? NULL : PCI_IOBASE + port; } #endif diff --git a/include/drm/drm_drv.h b/include/drm/drm_drv.h index 8830e3de3a86..3199ef70c007 100644 --- a/include/drm/drm_drv.h +++ b/include/drm/drm_drv.h @@ -674,7 +674,7 @@ static inline bool drm_core_check_feature(struct drm_device *dev, u32 feature) static inline bool drm_drv_uses_atomic_modeset(struct drm_device *dev) { return drm_core_check_feature(dev, DRIVER_ATOMIC) || - dev->mode_config.funcs->atomic_commit != NULL; + (dev->mode_config.funcs && dev->mode_config.funcs->atomic_commit != NULL); } diff --git a/include/drm/drm_fourcc.h b/include/drm/drm_fourcc.h index fac831c40106..865ef60c17af 100644 --- a/include/drm/drm_fourcc.h +++ b/include/drm/drm_fourcc.h @@ -88,6 +88,8 @@ const struct drm_format_info * drm_get_format_info(struct drm_device *dev, const struct drm_mode_fb_cmd2 *mode_cmd); uint32_t drm_mode_legacy_fb_format(uint32_t bpp, uint32_t depth); +uint32_t drm_driver_legacy_fb_format(struct drm_device *dev, + uint32_t bpp, uint32_t depth); int drm_format_num_planes(uint32_t format); int drm_format_plane_cpp(uint32_t format, int plane); int drm_format_horz_chroma_subsampling(uint32_t format); diff --git a/include/drm/drm_panel.h b/include/drm/drm_panel.h index a82c292af6c5..8c738c0e6e9f 100644 --- a/include/drm/drm_panel.h +++ b/include/drm/drm_panel.h @@ -90,7 +90,6 @@ struct drm_panel { struct drm_device *drm; struct drm_connector *connector; struct device *dev; - struct device_link *link; const struct drm_panel_funcs *funcs; diff --git a/include/drm/drm_pciids.h b/include/drm/drm_pciids.h index 683742826511..b7e899ce44f0 100644 --- a/include/drm/drm_pciids.h +++ b/include/drm/drm_pciids.h @@ -1,4 +1,4 @@ -/* SPDX-License-Identifier: GPL-2.0 */ +/* SPDX-License-Identifier: MIT */ #define radeon_PCI_IDS \ {0x1002, 0x1304, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_KAVERI|RADEON_IS_MOBILITY|RADEON_NEW_MEMMAP|RADEON_IS_IGP}, \ {0x1002, 0x1305, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_KAVERI|RADEON_NEW_MEMMAP|RADEON_IS_IGP}, \ diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index daec50f887b3..d87b268f1781 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -175,8 +175,6 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f); * finished to remove the job from the * @drm_gpu_scheduler.ring_mirror_list. * @node: used to append this struct to the @drm_gpu_scheduler.ring_mirror_list. - * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the timeout - * interval is over. * @id: a unique id assigned to each job scheduled on the scheduler. * @karma: increment on every hang caused by this job. If this exceeds the hang * limit of the scheduler then the job is marked guilty and will not @@ -195,7 +193,6 @@ struct drm_sched_job { struct dma_fence_cb finish_cb; struct work_struct finish_work; struct list_head node; - struct delayed_work work_tdr; uint64_t id; atomic_t karma; enum drm_sched_priority s_priority; @@ -259,6 +256,8 @@ struct drm_sched_backend_ops { * finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. + * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the + * timeout interval is over. * @thread: the kthread on which the scheduler which run. * @ring_mirror_list: the list of jobs which are currently in the job queue. * @job_list_lock: lock to protect the ring_mirror_list. @@ -278,6 +277,7 @@ struct drm_gpu_scheduler { wait_queue_head_t job_scheduled; atomic_t hw_rq_count; atomic64_t job_id_count; + struct delayed_work work_tdr; struct task_struct *thread; struct list_head ring_mirror_list; spinlock_t job_list_lock; diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 8c19470785e2..3fc4854dce49 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -313,6 +313,24 @@ ttm_bo_reference(struct ttm_buffer_object *bo) } /** + * ttm_bo_get_unless_zero - reference a struct ttm_buffer_object unless + * its refcount has already reached zero. + * @bo: The buffer object. + * + * Used to reference a TTM buffer object in lookups where the object is removed + * from the lookup structure during the destructor and for RCU lookups. + * + * Returns: @bo if the referencing was successful, NULL otherwise. + */ +static inline __must_check struct ttm_buffer_object * +ttm_bo_get_unless_zero(struct ttm_buffer_object *bo) +{ + if (!kref_get_unless_zero(&bo->kref)) + return NULL; + return bo; +} + +/** * ttm_bo_wait - wait for buffer idle. * * @bo: The buffer object. diff --git a/include/linux/blk-cgroup.h b/include/linux/blk-cgroup.h index 34aec30e06c7..6d766a19f2bb 100644 --- a/include/linux/blk-cgroup.h +++ b/include/linux/blk-cgroup.h @@ -56,6 +56,7 @@ struct blkcg { struct list_head all_blkcgs_node; #ifdef CONFIG_CGROUP_WRITEBACK struct list_head cgwb_list; + refcount_t cgwb_refcnt; #endif }; @@ -89,7 +90,6 @@ struct blkg_policy_data { /* the blkg and policy id this per-policy data belongs to */ struct blkcg_gq *blkg; int plid; - bool offline; }; /* @@ -387,6 +387,49 @@ static inline struct blkcg *cpd_to_blkcg(struct blkcg_policy_data *cpd) return cpd ? cpd->blkcg : NULL; } +extern void blkcg_destroy_blkgs(struct blkcg *blkcg); + +#ifdef CONFIG_CGROUP_WRITEBACK + +/** + * blkcg_cgwb_get - get a reference for blkcg->cgwb_list + * @blkcg: blkcg of interest + * + * This is used to track the number of active wb's related to a blkcg. + */ +static inline void blkcg_cgwb_get(struct blkcg *blkcg) +{ + refcount_inc(&blkcg->cgwb_refcnt); +} + +/** + * blkcg_cgwb_put - put a reference for @blkcg->cgwb_list + * @blkcg: blkcg of interest + * + * This is used to track the number of active wb's related to a blkcg. + * When this count goes to zero, all active wb has finished so the + * blkcg can continue destruction by calling blkcg_destroy_blkgs(). + * This work may occur in cgwb_release_workfn() on the cgwb_release + * workqueue. + */ +static inline void blkcg_cgwb_put(struct blkcg *blkcg) +{ + if (refcount_dec_and_test(&blkcg->cgwb_refcnt)) + blkcg_destroy_blkgs(blkcg); +} + +#else + +static inline void blkcg_cgwb_get(struct blkcg *blkcg) { } + +static inline void blkcg_cgwb_put(struct blkcg *blkcg) +{ + /* wb isn't being accounted, so trigger destruction right away */ + blkcg_destroy_blkgs(blkcg); +} + +#endif + /** * blkg_path - format cgroup path of blkg * @blkg: blkg of interest diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index d6869e0e2b64..6980014357d4 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -54,7 +54,7 @@ struct blk_stat_callback; * Maximum number of blkcg policies allowed to be registered concurrently. * Defined here to simplify include dependency. */ -#define BLKCG_MAX_POLS 3 +#define BLKCG_MAX_POLS 5 typedef void (rq_end_io_fn)(struct request *, blk_status_t); diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h index 763bbad1e258..4d36b27214fd 100644 --- a/include/linux/compiler-gcc.h +++ b/include/linux/compiler-gcc.h @@ -79,20 +79,6 @@ #define __noretpoline __attribute__((indirect_branch("keep"))) #endif -/* - * it doesn't make sense on ARM (currently the only user of __naked) - * to trace naked functions because then mcount is called without - * stack and frame pointer being set up and there is no chance to - * restore the lr register to the value before mcount was called. - * - * The asm() bodies of naked functions often depend on standard calling - * conventions, therefore they must be noinline and noclone. - * - * GCC 4.[56] currently fail to enforce this, so we must do so ourselves. - * See GCC PR44290. - */ -#define __naked __attribute__((naked)) noinline __noclone notrace - #define __UNIQUE_ID(prefix) __PASTE(__PASTE(__UNIQUE_ID_, prefix), __COUNTER__) #define __optimize(level) __attribute__((__optimize__(level))) diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index 3525c179698c..db192becfec4 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -226,6 +226,14 @@ struct ftrace_likely_data { #define notrace __attribute__((no_instrument_function)) #endif +/* + * it doesn't make sense on ARM (currently the only user of __naked) + * to trace naked functions because then mcount is called without + * stack and frame pointer being set up and there is no chance to + * restore the lr register to the value before mcount was called. + */ +#define __naked __attribute__((naked)) notrace + #define __compiler_offsetof(a, b) __builtin_offsetof(a, b) /* diff --git a/include/linux/fs.h b/include/linux/fs.h index 33322702c910..6c0b4a1c22ff 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1763,6 +1763,7 @@ struct file_operations { u64); int (*dedupe_file_range)(struct file *, loff_t, struct file *, loff_t, u64); + int (*fadvise)(struct file *, loff_t, loff_t, int); } __randomize_layout; struct inode_operations { @@ -3459,4 +3460,8 @@ static inline bool dir_relax_shared(struct inode *inode) extern bool path_noexec(const struct path *path); extern void inode_nohighmem(struct inode *inode); +/* mm/fadvise.c */ +extern int vfs_fadvise(struct file *file, loff_t offset, loff_t len, + int advice); + #endif /* _LINUX_FS_H */ diff --git a/include/linux/genhd.h b/include/linux/genhd.h index 57864422a2c8..25c08c6c7f99 100644 --- a/include/linux/genhd.h +++ b/include/linux/genhd.h @@ -83,10 +83,10 @@ struct partition { } __attribute__((packed)); struct disk_stats { + u64 nsecs[NR_STAT_GROUPS]; unsigned long sectors[NR_STAT_GROUPS]; unsigned long ios[NR_STAT_GROUPS]; unsigned long merges[NR_STAT_GROUPS]; - unsigned long ticks[NR_STAT_GROUPS]; unsigned long io_ticks; unsigned long time_in_queue; }; @@ -354,6 +354,9 @@ static inline void free_part_stats(struct hd_struct *part) #endif /* CONFIG_SMP */ +#define part_stat_read_msecs(part, which) \ + div_u64(part_stat_read(part, nsecs[which]), NSEC_PER_MSEC) + #define part_stat_read_accum(part, field) \ (part_stat_read(part, field[STAT_READ]) + \ part_stat_read(part, field[STAT_WRITE]) + \ diff --git a/include/linux/hid.h b/include/linux/hid.h index 834e6461a690..d44a78362942 100644 --- a/include/linux/hid.h +++ b/include/linux/hid.h @@ -526,6 +526,7 @@ struct hid_input { const char *name; bool registered; struct list_head reports; /* the list of reports */ + unsigned int application; /* application usage for this input */ }; enum hid_type { diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h index 0205aee44ded..c926698040e0 100644 --- a/include/linux/kvm_host.h +++ b/include/linux/kvm_host.h @@ -733,8 +733,6 @@ bool kvm_vcpu_wake_up(struct kvm_vcpu *vcpu); void kvm_vcpu_kick(struct kvm_vcpu *vcpu); int kvm_vcpu_yield_to(struct kvm_vcpu *target); void kvm_vcpu_on_spin(struct kvm_vcpu *vcpu, bool usermode_vcpu_not_eligible); -void kvm_load_guest_fpu(struct kvm_vcpu *vcpu); -void kvm_put_guest_fpu(struct kvm_vcpu *vcpu); void kvm_flush_remote_tlbs(struct kvm *kvm); void kvm_reload_remote_mmus(struct kvm *kvm); diff --git a/include/linux/list.h b/include/linux/list.h index de04cc5ed536..edb7628e46ed 100644 --- a/include/linux/list.h +++ b/include/linux/list.h @@ -184,6 +184,29 @@ static inline void list_move_tail(struct list_head *list, } /** + * list_bulk_move_tail - move a subsection of a list to its tail + * @head: the head that will follow our entry + * @first: first entry to move + * @last: last entry to move, can be the same as first + * + * Move all entries between @first and including @last before @head. + * All three entries must belong to the same linked list. + */ +static inline void list_bulk_move_tail(struct list_head *head, + struct list_head *first, + struct list_head *last) +{ + first->prev->next = last->next; + last->next->prev = first->prev; + + head->prev->next = first; + first->prev = head->prev; + + last->next = head; + head->prev = last; +} + +/** * list_is_last - tests whether @list is the last entry in list @head * @list: the entry to test * @head: the head of the list diff --git a/include/linux/mfd/da9063/pdata.h b/include/linux/mfd/da9063/pdata.h index 8a125701ef7b..50bed4f89c1a 100644 --- a/include/linux/mfd/da9063/pdata.h +++ b/include/linux/mfd/da9063/pdata.h @@ -21,7 +21,7 @@ /* * Regulator configuration */ -/* DA9063 regulator IDs */ +/* DA9063 and DA9063L regulator IDs */ enum { /* BUCKs */ DA9063_ID_BCORE1, @@ -37,18 +37,20 @@ enum { DA9063_ID_BMEM_BIO_MERGED, /* When two BUCKs are merged, they cannot be reused separately */ - /* LDOs */ + /* LDOs on both DA9063 and DA9063L */ + DA9063_ID_LDO3, + DA9063_ID_LDO7, + DA9063_ID_LDO8, + DA9063_ID_LDO9, + DA9063_ID_LDO11, + + /* DA9063-only LDOs */ DA9063_ID_LDO1, DA9063_ID_LDO2, - DA9063_ID_LDO3, DA9063_ID_LDO4, DA9063_ID_LDO5, DA9063_ID_LDO6, - DA9063_ID_LDO7, - DA9063_ID_LDO8, - DA9063_ID_LDO9, DA9063_ID_LDO10, - DA9063_ID_LDO11, }; /* Regulators platform data */ diff --git a/include/linux/mfd/rohm-bd718x7.h b/include/linux/mfd/rohm-bd718x7.h index a528747f8aed..e8338e5dc10b 100644 --- a/include/linux/mfd/rohm-bd718x7.h +++ b/include/linux/mfd/rohm-bd718x7.h @@ -78,9 +78,9 @@ enum { BD71837_REG_TRANS_COND0 = 0x1F, BD71837_REG_TRANS_COND1 = 0x20, BD71837_REG_VRFAULTEN = 0x21, - BD71837_REG_MVRFLTMASK0 = 0x22, - BD71837_REG_MVRFLTMASK1 = 0x23, - BD71837_REG_MVRFLTMASK2 = 0x24, + BD718XX_REG_MVRFLTMASK0 = 0x22, + BD718XX_REG_MVRFLTMASK1 = 0x23, + BD718XX_REG_MVRFLTMASK2 = 0x24, BD71837_REG_RCVCFG = 0x25, BD71837_REG_RCVNUM = 0x26, BD71837_REG_PWRONCONFIG0 = 0x27, @@ -159,6 +159,33 @@ enum { #define BUCK8_MASK 0x3F #define BUCK8_DEFAULT 0x1E +/* BD718XX Voltage monitoring masks */ +#define BD718XX_BUCK1_VRMON80 0x1 +#define BD718XX_BUCK1_VRMON130 0x2 +#define BD718XX_BUCK2_VRMON80 0x4 +#define BD718XX_BUCK2_VRMON130 0x8 +#define BD718XX_1ST_NODVS_BUCK_VRMON80 0x1 +#define BD718XX_1ST_NODVS_BUCK_VRMON130 0x2 +#define BD718XX_2ND_NODVS_BUCK_VRMON80 0x4 +#define BD718XX_2ND_NODVS_BUCK_VRMON130 0x8 +#define BD718XX_3RD_NODVS_BUCK_VRMON80 0x10 +#define BD718XX_3RD_NODVS_BUCK_VRMON130 0x20 +#define BD718XX_4TH_NODVS_BUCK_VRMON80 0x40 +#define BD718XX_4TH_NODVS_BUCK_VRMON130 0x80 +#define BD718XX_LDO1_VRMON80 0x1 +#define BD718XX_LDO2_VRMON80 0x2 +#define BD718XX_LDO3_VRMON80 0x4 +#define BD718XX_LDO4_VRMON80 0x8 +#define BD718XX_LDO5_VRMON80 0x10 +#define BD718XX_LDO6_VRMON80 0x20 + +/* BD71837 specific voltage monitoring masks */ +#define BD71837_BUCK3_VRMON80 0x10 +#define BD71837_BUCK3_VRMON130 0x20 +#define BD71837_BUCK4_VRMON80 0x40 +#define BD71837_BUCK4_VRMON130 0x80 +#define BD71837_LDO7_VRMON80 0x40 + /* BD71837_REG_IRQ bits */ #define IRQ_SWRST 0x40 #define IRQ_PWRON_S 0x20 diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 7a452716de4b..66d94b4557cf 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -362,8 +362,8 @@ struct mlx5_frag_buf { struct mlx5_frag_buf_ctrl { struct mlx5_frag_buf frag_buf; u32 sz_m1; - u32 frag_sz_m1; - u32 strides_offset; + u16 frag_sz_m1; + u16 strides_offset; u8 log_sz; u8 log_stride; u8 log_frag_strides; @@ -995,7 +995,7 @@ static inline u32 mlx5_base_mkey(const u32 key) } static inline void mlx5_fill_fbc_offset(u8 log_stride, u8 log_sz, - u32 strides_offset, + u16 strides_offset, struct mlx5_frag_buf_ctrl *fbc) { fbc->log_stride = log_stride; @@ -1052,7 +1052,7 @@ int mlx5_cmd_free_uar(struct mlx5_core_dev *dev, u32 uarn); void mlx5_health_cleanup(struct mlx5_core_dev *dev); int mlx5_health_init(struct mlx5_core_dev *dev); void mlx5_start_health_poll(struct mlx5_core_dev *dev); -void mlx5_stop_health_poll(struct mlx5_core_dev *dev); +void mlx5_stop_health_poll(struct mlx5_core_dev *dev, bool disable_health); void mlx5_drain_health_wq(struct mlx5_core_dev *dev); void mlx5_trigger_health_work(struct mlx5_core_dev *dev); void mlx5_drain_health_recovery(struct mlx5_core_dev *dev); diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index cd2bc939efd0..5ed8f6292a53 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -341,7 +341,7 @@ struct mm_struct { struct { struct vm_area_struct *mmap; /* list of VMAs */ struct rb_root mm_rb; - u32 vmacache_seqnum; /* per-thread vmacache */ + u64 vmacache_seqnum; /* per-thread vmacache */ #ifdef CONFIG_MMU unsigned long (*get_unmapped_area) (struct file *filp, unsigned long addr, unsigned long len, diff --git a/include/linux/mm_types_task.h b/include/linux/mm_types_task.h index 5fe87687664c..d7016dcb245e 100644 --- a/include/linux/mm_types_task.h +++ b/include/linux/mm_types_task.h @@ -32,7 +32,7 @@ #define VMACACHE_MASK (VMACACHE_SIZE - 1) struct vmacache { - u32 seqnum; + u64 seqnum; struct vm_area_struct *vmas[VMACACHE_SIZE]; }; diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h index 1298a7daa57d..01797cb4587e 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -754,6 +754,7 @@ struct tb_service_id { * struct typec_device_id - USB Type-C alternate mode identifiers * @svid: Standard or Vendor ID * @mode: Mode index + * @driver_data: Driver specific data */ struct typec_device_id { __u16 svid; diff --git a/include/linux/netpoll.h b/include/linux/netpoll.h index 67662d01130a..3ef82d3a78db 100644 --- a/include/linux/netpoll.h +++ b/include/linux/netpoll.h @@ -49,8 +49,9 @@ struct netpoll_info { }; #ifdef CONFIG_NETPOLL -extern void netpoll_poll_disable(struct net_device *dev); -extern void netpoll_poll_enable(struct net_device *dev); +void netpoll_poll_dev(struct net_device *dev); +void netpoll_poll_disable(struct net_device *dev); +void netpoll_poll_enable(struct net_device *dev); #else static inline void netpoll_poll_disable(struct net_device *dev) { return; } static inline void netpoll_poll_enable(struct net_device *dev) { return; } diff --git a/include/linux/pci.h b/include/linux/pci.h index e72ca8dd6241..6925828f9f25 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1235,6 +1235,9 @@ void pci_bus_remove_resources(struct pci_bus *bus); int devm_request_pci_bus_resources(struct device *dev, struct list_head *resources); +/* Temporary until new and working PCI SBR API in place */ +int pci_bridge_secondary_bus_reset(struct pci_dev *dev); + #define pci_bus_for_each_resource(bus, res, i) \ for (i = 0; \ (res = pci_bus_resource_n(bus, i)) || i < PCI_BRIDGE_RESOURCE_NUM; \ diff --git a/include/linux/pci_ids.h b/include/linux/pci_ids.h index 99d366cb0e9f..d157983b84cf 100644 --- a/include/linux/pci_ids.h +++ b/include/linux/pci_ids.h @@ -3084,4 +3084,6 @@ #define PCI_VENDOR_ID_OCZ 0x1b85 +#define PCI_VENDOR_ID_NCUBE 0x10ff + #endif /* _LINUX_PCI_IDS_H */ diff --git a/include/linux/regulator/machine.h b/include/linux/regulator/machine.h index 3468703d663a..a459a5e973a7 100644 --- a/include/linux/regulator/machine.h +++ b/include/linux/regulator/machine.h @@ -48,9 +48,9 @@ struct regulator; * DISABLE_IN_SUSPEND - turn off regulator in suspend states * ENABLE_IN_SUSPEND - keep regulator on in suspend states */ -#define DO_NOTHING_IN_SUSPEND (-1) -#define DISABLE_IN_SUSPEND 0 -#define ENABLE_IN_SUSPEND 1 +#define DO_NOTHING_IN_SUSPEND 0 +#define DISABLE_IN_SUSPEND 1 +#define ENABLE_IN_SUSPEND 2 /* Regulator active discharge flags */ enum regulator_active_discharge { diff --git a/include/linux/spi/spi-mem.h b/include/linux/spi/spi-mem.h index b2bd4b4127c4..69ee30456864 100644 --- a/include/linux/spi/spi-mem.h +++ b/include/linux/spi/spi-mem.h @@ -81,8 +81,10 @@ enum spi_mem_data_dir { * @dummy.buswidth: number of IO lanes used to transmit the dummy bytes * @data.buswidth: number of IO lanes used to send/receive the data * @data.dir: direction of the transfer - * @data.buf.in: input buffer - * @data.buf.out: output buffer + * @data.nbytes: number of data bytes to send/receive. Can be zero if the + * operation does not involve transferring data + * @data.buf.in: input buffer (must be DMA-able) + * @data.buf.out: output buffer (must be DMA-able) */ struct spi_mem_op { struct { @@ -105,7 +107,6 @@ struct spi_mem_op { u8 buswidth; enum spi_mem_data_dir dir; unsigned int nbytes; - /* buf.{in,out} must be DMA-able. */ union { void *in; const void *out; diff --git a/include/linux/stmmac.h b/include/linux/stmmac.h index c43e9a01b892..7ddfc65586b0 100644 --- a/include/linux/stmmac.h +++ b/include/linux/stmmac.h @@ -30,6 +30,7 @@ #define MTL_MAX_RX_QUEUES 8 #define MTL_MAX_TX_QUEUES 8 +#define STMMAC_CH_MAX 8 #define STMMAC_RX_COE_NONE 0 #define STMMAC_RX_COE_TYPE1 1 diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h index 5d738804e3d6..a5a3cfc3c2fa 100644 --- a/include/linux/timekeeping.h +++ b/include/linux/timekeeping.h @@ -258,8 +258,8 @@ extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot); extern int persistent_clock_is_local; extern void read_persistent_clock64(struct timespec64 *ts); -void read_persistent_clock_and_boot_offset(struct timespec64 *wall_clock, - struct timespec64 *boot_offset); +void read_persistent_wall_and_boot_offset(struct timespec64 *wall_clock, + struct timespec64 *boot_offset); extern int update_persistent_clock64(struct timespec64 now); /* diff --git a/include/linux/tracepoint.h b/include/linux/tracepoint.h index 7f2e16e76ac4..041f7e56a289 100644 --- a/include/linux/tracepoint.h +++ b/include/linux/tracepoint.h @@ -158,8 +158,10 @@ extern void syscall_unregfunc(void); * For rcuidle callers, use srcu since sched-rcu \ * doesn't work from the idle path. \ */ \ - if (rcuidle) \ + if (rcuidle) { \ idx = srcu_read_lock_notrace(&tracepoint_srcu); \ + rcu_irq_enter_irqson(); \ + } \ \ it_func_ptr = rcu_dereference_raw((tp)->funcs); \ \ @@ -171,8 +173,10 @@ extern void syscall_unregfunc(void); } while ((++it_func_ptr)->func); \ } \ \ - if (rcuidle) \ + if (rcuidle) { \ + rcu_irq_exit_irqson(); \ srcu_read_unlock_notrace(&tracepoint_srcu, idx);\ + } \ \ preempt_enable_notrace(); \ } while (0) diff --git a/include/linux/uio.h b/include/linux/uio.h index 409c845d4cd3..422b1c01ee0d 100644 --- a/include/linux/uio.h +++ b/include/linux/uio.h @@ -172,7 +172,7 @@ size_t copy_from_iter_flushcache(void *addr, size_t bytes, struct iov_iter *i) static __always_inline __must_check size_t copy_to_iter_mcsafe(void *addr, size_t bytes, struct iov_iter *i) { - if (unlikely(!check_copy_size(addr, bytes, false))) + if (unlikely(!check_copy_size(addr, bytes, true))) return 0; else return _copy_to_iter_mcsafe(addr, bytes, i); diff --git a/include/linux/vga_switcheroo.h b/include/linux/vga_switcheroo.h index a34539b7f750..7e6ac0114d55 100644 --- a/include/linux/vga_switcheroo.h +++ b/include/linux/vga_switcheroo.h @@ -133,15 +133,18 @@ struct vga_switcheroo_handler { * @can_switch: check if the device is in a position to switch now. * Mandatory. The client should return false if a user space process * has one of its device files open + * @gpu_bound: notify the client id to audio client when the GPU is bound. * * Client callbacks. A client can be either a GPU or an audio device on a GPU. * The @set_gpu_state and @can_switch methods are mandatory, @reprobe may be * set to NULL. For audio clients, the @reprobe member is bogus. + * OTOH, @gpu_bound is only for audio clients, and not used for GPU clients. */ struct vga_switcheroo_client_ops { void (*set_gpu_state)(struct pci_dev *dev, enum vga_switcheroo_state); void (*reprobe)(struct pci_dev *dev); bool (*can_switch)(struct pci_dev *dev); + void (*gpu_bound)(struct pci_dev *dev, enum vga_switcheroo_client_id); }; #if defined(CONFIG_VGA_SWITCHEROO) diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h index 5c7f010676a7..47a3441cf4c4 100644 --- a/include/linux/vm_event_item.h +++ b/include/linux/vm_event_item.h @@ -105,7 +105,6 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, #ifdef CONFIG_DEBUG_VM_VMACACHE VMACACHE_FIND_CALLS, VMACACHE_FIND_HITS, - VMACACHE_FULL_FLUSHES, #endif #ifdef CONFIG_SWAP SWAP_RA, diff --git a/include/linux/vmacache.h b/include/linux/vmacache.h index 3e9a963edd6a..6fce268a4588 100644 --- a/include/linux/vmacache.h +++ b/include/linux/vmacache.h @@ -10,7 +10,6 @@ static inline void vmacache_flush(struct task_struct *tsk) memset(tsk->vmacache.vmas, 0, sizeof(tsk->vmacache.vmas)); } -extern void vmacache_flush_all(struct mm_struct *mm); extern void vmacache_update(unsigned long addr, struct vm_area_struct *newvma); extern struct vm_area_struct *vmacache_find(struct mm_struct *mm, unsigned long addr); @@ -24,10 +23,6 @@ extern struct vm_area_struct *vmacache_find_exact(struct mm_struct *mm, static inline void vmacache_invalidate(struct mm_struct *mm) { mm->vmacache_seqnum++; - - /* deal with overflows */ - if (unlikely(mm->vmacache_seqnum == 0)) - vmacache_flush_all(mm); } #endif /* __LINUX_VMACACHE_H */ diff --git a/include/net/cfg80211.h b/include/net/cfg80211.h index 9a850973e09a..8ebabc9873d1 100644 --- a/include/net/cfg80211.h +++ b/include/net/cfg80211.h @@ -4865,8 +4865,8 @@ const char *reg_initiator_name(enum nl80211_reg_initiator initiator); * * Return: 0 on success. -ENODATA. */ -int reg_query_regdb_wmm(char *alpha2, int freq, u32 *ptr, - struct ieee80211_wmm_rule *rule); +int reg_query_regdb_wmm(char *alpha2, int freq, + struct ieee80211_reg_rule *rule); /* * callbacks for asynchronous cfg80211 methods, notification diff --git a/include/net/netfilter/nf_conntrack_timeout.h b/include/net/netfilter/nf_conntrack_timeout.h index d5f62cc6c2ae..3394d75e1c80 100644 --- a/include/net/netfilter/nf_conntrack_timeout.h +++ b/include/net/netfilter/nf_conntrack_timeout.h @@ -30,7 +30,7 @@ struct nf_conn_timeout { }; static inline unsigned int * -nf_ct_timeout_data(struct nf_conn_timeout *t) +nf_ct_timeout_data(const struct nf_conn_timeout *t) { struct nf_ct_timeout *timeout; diff --git a/include/net/nfc/hci.h b/include/net/nfc/hci.h index 316694dafa5b..008f466d1da7 100644 --- a/include/net/nfc/hci.h +++ b/include/net/nfc/hci.h @@ -87,7 +87,7 @@ struct nfc_hci_pipe { * According to specification 102 622 chapter 4.4 Pipes, * the pipe identifier is 7 bits long. */ -#define NFC_HCI_MAX_PIPES 127 +#define NFC_HCI_MAX_PIPES 128 struct nfc_hci_init_data { u8 gate_count; struct nfc_hci_gate gates[NFC_HCI_MAX_CUSTOM_GATES]; diff --git a/include/net/regulatory.h b/include/net/regulatory.h index 60f8cc86a447..3469750df0f4 100644 --- a/include/net/regulatory.h +++ b/include/net/regulatory.h @@ -217,15 +217,15 @@ struct ieee80211_wmm_rule { struct ieee80211_reg_rule { struct ieee80211_freq_range freq_range; struct ieee80211_power_rule power_rule; - struct ieee80211_wmm_rule *wmm_rule; + struct ieee80211_wmm_rule wmm_rule; u32 flags; u32 dfs_cac_ms; + bool has_wmm; }; struct ieee80211_regdomain { struct rcu_head rcu_head; u32 n_reg_rules; - u32 n_wmm_rules; char alpha2[3]; enum nl80211_dfs_regions dfs_region; struct ieee80211_reg_rule reg_rules[]; diff --git a/include/net/tls.h b/include/net/tls.h index d5c683e8bb22..0a769cf2f5f3 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -171,15 +171,14 @@ struct cipher_context { char *rec_seq; }; +union tls_crypto_context { + struct tls_crypto_info info; + struct tls12_crypto_info_aes_gcm_128 aes_gcm_128; +}; + struct tls_context { - union { - struct tls_crypto_info crypto_send; - struct tls12_crypto_info_aes_gcm_128 crypto_send_aes_gcm_128; - }; - union { - struct tls_crypto_info crypto_recv; - struct tls12_crypto_info_aes_gcm_128 crypto_recv_aes_gcm_128; - }; + union tls_crypto_context crypto_send; + union tls_crypto_context crypto_recv; struct list_head list; struct net_device *netdev; @@ -367,8 +366,8 @@ static inline void tls_fill_prepend(struct tls_context *ctx, * size KTLS_DTLS_HEADER_SIZE + KTLS_DTLS_NONCE_EXPLICIT_SIZE */ buf[0] = record_type; - buf[1] = TLS_VERSION_MINOR(ctx->crypto_send.version); - buf[2] = TLS_VERSION_MAJOR(ctx->crypto_send.version); + buf[1] = TLS_VERSION_MINOR(ctx->crypto_send.info.version); + buf[2] = TLS_VERSION_MAJOR(ctx->crypto_send.info.version); /* we can use IV for nonce explicit according to spec */ buf[3] = pkt_len >> 8; buf[4] = pkt_len & 0xFF; diff --git a/include/sound/hdaudio.h b/include/sound/hdaudio.h index 6f1e1f3b3063..cd1773d0e08f 100644 --- a/include/sound/hdaudio.h +++ b/include/sound/hdaudio.h @@ -412,6 +412,7 @@ void snd_hdac_bus_init_cmd_io(struct hdac_bus *bus); void snd_hdac_bus_stop_cmd_io(struct hdac_bus *bus); void snd_hdac_bus_enter_link_reset(struct hdac_bus *bus); void snd_hdac_bus_exit_link_reset(struct hdac_bus *bus); +int snd_hdac_bus_reset_link(struct hdac_bus *bus, bool full_reset); void snd_hdac_bus_update_rirb(struct hdac_bus *bus); int snd_hdac_bus_handle_stream_irq(struct hdac_bus *bus, unsigned int status, diff --git a/include/sound/soc-dapm.h b/include/sound/soc-dapm.h index af9ef16cc34d..fdaaafdc7a00 100644 --- a/include/sound/soc-dapm.h +++ b/include/sound/soc-dapm.h @@ -407,6 +407,7 @@ int snd_soc_dapm_new_dai_widgets(struct snd_soc_dapm_context *dapm, int snd_soc_dapm_link_dai_widgets(struct snd_soc_card *card); void snd_soc_dapm_connect_dai_link_widgets(struct snd_soc_card *card); int snd_soc_dapm_new_pcm(struct snd_soc_card *card, + struct snd_soc_pcm_runtime *rtd, const struct snd_soc_pcm_stream *params, unsigned int num_params, struct snd_soc_dapm_widget *source, diff --git a/include/uapi/drm/drm_fourcc.h b/include/uapi/drm/drm_fourcc.h index 139632b87181..0cd40ebfa1b1 100644 --- a/include/uapi/drm/drm_fourcc.h +++ b/include/uapi/drm/drm_fourcc.h @@ -339,6 +339,15 @@ extern "C" { #define DRM_FORMAT_MOD_SAMSUNG_64_32_TILE fourcc_mod_code(SAMSUNG, 1) /* + * Tiled, 16 (pixels) x 16 (lines) - sized macroblocks + * + * This is a simple tiled layout using tiles of 16x16 pixels in a row-major + * layout. For YCbCr formats Cb/Cr components are taken in such a way that + * they correspond to their 16x16 luma block. + */ +#define DRM_FORMAT_MOD_SAMSUNG_16_16_TILE fourcc_mod_code(SAMSUNG, 2) + +/* * Qualcomm Compressed Format * * Refers to a compressed variant of the base format that is compressed. diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h index 01674b56e14f..f5ff8a76e208 100644 --- a/include/uapi/linux/kfd_ioctl.h +++ b/include/uapi/linux/kfd_ioctl.h @@ -82,6 +82,14 @@ struct kfd_ioctl_set_cu_mask_args { __u64 cu_mask_ptr; /* to KFD */ }; +struct kfd_ioctl_get_queue_wave_state_args { + uint64_t ctl_stack_address; /* to KFD */ + uint32_t ctl_stack_used_size; /* from KFD */ + uint32_t save_area_used_size; /* from KFD */ + uint32_t queue_id; /* to KFD */ + uint32_t pad; +}; + /* For kfd_ioctl_set_memory_policy_args.default_policy and alternate_policy */ #define KFD_IOC_CACHE_POLICY_COHERENT 0 #define KFD_IOC_CACHE_POLICY_NONCOHERENT 1 @@ -475,7 +483,10 @@ struct kfd_ioctl_unmap_memory_from_gpu_args { #define AMDKFD_IOC_SET_CU_MASK \ AMDKFD_IOW(0x1A, struct kfd_ioctl_set_cu_mask_args) +#define AMDKFD_IOC_GET_QUEUE_WAVE_STATE \ + AMDKFD_IOWR(0x1B, struct kfd_ioctl_get_queue_wave_state_args) + #define AMDKFD_COMMAND_START 0x01 -#define AMDKFD_COMMAND_END 0x1B +#define AMDKFD_COMMAND_END 0x1C #endif diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h index 07548de5c988..251be353f950 100644 --- a/include/uapi/linux/kvm.h +++ b/include/uapi/linux/kvm.h @@ -952,6 +952,7 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_S390_HPAGE_1M 156 #define KVM_CAP_NESTED_STATE 157 #define KVM_CAP_ARM_INJECT_SERROR_ESR 158 +#define KVM_CAP_MSR_PLATFORM_INFO 159 #ifdef KVM_CAP_IRQ_ROUTING diff --git a/include/uapi/linux/perf_event.h b/include/uapi/linux/perf_event.h index eeb787b1c53c..f35eb72739c0 100644 --- a/include/uapi/linux/perf_event.h +++ b/include/uapi/linux/perf_event.h @@ -144,7 +144,7 @@ enum perf_event_sample_format { PERF_SAMPLE_MAX = 1U << 20, /* non-ABI */ - __PERF_SAMPLE_CALLCHAIN_EARLY = 1ULL << 63, + __PERF_SAMPLE_CALLCHAIN_EARLY = 1ULL << 63, /* non-ABI; internal use */ }; /* diff --git a/include/uapi/linux/rds.h b/include/uapi/linux/rds.h index dc520e1a4123..8b73cb603c5f 100644 --- a/include/uapi/linux/rds.h +++ b/include/uapi/linux/rds.h @@ -37,6 +37,7 @@ #include <linux/types.h> #include <linux/socket.h> /* For __kernel_sockaddr_storage. */ +#include <linux/in6.h> /* For struct in6_addr. */ #define RDS_IB_ABI_VERSION 0x301 diff --git a/include/uapi/linux/vhost.h b/include/uapi/linux/vhost.h index b1e22c40c4b6..84c3de89696a 100644 --- a/include/uapi/linux/vhost.h +++ b/include/uapi/linux/vhost.h @@ -176,7 +176,7 @@ struct vhost_memory { #define VHOST_BACKEND_F_IOTLB_MSG_V2 0x1 #define VHOST_SET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x25, __u64) -#define VHOST_GET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x26, __u64) +#define VHOST_GET_BACKEND_FEATURES _IOR(VHOST_VIRTIO, 0x26, __u64) /* VHOST_NET specific defines */ diff --git a/include/uapi/sound/skl-tplg-interface.h b/include/uapi/sound/skl-tplg-interface.h index f58cafa42f18..f39352cef382 100644 --- a/include/uapi/sound/skl-tplg-interface.h +++ b/include/uapi/sound/skl-tplg-interface.h @@ -10,6 +10,8 @@ #ifndef __HDA_TPLG_INTERFACE_H__ #define __HDA_TPLG_INTERFACE_H__ +#include <linux/types.h> + /* * Default types range from 0~12. type can range from 0 to 0xff * SST types start at higher to avoid any overlapping in future @@ -143,10 +145,10 @@ enum skl_module_param_type { }; struct skl_dfw_algo_data { - u32 set_params:2; - u32 rsvd:30; - u32 param_id; - u32 max; + __u32 set_params:2; + __u32 rsvd:30; + __u32 param_id; + __u32 max; char params[0]; } __packed; @@ -163,68 +165,68 @@ enum skl_tuple_type { /* v4 configuration data */ struct skl_dfw_v4_module_pin { - u16 module_id; - u16 instance_id; + __u16 module_id; + __u16 instance_id; } __packed; struct skl_dfw_v4_module_fmt { - u32 channels; - u32 freq; - u32 bit_depth; - u32 valid_bit_depth; - u32 ch_cfg; - u32 interleaving_style; - u32 sample_type; - u32 ch_map; + __u32 channels; + __u32 freq; + __u32 bit_depth; + __u32 valid_bit_depth; + __u32 ch_cfg; + __u32 interleaving_style; + __u32 sample_type; + __u32 ch_map; } __packed; struct skl_dfw_v4_module_caps { - u32 set_params:2; - u32 rsvd:30; - u32 param_id; - u32 caps_size; - u32 caps[HDA_SST_CFG_MAX]; + __u32 set_params:2; + __u32 rsvd:30; + __u32 param_id; + __u32 caps_size; + __u32 caps[HDA_SST_CFG_MAX]; } __packed; struct skl_dfw_v4_pipe { - u8 pipe_id; - u8 pipe_priority; - u16 conn_type:4; - u16 rsvd:4; - u16 memory_pages:8; + __u8 pipe_id; + __u8 pipe_priority; + __u16 conn_type:4; + __u16 rsvd:4; + __u16 memory_pages:8; } __packed; struct skl_dfw_v4_module { char uuid[SKL_UUID_STR_SZ]; - u16 module_id; - u16 instance_id; - u32 max_mcps; - u32 mem_pages; - u32 obs; - u32 ibs; - u32 vbus_id; - - u32 max_in_queue:8; - u32 max_out_queue:8; - u32 time_slot:8; - u32 core_id:4; - u32 rsvd1:4; - - u32 module_type:8; - u32 conn_type:4; - u32 dev_type:4; - u32 hw_conn_type:4; - u32 rsvd2:12; - - u32 params_fixup:8; - u32 converter:8; - u32 input_pin_type:1; - u32 output_pin_type:1; - u32 is_dynamic_in_pin:1; - u32 is_dynamic_out_pin:1; - u32 is_loadable:1; - u32 rsvd3:11; + __u16 module_id; + __u16 instance_id; + __u32 max_mcps; + __u32 mem_pages; + __u32 obs; + __u32 ibs; + __u32 vbus_id; + + __u32 max_in_queue:8; + __u32 max_out_queue:8; + __u32 time_slot:8; + __u32 core_id:4; + __u32 rsvd1:4; + + __u32 module_type:8; + __u32 conn_type:4; + __u32 dev_type:4; + __u32 hw_conn_type:4; + __u32 rsvd2:12; + + __u32 params_fixup:8; + __u32 converter:8; + __u32 input_pin_type:1; + __u32 output_pin_type:1; + __u32 is_dynamic_in_pin:1; + __u32 is_dynamic_out_pin:1; + __u32 is_loadable:1; + __u32 rsvd3:11; struct skl_dfw_v4_pipe pipe; struct skl_dfw_v4_module_fmt in_fmt[MAX_IN_QUEUE]; diff --git a/include/xen/mem-reservation.h b/include/xen/mem-reservation.h index 80b52b4945e9..a2ab516fcd2c 100644 --- a/include/xen/mem-reservation.h +++ b/include/xen/mem-reservation.h @@ -17,11 +17,12 @@ #include <xen/page.h> +extern bool xen_scrub_pages; + static inline void xenmem_reservation_scrub_page(struct page *page) { -#ifdef CONFIG_XEN_SCRUB_PAGES - clear_highpage(page); -#endif + if (xen_scrub_pages) + clear_highpage(page); } #ifdef CONFIG_XEN_HAVE_PVMMU diff --git a/ipc/shm.c b/ipc/shm.c index b0eb3757ab89..4cd402e4cfeb 100644 --- a/ipc/shm.c +++ b/ipc/shm.c @@ -199,6 +199,7 @@ static inline struct shmid_kernel *shm_lock(struct ipc_namespace *ns, int id) } ipc_unlock_object(ipcp); + ipcp = ERR_PTR(-EIDRM); err: rcu_read_unlock(); /* diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 2590700237c1..138f0302692e 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -1844,7 +1844,7 @@ static int btf_check_all_metas(struct btf_verifier_env *env) hdr = &btf->hdr; cur = btf->nohdr_data + hdr->type_off; - end = btf->nohdr_data + hdr->type_len; + end = cur + hdr->type_len; env->log_type_id = 1; while (cur < end) { diff --git a/kernel/bpf/sockmap.c b/kernel/bpf/sockmap.c index cf5195c7c331..0a0f2ec75370 100644 --- a/kernel/bpf/sockmap.c +++ b/kernel/bpf/sockmap.c @@ -132,6 +132,7 @@ struct smap_psock { struct work_struct gc_work; struct proto *sk_proto; + void (*save_unhash)(struct sock *sk); void (*save_close)(struct sock *sk, long timeout); void (*save_data_ready)(struct sock *sk); void (*save_write_space)(struct sock *sk); @@ -143,6 +144,7 @@ static int bpf_tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, static int bpf_tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); static int bpf_tcp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); +static void bpf_tcp_unhash(struct sock *sk); static void bpf_tcp_close(struct sock *sk, long timeout); static inline struct smap_psock *smap_psock_sk(const struct sock *sk) @@ -184,6 +186,7 @@ static void build_protos(struct proto prot[SOCKMAP_NUM_CONFIGS], struct proto *base) { prot[SOCKMAP_BASE] = *base; + prot[SOCKMAP_BASE].unhash = bpf_tcp_unhash; prot[SOCKMAP_BASE].close = bpf_tcp_close; prot[SOCKMAP_BASE].recvmsg = bpf_tcp_recvmsg; prot[SOCKMAP_BASE].stream_memory_read = bpf_tcp_stream_read; @@ -217,6 +220,7 @@ static int bpf_tcp_init(struct sock *sk) return -EBUSY; } + psock->save_unhash = sk->sk_prot->unhash; psock->save_close = sk->sk_prot->close; psock->sk_proto = sk->sk_prot; @@ -236,7 +240,7 @@ static int bpf_tcp_init(struct sock *sk) } static void smap_release_sock(struct smap_psock *psock, struct sock *sock); -static int free_start_sg(struct sock *sk, struct sk_msg_buff *md); +static int free_start_sg(struct sock *sk, struct sk_msg_buff *md, bool charge); static void bpf_tcp_release(struct sock *sk) { @@ -248,7 +252,7 @@ static void bpf_tcp_release(struct sock *sk) goto out; if (psock->cork) { - free_start_sg(psock->sock, psock->cork); + free_start_sg(psock->sock, psock->cork, true); kfree(psock->cork); psock->cork = NULL; } @@ -305,39 +309,21 @@ static struct smap_psock_map_entry *psock_map_pop(struct sock *sk, return e; } -static void bpf_tcp_close(struct sock *sk, long timeout) +static void bpf_tcp_remove(struct sock *sk, struct smap_psock *psock) { - void (*close_fun)(struct sock *sk, long timeout); struct smap_psock_map_entry *e; struct sk_msg_buff *md, *mtmp; - struct smap_psock *psock; struct sock *osk; - lock_sock(sk); - rcu_read_lock(); - psock = smap_psock_sk(sk); - if (unlikely(!psock)) { - rcu_read_unlock(); - release_sock(sk); - return sk->sk_prot->close(sk, timeout); - } - - /* The psock may be destroyed anytime after exiting the RCU critial - * section so by the time we use close_fun the psock may no longer - * be valid. However, bpf_tcp_close is called with the sock lock - * held so the close hook and sk are still valid. - */ - close_fun = psock->save_close; - if (psock->cork) { - free_start_sg(psock->sock, psock->cork); + free_start_sg(psock->sock, psock->cork, true); kfree(psock->cork); psock->cork = NULL; } list_for_each_entry_safe(md, mtmp, &psock->ingress, list) { list_del(&md->list); - free_start_sg(psock->sock, md); + free_start_sg(psock->sock, md, true); kfree(md); } @@ -369,7 +355,7 @@ static void bpf_tcp_close(struct sock *sk, long timeout) /* If another thread deleted this object skip deletion. * The refcnt on psock may or may not be zero. */ - if (l) { + if (l && l == link) { hlist_del_rcu(&link->hash_node); smap_release_sock(psock, link->sk); free_htab_elem(htab, link); @@ -379,6 +365,42 @@ static void bpf_tcp_close(struct sock *sk, long timeout) kfree(e); e = psock_map_pop(sk, psock); } +} + +static void bpf_tcp_unhash(struct sock *sk) +{ + void (*unhash_fun)(struct sock *sk); + struct smap_psock *psock; + + rcu_read_lock(); + psock = smap_psock_sk(sk); + if (unlikely(!psock)) { + rcu_read_unlock(); + if (sk->sk_prot->unhash) + sk->sk_prot->unhash(sk); + return; + } + unhash_fun = psock->save_unhash; + bpf_tcp_remove(sk, psock); + rcu_read_unlock(); + unhash_fun(sk); +} + +static void bpf_tcp_close(struct sock *sk, long timeout) +{ + void (*close_fun)(struct sock *sk, long timeout); + struct smap_psock *psock; + + lock_sock(sk); + rcu_read_lock(); + psock = smap_psock_sk(sk); + if (unlikely(!psock)) { + rcu_read_unlock(); + release_sock(sk); + return sk->sk_prot->close(sk, timeout); + } + close_fun = psock->save_close; + bpf_tcp_remove(sk, psock); rcu_read_unlock(); release_sock(sk); close_fun(sk, timeout); @@ -570,14 +592,16 @@ static void free_bytes_sg(struct sock *sk, int bytes, md->sg_start = i; } -static int free_sg(struct sock *sk, int start, struct sk_msg_buff *md) +static int free_sg(struct sock *sk, int start, + struct sk_msg_buff *md, bool charge) { struct scatterlist *sg = md->sg_data; int i = start, free = 0; while (sg[i].length) { free += sg[i].length; - sk_mem_uncharge(sk, sg[i].length); + if (charge) + sk_mem_uncharge(sk, sg[i].length); if (!md->skb) put_page(sg_page(&sg[i])); sg[i].length = 0; @@ -594,9 +618,9 @@ static int free_sg(struct sock *sk, int start, struct sk_msg_buff *md) return free; } -static int free_start_sg(struct sock *sk, struct sk_msg_buff *md) +static int free_start_sg(struct sock *sk, struct sk_msg_buff *md, bool charge) { - int free = free_sg(sk, md->sg_start, md); + int free = free_sg(sk, md->sg_start, md, charge); md->sg_start = md->sg_end; return free; @@ -604,7 +628,7 @@ static int free_start_sg(struct sock *sk, struct sk_msg_buff *md) static int free_curr_sg(struct sock *sk, struct sk_msg_buff *md) { - return free_sg(sk, md->sg_curr, md); + return free_sg(sk, md->sg_curr, md, true); } static int bpf_map_msg_verdict(int _rc, struct sk_msg_buff *md) @@ -718,7 +742,7 @@ static int bpf_tcp_ingress(struct sock *sk, int apply_bytes, list_add_tail(&r->list, &psock->ingress); sk->sk_data_ready(sk); } else { - free_start_sg(sk, r); + free_start_sg(sk, r, true); kfree(r); } @@ -752,14 +776,10 @@ static int bpf_tcp_sendmsg_do_redirect(struct sock *sk, int send, release_sock(sk); } smap_release_sock(psock, sk); - if (unlikely(err)) - goto out; - return 0; + return err; out_rcu: rcu_read_unlock(); -out: - free_bytes_sg(NULL, send, md, false); - return err; + return 0; } static inline void bpf_md_init(struct smap_psock *psock) @@ -822,7 +842,7 @@ more_data: case __SK_PASS: err = bpf_tcp_push(sk, send, m, flags, true); if (unlikely(err)) { - *copied -= free_start_sg(sk, m); + *copied -= free_start_sg(sk, m, true); break; } @@ -845,16 +865,17 @@ more_data: lock_sock(sk); if (unlikely(err < 0)) { - free_start_sg(sk, m); + int free = free_start_sg(sk, m, false); + psock->sg_size = 0; if (!cork) - *copied -= send; + *copied -= free; } else { psock->sg_size -= send; } if (cork) { - free_start_sg(sk, m); + free_start_sg(sk, m, true); psock->sg_size = 0; kfree(m); m = NULL; @@ -912,6 +933,8 @@ static int bpf_tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, if (unlikely(flags & MSG_ERRQUEUE)) return inet_recv_error(sk, msg, len, addr_len); + if (!skb_queue_empty(&sk->sk_receive_queue)) + return tcp_recvmsg(sk, msg, len, nonblock, flags, addr_len); rcu_read_lock(); psock = smap_psock_sk(sk); @@ -922,9 +945,6 @@ static int bpf_tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, goto out; rcu_read_unlock(); - if (!skb_queue_empty(&sk->sk_receive_queue)) - return tcp_recvmsg(sk, msg, len, nonblock, flags, addr_len); - lock_sock(sk); bytes_ready: while (copied != len) { @@ -1122,7 +1142,7 @@ wait_for_memory: err = sk_stream_wait_memory(sk, &timeo); if (err) { if (m && m != psock->cork) - free_start_sg(sk, m); + free_start_sg(sk, m, true); goto out_err; } } @@ -1464,10 +1484,16 @@ static void smap_destroy_psock(struct rcu_head *rcu) schedule_work(&psock->gc_work); } +static bool psock_is_smap_sk(struct sock *sk) +{ + return inet_csk(sk)->icsk_ulp_ops == &bpf_tcp_ulp_ops; +} + static void smap_release_sock(struct smap_psock *psock, struct sock *sock) { if (refcount_dec_and_test(&psock->refcnt)) { - tcp_cleanup_ulp(sock); + if (psock_is_smap_sk(sock)) + tcp_cleanup_ulp(sock); write_lock_bh(&sock->sk_callback_lock); smap_stop_sock(psock, sock); write_unlock_bh(&sock->sk_callback_lock); @@ -1581,13 +1607,13 @@ static void smap_gc_work(struct work_struct *w) bpf_prog_put(psock->bpf_tx_msg); if (psock->cork) { - free_start_sg(psock->sock, psock->cork); + free_start_sg(psock->sock, psock->cork, true); kfree(psock->cork); } list_for_each_entry_safe(md, mtmp, &psock->ingress, list) { list_del(&md->list); - free_start_sg(psock->sock, md); + free_start_sg(psock->sock, md, true); kfree(md); } @@ -1894,6 +1920,10 @@ static int __sock_map_ctx_update_elem(struct bpf_map *map, * doesn't update user data. */ if (psock) { + if (!psock_is_smap_sk(sock)) { + err = -EBUSY; + goto out_progs; + } if (READ_ONCE(psock->bpf_parse) && parse) { err = -EBUSY; goto out_progs; @@ -2089,8 +2119,12 @@ static int sock_map_update_elem(struct bpf_map *map, return -EINVAL; } + /* ULPs are currently supported only for TCP sockets in ESTABLISHED + * state. + */ if (skops.sk->sk_type != SOCK_STREAM || - skops.sk->sk_protocol != IPPROTO_TCP) { + skops.sk->sk_protocol != IPPROTO_TCP || + skops.sk->sk_state != TCP_ESTABLISHED) { fput(socket->file); return -EOPNOTSUPP; } @@ -2445,6 +2479,16 @@ static int sock_hash_update_elem(struct bpf_map *map, return -EINVAL; } + /* ULPs are currently supported only for TCP sockets in ESTABLISHED + * state. + */ + if (skops.sk->sk_type != SOCK_STREAM || + skops.sk->sk_protocol != IPPROTO_TCP || + skops.sk->sk_state != TCP_ESTABLISHED) { + fput(socket->file); + return -EOPNOTSUPP; + } + lock_sock(skops.sk); preempt_disable(); rcu_read_lock(); @@ -2535,10 +2579,22 @@ const struct bpf_map_ops sock_hash_ops = { .map_check_btf = map_check_no_btf, }; +static bool bpf_is_valid_sock_op(struct bpf_sock_ops_kern *ops) +{ + return ops->op == BPF_SOCK_OPS_PASSIVE_ESTABLISHED_CB || + ops->op == BPF_SOCK_OPS_ACTIVE_ESTABLISHED_CB; +} BPF_CALL_4(bpf_sock_map_update, struct bpf_sock_ops_kern *, bpf_sock, struct bpf_map *, map, void *, key, u64, flags) { WARN_ON_ONCE(!rcu_read_lock_held()); + + /* ULPs are currently supported only for TCP sockets in ESTABLISHED + * state. This checks that the sock ops triggering the update is + * one indicating we are (or will be soon) in an ESTABLISHED state. + */ + if (!bpf_is_valid_sock_op(bpf_sock)) + return -EOPNOTSUPP; return sock_map_ctx_update_elem(bpf_sock, map, key, flags); } @@ -2557,6 +2613,9 @@ BPF_CALL_4(bpf_sock_hash_update, struct bpf_sock_ops_kern *, bpf_sock, struct bpf_map *, map, void *, key, u64, flags) { WARN_ON_ONCE(!rcu_read_lock_held()); + + if (!bpf_is_valid_sock_op(bpf_sock)) + return -EOPNOTSUPP; return sock_hash_ctx_update_elem(bpf_sock, map, key, flags); } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 92246117d2b0..bb07e74b34a2 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -3163,7 +3163,7 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, * an arbitrary scalar. Disallow all math except * pointer subtraction */ - if (opcode == BPF_SUB){ + if (opcode == BPF_SUB && env->allow_ptr_leaks) { mark_reg_unknown(env, regs, insn->dst_reg); return 0; } diff --git a/kernel/cpu.c b/kernel/cpu.c index aa7fe85ad62e..0097acec1c71 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -607,15 +607,15 @@ static void cpuhp_thread_fun(unsigned int cpu) bool bringup = st->bringup; enum cpuhp_state state; + if (WARN_ON_ONCE(!st->should_run)) + return; + /* * ACQUIRE for the cpuhp_should_run() load of ->should_run. Ensures * that if we see ->should_run we also see the rest of the state. */ smp_mb(); - if (WARN_ON_ONCE(!st->should_run)) - return; - cpuhp_lock_acquire(bringup); if (st->single) { @@ -916,7 +916,8 @@ static int cpuhp_down_callbacks(unsigned int cpu, struct cpuhp_cpu_state *st, ret = cpuhp_invoke_callback(cpu, st->state, false, NULL, NULL); if (ret) { st->target = prev_state; - undo_cpu_down(cpu, st); + if (st->state < prev_state) + undo_cpu_down(cpu, st); break; } } @@ -969,7 +970,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen, * to do the further cleanups. */ ret = cpuhp_down_callbacks(cpu, st, target); - if (ret && st->state > CPUHP_TEARDOWN_CPU && st->state < prev_state) { + if (ret && st->state == CPUHP_TEARDOWN_CPU && st->state < prev_state) { cpuhp_reset_state(st, prev_state); __cpuhp_kick_ap(st); } diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index 9bd54304446f..1b1d63b3634b 100644 --- a/kernel/dma/Kconfig +++ b/kernel/dma/Kconfig @@ -23,6 +23,9 @@ config ARCH_HAS_SYNC_DMA_FOR_CPU bool select NEED_DMA_MAP_STATE +config ARCH_HAS_SYNC_DMA_FOR_CPU_ALL + bool + config DMA_DIRECT_OPS bool depends on HAS_DMA diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c index 1c35b7b945d0..de87b0282e74 100644 --- a/kernel/dma/direct.c +++ b/kernel/dma/direct.c @@ -168,7 +168,7 @@ int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl, int nents, int dma_direct_supported(struct device *dev, u64 mask) { #ifdef CONFIG_ZONE_DMA - if (mask < DMA_BIT_MASK(ARCH_ZONE_DMA_BITS)) + if (mask < phys_to_dma(dev, DMA_BIT_MASK(ARCH_ZONE_DMA_BITS))) return 0; #else /* @@ -177,7 +177,7 @@ int dma_direct_supported(struct device *dev, u64 mask) * memory, or by providing a ZONE_DMA32. If neither is the case, the * architecture needs to use an IOMMU instead of the direct mapping. */ - if (mask < DMA_BIT_MASK(32)) + if (mask < phys_to_dma(dev, DMA_BIT_MASK(32))) return 0; #endif /* diff --git a/kernel/events/core.c b/kernel/events/core.c index 2a62b96600ad..dcb093e7b377 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -2867,16 +2867,11 @@ static int perf_event_modify_breakpoint(struct perf_event *bp, _perf_event_disable(bp); err = modify_user_hw_breakpoint_check(bp, attr, true); - if (err) { - if (!bp->attr.disabled) - _perf_event_enable(bp); - return err; - } - - if (!attr->disabled) + if (!bp->attr.disabled) _perf_event_enable(bp); - return 0; + + return err; } static int perf_event_modify_attr(struct perf_event *event, @@ -3940,6 +3935,12 @@ int perf_event_read_local(struct perf_event *event, u64 *value, goto out; } + /* If this is a pinned event it must be running on this CPU */ + if (event->attr.pinned && event->oncpu != smp_processor_id()) { + ret = -EBUSY; + goto out; + } + /* * If the event is currently on this CPU, its either a per-task event, * or local to this CPU. Furthermore it means its ACTIVE (otherwise @@ -5948,6 +5949,7 @@ perf_output_sample_ustack(struct perf_output_handle *handle, u64 dump_size, unsigned long sp; unsigned int rem; u64 dyn_size; + mm_segment_t fs; /* * We dump: @@ -5965,7 +5967,10 @@ perf_output_sample_ustack(struct perf_output_handle *handle, u64 dump_size, /* Data. */ sp = perf_user_stack_pointer(regs); + fs = get_fs(); + set_fs(USER_DS); rem = __output_copy_user(handle, (void *) sp, dump_size); + set_fs(fs); dyn_size = dump_size - rem; perf_output_skip(handle, rem); diff --git a/kernel/events/hw_breakpoint.c b/kernel/events/hw_breakpoint.c index b3814fce5ecb..d6b56180827c 100644 --- a/kernel/events/hw_breakpoint.c +++ b/kernel/events/hw_breakpoint.c @@ -509,6 +509,8 @@ modify_user_hw_breakpoint_check(struct perf_event *bp, struct perf_event_attr *a */ int modify_user_hw_breakpoint(struct perf_event *bp, struct perf_event_attr *attr) { + int err; + /* * modify_user_hw_breakpoint can be invoked with IRQs disabled and hence it * will not be possible to raise IPIs that invoke __perf_event_disable. @@ -520,15 +522,12 @@ int modify_user_hw_breakpoint(struct perf_event *bp, struct perf_event_attr *att else perf_event_disable(bp); - if (!attr->disabled) { - int err = modify_user_hw_breakpoint_check(bp, attr, false); + err = modify_user_hw_breakpoint_check(bp, attr, false); - if (err) - return err; + if (!bp->attr.disabled) perf_event_enable(bp); - bp->attr.disabled = 0; - } - return 0; + + return err; } EXPORT_SYMBOL_GPL(modify_user_hw_breakpoint); diff --git a/kernel/fork.c b/kernel/fork.c index d896e9ca38b0..f0b58479534f 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -550,8 +550,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm, goto out; } /* a new mm has just been created */ - arch_dup_mmap(oldmm, mm); - retval = 0; + retval = arch_dup_mmap(oldmm, mm); out: up_write(&mm->mmap_sem); flush_tlb_mm(oldmm); diff --git a/kernel/jump_label.c b/kernel/jump_label.c index 01ebdf1f9f40..2e62503bea0d 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -678,7 +678,7 @@ jump_label_module_notify(struct notifier_block *self, unsigned long val, case MODULE_STATE_COMING: ret = jump_label_add_module(mod); if (ret) { - WARN(1, "Failed to allocatote memory: jump_label may not work properly.\n"); + WARN(1, "Failed to allocate memory: jump_label may not work properly.\n"); jump_label_del_module(mod); } break; diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index e406c5fdb41e..dd13f865ad40 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -55,7 +55,6 @@ #include "lockdep_internals.h" -#include <trace/events/preemptirq.h> #define CREATE_TRACE_POINTS #include <trace/events/lock.h> diff --git a/kernel/locking/mutex.c b/kernel/locking/mutex.c index 1a81a1257b3f..3f8a35104285 100644 --- a/kernel/locking/mutex.c +++ b/kernel/locking/mutex.c @@ -389,7 +389,7 @@ static bool __ww_mutex_wound(struct mutex *lock, /* * wake_up_process() paired with set_current_state() * inserts sufficient barriers to make sure @owner either sees - * it's wounded in __ww_mutex_lock_check_stamp() or has a + * it's wounded in __ww_mutex_check_kill() or has a * wakeup pending to re-read the wounded state. */ if (owner != current) @@ -946,7 +946,6 @@ __mutex_lock_common(struct mutex *lock, long state, unsigned int subclass, } debug_mutex_lock_common(lock, &waiter); - debug_mutex_add_waiter(lock, &waiter, current); lock_contended(&lock->dep_map, ip); diff --git a/kernel/locking/test-ww_mutex.c b/kernel/locking/test-ww_mutex.c index 5b915b370d5a..0be047dbd897 100644 --- a/kernel/locking/test-ww_mutex.c +++ b/kernel/locking/test-ww_mutex.c @@ -324,7 +324,7 @@ static int __test_cycle(unsigned int nthreads) if (!cycle->result) continue; - pr_err("cylic deadlock not resolved, ret[%d/%d] = %d\n", + pr_err("cyclic deadlock not resolved, ret[%d/%d] = %d\n", n, nthreads, cycle->result); ret = -EINVAL; break; diff --git a/kernel/pid.c b/kernel/pid.c index de1cfc4f75a2..cdf63e53a014 100644 --- a/kernel/pid.c +++ b/kernel/pid.c @@ -195,7 +195,7 @@ struct pid *alloc_pid(struct pid_namespace *ns) idr_preload_end(); if (nr < 0) { - retval = nr; + retval = (nr == -ENOSPC) ? -EAGAIN : nr; goto out_free; } diff --git a/kernel/printk/printk.c b/kernel/printk/printk.c index fd6f8ed28e01..9bf5404397e0 100644 --- a/kernel/printk/printk.c +++ b/kernel/printk/printk.c @@ -351,7 +351,6 @@ static int console_msg_format = MSG_FORMAT_DEFAULT; */ enum log_flags { - LOG_NOCONS = 1, /* suppress print, do not print to console */ LOG_NEWLINE = 2, /* text ended with a newline */ LOG_PREFIX = 4, /* text started with a prefix */ LOG_CONT = 8, /* text is a fragment of a continuation line */ @@ -1881,9 +1880,6 @@ int vprintk_store(int facility, int level, if (dict) lflags |= LOG_PREFIX|LOG_NEWLINE; - if (suppress_message_printing(level)) - lflags |= LOG_NOCONS; - return log_output(facility, level, lflags, dict, dictlen, text, text_len); } @@ -2032,6 +2028,7 @@ static void call_console_drivers(const char *ext_text, size_t ext_len, const char *text, size_t len) {} static size_t msg_print_text(const struct printk_log *msg, bool syslog, char *buf, size_t size) { return 0; } +static bool suppress_message_printing(int level) { return false; } #endif /* CONFIG_PRINTK */ @@ -2368,10 +2365,11 @@ skip: break; msg = log_from_idx(console_idx); - if (msg->flags & LOG_NOCONS) { + if (suppress_message_printing(msg->level)) { /* - * Skip record if !ignore_loglevel, and - * record has level above the console loglevel. + * Skip record we have buffered and already printed + * directly to the console when we received it, and + * record that has level above the console loglevel. */ console_idx = log_next(console_idx); console_seq++; diff --git a/kernel/printk/printk_safe.c b/kernel/printk/printk_safe.c index a0a74c533e4b..0913b4d385de 100644 --- a/kernel/printk/printk_safe.c +++ b/kernel/printk/printk_safe.c @@ -306,12 +306,12 @@ static __printf(1, 0) int vprintk_nmi(const char *fmt, va_list args) return printk_safe_log_store(s, fmt, args); } -void printk_nmi_enter(void) +void notrace printk_nmi_enter(void) { this_cpu_or(printk_context, PRINTK_NMI_CONTEXT_MASK); } -void printk_nmi_exit(void) +void notrace printk_nmi_exit(void) { this_cpu_and(printk_context, ~PRINTK_NMI_CONTEXT_MASK); } diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 60caf1fb94e0..6383aa6a60ca 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -89,12 +89,12 @@ struct static_key sched_feat_keys[__SCHED_FEAT_NR] = { static void sched_feat_disable(int i) { - static_key_disable(&sched_feat_keys[i]); + static_key_disable_cpuslocked(&sched_feat_keys[i]); } static void sched_feat_enable(int i) { - static_key_enable(&sched_feat_keys[i]); + static_key_enable_cpuslocked(&sched_feat_keys[i]); } #else static void sched_feat_disable(int i) { }; @@ -146,9 +146,11 @@ sched_feat_write(struct file *filp, const char __user *ubuf, /* Ensure the static_key remains in a consistent state */ inode = file_inode(filp); + cpus_read_lock(); inode_lock(inode); ret = sched_feat_set(cmp); inode_unlock(inode); + cpus_read_unlock(); if (ret < 0) return ret; diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b39fb596f6c1..f808ddf2a868 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -3362,6 +3362,7 @@ update_cfs_rq_load_avg(u64 now, struct cfs_rq *cfs_rq) * attach_entity_load_avg - attach this entity to its cfs_rq load avg * @cfs_rq: cfs_rq to attach to * @se: sched_entity to attach + * @flags: migration hints * * Must call update_cfs_rq_load_avg() before this, since we rely on * cfs_rq->avg.last_update_time being current. @@ -7263,6 +7264,7 @@ static void update_blocked_averages(int cpu) { struct rq *rq = cpu_rq(cpu); struct cfs_rq *cfs_rq, *pos; + const struct sched_class *curr_class; struct rq_flags rf; bool done = true; @@ -7299,8 +7301,10 @@ static void update_blocked_averages(int cpu) if (cfs_rq_has_blocked(cfs_rq)) done = false; } - update_rt_rq_load_avg(rq_clock_task(rq), rq, 0); - update_dl_rq_load_avg(rq_clock_task(rq), rq, 0); + + curr_class = rq->curr->sched_class; + update_rt_rq_load_avg(rq_clock_task(rq), rq, curr_class == &rt_sched_class); + update_dl_rq_load_avg(rq_clock_task(rq), rq, curr_class == &dl_sched_class); update_irq_load_avg(rq, 0); /* Don't need periodic decay once load/util_avg are null */ if (others_have_blocked(rq)) @@ -7365,13 +7369,16 @@ static inline void update_blocked_averages(int cpu) { struct rq *rq = cpu_rq(cpu); struct cfs_rq *cfs_rq = &rq->cfs; + const struct sched_class *curr_class; struct rq_flags rf; rq_lock_irqsave(rq, &rf); update_rq_clock(rq); update_cfs_rq_load_avg(cfs_rq_clock_task(cfs_rq), cfs_rq); - update_rt_rq_load_avg(rq_clock_task(rq), rq, 0); - update_dl_rq_load_avg(rq_clock_task(rq), rq, 0); + + curr_class = rq->curr->sched_class; + update_rt_rq_load_avg(rq_clock_task(rq), rq, curr_class == &rt_sched_class); + update_dl_rq_load_avg(rq_clock_task(rq), rq, curr_class == &dl_sched_class); update_irq_load_avg(rq, 0); #ifdef CONFIG_NO_HZ_COMMON rq->last_blocked_load_update_tick = jiffies; @@ -7482,10 +7489,10 @@ static inline int get_sd_load_idx(struct sched_domain *sd, return load_idx; } -static unsigned long scale_rt_capacity(int cpu) +static unsigned long scale_rt_capacity(struct sched_domain *sd, int cpu) { struct rq *rq = cpu_rq(cpu); - unsigned long max = arch_scale_cpu_capacity(NULL, cpu); + unsigned long max = arch_scale_cpu_capacity(sd, cpu); unsigned long used, free; unsigned long irq; @@ -7507,7 +7514,7 @@ static unsigned long scale_rt_capacity(int cpu) static void update_cpu_capacity(struct sched_domain *sd, int cpu) { - unsigned long capacity = scale_rt_capacity(cpu); + unsigned long capacity = scale_rt_capacity(sd, cpu); struct sched_group *sdg = sd->groups; cpu_rq(cpu)->cpu_capacity_orig = arch_scale_cpu_capacity(sd, cpu); @@ -8269,7 +8276,7 @@ static struct sched_group *find_busiest_group(struct lb_env *env) force_balance: /* Looks like there is an imbalance. Compute it */ calculate_imbalance(env, &sds); - return sds.busiest; + return env->imbalance ? sds.busiest : NULL; out_balanced: env->imbalance = 0; @@ -9638,7 +9645,8 @@ static inline bool vruntime_normalized(struct task_struct *p) * - A task which has been woken up by try_to_wake_up() and * waiting for actually being woken up by sched_ttwu_pending(). */ - if (!se->sum_exec_runtime || p->state == TASK_WAKING) + if (!se->sum_exec_runtime || + (p->state == TASK_WAKING && p->sched_remote_wakeup)) return true; return false; diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c index 56a0fed30c0a..505a41c42b96 100644 --- a/kernel/sched/topology.c +++ b/kernel/sched/topology.c @@ -1295,7 +1295,7 @@ static void init_numa_topology_type(void) n = sched_max_numa_distance; - if (sched_domains_numa_levels <= 1) { + if (sched_domains_numa_levels <= 2) { sched_numa_topology_type = NUMA_DIRECT; return; } @@ -1380,9 +1380,6 @@ void sched_init_numa(void) break; } - if (!level) - return; - /* * 'level' contains the number of unique distances * diff --git a/kernel/sys.c b/kernel/sys.c index cf5c67533ff1..123bd73046ec 100644 --- a/kernel/sys.c +++ b/kernel/sys.c @@ -71,9 +71,6 @@ #include <asm/io.h> #include <asm/unistd.h> -/* Hardening for Spectre-v1 */ -#include <linux/nospec.h> - #include "uid16.h" #ifndef SET_UNALIGN_CTL diff --git a/kernel/time/clocksource.c b/kernel/time/clocksource.c index f74fb00d8064..0e6e97a01942 100644 --- a/kernel/time/clocksource.c +++ b/kernel/time/clocksource.c @@ -133,19 +133,40 @@ static void inline clocksource_watchdog_unlock(unsigned long *flags) spin_unlock_irqrestore(&watchdog_lock, *flags); } +static int clocksource_watchdog_kthread(void *data); +static void __clocksource_change_rating(struct clocksource *cs, int rating); + /* * Interval: 0.5sec Threshold: 0.0625s */ #define WATCHDOG_INTERVAL (HZ >> 1) #define WATCHDOG_THRESHOLD (NSEC_PER_SEC >> 4) +static void clocksource_watchdog_work(struct work_struct *work) +{ + /* + * We cannot directly run clocksource_watchdog_kthread() here, because + * clocksource_select() calls timekeeping_notify() which uses + * stop_machine(). One cannot use stop_machine() from a workqueue() due + * lock inversions wrt CPU hotplug. + * + * Also, we only ever run this work once or twice during the lifetime + * of the kernel, so there is no point in creating a more permanent + * kthread for this. + * + * If kthread_run fails the next watchdog scan over the + * watchdog_list will find the unstable clock again. + */ + kthread_run(clocksource_watchdog_kthread, NULL, "kwatchdog"); +} + static void __clocksource_unstable(struct clocksource *cs) { cs->flags &= ~(CLOCK_SOURCE_VALID_FOR_HRES | CLOCK_SOURCE_WATCHDOG); cs->flags |= CLOCK_SOURCE_UNSTABLE; /* - * If the clocksource is registered clocksource_watchdog_work() will + * If the clocksource is registered clocksource_watchdog_kthread() will * re-rate and re-select. */ if (list_empty(&cs->list)) { @@ -156,7 +177,7 @@ static void __clocksource_unstable(struct clocksource *cs) if (cs->mark_unstable) cs->mark_unstable(cs); - /* kick clocksource_watchdog_work() */ + /* kick clocksource_watchdog_kthread() */ if (finished_booting) schedule_work(&watchdog_work); } @@ -166,7 +187,7 @@ static void __clocksource_unstable(struct clocksource *cs) * @cs: clocksource to be marked unstable * * This function is called by the x86 TSC code to mark clocksources as unstable; - * it defers demotion and re-selection to a work. + * it defers demotion and re-selection to a kthread. */ void clocksource_mark_unstable(struct clocksource *cs) { @@ -391,9 +412,7 @@ static void clocksource_dequeue_watchdog(struct clocksource *cs) } } -static void __clocksource_change_rating(struct clocksource *cs, int rating); - -static int __clocksource_watchdog_work(void) +static int __clocksource_watchdog_kthread(void) { struct clocksource *cs, *tmp; unsigned long flags; @@ -418,12 +437,13 @@ static int __clocksource_watchdog_work(void) return select; } -static void clocksource_watchdog_work(struct work_struct *work) +static int clocksource_watchdog_kthread(void *data) { mutex_lock(&clocksource_mutex); - if (__clocksource_watchdog_work()) + if (__clocksource_watchdog_kthread()) clocksource_select(); mutex_unlock(&clocksource_mutex); + return 0; } static bool clocksource_is_watchdog(struct clocksource *cs) @@ -442,7 +462,7 @@ static void clocksource_enqueue_watchdog(struct clocksource *cs) static void clocksource_select_watchdog(bool fallback) { } static inline void clocksource_dequeue_watchdog(struct clocksource *cs) { } static inline void clocksource_resume_watchdog(void) { } -static inline int __clocksource_watchdog_work(void) { return 0; } +static inline int __clocksource_watchdog_kthread(void) { return 0; } static bool clocksource_is_watchdog(struct clocksource *cs) { return false; } void clocksource_mark_unstable(struct clocksource *cs) { } @@ -810,7 +830,7 @@ static int __init clocksource_done_booting(void) /* * Run the watchdog first to eliminate unstable clock sources */ - __clocksource_watchdog_work(); + __clocksource_watchdog_kthread(); clocksource_select(); mutex_unlock(&clocksource_mutex); return 0; diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 1d92d4a982fd..65bd4616220d 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -1546,6 +1546,8 @@ rb_remove_pages(struct ring_buffer_per_cpu *cpu_buffer, unsigned long nr_pages) tmp_iter_page = first_page; do { + cond_resched(); + to_remove_page = tmp_iter_page; rb_inc_page(cpu_buffer, &tmp_iter_page); diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 613316724c6a..4966c4fbe7f7 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1277,13 +1277,13 @@ config WARN_ALL_UNSEEDED_RANDOM time. This is really bad from a security perspective, and so architecture maintainers really need to do what they can to get the CRNG seeded sooner after the system is booted. - However, since users can not do anything actionble to + However, since users cannot do anything actionable to address this, by default the kernel will issue only a single warning for the first use of unseeded randomness. Say Y here if you want to receive warnings for all uses of unseeded randomness. This will be of use primarily for - those developers interersted in improving the security of + those developers interested in improving the security of Linux kernels running on their architecture (or subarchitecture). diff --git a/mm/Kconfig b/mm/Kconfig index a550635ea5c3..de64ea658716 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -637,6 +637,7 @@ config DEFERRED_STRUCT_PAGE_INIT depends on NO_BOOTMEM depends on SPARSEMEM depends on !NEED_PER_CPU_KM + depends on 64BIT help Ordinarily all struct pages are initialised during early boot in a single thread. On very large machines this can take a considerable diff --git a/mm/Makefile b/mm/Makefile index 8716bdabe1e6..26ef77a3883b 100644 --- a/mm/Makefile +++ b/mm/Makefile @@ -32,7 +32,7 @@ ifdef CONFIG_CROSS_MEMORY_ATTACH mmu-$(CONFIG_MMU) += process_vm_access.o endif -obj-y := filemap.o mempool.o oom_kill.o \ +obj-y := filemap.o mempool.o oom_kill.o fadvise.o \ maccess.o page_alloc.o page-writeback.o \ readahead.o swap.o truncate.o vmscan.o shmem.o \ util.o mmzone.o vmstat.o backing-dev.o \ @@ -49,7 +49,6 @@ else obj-y += bootmem.o endif -obj-$(CONFIG_ADVISE_SYSCALLS) += fadvise.o ifdef CONFIG_MMU obj-$(CONFIG_ADVISE_SYSCALLS) += madvise.o endif diff --git a/mm/backing-dev.c b/mm/backing-dev.c index f5981e9d6ae2..8a8bb8796c6c 100644 --- a/mm/backing-dev.c +++ b/mm/backing-dev.c @@ -491,6 +491,7 @@ static void cgwb_release_workfn(struct work_struct *work) { struct bdi_writeback *wb = container_of(work, struct bdi_writeback, release_work); + struct blkcg *blkcg = css_to_blkcg(wb->blkcg_css); mutex_lock(&wb->bdi->cgwb_release_mutex); wb_shutdown(wb); @@ -499,6 +500,9 @@ static void cgwb_release_workfn(struct work_struct *work) css_put(wb->blkcg_css); mutex_unlock(&wb->bdi->cgwb_release_mutex); + /* triggers blkg destruction if cgwb_refcnt becomes zero */ + blkcg_cgwb_put(blkcg); + fprop_local_destroy_percpu(&wb->memcg_completions); percpu_ref_exit(&wb->refcnt); wb_exit(wb); @@ -597,6 +601,7 @@ static int cgwb_create(struct backing_dev_info *bdi, list_add_tail_rcu(&wb->bdi_node, &bdi->wb_list); list_add(&wb->memcg_node, memcg_cgwb_list); list_add(&wb->blkcg_node, blkcg_cgwb_list); + blkcg_cgwb_get(blkcg); css_get(memcg_css); css_get(blkcg_css); } diff --git a/mm/debug.c b/mm/debug.c index 38c926520c97..bd10aad8539a 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -114,7 +114,7 @@ EXPORT_SYMBOL(dump_vma); void dump_mm(const struct mm_struct *mm) { - pr_emerg("mm %px mmap %px seqnum %d task_size %lu\n" + pr_emerg("mm %px mmap %px seqnum %llu task_size %lu\n" #ifdef CONFIG_MMU "get_unmapped_area %px\n" #endif @@ -142,7 +142,7 @@ void dump_mm(const struct mm_struct *mm) "tlb_flush_pending %d\n" "def_flags: %#lx(%pGv)\n", - mm, mm->mmap, mm->vmacache_seqnum, mm->task_size, + mm, mm->mmap, (long long) mm->vmacache_seqnum, mm->task_size, #ifdef CONFIG_MMU mm->get_unmapped_area, #endif diff --git a/mm/fadvise.c b/mm/fadvise.c index 2d8376e3c640..467bcd032037 100644 --- a/mm/fadvise.c +++ b/mm/fadvise.c @@ -27,9 +27,9 @@ * deactivate the pages and clear PG_Referenced. */ -int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice) +static int generic_fadvise(struct file *file, loff_t offset, loff_t len, + int advice) { - struct fd f = fdget(fd); struct inode *inode; struct address_space *mapping; struct backing_dev_info *bdi; @@ -37,22 +37,14 @@ int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice) pgoff_t start_index; pgoff_t end_index; unsigned long nrpages; - int ret = 0; - - if (!f.file) - return -EBADF; - inode = file_inode(f.file); - if (S_ISFIFO(inode->i_mode)) { - ret = -ESPIPE; - goto out; - } + inode = file_inode(file); + if (S_ISFIFO(inode->i_mode)) + return -ESPIPE; - mapping = f.file->f_mapping; - if (!mapping || len < 0) { - ret = -EINVAL; - goto out; - } + mapping = file->f_mapping; + if (!mapping || len < 0) + return -EINVAL; bdi = inode_to_bdi(mapping->host); @@ -67,9 +59,9 @@ int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice) /* no bad return value, but ignore advice */ break; default: - ret = -EINVAL; + return -EINVAL; } - goto out; + return 0; } /* @@ -85,21 +77,21 @@ int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice) switch (advice) { case POSIX_FADV_NORMAL: - f.file->f_ra.ra_pages = bdi->ra_pages; - spin_lock(&f.file->f_lock); - f.file->f_mode &= ~FMODE_RANDOM; - spin_unlock(&f.file->f_lock); + file->f_ra.ra_pages = bdi->ra_pages; + spin_lock(&file->f_lock); + file->f_mode &= ~FMODE_RANDOM; + spin_unlock(&file->f_lock); break; case POSIX_FADV_RANDOM: - spin_lock(&f.file->f_lock); - f.file->f_mode |= FMODE_RANDOM; - spin_unlock(&f.file->f_lock); + spin_lock(&file->f_lock); + file->f_mode |= FMODE_RANDOM; + spin_unlock(&file->f_lock); break; case POSIX_FADV_SEQUENTIAL: - f.file->f_ra.ra_pages = bdi->ra_pages * 2; - spin_lock(&f.file->f_lock); - f.file->f_mode &= ~FMODE_RANDOM; - spin_unlock(&f.file->f_lock); + file->f_ra.ra_pages = bdi->ra_pages * 2; + spin_lock(&file->f_lock); + file->f_mode &= ~FMODE_RANDOM; + spin_unlock(&file->f_lock); break; case POSIX_FADV_WILLNEED: /* First and last PARTIAL page! */ @@ -115,8 +107,7 @@ int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice) * Ignore return value because fadvise() shall return * success even if filesystem can't retrieve a hint, */ - force_page_cache_readahead(mapping, f.file, start_index, - nrpages); + force_page_cache_readahead(mapping, file, start_index, nrpages); break; case POSIX_FADV_NOREUSE: break; @@ -183,9 +174,32 @@ int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice) } break; default: - ret = -EINVAL; + return -EINVAL; } -out: + return 0; +} + +int vfs_fadvise(struct file *file, loff_t offset, loff_t len, int advice) +{ + if (file->f_op->fadvise) + return file->f_op->fadvise(file, offset, len, advice); + + return generic_fadvise(file, offset, len, advice); +} +EXPORT_SYMBOL(vfs_fadvise); + +#ifdef CONFIG_ADVISE_SYSCALLS + +int ksys_fadvise64_64(int fd, loff_t offset, loff_t len, int advice) +{ + struct fd f = fdget(fd); + int ret; + + if (!f.file) + return -EBADF; + + ret = vfs_fadvise(f.file, offset, len, advice); + fdput(f); return ret; } @@ -203,3 +217,4 @@ SYSCALL_DEFINE4(fadvise64, int, fd, loff_t, offset, size_t, len, int, advice) } #endif +#endif diff --git a/mm/huge_memory.c b/mm/huge_memory.c index c3bc7e9c9a2a..533f9b00147d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -821,11 +821,11 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr, * but we need to be consistent with PTEs and architectures that * can't support a 'special' bit. */ - BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP))); + BUG_ON(!(vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) && + !pfn_t_devmap(pfn)); BUG_ON((vma->vm_flags & (VM_PFNMAP|VM_MIXEDMAP)) == (VM_PFNMAP|VM_MIXEDMAP)); BUG_ON((vma->vm_flags & VM_PFNMAP) && is_cow_mapping(vma->vm_flags)); - BUG_ON(!pfn_t_devmap(pfn)); if (addr < vma->vm_start || addr >= vma->vm_end) return VM_FAULT_SIGBUS; diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 9a085d525bbc..17dd883198ae 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -2097,6 +2097,11 @@ static int __init kmemleak_late_init(void) kmemleak_initialized = 1; + dentry = debugfs_create_file("kmemleak", 0644, NULL, NULL, + &kmemleak_fops); + if (!dentry) + pr_warn("Failed to create the debugfs kmemleak file\n"); + if (kmemleak_error) { /* * Some error occurred and kmemleak was disabled. There is a @@ -2108,10 +2113,6 @@ static int __init kmemleak_late_init(void) return -ENOMEM; } - dentry = debugfs_create_file("kmemleak", 0644, NULL, NULL, - &kmemleak_fops); - if (!dentry) - pr_warn("Failed to create the debugfs kmemleak file\n"); mutex_lock(&scan_mutex); start_scan_thread(); mutex_unlock(&scan_mutex); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4ead5a4817de..e79cb59552d9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1701,8 +1701,6 @@ static enum oom_status mem_cgroup_oom(struct mem_cgroup *memcg, gfp_t mask, int if (mem_cgroup_out_of_memory(memcg, mask, order)) return OOM_SUCCESS; - WARN(1,"Memory cgroup charge failed because of no reclaimable memory! " - "This looks like a misconfiguration or a kernel bug."); return OOM_FAILED; } diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c index 9eea6e809a4e..38d94b703e9d 100644 --- a/mm/memory_hotplug.c +++ b/mm/memory_hotplug.c @@ -1333,7 +1333,8 @@ static unsigned long scan_movable_pages(unsigned long start, unsigned long end) if (__PageMovable(page)) return pfn; if (PageHuge(page)) { - if (page_huge_active(page)) + if (hugepage_migration_supported(page_hstate(page)) && + page_huge_active(page)) return pfn; else pfn = round_up(pfn + 1, diff --git a/mm/oom_kill.c b/mm/oom_kill.c index b5b25e4dcbbb..f10aa5360616 100644 --- a/mm/oom_kill.c +++ b/mm/oom_kill.c @@ -522,6 +522,7 @@ bool __oom_reap_task_mm(struct mm_struct *mm) tlb_gather_mmu(&tlb, mm, start, end); if (mmu_notifier_invalidate_range_start_nonblock(mm, start, end)) { + tlb_finish_mmu(&tlb, start, end); ret = false; continue; } @@ -1103,10 +1104,17 @@ bool out_of_memory(struct oom_control *oc) } select_bad_process(oc); - /* Found nothing?!?! Either we hang forever, or we panic. */ - if (!oc->chosen && !is_sysrq_oom(oc) && !is_memcg_oom(oc)) { + /* Found nothing?!?! */ + if (!oc->chosen) { dump_header(oc, NULL); - panic("Out of memory and no killable processes...\n"); + pr_warn("Out of memory and no killable processes...\n"); + /* + * If we got here due to an actual allocation at the + * system level, we cannot survive this and will enter + * an endless loop in the allocator. Bail out now. + */ + if (!is_sysrq_oom(oc) && !is_memcg_oom(oc)) + panic("System is deadlocked on memory\n"); } if (oc->chosen && oc->chosen != (void *)-1UL) oom_kill_process(oc, !is_memcg_oom(oc) ? "Out of memory" : diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 05e983f42316..89d2a2ab3fe6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -7708,6 +7708,10 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count, * handle each tail page individually in migration. */ if (PageHuge(page)) { + + if (!hugepage_migration_supported(page_hstate(page))) + goto unmovable; + iter = round_up(iter + 1, 1<<compound_order(page)) - 1; continue; } diff --git a/mm/readahead.c b/mm/readahead.c index a59ea70527b9..4e630143a0ba 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -20,6 +20,7 @@ #include <linux/file.h> #include <linux/mm_inline.h> #include <linux/blk-cgroup.h> +#include <linux/fadvise.h> #include "internal.h" @@ -575,24 +576,6 @@ page_cache_async_readahead(struct address_space *mapping, } EXPORT_SYMBOL_GPL(page_cache_async_readahead); -static ssize_t -do_readahead(struct address_space *mapping, struct file *filp, - pgoff_t index, unsigned long nr) -{ - if (!mapping || !mapping->a_ops) - return -EINVAL; - - /* - * Readahead doesn't make sense for DAX inodes, but we don't want it - * to report a failure either. Instead, we just return success and - * don't do any work. - */ - if (dax_mapping(mapping)) - return 0; - - return force_page_cache_readahead(mapping, filp, index, nr); -} - ssize_t ksys_readahead(int fd, loff_t offset, size_t count) { ssize_t ret; @@ -600,16 +583,22 @@ ssize_t ksys_readahead(int fd, loff_t offset, size_t count) ret = -EBADF; f = fdget(fd); - if (f.file) { - if (f.file->f_mode & FMODE_READ) { - struct address_space *mapping = f.file->f_mapping; - pgoff_t start = offset >> PAGE_SHIFT; - pgoff_t end = (offset + count - 1) >> PAGE_SHIFT; - unsigned long len = end - start + 1; - ret = do_readahead(mapping, f.file, start, len); - } - fdput(f); - } + if (!f.file || !(f.file->f_mode & FMODE_READ)) + goto out; + + /* + * The readahead() syscall is intended to run only on files + * that can execute readahead. If readahead is not possible + * on this file, then we must return -EINVAL. + */ + ret = -EINVAL; + if (!f.file->f_mapping || !f.file->f_mapping->a_ops || + !S_ISREG(file_inode(f.file)->i_mode)) + goto out; + + ret = vfs_fadvise(f.file, offset, count, POSIX_FADV_WILLNEED); +out: + fdput(f); return ret; } diff --git a/mm/shmem.c b/mm/shmem.c index 0376c124b043..446942677cd4 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2227,6 +2227,8 @@ static struct inode *shmem_get_inode(struct super_block *sb, const struct inode mpol_shared_policy_init(&info->policy, NULL); break; } + + lockdep_annotate_inode_mutex_key(inode); } else shmem_free_inode(sb); return inode; diff --git a/mm/util.c b/mm/util.c index d2890a407332..9e3ebd2ef65f 100644 --- a/mm/util.c +++ b/mm/util.c @@ -435,11 +435,14 @@ void *kvmalloc_node(size_t size, gfp_t flags, int node) EXPORT_SYMBOL(kvmalloc_node); /** - * kvfree - free memory allocated with kvmalloc - * @addr: pointer returned by kvmalloc + * kvfree() - Free memory. + * @addr: Pointer to allocated memory. * - * If the memory is allocated from vmalloc area it is freed with vfree(). - * Otherwise kfree() is used. + * kvfree frees memory allocated by any of vmalloc(), kmalloc() or kvmalloc(). + * It is slightly more efficient to use kfree() or vfree() if you are certain + * that you know which one to use. + * + * Context: Any context except NMI. */ void kvfree(const void *addr) { diff --git a/mm/vmacache.c b/mm/vmacache.c index ea517bef7dc5..cdc32a3b02fa 100644 --- a/mm/vmacache.c +++ b/mm/vmacache.c @@ -20,44 +20,6 @@ #define VMACACHE_HASH(addr) ((addr >> VMACACHE_SHIFT) & VMACACHE_MASK) /* - * Flush vma caches for threads that share a given mm. - * - * The operation is safe because the caller holds the mmap_sem - * exclusively and other threads accessing the vma cache will - * have mmap_sem held at least for read, so no extra locking - * is required to maintain the vma cache. - */ -void vmacache_flush_all(struct mm_struct *mm) -{ - struct task_struct *g, *p; - - count_vm_vmacache_event(VMACACHE_FULL_FLUSHES); - - /* - * Single threaded tasks need not iterate the entire - * list of process. We can avoid the flushing as well - * since the mm's seqnum was increased and don't have - * to worry about other threads' seqnum. Current's - * flush will occur upon the next lookup. - */ - if (atomic_read(&mm->mm_users) == 1) - return; - - rcu_read_lock(); - for_each_process_thread(g, p) { - /* - * Only flush the vmacache pointers as the - * mm seqnum is already set and curr's will - * be set upon invalidation when the next - * lookup is done. - */ - if (mm == p->mm) - vmacache_flush(p); - } - rcu_read_unlock(); -} - -/* * This task may be accessing a foreign mm via (for example) * get_user_pages()->find_vma(). The vmacache is task-local and this * task's vmacache pertains to a different mm (ie, its own). There is diff --git a/mm/vmscan.c b/mm/vmscan.c index 7e7d25504651..c7ce2c161225 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -476,6 +476,17 @@ static unsigned long do_shrink_slab(struct shrink_control *shrinkctl, delta = freeable >> priority; delta *= 4; do_div(delta, shrinker->seeks); + + /* + * Make sure we apply some minimal pressure on default priority + * even on small cgroups. Stale objects are not only consuming memory + * by themselves, but can also hold a reference to a dying cgroup, + * preventing it from being reclaimed. A dying cgroup with all + * corresponding structures like per-cpu stats and kmem caches + * can be really big, so it may lead to a significant waste of memory. + */ + delta = max_t(unsigned long long, delta, min(freeable, batch_size)); + total_scan += delta; if (total_scan < 0) { pr_err("shrink_slab: %pF negative objects to delete nr=%ld\n", diff --git a/net/batman-adv/bat_v_elp.c b/net/batman-adv/bat_v_elp.c index 71c20c1d4002..9f481cfdf77d 100644 --- a/net/batman-adv/bat_v_elp.c +++ b/net/batman-adv/bat_v_elp.c @@ -241,7 +241,7 @@ batadv_v_elp_wifi_neigh_probe(struct batadv_hardif_neigh_node *neigh) * the packet to be exactly of that size to make the link * throughput estimation effective. */ - skb_put(skb, probe_len - hard_iface->bat_v.elp_skb->len); + skb_put_zero(skb, probe_len - hard_iface->bat_v.elp_skb->len); batadv_dbg(BATADV_DBG_BATMAN, bat_priv, "Sending unicast (probe) ELP packet on interface %s to %pM\n", @@ -268,6 +268,7 @@ static void batadv_v_elp_periodic_work(struct work_struct *work) struct batadv_priv *bat_priv; struct sk_buff *skb; u32 elp_interval; + bool ret; bat_v = container_of(work, struct batadv_hard_iface_bat_v, elp_wq.work); hard_iface = container_of(bat_v, struct batadv_hard_iface, bat_v); @@ -329,8 +330,11 @@ static void batadv_v_elp_periodic_work(struct work_struct *work) * may sleep and that is not allowed in an rcu protected * context. Therefore schedule a task for that. */ - queue_work(batadv_event_workqueue, - &hardif_neigh->bat_v.metric_work); + ret = queue_work(batadv_event_workqueue, + &hardif_neigh->bat_v.metric_work); + + if (!ret) + batadv_hardif_neigh_put(hardif_neigh); } rcu_read_unlock(); diff --git a/net/batman-adv/bridge_loop_avoidance.c b/net/batman-adv/bridge_loop_avoidance.c index ff9659af6b91..5f1aeeded0e3 100644 --- a/net/batman-adv/bridge_loop_avoidance.c +++ b/net/batman-adv/bridge_loop_avoidance.c @@ -1772,6 +1772,7 @@ batadv_bla_loopdetect_check(struct batadv_priv *bat_priv, struct sk_buff *skb, { struct batadv_bla_backbone_gw *backbone_gw; struct ethhdr *ethhdr; + bool ret; ethhdr = eth_hdr(skb); @@ -1795,8 +1796,13 @@ batadv_bla_loopdetect_check(struct batadv_priv *bat_priv, struct sk_buff *skb, if (unlikely(!backbone_gw)) return true; - queue_work(batadv_event_workqueue, &backbone_gw->report_work); - /* backbone_gw is unreferenced in the report work function function */ + ret = queue_work(batadv_event_workqueue, &backbone_gw->report_work); + + /* backbone_gw is unreferenced in the report work function function + * if queue_work() call was successful + */ + if (!ret) + batadv_backbone_gw_put(backbone_gw); return true; } diff --git a/net/batman-adv/gateway_client.c b/net/batman-adv/gateway_client.c index 8b198ee798c9..140c61a3f1ec 100644 --- a/net/batman-adv/gateway_client.c +++ b/net/batman-adv/gateway_client.c @@ -32,6 +32,7 @@ #include <linux/kernel.h> #include <linux/kref.h> #include <linux/list.h> +#include <linux/lockdep.h> #include <linux/netdevice.h> #include <linux/netlink.h> #include <linux/rculist.h> @@ -348,6 +349,9 @@ out: * @bat_priv: the bat priv with all the soft interface information * @orig_node: originator announcing gateway capabilities * @gateway: announced bandwidth information + * + * Has to be called with the appropriate locks being acquired + * (gw.list_lock). */ static void batadv_gw_node_add(struct batadv_priv *bat_priv, struct batadv_orig_node *orig_node, @@ -355,6 +359,8 @@ static void batadv_gw_node_add(struct batadv_priv *bat_priv, { struct batadv_gw_node *gw_node; + lockdep_assert_held(&bat_priv->gw.list_lock); + if (gateway->bandwidth_down == 0) return; @@ -369,10 +375,8 @@ static void batadv_gw_node_add(struct batadv_priv *bat_priv, gw_node->bandwidth_down = ntohl(gateway->bandwidth_down); gw_node->bandwidth_up = ntohl(gateway->bandwidth_up); - spin_lock_bh(&bat_priv->gw.list_lock); kref_get(&gw_node->refcount); hlist_add_head_rcu(&gw_node->list, &bat_priv->gw.gateway_list); - spin_unlock_bh(&bat_priv->gw.list_lock); batadv_dbg(BATADV_DBG_BATMAN, bat_priv, "Found new gateway %pM -> gw bandwidth: %u.%u/%u.%u MBit\n", @@ -428,11 +432,14 @@ void batadv_gw_node_update(struct batadv_priv *bat_priv, { struct batadv_gw_node *gw_node, *curr_gw = NULL; + spin_lock_bh(&bat_priv->gw.list_lock); gw_node = batadv_gw_node_get(bat_priv, orig_node); if (!gw_node) { batadv_gw_node_add(bat_priv, orig_node, gateway); + spin_unlock_bh(&bat_priv->gw.list_lock); goto out; } + spin_unlock_bh(&bat_priv->gw.list_lock); if (gw_node->bandwidth_down == ntohl(gateway->bandwidth_down) && gw_node->bandwidth_up == ntohl(gateway->bandwidth_up)) diff --git a/net/batman-adv/main.h b/net/batman-adv/main.h index 8da3c9336111..3ccc75ee719c 100644 --- a/net/batman-adv/main.h +++ b/net/batman-adv/main.h @@ -25,7 +25,7 @@ #define BATADV_DRIVER_DEVICE "batman-adv" #ifndef BATADV_SOURCE_VERSION -#define BATADV_SOURCE_VERSION "2018.2" +#define BATADV_SOURCE_VERSION "2018.3" #endif /* B.A.T.M.A.N. parameters */ diff --git a/net/batman-adv/network-coding.c b/net/batman-adv/network-coding.c index c3578444f3cb..34caf129a9bf 100644 --- a/net/batman-adv/network-coding.c +++ b/net/batman-adv/network-coding.c @@ -854,16 +854,27 @@ batadv_nc_get_nc_node(struct batadv_priv *bat_priv, spinlock_t *lock; /* Used to lock list selected by "int in_coding" */ struct list_head *list; + /* Select ingoing or outgoing coding node */ + if (in_coding) { + lock = &orig_neigh_node->in_coding_list_lock; + list = &orig_neigh_node->in_coding_list; + } else { + lock = &orig_neigh_node->out_coding_list_lock; + list = &orig_neigh_node->out_coding_list; + } + + spin_lock_bh(lock); + /* Check if nc_node is already added */ nc_node = batadv_nc_find_nc_node(orig_node, orig_neigh_node, in_coding); /* Node found */ if (nc_node) - return nc_node; + goto unlock; nc_node = kzalloc(sizeof(*nc_node), GFP_ATOMIC); if (!nc_node) - return NULL; + goto unlock; /* Initialize nc_node */ INIT_LIST_HEAD(&nc_node->list); @@ -872,22 +883,14 @@ batadv_nc_get_nc_node(struct batadv_priv *bat_priv, kref_get(&orig_neigh_node->refcount); nc_node->orig_node = orig_neigh_node; - /* Select ingoing or outgoing coding node */ - if (in_coding) { - lock = &orig_neigh_node->in_coding_list_lock; - list = &orig_neigh_node->in_coding_list; - } else { - lock = &orig_neigh_node->out_coding_list_lock; - list = &orig_neigh_node->out_coding_list; - } - batadv_dbg(BATADV_DBG_NC, bat_priv, "Adding nc_node %pM -> %pM\n", nc_node->addr, nc_node->orig_node->orig); /* Add nc_node to orig_node */ - spin_lock_bh(lock); kref_get(&nc_node->refcount); list_add_tail_rcu(&nc_node->list, list); + +unlock: spin_unlock_bh(lock); return nc_node; diff --git a/net/batman-adv/soft-interface.c b/net/batman-adv/soft-interface.c index 1485263a348b..626ddca332db 100644 --- a/net/batman-adv/soft-interface.c +++ b/net/batman-adv/soft-interface.c @@ -574,15 +574,20 @@ int batadv_softif_create_vlan(struct batadv_priv *bat_priv, unsigned short vid) struct batadv_softif_vlan *vlan; int err; + spin_lock_bh(&bat_priv->softif_vlan_list_lock); + vlan = batadv_softif_vlan_get(bat_priv, vid); if (vlan) { batadv_softif_vlan_put(vlan); + spin_unlock_bh(&bat_priv->softif_vlan_list_lock); return -EEXIST; } vlan = kzalloc(sizeof(*vlan), GFP_ATOMIC); - if (!vlan) + if (!vlan) { + spin_unlock_bh(&bat_priv->softif_vlan_list_lock); return -ENOMEM; + } vlan->bat_priv = bat_priv; vlan->vid = vid; @@ -590,17 +595,23 @@ int batadv_softif_create_vlan(struct batadv_priv *bat_priv, unsigned short vid) atomic_set(&vlan->ap_isolation, 0); + kref_get(&vlan->refcount); + hlist_add_head_rcu(&vlan->list, &bat_priv->softif_vlan_list); + spin_unlock_bh(&bat_priv->softif_vlan_list_lock); + + /* batadv_sysfs_add_vlan cannot be in the spinlock section due to the + * sleeping behavior of the sysfs functions and the fs_reclaim lock + */ err = batadv_sysfs_add_vlan(bat_priv->soft_iface, vlan); if (err) { - kfree(vlan); + /* ref for the function */ + batadv_softif_vlan_put(vlan); + + /* ref for the list */ + batadv_softif_vlan_put(vlan); return err; } - spin_lock_bh(&bat_priv->softif_vlan_list_lock); - kref_get(&vlan->refcount); - hlist_add_head_rcu(&vlan->list, &bat_priv->softif_vlan_list); - spin_unlock_bh(&bat_priv->softif_vlan_list_lock); - /* add a new TT local entry. This one will be marked with the NOPURGE * flag */ diff --git a/net/batman-adv/sysfs.c b/net/batman-adv/sysfs.c index f2eef43bd2ec..09427fc6494a 100644 --- a/net/batman-adv/sysfs.c +++ b/net/batman-adv/sysfs.c @@ -188,7 +188,8 @@ ssize_t batadv_store_##_name(struct kobject *kobj, \ \ return __batadv_store_uint_attr(buff, count, _min, _max, \ _post_func, attr, \ - &bat_priv->_var, net_dev); \ + &bat_priv->_var, net_dev, \ + NULL); \ } #define BATADV_ATTR_SIF_SHOW_UINT(_name, _var) \ @@ -262,7 +263,9 @@ ssize_t batadv_store_##_name(struct kobject *kobj, \ \ length = __batadv_store_uint_attr(buff, count, _min, _max, \ _post_func, attr, \ - &hard_iface->_var, net_dev); \ + &hard_iface->_var, \ + hard_iface->soft_iface, \ + net_dev); \ \ batadv_hardif_put(hard_iface); \ return length; \ @@ -356,10 +359,12 @@ __batadv_store_bool_attr(char *buff, size_t count, static int batadv_store_uint_attr(const char *buff, size_t count, struct net_device *net_dev, + struct net_device *slave_dev, const char *attr_name, unsigned int min, unsigned int max, atomic_t *attr) { + char ifname[IFNAMSIZ + 3] = ""; unsigned long uint_val; int ret; @@ -385,8 +390,11 @@ static int batadv_store_uint_attr(const char *buff, size_t count, if (atomic_read(attr) == uint_val) return count; - batadv_info(net_dev, "%s: Changing from: %i to: %lu\n", - attr_name, atomic_read(attr), uint_val); + if (slave_dev) + snprintf(ifname, sizeof(ifname), "%s: ", slave_dev->name); + + batadv_info(net_dev, "%s: %sChanging from: %i to: %lu\n", + attr_name, ifname, atomic_read(attr), uint_val); atomic_set(attr, uint_val); return count; @@ -397,12 +405,13 @@ static ssize_t __batadv_store_uint_attr(const char *buff, size_t count, void (*post_func)(struct net_device *), const struct attribute *attr, atomic_t *attr_store, - struct net_device *net_dev) + struct net_device *net_dev, + struct net_device *slave_dev) { int ret; - ret = batadv_store_uint_attr(buff, count, net_dev, attr->name, min, max, - attr_store); + ret = batadv_store_uint_attr(buff, count, net_dev, slave_dev, + attr->name, min, max, attr_store); if (post_func && ret) post_func(net_dev); @@ -571,7 +580,7 @@ static ssize_t batadv_store_gw_sel_class(struct kobject *kobj, return __batadv_store_uint_attr(buff, count, 1, BATADV_TQ_MAX_VALUE, batadv_post_gw_reselect, attr, &bat_priv->gw.sel_class, - bat_priv->soft_iface); + bat_priv->soft_iface, NULL); } static ssize_t batadv_show_gw_bwidth(struct kobject *kobj, @@ -1090,8 +1099,9 @@ static ssize_t batadv_store_throughput_override(struct kobject *kobj, if (old_tp_override == tp_override) goto out; - batadv_info(net_dev, "%s: Changing from: %u.%u MBit to: %u.%u MBit\n", - "throughput_override", + batadv_info(hard_iface->soft_iface, + "%s: %s: Changing from: %u.%u MBit to: %u.%u MBit\n", + "throughput_override", net_dev->name, old_tp_override / 10, old_tp_override % 10, tp_override / 10, tp_override % 10); diff --git a/net/batman-adv/translation-table.c b/net/batman-adv/translation-table.c index 12a2b7d21376..d21624c44665 100644 --- a/net/batman-adv/translation-table.c +++ b/net/batman-adv/translation-table.c @@ -1613,6 +1613,8 @@ batadv_tt_global_orig_entry_add(struct batadv_tt_global_entry *tt_global, { struct batadv_tt_orig_list_entry *orig_entry; + spin_lock_bh(&tt_global->list_lock); + orig_entry = batadv_tt_global_orig_entry_find(tt_global, orig_node); if (orig_entry) { /* refresh the ttvn: the current value could be a bogus one that @@ -1635,11 +1637,9 @@ batadv_tt_global_orig_entry_add(struct batadv_tt_global_entry *tt_global, orig_entry->flags = flags; kref_init(&orig_entry->refcount); - spin_lock_bh(&tt_global->list_lock); kref_get(&orig_entry->refcount); hlist_add_head_rcu(&orig_entry->list, &tt_global->orig_list); - spin_unlock_bh(&tt_global->list_lock); atomic_inc(&tt_global->orig_list_count); sync_flags: @@ -1647,6 +1647,8 @@ sync_flags: out: if (orig_entry) batadv_tt_orig_list_entry_put(orig_entry); + + spin_unlock_bh(&tt_global->list_lock); } /** diff --git a/net/batman-adv/tvlv.c b/net/batman-adv/tvlv.c index a637458205d1..40e69c9346d2 100644 --- a/net/batman-adv/tvlv.c +++ b/net/batman-adv/tvlv.c @@ -529,15 +529,20 @@ void batadv_tvlv_handler_register(struct batadv_priv *bat_priv, { struct batadv_tvlv_handler *tvlv_handler; + spin_lock_bh(&bat_priv->tvlv.handler_list_lock); + tvlv_handler = batadv_tvlv_handler_get(bat_priv, type, version); if (tvlv_handler) { + spin_unlock_bh(&bat_priv->tvlv.handler_list_lock); batadv_tvlv_handler_put(tvlv_handler); return; } tvlv_handler = kzalloc(sizeof(*tvlv_handler), GFP_ATOMIC); - if (!tvlv_handler) + if (!tvlv_handler) { + spin_unlock_bh(&bat_priv->tvlv.handler_list_lock); return; + } tvlv_handler->ogm_handler = optr; tvlv_handler->unicast_handler = uptr; @@ -547,7 +552,6 @@ void batadv_tvlv_handler_register(struct batadv_priv *bat_priv, kref_init(&tvlv_handler->refcount); INIT_HLIST_NODE(&tvlv_handler->list); - spin_lock_bh(&bat_priv->tvlv.handler_list_lock); kref_get(&tvlv_handler->refcount); hlist_add_head_rcu(&tvlv_handler->list, &bat_priv->tvlv.handler_list); spin_unlock_bh(&bat_priv->tvlv.handler_list_lock); diff --git a/net/bluetooth/smp.c b/net/bluetooth/smp.c index ae91e2d40056..3a7b0773536b 100644 --- a/net/bluetooth/smp.c +++ b/net/bluetooth/smp.c @@ -83,6 +83,7 @@ enum { struct smp_dev { /* Secure Connections OOB data */ + bool local_oob; u8 local_pk[64]; u8 local_rand[16]; bool debug_key; @@ -599,6 +600,8 @@ int smp_generate_oob(struct hci_dev *hdev, u8 hash[16], u8 rand[16]) memcpy(rand, smp->local_rand, 16); + smp->local_oob = true; + return 0; } @@ -1785,7 +1788,7 @@ static u8 smp_cmd_pairing_req(struct l2cap_conn *conn, struct sk_buff *skb) * successfully received our local OOB data - therefore set the * flag to indicate that local OOB is in use. */ - if (req->oob_flag == SMP_OOB_PRESENT) + if (req->oob_flag == SMP_OOB_PRESENT && SMP_DEV(hdev)->local_oob) set_bit(SMP_FLAG_LOCAL_OOB, &smp->flags); /* SMP over BR/EDR requires special treatment */ @@ -1967,7 +1970,7 @@ static u8 smp_cmd_pairing_rsp(struct l2cap_conn *conn, struct sk_buff *skb) * successfully received our local OOB data - therefore set the * flag to indicate that local OOB is in use. */ - if (rsp->oob_flag == SMP_OOB_PRESENT) + if (rsp->oob_flag == SMP_OOB_PRESENT && SMP_DEV(hdev)->local_oob) set_bit(SMP_FLAG_LOCAL_OOB, &smp->flags); smp->prsp[0] = SMP_CMD_PAIRING_RSP; @@ -2697,7 +2700,13 @@ static int smp_cmd_public_key(struct l2cap_conn *conn, struct sk_buff *skb) * key was set/generated. */ if (test_bit(SMP_FLAG_LOCAL_OOB, &smp->flags)) { - struct smp_dev *smp_dev = chan->data; + struct l2cap_chan *hchan = hdev->smp_data; + struct smp_dev *smp_dev; + + if (!hchan || !hchan->data) + return SMP_UNSPECIFIED; + + smp_dev = hchan->data; tfm_ecdh = smp_dev->tfm_ecdh; } else { @@ -3230,6 +3239,7 @@ static struct l2cap_chan *smp_add_cid(struct hci_dev *hdev, u16 cid) return ERR_CAST(tfm_ecdh); } + smp->local_oob = false; smp->tfm_aes = tfm_aes; smp->tfm_cmac = tfm_cmac; smp->tfm_ecdh = tfm_ecdh; diff --git a/net/core/devlink.c b/net/core/devlink.c index 65fc366a78a4..8c0ed225e280 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -2592,7 +2592,7 @@ send_done: if (!nlh) { err = devlink_dpipe_send_and_alloc_skb(&skb, info); if (err) - goto err_skb_send_alloc; + return err; goto send_done; } return genlmsg_reply(skb, info); @@ -2600,7 +2600,6 @@ send_done: nla_put_failure: err = -EMSGSIZE; err_resource_put: -err_skb_send_alloc: nlmsg_free(skb); return err; } diff --git a/net/core/ethtool.c b/net/core/ethtool.c index c9993c6c2fd4..234a0ec2e932 100644 --- a/net/core/ethtool.c +++ b/net/core/ethtool.c @@ -2624,6 +2624,7 @@ int dev_ethtool(struct net *net, struct ifreq *ifr) case ETHTOOL_GPHYSTATS: case ETHTOOL_GTSO: case ETHTOOL_GPERMADDR: + case ETHTOOL_GUFO: case ETHTOOL_GGSO: case ETHTOOL_GGRO: case ETHTOOL_GFLAGS: diff --git a/net/core/filter.c b/net/core/filter.c index c25eb36f1320..5e00f2b85a56 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -2282,14 +2282,21 @@ static const struct bpf_func_proto bpf_msg_cork_bytes_proto = { .arg2_type = ARG_ANYTHING, }; +#define sk_msg_iter_var(var) \ + do { \ + var++; \ + if (var == MAX_SKB_FRAGS) \ + var = 0; \ + } while (0) + BPF_CALL_4(bpf_msg_pull_data, struct sk_msg_buff *, msg, u32, start, u32, end, u64, flags) { - unsigned int len = 0, offset = 0, copy = 0; + unsigned int len = 0, offset = 0, copy = 0, poffset = 0; + int bytes = end - start, bytes_sg_total; struct scatterlist *sg = msg->sg_data; int first_sg, last_sg, i, shift; unsigned char *p, *to, *from; - int bytes = end - start; struct page *page; if (unlikely(flags || end <= start)) @@ -2299,21 +2306,22 @@ BPF_CALL_4(bpf_msg_pull_data, i = msg->sg_start; do { len = sg[i].length; - offset += len; if (start < offset + len) break; - i++; - if (i == MAX_SKB_FRAGS) - i = 0; + offset += len; + sk_msg_iter_var(i); } while (i != msg->sg_end); if (unlikely(start >= offset + len)) return -EINVAL; - if (!msg->sg_copy[i] && bytes <= len) - goto out; - first_sg = i; + /* The start may point into the sg element so we need to also + * account for the headroom. + */ + bytes_sg_total = start - offset + bytes; + if (!msg->sg_copy[i] && bytes_sg_total <= len) + goto out; /* At this point we need to linearize multiple scatterlist * elements or a single shared page. Either way we need to @@ -2327,37 +2335,33 @@ BPF_CALL_4(bpf_msg_pull_data, */ do { copy += sg[i].length; - i++; - if (i == MAX_SKB_FRAGS) - i = 0; - if (bytes < copy) + sk_msg_iter_var(i); + if (bytes_sg_total <= copy) break; } while (i != msg->sg_end); last_sg = i; - if (unlikely(copy < end - start)) + if (unlikely(bytes_sg_total > copy)) return -EINVAL; - page = alloc_pages(__GFP_NOWARN | GFP_ATOMIC, get_order(copy)); + page = alloc_pages(__GFP_NOWARN | GFP_ATOMIC | __GFP_COMP, + get_order(copy)); if (unlikely(!page)) return -ENOMEM; p = page_address(page); - offset = 0; i = first_sg; do { from = sg_virt(&sg[i]); len = sg[i].length; - to = p + offset; + to = p + poffset; memcpy(to, from, len); - offset += len; + poffset += len; sg[i].length = 0; put_page(sg_page(&sg[i])); - i++; - if (i == MAX_SKB_FRAGS) - i = 0; + sk_msg_iter_var(i); } while (i != last_sg); sg[first_sg].length = copy; @@ -2367,11 +2371,15 @@ BPF_CALL_4(bpf_msg_pull_data, * had a single entry though we can just replace it and * be done. Otherwise walk the ring and shift the entries. */ - shift = last_sg - first_sg - 1; + WARN_ON_ONCE(last_sg == first_sg); + shift = last_sg > first_sg ? + last_sg - first_sg - 1 : + MAX_SKB_FRAGS - first_sg + last_sg - 1; if (!shift) goto out; - i = first_sg + 1; + i = first_sg; + sk_msg_iter_var(i); do { int move_from; @@ -2388,15 +2396,13 @@ BPF_CALL_4(bpf_msg_pull_data, sg[move_from].page_link = 0; sg[move_from].offset = 0; - i++; - if (i == MAX_SKB_FRAGS) - i = 0; + sk_msg_iter_var(i); } while (1); msg->sg_end -= shift; if (msg->sg_end < 0) msg->sg_end += MAX_SKB_FRAGS; out: - msg->data = sg_virt(&sg[i]) + start - offset; + msg->data = sg_virt(&sg[first_sg]) + start - offset; msg->data_end = msg->data + bytes; return 0; @@ -7281,7 +7287,7 @@ static u32 sk_reuseport_convert_ctx_access(enum bpf_access_type type, break; case offsetof(struct sk_reuseport_md, ip_protocol): - BUILD_BUG_ON(hweight_long(SK_FL_PROTO_MASK) != BITS_PER_BYTE); + BUILD_BUG_ON(HWEIGHT32(SK_FL_PROTO_MASK) != BITS_PER_BYTE); SK_REUSEPORT_LOAD_SK_FIELD_SIZE_OFF(__sk_flags_offset, BPF_W, 0); *insn++ = BPF_ALU32_IMM(BPF_AND, si->dst_reg, SK_FL_PROTO_MASK); diff --git a/net/core/neighbour.c b/net/core/neighbour.c index aa19d86937af..91592fceeaad 100644 --- a/net/core/neighbour.c +++ b/net/core/neighbour.c @@ -1180,6 +1180,12 @@ int neigh_update(struct neighbour *neigh, const u8 *lladdr, u8 new, lladdr = neigh->ha; } + /* Update confirmed timestamp for neighbour entry after we + * received ARP packet even if it doesn't change IP to MAC binding. + */ + if (new & NUD_CONNECTED) + neigh->confirmed = jiffies; + /* If entry was valid and address is not changed, do not change entry state, if new one is STALE. */ @@ -1201,15 +1207,12 @@ int neigh_update(struct neighbour *neigh, const u8 *lladdr, u8 new, } } - /* Update timestamps only once we know we will make a change to the + /* Update timestamp only once we know we will make a change to the * neighbour entry. Otherwise we risk to move the locktime window with * noop updates and ignore relevant ARP updates. */ - if (new != old || lladdr != neigh->ha) { - if (new & NUD_CONNECTED) - neigh->confirmed = jiffies; + if (new != old || lladdr != neigh->ha) neigh->updated = jiffies; - } if (new != old) { neigh_del_timer(neigh); diff --git a/net/core/netpoll.c b/net/core/netpoll.c index 57557a6a950c..3219a2932463 100644 --- a/net/core/netpoll.c +++ b/net/core/netpoll.c @@ -187,16 +187,16 @@ static void poll_napi(struct net_device *dev) } } -static void netpoll_poll_dev(struct net_device *dev) +void netpoll_poll_dev(struct net_device *dev) { - const struct net_device_ops *ops; struct netpoll_info *ni = rcu_dereference_bh(dev->npinfo); + const struct net_device_ops *ops; /* Don't do any rx activity if the dev_lock mutex is held * the dev_open/close paths use this to block netpoll activity * while changing device state */ - if (down_trylock(&ni->dev_lock)) + if (!ni || down_trylock(&ni->dev_lock)) return; if (!netif_running(dev)) { @@ -205,13 +205,8 @@ static void netpoll_poll_dev(struct net_device *dev) } ops = dev->netdev_ops; - if (!ops->ndo_poll_controller) { - up(&ni->dev_lock); - return; - } - - /* Process pending work on NIC */ - ops->ndo_poll_controller(dev); + if (ops->ndo_poll_controller) + ops->ndo_poll_controller(dev); poll_napi(dev); @@ -219,6 +214,7 @@ static void netpoll_poll_dev(struct net_device *dev) zap_completion_queue(); } +EXPORT_SYMBOL(netpoll_poll_dev); void netpoll_poll_disable(struct net_device *dev) { @@ -613,8 +609,7 @@ int __netpoll_setup(struct netpoll *np, struct net_device *ndev) strlcpy(np->dev_name, ndev->name, IFNAMSIZ); INIT_WORK(&np->cleanup_work, netpoll_async_cleanup); - if ((ndev->priv_flags & IFF_DISABLE_NETPOLL) || - !ndev->netdev_ops->ndo_poll_controller) { + if (ndev->priv_flags & IFF_DISABLE_NETPOLL) { np_err(np, "%s doesn't support polling, aborting\n", np->dev_name); err = -ENOTSUPP; diff --git a/net/core/rtnetlink.c b/net/core/rtnetlink.c index 24431e578310..63ce2283a456 100644 --- a/net/core/rtnetlink.c +++ b/net/core/rtnetlink.c @@ -324,6 +324,10 @@ void rtnl_unregister_all(int protocol) rtnl_lock(); tab = rtnl_msg_handlers[protocol]; + if (!tab) { + rtnl_unlock(); + return; + } RCU_INIT_POINTER(rtnl_msg_handlers[protocol], NULL); for (msgindex = 0; msgindex < RTM_NR_MSGTYPES; msgindex++) { link = tab[msgindex]; @@ -2806,7 +2810,7 @@ int rtnl_configure_link(struct net_device *dev, const struct ifinfomsg *ifm) } if (dev->rtnl_link_state == RTNL_LINK_INITIALIZED) { - __dev_notify_flags(dev, old_flags, 0U); + __dev_notify_flags(dev, old_flags, (old_flags ^ dev->flags)); } else { dev->rtnl_link_state = RTNL_LINK_INITIALIZED; __dev_notify_flags(dev, old_flags, ~0U); diff --git a/net/core/skbuff.c b/net/core/skbuff.c index c996c09d095f..b2c807f67aba 100644 --- a/net/core/skbuff.c +++ b/net/core/skbuff.c @@ -939,9 +939,6 @@ struct ubuf_info *sock_zerocopy_alloc(struct sock *sk, size_t size) WARN_ON_ONCE(!in_task()); - if (!sock_flag(sk, SOCK_ZEROCOPY)) - return NULL; - skb = sock_omalloc(sk, 0, GFP_KERNEL); if (!skb) return NULL; diff --git a/net/dsa/dsa.c b/net/dsa/dsa.c index e63c554e0623..9f3209ff7ffd 100644 --- a/net/dsa/dsa.c +++ b/net/dsa/dsa.c @@ -19,12 +19,10 @@ #include <linux/of_mdio.h> #include <linux/of_platform.h> #include <linux/of_net.h> -#include <linux/of_gpio.h> #include <linux/netdevice.h> #include <linux/sysfs.h> #include <linux/phy_fixed.h> #include <linux/ptp_classify.h> -#include <linux/gpio/consumer.h> #include <linux/etherdevice.h> #include "dsa_priv.h" diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index 20fda8fb8ffd..1fbe2f815474 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -1377,6 +1377,7 @@ struct sk_buff *inet_gso_segment(struct sk_buff *skb, if (encap) skb_reset_inner_headers(skb); skb->network_header = (u8 *)iph - skb->head; + skb_reset_mac_len(skb); } while ((skb = skb->next)); out: diff --git a/net/ipv4/igmp.c b/net/ipv4/igmp.c index cf75f8944b05..4da39446da2d 100644 --- a/net/ipv4/igmp.c +++ b/net/ipv4/igmp.c @@ -820,10 +820,9 @@ static void igmp_timer_expire(struct timer_list *t) spin_lock(&im->lock); im->tm_running = 0; - if (im->unsolicit_count) { - im->unsolicit_count--; + if (im->unsolicit_count && --im->unsolicit_count) igmp_start_timer(im, unsolicited_report_interval(in_dev)); - } + im->reporter = 1; spin_unlock(&im->lock); @@ -1308,6 +1307,8 @@ static void igmp_group_added(struct ip_mc_list *im) if (in_dev->dead) return; + + im->unsolicit_count = net->ipv4.sysctl_igmp_qrv; if (IGMP_V1_SEEN(in_dev) || IGMP_V2_SEEN(in_dev)) { spin_lock_bh(&im->lock); igmp_start_timer(im, IGMP_INITIAL_REPORT_DELAY); @@ -1391,9 +1392,6 @@ static void __ip_mc_inc_group(struct in_device *in_dev, __be32 addr, unsigned int mode) { struct ip_mc_list *im; -#ifdef CONFIG_IP_MULTICAST - struct net *net = dev_net(in_dev->dev); -#endif ASSERT_RTNL(); @@ -1420,7 +1418,6 @@ static void __ip_mc_inc_group(struct in_device *in_dev, __be32 addr, spin_lock_init(&im->lock); #ifdef CONFIG_IP_MULTICAST timer_setup(&im->timer, igmp_timer_expire, 0); - im->unsolicit_count = net->ipv4.sysctl_igmp_qrv; #endif im->next_rcu = in_dev->mc_list; diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c index 88281fbce88c..e7227128df2c 100644 --- a/net/ipv4/ip_fragment.c +++ b/net/ipv4/ip_fragment.c @@ -599,6 +599,7 @@ static int ip_frag_reasm(struct ipq *qp, struct sk_buff *skb, nextp = &fp->next; fp->prev = NULL; memset(&fp->rbnode, 0, sizeof(fp->rbnode)); + fp->sk = NULL; head->data_len += fp->len; head->len += fp->len; if (head->ip_summed != fp->ip_summed) diff --git a/net/ipv4/ip_gre.c b/net/ipv4/ip_gre.c index 51a5d06085ac..8cce0e9ea08c 100644 --- a/net/ipv4/ip_gre.c +++ b/net/ipv4/ip_gre.c @@ -178,6 +178,9 @@ static void ipgre_err(struct sk_buff *skb, u32 info, if (tpi->proto == htons(ETH_P_TEB)) itn = net_generic(net, gre_tap_net_id); + else if (tpi->proto == htons(ETH_P_ERSPAN) || + tpi->proto == htons(ETH_P_ERSPAN2)) + itn = net_generic(net, erspan_net_id); else itn = net_generic(net, ipgre_net_id); @@ -328,6 +331,8 @@ static int erspan_rcv(struct sk_buff *skb, struct tnl_ptk_info *tpi, ip_tunnel_rcv(tunnel, skb, tpi, tun_dst, log_ecn_error); return PACKET_RCVD; } + return PACKET_REJECT; + drop: kfree_skb(skb); return PACKET_RCVD; @@ -1508,11 +1513,14 @@ nla_put_failure: static void erspan_setup(struct net_device *dev) { + struct ip_tunnel *t = netdev_priv(dev); + ether_setup(dev); dev->netdev_ops = &erspan_netdev_ops; dev->priv_flags &= ~IFF_TX_SKB_SHARING; dev->priv_flags |= IFF_LIVE_ADDR_CHANGE; ip_tunnel_setup(dev, erspan_net_id); + t->erspan_ver = 1; } static const struct nla_policy ipgre_policy[IFLA_GRE_MAX + 1] = { diff --git a/net/ipv4/ip_tunnel.c b/net/ipv4/ip_tunnel.c index c4f5602308ed..284a22154b4e 100644 --- a/net/ipv4/ip_tunnel.c +++ b/net/ipv4/ip_tunnel.c @@ -627,6 +627,7 @@ void ip_tunnel_xmit(struct sk_buff *skb, struct net_device *dev, const struct iphdr *tnl_params, u8 protocol) { struct ip_tunnel *tunnel = netdev_priv(dev); + unsigned int inner_nhdr_len = 0; const struct iphdr *inner_iph; struct flowi4 fl4; u8 tos, ttl; @@ -636,6 +637,14 @@ void ip_tunnel_xmit(struct sk_buff *skb, struct net_device *dev, __be32 dst; bool connected; + /* ensure we can access the inner net header, for several users below */ + if (skb->protocol == htons(ETH_P_IP)) + inner_nhdr_len = sizeof(struct iphdr); + else if (skb->protocol == htons(ETH_P_IPV6)) + inner_nhdr_len = sizeof(struct ipv6hdr); + if (unlikely(!pskb_may_pull(skb, inner_nhdr_len))) + goto tx_error; + inner_iph = (const struct iphdr *)skb_inner_network_header(skb); connected = (tunnel->parms.iph.daddr != 0); diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig index d9504adc47b3..184bf2e0a1ed 100644 --- a/net/ipv4/netfilter/Kconfig +++ b/net/ipv4/netfilter/Kconfig @@ -106,6 +106,10 @@ config NF_NAT_IPV4 if NF_NAT_IPV4 +config NF_NAT_MASQUERADE_IPV4 + bool + +if NF_TABLES config NFT_CHAIN_NAT_IPV4 depends on NF_TABLES_IPV4 tristate "IPv4 nf_tables nat chain support" @@ -115,9 +119,6 @@ config NFT_CHAIN_NAT_IPV4 packet transformations such as the source, destination address and source and destination ports. -config NF_NAT_MASQUERADE_IPV4 - bool - config NFT_MASQ_IPV4 tristate "IPv4 masquerading support for nf_tables" depends on NF_TABLES_IPV4 @@ -135,6 +136,7 @@ config NFT_REDIR_IPV4 help This is the expression that provides IPv4 redirect support for nf_tables. +endif # NF_TABLES config NF_NAT_SNMP_BASIC tristate "Basic SNMP-ALG support" diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index b8af2fec5ad5..10c6246396cc 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1185,7 +1185,7 @@ int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size) flags = msg->msg_flags; - if (flags & MSG_ZEROCOPY && size) { + if (flags & MSG_ZEROCOPY && size && sock_flag(sk, SOCK_ZEROCOPY)) { if (sk->sk_state != TCP_ESTABLISHED) { err = -EINVAL; goto out_err; diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index 4c2dd9f863f7..4cf2f7bb2802 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -6367,8 +6367,8 @@ static bool tcp_syn_flood_action(const struct sock *sk, if (!queue->synflood_warned && net->ipv4.sysctl_tcp_syncookies != 2 && xchg(&queue->synflood_warned, 1) == 0) - pr_info("%s: Possible SYN flooding on port %d. %s. Check SNMP counters.\n", - proto, ntohs(tcp_hdr(skb)->dest), msg); + net_info_ratelimited("%s: Possible SYN flooding on port %d. %s. Check SNMP counters.\n", + proto, ntohs(tcp_hdr(skb)->dest), msg); return want_cookie; } diff --git a/net/ipv4/tcp_minisocks.c b/net/ipv4/tcp_minisocks.c index 75ef332a7caf..12affb7864d9 100644 --- a/net/ipv4/tcp_minisocks.c +++ b/net/ipv4/tcp_minisocks.c @@ -184,8 +184,9 @@ kill: inet_twsk_deschedule_put(tw); return TCP_TW_SUCCESS; } + } else { + inet_twsk_reschedule(tw, TCP_TIMEWAIT_LEN); } - inet_twsk_reschedule(tw, TCP_TIMEWAIT_LEN); if (tmp_opt.saw_tstamp) { tcptw->tw_ts_recent = tmp_opt.rcv_tsval; diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index f4e35b2ff8b8..7d69dd6fa7e8 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -2124,6 +2124,28 @@ static inline int udp4_csum_init(struct sk_buff *skb, struct udphdr *uh, inet_compute_pseudo); } +/* wrapper for udp_queue_rcv_skb tacking care of csum conversion and + * return code conversion for ip layer consumption + */ +static int udp_unicast_rcv_skb(struct sock *sk, struct sk_buff *skb, + struct udphdr *uh) +{ + int ret; + + if (inet_get_convert_csum(sk) && uh->check && !IS_UDPLITE(sk)) + skb_checksum_try_convert(skb, IPPROTO_UDP, uh->check, + inet_compute_pseudo); + + ret = udp_queue_rcv_skb(sk, skb); + + /* a return value > 0 means to resubmit the input, but + * it wants the return to be -protocol, or 0 + */ + if (ret > 0) + return -ret; + return 0; +} + /* * All we need to do is get the socket, and then do a checksum. */ @@ -2170,14 +2192,9 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, if (unlikely(sk->sk_rx_dst != dst)) udp_sk_rx_dst_set(sk, dst); - ret = udp_queue_rcv_skb(sk, skb); + ret = udp_unicast_rcv_skb(sk, skb, uh); sock_put(sk); - /* a return value > 0 means to resubmit the input, but - * it wants the return to be -protocol, or 0 - */ - if (ret > 0) - return -ret; - return 0; + return ret; } if (rt->rt_flags & (RTCF_BROADCAST|RTCF_MULTICAST)) @@ -2185,22 +2202,8 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, saddr, daddr, udptable, proto); sk = __udp4_lib_lookup_skb(skb, uh->source, uh->dest, udptable); - if (sk) { - int ret; - - if (inet_get_convert_csum(sk) && uh->check && !IS_UDPLITE(sk)) - skb_checksum_try_convert(skb, IPPROTO_UDP, uh->check, - inet_compute_pseudo); - - ret = udp_queue_rcv_skb(sk, skb); - - /* a return value > 0 means to resubmit the input, but - * it wants the return to be -protocol, or 0 - */ - if (ret > 0) - return -ret; - return 0; - } + if (sk) + return udp_unicast_rcv_skb(sk, skb, uh); if (!xfrm4_policy_check(NULL, XFRM_POLICY_IN, skb)) goto drop; diff --git a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c index d51a8c0b3372..c63ccce6425f 100644 --- a/net/ipv6/addrconf.c +++ b/net/ipv6/addrconf.c @@ -4201,7 +4201,6 @@ static struct inet6_ifaddr *if6_get_first(struct seq_file *seq, loff_t pos) p++; continue; } - state->offset++; return ifa; } @@ -4225,13 +4224,12 @@ static struct inet6_ifaddr *if6_get_next(struct seq_file *seq, return ifa; } + state->offset = 0; while (++state->bucket < IN6_ADDR_HSIZE) { - state->offset = 0; hlist_for_each_entry_rcu(ifa, &inet6_addr_lst[state->bucket], addr_lst) { if (!net_eq(dev_net(ifa->idev->dev), net)) continue; - state->offset++; return ifa; } } diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c index 673bba31eb18..9a4261e50272 100644 --- a/net/ipv6/af_inet6.c +++ b/net/ipv6/af_inet6.c @@ -938,14 +938,14 @@ static int __init inet6_init(void) err = proto_register(&pingv6_prot, 1); if (err) - goto out_unregister_ping_proto; + goto out_unregister_raw_proto; /* We MUST register RAW sockets before we create the ICMP6, * IGMP6, or NDISC control sockets. */ err = rawv6_init(); if (err) - goto out_unregister_raw_proto; + goto out_unregister_ping_proto; /* Register the family here so that the init calls below will * be able to create sockets. (?? is this dangerous ??) @@ -1113,11 +1113,11 @@ netfilter_fail: igmp_fail: ndisc_cleanup(); ndisc_fail: - ip6_mr_cleanup(); + icmpv6_cleanup(); icmp_fail: - unregister_pernet_subsys(&inet6_net_ops); + ip6_mr_cleanup(); ipmr_fail: - icmpv6_cleanup(); + unregister_pernet_subsys(&inet6_net_ops); register_pernet_fail: sock_unregister(PF_INET6); rtnl_unregister_all(PF_INET6); diff --git a/net/ipv6/ip6_fib.c b/net/ipv6/ip6_fib.c index c861a6d4671d..5516f55e214b 100644 --- a/net/ipv6/ip6_fib.c +++ b/net/ipv6/ip6_fib.c @@ -989,7 +989,10 @@ static int fib6_add_rt2node(struct fib6_node *fn, struct fib6_info *rt, fib6_clean_expires(iter); else fib6_set_expires(iter, rt->expires); - fib6_metric_set(iter, RTAX_MTU, rt->fib6_pmtu); + + if (rt->fib6_pmtu) + fib6_metric_set(iter, RTAX_MTU, + rt->fib6_pmtu); return -EEXIST; } /* If we have the same destination and the same metric, diff --git a/net/ipv6/ip6_gre.c b/net/ipv6/ip6_gre.c index 18a3794b0f52..e493b041d4ac 100644 --- a/net/ipv6/ip6_gre.c +++ b/net/ipv6/ip6_gre.c @@ -1778,6 +1778,7 @@ static void ip6gre_netlink_parms(struct nlattr *data[], if (data[IFLA_GRE_COLLECT_METADATA]) parms->collect_md = true; + parms->erspan_ver = 1; if (data[IFLA_GRE_ERSPAN_VER]) parms->erspan_ver = nla_get_u8(data[IFLA_GRE_ERSPAN_VER]); diff --git a/net/ipv6/ip6_offload.c b/net/ipv6/ip6_offload.c index 37ff4805b20c..c7e495f12011 100644 --- a/net/ipv6/ip6_offload.c +++ b/net/ipv6/ip6_offload.c @@ -115,6 +115,7 @@ static struct sk_buff *ipv6_gso_segment(struct sk_buff *skb, payload_len = skb->len - nhoff - sizeof(*ipv6h); ipv6h->payload_len = htons(payload_len); skb->network_header = (u8 *)ipv6h - skb->head; + skb_reset_mac_len(skb); if (udpfrag) { int err = ip6_find_1stfragopt(skb, &prevhdr); diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c index 16f200f06500..f9f8f554d141 100644 --- a/net/ipv6/ip6_output.c +++ b/net/ipv6/ip6_output.c @@ -219,12 +219,10 @@ int ip6_xmit(const struct sock *sk, struct sk_buff *skb, struct flowi6 *fl6, kfree_skb(skb); return -ENOBUFS; } + if (skb->sk) + skb_set_owner_w(skb2, skb->sk); consume_skb(skb); skb = skb2; - /* skb_set_owner_w() changes sk->sk_wmem_alloc atomically, - * it is safe to call in our context (socket lock not held) - */ - skb_set_owner_w(skb, (struct sock *)sk); } if (opt->opt_flen) ipv6_push_frag_opts(skb, opt, &proto); diff --git a/net/ipv6/ip6_tunnel.c b/net/ipv6/ip6_tunnel.c index 5df2a58d945c..a0b6932c3afd 100644 --- a/net/ipv6/ip6_tunnel.c +++ b/net/ipv6/ip6_tunnel.c @@ -1188,7 +1188,15 @@ route_lookup: init_tel_txopt(&opt, encap_limit); ipv6_push_frag_opts(skb, &opt.ops, &proto); } - hop_limit = hop_limit ? : ip6_dst_hoplimit(dst); + + if (hop_limit == 0) { + if (skb->protocol == htons(ETH_P_IP)) + hop_limit = ip_hdr(skb)->ttl; + else if (skb->protocol == htons(ETH_P_IPV6)) + hop_limit = ipv6_hdr(skb)->hop_limit; + else + hop_limit = ip6_dst_hoplimit(dst); + } /* Calculate max headroom for all the headers and adjust * needed_headroom if necessary. @@ -1226,7 +1234,7 @@ static inline int ip4ip6_tnl_xmit(struct sk_buff *skb, struct net_device *dev) { struct ip6_tnl *t = netdev_priv(dev); - const struct iphdr *iph = ip_hdr(skb); + const struct iphdr *iph; int encap_limit = -1; struct flowi6 fl6; __u8 dsfield; @@ -1234,6 +1242,11 @@ ip4ip6_tnl_xmit(struct sk_buff *skb, struct net_device *dev) u8 tproto; int err; + /* ensure we can access the full inner ip header */ + if (!pskb_may_pull(skb, sizeof(struct iphdr))) + return -1; + + iph = ip_hdr(skb); memset(&(IPCB(skb)->opt), 0, sizeof(IPCB(skb)->opt)); tproto = READ_ONCE(t->parms.proto); @@ -1298,7 +1311,7 @@ static inline int ip6ip6_tnl_xmit(struct sk_buff *skb, struct net_device *dev) { struct ip6_tnl *t = netdev_priv(dev); - struct ipv6hdr *ipv6h = ipv6_hdr(skb); + struct ipv6hdr *ipv6h; int encap_limit = -1; __u16 offset; struct flowi6 fl6; @@ -1307,6 +1320,10 @@ ip6ip6_tnl_xmit(struct sk_buff *skb, struct net_device *dev) u8 tproto; int err; + if (unlikely(!pskb_may_pull(skb, sizeof(*ipv6h)))) + return -1; + + ipv6h = ipv6_hdr(skb); tproto = READ_ONCE(t->parms.proto); if ((tproto != IPPROTO_IPV6 && tproto != 0) || ip6_tnl_addr_conflict(t, ipv6h)) diff --git a/net/ipv6/ip6_vti.c b/net/ipv6/ip6_vti.c index 5095367c7204..eeaf7455d51e 100644 --- a/net/ipv6/ip6_vti.c +++ b/net/ipv6/ip6_vti.c @@ -481,7 +481,7 @@ vti6_xmit(struct sk_buff *skb, struct net_device *dev, struct flowi *fl) } mtu = dst_mtu(dst); - if (!skb->ignore_df && skb->len > mtu) { + if (skb->len > mtu) { skb_dst_update_pmtu(skb, mtu); if (skb->protocol == htons(ETH_P_IPV6)) { diff --git a/net/ipv6/netfilter/nf_conntrack_reasm.c b/net/ipv6/netfilter/nf_conntrack_reasm.c index 2a14d8b65924..8f68a518d9db 100644 --- a/net/ipv6/netfilter/nf_conntrack_reasm.c +++ b/net/ipv6/netfilter/nf_conntrack_reasm.c @@ -445,6 +445,7 @@ nf_ct_frag6_reasm(struct frag_queue *fq, struct sk_buff *prev, struct net_devic else if (head->ip_summed == CHECKSUM_COMPLETE) head->csum = csum_add(head->csum, fp->csum); head->truesize += fp->truesize; + fp->sk = NULL; } sub_frag_mem_limit(fq->q.net, head->truesize); diff --git a/net/ipv6/route.c b/net/ipv6/route.c index c4ea13e8360b..826b14de7dbb 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -364,11 +364,14 @@ EXPORT_SYMBOL(ip6_dst_alloc); static void ip6_dst_destroy(struct dst_entry *dst) { + struct dst_metrics *p = (struct dst_metrics *)DST_METRICS_PTR(dst); struct rt6_info *rt = (struct rt6_info *)dst; struct fib6_info *from; struct inet6_dev *idev; - dst_destroy_metrics_generic(dst); + if (p != &dst_default_metrics && refcount_dec_and_test(&p->refcnt)) + kfree(p); + rt6_uncached_list_del(rt); idev = rt->rt6i_idev; @@ -946,8 +949,6 @@ static void ip6_rt_init_dst_reject(struct rt6_info *rt, struct fib6_info *ort) static void ip6_rt_init_dst(struct rt6_info *rt, struct fib6_info *ort) { - rt->dst.flags |= fib6_info_dst_flags(ort); - if (ort->fib6_flags & RTF_REJECT) { ip6_rt_init_dst_reject(rt, ort); return; @@ -978,6 +979,10 @@ static void rt6_set_from(struct rt6_info *rt, struct fib6_info *from) rt->rt6i_flags &= ~RTF_EXPIRES; rcu_assign_pointer(rt->from, from); dst_init_metrics(&rt->dst, from->fib6_metrics->metrics, true); + if (from->fib6_metrics != &dst_default_metrics) { + rt->dst._metrics |= DST_METRICS_REFCOUNTED; + refcount_inc(&from->fib6_metrics->refcnt); + } } /* Caller must already hold reference to @ort */ @@ -996,7 +1001,6 @@ static void ip6_rt_copy_init(struct rt6_info *rt, struct fib6_info *ort) rt->rt6i_src = ort->fib6_src; #endif rt->rt6i_prefsrc = ort->fib6_prefsrc; - rt->dst.lwtstate = lwtstate_get(ort->fib6_nh.nh_lwtstate); } static struct fib6_node* fib6_backtrack(struct fib6_node *fn, @@ -4671,20 +4675,31 @@ static int rt6_fill_node(struct net *net, struct sk_buff *skb, int iif, int type, u32 portid, u32 seq, unsigned int flags) { - struct rtmsg *rtm; + struct rt6_info *rt6 = (struct rt6_info *)dst; + struct rt6key *rt6_dst, *rt6_src; + u32 *pmetrics, table, rt6_flags; struct nlmsghdr *nlh; + struct rtmsg *rtm; long expires = 0; - u32 *pmetrics; - u32 table; nlh = nlmsg_put(skb, portid, seq, type, sizeof(*rtm), flags); if (!nlh) return -EMSGSIZE; + if (rt6) { + rt6_dst = &rt6->rt6i_dst; + rt6_src = &rt6->rt6i_src; + rt6_flags = rt6->rt6i_flags; + } else { + rt6_dst = &rt->fib6_dst; + rt6_src = &rt->fib6_src; + rt6_flags = rt->fib6_flags; + } + rtm = nlmsg_data(nlh); rtm->rtm_family = AF_INET6; - rtm->rtm_dst_len = rt->fib6_dst.plen; - rtm->rtm_src_len = rt->fib6_src.plen; + rtm->rtm_dst_len = rt6_dst->plen; + rtm->rtm_src_len = rt6_src->plen; rtm->rtm_tos = 0; if (rt->fib6_table) table = rt->fib6_table->tb6_id; @@ -4699,7 +4714,7 @@ static int rt6_fill_node(struct net *net, struct sk_buff *skb, rtm->rtm_scope = RT_SCOPE_UNIVERSE; rtm->rtm_protocol = rt->fib6_protocol; - if (rt->fib6_flags & RTF_CACHE) + if (rt6_flags & RTF_CACHE) rtm->rtm_flags |= RTM_F_CLONED; if (dest) { @@ -4707,7 +4722,7 @@ static int rt6_fill_node(struct net *net, struct sk_buff *skb, goto nla_put_failure; rtm->rtm_dst_len = 128; } else if (rtm->rtm_dst_len) - if (nla_put_in6_addr(skb, RTA_DST, &rt->fib6_dst.addr)) + if (nla_put_in6_addr(skb, RTA_DST, &rt6_dst->addr)) goto nla_put_failure; #ifdef CONFIG_IPV6_SUBTREES if (src) { @@ -4715,12 +4730,12 @@ static int rt6_fill_node(struct net *net, struct sk_buff *skb, goto nla_put_failure; rtm->rtm_src_len = 128; } else if (rtm->rtm_src_len && - nla_put_in6_addr(skb, RTA_SRC, &rt->fib6_src.addr)) + nla_put_in6_addr(skb, RTA_SRC, &rt6_src->addr)) goto nla_put_failure; #endif if (iif) { #ifdef CONFIG_IPV6_MROUTE - if (ipv6_addr_is_multicast(&rt->fib6_dst.addr)) { + if (ipv6_addr_is_multicast(&rt6_dst->addr)) { int err = ip6mr_get_route(net, skb, rtm, portid); if (err == 0) @@ -4755,7 +4770,14 @@ static int rt6_fill_node(struct net *net, struct sk_buff *skb, /* For multipath routes, walk the siblings list and add * each as a nexthop within RTA_MULTIPATH. */ - if (rt->fib6_nsiblings) { + if (rt6) { + if (rt6_flags & RTF_GATEWAY && + nla_put_in6_addr(skb, RTA_GATEWAY, &rt6->rt6i_gateway)) + goto nla_put_failure; + + if (dst->dev && nla_put_u32(skb, RTA_OIF, dst->dev->ifindex)) + goto nla_put_failure; + } else if (rt->fib6_nsiblings) { struct fib6_info *sibling, *next_sibling; struct nlattr *mp; @@ -4778,7 +4800,7 @@ static int rt6_fill_node(struct net *net, struct sk_buff *skb, goto nla_put_failure; } - if (rt->fib6_flags & RTF_EXPIRES) { + if (rt6_flags & RTF_EXPIRES) { expires = dst ? dst->expires : rt->expires; expires -= jiffies; } @@ -4786,7 +4808,7 @@ static int rt6_fill_node(struct net *net, struct sk_buff *skb, if (rtnl_put_cacheinfo(skb, dst, 0, expires, dst ? dst->error : 0) < 0) goto nla_put_failure; - if (nla_put_u8(skb, RTA_PREF, IPV6_EXTRACT_PREF(rt->fib6_flags))) + if (nla_put_u8(skb, RTA_PREF, IPV6_EXTRACT_PREF(rt6_flags))) goto nla_put_failure; diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 83f4c77c79d8..28c4aa5078fc 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -752,6 +752,28 @@ static void udp6_sk_rx_dst_set(struct sock *sk, struct dst_entry *dst) } } +/* wrapper for udp_queue_rcv_skb tacking care of csum conversion and + * return code conversion for ip layer consumption + */ +static int udp6_unicast_rcv_skb(struct sock *sk, struct sk_buff *skb, + struct udphdr *uh) +{ + int ret; + + if (inet_get_convert_csum(sk) && uh->check && !IS_UDPLITE(sk)) + skb_checksum_try_convert(skb, IPPROTO_UDP, uh->check, + ip6_compute_pseudo); + + ret = udpv6_queue_rcv_skb(sk, skb); + + /* a return value > 0 means to resubmit the input, but + * it wants the return to be -protocol, or 0 + */ + if (ret > 0) + return -ret; + return 0; +} + int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, int proto) { @@ -803,13 +825,14 @@ int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, if (unlikely(sk->sk_rx_dst != dst)) udp6_sk_rx_dst_set(sk, dst); - ret = udpv6_queue_rcv_skb(sk, skb); - sock_put(sk); + if (!uh->check && !udp_sk(sk)->no_check6_rx) { + sock_put(sk); + goto report_csum_error; + } - /* a return value > 0 means to resubmit the input */ - if (ret > 0) - return ret; - return 0; + ret = udp6_unicast_rcv_skb(sk, skb, uh); + sock_put(sk); + return ret; } /* @@ -822,30 +845,13 @@ int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, /* Unicast */ sk = __udp6_lib_lookup_skb(skb, uh->source, uh->dest, udptable); if (sk) { - int ret; - - if (!uh->check && !udp_sk(sk)->no_check6_rx) { - udp6_csum_zero_error(skb); - goto csum_error; - } - - if (inet_get_convert_csum(sk) && uh->check && !IS_UDPLITE(sk)) - skb_checksum_try_convert(skb, IPPROTO_UDP, uh->check, - ip6_compute_pseudo); - - ret = udpv6_queue_rcv_skb(sk, skb); - - /* a return value > 0 means to resubmit the input */ - if (ret > 0) - return ret; - - return 0; + if (!uh->check && !udp_sk(sk)->no_check6_rx) + goto report_csum_error; + return udp6_unicast_rcv_skb(sk, skb, uh); } - if (!uh->check) { - udp6_csum_zero_error(skb); - goto csum_error; - } + if (!uh->check) + goto report_csum_error; if (!xfrm6_policy_check(NULL, XFRM_POLICY_IN, skb)) goto discard; @@ -866,6 +872,9 @@ short_packet: ulen, skb->len, daddr, ntohs(uh->dest)); goto discard; + +report_csum_error: + udp6_csum_zero_error(skb); csum_error: __UDP6_INC_STATS(net, UDP_MIB_CSUMERRORS, proto == IPPROTO_UDPLITE); discard: diff --git a/net/iucv/af_iucv.c b/net/iucv/af_iucv.c index a21d8ed0a325..e2f16a0173a9 100644 --- a/net/iucv/af_iucv.c +++ b/net/iucv/af_iucv.c @@ -351,20 +351,28 @@ static int afiucv_hs_send(struct iucv_message *imsg, struct sock *sock, memcpy(&phs_hdr->iucv_hdr, imsg, sizeof(struct iucv_message)); skb->dev = iucv->hs_dev; - if (!skb->dev) - return -ENODEV; - if (!(skb->dev->flags & IFF_UP) || !netif_carrier_ok(skb->dev)) - return -ENETDOWN; + if (!skb->dev) { + err = -ENODEV; + goto err_free; + } + if (!(skb->dev->flags & IFF_UP) || !netif_carrier_ok(skb->dev)) { + err = -ENETDOWN; + goto err_free; + } if (skb->len > skb->dev->mtu) { - if (sock->sk_type == SOCK_SEQPACKET) - return -EMSGSIZE; - else - skb_trim(skb, skb->dev->mtu); + if (sock->sk_type == SOCK_SEQPACKET) { + err = -EMSGSIZE; + goto err_free; + } + skb_trim(skb, skb->dev->mtu); } skb->protocol = cpu_to_be16(ETH_P_AF_IUCV); nskb = skb_clone(skb, GFP_ATOMIC); - if (!nskb) - return -ENOMEM; + if (!nskb) { + err = -ENOMEM; + goto err_free; + } + skb_queue_tail(&iucv->send_skb_q, nskb); err = dev_queue_xmit(skb); if (net_xmit_eval(err)) { @@ -375,6 +383,10 @@ static int afiucv_hs_send(struct iucv_message *imsg, struct sock *sock, WARN_ON(atomic_read(&iucv->msg_recv) < 0); } return net_xmit_eval(err); + +err_free: + kfree_skb(skb); + return err; } static struct sock *__iucv_get_sock_by_name(char *nm) @@ -1167,7 +1179,7 @@ static int iucv_sock_sendmsg(struct socket *sock, struct msghdr *msg, err = afiucv_hs_send(&txmsg, sk, skb, 0); if (err) { atomic_dec(&iucv->msg_sent); - goto fail; + goto out; } } else { /* Classic VM IUCV transport */ skb_queue_tail(&iucv->send_skb_q, skb); @@ -2155,8 +2167,8 @@ static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev, struct sock *sk; struct iucv_sock *iucv; struct af_iucv_trans_hdr *trans_hdr; + int err = NET_RX_SUCCESS; char nullstring[8]; - int err = 0; if (skb->len < (ETH_HLEN + sizeof(struct af_iucv_trans_hdr))) { WARN_ONCE(1, "AF_IUCV too short skb, len=%d, min=%d", @@ -2254,7 +2266,7 @@ static int afiucv_hs_rcv(struct sk_buff *skb, struct net_device *dev, err = afiucv_hs_callback_rx(sk, skb); break; default: - ; + kfree_skb(skb); } return err; diff --git a/net/iucv/iucv.c b/net/iucv/iucv.c index 8f7ef167c45a..eb502c6290c2 100644 --- a/net/iucv/iucv.c +++ b/net/iucv/iucv.c @@ -1874,7 +1874,7 @@ static void iucv_pm_complete(struct device *dev) * Returns 0 if there are still iucv pathes defined * 1 if there are no iucv pathes defined */ -int iucv_path_table_empty(void) +static int iucv_path_table_empty(void) { int i; diff --git a/net/mac80211/ibss.c b/net/mac80211/ibss.c index 6449a1c2283b..f0f5fedb8caa 100644 --- a/net/mac80211/ibss.c +++ b/net/mac80211/ibss.c @@ -947,8 +947,8 @@ static void ieee80211_rx_mgmt_deauth_ibss(struct ieee80211_sub_if_data *sdata, if (len < IEEE80211_DEAUTH_FRAME_LEN) return; - ibss_dbg(sdata, "RX DeAuth SA=%pM DA=%pM BSSID=%pM (reason: %d)\n", - mgmt->sa, mgmt->da, mgmt->bssid, reason); + ibss_dbg(sdata, "RX DeAuth SA=%pM DA=%pM\n", mgmt->sa, mgmt->da); + ibss_dbg(sdata, "\tBSSID=%pM (reason: %d)\n", mgmt->bssid, reason); sta_info_destroy_addr(sdata, mgmt->sa); } @@ -966,9 +966,9 @@ static void ieee80211_rx_mgmt_auth_ibss(struct ieee80211_sub_if_data *sdata, auth_alg = le16_to_cpu(mgmt->u.auth.auth_alg); auth_transaction = le16_to_cpu(mgmt->u.auth.auth_transaction); - ibss_dbg(sdata, - "RX Auth SA=%pM DA=%pM BSSID=%pM (auth_transaction=%d)\n", - mgmt->sa, mgmt->da, mgmt->bssid, auth_transaction); + ibss_dbg(sdata, "RX Auth SA=%pM DA=%pM\n", mgmt->sa, mgmt->da); + ibss_dbg(sdata, "\tBSSID=%pM (auth_transaction=%d)\n", + mgmt->bssid, auth_transaction); if (auth_alg != WLAN_AUTH_OPEN || auth_transaction != 1) return; @@ -1175,10 +1175,10 @@ static void ieee80211_rx_bss_info(struct ieee80211_sub_if_data *sdata, rx_timestamp = drv_get_tsf(local, sdata); } - ibss_dbg(sdata, - "RX beacon SA=%pM BSSID=%pM TSF=0x%llx BCN=0x%llx diff=%lld @%lu\n", + ibss_dbg(sdata, "RX beacon SA=%pM BSSID=%pM TSF=0x%llx\n", mgmt->sa, mgmt->bssid, - (unsigned long long)rx_timestamp, + (unsigned long long)rx_timestamp); + ibss_dbg(sdata, "\tBCN=0x%llx diff=%lld @%lu\n", (unsigned long long)beacon_timestamp, (unsigned long long)(rx_timestamp - beacon_timestamp), jiffies); @@ -1537,9 +1537,9 @@ static void ieee80211_rx_mgmt_probe_req(struct ieee80211_sub_if_data *sdata, tx_last_beacon = drv_tx_last_beacon(local); - ibss_dbg(sdata, - "RX ProbeReq SA=%pM DA=%pM BSSID=%pM (tx_last_beacon=%d)\n", - mgmt->sa, mgmt->da, mgmt->bssid, tx_last_beacon); + ibss_dbg(sdata, "RX ProbeReq SA=%pM DA=%pM\n", mgmt->sa, mgmt->da); + ibss_dbg(sdata, "\tBSSID=%pM (tx_last_beacon=%d)\n", + mgmt->bssid, tx_last_beacon); if (!tx_last_beacon && is_multicast_ether_addr(mgmt->da)) return; diff --git a/net/mac80211/main.c b/net/mac80211/main.c index 4fb2709cb527..513627896204 100644 --- a/net/mac80211/main.c +++ b/net/mac80211/main.c @@ -256,8 +256,27 @@ static void ieee80211_restart_work(struct work_struct *work) flush_work(&local->radar_detected_work); rtnl_lock(); - list_for_each_entry(sdata, &local->interfaces, list) + list_for_each_entry(sdata, &local->interfaces, list) { + /* + * XXX: there may be more work for other vif types and even + * for station mode: a good thing would be to run most of + * the iface type's dependent _stop (ieee80211_mg_stop, + * ieee80211_ibss_stop) etc... + * For now, fix only the specific bug that was seen: race + * between csa_connection_drop_work and us. + */ + if (sdata->vif.type == NL80211_IFTYPE_STATION) { + /* + * This worker is scheduled from the iface worker that + * runs on mac80211's workqueue, so we can't be + * scheduling this worker after the cancel right here. + * The exception is ieee80211_chswitch_done. + * Then we can have a race... + */ + cancel_work_sync(&sdata->u.mgd.csa_connection_drop_work); + } flush_delayed_work(&sdata->dec_tailroom_needed_wk); + } ieee80211_scan_cancel(local); /* make sure any new ROC will consider local->in_reconfig */ @@ -471,10 +490,7 @@ static const struct ieee80211_vht_cap mac80211_vht_capa_mod_mask = { cpu_to_le32(IEEE80211_VHT_CAP_RXLDPC | IEEE80211_VHT_CAP_SHORT_GI_80 | IEEE80211_VHT_CAP_SHORT_GI_160 | - IEEE80211_VHT_CAP_RXSTBC_1 | - IEEE80211_VHT_CAP_RXSTBC_2 | - IEEE80211_VHT_CAP_RXSTBC_3 | - IEEE80211_VHT_CAP_RXSTBC_4 | + IEEE80211_VHT_CAP_RXSTBC_MASK | IEEE80211_VHT_CAP_TXSTBC | IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE | IEEE80211_VHT_CAP_SU_BEAMFORMEE_CAPABLE | @@ -1208,6 +1224,7 @@ void ieee80211_unregister_hw(struct ieee80211_hw *hw) #if IS_ENABLED(CONFIG_IPV6) unregister_inet6addr_notifier(&local->ifa6_notifier); #endif + ieee80211_txq_teardown_flows(local); rtnl_lock(); @@ -1236,7 +1253,6 @@ void ieee80211_unregister_hw(struct ieee80211_hw *hw) skb_queue_purge(&local->skb_queue); skb_queue_purge(&local->skb_queue_unreliable); skb_queue_purge(&local->skb_queue_tdls_chsw); - ieee80211_txq_teardown_flows(local); destroy_workqueue(local->workqueue); wiphy_unregister(local->hw.wiphy); diff --git a/net/mac80211/mesh_hwmp.c b/net/mac80211/mesh_hwmp.c index 35ad3983ae4b..daf9db3c8f24 100644 --- a/net/mac80211/mesh_hwmp.c +++ b/net/mac80211/mesh_hwmp.c @@ -572,6 +572,10 @@ static void hwmp_preq_frame_process(struct ieee80211_sub_if_data *sdata, forward = false; reply = true; target_metric = 0; + + if (SN_GT(target_sn, ifmsh->sn)) + ifmsh->sn = target_sn; + if (time_after(jiffies, ifmsh->last_sn_update + net_traversal_jiffies(sdata)) || time_before(jiffies, ifmsh->last_sn_update)) { diff --git a/net/mac80211/mlme.c b/net/mac80211/mlme.c index 7fb9957359a3..3dbecae4be73 100644 --- a/net/mac80211/mlme.c +++ b/net/mac80211/mlme.c @@ -1073,6 +1073,10 @@ static void ieee80211_chswitch_work(struct work_struct *work) */ if (sdata->reserved_chanctx) { + struct ieee80211_supported_band *sband = NULL; + struct sta_info *mgd_sta = NULL; + enum ieee80211_sta_rx_bandwidth bw = IEEE80211_STA_RX_BW_20; + /* * with multi-vif csa driver may call ieee80211_csa_finish() * many times while waiting for other interfaces to use their @@ -1081,6 +1085,48 @@ static void ieee80211_chswitch_work(struct work_struct *work) if (sdata->reserved_ready) goto out; + if (sdata->vif.bss_conf.chandef.width != + sdata->csa_chandef.width) { + /* + * For managed interface, we need to also update the AP + * station bandwidth and align the rate scale algorithm + * on the bandwidth change. Here we only consider the + * bandwidth of the new channel definition (as channel + * switch flow does not have the full HT/VHT/HE + * information), assuming that if additional changes are + * required they would be done as part of the processing + * of the next beacon from the AP. + */ + switch (sdata->csa_chandef.width) { + case NL80211_CHAN_WIDTH_20_NOHT: + case NL80211_CHAN_WIDTH_20: + default: + bw = IEEE80211_STA_RX_BW_20; + break; + case NL80211_CHAN_WIDTH_40: + bw = IEEE80211_STA_RX_BW_40; + break; + case NL80211_CHAN_WIDTH_80: + bw = IEEE80211_STA_RX_BW_80; + break; + case NL80211_CHAN_WIDTH_80P80: + case NL80211_CHAN_WIDTH_160: + bw = IEEE80211_STA_RX_BW_160; + break; + } + + mgd_sta = sta_info_get(sdata, ifmgd->bssid); + sband = + local->hw.wiphy->bands[sdata->csa_chandef.chan->band]; + } + + if (sdata->vif.bss_conf.chandef.width > + sdata->csa_chandef.width) { + mgd_sta->sta.bandwidth = bw; + rate_control_rate_update(local, sband, mgd_sta, + IEEE80211_RC_BW_CHANGED); + } + ret = ieee80211_vif_use_reserved_context(sdata); if (ret) { sdata_info(sdata, @@ -1091,6 +1137,13 @@ static void ieee80211_chswitch_work(struct work_struct *work) goto out; } + if (sdata->vif.bss_conf.chandef.width < + sdata->csa_chandef.width) { + mgd_sta->sta.bandwidth = bw; + rate_control_rate_update(local, sband, mgd_sta, + IEEE80211_RC_BW_CHANGED); + } + goto out; } @@ -1312,6 +1365,16 @@ ieee80211_sta_process_chanswitch(struct ieee80211_sub_if_data *sdata, cbss->beacon_interval)); return; drop_connection: + /* + * This is just so that the disconnect flow will know that + * we were trying to switch channel and failed. In case the + * mode is 1 (we are not allowed to Tx), we will know not to + * send a deauthentication frame. Those two fields will be + * reset when the disconnection worker runs. + */ + sdata->vif.csa_active = true; + sdata->csa_block_tx = csa_ie.mode; + ieee80211_queue_work(&local->hw, &ifmgd->csa_connection_drop_work); mutex_unlock(&local->chanctx_mtx); mutex_unlock(&local->mtx); @@ -2522,6 +2585,7 @@ static void __ieee80211_disconnect(struct ieee80211_sub_if_data *sdata) struct ieee80211_local *local = sdata->local; struct ieee80211_if_managed *ifmgd = &sdata->u.mgd; u8 frame_buf[IEEE80211_DEAUTH_FRAME_LEN]; + bool tx; sdata_lock(sdata); if (!ifmgd->associated) { @@ -2529,6 +2593,8 @@ static void __ieee80211_disconnect(struct ieee80211_sub_if_data *sdata) return; } + tx = !sdata->csa_block_tx; + /* AP is probably out of range (or not reachable for another reason) so * remove the bss struct for that AP. */ @@ -2536,7 +2602,7 @@ static void __ieee80211_disconnect(struct ieee80211_sub_if_data *sdata) ieee80211_set_disassoc(sdata, IEEE80211_STYPE_DEAUTH, WLAN_REASON_DISASSOC_DUE_TO_INACTIVITY, - true, frame_buf); + tx, frame_buf); mutex_lock(&local->mtx); sdata->vif.csa_active = false; ifmgd->csa_waiting_bcn = false; @@ -2547,7 +2613,7 @@ static void __ieee80211_disconnect(struct ieee80211_sub_if_data *sdata) } mutex_unlock(&local->mtx); - ieee80211_report_disconnect(sdata, frame_buf, sizeof(frame_buf), true, + ieee80211_report_disconnect(sdata, frame_buf, sizeof(frame_buf), tx, WLAN_REASON_DISASSOC_DUE_TO_INACTIVITY); sdata_unlock(sdata); diff --git a/net/mac80211/rx.c b/net/mac80211/rx.c index 64742f2765c4..96611d5dfadb 100644 --- a/net/mac80211/rx.c +++ b/net/mac80211/rx.c @@ -1728,6 +1728,7 @@ ieee80211_rx_h_sta_process(struct ieee80211_rx_data *rx) */ if (!ieee80211_hw_check(&sta->local->hw, AP_LINK_PS) && !ieee80211_has_morefrags(hdr->frame_control) && + !is_multicast_ether_addr(hdr->addr1) && (ieee80211_is_mgmt(hdr->frame_control) || ieee80211_is_data(hdr->frame_control)) && !(status->rx_flags & IEEE80211_RX_DEFERRED_RELEASE) && diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c index cd332e3e1134..f353d9db54bc 100644 --- a/net/mac80211/tx.c +++ b/net/mac80211/tx.c @@ -3078,27 +3078,18 @@ void ieee80211_clear_fast_xmit(struct sta_info *sta) } static bool ieee80211_amsdu_realloc_pad(struct ieee80211_local *local, - struct sk_buff *skb, int headroom, - int *subframe_len) + struct sk_buff *skb, int headroom) { - int amsdu_len = *subframe_len + sizeof(struct ethhdr); - int padding = (4 - amsdu_len) & 3; - - if (skb_headroom(skb) < headroom || skb_tailroom(skb) < padding) { + if (skb_headroom(skb) < headroom) { I802_DEBUG_INC(local->tx_expand_skb_head); - if (pskb_expand_head(skb, headroom, padding, GFP_ATOMIC)) { + if (pskb_expand_head(skb, headroom, 0, GFP_ATOMIC)) { wiphy_debug(local->hw.wiphy, "failed to reallocate TX buffer\n"); return false; } } - if (padding) { - *subframe_len += padding; - skb_put_zero(skb, padding); - } - return true; } @@ -3122,8 +3113,7 @@ static bool ieee80211_amsdu_prepare_head(struct ieee80211_sub_if_data *sdata, if (info->control.flags & IEEE80211_TX_CTRL_AMSDU) return true; - if (!ieee80211_amsdu_realloc_pad(local, skb, sizeof(*amsdu_hdr), - &subframe_len)) + if (!ieee80211_amsdu_realloc_pad(local, skb, sizeof(*amsdu_hdr))) return false; data = skb_push(skb, sizeof(*amsdu_hdr)); @@ -3189,7 +3179,8 @@ static bool ieee80211_amsdu_aggregate(struct ieee80211_sub_if_data *sdata, void *data; bool ret = false; unsigned int orig_len; - int n = 1, nfrags; + int n = 2, nfrags, pad = 0; + u16 hdrlen; if (!ieee80211_hw_check(&local->hw, TX_AMSDU)) return false; @@ -3222,9 +3213,6 @@ static bool ieee80211_amsdu_aggregate(struct ieee80211_sub_if_data *sdata, if (skb->len + head->len > max_amsdu_len) goto out; - if (!ieee80211_amsdu_prepare_head(sdata, fast_tx, head)) - goto out; - nfrags = 1 + skb_shinfo(skb)->nr_frags; nfrags += 1 + skb_shinfo(head)->nr_frags; frag_tail = &skb_shinfo(head)->frag_list; @@ -3240,10 +3228,24 @@ static bool ieee80211_amsdu_aggregate(struct ieee80211_sub_if_data *sdata, if (max_frags && nfrags > max_frags) goto out; - if (!ieee80211_amsdu_realloc_pad(local, skb, sizeof(rfc1042_header) + 2, - &subframe_len)) + if (!ieee80211_amsdu_prepare_head(sdata, fast_tx, head)) goto out; + /* + * Pad out the previous subframe to a multiple of 4 by adding the + * padding to the next one, that's being added. Note that head->len + * is the length of the full A-MSDU, but that works since each time + * we add a new subframe we pad out the previous one to a multiple + * of 4 and thus it no longer matters in the next round. + */ + hdrlen = fast_tx->hdr_len - sizeof(rfc1042_header); + if ((head->len - hdrlen) & 3) + pad = 4 - ((head->len - hdrlen) & 3); + + if (!ieee80211_amsdu_realloc_pad(local, skb, sizeof(rfc1042_header) + + 2 + pad)) + goto out_recalc; + ret = true; data = skb_push(skb, ETH_ALEN + 2); memmove(data, data + ETH_ALEN + 2, 2 * ETH_ALEN); @@ -3253,15 +3255,19 @@ static bool ieee80211_amsdu_aggregate(struct ieee80211_sub_if_data *sdata, memcpy(data, &len, 2); memcpy(data + 2, rfc1042_header, sizeof(rfc1042_header)); + memset(skb_push(skb, pad), 0, pad); + head->len += skb->len; head->data_len += skb->len; *frag_tail = skb; - flow->backlog += head->len - orig_len; - tin->backlog_bytes += head->len - orig_len; - - fq_recalc_backlog(fq, tin, flow); +out_recalc: + if (head->len != orig_len) { + flow->backlog += head->len - orig_len; + tin->backlog_bytes += head->len - orig_len; + fq_recalc_backlog(fq, tin, flow); + } out: spin_unlock_bh(&fq->lock); diff --git a/net/mac80211/util.c b/net/mac80211/util.c index 88efda7c9f8a..716cd6442d86 100644 --- a/net/mac80211/util.c +++ b/net/mac80211/util.c @@ -1135,7 +1135,7 @@ void ieee80211_regulatory_limit_wmm_params(struct ieee80211_sub_if_data *sdata, { struct ieee80211_chanctx_conf *chanctx_conf; const struct ieee80211_reg_rule *rrule; - struct ieee80211_wmm_ac *wmm_ac; + const struct ieee80211_wmm_ac *wmm_ac; u16 center_freq = 0; if (sdata->vif.type != NL80211_IFTYPE_AP && @@ -1154,20 +1154,19 @@ void ieee80211_regulatory_limit_wmm_params(struct ieee80211_sub_if_data *sdata, rrule = freq_reg_info(sdata->wdev.wiphy, MHZ_TO_KHZ(center_freq)); - if (IS_ERR_OR_NULL(rrule) || !rrule->wmm_rule) { + if (IS_ERR_OR_NULL(rrule) || !rrule->has_wmm) { rcu_read_unlock(); return; } if (sdata->vif.type == NL80211_IFTYPE_AP) - wmm_ac = &rrule->wmm_rule->ap[ac]; + wmm_ac = &rrule->wmm_rule.ap[ac]; else - wmm_ac = &rrule->wmm_rule->client[ac]; + wmm_ac = &rrule->wmm_rule.client[ac]; qparam->cw_min = max_t(u16, qparam->cw_min, wmm_ac->cw_min); qparam->cw_max = max_t(u16, qparam->cw_max, wmm_ac->cw_max); qparam->aifs = max_t(u8, qparam->aifs, wmm_ac->aifsn); - qparam->txop = !qparam->txop ? wmm_ac->cot / 32 : - min_t(u16, qparam->txop, wmm_ac->cot / 32); + qparam->txop = min_t(u16, qparam->txop, wmm_ac->cot / 32); rcu_read_unlock(); } diff --git a/net/mpls/af_mpls.c b/net/mpls/af_mpls.c index 7a4de6d618b1..8fbe6cdbe255 100644 --- a/net/mpls/af_mpls.c +++ b/net/mpls/af_mpls.c @@ -1533,10 +1533,14 @@ static int mpls_dev_notify(struct notifier_block *this, unsigned long event, unsigned int flags; if (event == NETDEV_REGISTER) { - /* For now just support Ethernet, IPGRE, SIT and IPIP devices */ + + /* For now just support Ethernet, IPGRE, IP6GRE, SIT and + * IPIP devices + */ if (dev->type == ARPHRD_ETHER || dev->type == ARPHRD_LOOPBACK || dev->type == ARPHRD_IPGRE || + dev->type == ARPHRD_IP6GRE || dev->type == ARPHRD_SIT || dev->type == ARPHRD_TUNNEL) { mdev = mpls_add_dev(dev); diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig index 71709c104081..f61c306de1d0 100644 --- a/net/netfilter/Kconfig +++ b/net/netfilter/Kconfig @@ -771,13 +771,13 @@ config NETFILTER_XT_TARGET_CHECKSUM depends on NETFILTER_ADVANCED ---help--- This option adds a `CHECKSUM' target, which can be used in the iptables mangle - table. + table to work around buggy DHCP clients in virtualized environments. - You can use this target to compute and fill in the checksum in - a packet that lacks a checksum. This is particularly useful, - if you need to work around old applications such as dhcp clients, - that do not work well with checksum offloads, but don't want to disable - checksum offload in your device. + Some old DHCP clients drop packets because they are not aware + that the checksum would normally be offloaded to hardware and + thus should be considered valid. + This target can be used to fill in the checksum using iptables + when such packets are sent via a virtual network device. To compile it as a module, choose M here. If unsure, say N. diff --git a/net/netfilter/nf_conntrack_proto.c b/net/netfilter/nf_conntrack_proto.c index 9f14b0df6960..51c5d7eec0a3 100644 --- a/net/netfilter/nf_conntrack_proto.c +++ b/net/netfilter/nf_conntrack_proto.c @@ -776,9 +776,26 @@ static const struct nf_hook_ops ipv6_conntrack_ops[] = { }; #endif +static int nf_ct_tcp_fixup(struct nf_conn *ct, void *_nfproto) +{ + u8 nfproto = (unsigned long)_nfproto; + + if (nf_ct_l3num(ct) != nfproto) + return 0; + + if (nf_ct_protonum(ct) == IPPROTO_TCP && + ct->proto.tcp.state == TCP_CONNTRACK_ESTABLISHED) { + ct->proto.tcp.seen[0].td_maxwin = 0; + ct->proto.tcp.seen[1].td_maxwin = 0; + } + + return 0; +} + static int nf_ct_netns_do_get(struct net *net, u8 nfproto) { struct nf_conntrack_net *cnet = net_generic(net, nf_conntrack_net_id); + bool fixup_needed = false; int err = 0; mutex_lock(&nf_ct_proto_mutex); @@ -798,6 +815,8 @@ static int nf_ct_netns_do_get(struct net *net, u8 nfproto) ARRAY_SIZE(ipv4_conntrack_ops)); if (err) cnet->users4 = 0; + else + fixup_needed = true; break; #if IS_ENABLED(CONFIG_IPV6) case NFPROTO_IPV6: @@ -814,6 +833,8 @@ static int nf_ct_netns_do_get(struct net *net, u8 nfproto) ARRAY_SIZE(ipv6_conntrack_ops)); if (err) cnet->users6 = 0; + else + fixup_needed = true; break; #endif default: @@ -822,6 +843,11 @@ static int nf_ct_netns_do_get(struct net *net, u8 nfproto) } out_unlock: mutex_unlock(&nf_ct_proto_mutex); + + if (fixup_needed) + nf_ct_iterate_cleanup_net(net, nf_ct_tcp_fixup, + (void *)(unsigned long)nfproto, 0, 0); + return err; } diff --git a/net/netfilter/nf_conntrack_proto_dccp.c b/net/netfilter/nf_conntrack_proto_dccp.c index 8c58f96b59e7..f3f91ed2c21a 100644 --- a/net/netfilter/nf_conntrack_proto_dccp.c +++ b/net/netfilter/nf_conntrack_proto_dccp.c @@ -675,7 +675,7 @@ static int nlattr_to_dccp(struct nlattr *cda[], struct nf_conn *ct) } #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -697,6 +697,8 @@ static int dccp_timeout_nlattr_to_obj(struct nlattr *tb[], timeouts[i] = ntohl(nla_get_be32(tb[i])) * HZ; } } + + timeouts[CTA_TIMEOUT_DCCP_UNSPEC] = timeouts[CTA_TIMEOUT_DCCP_REQUEST]; return 0; } @@ -726,7 +728,7 @@ dccp_timeout_nla_policy[CTA_TIMEOUT_DCCP_MAX+1] = { [CTA_TIMEOUT_DCCP_CLOSING] = { .type = NLA_U32 }, [CTA_TIMEOUT_DCCP_TIMEWAIT] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ #ifdef CONFIG_SYSCTL /* template, data assigned later */ @@ -827,6 +829,11 @@ static int dccp_init_net(struct net *net, u_int16_t proto) dn->dccp_timeout[CT_DCCP_CLOSEREQ] = 64 * HZ; dn->dccp_timeout[CT_DCCP_CLOSING] = 64 * HZ; dn->dccp_timeout[CT_DCCP_TIMEWAIT] = 2 * DCCP_MSL; + + /* timeouts[0] is unused, make it same as SYN_SENT so + * ->timeouts[0] contains 'new' timeout, like udp or icmp. + */ + dn->dccp_timeout[CT_DCCP_NONE] = dn->dccp_timeout[CT_DCCP_REQUEST]; } return dccp_kmemdup_sysctl_table(net, pn, dn); @@ -856,7 +863,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_dccp4 = { .nlattr_to_tuple = nf_ct_port_nlattr_to_tuple, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = dccp_timeout_nlattr_to_obj, .obj_to_nlattr = dccp_timeout_obj_to_nlattr, @@ -864,7 +871,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_dccp4 = { .obj_size = sizeof(unsigned int) * CT_DCCP_MAX, .nla_policy = dccp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = dccp_init_net, .get_net_proto = dccp_get_net_proto, }; @@ -889,7 +896,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_dccp6 = { .nlattr_to_tuple = nf_ct_port_nlattr_to_tuple, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = dccp_timeout_nlattr_to_obj, .obj_to_nlattr = dccp_timeout_obj_to_nlattr, @@ -897,7 +904,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_dccp6 = { .obj_size = sizeof(unsigned int) * CT_DCCP_MAX, .nla_policy = dccp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = dccp_init_net, .get_net_proto = dccp_get_net_proto, }; diff --git a/net/netfilter/nf_conntrack_proto_generic.c b/net/netfilter/nf_conntrack_proto_generic.c index ac4a0b296dcd..1df3244ecd07 100644 --- a/net/netfilter/nf_conntrack_proto_generic.c +++ b/net/netfilter/nf_conntrack_proto_generic.c @@ -70,7 +70,7 @@ static bool generic_new(struct nf_conn *ct, const struct sk_buff *skb, return ret; } -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -113,7 +113,7 @@ static const struct nla_policy generic_timeout_nla_policy[CTA_TIMEOUT_GENERIC_MAX+1] = { [CTA_TIMEOUT_GENERIC_TIMEOUT] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ #ifdef CONFIG_SYSCTL static struct ctl_table generic_sysctl_table[] = { @@ -164,7 +164,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_generic = .pkt_to_tuple = generic_pkt_to_tuple, .packet = generic_packet, .new = generic_new, -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = generic_timeout_nlattr_to_obj, .obj_to_nlattr = generic_timeout_obj_to_nlattr, @@ -172,7 +172,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_generic = .obj_size = sizeof(unsigned int), .nla_policy = generic_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = generic_init_net, .get_net_proto = generic_get_net_proto, }; diff --git a/net/netfilter/nf_conntrack_proto_gre.c b/net/netfilter/nf_conntrack_proto_gre.c index d1632252bf5b..650eb4fba2c5 100644 --- a/net/netfilter/nf_conntrack_proto_gre.c +++ b/net/netfilter/nf_conntrack_proto_gre.c @@ -285,7 +285,7 @@ static void gre_destroy(struct nf_conn *ct) nf_ct_gre_keymap_destroy(master); } -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -334,7 +334,7 @@ gre_timeout_nla_policy[CTA_TIMEOUT_GRE_MAX+1] = { [CTA_TIMEOUT_GRE_UNREPLIED] = { .type = NLA_U32 }, [CTA_TIMEOUT_GRE_REPLIED] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ static int gre_init_net(struct net *net, u_int16_t proto) { @@ -367,7 +367,7 @@ static const struct nf_conntrack_l4proto nf_conntrack_l4proto_gre4 = { .nlattr_to_tuple = nf_ct_port_nlattr_to_tuple, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = gre_timeout_nlattr_to_obj, .obj_to_nlattr = gre_timeout_obj_to_nlattr, @@ -375,7 +375,7 @@ static const struct nf_conntrack_l4proto nf_conntrack_l4proto_gre4 = { .obj_size = sizeof(unsigned int) * GRE_CT_MAX, .nla_policy = gre_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .net_id = &proto_gre_net_id, .init_net = gre_init_net, }; diff --git a/net/netfilter/nf_conntrack_proto_icmp.c b/net/netfilter/nf_conntrack_proto_icmp.c index 036670b38282..43c7e1a217b9 100644 --- a/net/netfilter/nf_conntrack_proto_icmp.c +++ b/net/netfilter/nf_conntrack_proto_icmp.c @@ -273,7 +273,7 @@ static unsigned int icmp_nlattr_tuple_size(void) } #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -313,7 +313,7 @@ static const struct nla_policy icmp_timeout_nla_policy[CTA_TIMEOUT_ICMP_MAX+1] = { [CTA_TIMEOUT_ICMP_TIMEOUT] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ #ifdef CONFIG_SYSCTL static struct ctl_table icmp_sysctl_table[] = { @@ -374,7 +374,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_icmp = .nlattr_to_tuple = icmp_nlattr_to_tuple, .nla_policy = icmp_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = icmp_timeout_nlattr_to_obj, .obj_to_nlattr = icmp_timeout_obj_to_nlattr, @@ -382,7 +382,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_icmp = .obj_size = sizeof(unsigned int), .nla_policy = icmp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = icmp_init_net, .get_net_proto = icmp_get_net_proto, }; diff --git a/net/netfilter/nf_conntrack_proto_icmpv6.c b/net/netfilter/nf_conntrack_proto_icmpv6.c index bed07b998a10..97e40f77d678 100644 --- a/net/netfilter/nf_conntrack_proto_icmpv6.c +++ b/net/netfilter/nf_conntrack_proto_icmpv6.c @@ -274,7 +274,7 @@ static unsigned int icmpv6_nlattr_tuple_size(void) } #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -314,7 +314,7 @@ static const struct nla_policy icmpv6_timeout_nla_policy[CTA_TIMEOUT_ICMPV6_MAX+1] = { [CTA_TIMEOUT_ICMPV6_TIMEOUT] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ #ifdef CONFIG_SYSCTL static struct ctl_table icmpv6_sysctl_table[] = { @@ -373,7 +373,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_icmpv6 = .nlattr_to_tuple = icmpv6_nlattr_to_tuple, .nla_policy = icmpv6_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = icmpv6_timeout_nlattr_to_obj, .obj_to_nlattr = icmpv6_timeout_obj_to_nlattr, @@ -381,7 +381,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_icmpv6 = .obj_size = sizeof(unsigned int), .nla_policy = icmpv6_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = icmpv6_init_net, .get_net_proto = icmpv6_get_net_proto, }; diff --git a/net/netfilter/nf_conntrack_proto_sctp.c b/net/netfilter/nf_conntrack_proto_sctp.c index 8d1e085fc14a..e4d738d34cd0 100644 --- a/net/netfilter/nf_conntrack_proto_sctp.c +++ b/net/netfilter/nf_conntrack_proto_sctp.c @@ -591,7 +591,7 @@ static int nlattr_to_sctp(struct nlattr *cda[], struct nf_conn *ct) } #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -613,6 +613,8 @@ static int sctp_timeout_nlattr_to_obj(struct nlattr *tb[], timeouts[i] = ntohl(nla_get_be32(tb[i])) * HZ; } } + + timeouts[CTA_TIMEOUT_SCTP_UNSPEC] = timeouts[CTA_TIMEOUT_SCTP_CLOSED]; return 0; } @@ -644,7 +646,7 @@ sctp_timeout_nla_policy[CTA_TIMEOUT_SCTP_MAX+1] = { [CTA_TIMEOUT_SCTP_HEARTBEAT_SENT] = { .type = NLA_U32 }, [CTA_TIMEOUT_SCTP_HEARTBEAT_ACKED] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ #ifdef CONFIG_SYSCTL @@ -743,6 +745,11 @@ static int sctp_init_net(struct net *net, u_int16_t proto) for (i = 0; i < SCTP_CONNTRACK_MAX; i++) sn->timeouts[i] = sctp_timeouts[i]; + + /* timeouts[0] is unused, init it so ->timeouts[0] contains + * 'new' timeout, like udp or icmp. + */ + sn->timeouts[0] = sctp_timeouts[SCTP_CONNTRACK_CLOSED]; } return sctp_kmemdup_sysctl_table(pn, sn); @@ -773,7 +780,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_sctp4 = { .nlattr_to_tuple = nf_ct_port_nlattr_to_tuple, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = sctp_timeout_nlattr_to_obj, .obj_to_nlattr = sctp_timeout_obj_to_nlattr, @@ -781,7 +788,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_sctp4 = { .obj_size = sizeof(unsigned int) * SCTP_CONNTRACK_MAX, .nla_policy = sctp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = sctp_init_net, .get_net_proto = sctp_get_net_proto, }; @@ -806,7 +813,8 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_sctp6 = { .nlattr_tuple_size = nf_ct_port_nlattr_tuple_size, .nlattr_to_tuple = nf_ct_port_nlattr_to_tuple, .nla_policy = nf_ct_port_nla_policy, -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#endif +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = sctp_timeout_nlattr_to_obj, .obj_to_nlattr = sctp_timeout_obj_to_nlattr, @@ -814,8 +822,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_sctp6 = { .obj_size = sizeof(unsigned int) * SCTP_CONNTRACK_MAX, .nla_policy = sctp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ -#endif +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = sctp_init_net, .get_net_proto = sctp_get_net_proto, }; diff --git a/net/netfilter/nf_conntrack_proto_tcp.c b/net/netfilter/nf_conntrack_proto_tcp.c index d80d322b9d8b..b4bdf9eda7b7 100644 --- a/net/netfilter/nf_conntrack_proto_tcp.c +++ b/net/netfilter/nf_conntrack_proto_tcp.c @@ -1279,7 +1279,7 @@ static unsigned int tcp_nlattr_tuple_size(void) } #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -1301,6 +1301,7 @@ static int tcp_timeout_nlattr_to_obj(struct nlattr *tb[], timeouts[TCP_CONNTRACK_SYN_SENT] = ntohl(nla_get_be32(tb[CTA_TIMEOUT_TCP_SYN_SENT]))*HZ; } + if (tb[CTA_TIMEOUT_TCP_SYN_RECV]) { timeouts[TCP_CONNTRACK_SYN_RECV] = ntohl(nla_get_be32(tb[CTA_TIMEOUT_TCP_SYN_RECV]))*HZ; @@ -1341,6 +1342,8 @@ static int tcp_timeout_nlattr_to_obj(struct nlattr *tb[], timeouts[TCP_CONNTRACK_UNACK] = ntohl(nla_get_be32(tb[CTA_TIMEOUT_TCP_UNACK]))*HZ; } + + timeouts[CTA_TIMEOUT_TCP_UNSPEC] = timeouts[CTA_TIMEOUT_TCP_SYN_SENT]; return 0; } @@ -1391,7 +1394,7 @@ static const struct nla_policy tcp_timeout_nla_policy[CTA_TIMEOUT_TCP_MAX+1] = { [CTA_TIMEOUT_TCP_RETRANS] = { .type = NLA_U32 }, [CTA_TIMEOUT_TCP_UNACK] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ #ifdef CONFIG_SYSCTL static struct ctl_table tcp_sysctl_table[] = { @@ -1518,6 +1521,10 @@ static int tcp_init_net(struct net *net, u_int16_t proto) for (i = 0; i < TCP_CONNTRACK_TIMEOUT_MAX; i++) tn->timeouts[i] = tcp_timeouts[i]; + /* timeouts[0] is unused, make it same as SYN_SENT so + * ->timeouts[0] contains 'new' timeout, like udp or icmp. + */ + tn->timeouts[0] = tcp_timeouts[TCP_CONNTRACK_SYN_SENT]; tn->tcp_loose = nf_ct_tcp_loose; tn->tcp_be_liberal = nf_ct_tcp_be_liberal; tn->tcp_max_retrans = nf_ct_tcp_max_retrans; @@ -1551,7 +1558,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_tcp4 = .nlattr_size = TCP_NLATTR_SIZE, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = tcp_timeout_nlattr_to_obj, .obj_to_nlattr = tcp_timeout_obj_to_nlattr, @@ -1560,7 +1567,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_tcp4 = TCP_CONNTRACK_TIMEOUT_MAX, .nla_policy = tcp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = tcp_init_net, .get_net_proto = tcp_get_net_proto, }; @@ -1586,7 +1593,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_tcp6 = .nlattr_tuple_size = tcp_nlattr_tuple_size, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = tcp_timeout_nlattr_to_obj, .obj_to_nlattr = tcp_timeout_obj_to_nlattr, @@ -1595,7 +1602,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_tcp6 = TCP_CONNTRACK_TIMEOUT_MAX, .nla_policy = tcp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = tcp_init_net, .get_net_proto = tcp_get_net_proto, }; diff --git a/net/netfilter/nf_conntrack_proto_udp.c b/net/netfilter/nf_conntrack_proto_udp.c index 7a1b8988a931..3065fb8ef91b 100644 --- a/net/netfilter/nf_conntrack_proto_udp.c +++ b/net/netfilter/nf_conntrack_proto_udp.c @@ -171,7 +171,7 @@ static int udp_error(struct net *net, struct nf_conn *tmpl, struct sk_buff *skb, return NF_ACCEPT; } -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT #include <linux/netfilter/nfnetlink.h> #include <linux/netfilter/nfnetlink_cttimeout.h> @@ -221,7 +221,7 @@ udp_timeout_nla_policy[CTA_TIMEOUT_UDP_MAX+1] = { [CTA_TIMEOUT_UDP_UNREPLIED] = { .type = NLA_U32 }, [CTA_TIMEOUT_UDP_REPLIED] = { .type = NLA_U32 }, }; -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ #ifdef CONFIG_SYSCTL static struct ctl_table udp_sysctl_table[] = { @@ -292,7 +292,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udp4 = .nlattr_tuple_size = nf_ct_port_nlattr_tuple_size, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = udp_timeout_nlattr_to_obj, .obj_to_nlattr = udp_timeout_obj_to_nlattr, @@ -300,7 +300,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udp4 = .obj_size = sizeof(unsigned int) * CTA_TIMEOUT_UDP_MAX, .nla_policy = udp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = udp_init_net, .get_net_proto = udp_get_net_proto, }; @@ -321,7 +321,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udplite4 = .nlattr_tuple_size = nf_ct_port_nlattr_tuple_size, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = udp_timeout_nlattr_to_obj, .obj_to_nlattr = udp_timeout_obj_to_nlattr, @@ -329,7 +329,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udplite4 = .obj_size = sizeof(unsigned int) * CTA_TIMEOUT_UDP_MAX, .nla_policy = udp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = udp_init_net, .get_net_proto = udp_get_net_proto, }; @@ -350,7 +350,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udp6 = .nlattr_tuple_size = nf_ct_port_nlattr_tuple_size, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = udp_timeout_nlattr_to_obj, .obj_to_nlattr = udp_timeout_obj_to_nlattr, @@ -358,7 +358,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udp6 = .obj_size = sizeof(unsigned int) * CTA_TIMEOUT_UDP_MAX, .nla_policy = udp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = udp_init_net, .get_net_proto = udp_get_net_proto, }; @@ -379,7 +379,7 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udplite6 = .nlattr_tuple_size = nf_ct_port_nlattr_tuple_size, .nla_policy = nf_ct_port_nla_policy, #endif -#if IS_ENABLED(CONFIG_NF_CT_NETLINK_TIMEOUT) +#ifdef CONFIG_NF_CONNTRACK_TIMEOUT .ctnl_timeout = { .nlattr_to_obj = udp_timeout_nlattr_to_obj, .obj_to_nlattr = udp_timeout_obj_to_nlattr, @@ -387,10 +387,9 @@ const struct nf_conntrack_l4proto nf_conntrack_l4proto_udplite6 = .obj_size = sizeof(unsigned int) * CTA_TIMEOUT_UDP_MAX, .nla_policy = udp_timeout_nla_policy, }, -#endif /* CONFIG_NF_CT_NETLINK_TIMEOUT */ +#endif /* CONFIG_NF_CONNTRACK_TIMEOUT */ .init_net = udp_init_net, .get_net_proto = udp_get_net_proto, }; EXPORT_SYMBOL_GPL(nf_conntrack_l4proto_udplite6); #endif -#include <net/netfilter/nf_conntrack_timeout.h> diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c index 1dca5683f59f..2cfb173cd0b2 100644 --- a/net/netfilter/nf_tables_api.c +++ b/net/netfilter/nf_tables_api.c @@ -4637,6 +4637,7 @@ static int nft_flush_set(const struct nft_ctx *ctx, } set->ndeact++; + nft_set_elem_deactivate(ctx->net, set, elem); nft_trans_elem_set(trans) = set; nft_trans_elem(trans) = *elem; list_add_tail(&trans->list, &ctx->net->nft.commit_list); diff --git a/net/netfilter/nfnetlink_cttimeout.c b/net/netfilter/nfnetlink_cttimeout.c index d46a236cdf31..a30f8ba4b89a 100644 --- a/net/netfilter/nfnetlink_cttimeout.c +++ b/net/netfilter/nfnetlink_cttimeout.c @@ -489,8 +489,8 @@ err: return err; } -static struct ctnl_timeout * -ctnl_timeout_find_get(struct net *net, const char *name) +static struct nf_ct_timeout *ctnl_timeout_find_get(struct net *net, + const char *name) { struct ctnl_timeout *timeout, *matching = NULL; @@ -509,7 +509,7 @@ ctnl_timeout_find_get(struct net *net, const char *name) break; } err: - return matching; + return matching ? &matching->timeout : NULL; } static void ctnl_timeout_put(struct nf_ct_timeout *t) diff --git a/net/netfilter/nfnetlink_queue.c b/net/netfilter/nfnetlink_queue.c index ea4ba551abb2..d33094f4ec41 100644 --- a/net/netfilter/nfnetlink_queue.c +++ b/net/netfilter/nfnetlink_queue.c @@ -233,6 +233,7 @@ static void nfqnl_reinject(struct nf_queue_entry *entry, unsigned int verdict) int err; if (verdict == NF_ACCEPT || + verdict == NF_REPEAT || verdict == NF_STOP) { rcu_read_lock(); ct_hook = rcu_dereference(nf_ct_hook); diff --git a/net/netfilter/nft_ct.c b/net/netfilter/nft_ct.c index 26a8baebd072..5dd87748afa8 100644 --- a/net/netfilter/nft_ct.c +++ b/net/netfilter/nft_ct.c @@ -799,7 +799,7 @@ err: } struct nft_ct_timeout_obj { - struct nf_conn *tmpl; + struct nf_ct_timeout *timeout; u8 l4proto; }; @@ -809,26 +809,42 @@ static void nft_ct_timeout_obj_eval(struct nft_object *obj, { const struct nft_ct_timeout_obj *priv = nft_obj_data(obj); struct nf_conn *ct = (struct nf_conn *)skb_nfct(pkt->skb); - struct sk_buff *skb = pkt->skb; + struct nf_conn_timeout *timeout; + const unsigned int *values; + + if (priv->l4proto != pkt->tprot) + return; - if (ct || - priv->l4proto != pkt->tprot) + if (!ct || nf_ct_is_template(ct) || nf_ct_is_confirmed(ct)) return; - nf_ct_set(skb, priv->tmpl, IP_CT_NEW); + timeout = nf_ct_timeout_find(ct); + if (!timeout) { + timeout = nf_ct_timeout_ext_add(ct, priv->timeout, GFP_ATOMIC); + if (!timeout) { + regs->verdict.code = NF_DROP; + return; + } + } + + rcu_assign_pointer(timeout->timeout, priv->timeout); + + /* adjust the timeout as per 'new' state. ct is unconfirmed, + * so the current timestamp must not be added. + */ + values = nf_ct_timeout_data(timeout); + if (values) + nf_ct_refresh(ct, pkt->skb, values[0]); } static int nft_ct_timeout_obj_init(const struct nft_ctx *ctx, const struct nlattr * const tb[], struct nft_object *obj) { - const struct nf_conntrack_zone *zone = &nf_ct_zone_dflt; struct nft_ct_timeout_obj *priv = nft_obj_data(obj); const struct nf_conntrack_l4proto *l4proto; - struct nf_conn_timeout *timeout_ext; struct nf_ct_timeout *timeout; int l3num = ctx->family; - struct nf_conn *tmpl; __u8 l4num; int ret; @@ -863,28 +879,14 @@ static int nft_ct_timeout_obj_init(const struct nft_ctx *ctx, timeout->l3num = l3num; timeout->l4proto = l4proto; - tmpl = nf_ct_tmpl_alloc(ctx->net, zone, GFP_ATOMIC); - if (!tmpl) { - ret = -ENOMEM; - goto err_free_timeout; - } - - timeout_ext = nf_ct_timeout_ext_add(tmpl, timeout, GFP_ATOMIC); - if (!timeout_ext) { - ret = -ENOMEM; - goto err_free_tmpl; - } ret = nf_ct_netns_get(ctx->net, ctx->family); if (ret < 0) - goto err_free_tmpl; - - priv->tmpl = tmpl; + goto err_free_timeout; + priv->timeout = timeout; return 0; -err_free_tmpl: - nf_ct_tmpl_free(tmpl); err_free_timeout: kfree(timeout); err_proto_put: @@ -896,22 +898,19 @@ static void nft_ct_timeout_obj_destroy(const struct nft_ctx *ctx, struct nft_object *obj) { struct nft_ct_timeout_obj *priv = nft_obj_data(obj); - struct nf_conn_timeout *t = nf_ct_timeout_find(priv->tmpl); - struct nf_ct_timeout *timeout; + struct nf_ct_timeout *timeout = priv->timeout; - timeout = rcu_dereference_raw(t->timeout); nf_ct_untimeout(ctx->net, timeout); nf_ct_l4proto_put(timeout->l4proto); nf_ct_netns_put(ctx->net, ctx->family); - nf_ct_tmpl_free(priv->tmpl); + kfree(priv->timeout); } static int nft_ct_timeout_obj_dump(struct sk_buff *skb, struct nft_object *obj, bool reset) { const struct nft_ct_timeout_obj *priv = nft_obj_data(obj); - const struct nf_conn_timeout *t = nf_ct_timeout_find(priv->tmpl); - const struct nf_ct_timeout *timeout = rcu_dereference_raw(t->timeout); + const struct nf_ct_timeout *timeout = priv->timeout; struct nlattr *nest_params; int ret; diff --git a/net/netfilter/xt_CHECKSUM.c b/net/netfilter/xt_CHECKSUM.c index 9f4151ec3e06..6c7aa6a0a0d2 100644 --- a/net/netfilter/xt_CHECKSUM.c +++ b/net/netfilter/xt_CHECKSUM.c @@ -16,6 +16,9 @@ #include <linux/netfilter/x_tables.h> #include <linux/netfilter/xt_CHECKSUM.h> +#include <linux/netfilter_ipv4/ip_tables.h> +#include <linux/netfilter_ipv6/ip6_tables.h> + MODULE_LICENSE("GPL"); MODULE_AUTHOR("Michael S. Tsirkin <mst@redhat.com>"); MODULE_DESCRIPTION("Xtables: checksum modification"); @@ -25,7 +28,7 @@ MODULE_ALIAS("ip6t_CHECKSUM"); static unsigned int checksum_tg(struct sk_buff *skb, const struct xt_action_param *par) { - if (skb->ip_summed == CHECKSUM_PARTIAL) + if (skb->ip_summed == CHECKSUM_PARTIAL && !skb_is_gso(skb)) skb_checksum_help(skb); return XT_CONTINUE; @@ -34,6 +37,8 @@ checksum_tg(struct sk_buff *skb, const struct xt_action_param *par) static int checksum_tg_check(const struct xt_tgchk_param *par) { const struct xt_CHECKSUM_info *einfo = par->targinfo; + const struct ip6t_ip6 *i6 = par->entryinfo; + const struct ipt_ip *i4 = par->entryinfo; if (einfo->operation & ~XT_CHECKSUM_OP_FILL) { pr_info_ratelimited("unsupported CHECKSUM operation %x\n", @@ -43,6 +48,21 @@ static int checksum_tg_check(const struct xt_tgchk_param *par) if (!einfo->operation) return -EINVAL; + switch (par->family) { + case NFPROTO_IPV4: + if (i4->proto == IPPROTO_UDP && + (i4->invflags & XT_INV_PROTO) == 0) + return 0; + break; + case NFPROTO_IPV6: + if ((i6->flags & IP6T_F_PROTO) && + i6->proto == IPPROTO_UDP && + (i6->invflags & XT_INV_PROTO) == 0) + return 0; + break; + } + + pr_warn_once("CHECKSUM should be avoided. If really needed, restrict with \"-p udp\" and only use in OUTPUT\n"); return 0; } diff --git a/net/netfilter/xt_cluster.c b/net/netfilter/xt_cluster.c index dfbdbb2fc0ed..51d0c257e7a5 100644 --- a/net/netfilter/xt_cluster.c +++ b/net/netfilter/xt_cluster.c @@ -125,6 +125,7 @@ xt_cluster_mt(const struct sk_buff *skb, struct xt_action_param *par) static int xt_cluster_mt_checkentry(const struct xt_mtchk_param *par) { struct xt_cluster_match_info *info = par->matchinfo; + int ret; if (info->total_nodes > XT_CLUSTER_NODES_MAX) { pr_info_ratelimited("you have exceeded the maximum number of cluster nodes (%u > %u)\n", @@ -135,7 +136,17 @@ static int xt_cluster_mt_checkentry(const struct xt_mtchk_param *par) pr_info_ratelimited("node mask cannot exceed total number of nodes\n"); return -EDOM; } - return 0; + + ret = nf_ct_netns_get(par->net, par->family); + if (ret < 0) + pr_info_ratelimited("cannot load conntrack support for proto=%u\n", + par->family); + return ret; +} + +static void xt_cluster_mt_destroy(const struct xt_mtdtor_param *par) +{ + nf_ct_netns_put(par->net, par->family); } static struct xt_match xt_cluster_match __read_mostly = { @@ -144,6 +155,7 @@ static struct xt_match xt_cluster_match __read_mostly = { .match = xt_cluster_mt, .checkentry = xt_cluster_mt_checkentry, .matchsize = sizeof(struct xt_cluster_match_info), + .destroy = xt_cluster_mt_destroy, .me = THIS_MODULE, }; diff --git a/net/netfilter/xt_hashlimit.c b/net/netfilter/xt_hashlimit.c index 9b16402f29af..3e7d259e5d8d 100644 --- a/net/netfilter/xt_hashlimit.c +++ b/net/netfilter/xt_hashlimit.c @@ -1057,7 +1057,7 @@ static struct xt_match hashlimit_mt_reg[] __read_mostly = { static void *dl_seq_start(struct seq_file *s, loff_t *pos) __acquires(htable->lock) { - struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->file)); unsigned int *bucket; spin_lock_bh(&htable->lock); @@ -1074,7 +1074,7 @@ static void *dl_seq_start(struct seq_file *s, loff_t *pos) static void *dl_seq_next(struct seq_file *s, void *v, loff_t *pos) { - struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->file)); unsigned int *bucket = v; *pos = ++(*bucket); @@ -1088,7 +1088,7 @@ static void *dl_seq_next(struct seq_file *s, void *v, loff_t *pos) static void dl_seq_stop(struct seq_file *s, void *v) __releases(htable->lock) { - struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->file)); unsigned int *bucket = v; if (!IS_ERR(bucket)) @@ -1130,7 +1130,7 @@ static void dl_seq_print(struct dsthash_ent *ent, u_int8_t family, static int dl_seq_real_show_v2(struct dsthash_ent *ent, u_int8_t family, struct seq_file *s) { - struct xt_hashlimit_htable *ht = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *ht = PDE_DATA(file_inode(s->file)); spin_lock(&ent->lock); /* recalculate to show accurate numbers */ @@ -1145,7 +1145,7 @@ static int dl_seq_real_show_v2(struct dsthash_ent *ent, u_int8_t family, static int dl_seq_real_show_v1(struct dsthash_ent *ent, u_int8_t family, struct seq_file *s) { - struct xt_hashlimit_htable *ht = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *ht = PDE_DATA(file_inode(s->file)); spin_lock(&ent->lock); /* recalculate to show accurate numbers */ @@ -1160,7 +1160,7 @@ static int dl_seq_real_show_v1(struct dsthash_ent *ent, u_int8_t family, static int dl_seq_real_show(struct dsthash_ent *ent, u_int8_t family, struct seq_file *s) { - struct xt_hashlimit_htable *ht = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *ht = PDE_DATA(file_inode(s->file)); spin_lock(&ent->lock); /* recalculate to show accurate numbers */ @@ -1174,7 +1174,7 @@ static int dl_seq_real_show(struct dsthash_ent *ent, u_int8_t family, static int dl_seq_show_v2(struct seq_file *s, void *v) { - struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->file)); unsigned int *bucket = (unsigned int *)v; struct dsthash_ent *ent; @@ -1188,7 +1188,7 @@ static int dl_seq_show_v2(struct seq_file *s, void *v) static int dl_seq_show_v1(struct seq_file *s, void *v) { - struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->file)); unsigned int *bucket = v; struct dsthash_ent *ent; @@ -1202,7 +1202,7 @@ static int dl_seq_show_v1(struct seq_file *s, void *v) static int dl_seq_show(struct seq_file *s, void *v) { - struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->private)); + struct xt_hashlimit_htable *htable = PDE_DATA(file_inode(s->file)); unsigned int *bucket = v; struct dsthash_ent *ent; diff --git a/net/netlabel/netlabel_unlabeled.c b/net/netlabel/netlabel_unlabeled.c index c070dfc0190a..c92894c3e40a 100644 --- a/net/netlabel/netlabel_unlabeled.c +++ b/net/netlabel/netlabel_unlabeled.c @@ -781,7 +781,8 @@ static int netlbl_unlabel_addrinfo_get(struct genl_info *info, { u32 addr_len; - if (info->attrs[NLBL_UNLABEL_A_IPV4ADDR]) { + if (info->attrs[NLBL_UNLABEL_A_IPV4ADDR] && + info->attrs[NLBL_UNLABEL_A_IPV4MASK]) { addr_len = nla_len(info->attrs[NLBL_UNLABEL_A_IPV4ADDR]); if (addr_len != sizeof(struct in_addr) && addr_len != nla_len(info->attrs[NLBL_UNLABEL_A_IPV4MASK])) diff --git a/net/nfc/hci/core.c b/net/nfc/hci/core.c index ac8030c4bcf8..19cb2e473ea6 100644 --- a/net/nfc/hci/core.c +++ b/net/nfc/hci/core.c @@ -209,6 +209,11 @@ void nfc_hci_cmd_received(struct nfc_hci_dev *hdev, u8 pipe, u8 cmd, } create_info = (struct hci_create_pipe_resp *)skb->data; + if (create_info->pipe >= NFC_HCI_MAX_PIPES) { + status = NFC_HCI_ANY_E_NOK; + goto exit; + } + /* Save the new created pipe and bind with local gate, * the description for skb->data[3] is destination gate id * but since we received this cmd from host controller, we @@ -232,6 +237,11 @@ void nfc_hci_cmd_received(struct nfc_hci_dev *hdev, u8 pipe, u8 cmd, } delete_info = (struct hci_delete_pipe_noti *)skb->data; + if (delete_info->pipe >= NFC_HCI_MAX_PIPES) { + status = NFC_HCI_ANY_E_NOK; + goto exit; + } + hdev->pipes[delete_info->pipe].gate = NFC_HCI_INVALID_GATE; hdev->pipes[delete_info->pipe].dest_host = NFC_HCI_INVALID_HOST; break; diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index 5610061e7f2e..75c92a87e7b2 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -4137,36 +4137,52 @@ static const struct vm_operations_struct packet_mmap_ops = { .close = packet_mm_close, }; -static void free_pg_vec(struct pgv *pg_vec, unsigned int len) +static void free_pg_vec(struct pgv *pg_vec, unsigned int order, + unsigned int len) { int i; for (i = 0; i < len; i++) { if (likely(pg_vec[i].buffer)) { - kvfree(pg_vec[i].buffer); + if (is_vmalloc_addr(pg_vec[i].buffer)) + vfree(pg_vec[i].buffer); + else + free_pages((unsigned long)pg_vec[i].buffer, + order); pg_vec[i].buffer = NULL; } } kfree(pg_vec); } -static char *alloc_one_pg_vec_page(unsigned long size) +static char *alloc_one_pg_vec_page(unsigned long order) { char *buffer; + gfp_t gfp_flags = GFP_KERNEL | __GFP_COMP | + __GFP_ZERO | __GFP_NOWARN | __GFP_NORETRY; - buffer = kvzalloc(size, GFP_KERNEL); + buffer = (char *) __get_free_pages(gfp_flags, order); if (buffer) return buffer; - buffer = kvzalloc(size, GFP_KERNEL | __GFP_RETRY_MAYFAIL); + /* __get_free_pages failed, fall back to vmalloc */ + buffer = vzalloc(array_size((1 << order), PAGE_SIZE)); + if (buffer) + return buffer; - return buffer; + /* vmalloc failed, lets dig into swap here */ + gfp_flags &= ~__GFP_NORETRY; + buffer = (char *) __get_free_pages(gfp_flags, order); + if (buffer) + return buffer; + + /* complete and utter failure */ + return NULL; } -static struct pgv *alloc_pg_vec(struct tpacket_req *req) +static struct pgv *alloc_pg_vec(struct tpacket_req *req, int order) { unsigned int block_nr = req->tp_block_nr; - unsigned long size = req->tp_block_size; struct pgv *pg_vec; int i; @@ -4175,7 +4191,7 @@ static struct pgv *alloc_pg_vec(struct tpacket_req *req) goto out; for (i = 0; i < block_nr; i++) { - pg_vec[i].buffer = alloc_one_pg_vec_page(size); + pg_vec[i].buffer = alloc_one_pg_vec_page(order); if (unlikely(!pg_vec[i].buffer)) goto out_free_pgvec; } @@ -4184,7 +4200,7 @@ out: return pg_vec; out_free_pgvec: - free_pg_vec(pg_vec, block_nr); + free_pg_vec(pg_vec, order, block_nr); pg_vec = NULL; goto out; } @@ -4194,9 +4210,9 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u, { struct pgv *pg_vec = NULL; struct packet_sock *po = pkt_sk(sk); + int was_running, order = 0; struct packet_ring_buffer *rb; struct sk_buff_head *rb_queue; - int was_running; __be16 num; int err = -EINVAL; /* Added to avoid minimal code churn */ @@ -4258,7 +4274,8 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u, goto out; err = -ENOMEM; - pg_vec = alloc_pg_vec(req); + order = get_order(req->tp_block_size); + pg_vec = alloc_pg_vec(req, order); if (unlikely(!pg_vec)) goto out; switch (po->tp_version) { @@ -4312,6 +4329,7 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u, rb->frame_size = req->tp_frame_size; spin_unlock_bh(&rb_queue->lock); + swap(rb->pg_vec_order, order); swap(rb->pg_vec_len, req->tp_block_nr); rb->pg_vec_pages = req->tp_block_size/PAGE_SIZE; @@ -4337,7 +4355,7 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u, } if (pg_vec) - free_pg_vec(pg_vec, req->tp_block_nr); + free_pg_vec(pg_vec, order, req->tp_block_nr); out: return err; } diff --git a/net/packet/internal.h b/net/packet/internal.h index 8f50036f62f0..3bb7c5fb3bff 100644 --- a/net/packet/internal.h +++ b/net/packet/internal.h @@ -64,6 +64,7 @@ struct packet_ring_buffer { unsigned int frame_size; unsigned int frame_max; + unsigned int pg_vec_order; unsigned int pg_vec_pages; unsigned int pg_vec_len; diff --git a/net/rds/Kconfig b/net/rds/Kconfig index 01b3bd6a3708..b9092111bc45 100644 --- a/net/rds/Kconfig +++ b/net/rds/Kconfig @@ -1,6 +1,6 @@ config RDS - tristate "The RDS Protocol" + tristate "The Reliable Datagram Sockets Protocol" depends on INET ---help--- The RDS (Reliable Datagram Sockets) protocol provides reliable, diff --git a/net/rds/bind.c b/net/rds/bind.c index 3ab55784b637..762d2c6788a3 100644 --- a/net/rds/bind.c +++ b/net/rds/bind.c @@ -76,11 +76,13 @@ struct rds_sock *rds_find_bound(const struct in6_addr *addr, __be16 port, struct rds_sock *rs; __rds_create_bind_key(key, addr, port, scope_id); - rs = rhashtable_lookup_fast(&bind_hash_table, key, ht_parms); + rcu_read_lock(); + rs = rhashtable_lookup(&bind_hash_table, key, ht_parms); if (rs && !sock_flag(rds_rs_to_sk(rs), SOCK_DEAD)) rds_sock_addref(rs); else rs = NULL; + rcu_read_unlock(); rdsdebug("returning rs %p for %pI6c:%u\n", rs, addr, ntohs(port)); @@ -235,6 +237,7 @@ int rds_bind(struct socket *sock, struct sockaddr *uaddr, int addr_len) goto out; } + sock_set_flag(sk, SOCK_RCU_FREE); ret = rds_add_bound(rs, binding_addr, &port, scope_id); if (ret) goto out; diff --git a/net/rds/ib.c b/net/rds/ib.c index c1d97640c0be..eba75c1ba359 100644 --- a/net/rds/ib.c +++ b/net/rds/ib.c @@ -341,15 +341,10 @@ static int rds6_ib_conn_info_visitor(struct rds_connection *conn, if (rds_conn_state(conn) == RDS_CONN_UP) { struct rds_ib_device *rds_ibdev; - struct rdma_dev_addr *dev_addr; ic = conn->c_transport_data; - dev_addr = &ic->i_cm_id->route.addr.dev_addr; - rdma_addr_get_sgid(dev_addr, - (union ib_gid *)&iinfo6->src_gid); - rdma_addr_get_dgid(dev_addr, - (union ib_gid *)&iinfo6->dst_gid); - + rdma_read_gids(ic->i_cm_id, (union ib_gid *)&iinfo6->src_gid, + (union ib_gid *)&iinfo6->dst_gid); rds_ibdev = ic->rds_ibdev; iinfo6->max_send_wr = ic->i_send_ring.w_nr; iinfo6->max_recv_wr = ic->i_recv_ring.w_nr; diff --git a/net/rds/ib.h b/net/rds/ib.h index 73427ff439f9..71ff356ee702 100644 --- a/net/rds/ib.h +++ b/net/rds/ib.h @@ -443,7 +443,7 @@ int rds_ib_send_grab_credits(struct rds_ib_connection *ic, u32 wanted, int rds_ib_xmit_atomic(struct rds_connection *conn, struct rm_atomic_op *op); /* ib_stats.c */ -DECLARE_PER_CPU(struct rds_ib_statistics, rds_ib_stats); +DECLARE_PER_CPU_SHARED_ALIGNED(struct rds_ib_statistics, rds_ib_stats); #define rds_ib_stats_inc(member) rds_stats_inc_which(rds_ib_stats, member) #define rds_ib_stats_add(member, count) \ rds_stats_add_which(rds_ib_stats, member, count) diff --git a/net/rfkill/rfkill-gpio.c b/net/rfkill/rfkill-gpio.c index 00192a996be0..0f8465852254 100644 --- a/net/rfkill/rfkill-gpio.c +++ b/net/rfkill/rfkill-gpio.c @@ -20,6 +20,7 @@ #include <linux/init.h> #include <linux/kernel.h> #include <linux/module.h> +#include <linux/mod_devicetable.h> #include <linux/rfkill.h> #include <linux/platform_device.h> #include <linux/clk.h> diff --git a/net/sched/act_api.c b/net/sched/act_api.c index db83dac1e7f4..e12f8ef7baa4 100644 --- a/net/sched/act_api.c +++ b/net/sched/act_api.c @@ -662,6 +662,13 @@ int tcf_action_destroy(struct tc_action *actions[], int bind) return ret; } +static int tcf_action_destroy_1(struct tc_action *a, int bind) +{ + struct tc_action *actions[] = { a, NULL }; + + return tcf_action_destroy(actions, bind); +} + static int tcf_action_put(struct tc_action *p) { return __tcf_action_put(p, false); @@ -881,17 +888,16 @@ struct tc_action *tcf_action_init_1(struct net *net, struct tcf_proto *tp, if (TC_ACT_EXT_CMP(a->tcfa_action, TC_ACT_GOTO_CHAIN)) { err = tcf_action_goto_chain_init(a, tp); if (err) { - struct tc_action *actions[] = { a, NULL }; - - tcf_action_destroy(actions, bind); + tcf_action_destroy_1(a, bind); NL_SET_ERR_MSG(extack, "Failed to init TC action chain"); return ERR_PTR(err); } } if (!tcf_action_valid(a->tcfa_action)) { - NL_SET_ERR_MSG(extack, "invalid action value, using TC_ACT_UNSPEC instead"); - a->tcfa_action = TC_ACT_UNSPEC; + tcf_action_destroy_1(a, bind); + NL_SET_ERR_MSG(extack, "Invalid control action value"); + return ERR_PTR(-EINVAL); } return a; @@ -1173,6 +1179,7 @@ static int tcf_action_delete(struct net *net, struct tc_action *actions[]) struct tcf_idrinfo *idrinfo = a->idrinfo; u32 act_index = a->tcfa_index; + actions[i] = NULL; if (tcf_action_put(a)) { /* last reference, action was deleted concurrently */ module_put(ops->owner); @@ -1184,7 +1191,6 @@ static int tcf_action_delete(struct net *net, struct tc_action *actions[]) if (ret < 0) return ret; } - actions[i] = NULL; } return 0; } diff --git a/net/sched/act_ife.c b/net/sched/act_ife.c index 196430aefe87..06a3d4801878 100644 --- a/net/sched/act_ife.c +++ b/net/sched/act_ife.c @@ -326,6 +326,20 @@ static int __add_metainfo(const struct tcf_meta_ops *ops, return ret; } +static int add_metainfo_and_get_ops(const struct tcf_meta_ops *ops, + struct tcf_ife_info *ife, u32 metaid, + bool exists) +{ + int ret; + + if (!try_module_get(ops->owner)) + return -ENOENT; + ret = __add_metainfo(ops, ife, metaid, NULL, 0, true, exists); + if (ret) + module_put(ops->owner); + return ret; +} + static int add_metainfo(struct tcf_ife_info *ife, u32 metaid, void *metaval, int len, bool exists) { @@ -349,7 +363,7 @@ static int use_all_metadata(struct tcf_ife_info *ife, bool exists) read_lock(&ife_mod_lock); list_for_each_entry(o, &ifeoplist, list) { - rc = __add_metainfo(o, ife, o->metaid, NULL, 0, true, exists); + rc = add_metainfo_and_get_ops(o, ife, o->metaid, exists); if (rc == 0) installed += 1; } @@ -400,7 +414,6 @@ static void _tcf_ife_cleanup(struct tc_action *a) struct tcf_meta_info *e, *n; list_for_each_entry_safe(e, n, &ife->metalist, metalist) { - module_put(e->ops->owner); list_del(&e->metalist); if (e->metaval) { if (e->ops->release) @@ -408,6 +421,7 @@ static void _tcf_ife_cleanup(struct tc_action *a) else kfree(e->metaval); } + module_put(e->ops->owner); kfree(e); } } diff --git a/net/sched/act_pedit.c b/net/sched/act_pedit.c index 107034070019..ad99a99f11f6 100644 --- a/net/sched/act_pedit.c +++ b/net/sched/act_pedit.c @@ -109,16 +109,18 @@ static int tcf_pedit_key_ex_dump(struct sk_buff *skb, { struct nlattr *keys_start = nla_nest_start(skb, TCA_PEDIT_KEYS_EX); + if (!keys_start) + goto nla_failure; for (; n > 0; n--) { struct nlattr *key_start; key_start = nla_nest_start(skb, TCA_PEDIT_KEY_EX); + if (!key_start) + goto nla_failure; if (nla_put_u16(skb, TCA_PEDIT_KEY_EX_HTYPE, keys_ex->htype) || - nla_put_u16(skb, TCA_PEDIT_KEY_EX_CMD, keys_ex->cmd)) { - nlmsg_trim(skb, keys_start); - return -EINVAL; - } + nla_put_u16(skb, TCA_PEDIT_KEY_EX_CMD, keys_ex->cmd)) + goto nla_failure; nla_nest_end(skb, key_start); @@ -128,6 +130,9 @@ static int tcf_pedit_key_ex_dump(struct sk_buff *skb, nla_nest_end(skb, keys_start); return 0; +nla_failure: + nla_nest_cancel(skb, keys_start); + return -EINVAL; } static int tcf_pedit_init(struct net *net, struct nlattr *nla, @@ -418,7 +423,10 @@ static int tcf_pedit_dump(struct sk_buff *skb, struct tc_action *a, opt->bindcnt = atomic_read(&p->tcf_bindcnt) - bind; if (p->tcfp_keys_ex) { - tcf_pedit_key_ex_dump(skb, p->tcfp_keys_ex, p->tcfp_nkeys); + if (tcf_pedit_key_ex_dump(skb, + p->tcfp_keys_ex, + p->tcfp_nkeys)) + goto nla_put_failure; if (nla_put(skb, TCA_PEDIT_PARMS_EX, s, opt)) goto nla_put_failure; diff --git a/net/sched/act_sample.c b/net/sched/act_sample.c index 44e9c00657bc..6b67aa13d2dd 100644 --- a/net/sched/act_sample.c +++ b/net/sched/act_sample.c @@ -69,7 +69,7 @@ static int tcf_sample_init(struct net *net, struct nlattr *nla, if (!exists) { ret = tcf_idr_create(tn, parm->index, est, a, - &act_sample_ops, bind, false); + &act_sample_ops, bind, true); if (ret) { tcf_idr_cleanup(tn, parm->index); return ret; diff --git a/net/sched/act_tunnel_key.c b/net/sched/act_tunnel_key.c index 420759153d5f..681f6f04e7da 100644 --- a/net/sched/act_tunnel_key.c +++ b/net/sched/act_tunnel_key.c @@ -317,7 +317,7 @@ static int tunnel_key_init(struct net *net, struct nlattr *nla, &metadata->u.tun_info, opts_len, extack); if (ret < 0) - goto err_out; + goto release_tun_meta; } metadata->u.tun_info.mode |= IP_TUNNEL_INFO_TX; @@ -333,23 +333,24 @@ static int tunnel_key_init(struct net *net, struct nlattr *nla, &act_tunnel_key_ops, bind, true); if (ret) { NL_SET_ERR_MSG(extack, "Cannot create TC IDR"); - goto err_out; + goto release_tun_meta; } ret = ACT_P_CREATED; } else if (!ovr) { - tcf_idr_release(*a, bind); NL_SET_ERR_MSG(extack, "TC IDR already exists"); - return -EEXIST; + ret = -EEXIST; + goto release_tun_meta; } t = to_tunnel_key(*a); params_new = kzalloc(sizeof(*params_new), GFP_KERNEL); if (unlikely(!params_new)) { - tcf_idr_release(*a, bind); NL_SET_ERR_MSG(extack, "Cannot allocate tunnel key parameters"); - return -ENOMEM; + ret = -ENOMEM; + exists = true; + goto release_tun_meta; } params_new->tcft_action = parm->t_action; params_new->tcft_enc_metadata = metadata; @@ -367,6 +368,9 @@ static int tunnel_key_init(struct net *net, struct nlattr *nla, return ret; +release_tun_meta: + dst_release(&metadata->dst); + err_out: if (exists) tcf_idr_release(*a, bind); @@ -408,8 +412,10 @@ static int tunnel_key_geneve_opts_dump(struct sk_buff *skb, nla_put_u8(skb, TCA_TUNNEL_KEY_ENC_OPT_GENEVE_TYPE, opt->type) || nla_put(skb, TCA_TUNNEL_KEY_ENC_OPT_GENEVE_DATA, - opt->length * 4, opt + 1)) + opt->length * 4, opt + 1)) { + nla_nest_cancel(skb, start); return -EMSGSIZE; + } len -= sizeof(struct geneve_opt) + opt->length * 4; src += sizeof(struct geneve_opt) + opt->length * 4; @@ -423,7 +429,7 @@ static int tunnel_key_opts_dump(struct sk_buff *skb, const struct ip_tunnel_info *info) { struct nlattr *start; - int err; + int err = -EINVAL; if (!info->options_len) return 0; @@ -435,9 +441,11 @@ static int tunnel_key_opts_dump(struct sk_buff *skb, if (info->key.tun_flags & TUNNEL_GENEVE_OPT) { err = tunnel_key_geneve_opts_dump(skb, info); if (err) - return err; + goto err_out; } else { - return -EINVAL; +err_out: + nla_nest_cancel(skb, start); + return err; } nla_nest_end(skb, start); diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c index 31bd1439cf60..0a75cb2e5e7b 100644 --- a/net/sched/cls_api.c +++ b/net/sched/cls_api.c @@ -1252,7 +1252,7 @@ replay: } chain = tcf_chain_get(block, chain_index, true); if (!chain) { - NL_SET_ERR_MSG(extack, "Cannot find specified filter chain"); + NL_SET_ERR_MSG(extack, "Cannot create specified filter chain"); err = -ENOMEM; goto errout; } @@ -1399,7 +1399,7 @@ static int tc_del_tfilter(struct sk_buff *skb, struct nlmsghdr *n, goto errout; } NL_SET_ERR_MSG(extack, "Cannot find specified filter chain"); - err = -EINVAL; + err = -ENOENT; goto errout; } @@ -1902,6 +1902,8 @@ replay: RTM_NEWCHAIN, false); break; case RTM_DELCHAIN: + tfilter_notify_chain(net, skb, block, q, parent, n, + chain, RTM_DELTFILTER); /* Flush the chain first as the user requested chain removal. */ tcf_chain_flush(chain); /* In case the chain was successfully deleted, put a reference diff --git a/net/sctp/proc.c b/net/sctp/proc.c index ef5c9a82d4e8..a644292f9faf 100644 --- a/net/sctp/proc.c +++ b/net/sctp/proc.c @@ -215,7 +215,6 @@ static const struct seq_operations sctp_eps_ops = { struct sctp_ht_iter { struct seq_net_private p; struct rhashtable_iter hti; - int start_fail; }; static void *sctp_transport_seq_start(struct seq_file *seq, loff_t *pos) @@ -224,7 +223,6 @@ static void *sctp_transport_seq_start(struct seq_file *seq, loff_t *pos) sctp_transport_walk_start(&iter->hti); - iter->start_fail = 0; return sctp_transport_get_idx(seq_file_net(seq), &iter->hti, *pos); } @@ -232,8 +230,6 @@ static void sctp_transport_seq_stop(struct seq_file *seq, void *v) { struct sctp_ht_iter *iter = seq->private; - if (iter->start_fail) - return; sctp_transport_walk_stop(&iter->hti); } @@ -264,8 +260,6 @@ static int sctp_assocs_seq_show(struct seq_file *seq, void *v) } transport = (struct sctp_transport *)v; - if (!sctp_transport_hold(transport)) - return 0; assoc = transport->asoc; epb = &assoc->base; sk = epb->sk; @@ -322,8 +316,6 @@ static int sctp_remaddr_seq_show(struct seq_file *seq, void *v) } transport = (struct sctp_transport *)v; - if (!sctp_transport_hold(transport)) - return 0; assoc = transport->asoc; list_for_each_entry_rcu(tsp, &assoc->peer.transport_addr_list, diff --git a/net/sctp/socket.c b/net/sctp/socket.c index e96b15a66aba..f73e9d38d5ba 100644 --- a/net/sctp/socket.c +++ b/net/sctp/socket.c @@ -2658,20 +2658,23 @@ static int sctp_apply_peer_addr_params(struct sctp_paddrparams *params, } if (params->spp_flags & SPP_IPV6_FLOWLABEL) { - if (trans && trans->ipaddr.sa.sa_family == AF_INET6) { - trans->flowlabel = params->spp_ipv6_flowlabel & - SCTP_FLOWLABEL_VAL_MASK; - trans->flowlabel |= SCTP_FLOWLABEL_SET_MASK; - } else if (asoc) { - list_for_each_entry(trans, - &asoc->peer.transport_addr_list, - transports) { - if (trans->ipaddr.sa.sa_family != AF_INET6) - continue; + if (trans) { + if (trans->ipaddr.sa.sa_family == AF_INET6) { trans->flowlabel = params->spp_ipv6_flowlabel & SCTP_FLOWLABEL_VAL_MASK; trans->flowlabel |= SCTP_FLOWLABEL_SET_MASK; } + } else if (asoc) { + struct sctp_transport *t; + + list_for_each_entry(t, &asoc->peer.transport_addr_list, + transports) { + if (t->ipaddr.sa.sa_family != AF_INET6) + continue; + t->flowlabel = params->spp_ipv6_flowlabel & + SCTP_FLOWLABEL_VAL_MASK; + t->flowlabel |= SCTP_FLOWLABEL_SET_MASK; + } asoc->flowlabel = params->spp_ipv6_flowlabel & SCTP_FLOWLABEL_VAL_MASK; asoc->flowlabel |= SCTP_FLOWLABEL_SET_MASK; @@ -2687,12 +2690,13 @@ static int sctp_apply_peer_addr_params(struct sctp_paddrparams *params, trans->dscp = params->spp_dscp & SCTP_DSCP_VAL_MASK; trans->dscp |= SCTP_DSCP_SET_MASK; } else if (asoc) { - list_for_each_entry(trans, - &asoc->peer.transport_addr_list, + struct sctp_transport *t; + + list_for_each_entry(t, &asoc->peer.transport_addr_list, transports) { - trans->dscp = params->spp_dscp & - SCTP_DSCP_VAL_MASK; - trans->dscp |= SCTP_DSCP_SET_MASK; + t->dscp = params->spp_dscp & + SCTP_DSCP_VAL_MASK; + t->dscp |= SCTP_DSCP_SET_MASK; } asoc->dscp = params->spp_dscp & SCTP_DSCP_VAL_MASK; asoc->dscp |= SCTP_DSCP_SET_MASK; @@ -5005,9 +5009,14 @@ struct sctp_transport *sctp_transport_get_next(struct net *net, break; } + if (!sctp_transport_hold(t)) + continue; + if (net_eq(sock_net(t->asoc->base.sk), net) && t->asoc->peer.primary_path == t) break; + + sctp_transport_put(t); } return t; @@ -5017,13 +5026,18 @@ struct sctp_transport *sctp_transport_get_idx(struct net *net, struct rhashtable_iter *iter, int pos) { - void *obj = SEQ_START_TOKEN; + struct sctp_transport *t; - while (pos && (obj = sctp_transport_get_next(net, iter)) && - !IS_ERR(obj)) - pos--; + if (!pos) + return SEQ_START_TOKEN; - return obj; + while ((t = sctp_transport_get_next(net, iter)) && !IS_ERR(t)) { + if (!--pos) + break; + sctp_transport_put(t); + } + + return t; } int sctp_for_each_endpoint(int (*cb)(struct sctp_endpoint *, void *), @@ -5082,8 +5096,6 @@ again: tsp = sctp_transport_get_idx(net, &hti, *pos + 1); for (; !IS_ERR_OR_NULL(tsp); tsp = sctp_transport_get_next(net, &hti)) { - if (!sctp_transport_hold(tsp)) - continue; ret = cb(tsp, p); if (ret) break; diff --git a/net/sctp/transport.c b/net/sctp/transport.c index 12cac85da994..033696e6f74f 100644 --- a/net/sctp/transport.c +++ b/net/sctp/transport.c @@ -260,6 +260,7 @@ void sctp_transport_pmtu(struct sctp_transport *transport, struct sock *sk) bool sctp_transport_update_pmtu(struct sctp_transport *t, u32 pmtu) { struct dst_entry *dst = sctp_transport_dst_check(t); + struct sock *sk = t->asoc->base.sk; bool change = true; if (unlikely(pmtu < SCTP_DEFAULT_MINSEGMENT)) { @@ -271,12 +272,19 @@ bool sctp_transport_update_pmtu(struct sctp_transport *t, u32 pmtu) pmtu = SCTP_TRUNC4(pmtu); if (dst) { - dst->ops->update_pmtu(dst, t->asoc->base.sk, NULL, pmtu); + struct sctp_pf *pf = sctp_get_pf_specific(dst->ops->family); + union sctp_addr addr; + + pf->af->from_sk(&addr, sk); + pf->to_sk_daddr(&t->ipaddr, sk); + dst->ops->update_pmtu(dst, sk, NULL, pmtu); + pf->to_sk_daddr(&addr, sk); + dst = sctp_transport_dst_check(t); } if (!dst) { - t->af_specific->get_dst(t, &t->saddr, &t->fl, t->asoc->base.sk); + t->af_specific->get_dst(t, &t->saddr, &t->fl, sk); dst = t->dst; } diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 2d8a1e15e4f9..015231789ed2 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -742,7 +742,10 @@ static void smc_connect_work(struct work_struct *work) smc->sk.sk_err = -rc; out: - smc->sk.sk_state_change(&smc->sk); + if (smc->sk.sk_err) + smc->sk.sk_state_change(&smc->sk); + else + smc->sk.sk_write_space(&smc->sk); kfree(smc->connect_info); smc->connect_info = NULL; release_sock(&smc->sk); @@ -1150,9 +1153,9 @@ static int smc_listen_rdma_reg(struct smc_sock *new_smc, int local_contact) } /* listen worker: finish RDMA setup */ -static void smc_listen_rdma_finish(struct smc_sock *new_smc, - struct smc_clc_msg_accept_confirm *cclc, - int local_contact) +static int smc_listen_rdma_finish(struct smc_sock *new_smc, + struct smc_clc_msg_accept_confirm *cclc, + int local_contact) { struct smc_link *link = &new_smc->conn.lgr->lnk[SMC_SINGLE_LINK]; int reason_code = 0; @@ -1175,11 +1178,12 @@ static void smc_listen_rdma_finish(struct smc_sock *new_smc, if (reason_code) goto decline; } - return; + return 0; decline: mutex_unlock(&smc_create_lgr_pending); smc_listen_decline(new_smc, reason_code, local_contact); + return reason_code; } /* setup for RDMA connection of server */ @@ -1276,8 +1280,10 @@ static void smc_listen_work(struct work_struct *work) } /* finish worker */ - if (!ism_supported) - smc_listen_rdma_finish(new_smc, &cclc, local_contact); + if (!ism_supported) { + if (smc_listen_rdma_finish(new_smc, &cclc, local_contact)) + return; + } smc_conn_save_peer_info(new_smc, &cclc); mutex_unlock(&smc_create_lgr_pending); smc_listen_out_connected(new_smc); @@ -1529,7 +1535,7 @@ static __poll_t smc_poll(struct file *file, struct socket *sock, return EPOLLNVAL; smc = smc_sk(sock->sk); - if ((sk->sk_state == SMC_INIT) || smc->use_fallback) { + if (smc->use_fallback) { /* delegate to CLC child sock */ mask = smc->clcsock->ops->poll(file, smc->clcsock, wait); sk->sk_err = smc->clcsock->sk->sk_err; @@ -1560,9 +1566,9 @@ static __poll_t smc_poll(struct file *file, struct socket *sock, mask |= EPOLLIN | EPOLLRDNORM | EPOLLRDHUP; if (sk->sk_state == SMC_APPCLOSEWAIT1) mask |= EPOLLIN; + if (smc->conn.urg_state == SMC_URG_VALID) + mask |= EPOLLPRI; } - if (smc->conn.urg_state == SMC_URG_VALID) - mask |= EPOLLPRI; } return mask; diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index 83aba9ade060..52241d679cc9 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -446,14 +446,12 @@ int smc_clc_send_proposal(struct smc_sock *smc, int smc_type, vec[i++].iov_len = sizeof(trl); /* due to the few bytes needed for clc-handshake this cannot block */ len = kernel_sendmsg(smc->clcsock, &msg, vec, i, plen); - if (len < sizeof(pclc)) { - if (len >= 0) { - reason_code = -ENETUNREACH; - smc->sk.sk_err = -reason_code; - } else { - smc->sk.sk_err = smc->clcsock->sk->sk_err; - reason_code = -smc->sk.sk_err; - } + if (len < 0) { + smc->sk.sk_err = smc->clcsock->sk->sk_err; + reason_code = -smc->sk.sk_err; + } else if (len < (int)sizeof(pclc)) { + reason_code = -ENETUNREACH; + smc->sk.sk_err = -reason_code; } return reason_code; diff --git a/net/smc/smc_close.c b/net/smc/smc_close.c index ac961dfb1ea1..ea2b87f29469 100644 --- a/net/smc/smc_close.c +++ b/net/smc/smc_close.c @@ -100,15 +100,14 @@ static void smc_close_active_abort(struct smc_sock *smc) struct smc_cdc_conn_state_flags *txflags = &smc->conn.local_tx_ctrl.conn_state_flags; - sk->sk_err = ECONNABORTED; - if (smc->clcsock && smc->clcsock->sk) { - smc->clcsock->sk->sk_err = ECONNABORTED; - smc->clcsock->sk->sk_state_change(smc->clcsock->sk); + if (sk->sk_state != SMC_INIT && smc->clcsock && smc->clcsock->sk) { + sk->sk_err = ECONNABORTED; + if (smc->clcsock && smc->clcsock->sk) { + smc->clcsock->sk->sk_err = ECONNABORTED; + smc->clcsock->sk->sk_state_change(smc->clcsock->sk); + } } switch (sk->sk_state) { - case SMC_INIT: - sk->sk_state = SMC_PEERABORTWAIT; - break; case SMC_ACTIVE: sk->sk_state = SMC_PEERABORTWAIT; release_sock(sk); @@ -143,6 +142,7 @@ static void smc_close_active_abort(struct smc_sock *smc) case SMC_PEERFINCLOSEWAIT: sock_put(sk); /* passive closing */ break; + case SMC_INIT: case SMC_PEERABORTWAIT: case SMC_CLOSED: break; diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c index 01c6ce042a1c..7cb3e4f07c10 100644 --- a/net/smc/smc_pnet.c +++ b/net/smc/smc_pnet.c @@ -461,7 +461,7 @@ static const struct genl_ops smc_pnet_ops[] = { }; /* SMC_PNETID family definition */ -static struct genl_family smc_pnet_nl_family = { +static struct genl_family smc_pnet_nl_family __ro_after_init = { .hdrsize = 0, .name = SMCR_GENL_FAMILY_NAME, .version = SMCR_GENL_FAMILY_VERSION, diff --git a/net/socket.c b/net/socket.c index e6945e318f02..01f3f8f32d6f 100644 --- a/net/socket.c +++ b/net/socket.c @@ -941,7 +941,8 @@ void dlci_ioctl_set(int (*hook) (unsigned int, void __user *)) EXPORT_SYMBOL(dlci_ioctl_set); static long sock_do_ioctl(struct net *net, struct socket *sock, - unsigned int cmd, unsigned long arg) + unsigned int cmd, unsigned long arg, + unsigned int ifreq_size) { int err; void __user *argp = (void __user *)arg; @@ -967,11 +968,11 @@ static long sock_do_ioctl(struct net *net, struct socket *sock, } else { struct ifreq ifr; bool need_copyout; - if (copy_from_user(&ifr, argp, sizeof(struct ifreq))) + if (copy_from_user(&ifr, argp, ifreq_size)) return -EFAULT; err = dev_ioctl(net, cmd, &ifr, &need_copyout); if (!err && need_copyout) - if (copy_to_user(argp, &ifr, sizeof(struct ifreq))) + if (copy_to_user(argp, &ifr, ifreq_size)) return -EFAULT; } return err; @@ -1070,7 +1071,8 @@ static long sock_ioctl(struct file *file, unsigned cmd, unsigned long arg) err = open_related_ns(&net->ns, get_net_ns); break; default: - err = sock_do_ioctl(net, sock, cmd, arg); + err = sock_do_ioctl(net, sock, cmd, arg, + sizeof(struct ifreq)); break; } return err; @@ -2750,7 +2752,8 @@ static int do_siocgstamp(struct net *net, struct socket *sock, int err; set_fs(KERNEL_DS); - err = sock_do_ioctl(net, sock, cmd, (unsigned long)&ktv); + err = sock_do_ioctl(net, sock, cmd, (unsigned long)&ktv, + sizeof(struct compat_ifreq)); set_fs(old_fs); if (!err) err = compat_put_timeval(&ktv, up); @@ -2766,7 +2769,8 @@ static int do_siocgstampns(struct net *net, struct socket *sock, int err; set_fs(KERNEL_DS); - err = sock_do_ioctl(net, sock, cmd, (unsigned long)&kts); + err = sock_do_ioctl(net, sock, cmd, (unsigned long)&kts, + sizeof(struct compat_ifreq)); set_fs(old_fs); if (!err) err = compat_put_timespec(&kts, up); @@ -3072,7 +3076,8 @@ static int routing_ioctl(struct net *net, struct socket *sock, } set_fs(KERNEL_DS); - ret = sock_do_ioctl(net, sock, cmd, (unsigned long) r); + ret = sock_do_ioctl(net, sock, cmd, (unsigned long) r, + sizeof(struct compat_ifreq)); set_fs(old_fs); out: @@ -3185,7 +3190,8 @@ static int compat_sock_ioctl_trans(struct file *file, struct socket *sock, case SIOCBONDSETHWADDR: case SIOCBONDCHANGEACTIVE: case SIOCGIFNAME: - return sock_do_ioctl(net, sock, cmd, arg); + return sock_do_ioctl(net, sock, cmd, arg, + sizeof(struct compat_ifreq)); } return -ENOIOCTLCMD; diff --git a/net/tipc/bcast.c b/net/tipc/bcast.c index 9ee6cfea56dd..d8026543bf4c 100644 --- a/net/tipc/bcast.c +++ b/net/tipc/bcast.c @@ -51,12 +51,12 @@ const char tipc_bclink_name[] = "broadcast-link"; * struct tipc_bc_base - base structure for keeping broadcast send state * @link: broadcast send link structure * @inputq: data input queue; will only carry SOCK_WAKEUP messages - * @dest: array keeping number of reachable destinations per bearer + * @dests: array keeping number of reachable destinations per bearer * @primary_bearer: a bearer having links to all broadcast destinations, if any * @bcast_support: indicates if primary bearer, if any, supports broadcast * @rcast_support: indicates if all peer nodes support replicast * @rc_ratio: dest count as percentage of cluster size where send method changes - * @bc_threshold: calculated drom rc_ratio; if dests > threshold use broadcast + * @bc_threshold: calculated from rc_ratio; if dests > threshold use broadcast */ struct tipc_bc_base { struct tipc_link *link; diff --git a/net/tipc/diag.c b/net/tipc/diag.c index aaabb0b776dd..73137f4aeb68 100644 --- a/net/tipc/diag.c +++ b/net/tipc/diag.c @@ -84,7 +84,9 @@ static int tipc_sock_diag_handler_dump(struct sk_buff *skb, if (h->nlmsg_flags & NLM_F_DUMP) { struct netlink_dump_control c = { + .start = tipc_dump_start, .dump = tipc_diag_dump, + .done = tipc_dump_done, }; netlink_dump_start(net->diag_nlsk, skb, h, &c); return 0; diff --git a/net/tipc/name_table.c b/net/tipc/name_table.c index 88f027b502f6..66d5b2c5987a 100644 --- a/net/tipc/name_table.c +++ b/net/tipc/name_table.c @@ -980,20 +980,17 @@ int tipc_nl_name_table_dump(struct sk_buff *skb, struct netlink_callback *cb) struct tipc_dest *tipc_dest_find(struct list_head *l, u32 node, u32 port) { - u64 value = (u64)node << 32 | port; struct tipc_dest *dst; list_for_each_entry(dst, l, list) { - if (dst->value != value) - continue; - return dst; + if (dst->node == node && dst->port == port) + return dst; } return NULL; } bool tipc_dest_push(struct list_head *l, u32 node, u32 port) { - u64 value = (u64)node << 32 | port; struct tipc_dest *dst; if (tipc_dest_find(l, node, port)) @@ -1002,7 +999,8 @@ bool tipc_dest_push(struct list_head *l, u32 node, u32 port) dst = kmalloc(sizeof(*dst), GFP_ATOMIC); if (unlikely(!dst)) return false; - dst->value = value; + dst->node = node; + dst->port = port; list_add(&dst->list, l); return true; } diff --git a/net/tipc/name_table.h b/net/tipc/name_table.h index 0febba41da86..892bd750b85f 100644 --- a/net/tipc/name_table.h +++ b/net/tipc/name_table.h @@ -133,13 +133,8 @@ void tipc_nametbl_stop(struct net *net); struct tipc_dest { struct list_head list; - union { - struct { - u32 port; - u32 node; - }; - u64 value; - }; + u32 port; + u32 node; }; struct tipc_dest *tipc_dest_find(struct list_head *l, u32 node, u32 port); diff --git a/net/tipc/netlink.c b/net/tipc/netlink.c index 6ff2254088f6..99ee419210ba 100644 --- a/net/tipc/netlink.c +++ b/net/tipc/netlink.c @@ -167,7 +167,9 @@ static const struct genl_ops tipc_genl_v2_ops[] = { }, { .cmd = TIPC_NL_SOCK_GET, + .start = tipc_dump_start, .dumpit = tipc_nl_sk_dump, + .done = tipc_dump_done, .policy = tipc_nl_policy, }, { diff --git a/net/tipc/netlink_compat.c b/net/tipc/netlink_compat.c index a2f76743c73a..6376467e78f8 100644 --- a/net/tipc/netlink_compat.c +++ b/net/tipc/netlink_compat.c @@ -185,6 +185,10 @@ static int __tipc_nl_compat_dumpit(struct tipc_nl_compat_cmd_dump *cmd, return -ENOMEM; buf->sk = msg->dst_sk; + if (__tipc_dump_start(&cb, msg->net)) { + kfree_skb(buf); + return -ENOMEM; + } do { int rem; @@ -216,6 +220,7 @@ static int __tipc_nl_compat_dumpit(struct tipc_nl_compat_cmd_dump *cmd, err = 0; err_out: + tipc_dump_done(&cb); kfree_skb(buf); if (err == -EMSGSIZE) { diff --git a/net/tipc/socket.c b/net/tipc/socket.c index c1e93c9515bc..3f03ddd0e35b 100644 --- a/net/tipc/socket.c +++ b/net/tipc/socket.c @@ -576,6 +576,7 @@ static int tipc_release(struct socket *sock) sk_stop_timer(sk, &sk->sk_timer); tipc_sk_remove(tsk); + sock_orphan(sk); /* Reject any messages that accumulated in backlog queue */ release_sock(sk); tipc_dest_list_purge(&tsk->cong_links); @@ -2672,6 +2673,8 @@ void tipc_sk_reinit(struct net *net) rhashtable_walk_stop(&iter); } while (tsk == ERR_PTR(-EAGAIN)); + + rhashtable_walk_exit(&iter); } static struct tipc_sock *tipc_sk_lookup(struct net *net, u32 portid) @@ -3227,45 +3230,74 @@ int tipc_nl_sk_walk(struct sk_buff *skb, struct netlink_callback *cb, struct netlink_callback *cb, struct tipc_sock *tsk)) { - struct net *net = sock_net(skb->sk); - struct tipc_net *tn = tipc_net(net); - const struct bucket_table *tbl; - u32 prev_portid = cb->args[1]; - u32 tbl_id = cb->args[0]; - struct rhash_head *pos; + struct rhashtable_iter *iter = (void *)cb->args[4]; struct tipc_sock *tsk; int err; - rcu_read_lock(); - tbl = rht_dereference_rcu((&tn->sk_rht)->tbl, &tn->sk_rht); - for (; tbl_id < tbl->size; tbl_id++) { - rht_for_each_entry_rcu(tsk, pos, tbl, tbl_id, node) { - spin_lock_bh(&tsk->sk.sk_lock.slock); - if (prev_portid && prev_portid != tsk->portid) { - spin_unlock_bh(&tsk->sk.sk_lock.slock); + rhashtable_walk_start(iter); + while ((tsk = rhashtable_walk_next(iter)) != NULL) { + if (IS_ERR(tsk)) { + err = PTR_ERR(tsk); + if (err == -EAGAIN) { + err = 0; continue; } + break; + } - err = skb_handler(skb, cb, tsk); - if (err) { - prev_portid = tsk->portid; - spin_unlock_bh(&tsk->sk.sk_lock.slock); - goto out; - } - - prev_portid = 0; - spin_unlock_bh(&tsk->sk.sk_lock.slock); + sock_hold(&tsk->sk); + rhashtable_walk_stop(iter); + lock_sock(&tsk->sk); + err = skb_handler(skb, cb, tsk); + if (err) { + release_sock(&tsk->sk); + sock_put(&tsk->sk); + goto out; } + release_sock(&tsk->sk); + rhashtable_walk_start(iter); + sock_put(&tsk->sk); } + rhashtable_walk_stop(iter); out: - rcu_read_unlock(); - cb->args[0] = tbl_id; - cb->args[1] = prev_portid; - return skb->len; } EXPORT_SYMBOL(tipc_nl_sk_walk); +int tipc_dump_start(struct netlink_callback *cb) +{ + return __tipc_dump_start(cb, sock_net(cb->skb->sk)); +} +EXPORT_SYMBOL(tipc_dump_start); + +int __tipc_dump_start(struct netlink_callback *cb, struct net *net) +{ + /* tipc_nl_name_table_dump() uses cb->args[0...3]. */ + struct rhashtable_iter *iter = (void *)cb->args[4]; + struct tipc_net *tn = tipc_net(net); + + if (!iter) { + iter = kmalloc(sizeof(*iter), GFP_KERNEL); + if (!iter) + return -ENOMEM; + + cb->args[4] = (long)iter; + } + + rhashtable_walk_enter(&tn->sk_rht, iter); + return 0; +} + +int tipc_dump_done(struct netlink_callback *cb) +{ + struct rhashtable_iter *hti = (void *)cb->args[4]; + + rhashtable_walk_exit(hti); + kfree(hti); + return 0; +} +EXPORT_SYMBOL(tipc_dump_done); + int tipc_sk_fill_sock_diag(struct sk_buff *skb, struct netlink_callback *cb, struct tipc_sock *tsk, u32 sk_filter_state, u64 (*tipc_diag_gen_cookie)(struct sock *sk)) diff --git a/net/tipc/socket.h b/net/tipc/socket.h index aff9b2ae5a1f..5e575f205afe 100644 --- a/net/tipc/socket.h +++ b/net/tipc/socket.h @@ -68,4 +68,7 @@ int tipc_nl_sk_walk(struct sk_buff *skb, struct netlink_callback *cb, int (*skb_handler)(struct sk_buff *skb, struct netlink_callback *cb, struct tipc_sock *tsk)); +int tipc_dump_start(struct netlink_callback *cb); +int __tipc_dump_start(struct netlink_callback *cb, struct net *net); +int tipc_dump_done(struct netlink_callback *cb); #endif diff --git a/net/tipc/topsrv.c b/net/tipc/topsrv.c index c8e34ef22c30..2627b5d812e9 100644 --- a/net/tipc/topsrv.c +++ b/net/tipc/topsrv.c @@ -313,8 +313,8 @@ static void tipc_conn_send_work(struct work_struct *work) conn_put(con); } -/* tipc_conn_queue_evt() - interrupt level call from a subscription instance - * The queued work is launched into tipc_send_work()->tipc_send_to_sock() +/* tipc_topsrv_queue_evt() - interrupt level call from a subscription instance + * The queued work is launched into tipc_conn_send_work()->tipc_conn_send_to_sock() */ void tipc_topsrv_queue_evt(struct net *net, int conid, u32 event, struct tipc_event *evt) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 292742e50bfa..961b07d4d41c 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -686,7 +686,7 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx) goto free_marker_record; } - crypto_info = &ctx->crypto_send; + crypto_info = &ctx->crypto_send.info; switch (crypto_info->cipher_type) { case TLS_CIPHER_AES_GCM_128: nonce_size = TLS_CIPHER_AES_GCM_128_IV_SIZE; @@ -780,7 +780,7 @@ int tls_set_device_offload(struct sock *sk, struct tls_context *ctx) ctx->priv_ctx_tx = offload_ctx; rc = netdev->tlsdev_ops->tls_dev_add(netdev, sk, TLS_OFFLOAD_CTX_DIR_TX, - &ctx->crypto_send, + &ctx->crypto_send.info, tcp_sk(sk)->write_seq); if (rc) goto release_netdev; @@ -862,7 +862,7 @@ int tls_set_device_offload_rx(struct sock *sk, struct tls_context *ctx) goto release_ctx; rc = netdev->tlsdev_ops->tls_dev_add(netdev, sk, TLS_OFFLOAD_CTX_DIR_RX, - &ctx->crypto_recv, + &ctx->crypto_recv.info, tcp_sk(sk)->copied_seq); if (rc) { pr_err_ratelimited("%s: The netdev has refused to offload this socket\n", diff --git a/net/tls/tls_device_fallback.c b/net/tls/tls_device_fallback.c index 6102169239d1..450a6dbc5a88 100644 --- a/net/tls/tls_device_fallback.c +++ b/net/tls/tls_device_fallback.c @@ -320,7 +320,7 @@ static struct sk_buff *tls_enc_skb(struct tls_context *tls_ctx, goto free_req; iv = buf; - memcpy(iv, tls_ctx->crypto_send_aes_gcm_128.salt, + memcpy(iv, tls_ctx->crypto_send.aes_gcm_128.salt, TLS_CIPHER_AES_GCM_128_SALT_SIZE); aad = buf + TLS_CIPHER_AES_GCM_128_SALT_SIZE + TLS_CIPHER_AES_GCM_128_IV_SIZE; diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index 180b6640e531..523622dc74f8 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -241,6 +241,16 @@ static void tls_write_space(struct sock *sk) ctx->sk_write_space(sk); } +static void tls_ctx_free(struct tls_context *ctx) +{ + if (!ctx) + return; + + memzero_explicit(&ctx->crypto_send, sizeof(ctx->crypto_send)); + memzero_explicit(&ctx->crypto_recv, sizeof(ctx->crypto_recv)); + kfree(ctx); +} + static void tls_sk_proto_close(struct sock *sk, long timeout) { struct tls_context *ctx = tls_get_ctx(sk); @@ -294,7 +304,7 @@ static void tls_sk_proto_close(struct sock *sk, long timeout) #else { #endif - kfree(ctx); + tls_ctx_free(ctx); ctx = NULL; } @@ -305,7 +315,7 @@ skip_tx_cleanup: * for sk->sk_prot->unhash [tls_hw_unhash] */ if (free_ctx) - kfree(ctx); + tls_ctx_free(ctx); } static int do_tls_getsockopt_tx(struct sock *sk, char __user *optval, @@ -330,7 +340,7 @@ static int do_tls_getsockopt_tx(struct sock *sk, char __user *optval, } /* get user crypto info */ - crypto_info = &ctx->crypto_send; + crypto_info = &ctx->crypto_send.info; if (!TLS_CRYPTO_INFO_READY(crypto_info)) { rc = -EBUSY; @@ -417,9 +427,9 @@ static int do_tls_setsockopt_conf(struct sock *sk, char __user *optval, } if (tx) - crypto_info = &ctx->crypto_send; + crypto_info = &ctx->crypto_send.info; else - crypto_info = &ctx->crypto_recv; + crypto_info = &ctx->crypto_recv.info; /* Currently we don't support set crypto info more than one time */ if (TLS_CRYPTO_INFO_READY(crypto_info)) { @@ -499,7 +509,7 @@ static int do_tls_setsockopt_conf(struct sock *sk, char __user *optval, goto out; err_crypto_info: - memset(crypto_info, 0, sizeof(*crypto_info)); + memzero_explicit(crypto_info, sizeof(union tls_crypto_context)); out: return rc; } diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 52fbe727d7c1..b9c6ecfbcfea 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -125,6 +125,9 @@ static int alloc_encrypted_sg(struct sock *sk, int len) &ctx->sg_encrypted_num_elem, &ctx->sg_encrypted_size, 0); + if (rc == -ENOSPC) + ctx->sg_encrypted_num_elem = ARRAY_SIZE(ctx->sg_encrypted_data); + return rc; } @@ -138,6 +141,9 @@ static int alloc_plaintext_sg(struct sock *sk, int len) &ctx->sg_plaintext_num_elem, &ctx->sg_plaintext_size, tls_ctx->pending_open_record_frags); + if (rc == -ENOSPC) + ctx->sg_plaintext_num_elem = ARRAY_SIZE(ctx->sg_plaintext_data); + return rc; } @@ -925,7 +931,15 @@ int tls_sw_recvmsg(struct sock *sk, if (control != TLS_RECORD_TYPE_DATA) goto recv_end; } + } else { + /* MSG_PEEK right now cannot look beyond current skb + * from strparser, meaning we cannot advance skb here + * and thus unpause strparser since we'd loose original + * one. + */ + break; } + /* If we have a new message from strparser, continue now. */ if (copied >= target && !ctx->recv_pkt) break; @@ -1049,8 +1063,8 @@ static int tls_read_size(struct strparser *strp, struct sk_buff *skb) goto read_failure; } - if (header[1] != TLS_VERSION_MINOR(tls_ctx->crypto_recv.version) || - header[2] != TLS_VERSION_MAJOR(tls_ctx->crypto_recv.version)) { + if (header[1] != TLS_VERSION_MINOR(tls_ctx->crypto_recv.info.version) || + header[2] != TLS_VERSION_MAJOR(tls_ctx->crypto_recv.info.version)) { ret = -EINVAL; goto read_failure; } @@ -1130,7 +1144,6 @@ void tls_sw_free_resources_rx(struct sock *sk) int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx) { - char keyval[TLS_CIPHER_AES_GCM_128_KEY_SIZE]; struct tls_crypto_info *crypto_info; struct tls12_crypto_info_aes_gcm_128 *gcm_128_info; struct tls_sw_context_tx *sw_ctx_tx = NULL; @@ -1175,12 +1188,12 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx) if (tx) { crypto_init_wait(&sw_ctx_tx->async_wait); - crypto_info = &ctx->crypto_send; + crypto_info = &ctx->crypto_send.info; cctx = &ctx->tx; aead = &sw_ctx_tx->aead_send; } else { crypto_init_wait(&sw_ctx_rx->async_wait); - crypto_info = &ctx->crypto_recv; + crypto_info = &ctx->crypto_recv.info; cctx = &ctx->rx; aead = &sw_ctx_rx->aead_recv; } @@ -1259,9 +1272,7 @@ int tls_set_sw_offload(struct sock *sk, struct tls_context *ctx, int tx) ctx->push_pending_record = tls_sw_push_pending_record; - memcpy(keyval, gcm_128_info->key, TLS_CIPHER_AES_GCM_128_KEY_SIZE); - - rc = crypto_aead_setkey(*aead, keyval, + rc = crypto_aead_setkey(*aead, gcm_128_info->key, TLS_CIPHER_AES_GCM_128_KEY_SIZE); if (rc) goto free_aead; diff --git a/net/wireless/nl80211.c b/net/wireless/nl80211.c index 5fb9b7dd9831..4b8ec659e797 100644 --- a/net/wireless/nl80211.c +++ b/net/wireless/nl80211.c @@ -669,13 +669,13 @@ static int nl80211_msg_put_wmm_rules(struct sk_buff *msg, goto nla_put_failure; if (nla_put_u16(msg, NL80211_WMMR_CW_MIN, - rule->wmm_rule->client[j].cw_min) || + rule->wmm_rule.client[j].cw_min) || nla_put_u16(msg, NL80211_WMMR_CW_MAX, - rule->wmm_rule->client[j].cw_max) || + rule->wmm_rule.client[j].cw_max) || nla_put_u8(msg, NL80211_WMMR_AIFSN, - rule->wmm_rule->client[j].aifsn) || - nla_put_u8(msg, NL80211_WMMR_TXOP, - rule->wmm_rule->client[j].cot)) + rule->wmm_rule.client[j].aifsn) || + nla_put_u16(msg, NL80211_WMMR_TXOP, + rule->wmm_rule.client[j].cot)) goto nla_put_failure; nla_nest_end(msg, nl_wmm_rule); @@ -766,9 +766,9 @@ static int nl80211_msg_put_channel(struct sk_buff *msg, struct wiphy *wiphy, if (large) { const struct ieee80211_reg_rule *rule = - freq_reg_info(wiphy, chan->center_freq); + freq_reg_info(wiphy, MHZ_TO_KHZ(chan->center_freq)); - if (!IS_ERR(rule) && rule->wmm_rule) { + if (!IS_ERR_OR_NULL(rule) && rule->has_wmm) { if (nl80211_msg_put_wmm_rules(msg, rule)) goto nla_put_failure; } @@ -12205,6 +12205,7 @@ static int nl80211_update_ft_ies(struct sk_buff *skb, struct genl_info *info) return -EOPNOTSUPP; if (!info->attrs[NL80211_ATTR_MDID] || + !info->attrs[NL80211_ATTR_IE] || !is_valid_ie_attr(info->attrs[NL80211_ATTR_IE])) return -EINVAL; diff --git a/net/wireless/reg.c b/net/wireless/reg.c index 4fc66a117b7d..2f702adf2912 100644 --- a/net/wireless/reg.c +++ b/net/wireless/reg.c @@ -425,36 +425,23 @@ static const struct ieee80211_regdomain * reg_copy_regd(const struct ieee80211_regdomain *src_regd) { struct ieee80211_regdomain *regd; - int size_of_regd, size_of_wmms; + int size_of_regd; unsigned int i; - struct ieee80211_wmm_rule *d_wmm, *s_wmm; size_of_regd = sizeof(struct ieee80211_regdomain) + src_regd->n_reg_rules * sizeof(struct ieee80211_reg_rule); - size_of_wmms = src_regd->n_wmm_rules * - sizeof(struct ieee80211_wmm_rule); - regd = kzalloc(size_of_regd + size_of_wmms, GFP_KERNEL); + regd = kzalloc(size_of_regd, GFP_KERNEL); if (!regd) return ERR_PTR(-ENOMEM); memcpy(regd, src_regd, sizeof(struct ieee80211_regdomain)); - d_wmm = (struct ieee80211_wmm_rule *)((u8 *)regd + size_of_regd); - s_wmm = (struct ieee80211_wmm_rule *)((u8 *)src_regd + size_of_regd); - memcpy(d_wmm, s_wmm, size_of_wmms); - - for (i = 0; i < src_regd->n_reg_rules; i++) { + for (i = 0; i < src_regd->n_reg_rules; i++) memcpy(®d->reg_rules[i], &src_regd->reg_rules[i], sizeof(struct ieee80211_reg_rule)); - if (!src_regd->reg_rules[i].wmm_rule) - continue; - regd->reg_rules[i].wmm_rule = d_wmm + - (src_regd->reg_rules[i].wmm_rule - s_wmm) / - sizeof(struct ieee80211_wmm_rule); - } return regd; } @@ -860,9 +847,10 @@ static bool valid_regdb(const u8 *data, unsigned int size) return true; } -static void set_wmm_rule(struct ieee80211_wmm_rule *rule, +static void set_wmm_rule(struct ieee80211_reg_rule *rrule, struct fwdb_wmm_rule *wmm) { + struct ieee80211_wmm_rule *rule = &rrule->wmm_rule; unsigned int i; for (i = 0; i < IEEE80211_NUM_ACS; i++) { @@ -876,11 +864,13 @@ static void set_wmm_rule(struct ieee80211_wmm_rule *rule, rule->ap[i].aifsn = wmm->ap[i].aifsn; rule->ap[i].cot = 1000 * be16_to_cpu(wmm->ap[i].cot); } + + rrule->has_wmm = true; } static int __regdb_query_wmm(const struct fwdb_header *db, const struct fwdb_country *country, int freq, - u32 *dbptr, struct ieee80211_wmm_rule *rule) + struct ieee80211_reg_rule *rule) { unsigned int ptr = be16_to_cpu(country->coll_ptr) << 2; struct fwdb_collection *coll = (void *)((u8 *)db + ptr); @@ -901,8 +891,6 @@ static int __regdb_query_wmm(const struct fwdb_header *db, wmm_ptr = be16_to_cpu(rrule->wmm_ptr) << 2; wmm = (void *)((u8 *)db + wmm_ptr); set_wmm_rule(rule, wmm); - if (dbptr) - *dbptr = wmm_ptr; return 0; } } @@ -910,8 +898,7 @@ static int __regdb_query_wmm(const struct fwdb_header *db, return -ENODATA; } -int reg_query_regdb_wmm(char *alpha2, int freq, u32 *dbptr, - struct ieee80211_wmm_rule *rule) +int reg_query_regdb_wmm(char *alpha2, int freq, struct ieee80211_reg_rule *rule) { const struct fwdb_header *hdr = regdb; const struct fwdb_country *country; @@ -925,8 +912,7 @@ int reg_query_regdb_wmm(char *alpha2, int freq, u32 *dbptr, country = &hdr->country[0]; while (country->coll_ptr) { if (alpha2_equal(alpha2, country->alpha2)) - return __regdb_query_wmm(regdb, country, freq, dbptr, - rule); + return __regdb_query_wmm(regdb, country, freq, rule); country++; } @@ -935,32 +921,13 @@ int reg_query_regdb_wmm(char *alpha2, int freq, u32 *dbptr, } EXPORT_SYMBOL(reg_query_regdb_wmm); -struct wmm_ptrs { - struct ieee80211_wmm_rule *rule; - u32 ptr; -}; - -static struct ieee80211_wmm_rule *find_wmm_ptr(struct wmm_ptrs *wmm_ptrs, - u32 wmm_ptr, int n_wmms) -{ - int i; - - for (i = 0; i < n_wmms; i++) { - if (wmm_ptrs[i].ptr == wmm_ptr) - return wmm_ptrs[i].rule; - } - return NULL; -} - static int regdb_query_country(const struct fwdb_header *db, const struct fwdb_country *country) { unsigned int ptr = be16_to_cpu(country->coll_ptr) << 2; struct fwdb_collection *coll = (void *)((u8 *)db + ptr); struct ieee80211_regdomain *regdom; - struct ieee80211_regdomain *tmp_rd; - unsigned int size_of_regd, i, n_wmms = 0; - struct wmm_ptrs *wmm_ptrs; + unsigned int size_of_regd, i; size_of_regd = sizeof(struct ieee80211_regdomain) + coll->n_rules * sizeof(struct ieee80211_reg_rule); @@ -969,12 +936,6 @@ static int regdb_query_country(const struct fwdb_header *db, if (!regdom) return -ENOMEM; - wmm_ptrs = kcalloc(coll->n_rules, sizeof(*wmm_ptrs), GFP_KERNEL); - if (!wmm_ptrs) { - kfree(regdom); - return -ENOMEM; - } - regdom->n_reg_rules = coll->n_rules; regdom->alpha2[0] = country->alpha2[0]; regdom->alpha2[1] = country->alpha2[1]; @@ -1013,37 +974,11 @@ static int regdb_query_country(const struct fwdb_header *db, 1000 * be16_to_cpu(rule->cac_timeout); if (rule->len >= offsetofend(struct fwdb_rule, wmm_ptr)) { u32 wmm_ptr = be16_to_cpu(rule->wmm_ptr) << 2; - struct ieee80211_wmm_rule *wmm_pos = - find_wmm_ptr(wmm_ptrs, wmm_ptr, n_wmms); - struct fwdb_wmm_rule *wmm; - struct ieee80211_wmm_rule *wmm_rule; - - if (wmm_pos) { - rrule->wmm_rule = wmm_pos; - continue; - } - wmm = (void *)((u8 *)db + wmm_ptr); - tmp_rd = krealloc(regdom, size_of_regd + (n_wmms + 1) * - sizeof(struct ieee80211_wmm_rule), - GFP_KERNEL); - - if (!tmp_rd) { - kfree(regdom); - kfree(wmm_ptrs); - return -ENOMEM; - } - regdom = tmp_rd; - - wmm_rule = (struct ieee80211_wmm_rule *) - ((u8 *)regdom + size_of_regd + n_wmms * - sizeof(struct ieee80211_wmm_rule)); + struct fwdb_wmm_rule *wmm = (void *)((u8 *)db + wmm_ptr); - set_wmm_rule(wmm_rule, wmm); - wmm_ptrs[n_wmms].ptr = wmm_ptr; - wmm_ptrs[n_wmms++].rule = wmm_rule; + set_wmm_rule(rrule, wmm); } } - kfree(wmm_ptrs); return reg_schedule_apply(regdom); } diff --git a/net/wireless/util.c b/net/wireless/util.c index e0825a019e9f..959ed3acd240 100644 --- a/net/wireless/util.c +++ b/net/wireless/util.c @@ -1456,7 +1456,7 @@ bool ieee80211_chandef_to_operating_class(struct cfg80211_chan_def *chandef, u8 *op_class) { u8 vht_opclass; - u16 freq = chandef->center_freq1; + u32 freq = chandef->center_freq1; if (freq >= 2412 && freq <= 2472) { if (chandef->width > NL80211_CHAN_WIDTH_40) diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index 5219280bf7ff..161b0224d6ae 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -380,6 +380,7 @@ our $Attribute = qr{ __noclone| __deprecated| __read_mostly| + __ro_after_init| __kprobes| $InitAttribute| ____cacheline_aligned| @@ -3311,7 +3312,7 @@ sub process { # known declaration macros $sline =~ /^\+\s+$declaration_macros/ || # start of struct or union or enum - $sline =~ /^\+\s+(?:union|struct|enum|typedef)\b/ || + $sline =~ /^\+\s+(?:static\s+)?(?:const\s+)?(?:union|struct|enum|typedef)\b/ || # start or end of block or continuation of declaration $sline =~ /^\+\s+(?:$|[\{\}\.\#\"\?\:\(\[])/ || # bitfield continuation diff --git a/scripts/depmod.sh b/scripts/depmod.sh index 999d585eaa73..e083bcae343f 100755 --- a/scripts/depmod.sh +++ b/scripts/depmod.sh @@ -11,13 +11,14 @@ DEPMOD=$1 KERNELRELEASE=$2 if ! test -r System.map ; then + echo "Warning: modules_install: missing 'System.map' file. Skipping depmod." >&2 exit 0 fi if [ -z $(command -v $DEPMOD) ]; then - echo "'make modules_install' requires $DEPMOD. Please install it." >&2 + echo "Warning: 'make modules_install' requires $DEPMOD. Please install it." >&2 echo "This is probably in the kmod package." >&2 - exit 1 + exit 0 fi # older versions of depmod require the version string to start with three diff --git a/scripts/kconfig/Makefile b/scripts/kconfig/Makefile index 4a7bd2192073..67ed9f6ccdf8 100644 --- a/scripts/kconfig/Makefile +++ b/scripts/kconfig/Makefile @@ -221,7 +221,6 @@ $(obj)/zconf.tab.o: $(obj)/zconf.lex.c # check if necessary packages are available, and configure build flags define filechk_conf_cfg - $(CONFIG_SHELL) $(srctree)/scripts/kconfig/check-pkgconfig.sh; \ $(CONFIG_SHELL) $< endef diff --git a/scripts/kconfig/check-pkgconfig.sh b/scripts/kconfig/check-pkgconfig.sh deleted file mode 100644 index 7a1c40bfb58c..000000000000 --- a/scripts/kconfig/check-pkgconfig.sh +++ /dev/null @@ -1,8 +0,0 @@ -#!/bin/sh -# SPDX-License-Identifier: GPL-2.0 -# Check for pkg-config presence - -if [ -z $(command -v pkg-config) ]; then - echo "'make *config' requires 'pkg-config'. Please install it." 1>&2 - exit 1 -fi diff --git a/scripts/kconfig/gconf-cfg.sh b/scripts/kconfig/gconf-cfg.sh index 533b3d8f8f08..480ecd8b9f41 100755 --- a/scripts/kconfig/gconf-cfg.sh +++ b/scripts/kconfig/gconf-cfg.sh @@ -3,6 +3,13 @@ PKG="gtk+-2.0 gmodule-2.0 libglade-2.0" +if [ -z "$(command -v pkg-config)" ]; then + echo >&2 "*" + echo >&2 "* 'make gconfig' requires 'pkg-config'. Please install it." + echo >&2 "*" + exit 1 +fi + if ! pkg-config --exists $PKG; then echo >&2 "*" echo >&2 "* Unable to find the GTK+ installation. Please make sure that" diff --git a/scripts/kconfig/mconf-cfg.sh b/scripts/kconfig/mconf-cfg.sh index e6f9facd0077..c812872d7f9d 100755 --- a/scripts/kconfig/mconf-cfg.sh +++ b/scripts/kconfig/mconf-cfg.sh @@ -4,20 +4,23 @@ PKG="ncursesw" PKG2="ncurses" -if pkg-config --exists $PKG; then - echo cflags=\"$(pkg-config --cflags $PKG)\" - echo libs=\"$(pkg-config --libs $PKG)\" - exit 0 -fi +if [ -n "$(command -v pkg-config)" ]; then + if pkg-config --exists $PKG; then + echo cflags=\"$(pkg-config --cflags $PKG)\" + echo libs=\"$(pkg-config --libs $PKG)\" + exit 0 + fi -if pkg-config --exists $PKG2; then - echo cflags=\"$(pkg-config --cflags $PKG2)\" - echo libs=\"$(pkg-config --libs $PKG2)\" - exit 0 + if pkg-config --exists $PKG2; then + echo cflags=\"$(pkg-config --cflags $PKG2)\" + echo libs=\"$(pkg-config --libs $PKG2)\" + exit 0 + fi fi -# Unfortunately, some distributions (e.g. openSUSE) cannot find ncurses -# by pkg-config. +# Check the default paths in case pkg-config is not installed. +# (Even if it is installed, some distributions such as openSUSE cannot +# find ncurses by pkg-config.) if [ -f /usr/include/ncursesw/ncurses.h ]; then echo cflags=\"-D_GNU_SOURCE -I/usr/include/ncursesw\" echo libs=\"-lncursesw\" diff --git a/scripts/kconfig/mconf.c b/scripts/kconfig/mconf.c index 83b5836615fb..143c05fec161 100644 --- a/scripts/kconfig/mconf.c +++ b/scripts/kconfig/mconf.c @@ -490,7 +490,6 @@ static void build_conf(struct menu *menu) switch (prop->type) { case P_MENU: child_count++; - prompt = prompt; if (single_menu_mode) { item_make("%s%*c%s", menu->data ? "-->" : "++>", diff --git a/scripts/kconfig/nconf-cfg.sh b/scripts/kconfig/nconf-cfg.sh index 42f5ac73548e..001559ef0a60 100644 --- a/scripts/kconfig/nconf-cfg.sh +++ b/scripts/kconfig/nconf-cfg.sh @@ -4,20 +4,23 @@ PKG="ncursesw menuw panelw" PKG2="ncurses menu panel" -if pkg-config --exists $PKG; then - echo cflags=\"$(pkg-config --cflags $PKG)\" - echo libs=\"$(pkg-config --libs $PKG)\" - exit 0 -fi +if [ -n "$(command -v pkg-config)" ]; then + if pkg-config --exists $PKG; then + echo cflags=\"$(pkg-config --cflags $PKG)\" + echo libs=\"$(pkg-config --libs $PKG)\" + exit 0 + fi -if pkg-config --exists $PKG2; then - echo cflags=\"$(pkg-config --cflags $PKG2)\" - echo libs=\"$(pkg-config --libs $PKG2)\" - exit 0 + if pkg-config --exists $PKG2; then + echo cflags=\"$(pkg-config --cflags $PKG2)\" + echo libs=\"$(pkg-config --libs $PKG2)\" + exit 0 + fi fi -# Unfortunately, some distributions (e.g. openSUSE) cannot find ncurses -# by pkg-config. +# Check the default paths in case pkg-config is not installed. +# (Even if it is installed, some distributions such as openSUSE cannot +# find ncurses by pkg-config.) if [ -f /usr/include/ncursesw/ncurses.h ]; then echo cflags=\"-D_GNU_SOURCE -I/usr/include/ncursesw\" echo libs=\"-lncursesw -lmenuw -lpanelw\" diff --git a/scripts/kconfig/qconf-cfg.sh b/scripts/kconfig/qconf-cfg.sh index 0862e1562536..02ccc0ae1031 100755 --- a/scripts/kconfig/qconf-cfg.sh +++ b/scripts/kconfig/qconf-cfg.sh @@ -4,6 +4,13 @@ PKG="Qt5Core Qt5Gui Qt5Widgets" PKG2="QtCore QtGui" +if [ -z "$(command -v pkg-config)" ]; then + echo >&2 "*" + echo >&2 "* 'make xconfig' requires 'pkg-config'. Please install it." + echo >&2 "*" + exit 1 +fi + if pkg-config --exists $PKG; then echo cflags=\"-std=c++11 -fPIC $(pkg-config --cflags Qt5Core Qt5Gui Qt5Widgets)\" echo libs=\"$(pkg-config --libs $PKG)\" diff --git a/scripts/recordmcount.pl b/scripts/recordmcount.pl index fe06e77c15eb..f599031260d5 100755 --- a/scripts/recordmcount.pl +++ b/scripts/recordmcount.pl @@ -389,6 +389,9 @@ if ($arch eq "x86_64") { $mcount_regex = "^\\s*([0-9a-fA-F]+):\\sR_RISCV_CALL\\s_mcount\$"; $type = ".quad"; $alignment = 2; +} elsif ($arch eq "nds32") { + $mcount_regex = "^\\s*([0-9a-fA-F]+):\\s*R_NDS32_HI20_RELA\\s+_mcount\$"; + $alignment = 2; } else { die "Arch $arch is not supported with CONFIG_FTRACE_MCOUNT_RECORD"; } diff --git a/scripts/setlocalversion b/scripts/setlocalversion index 71f39410691b..79f7dd57d571 100755 --- a/scripts/setlocalversion +++ b/scripts/setlocalversion @@ -74,7 +74,7 @@ scm_version() fi # Check for uncommitted changes - if git diff-index --name-only HEAD | grep -qv "^scripts/package"; then + if git status -uno --porcelain | grep -qv '^.. scripts/package'; then printf '%s' -dirty fi diff --git a/scripts/subarch.include b/scripts/subarch.include new file mode 100644 index 000000000000..650682821126 --- /dev/null +++ b/scripts/subarch.include @@ -0,0 +1,13 @@ +# SUBARCH tells the usermode build what the underlying arch is. That is set +# first, and if a usermode build is happening, the "ARCH=um" on the command +# line overrides the setting of ARCH below. If a native build is happening, +# then ARCH is assigned, getting whatever value it gets normally, and +# SUBARCH is subsequently ignored. + +SUBARCH := $(shell uname -m | sed -e s/i.86/x86/ -e s/x86_64/x86/ \ + -e s/sun4u/sparc64/ \ + -e s/arm.*/arm/ -e s/sa110/arm/ \ + -e s/s390x/s390/ -e s/parisc64/parisc/ \ + -e s/ppc.*/powerpc/ -e s/mips.*/mips/ \ + -e s/sh[234].*/sh/ -e s/aarch64.*/arm64/ \ + -e s/riscv.*/riscv/) diff --git a/security/Kconfig b/security/Kconfig index 27d8b2688f75..d9aa521b5206 100644 --- a/security/Kconfig +++ b/security/Kconfig @@ -57,7 +57,7 @@ config SECURITY_NETWORK config PAGE_TABLE_ISOLATION bool "Remove the kernel mapping in user mode" default y - depends on X86 && !UML + depends on (X86_64 || X86_PAE) && !UML help This feature reduces the number of hardware side channels by ensuring that the majority of kernel addresses are not mapped diff --git a/security/apparmor/secid.c b/security/apparmor/secid.c index f2f22d00db18..4ccec1bcf6f5 100644 --- a/security/apparmor/secid.c +++ b/security/apparmor/secid.c @@ -79,7 +79,6 @@ int apparmor_secid_to_secctx(u32 secid, char **secdata, u32 *seclen) struct aa_label *label = aa_secid_to_label(secid); int len; - AA_BUG(!secdata); AA_BUG(!seclen); if (!label) diff --git a/sound/core/rawmidi.c b/sound/core/rawmidi.c index 69517e18ef07..08d5662039e3 100644 --- a/sound/core/rawmidi.c +++ b/sound/core/rawmidi.c @@ -129,7 +129,7 @@ static int snd_rawmidi_runtime_create(struct snd_rawmidi_substream *substream) runtime->avail = 0; else runtime->avail = runtime->buffer_size; - runtime->buffer = kvmalloc(runtime->buffer_size, GFP_KERNEL); + runtime->buffer = kvzalloc(runtime->buffer_size, GFP_KERNEL); if (!runtime->buffer) { kfree(runtime); return -ENOMEM; @@ -655,7 +655,7 @@ static int resize_runtime_buffer(struct snd_rawmidi_runtime *runtime, if (params->avail_min < 1 || params->avail_min > params->buffer_size) return -EINVAL; if (params->buffer_size != runtime->buffer_size) { - newbuf = kvmalloc(params->buffer_size, GFP_KERNEL); + newbuf = kvzalloc(params->buffer_size, GFP_KERNEL); if (!newbuf) return -ENOMEM; spin_lock_irq(&runtime->lock); diff --git a/sound/firewire/bebob/bebob.c b/sound/firewire/bebob/bebob.c index 730ea91d9be8..93676354f87f 100644 --- a/sound/firewire/bebob/bebob.c +++ b/sound/firewire/bebob/bebob.c @@ -263,6 +263,8 @@ do_registration(struct work_struct *work) error: mutex_unlock(&devices_mutex); snd_bebob_stream_destroy_duplex(bebob); + kfree(bebob->maudio_special_quirk); + bebob->maudio_special_quirk = NULL; snd_card_free(bebob->card); dev_info(&bebob->unit->device, "Sound card registration failed: %d\n", err); diff --git a/sound/firewire/bebob/bebob_maudio.c b/sound/firewire/bebob/bebob_maudio.c index bd55620c6a47..c266997ad299 100644 --- a/sound/firewire/bebob/bebob_maudio.c +++ b/sound/firewire/bebob/bebob_maudio.c @@ -96,17 +96,13 @@ int snd_bebob_maudio_load_firmware(struct fw_unit *unit) struct fw_device *device = fw_parent_device(unit); int err, rcode; u64 date; - __le32 cues[3] = { - cpu_to_le32(MAUDIO_BOOTLOADER_CUE1), - cpu_to_le32(MAUDIO_BOOTLOADER_CUE2), - cpu_to_le32(MAUDIO_BOOTLOADER_CUE3) - }; + __le32 *cues; /* check date of software used to build */ err = snd_bebob_read_block(unit, INFO_OFFSET_SW_DATE, &date, sizeof(u64)); if (err < 0) - goto end; + return err; /* * firmware version 5058 or later has date later than "20070401", but * 'date' is not null-terminated. @@ -114,20 +110,28 @@ int snd_bebob_maudio_load_firmware(struct fw_unit *unit) if (date < 0x3230303730343031LL) { dev_err(&unit->device, "Use firmware version 5058 or later\n"); - err = -ENOSYS; - goto end; + return -ENXIO; } + cues = kmalloc_array(3, sizeof(*cues), GFP_KERNEL); + if (!cues) + return -ENOMEM; + + cues[0] = cpu_to_le32(MAUDIO_BOOTLOADER_CUE1); + cues[1] = cpu_to_le32(MAUDIO_BOOTLOADER_CUE2); + cues[2] = cpu_to_le32(MAUDIO_BOOTLOADER_CUE3); + rcode = fw_run_transaction(device->card, TCODE_WRITE_BLOCK_REQUEST, device->node_id, device->generation, device->max_speed, BEBOB_ADDR_REG_REQ, - cues, sizeof(cues)); + cues, 3 * sizeof(*cues)); + kfree(cues); if (rcode != RCODE_COMPLETE) { dev_err(&unit->device, "Failed to send a cue to load firmware\n"); err = -EIO; } -end: + return err; } @@ -290,10 +294,6 @@ snd_bebob_maudio_special_discover(struct snd_bebob *bebob, bool is1814) bebob->midi_output_ports = 2; } end: - if (err < 0) { - kfree(params); - bebob->maudio_special_quirk = NULL; - } mutex_unlock(&bebob->mutex); return err; } diff --git a/sound/firewire/digi00x/digi00x.c b/sound/firewire/digi00x/digi00x.c index 1f5e1d23f31a..ef689997d6a5 100644 --- a/sound/firewire/digi00x/digi00x.c +++ b/sound/firewire/digi00x/digi00x.c @@ -49,6 +49,7 @@ static void dg00x_free(struct snd_dg00x *dg00x) fw_unit_put(dg00x->unit); mutex_destroy(&dg00x->mutex); + kfree(dg00x); } static void dg00x_card_free(struct snd_card *card) diff --git a/sound/firewire/fireface/ff-protocol-ff400.c b/sound/firewire/fireface/ff-protocol-ff400.c index ad7a0a32557d..64c3cb0fb926 100644 --- a/sound/firewire/fireface/ff-protocol-ff400.c +++ b/sound/firewire/fireface/ff-protocol-ff400.c @@ -146,6 +146,7 @@ static int ff400_switch_fetching_mode(struct snd_ff *ff, bool enable) { __le32 *reg; int i; + int err; reg = kcalloc(18, sizeof(__le32), GFP_KERNEL); if (reg == NULL) @@ -163,9 +164,11 @@ static int ff400_switch_fetching_mode(struct snd_ff *ff, bool enable) reg[i] = cpu_to_le32(0x00000001); } - return snd_fw_transaction(ff->unit, TCODE_WRITE_BLOCK_REQUEST, - FF400_FETCH_PCM_FRAMES, reg, - sizeof(__le32) * 18, 0); + err = snd_fw_transaction(ff->unit, TCODE_WRITE_BLOCK_REQUEST, + FF400_FETCH_PCM_FRAMES, reg, + sizeof(__le32) * 18, 0); + kfree(reg); + return err; } static void ff400_dump_sync_status(struct snd_ff *ff, diff --git a/sound/firewire/fireworks/fireworks.c b/sound/firewire/fireworks/fireworks.c index 71a0613d3da0..f2d073365cf6 100644 --- a/sound/firewire/fireworks/fireworks.c +++ b/sound/firewire/fireworks/fireworks.c @@ -301,6 +301,8 @@ error: snd_efw_transaction_remove_instance(efw); snd_efw_stream_destroy_duplex(efw); snd_card_free(efw->card); + kfree(efw->resp_buf); + efw->resp_buf = NULL; dev_info(&efw->unit->device, "Sound card registration failed: %d\n", err); } diff --git a/sound/firewire/oxfw/oxfw.c b/sound/firewire/oxfw/oxfw.c index 1e5b2c802635..2ea8be6c8584 100644 --- a/sound/firewire/oxfw/oxfw.c +++ b/sound/firewire/oxfw/oxfw.c @@ -130,6 +130,7 @@ static void oxfw_free(struct snd_oxfw *oxfw) kfree(oxfw->spec); mutex_destroy(&oxfw->mutex); + kfree(oxfw); } /* @@ -207,6 +208,7 @@ static int detect_quirks(struct snd_oxfw *oxfw) static void do_registration(struct work_struct *work) { struct snd_oxfw *oxfw = container_of(work, struct snd_oxfw, dwork.work); + int i; int err; if (oxfw->registered) @@ -269,7 +271,15 @@ error: snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->rx_stream); if (oxfw->has_output) snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->tx_stream); + for (i = 0; i < SND_OXFW_STREAM_FORMAT_ENTRIES; ++i) { + kfree(oxfw->tx_stream_formats[i]); + oxfw->tx_stream_formats[i] = NULL; + kfree(oxfw->rx_stream_formats[i]); + oxfw->rx_stream_formats[i] = NULL; + } snd_card_free(oxfw->card); + kfree(oxfw->spec); + oxfw->spec = NULL; dev_info(&oxfw->unit->device, "Sound card registration failed: %d\n", err); } diff --git a/sound/firewire/tascam/tascam.c b/sound/firewire/tascam/tascam.c index 44ad41fb7374..d3fdc463a884 100644 --- a/sound/firewire/tascam/tascam.c +++ b/sound/firewire/tascam/tascam.c @@ -93,6 +93,7 @@ static void tscm_free(struct snd_tscm *tscm) fw_unit_put(tscm->unit); mutex_destroy(&tscm->mutex); + kfree(tscm); } static void tscm_card_free(struct snd_card *card) diff --git a/sound/hda/ext/hdac_ext_stream.c b/sound/hda/ext/hdac_ext_stream.c index 1bd27576db98..a835558ddbc9 100644 --- a/sound/hda/ext/hdac_ext_stream.c +++ b/sound/hda/ext/hdac_ext_stream.c @@ -146,7 +146,8 @@ EXPORT_SYMBOL_GPL(snd_hdac_ext_stream_decouple); */ void snd_hdac_ext_link_stream_start(struct hdac_ext_stream *stream) { - snd_hdac_updatel(stream->pplc_addr, AZX_REG_PPLCCTL, 0, AZX_PPLCCTL_RUN); + snd_hdac_updatel(stream->pplc_addr, AZX_REG_PPLCCTL, + AZX_PPLCCTL_RUN, AZX_PPLCCTL_RUN); } EXPORT_SYMBOL_GPL(snd_hdac_ext_link_stream_start); @@ -171,7 +172,8 @@ void snd_hdac_ext_link_stream_reset(struct hdac_ext_stream *stream) snd_hdac_ext_link_stream_clear(stream); - snd_hdac_updatel(stream->pplc_addr, AZX_REG_PPLCCTL, 0, AZX_PPLCCTL_STRST); + snd_hdac_updatel(stream->pplc_addr, AZX_REG_PPLCCTL, + AZX_PPLCCTL_STRST, AZX_PPLCCTL_STRST); udelay(3); timeout = 50; do { @@ -242,7 +244,7 @@ EXPORT_SYMBOL_GPL(snd_hdac_ext_link_set_stream_id); void snd_hdac_ext_link_clear_stream_id(struct hdac_ext_link *link, int stream) { - snd_hdac_updatew(link->ml_addr, AZX_REG_ML_LOSIDV, 0, (1 << stream)); + snd_hdac_updatew(link->ml_addr, AZX_REG_ML_LOSIDV, (1 << stream), 0); } EXPORT_SYMBOL_GPL(snd_hdac_ext_link_clear_stream_id); @@ -415,7 +417,6 @@ void snd_hdac_ext_stream_spbcap_enable(struct hdac_bus *bus, bool enable, int index) { u32 mask = 0; - u32 register_mask = 0; if (!bus->spbcap) { dev_err(bus->dev, "Address of SPB capability is NULL\n"); @@ -424,12 +425,8 @@ void snd_hdac_ext_stream_spbcap_enable(struct hdac_bus *bus, mask |= (1 << index); - register_mask = readl(bus->spbcap + AZX_REG_SPB_SPBFCCTL); - - mask |= register_mask; - if (enable) - snd_hdac_updatel(bus->spbcap, AZX_REG_SPB_SPBFCCTL, 0, mask); + snd_hdac_updatel(bus->spbcap, AZX_REG_SPB_SPBFCCTL, mask, mask); else snd_hdac_updatel(bus->spbcap, AZX_REG_SPB_SPBFCCTL, mask, 0); } @@ -503,7 +500,6 @@ void snd_hdac_ext_stream_drsm_enable(struct hdac_bus *bus, bool enable, int index) { u32 mask = 0; - u32 register_mask = 0; if (!bus->drsmcap) { dev_err(bus->dev, "Address of DRSM capability is NULL\n"); @@ -512,12 +508,8 @@ void snd_hdac_ext_stream_drsm_enable(struct hdac_bus *bus, mask |= (1 << index); - register_mask = readl(bus->drsmcap + AZX_REG_SPB_SPBFCCTL); - - mask |= register_mask; - if (enable) - snd_hdac_updatel(bus->drsmcap, AZX_REG_DRSM_CTL, 0, mask); + snd_hdac_updatel(bus->drsmcap, AZX_REG_DRSM_CTL, mask, mask); else snd_hdac_updatel(bus->drsmcap, AZX_REG_DRSM_CTL, mask, 0); } diff --git a/sound/hda/hdac_controller.c b/sound/hda/hdac_controller.c index 560ec0986e1a..74244d8e2909 100644 --- a/sound/hda/hdac_controller.c +++ b/sound/hda/hdac_controller.c @@ -40,6 +40,8 @@ static void azx_clear_corbrp(struct hdac_bus *bus) */ void snd_hdac_bus_init_cmd_io(struct hdac_bus *bus) { + WARN_ON_ONCE(!bus->rb.area); + spin_lock_irq(&bus->reg_lock); /* CORB set up */ bus->corb.addr = bus->rb.addr; @@ -383,7 +385,7 @@ void snd_hdac_bus_exit_link_reset(struct hdac_bus *bus) EXPORT_SYMBOL_GPL(snd_hdac_bus_exit_link_reset); /* reset codec link */ -static int azx_reset(struct hdac_bus *bus, bool full_reset) +int snd_hdac_bus_reset_link(struct hdac_bus *bus, bool full_reset) { if (!full_reset) goto skip_reset; @@ -408,7 +410,7 @@ static int azx_reset(struct hdac_bus *bus, bool full_reset) skip_reset: /* check to see if controller is ready */ if (!snd_hdac_chip_readb(bus, GCTL)) { - dev_dbg(bus->dev, "azx_reset: controller not ready!\n"); + dev_dbg(bus->dev, "controller not ready!\n"); return -EBUSY; } @@ -423,6 +425,7 @@ static int azx_reset(struct hdac_bus *bus, bool full_reset) return 0; } +EXPORT_SYMBOL_GPL(snd_hdac_bus_reset_link); /* enable interrupts */ static void azx_int_enable(struct hdac_bus *bus) @@ -477,15 +480,17 @@ bool snd_hdac_bus_init_chip(struct hdac_bus *bus, bool full_reset) return false; /* reset controller */ - azx_reset(bus, full_reset); + snd_hdac_bus_reset_link(bus, full_reset); - /* initialize interrupts */ + /* clear interrupts */ azx_int_clear(bus); - azx_int_enable(bus); /* initialize the codec command I/O */ snd_hdac_bus_init_cmd_io(bus); + /* enable interrupts after CORB/RIRB buffers are initialized above */ + azx_int_enable(bus); + /* program the position buffer */ if (bus->use_posbuf && bus->posbuf.addr) { snd_hdac_chip_writel(bus, DPLBASE, (u32)bus->posbuf.addr); diff --git a/sound/pci/emu10k1/emufx.c b/sound/pci/emu10k1/emufx.c index 90713741c2dc..6ebe817801ea 100644 --- a/sound/pci/emu10k1/emufx.c +++ b/sound/pci/emu10k1/emufx.c @@ -2540,7 +2540,7 @@ static int snd_emu10k1_fx8010_ioctl(struct snd_hwdep * hw, struct file *file, un emu->support_tlv = 1; return put_user(SNDRV_EMU10K1_VERSION, (int __user *)argp); case SNDRV_EMU10K1_IOCTL_INFO: - info = kmalloc(sizeof(*info), GFP_KERNEL); + info = kzalloc(sizeof(*info), GFP_KERNEL); if (!info) return -ENOMEM; snd_emu10k1_fx8010_info(emu, info); diff --git a/sound/pci/hda/hda_codec.c b/sound/pci/hda/hda_codec.c index 0a5085537034..26d348b47867 100644 --- a/sound/pci/hda/hda_codec.c +++ b/sound/pci/hda/hda_codec.c @@ -3935,7 +3935,8 @@ void snd_hda_bus_reset_codecs(struct hda_bus *bus) list_for_each_codec(codec, bus) { /* FIXME: maybe a better way needed for forced reset */ - cancel_delayed_work_sync(&codec->jackpoll_work); + if (current_work() != &codec->jackpoll_work.work) + cancel_delayed_work_sync(&codec->jackpoll_work); #ifdef CONFIG_PM if (hda_codec_is_power_on(codec)) { hda_call_codec_suspend(codec); diff --git a/sound/pci/hda/hda_intel.c b/sound/pci/hda/hda_intel.c index 1b2ce304152a..aa4c672dbaf7 100644 --- a/sound/pci/hda/hda_intel.c +++ b/sound/pci/hda/hda_intel.c @@ -365,8 +365,10 @@ enum { */ #ifdef SUPPORT_VGA_SWITCHEROO #define use_vga_switcheroo(chip) ((chip)->use_vga_switcheroo) +#define needs_eld_notify_link(chip) ((chip)->need_eld_notify_link) #else #define use_vga_switcheroo(chip) 0 +#define needs_eld_notify_link(chip) false #endif #define CONTROLLER_IN_GPU(pci) (((pci)->device == 0x0a0c) || \ @@ -453,6 +455,7 @@ static inline void mark_runtime_wc(struct azx *chip, struct azx_dev *azx_dev, #endif static int azx_acquire_irq(struct azx *chip, int do_disconnect); +static void set_default_power_save(struct azx *chip); /* * initialize the PCI registers @@ -1201,6 +1204,10 @@ static int azx_runtime_idle(struct device *dev) azx_bus(chip)->codec_powered || !chip->running) return -EBUSY; + /* ELD notification gets broken when HD-audio bus is off */ + if (needs_eld_notify_link(hda)) + return -EBUSY; + return 0; } @@ -1298,6 +1305,36 @@ static bool azx_vs_can_switch(struct pci_dev *pci) return true; } +/* + * The discrete GPU cannot power down unless the HDA controller runtime + * suspends, so activate runtime PM on codecs even if power_save == 0. + */ +static void setup_vga_switcheroo_runtime_pm(struct azx *chip) +{ + struct hda_intel *hda = container_of(chip, struct hda_intel, chip); + struct hda_codec *codec; + + if (hda->use_vga_switcheroo && !hda->need_eld_notify_link) { + list_for_each_codec(codec, &chip->bus) + codec->auto_runtime_pm = 1; + /* reset the power save setup */ + if (chip->running) + set_default_power_save(chip); + } +} + +static void azx_vs_gpu_bound(struct pci_dev *pci, + enum vga_switcheroo_client_id client_id) +{ + struct snd_card *card = pci_get_drvdata(pci); + struct azx *chip = card->private_data; + struct hda_intel *hda = container_of(chip, struct hda_intel, chip); + + if (client_id == VGA_SWITCHEROO_DIS) + hda->need_eld_notify_link = 0; + setup_vga_switcheroo_runtime_pm(chip); +} + static void init_vga_switcheroo(struct azx *chip) { struct hda_intel *hda = container_of(chip, struct hda_intel, chip); @@ -1306,6 +1343,7 @@ static void init_vga_switcheroo(struct azx *chip) dev_info(chip->card->dev, "Handle vga_switcheroo audio client\n"); hda->use_vga_switcheroo = 1; + hda->need_eld_notify_link = 1; /* cleared in gpu_bound op */ chip->driver_caps |= AZX_DCAPS_PM_RUNTIME; pci_dev_put(p); } @@ -1314,6 +1352,7 @@ static void init_vga_switcheroo(struct azx *chip) static const struct vga_switcheroo_client_ops azx_vs_ops = { .set_gpu_state = azx_vs_set_state, .can_switch = azx_vs_can_switch, + .gpu_bound = azx_vs_gpu_bound, }; static int register_vga_switcheroo(struct azx *chip) @@ -1339,6 +1378,7 @@ static int register_vga_switcheroo(struct azx *chip) #define init_vga_switcheroo(chip) /* NOP */ #define register_vga_switcheroo(chip) 0 #define check_hdmi_disabled(pci) false +#define setup_vga_switcheroo_runtime_pm(chip) /* NOP */ #endif /* SUPPORT_VGA_SWITCHER */ /* @@ -1352,6 +1392,7 @@ static int azx_free(struct azx *chip) if (azx_has_pm_runtime(chip) && chip->running) pm_runtime_get_noresume(&pci->dev); + chip->running = 0; azx_del_card_list(chip); @@ -2230,6 +2271,25 @@ static struct snd_pci_quirk power_save_blacklist[] = { }; #endif /* CONFIG_PM */ +static void set_default_power_save(struct azx *chip) +{ + int val = power_save; + +#ifdef CONFIG_PM + if (pm_blacklist) { + const struct snd_pci_quirk *q; + + q = snd_pci_quirk_lookup(chip->pci, power_save_blacklist); + if (q && val) { + dev_info(chip->card->dev, "device %04x:%04x is on the power_save blacklist, forcing power_save to 0\n", + q->subvendor, q->subdevice); + val = 0; + } + } +#endif /* CONFIG_PM */ + snd_hda_set_power_save(&chip->bus, val * 1000); +} + /* number of codec slots for each chipset: 0 = default slots (i.e. 4) */ static unsigned int azx_max_codecs[AZX_NUM_DRIVERS] = { [AZX_DRIVER_NVIDIA] = 8, @@ -2241,9 +2301,7 @@ static int azx_probe_continue(struct azx *chip) struct hda_intel *hda = container_of(chip, struct hda_intel, chip); struct hdac_bus *bus = azx_bus(chip); struct pci_dev *pci = chip->pci; - struct hda_codec *codec; int dev = chip->dev_index; - int val; int err; hda->probe_continued = 1; @@ -2322,31 +2380,13 @@ static int azx_probe_continue(struct azx *chip) if (err < 0) goto out_free; + setup_vga_switcheroo_runtime_pm(chip); + chip->running = 1; azx_add_card_list(chip); - val = power_save; -#ifdef CONFIG_PM - if (pm_blacklist) { - const struct snd_pci_quirk *q; - - q = snd_pci_quirk_lookup(chip->pci, power_save_blacklist); - if (q && val) { - dev_info(chip->card->dev, "device %04x:%04x is on the power_save blacklist, forcing power_save to 0\n", - q->subvendor, q->subdevice); - val = 0; - } - } -#endif /* CONFIG_PM */ - /* - * The discrete GPU cannot power down unless the HDA controller runtime - * suspends, so activate runtime PM on codecs even if power_save == 0. - */ - if (use_vga_switcheroo(hda)) - list_for_each_codec(codec, &chip->bus) - codec->auto_runtime_pm = 1; + set_default_power_save(chip); - snd_hda_set_power_save(&chip->bus, val * 1000); if (azx_has_pm_runtime(chip)) pm_runtime_put_autosuspend(&pci->dev); diff --git a/sound/pci/hda/hda_intel.h b/sound/pci/hda/hda_intel.h index e3a3d318d2e5..f59719e06b91 100644 --- a/sound/pci/hda/hda_intel.h +++ b/sound/pci/hda/hda_intel.h @@ -37,6 +37,7 @@ struct hda_intel { /* vga_switcheroo setup */ unsigned int use_vga_switcheroo:1; + unsigned int need_eld_notify_link:1; unsigned int vga_switcheroo_registered:1; unsigned int init_failed:1; /* delayed init failed */ diff --git a/sound/soc/amd/acp-pcm-dma.c b/sound/soc/amd/acp-pcm-dma.c index e359938e3d7e..77b265bd0505 100644 --- a/sound/soc/amd/acp-pcm-dma.c +++ b/sound/soc/amd/acp-pcm-dma.c @@ -16,6 +16,7 @@ #include <linux/module.h> #include <linux/delay.h> #include <linux/io.h> +#include <linux/iopoll.h> #include <linux/sizes.h> #include <linux/pm_runtime.h> @@ -184,6 +185,24 @@ static void config_dma_descriptor_in_sram(void __iomem *acp_mmio, acp_reg_write(descr_info->xfer_val, acp_mmio, mmACP_SRBM_Targ_Idx_Data); } +static void pre_config_reset(void __iomem *acp_mmio, u16 ch_num) +{ + u32 dma_ctrl; + int ret; + + /* clear the reset bit */ + dma_ctrl = acp_reg_read(acp_mmio, mmACP_DMA_CNTL_0 + ch_num); + dma_ctrl &= ~ACP_DMA_CNTL_0__DMAChRst_MASK; + acp_reg_write(dma_ctrl, acp_mmio, mmACP_DMA_CNTL_0 + ch_num); + /* check the reset bit before programming configuration registers */ + ret = readl_poll_timeout(acp_mmio + ((mmACP_DMA_CNTL_0 + ch_num) * 4), + dma_ctrl, + !(dma_ctrl & ACP_DMA_CNTL_0__DMAChRst_MASK), + 100, ACP_DMA_RESET_TIME); + if (ret < 0) + pr_err("Failed to clear reset of channel : %d\n", ch_num); +} + /* * Initialize the DMA descriptor information for transfer between * system memory <-> ACP SRAM @@ -236,6 +255,7 @@ static void set_acp_sysmem_dma_descriptors(void __iomem *acp_mmio, config_dma_descriptor_in_sram(acp_mmio, dma_dscr_idx, &dmadscr[i]); } + pre_config_reset(acp_mmio, ch); config_acp_dma_channel(acp_mmio, ch, dma_dscr_idx - 1, NUM_DSCRS_PER_CHANNEL, @@ -275,6 +295,7 @@ static void set_acp_to_i2s_dma_descriptors(void __iomem *acp_mmio, u32 size, config_dma_descriptor_in_sram(acp_mmio, dma_dscr_idx, &dmadscr[i]); } + pre_config_reset(acp_mmio, ch); /* Configure the DMA channel with the above descriptore */ config_acp_dma_channel(acp_mmio, ch, dma_dscr_idx - 1, NUM_DSCRS_PER_CHANNEL, diff --git a/sound/soc/codecs/cs4265.c b/sound/soc/codecs/cs4265.c index 275677de669f..407554175282 100644 --- a/sound/soc/codecs/cs4265.c +++ b/sound/soc/codecs/cs4265.c @@ -157,8 +157,8 @@ static const struct snd_kcontrol_new cs4265_snd_controls[] = { SOC_SINGLE("Validity Bit Control Switch", CS4265_SPDIF_CTL2, 3, 1, 0), SOC_ENUM("SPDIF Mono/Stereo", spdif_mono_stereo_enum), - SOC_SINGLE("MMTLR Data Switch", 0, - 1, 1, 0), + SOC_SINGLE("MMTLR Data Switch", CS4265_SPDIF_CTL2, + 0, 1, 0), SOC_ENUM("Mono Channel Select", spdif_mono_select_enum), SND_SOC_BYTES("C Data Buffer", CS4265_C_DATA_BUFF, 24), }; diff --git a/sound/soc/codecs/max98373.c b/sound/soc/codecs/max98373.c index 92b7125ea169..1093f766d0d2 100644 --- a/sound/soc/codecs/max98373.c +++ b/sound/soc/codecs/max98373.c @@ -520,6 +520,7 @@ static bool max98373_volatile_reg(struct device *dev, unsigned int reg) { switch (reg) { case MAX98373_R2000_SW_RESET ... MAX98373_R2009_INT_FLAG3: + case MAX98373_R203E_AMP_PATH_GAIN: case MAX98373_R2054_MEAS_ADC_PVDD_CH_READBACK: case MAX98373_R2055_MEAS_ADC_THERM_CH_READBACK: case MAX98373_R20B6_BDE_CUR_STATE_READBACK: @@ -729,6 +730,7 @@ static int max98373_probe(struct snd_soc_component *component) /* Software Reset */ regmap_write(max98373->regmap, MAX98373_R2000_SW_RESET, MAX98373_SOFT_RESET); + usleep_range(10000, 11000); /* IV default slot configuration */ regmap_write(max98373->regmap, @@ -817,6 +819,7 @@ static int max98373_resume(struct device *dev) regmap_write(max98373->regmap, MAX98373_R2000_SW_RESET, MAX98373_SOFT_RESET); + usleep_range(10000, 11000); regcache_cache_only(max98373->regmap, false); regcache_sync(max98373->regmap); return 0; diff --git a/sound/soc/codecs/rt5514.c b/sound/soc/codecs/rt5514.c index dca82dd6e3bf..32fe76c3134a 100644 --- a/sound/soc/codecs/rt5514.c +++ b/sound/soc/codecs/rt5514.c @@ -64,8 +64,8 @@ static const struct reg_sequence rt5514_patch[] = { {RT5514_ANA_CTRL_LDO10, 0x00028604}, {RT5514_ANA_CTRL_ADCFED, 0x00000800}, {RT5514_ASRC_IN_CTRL1, 0x00000003}, - {RT5514_DOWNFILTER0_CTRL3, 0x10000352}, - {RT5514_DOWNFILTER1_CTRL3, 0x10000352}, + {RT5514_DOWNFILTER0_CTRL3, 0x10000342}, + {RT5514_DOWNFILTER1_CTRL3, 0x10000342}, }; static const struct reg_default rt5514_reg[] = { @@ -92,10 +92,10 @@ static const struct reg_default rt5514_reg[] = { {RT5514_ASRC_IN_CTRL1, 0x00000003}, {RT5514_DOWNFILTER0_CTRL1, 0x00020c2f}, {RT5514_DOWNFILTER0_CTRL2, 0x00020c2f}, - {RT5514_DOWNFILTER0_CTRL3, 0x10000352}, + {RT5514_DOWNFILTER0_CTRL3, 0x10000342}, {RT5514_DOWNFILTER1_CTRL1, 0x00020c2f}, {RT5514_DOWNFILTER1_CTRL2, 0x00020c2f}, - {RT5514_DOWNFILTER1_CTRL3, 0x10000352}, + {RT5514_DOWNFILTER1_CTRL3, 0x10000342}, {RT5514_ANA_CTRL_LDO10, 0x00028604}, {RT5514_ANA_CTRL_LDO18_16, 0x02000345}, {RT5514_ANA_CTRL_ADC12, 0x0000a2a8}, diff --git a/sound/soc/codecs/rt5682.c b/sound/soc/codecs/rt5682.c index 640d400ca013..afe7d5b19313 100644 --- a/sound/soc/codecs/rt5682.c +++ b/sound/soc/codecs/rt5682.c @@ -750,8 +750,8 @@ static bool rt5682_readable_register(struct device *dev, unsigned int reg) } static const DECLARE_TLV_DB_SCALE(hp_vol_tlv, -2250, 150, 0); -static const DECLARE_TLV_DB_SCALE(dac_vol_tlv, -65625, 375, 0); -static const DECLARE_TLV_DB_SCALE(adc_vol_tlv, -17625, 375, 0); +static const DECLARE_TLV_DB_SCALE(dac_vol_tlv, -6525, 75, 0); +static const DECLARE_TLV_DB_SCALE(adc_vol_tlv, -1725, 75, 0); static const DECLARE_TLV_DB_SCALE(adc_bst_tlv, 0, 1200, 0); /* {0, +20, +24, +30, +35, +40, +44, +50, +52} dB */ @@ -1114,7 +1114,7 @@ static const struct snd_kcontrol_new rt5682_snd_controls[] = { /* DAC Digital Volume */ SOC_DOUBLE_TLV("DAC1 Playback Volume", RT5682_DAC1_DIG_VOL, - RT5682_L_VOL_SFT, RT5682_R_VOL_SFT, 175, 0, dac_vol_tlv), + RT5682_L_VOL_SFT + 1, RT5682_R_VOL_SFT + 1, 86, 0, dac_vol_tlv), /* IN Boost Volume */ SOC_SINGLE_TLV("CBJ Boost Volume", RT5682_CBJ_BST_CTRL, @@ -1124,7 +1124,7 @@ static const struct snd_kcontrol_new rt5682_snd_controls[] = { SOC_DOUBLE("STO1 ADC Capture Switch", RT5682_STO1_ADC_DIG_VOL, RT5682_L_MUTE_SFT, RT5682_R_MUTE_SFT, 1, 1), SOC_DOUBLE_TLV("STO1 ADC Capture Volume", RT5682_STO1_ADC_DIG_VOL, - RT5682_L_VOL_SFT, RT5682_R_VOL_SFT, 127, 0, adc_vol_tlv), + RT5682_L_VOL_SFT + 1, RT5682_R_VOL_SFT + 1, 63, 0, adc_vol_tlv), /* ADC Boost Volume Control */ SOC_DOUBLE_TLV("STO1 ADC Boost Gain Volume", RT5682_STO1_ADC_BOOST, diff --git a/sound/soc/codecs/sigmadsp.c b/sound/soc/codecs/sigmadsp.c index d53680ac78e4..6df158669420 100644 --- a/sound/soc/codecs/sigmadsp.c +++ b/sound/soc/codecs/sigmadsp.c @@ -117,8 +117,7 @@ static int sigmadsp_ctrl_write(struct sigmadsp *sigmadsp, struct sigmadsp_control *ctrl, void *data) { /* safeload loads up to 20 bytes in a atomic operation */ - if (ctrl->num_bytes > 4 && ctrl->num_bytes <= 20 && sigmadsp->ops && - sigmadsp->ops->safeload) + if (ctrl->num_bytes <= 20 && sigmadsp->ops && sigmadsp->ops->safeload) return sigmadsp->ops->safeload(sigmadsp, ctrl->addr, data, ctrl->num_bytes); else diff --git a/sound/soc/codecs/tas6424.c b/sound/soc/codecs/tas6424.c index 14999b999fd3..0d6145549a98 100644 --- a/sound/soc/codecs/tas6424.c +++ b/sound/soc/codecs/tas6424.c @@ -424,8 +424,10 @@ static void tas6424_fault_check_work(struct work_struct *work) TAS6424_FAULT_PVDD_UV | TAS6424_FAULT_VBAT_UV; - if (reg) + if (!reg) { + tas6424->last_fault1 = reg; goto check_global_fault2_reg; + } /* * Only flag errors once for a given occurrence. This is needed as @@ -461,8 +463,10 @@ check_global_fault2_reg: TAS6424_FAULT_OTSD_CH3 | TAS6424_FAULT_OTSD_CH4; - if (!reg) + if (!reg) { + tas6424->last_fault2 = reg; goto check_warn_reg; + } if ((reg & TAS6424_FAULT_OTSD) && !(tas6424->last_fault2 & TAS6424_FAULT_OTSD)) dev_crit(dev, "experienced a global overtemp shutdown\n"); @@ -497,8 +501,10 @@ check_warn_reg: TAS6424_WARN_VDD_OTW_CH3 | TAS6424_WARN_VDD_OTW_CH4; - if (!reg) + if (!reg) { + tas6424->last_warn = reg; goto out; + } if ((reg & TAS6424_WARN_VDD_UV) && !(tas6424->last_warn & TAS6424_WARN_VDD_UV)) dev_warn(dev, "experienced a VDD under voltage condition\n"); diff --git a/sound/soc/codecs/wm8804-i2c.c b/sound/soc/codecs/wm8804-i2c.c index f27464c2c5ba..79541960f45d 100644 --- a/sound/soc/codecs/wm8804-i2c.c +++ b/sound/soc/codecs/wm8804-i2c.c @@ -13,6 +13,7 @@ #include <linux/init.h> #include <linux/module.h> #include <linux/i2c.h> +#include <linux/acpi.h> #include "wm8804.h" @@ -40,17 +41,29 @@ static const struct i2c_device_id wm8804_i2c_id[] = { }; MODULE_DEVICE_TABLE(i2c, wm8804_i2c_id); +#if defined(CONFIG_OF) static const struct of_device_id wm8804_of_match[] = { { .compatible = "wlf,wm8804", }, { } }; MODULE_DEVICE_TABLE(of, wm8804_of_match); +#endif + +#ifdef CONFIG_ACPI +static const struct acpi_device_id wm8804_acpi_match[] = { + { "1AEC8804", 0 }, /* Wolfson PCI ID + part ID */ + { "10138804", 0 }, /* Cirrus Logic PCI ID + part ID */ + { }, +}; +MODULE_DEVICE_TABLE(acpi, wm8804_acpi_match); +#endif static struct i2c_driver wm8804_i2c_driver = { .driver = { .name = "wm8804", .pm = &wm8804_pm, - .of_match_table = wm8804_of_match, + .of_match_table = of_match_ptr(wm8804_of_match), + .acpi_match_table = ACPI_PTR(wm8804_acpi_match), }, .probe = wm8804_i2c_probe, .remove = wm8804_i2c_remove, diff --git a/sound/soc/codecs/wm9712.c b/sound/soc/codecs/wm9712.c index 953d94d50586..ade34c26ad2f 100644 --- a/sound/soc/codecs/wm9712.c +++ b/sound/soc/codecs/wm9712.c @@ -719,7 +719,7 @@ static int wm9712_probe(struct platform_device *pdev) static struct platform_driver wm9712_component_driver = { .driver = { - .name = "wm9712-component", + .name = "wm9712-codec", }, .probe = wm9712_probe, diff --git a/sound/soc/intel/boards/bytcr_rt5640.c b/sound/soc/intel/boards/bytcr_rt5640.c index d32844f94d74..b6dc524830b2 100644 --- a/sound/soc/intel/boards/bytcr_rt5640.c +++ b/sound/soc/intel/boards/bytcr_rt5640.c @@ -575,6 +575,17 @@ static const struct dmi_system_id byt_rt5640_quirk_table[] = { BYT_RT5640_MONO_SPEAKER | BYT_RT5640_MCLK_EN), }, + { /* Linx Linx7 tablet */ + .matches = { + DMI_EXACT_MATCH(DMI_SYS_VENDOR, "LINX"), + DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "LINX7"), + }, + .driver_data = (void *)(BYTCR_INPUT_DEFAULTS | + BYT_RT5640_MONO_SPEAKER | + BYT_RT5640_JD_NOT_INV | + BYT_RT5640_SSP0_AIF1 | + BYT_RT5640_MCLK_EN), + }, { /* MSI S100 tablet */ .matches = { DMI_EXACT_MATCH(DMI_SYS_VENDOR, "Micro-Star International Co., Ltd."), @@ -602,6 +613,21 @@ static const struct dmi_system_id byt_rt5640_quirk_table[] = { BYT_RT5640_SSP0_AIF1 | BYT_RT5640_MCLK_EN), }, + { /* Onda v975w */ + .matches = { + DMI_EXACT_MATCH(DMI_BOARD_VENDOR, "AMI Corporation"), + DMI_EXACT_MATCH(DMI_BOARD_NAME, "Aptio CRB"), + /* The above are too generic, also match BIOS info */ + DMI_EXACT_MATCH(DMI_BIOS_VERSION, "5.6.5"), + DMI_EXACT_MATCH(DMI_BIOS_DATE, "07/25/2014"), + }, + .driver_data = (void *)(BYT_RT5640_IN1_MAP | + BYT_RT5640_JD_SRC_JD2_IN4N | + BYT_RT5640_OVCD_TH_2000UA | + BYT_RT5640_OVCD_SF_0P75 | + BYT_RT5640_DIFF_MIC | + BYT_RT5640_MCLK_EN), + }, { /* Pipo W4 */ .matches = { DMI_EXACT_MATCH(DMI_BOARD_VENDOR, "AMI Corporation"), diff --git a/sound/soc/intel/skylake/skl.c b/sound/soc/intel/skylake/skl.c index dce649485649..1d17be0f78a0 100644 --- a/sound/soc/intel/skylake/skl.c +++ b/sound/soc/intel/skylake/skl.c @@ -834,7 +834,7 @@ static int skl_first_init(struct hdac_bus *bus) return -ENXIO; } - skl_init_chip(bus, true); + snd_hdac_bus_reset_link(bus, true); snd_hdac_bus_parse_capabilities(bus); diff --git a/sound/soc/qcom/qdsp6/q6routing.c b/sound/soc/qcom/qdsp6/q6routing.c index dc94c5c53788..c6b51571be94 100644 --- a/sound/soc/qcom/qdsp6/q6routing.c +++ b/sound/soc/qcom/qdsp6/q6routing.c @@ -960,8 +960,10 @@ static int msm_routing_probe(struct snd_soc_component *c) { int i; - for (i = 0; i < MAX_SESSIONS; i++) + for (i = 0; i < MAX_SESSIONS; i++) { routing_data->sessions[i].port_id = -1; + routing_data->sessions[i].fedai_id = -1; + } return 0; } diff --git a/sound/soc/sh/rcar/adg.c b/sound/soc/sh/rcar/adg.c index 3a3064dda57f..051f96405346 100644 --- a/sound/soc/sh/rcar/adg.c +++ b/sound/soc/sh/rcar/adg.c @@ -462,6 +462,11 @@ static void rsnd_adg_get_clkout(struct rsnd_priv *priv, goto rsnd_adg_get_clkout_end; req_size = prop->length / sizeof(u32); + if (req_size > REQ_SIZE) { + dev_err(dev, + "too many clock-frequency, use top %d\n", REQ_SIZE); + req_size = REQ_SIZE; + } of_property_read_u32_array(np, "clock-frequency", req_rate, req_size); req_48kHz_rate = 0; diff --git a/sound/soc/sh/rcar/core.c b/sound/soc/sh/rcar/core.c index f8425d8b44d2..d23c2bbff0cf 100644 --- a/sound/soc/sh/rcar/core.c +++ b/sound/soc/sh/rcar/core.c @@ -478,7 +478,7 @@ static int rsnd_status_update(u32 *status, (func_call && (mod)->ops->fn) ? #fn : ""); \ if (func_call && (mod)->ops->fn) \ tmp = (mod)->ops->fn(mod, io, param); \ - if (tmp) \ + if (tmp && (tmp != -EPROBE_DEFER)) \ dev_err(dev, "%s[%d] : %s error %d\n", \ rsnd_mod_name(mod), rsnd_mod_id(mod), \ #fn, tmp); \ @@ -958,12 +958,23 @@ static void rsnd_soc_dai_shutdown(struct snd_pcm_substream *substream, rsnd_dai_stream_quit(io); } +static int rsnd_soc_dai_prepare(struct snd_pcm_substream *substream, + struct snd_soc_dai *dai) +{ + struct rsnd_priv *priv = rsnd_dai_to_priv(dai); + struct rsnd_dai *rdai = rsnd_dai_to_rdai(dai); + struct rsnd_dai_stream *io = rsnd_rdai_to_io(rdai, substream); + + return rsnd_dai_call(prepare, io, priv); +} + static const struct snd_soc_dai_ops rsnd_soc_dai_ops = { .startup = rsnd_soc_dai_startup, .shutdown = rsnd_soc_dai_shutdown, .trigger = rsnd_soc_dai_trigger, .set_fmt = rsnd_soc_dai_set_fmt, .set_tdm_slot = rsnd_soc_set_dai_tdm_slot, + .prepare = rsnd_soc_dai_prepare, }; void rsnd_parse_connect_common(struct rsnd_dai *rdai, @@ -1550,6 +1561,14 @@ exit_snd_probe: rsnd_dai_call(remove, &rdai->capture, priv); } + /* + * adg is very special mod which can't use rsnd_dai_call(remove), + * and it registers ADG clock on probe. + * It should be unregister if probe failed. + * Mainly it is assuming -EPROBE_DEFER case + */ + rsnd_adg_remove(priv); + return ret; } diff --git a/sound/soc/sh/rcar/dma.c b/sound/soc/sh/rcar/dma.c index fe63ef8600d0..d65ea7bc4dac 100644 --- a/sound/soc/sh/rcar/dma.c +++ b/sound/soc/sh/rcar/dma.c @@ -241,6 +241,10 @@ static int rsnd_dmaen_attach(struct rsnd_dai_stream *io, /* try to get DMAEngine channel */ chan = rsnd_dmaen_request_channel(io, mod_from, mod_to); if (IS_ERR_OR_NULL(chan)) { + /* Let's follow when -EPROBE_DEFER case */ + if (PTR_ERR(chan) == -EPROBE_DEFER) + return PTR_ERR(chan); + /* * DMA failed. try to PIO mode * see diff --git a/sound/soc/sh/rcar/rsnd.h b/sound/soc/sh/rcar/rsnd.h index 96d93330b1e1..8f7a0abfa751 100644 --- a/sound/soc/sh/rcar/rsnd.h +++ b/sound/soc/sh/rcar/rsnd.h @@ -280,6 +280,9 @@ struct rsnd_mod_ops { int (*nolock_stop)(struct rsnd_mod *mod, struct rsnd_dai_stream *io, struct rsnd_priv *priv); + int (*prepare)(struct rsnd_mod *mod, + struct rsnd_dai_stream *io, + struct rsnd_priv *priv); }; struct rsnd_dai_stream; @@ -309,6 +312,7 @@ struct rsnd_mod { * H 0: fallback * H 0: hw_params * H 0: pointer + * H 0: prepare */ #define __rsnd_mod_shift_nolock_start 0 #define __rsnd_mod_shift_nolock_stop 0 @@ -323,6 +327,7 @@ struct rsnd_mod { #define __rsnd_mod_shift_fallback 28 /* always called */ #define __rsnd_mod_shift_hw_params 28 /* always called */ #define __rsnd_mod_shift_pointer 28 /* always called */ +#define __rsnd_mod_shift_prepare 28 /* always called */ #define __rsnd_mod_add_probe 0 #define __rsnd_mod_add_remove 0 @@ -337,6 +342,7 @@ struct rsnd_mod { #define __rsnd_mod_add_fallback 0 #define __rsnd_mod_add_hw_params 0 #define __rsnd_mod_add_pointer 0 +#define __rsnd_mod_add_prepare 0 #define __rsnd_mod_call_probe 0 #define __rsnd_mod_call_remove 0 @@ -351,6 +357,7 @@ struct rsnd_mod { #define __rsnd_mod_call_pointer 0 #define __rsnd_mod_call_nolock_start 0 #define __rsnd_mod_call_nolock_stop 1 +#define __rsnd_mod_call_prepare 0 #define rsnd_mod_to_priv(mod) ((mod)->priv) #define rsnd_mod_name(mod) ((mod)->ops->name) diff --git a/sound/soc/sh/rcar/ssi.c b/sound/soc/sh/rcar/ssi.c index 8304e4ec9242..3f880ec66459 100644 --- a/sound/soc/sh/rcar/ssi.c +++ b/sound/soc/sh/rcar/ssi.c @@ -283,7 +283,7 @@ static int rsnd_ssi_master_clk_start(struct rsnd_mod *mod, if (rsnd_ssi_is_multi_slave(mod, io)) return 0; - if (ssi->usrcnt > 1) { + if (ssi->rate) { if (ssi->rate != rate) { dev_err(dev, "SSI parent/child should use same rate\n"); return -EINVAL; @@ -434,7 +434,6 @@ static int rsnd_ssi_init(struct rsnd_mod *mod, struct rsnd_priv *priv) { struct rsnd_ssi *ssi = rsnd_mod_to_ssi(mod); - int ret; if (!rsnd_ssi_is_run_mods(mod, io)) return 0; @@ -443,10 +442,6 @@ static int rsnd_ssi_init(struct rsnd_mod *mod, rsnd_mod_power_on(mod); - ret = rsnd_ssi_master_clk_start(mod, io); - if (ret < 0) - return ret; - rsnd_ssi_config_init(mod, io); rsnd_ssi_register_setup(mod); @@ -852,6 +847,13 @@ static int rsnd_ssi_pio_pointer(struct rsnd_mod *mod, return 0; } +static int rsnd_ssi_prepare(struct rsnd_mod *mod, + struct rsnd_dai_stream *io, + struct rsnd_priv *priv) +{ + return rsnd_ssi_master_clk_start(mod, io); +} + static struct rsnd_mod_ops rsnd_ssi_pio_ops = { .name = SSI_NAME, .probe = rsnd_ssi_common_probe, @@ -864,6 +866,7 @@ static struct rsnd_mod_ops rsnd_ssi_pio_ops = { .pointer = rsnd_ssi_pio_pointer, .pcm_new = rsnd_ssi_pcm_new, .hw_params = rsnd_ssi_hw_params, + .prepare = rsnd_ssi_prepare, }; static int rsnd_ssi_dma_probe(struct rsnd_mod *mod, @@ -940,6 +943,7 @@ static struct rsnd_mod_ops rsnd_ssi_dma_ops = { .pcm_new = rsnd_ssi_pcm_new, .fallback = rsnd_ssi_fallback, .hw_params = rsnd_ssi_hw_params, + .prepare = rsnd_ssi_prepare, }; int rsnd_ssi_is_dma_mode(struct rsnd_mod *mod) diff --git a/sound/soc/soc-core.c b/sound/soc/soc-core.c index 9cfe10d8040c..473eefe8658e 100644 --- a/sound/soc/soc-core.c +++ b/sound/soc/soc-core.c @@ -1447,7 +1447,7 @@ static int soc_link_dai_widgets(struct snd_soc_card *card, sink = codec_dai->playback_widget; source = cpu_dai->capture_widget; if (sink && source) { - ret = snd_soc_dapm_new_pcm(card, dai_link->params, + ret = snd_soc_dapm_new_pcm(card, rtd, dai_link->params, dai_link->num_params, source, sink); if (ret != 0) { @@ -1460,7 +1460,7 @@ static int soc_link_dai_widgets(struct snd_soc_card *card, sink = cpu_dai->playback_widget; source = codec_dai->capture_widget; if (sink && source) { - ret = snd_soc_dapm_new_pcm(card, dai_link->params, + ret = snd_soc_dapm_new_pcm(card, rtd, dai_link->params, dai_link->num_params, source, sink); if (ret != 0) { diff --git a/sound/soc/soc-dapm.c b/sound/soc/soc-dapm.c index 7e96793050c9..461d951917c0 100644 --- a/sound/soc/soc-dapm.c +++ b/sound/soc/soc-dapm.c @@ -3652,6 +3652,7 @@ static int snd_soc_dai_link_event(struct snd_soc_dapm_widget *w, { struct snd_soc_dapm_path *source_p, *sink_p; struct snd_soc_dai *source, *sink; + struct snd_soc_pcm_runtime *rtd = w->priv; const struct snd_soc_pcm_stream *config = w->params + w->params_select; struct snd_pcm_substream substream; struct snd_pcm_hw_params *params = NULL; @@ -3711,6 +3712,7 @@ static int snd_soc_dai_link_event(struct snd_soc_dapm_widget *w, goto out; } substream.runtime = runtime; + substream.private_data = rtd; switch (event) { case SND_SOC_DAPM_PRE_PMU: @@ -3895,6 +3897,7 @@ outfree_w_param: } int snd_soc_dapm_new_pcm(struct snd_soc_card *card, + struct snd_soc_pcm_runtime *rtd, const struct snd_soc_pcm_stream *params, unsigned int num_params, struct snd_soc_dapm_widget *source, @@ -3963,6 +3966,7 @@ int snd_soc_dapm_new_pcm(struct snd_soc_card *card, w->params = params; w->num_params = num_params; + w->priv = rtd; ret = snd_soc_dapm_add_path(&card->dapm, source, w, NULL, NULL); if (ret) diff --git a/tools/arch/arm/include/uapi/asm/kvm.h b/tools/arch/arm/include/uapi/asm/kvm.h index 16e006f708ca..4602464ebdfb 100644 --- a/tools/arch/arm/include/uapi/asm/kvm.h +++ b/tools/arch/arm/include/uapi/asm/kvm.h @@ -27,6 +27,7 @@ #define __KVM_HAVE_GUEST_DEBUG #define __KVM_HAVE_IRQ_LINE #define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_VCPU_EVENTS #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 @@ -125,6 +126,18 @@ struct kvm_sync_regs { struct kvm_arch_memory_slot { }; +/* for KVM_GET/SET_VCPU_EVENTS */ +struct kvm_vcpu_events { + struct { + __u8 serror_pending; + __u8 serror_has_esr; + /* Align it to 8 bytes */ + __u8 pad[6]; + __u64 serror_esr; + } exception; + __u32 reserved[12]; +}; + /* If you need to interpret the index values, here is the key: */ #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 #define KVM_REG_ARM_COPROC_SHIFT 16 diff --git a/tools/arch/arm64/include/uapi/asm/kvm.h b/tools/arch/arm64/include/uapi/asm/kvm.h index 4e76630dd655..97c3478ee6e7 100644 --- a/tools/arch/arm64/include/uapi/asm/kvm.h +++ b/tools/arch/arm64/include/uapi/asm/kvm.h @@ -39,6 +39,7 @@ #define __KVM_HAVE_GUEST_DEBUG #define __KVM_HAVE_IRQ_LINE #define __KVM_HAVE_READONLY_MEM +#define __KVM_HAVE_VCPU_EVENTS #define KVM_COALESCED_MMIO_PAGE_OFFSET 1 @@ -154,6 +155,18 @@ struct kvm_sync_regs { struct kvm_arch_memory_slot { }; +/* for KVM_GET/SET_VCPU_EVENTS */ +struct kvm_vcpu_events { + struct { + __u8 serror_pending; + __u8 serror_has_esr; + /* Align it to 8 bytes */ + __u8 pad[6]; + __u64 serror_esr; + } exception; + __u32 reserved[12]; +}; + /* If you need to interpret the index values, here is the key: */ #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 #define KVM_REG_ARM_COPROC_SHIFT 16 diff --git a/tools/arch/s390/include/uapi/asm/kvm.h b/tools/arch/s390/include/uapi/asm/kvm.h index 4cdaa55fabfe..9a50f02b9894 100644 --- a/tools/arch/s390/include/uapi/asm/kvm.h +++ b/tools/arch/s390/include/uapi/asm/kvm.h @@ -4,7 +4,7 @@ /* * KVM s390 specific structures and definitions * - * Copyright IBM Corp. 2008 + * Copyright IBM Corp. 2008, 2018 * * Author(s): Carsten Otte <cotte@de.ibm.com> * Christian Borntraeger <borntraeger@de.ibm.com> @@ -225,6 +225,7 @@ struct kvm_guest_debug_arch { #define KVM_SYNC_FPRS (1UL << 8) #define KVM_SYNC_GSCB (1UL << 9) #define KVM_SYNC_BPBC (1UL << 10) +#define KVM_SYNC_ETOKEN (1UL << 11) /* length and alignment of the sdnx as a power of two */ #define SDNXC 8 #define SDNXL (1UL << SDNXC) @@ -258,6 +259,8 @@ struct kvm_sync_regs { struct { __u64 reserved1[2]; __u64 gscb[4]; + __u64 etoken; + __u64 etoken_extension; }; }; }; diff --git a/tools/arch/x86/include/uapi/asm/kvm.h b/tools/arch/x86/include/uapi/asm/kvm.h index c535c2fdea13..86299efa804a 100644 --- a/tools/arch/x86/include/uapi/asm/kvm.h +++ b/tools/arch/x86/include/uapi/asm/kvm.h @@ -378,4 +378,41 @@ struct kvm_sync_regs { #define KVM_X86_QUIRK_LINT0_REENABLED (1 << 0) #define KVM_X86_QUIRK_CD_NW_CLEARED (1 << 1) +#define KVM_STATE_NESTED_GUEST_MODE 0x00000001 +#define KVM_STATE_NESTED_RUN_PENDING 0x00000002 + +#define KVM_STATE_NESTED_SMM_GUEST_MODE 0x00000001 +#define KVM_STATE_NESTED_SMM_VMXON 0x00000002 + +struct kvm_vmx_nested_state { + __u64 vmxon_pa; + __u64 vmcs_pa; + + struct { + __u16 flags; + } smm; +}; + +/* for KVM_CAP_NESTED_STATE */ +struct kvm_nested_state { + /* KVM_STATE_* flags */ + __u16 flags; + + /* 0 for VMX, 1 for SVM. */ + __u16 format; + + /* 128 for SVM, 128 + VMCS size for VMX. */ + __u32 size; + + union { + /* VMXON, VMCS */ + struct kvm_vmx_nested_state vmx; + + /* Pad the header to 128 bytes. */ + __u8 pad[120]; + }; + + __u8 data[0]; +}; + #endif /* _ASM_X86_KVM_H */ diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c index b2ec20e562bd..b455930a3eaf 100644 --- a/tools/bpf/bpftool/map.c +++ b/tools/bpf/bpftool/map.c @@ -68,6 +68,7 @@ static const char * const map_type_name[] = { [BPF_MAP_TYPE_DEVMAP] = "devmap", [BPF_MAP_TYPE_SOCKMAP] = "sockmap", [BPF_MAP_TYPE_CPUMAP] = "cpumap", + [BPF_MAP_TYPE_XSKMAP] = "xskmap", [BPF_MAP_TYPE_SOCKHASH] = "sockhash", [BPF_MAP_TYPE_CGROUP_STORAGE] = "cgroup_storage", }; diff --git a/tools/hv/hv_kvp_daemon.c b/tools/hv/hv_kvp_daemon.c index dbf6e8bd98ba..bbb2a8ef367c 100644 --- a/tools/hv/hv_kvp_daemon.c +++ b/tools/hv/hv_kvp_daemon.c @@ -286,7 +286,7 @@ static int kvp_key_delete(int pool, const __u8 *key, int key_size) * Found a match; just move the remaining * entries up. */ - if (i == num_records) { + if (i == (num_records - 1)) { kvp_file_info[pool].num_records--; kvp_update_file(pool); return 0; diff --git a/tools/include/linux/lockdep.h b/tools/include/linux/lockdep.h index 6b0c36a58fcb..e56997288f2b 100644 --- a/tools/include/linux/lockdep.h +++ b/tools/include/linux/lockdep.h @@ -30,9 +30,12 @@ struct task_struct { struct held_lock held_locks[MAX_LOCK_DEPTH]; gfp_t lockdep_reclaim_gfp; int pid; + int state; char comm[17]; }; +#define TASK_RUNNING 0 + extern struct task_struct *__curr(void); #define current (__curr()) diff --git a/tools/include/linux/nmi.h b/tools/include/linux/nmi.h new file mode 100644 index 000000000000..e69de29bb2d1 --- /dev/null +++ b/tools/include/linux/nmi.h diff --git a/tools/include/tools/libc_compat.h b/tools/include/tools/libc_compat.h index 664ced8cb1b0..e907ba6f15e5 100644 --- a/tools/include/tools/libc_compat.h +++ b/tools/include/tools/libc_compat.h @@ -1,4 +1,4 @@ -// SPDX-License-Identifier: GPL-2.0+ +// SPDX-License-Identifier: (LGPL-2.0+ OR BSD-2-Clause) /* Copyright (C) 2018 Netronome Systems, Inc. */ #ifndef __TOOLS_LIBC_COMPAT_H diff --git a/tools/include/uapi/asm-generic/unistd.h b/tools/include/uapi/asm-generic/unistd.h index 42990676a55e..df4bedb9b01c 100644 --- a/tools/include/uapi/asm-generic/unistd.h +++ b/tools/include/uapi/asm-generic/unistd.h @@ -734,9 +734,11 @@ __SYSCALL(__NR_pkey_free, sys_pkey_free) __SYSCALL(__NR_statx, sys_statx) #define __NR_io_pgetevents 292 __SC_COMP(__NR_io_pgetevents, sys_io_pgetevents, compat_sys_io_pgetevents) +#define __NR_rseq 293 +__SYSCALL(__NR_rseq, sys_rseq) #undef __NR_syscalls -#define __NR_syscalls 293 +#define __NR_syscalls 294 /* * 32 bit systems traditionally used different diff --git a/tools/include/uapi/drm/drm.h b/tools/include/uapi/drm/drm.h index 9c660e1688ab..300f336633f2 100644 --- a/tools/include/uapi/drm/drm.h +++ b/tools/include/uapi/drm/drm.h @@ -687,6 +687,15 @@ struct drm_get_cap { */ #define DRM_CLIENT_CAP_ASPECT_RATIO 4 +/** + * DRM_CLIENT_CAP_WRITEBACK_CONNECTORS + * + * If set to 1, the DRM core will expose special connectors to be used for + * writing back to memory the scene setup in the commit. Depends on client + * also supporting DRM_CLIENT_CAP_ATOMIC + */ +#define DRM_CLIENT_CAP_WRITEBACK_CONNECTORS 5 + /** DRM_IOCTL_SET_CLIENT_CAP ioctl argument type */ struct drm_set_client_cap { __u64 capability; diff --git a/tools/include/uapi/linux/if_link.h b/tools/include/uapi/linux/if_link.h index cf01b6824244..43391e2d1153 100644 --- a/tools/include/uapi/linux/if_link.h +++ b/tools/include/uapi/linux/if_link.h @@ -164,6 +164,8 @@ enum { IFLA_CARRIER_UP_COUNT, IFLA_CARRIER_DOWN_COUNT, IFLA_NEW_IFINDEX, + IFLA_MIN_MTU, + IFLA_MAX_MTU, __IFLA_MAX }; @@ -334,6 +336,7 @@ enum { IFLA_BRPORT_GROUP_FWD_MASK, IFLA_BRPORT_NEIGH_SUPPRESS, IFLA_BRPORT_ISOLATED, + IFLA_BRPORT_BACKUP_PORT, __IFLA_BRPORT_MAX }; #define IFLA_BRPORT_MAX (__IFLA_BRPORT_MAX - 1) @@ -459,6 +462,16 @@ enum { #define IFLA_MACSEC_MAX (__IFLA_MACSEC_MAX - 1) +/* XFRM section */ +enum { + IFLA_XFRM_UNSPEC, + IFLA_XFRM_LINK, + IFLA_XFRM_IF_ID, + __IFLA_XFRM_MAX +}; + +#define IFLA_XFRM_MAX (__IFLA_XFRM_MAX - 1) + enum macsec_validation_type { MACSEC_VALIDATE_DISABLED = 0, MACSEC_VALIDATE_CHECK = 1, @@ -920,6 +933,7 @@ enum { XDP_ATTACHED_DRV, XDP_ATTACHED_SKB, XDP_ATTACHED_HW, + XDP_ATTACHED_MULTI, }; enum { @@ -928,6 +942,9 @@ enum { IFLA_XDP_ATTACHED, IFLA_XDP_FLAGS, IFLA_XDP_PROG_ID, + IFLA_XDP_DRV_PROG_ID, + IFLA_XDP_SKB_PROG_ID, + IFLA_XDP_HW_PROG_ID, __IFLA_XDP_MAX, }; diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h index b6270a3b38e9..07548de5c988 100644 --- a/tools/include/uapi/linux/kvm.h +++ b/tools/include/uapi/linux/kvm.h @@ -949,6 +949,9 @@ struct kvm_ppc_resize_hpt { #define KVM_CAP_GET_MSR_FEATURES 153 #define KVM_CAP_HYPERV_EVENTFD 154 #define KVM_CAP_HYPERV_TLBFLUSH 155 +#define KVM_CAP_S390_HPAGE_1M 156 +#define KVM_CAP_NESTED_STATE 157 +#define KVM_CAP_ARM_INJECT_SERROR_ESR 158 #ifdef KVM_CAP_IRQ_ROUTING @@ -1391,6 +1394,9 @@ struct kvm_enc_region { /* Available with KVM_CAP_HYPERV_EVENTFD */ #define KVM_HYPERV_EVENTFD _IOW(KVMIO, 0xbd, struct kvm_hyperv_eventfd) +/* Available with KVM_CAP_NESTED_STATE */ +#define KVM_GET_NESTED_STATE _IOWR(KVMIO, 0xbe, struct kvm_nested_state) +#define KVM_SET_NESTED_STATE _IOW(KVMIO, 0xbf, struct kvm_nested_state) /* Secure Encrypted Virtualization command */ enum sev_cmd_id { diff --git a/tools/include/uapi/linux/perf_event.h b/tools/include/uapi/linux/perf_event.h index eeb787b1c53c..f35eb72739c0 100644 --- a/tools/include/uapi/linux/perf_event.h +++ b/tools/include/uapi/linux/perf_event.h @@ -144,7 +144,7 @@ enum perf_event_sample_format { PERF_SAMPLE_MAX = 1U << 20, /* non-ABI */ - __PERF_SAMPLE_CALLCHAIN_EARLY = 1ULL << 63, + __PERF_SAMPLE_CALLCHAIN_EARLY = 1ULL << 63, /* non-ABI; internal use */ }; /* diff --git a/tools/include/uapi/linux/vhost.h b/tools/include/uapi/linux/vhost.h index c51f8e5cc608..84c3de89696a 100644 --- a/tools/include/uapi/linux/vhost.h +++ b/tools/include/uapi/linux/vhost.h @@ -65,6 +65,7 @@ struct vhost_iotlb_msg { }; #define VHOST_IOTLB_MSG 0x1 +#define VHOST_IOTLB_MSG_V2 0x2 struct vhost_msg { int type; @@ -74,6 +75,15 @@ struct vhost_msg { }; }; +struct vhost_msg_v2 { + __u32 type; + __u32 reserved; + union { + struct vhost_iotlb_msg iotlb; + __u8 padding[64]; + }; +}; + struct vhost_memory_region { __u64 guest_phys_addr; __u64 memory_size; /* bytes */ @@ -160,6 +170,14 @@ struct vhost_memory { #define VHOST_GET_VRING_BUSYLOOP_TIMEOUT _IOW(VHOST_VIRTIO, 0x24, \ struct vhost_vring_state) +/* Set or get vhost backend capability */ + +/* Use message type V2 */ +#define VHOST_BACKEND_F_IOTLB_MSG_V2 0x1 + +#define VHOST_SET_BACKEND_FEATURES _IOW(VHOST_VIRTIO, 0x25, __u64) +#define VHOST_GET_BACKEND_FEATURES _IOR(VHOST_VIRTIO, 0x26, __u64) + /* VHOST_NET specific defines */ /* Attach virtio net ring to a raw socket, or tap device. diff --git a/tools/kvm/kvm_stat/kvm_stat b/tools/kvm/kvm_stat/kvm_stat index 56c4b3f8a01b..439b8a27488d 100755 --- a/tools/kvm/kvm_stat/kvm_stat +++ b/tools/kvm/kvm_stat/kvm_stat @@ -759,12 +759,18 @@ class DebugfsProvider(Provider): if len(vms) == 0: self.do_read = False - self.paths = filter(lambda x: "{}-".format(pid) in x, vms) + self.paths = list(filter(lambda x: "{}-".format(pid) in x, vms)) else: self.paths = [] self.do_read = True - self.reset() + + def _verify_paths(self): + """Remove invalid paths""" + for path in self.paths: + if not os.path.exists(os.path.join(PATH_DEBUGFS_KVM, path)): + self.paths.remove(path) + continue def read(self, reset=0, by_guest=0): """Returns a dict with format:'file name / field -> current value'. @@ -780,6 +786,7 @@ class DebugfsProvider(Provider): # If no debugfs filtering support is available, then don't read. if not self.do_read: return results + self._verify_paths() paths = self.paths if self._pid == 0: @@ -1096,15 +1103,16 @@ class Tui(object): pid = self.stats.pid_filter self.screen.erase() gname = self.get_gname_from_pid(pid) + self._gname = gname if gname: gname = ('({})'.format(gname[:MAX_GUEST_NAME_LEN] + '...' if len(gname) > MAX_GUEST_NAME_LEN else gname)) if pid > 0: - self.screen.addstr(0, 0, 'kvm statistics - pid {0} {1}' - .format(pid, gname), curses.A_BOLD) + self._headline = 'kvm statistics - pid {0} {1}'.format(pid, gname) else: - self.screen.addstr(0, 0, 'kvm statistics - summary', curses.A_BOLD) + self._headline = 'kvm statistics - summary' + self.screen.addstr(0, 0, self._headline, curses.A_BOLD) if self.stats.fields_filter: regex = self.stats.fields_filter if len(regex) > MAX_REGEX_LEN: @@ -1162,6 +1170,19 @@ class Tui(object): return sorted_items + if not self._is_running_guest(self.stats.pid_filter): + if self._gname: + try: # ...to identify the guest by name in case it's back + pids = self.get_pid_from_gname(self._gname) + if len(pids) == 1: + self._refresh_header(pids[0]) + self._update_pid(pids[0]) + return + except: + pass + self._display_guest_dead() + # leave final data on screen + return row = 3 self.screen.move(row, 0) self.screen.clrtobot() @@ -1184,6 +1205,7 @@ class Tui(object): # print events tavg = 0 tcur = 0 + guest_removed = False for key, values in get_sorted_events(self, stats): if row >= self.screen.getmaxyx()[0] - 1 or values == (0, 0): break @@ -1191,7 +1213,10 @@ class Tui(object): key = self.get_gname_from_pid(key) if not key: continue - cur = int(round(values.delta / sleeptime)) if values.delta else '' + cur = int(round(values.delta / sleeptime)) if values.delta else 0 + if cur < 0: + guest_removed = True + continue if key[0] != ' ': if values.delta: tcur += values.delta @@ -1204,13 +1229,21 @@ class Tui(object): values.value * 100 / float(ltotal), cur)) row += 1 if row == 3: - self.screen.addstr(4, 1, 'No matching events reported yet') + if guest_removed: + self.screen.addstr(4, 1, 'Guest removed, updating...') + else: + self.screen.addstr(4, 1, 'No matching events reported yet') if row > 4: tavg = int(round(tcur / sleeptime)) if tcur > 0 else '' self.screen.addstr(row, 1, '%-40s %10d %8s' % ('Total', total, tavg), curses.A_BOLD) self.screen.refresh() + def _display_guest_dead(self): + marker = ' Guest is DEAD ' + y = min(len(self._headline), 80 - len(marker)) + self.screen.addstr(0, y, marker, curses.A_BLINK | curses.A_STANDOUT) + def _show_msg(self, text): """Display message centered text and exit on key press""" hint = 'Press any key to continue' @@ -1219,10 +1252,10 @@ class Tui(object): (x, term_width) = self.screen.getmaxyx() row = 2 for line in text: - start = (term_width - len(line)) / 2 + start = (term_width - len(line)) // 2 self.screen.addstr(row, start, line) row += 1 - self.screen.addstr(row + 1, (term_width - len(hint)) / 2, hint, + self.screen.addstr(row + 1, (term_width - len(hint)) // 2, hint, curses.A_STANDOUT) self.screen.getkey() @@ -1319,6 +1352,12 @@ class Tui(object): msg = '"' + str(val) + '": Invalid value' self._refresh_header() + def _is_running_guest(self, pid): + """Check if pid is still a running process.""" + if not pid: + return True + return os.path.isdir(os.path.join('/proc/', str(pid))) + def _show_vm_selection_by_guest(self): """Draws guest selection mask. @@ -1346,7 +1385,7 @@ class Tui(object): if not guest or guest == '0': break if guest.isdigit(): - if not os.path.isdir(os.path.join('/proc/', guest)): + if not self._is_running_guest(guest): msg = '"' + guest + '": Not a running process' continue pid = int(guest) diff --git a/tools/lib/bpf/Build b/tools/lib/bpf/Build index 13a861135127..6eb9bacd1948 100644 --- a/tools/lib/bpf/Build +++ b/tools/lib/bpf/Build @@ -1 +1 @@ -libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o +libbpf-y := libbpf.o bpf.o nlattr.o btf.o libbpf_errno.o str_error.o diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 2abd0f112627..bdb94939fd60 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -50,6 +50,7 @@ #include "libbpf.h" #include "bpf.h" #include "btf.h" +#include "str_error.h" #ifndef EM_BPF #define EM_BPF 247 @@ -469,7 +470,7 @@ static int bpf_object__elf_init(struct bpf_object *obj) obj->efile.fd = open(obj->path, O_RDONLY); if (obj->efile.fd < 0) { char errmsg[STRERR_BUFSIZE]; - char *cp = strerror_r(errno, errmsg, sizeof(errmsg)); + char *cp = str_error(errno, errmsg, sizeof(errmsg)); pr_warning("failed to open %s: %s\n", obj->path, cp); return -errno; @@ -810,8 +811,7 @@ static int bpf_object__elf_collect(struct bpf_object *obj) data->d_size, name, idx); if (err) { char errmsg[STRERR_BUFSIZE]; - char *cp = strerror_r(-err, errmsg, - sizeof(errmsg)); + char *cp = str_error(-err, errmsg, sizeof(errmsg)); pr_warning("failed to alloc program %s (%s): %s", name, obj->path, cp); @@ -1140,7 +1140,7 @@ bpf_object__create_maps(struct bpf_object *obj) *pfd = bpf_create_map_xattr(&create_attr); if (*pfd < 0 && create_attr.btf_key_type_id) { - cp = strerror_r(errno, errmsg, sizeof(errmsg)); + cp = str_error(errno, errmsg, sizeof(errmsg)); pr_warning("Error in bpf_create_map_xattr(%s):%s(%d). Retrying without BTF.\n", map->name, cp, errno); create_attr.btf_fd = 0; @@ -1155,7 +1155,7 @@ bpf_object__create_maps(struct bpf_object *obj) size_t j; err = *pfd; - cp = strerror_r(errno, errmsg, sizeof(errmsg)); + cp = str_error(errno, errmsg, sizeof(errmsg)); pr_warning("failed to create map (name: '%s'): %s\n", map->name, cp); for (j = 0; j < i; j++) @@ -1339,7 +1339,7 @@ load_program(enum bpf_prog_type type, enum bpf_attach_type expected_attach_type, } ret = -LIBBPF_ERRNO__LOAD; - cp = strerror_r(errno, errmsg, sizeof(errmsg)); + cp = str_error(errno, errmsg, sizeof(errmsg)); pr_warning("load bpf program failed: %s\n", cp); if (log_buf && log_buf[0] != '\0') { @@ -1654,7 +1654,7 @@ static int check_path(const char *path) dir = dirname(dname); if (statfs(dir, &st_fs)) { - cp = strerror_r(errno, errmsg, sizeof(errmsg)); + cp = str_error(errno, errmsg, sizeof(errmsg)); pr_warning("failed to statfs %s: %s\n", dir, cp); err = -errno; } @@ -1690,7 +1690,7 @@ int bpf_program__pin_instance(struct bpf_program *prog, const char *path, } if (bpf_obj_pin(prog->instances.fds[instance], path)) { - cp = strerror_r(errno, errmsg, sizeof(errmsg)); + cp = str_error(errno, errmsg, sizeof(errmsg)); pr_warning("failed to pin program: %s\n", cp); return -errno; } @@ -1708,7 +1708,7 @@ static int make_dir(const char *path) err = -errno; if (err) { - cp = strerror_r(-err, errmsg, sizeof(errmsg)); + cp = str_error(-err, errmsg, sizeof(errmsg)); pr_warning("failed to mkdir %s: %s\n", path, cp); } return err; @@ -1770,7 +1770,7 @@ int bpf_map__pin(struct bpf_map *map, const char *path) } if (bpf_obj_pin(map->fd, path)) { - cp = strerror_r(errno, errmsg, sizeof(errmsg)); + cp = str_error(errno, errmsg, sizeof(errmsg)); pr_warning("failed to pin map: %s\n", cp); return -errno; } diff --git a/tools/lib/bpf/str_error.c b/tools/lib/bpf/str_error.c new file mode 100644 index 000000000000..b8798114a357 --- /dev/null +++ b/tools/lib/bpf/str_error.c @@ -0,0 +1,18 @@ +// SPDX-License-Identifier: LGPL-2.1 +#undef _GNU_SOURCE +#include <string.h> +#include <stdio.h> +#include "str_error.h" + +/* + * Wrapper to allow for building in non-GNU systems such as Alpine Linux's musl + * libc, while checking strerror_r() return to avoid having to check this in + * all places calling it. + */ +char *str_error(int err, char *dst, int len) +{ + int ret = strerror_r(err, dst, len); + if (ret) + snprintf(dst, len, "ERROR: strerror_r(%d)=%d", err, ret); + return dst; +} diff --git a/tools/lib/bpf/str_error.h b/tools/lib/bpf/str_error.h new file mode 100644 index 000000000000..355b1db571d1 --- /dev/null +++ b/tools/lib/bpf/str_error.h @@ -0,0 +1,6 @@ +// SPDX-License-Identifier: LGPL-2.1 +#ifndef BPF_STR_ERROR +#define BPF_STR_ERROR + +char *str_error(int err, char *dst, int len); +#endif // BPF_STR_ERROR diff --git a/tools/perf/Documentation/Makefile b/tools/perf/Documentation/Makefile index 42261a9b280e..ac841bc5c35b 100644 --- a/tools/perf/Documentation/Makefile +++ b/tools/perf/Documentation/Makefile @@ -280,7 +280,7 @@ $(MAN_HTML): $(OUTPUT)%.html : %.txt mv $@+ $@ ifdef USE_ASCIIDOCTOR -$(OUTPUT)%.1 $(OUTPUT)%.5 $(OUTPUT)%.7 : $(OUTPUT)%.txt +$(OUTPUT)%.1 $(OUTPUT)%.5 $(OUTPUT)%.7 : %.txt $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ $(ASCIIDOC) -b manpage -d manpage \ $(ASCIIDOC_EXTRA) -aperf_version=$(PERF_VERSION) -o $@+ $< && \ diff --git a/tools/perf/Makefile.perf b/tools/perf/Makefile.perf index b3d1b12a5081..5224ade3d5af 100644 --- a/tools/perf/Makefile.perf +++ b/tools/perf/Makefile.perf @@ -777,14 +777,12 @@ endif $(call QUIET_INSTALL, libexec) \ $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perfexec_instdir_SQ)' ifndef NO_LIBBPF - $(call QUIET_INSTALL, lib) \ - $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf' - $(call QUIET_INSTALL, include/bpf) \ - $(INSTALL) include/bpf/*.h '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf' - $(call QUIET_INSTALL, lib) \ - $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf' - $(call QUIET_INSTALL, examples/bpf) \ - $(INSTALL) examples/bpf/*.c '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf' + $(call QUIET_INSTALL, bpf-headers) \ + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf'; \ + $(INSTALL) include/bpf/*.h -t '$(DESTDIR_SQ)$(perf_include_instdir_SQ)/bpf' + $(call QUIET_INSTALL, bpf-examples) \ + $(INSTALL) -d -m 755 '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf'; \ + $(INSTALL) examples/bpf/*.c -t '$(DESTDIR_SQ)$(perf_examples_instdir_SQ)/bpf' endif $(call QUIET_INSTALL, perf-archive) \ $(INSTALL) $(OUTPUT)perf-archive -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)' diff --git a/tools/perf/arch/arm64/Makefile b/tools/perf/arch/arm64/Makefile index f013b115dc86..dbef716a1913 100644 --- a/tools/perf/arch/arm64/Makefile +++ b/tools/perf/arch/arm64/Makefile @@ -11,7 +11,8 @@ PERF_HAVE_ARCH_REGS_QUERY_REGISTER_OFFSET := 1 out := $(OUTPUT)arch/arm64/include/generated/asm header := $(out)/syscalls.c -sysdef := $(srctree)/tools/include/uapi/asm-generic/unistd.h +incpath := $(srctree)/tools +sysdef := $(srctree)/tools/arch/arm64/include/uapi/asm/unistd.h sysprf := $(srctree)/tools/perf/arch/arm64/entry/syscalls/ systbl := $(sysprf)/mksyscalltbl @@ -19,7 +20,7 @@ systbl := $(sysprf)/mksyscalltbl _dummy := $(shell [ -d '$(out)' ] || mkdir -p '$(out)') $(header): $(sysdef) $(systbl) - $(Q)$(SHELL) '$(systbl)' '$(CC)' '$(HOSTCC)' $(sysdef) > $@ + $(Q)$(SHELL) '$(systbl)' '$(CC)' '$(HOSTCC)' $(incpath) $(sysdef) > $@ clean:: $(call QUIET_CLEAN, arm64) $(RM) $(header) diff --git a/tools/perf/arch/arm64/entry/syscalls/mksyscalltbl b/tools/perf/arch/arm64/entry/syscalls/mksyscalltbl index 52e197317d3e..2dbb8cade048 100755 --- a/tools/perf/arch/arm64/entry/syscalls/mksyscalltbl +++ b/tools/perf/arch/arm64/entry/syscalls/mksyscalltbl @@ -11,7 +11,8 @@ gcc=$1 hostcc=$2 -input=$3 +incpath=$3 +input=$4 if ! test -r $input; then echo "Could not read input file" >&2 @@ -28,7 +29,6 @@ create_table_from_c() cat <<-_EoHEADER #include <stdio.h> - #define __ARCH_WANT_RENAMEAT #include "$input" int main(int argc, char *argv[]) { @@ -42,7 +42,7 @@ create_table_from_c() printf "%s\n" " printf(\"#define SYSCALLTBL_ARM64_MAX_ID %d\\n\", __NR_$last_sc);" printf "}\n" - } | $hostcc -o $create_table_exe -x c - + } | $hostcc -I $incpath/include/uapi -o $create_table_exe -x c - $create_table_exe diff --git a/tools/perf/arch/powerpc/util/sym-handling.c b/tools/perf/arch/powerpc/util/sym-handling.c index 20e7d74d86cd..10a44e946f77 100644 --- a/tools/perf/arch/powerpc/util/sym-handling.c +++ b/tools/perf/arch/powerpc/util/sym-handling.c @@ -22,15 +22,16 @@ bool elf__needs_adjust_symbols(GElf_Ehdr ehdr) #endif -#if !defined(_CALL_ELF) || _CALL_ELF != 2 int arch__choose_best_symbol(struct symbol *syma, struct symbol *symb __maybe_unused) { char *sym = syma->name; +#if !defined(_CALL_ELF) || _CALL_ELF != 2 /* Skip over any initial dot */ if (*sym == '.') sym++; +#endif /* Avoid "SyS" kernel syscall aliases */ if (strlen(sym) >= 3 && !strncmp(sym, "SyS", 3)) @@ -41,6 +42,7 @@ int arch__choose_best_symbol(struct symbol *syma, return SYMBOL_A; } +#if !defined(_CALL_ELF) || _CALL_ELF != 2 /* Allow matching against dot variants */ int arch__compare_symbol_names(const char *namea, const char *nameb) { diff --git a/tools/perf/arch/x86/include/arch-tests.h b/tools/perf/arch/x86/include/arch-tests.h index c1bd979b957b..613709cfbbd0 100644 --- a/tools/perf/arch/x86/include/arch-tests.h +++ b/tools/perf/arch/x86/include/arch-tests.h @@ -9,6 +9,7 @@ struct test; int test__rdpmc(struct test *test __maybe_unused, int subtest); int test__perf_time_to_tsc(struct test *test __maybe_unused, int subtest); int test__insn_x86(struct test *test __maybe_unused, int subtest); +int test__bp_modify(struct test *test, int subtest); #ifdef HAVE_DWARF_UNWIND_SUPPORT struct thread; diff --git a/tools/perf/arch/x86/tests/Build b/tools/perf/arch/x86/tests/Build index 8e2c5a38c3b9..586849ff83a0 100644 --- a/tools/perf/arch/x86/tests/Build +++ b/tools/perf/arch/x86/tests/Build @@ -5,3 +5,4 @@ libperf-y += arch-tests.o libperf-y += rdpmc.o libperf-y += perf-time-to-tsc.o libperf-$(CONFIG_AUXTRACE) += insn-x86.o +libperf-$(CONFIG_X86_64) += bp-modify.o diff --git a/tools/perf/arch/x86/tests/arch-tests.c b/tools/perf/arch/x86/tests/arch-tests.c index cc1802ff5410..d47d3f8e3c8e 100644 --- a/tools/perf/arch/x86/tests/arch-tests.c +++ b/tools/perf/arch/x86/tests/arch-tests.c @@ -24,6 +24,12 @@ struct test arch_tests[] = { .func = test__insn_x86, }, #endif +#if defined(__x86_64__) + { + .desc = "x86 bp modify", + .func = test__bp_modify, + }, +#endif { .func = NULL, }, diff --git a/tools/perf/arch/x86/tests/bp-modify.c b/tools/perf/arch/x86/tests/bp-modify.c new file mode 100644 index 000000000000..f53e4406709f --- /dev/null +++ b/tools/perf/arch/x86/tests/bp-modify.c @@ -0,0 +1,213 @@ +// SPDX-License-Identifier: GPL-2.0 +#include <linux/compiler.h> +#include <sys/types.h> +#include <sys/wait.h> +#include <sys/user.h> +#include <syscall.h> +#include <unistd.h> +#include <stdio.h> +#include <stdlib.h> +#include <sys/ptrace.h> +#include <asm/ptrace.h> +#include <errno.h> +#include "debug.h" +#include "tests/tests.h" +#include "arch-tests.h" + +static noinline int bp_1(void) +{ + pr_debug("in %s\n", __func__); + return 0; +} + +static noinline int bp_2(void) +{ + pr_debug("in %s\n", __func__); + return 0; +} + +static int spawn_child(void) +{ + int child = fork(); + + if (child == 0) { + /* + * The child sets itself for as tracee and + * waits in signal for parent to trace it, + * then it calls bp_1 and quits. + */ + int err = ptrace(PTRACE_TRACEME, 0, NULL, NULL); + + if (err) { + pr_debug("failed to PTRACE_TRACEME\n"); + exit(1); + } + + raise(SIGCONT); + bp_1(); + exit(0); + } + + return child; +} + +/* + * This tests creates HW breakpoint, tries to + * change it and checks it was properly changed. + */ +static int bp_modify1(void) +{ + pid_t child; + int status; + unsigned long rip = 0, dr7 = 1; + + child = spawn_child(); + + waitpid(child, &status, 0); + if (WIFEXITED(status)) { + pr_debug("tracee exited prematurely 1\n"); + return TEST_FAIL; + } + + /* + * The parent does following steps: + * - creates a new breakpoint (id 0) for bp_2 function + * - changes that breakponit to bp_1 function + * - waits for the breakpoint to hit and checks + * it has proper rip of bp_1 function + * - detaches the child + */ + if (ptrace(PTRACE_POKEUSER, child, + offsetof(struct user, u_debugreg[0]), bp_2)) { + pr_debug("failed to set breakpoint, 1st time: %s\n", + strerror(errno)); + goto out; + } + + if (ptrace(PTRACE_POKEUSER, child, + offsetof(struct user, u_debugreg[0]), bp_1)) { + pr_debug("failed to set breakpoint, 2nd time: %s\n", + strerror(errno)); + goto out; + } + + if (ptrace(PTRACE_POKEUSER, child, + offsetof(struct user, u_debugreg[7]), dr7)) { + pr_debug("failed to set dr7: %s\n", strerror(errno)); + goto out; + } + + if (ptrace(PTRACE_CONT, child, NULL, NULL)) { + pr_debug("failed to PTRACE_CONT: %s\n", strerror(errno)); + goto out; + } + + waitpid(child, &status, 0); + if (WIFEXITED(status)) { + pr_debug("tracee exited prematurely 2\n"); + return TEST_FAIL; + } + + rip = ptrace(PTRACE_PEEKUSER, child, + offsetof(struct user_regs_struct, rip), NULL); + if (rip == (unsigned long) -1) { + pr_debug("failed to PTRACE_PEEKUSER: %s\n", + strerror(errno)); + goto out; + } + + pr_debug("rip %lx, bp_1 %p\n", rip, bp_1); + +out: + if (ptrace(PTRACE_DETACH, child, NULL, NULL)) { + pr_debug("failed to PTRACE_DETACH: %s", strerror(errno)); + return TEST_FAIL; + } + + return rip == (unsigned long) bp_1 ? TEST_OK : TEST_FAIL; +} + +/* + * This tests creates HW breakpoint, tries to + * change it to bogus value and checks the original + * breakpoint is hit. + */ +static int bp_modify2(void) +{ + pid_t child; + int status; + unsigned long rip = 0, dr7 = 1; + + child = spawn_child(); + + waitpid(child, &status, 0); + if (WIFEXITED(status)) { + pr_debug("tracee exited prematurely 1\n"); + return TEST_FAIL; + } + + /* + * The parent does following steps: + * - creates a new breakpoint (id 0) for bp_1 function + * - tries to change that breakpoint to (-1) address + * - waits for the breakpoint to hit and checks + * it has proper rip of bp_1 function + * - detaches the child + */ + if (ptrace(PTRACE_POKEUSER, child, + offsetof(struct user, u_debugreg[0]), bp_1)) { + pr_debug("failed to set breakpoint: %s\n", + strerror(errno)); + goto out; + } + + if (ptrace(PTRACE_POKEUSER, child, + offsetof(struct user, u_debugreg[7]), dr7)) { + pr_debug("failed to set dr7: %s\n", strerror(errno)); + goto out; + } + + if (!ptrace(PTRACE_POKEUSER, child, + offsetof(struct user, u_debugreg[0]), (unsigned long) (-1))) { + pr_debug("failed, breakpoint set to bogus address\n"); + goto out; + } + + if (ptrace(PTRACE_CONT, child, NULL, NULL)) { + pr_debug("failed to PTRACE_CONT: %s\n", strerror(errno)); + goto out; + } + + waitpid(child, &status, 0); + if (WIFEXITED(status)) { + pr_debug("tracee exited prematurely 2\n"); + return TEST_FAIL; + } + + rip = ptrace(PTRACE_PEEKUSER, child, + offsetof(struct user_regs_struct, rip), NULL); + if (rip == (unsigned long) -1) { + pr_debug("failed to PTRACE_PEEKUSER: %s\n", + strerror(errno)); + goto out; + } + + pr_debug("rip %lx, bp_1 %p\n", rip, bp_1); + +out: + if (ptrace(PTRACE_DETACH, child, NULL, NULL)) { + pr_debug("failed to PTRACE_DETACH: %s", strerror(errno)); + return TEST_FAIL; + } + + return rip == (unsigned long) bp_1 ? TEST_OK : TEST_FAIL; +} + +int test__bp_modify(struct test *test __maybe_unused, + int subtest __maybe_unused) +{ + TEST_ASSERT_VAL("modify test 1 failed\n", !bp_modify1()); + TEST_ASSERT_VAL("modify test 2 failed\n", !bp_modify2()); + + return 0; +} diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c index 20061cf42288..28cd6a17491b 100644 --- a/tools/perf/util/annotate.c +++ b/tools/perf/util/annotate.c @@ -246,8 +246,14 @@ find_target: indirect_call: tok = strchr(endptr, '*'); - if (tok != NULL) - ops->target.addr = strtoull(tok + 1, NULL, 16); + if (tok != NULL) { + endptr++; + + /* Indirect call can use a non-rip register and offset: callq *0x8(%rbx). + * Do not parse such instruction. */ + if (strstr(endptr, "(%r") == NULL) + ops->target.addr = strtoull(endptr, NULL, 16); + } goto find_target; } @@ -276,7 +282,19 @@ bool ins__is_call(const struct ins *ins) return ins->ops == &call_ops || ins->ops == &s390_call_ops; } -static int jump__parse(struct arch *arch __maybe_unused, struct ins_operands *ops, struct map_symbol *ms) +/* + * Prevents from matching commas in the comment section, e.g.: + * ffff200008446e70: b.cs ffff2000084470f4 <generic_exec_single+0x314> // b.hs, b.nlast + */ +static inline const char *validate_comma(const char *c, struct ins_operands *ops) +{ + if (ops->raw_comment && c > ops->raw_comment) + return NULL; + + return c; +} + +static int jump__parse(struct arch *arch, struct ins_operands *ops, struct map_symbol *ms) { struct map *map = ms->map; struct symbol *sym = ms->sym; @@ -285,6 +303,10 @@ static int jump__parse(struct arch *arch __maybe_unused, struct ins_operands *op }; const char *c = strchr(ops->raw, ','); u64 start, end; + + ops->raw_comment = strchr(ops->raw, arch->objdump.comment_char); + c = validate_comma(c, ops); + /* * Examples of lines to parse for the _cpp_lex_token@@Base * function: @@ -304,6 +326,7 @@ static int jump__parse(struct arch *arch __maybe_unused, struct ins_operands *op ops->target.addr = strtoull(c, NULL, 16); if (!ops->target.addr) { c = strchr(c, ','); + c = validate_comma(c, ops); if (c++ != NULL) ops->target.addr = strtoull(c, NULL, 16); } @@ -361,9 +384,12 @@ static int jump__scnprintf(struct ins *ins, char *bf, size_t size, return scnprintf(bf, size, "%-6s %s", ins->name, ops->target.sym->name); c = strchr(ops->raw, ','); + c = validate_comma(c, ops); + if (c != NULL) { const char *c2 = strchr(c + 1, ','); + c2 = validate_comma(c2, ops); /* check for 3-op insn */ if (c2 != NULL) c = c2; diff --git a/tools/perf/util/annotate.h b/tools/perf/util/annotate.h index 005a5fe8a8c6..5399ba2321bb 100644 --- a/tools/perf/util/annotate.h +++ b/tools/perf/util/annotate.h @@ -22,6 +22,7 @@ struct ins { struct ins_operands { char *raw; + char *raw_comment; struct { char *raw; char *name; diff --git a/tools/perf/util/evsel.c b/tools/perf/util/evsel.c index c980bbff6353..1a61628a1c12 100644 --- a/tools/perf/util/evsel.c +++ b/tools/perf/util/evsel.c @@ -251,8 +251,9 @@ struct perf_evsel *perf_evsel__new_idx(struct perf_event_attr *attr, int idx) { struct perf_evsel *evsel = zalloc(perf_evsel__object.size); - if (evsel != NULL) - perf_evsel__init(evsel, attr, idx); + if (!evsel) + return NULL; + perf_evsel__init(evsel, attr, idx); if (perf_evsel__is_bpf_output(evsel)) { evsel->attr.sample_type |= (PERF_SAMPLE_RAW | PERF_SAMPLE_TIME | diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c index 36d0763311ef..6a6929f208b4 100644 --- a/tools/perf/util/map.c +++ b/tools/perf/util/map.c @@ -576,6 +576,13 @@ struct symbol *map_groups__find_symbol(struct map_groups *mg, return NULL; } +static bool map__contains_symbol(struct map *map, struct symbol *sym) +{ + u64 ip = map->unmap_ip(map, sym->start); + + return ip >= map->start && ip < map->end; +} + struct symbol *maps__find_symbol_by_name(struct maps *maps, const char *name, struct map **mapp) { @@ -591,6 +598,10 @@ struct symbol *maps__find_symbol_by_name(struct maps *maps, const char *name, if (sym == NULL) continue; + if (!map__contains_symbol(pos, sym)) { + sym = NULL; + continue; + } if (mapp != NULL) *mapp = pos; goto out; diff --git a/tools/perf/util/trace-event-info.c b/tools/perf/util/trace-event-info.c index c85d0d1a65ed..7b0ca7cbb7de 100644 --- a/tools/perf/util/trace-event-info.c +++ b/tools/perf/util/trace-event-info.c @@ -377,7 +377,7 @@ out: static int record_saved_cmdline(void) { - unsigned int size; + unsigned long long size; char *path; struct stat st; int ret, err = 0; diff --git a/tools/perf/util/trace-event-parse.c b/tools/perf/util/trace-event-parse.c index 920b1d58a068..e76214f8d596 100644 --- a/tools/perf/util/trace-event-parse.c +++ b/tools/perf/util/trace-event-parse.c @@ -164,16 +164,15 @@ void parse_ftrace_printk(struct tep_handle *pevent, void parse_saved_cmdline(struct tep_handle *pevent, char *file, unsigned int size __maybe_unused) { - char *comm; + char comm[17]; /* Max comm length in the kernel is 16. */ char *line; char *next = NULL; int pid; line = strtok_r(file, "\n", &next); while (line) { - sscanf(line, "%d %ms", &pid, &comm); - tep_register_comm(pevent, comm, pid); - free(comm); + if (sscanf(line, "%d %16s", &pid, comm) == 2) + tep_register_comm(pevent, comm, pid); line = strtok_r(NULL, "\n", &next); } } diff --git a/tools/testing/selftests/android/Makefile b/tools/testing/selftests/android/Makefile index 72c25a3cb658..d9a725478375 100644 --- a/tools/testing/selftests/android/Makefile +++ b/tools/testing/selftests/android/Makefile @@ -6,7 +6,7 @@ TEST_PROGS := run.sh include ../lib.mk -all: +all: khdr @for DIR in $(SUBDIRS); do \ BUILD_TARGET=$(OUTPUT)/$$DIR; \ mkdir $$BUILD_TARGET -p; \ diff --git a/tools/testing/selftests/android/ion/config b/tools/testing/selftests/android/config index b4ad748a9dd9..b4ad748a9dd9 100644 --- a/tools/testing/selftests/android/ion/config +++ b/tools/testing/selftests/android/config diff --git a/tools/testing/selftests/android/ion/Makefile b/tools/testing/selftests/android/ion/Makefile index e03695287f76..88cfe88e466f 100644 --- a/tools/testing/selftests/android/ion/Makefile +++ b/tools/testing/selftests/android/ion/Makefile @@ -10,6 +10,8 @@ $(TEST_GEN_FILES): ipcsocket.c ionutils.c TEST_PROGS := ion_test.sh +KSFT_KHDR_INSTALL := 1 +top_srcdir = ../../../../.. include ../../lib.mk $(OUTPUT)/ionapp_export: ionapp_export.c ipcsocket.c ionutils.c diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c index 6f54f84144a0..9b552c0fc47d 100644 --- a/tools/testing/selftests/bpf/test_maps.c +++ b/tools/testing/selftests/bpf/test_maps.c @@ -580,7 +580,11 @@ static void test_sockmap(int tasks, void *data) /* Test update without programs */ for (i = 0; i < 6; i++) { err = bpf_map_update_elem(fd, &i, &sfd[i], BPF_ANY); - if (err) { + if (i < 2 && !err) { + printf("Allowed update sockmap '%i:%i' not in ESTABLISHED\n", + i, sfd[i]); + goto out_sockmap; + } else if (i >= 2 && err) { printf("Failed noprog update sockmap '%i:%i'\n", i, sfd[i]); goto out_sockmap; @@ -741,7 +745,7 @@ static void test_sockmap(int tasks, void *data) } /* Test map update elem afterwards fd lives in fd and map_fd */ - for (i = 0; i < 6; i++) { + for (i = 2; i < 6; i++) { err = bpf_map_update_elem(map_fd_rx, &i, &sfd[i], BPF_ANY); if (err) { printf("Failed map_fd_rx update sockmap %i '%i:%i'\n", @@ -845,7 +849,7 @@ static void test_sockmap(int tasks, void *data) } /* Delete the elems without programs */ - for (i = 0; i < 6; i++) { + for (i = 2; i < 6; i++) { err = bpf_map_delete_elem(fd, &i); if (err) { printf("Failed delete sockmap %i '%i:%i'\n", diff --git a/tools/testing/selftests/cgroup/.gitignore b/tools/testing/selftests/cgroup/.gitignore index 95eb3a53c381..adacda50a4b2 100644 --- a/tools/testing/selftests/cgroup/.gitignore +++ b/tools/testing/selftests/cgroup/.gitignore @@ -1 +1,2 @@ test_memcontrol +test_core diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c index 1c5d2b2a583b..14c9fe284806 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.c +++ b/tools/testing/selftests/cgroup/cgroup_util.c @@ -89,17 +89,28 @@ int cg_read(const char *cgroup, const char *control, char *buf, size_t len) int cg_read_strcmp(const char *cgroup, const char *control, const char *expected) { - size_t size = strlen(expected) + 1; + size_t size; char *buf; + int ret; + + /* Handle the case of comparing against empty string */ + if (!expected) + size = 32; + else + size = strlen(expected) + 1; buf = malloc(size); if (!buf) return -1; - if (cg_read(cgroup, control, buf, size)) + if (cg_read(cgroup, control, buf, size)) { + free(buf); return -1; + } - return strcmp(expected, buf); + ret = strcmp(expected, buf); + free(buf); + return ret; } int cg_read_strstr(const char *cgroup, const char *control, const char *needle) @@ -337,3 +348,24 @@ int is_swap_enabled(void) return cnt > 1; } + +int set_oom_adj_score(int pid, int score) +{ + char path[PATH_MAX]; + int fd, len; + + sprintf(path, "/proc/%d/oom_score_adj", pid); + + fd = open(path, O_WRONLY | O_APPEND); + if (fd < 0) + return fd; + + len = dprintf(fd, "%d", score); + if (len < 0) { + close(fd); + return len; + } + + close(fd); + return 0; +} diff --git a/tools/testing/selftests/cgroup/cgroup_util.h b/tools/testing/selftests/cgroup/cgroup_util.h index 1ff6f9f1abdc..9ac8b7958f83 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.h +++ b/tools/testing/selftests/cgroup/cgroup_util.h @@ -40,3 +40,4 @@ extern int get_temp_fd(void); extern int alloc_pagecache(int fd, size_t size); extern int alloc_anon(const char *cgroup, void *arg); extern int is_swap_enabled(void); +extern int set_oom_adj_score(int pid, int score); diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index cf0bddc9d271..28d321ba311b 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -2,6 +2,7 @@ #define _GNU_SOURCE #include <linux/limits.h> +#include <linux/oom.h> #include <fcntl.h> #include <stdio.h> #include <stdlib.h> @@ -202,6 +203,36 @@ static int alloc_pagecache_50M_noexit(const char *cgroup, void *arg) return 0; } +static int alloc_anon_noexit(const char *cgroup, void *arg) +{ + int ppid = getppid(); + + if (alloc_anon(cgroup, arg)) + return -1; + + while (getppid() == ppid) + sleep(1); + + return 0; +} + +/* + * Wait until processes are killed asynchronously by the OOM killer + * If we exceed a timeout, fail. + */ +static int cg_test_proc_killed(const char *cgroup) +{ + int limit; + + for (limit = 10; limit > 0; limit--) { + if (cg_read_strcmp(cgroup, "cgroup.procs", "") == 0) + return 0; + + usleep(100000); + } + return -1; +} + /* * First, this test creates the following hierarchy: * A memory.min = 50M, memory.max = 200M @@ -964,6 +995,177 @@ cleanup: return ret; } +/* + * This test disables swapping and tries to allocate anonymous memory + * up to OOM with memory.group.oom set. Then it checks that all + * processes in the leaf (but not the parent) were killed. + */ +static int test_memcg_oom_group_leaf_events(const char *root) +{ + int ret = KSFT_FAIL; + char *parent, *child; + + parent = cg_name(root, "memcg_test_0"); + child = cg_name(root, "memcg_test_0/memcg_test_1"); + + if (!parent || !child) + goto cleanup; + + if (cg_create(parent)) + goto cleanup; + + if (cg_create(child)) + goto cleanup; + + if (cg_write(parent, "cgroup.subtree_control", "+memory")) + goto cleanup; + + if (cg_write(child, "memory.max", "50M")) + goto cleanup; + + if (cg_write(child, "memory.swap.max", "0")) + goto cleanup; + + if (cg_write(child, "memory.oom.group", "1")) + goto cleanup; + + cg_run_nowait(parent, alloc_anon_noexit, (void *) MB(60)); + cg_run_nowait(child, alloc_anon_noexit, (void *) MB(1)); + cg_run_nowait(child, alloc_anon_noexit, (void *) MB(1)); + if (!cg_run(child, alloc_anon, (void *)MB(100))) + goto cleanup; + + if (cg_test_proc_killed(child)) + goto cleanup; + + if (cg_read_key_long(child, "memory.events", "oom_kill ") <= 0) + goto cleanup; + + if (cg_read_key_long(parent, "memory.events", "oom_kill ") != 0) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (child) + cg_destroy(child); + if (parent) + cg_destroy(parent); + free(child); + free(parent); + + return ret; +} + +/* + * This test disables swapping and tries to allocate anonymous memory + * up to OOM with memory.group.oom set. Then it checks that all + * processes in the parent and leaf were killed. + */ +static int test_memcg_oom_group_parent_events(const char *root) +{ + int ret = KSFT_FAIL; + char *parent, *child; + + parent = cg_name(root, "memcg_test_0"); + child = cg_name(root, "memcg_test_0/memcg_test_1"); + + if (!parent || !child) + goto cleanup; + + if (cg_create(parent)) + goto cleanup; + + if (cg_create(child)) + goto cleanup; + + if (cg_write(parent, "memory.max", "80M")) + goto cleanup; + + if (cg_write(parent, "memory.swap.max", "0")) + goto cleanup; + + if (cg_write(parent, "memory.oom.group", "1")) + goto cleanup; + + cg_run_nowait(parent, alloc_anon_noexit, (void *) MB(60)); + cg_run_nowait(child, alloc_anon_noexit, (void *) MB(1)); + cg_run_nowait(child, alloc_anon_noexit, (void *) MB(1)); + + if (!cg_run(child, alloc_anon, (void *)MB(100))) + goto cleanup; + + if (cg_test_proc_killed(child)) + goto cleanup; + if (cg_test_proc_killed(parent)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (child) + cg_destroy(child); + if (parent) + cg_destroy(parent); + free(child); + free(parent); + + return ret; +} + +/* + * This test disables swapping and tries to allocate anonymous memory + * up to OOM with memory.group.oom set. Then it checks that all + * processes were killed except those set with OOM_SCORE_ADJ_MIN + */ +static int test_memcg_oom_group_score_events(const char *root) +{ + int ret = KSFT_FAIL; + char *memcg; + int safe_pid; + + memcg = cg_name(root, "memcg_test_0"); + + if (!memcg) + goto cleanup; + + if (cg_create(memcg)) + goto cleanup; + + if (cg_write(memcg, "memory.max", "50M")) + goto cleanup; + + if (cg_write(memcg, "memory.swap.max", "0")) + goto cleanup; + + if (cg_write(memcg, "memory.oom.group", "1")) + goto cleanup; + + safe_pid = cg_run_nowait(memcg, alloc_anon_noexit, (void *) MB(1)); + if (set_oom_adj_score(safe_pid, OOM_SCORE_ADJ_MIN)) + goto cleanup; + + cg_run_nowait(memcg, alloc_anon_noexit, (void *) MB(1)); + if (!cg_run(memcg, alloc_anon, (void *)MB(100))) + goto cleanup; + + if (cg_read_key_long(memcg, "memory.events", "oom_kill ") != 3) + goto cleanup; + + if (kill(safe_pid, SIGKILL)) + goto cleanup; + + ret = KSFT_PASS; + +cleanup: + if (memcg) + cg_destroy(memcg); + free(memcg); + + return ret; +} + + #define T(x) { x, #x } struct memcg_test { int (*fn)(const char *root); @@ -978,6 +1180,9 @@ struct memcg_test { T(test_memcg_oom_events), T(test_memcg_swap_max), T(test_memcg_sock), + T(test_memcg_oom_group_leaf_events), + T(test_memcg_oom_group_parent_events), + T(test_memcg_oom_group_score_events), }; #undef T diff --git a/tools/testing/selftests/efivarfs/config b/tools/testing/selftests/efivarfs/config new file mode 100644 index 000000000000..4e151f1005b2 --- /dev/null +++ b/tools/testing/selftests/efivarfs/config @@ -0,0 +1 @@ +CONFIG_EFIVAR_FS=y diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile index ff8feca49746..ad1eeb14fda7 100644 --- a/tools/testing/selftests/futex/functional/Makefile +++ b/tools/testing/selftests/futex/functional/Makefile @@ -18,6 +18,7 @@ TEST_GEN_FILES := \ TEST_PROGS := run.sh +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_FILES): $(HEADERS) diff --git a/tools/testing/selftests/gpio/Makefile b/tools/testing/selftests/gpio/Makefile index 1bbb47565c55..4665cdbf1a8d 100644 --- a/tools/testing/selftests/gpio/Makefile +++ b/tools/testing/selftests/gpio/Makefile @@ -21,11 +21,8 @@ endef CFLAGS += -O2 -g -std=gnu99 -Wall -I../../../../usr/include/ LDLIBS += -lmount -I/usr/include/libmount -$(BINARIES): ../../../gpio/gpio-utils.o ../../../../usr/include/linux/gpio.h +$(BINARIES):| khdr +$(BINARIES): ../../../gpio/gpio-utils.o ../../../gpio/gpio-utils.o: make ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) -C ../../../gpio - -../../../../usr/include/linux/gpio.h: - make -C ../../../.. headers_install INSTALL_HDR_PATH=$(shell pwd)/../../../../usr/ - diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h index 15e6b75fc3a5..a3edb2c8e43d 100644 --- a/tools/testing/selftests/kselftest.h +++ b/tools/testing/selftests/kselftest.h @@ -19,7 +19,6 @@ #define KSFT_FAIL 1 #define KSFT_XFAIL 2 #define KSFT_XPASS 3 -/* Treat skip as pass */ #define KSFT_SKIP 4 /* counters */ diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore index 4202139d81d9..5c34752e1cff 100644 --- a/tools/testing/selftests/kvm/.gitignore +++ b/tools/testing/selftests/kvm/.gitignore @@ -1,4 +1,5 @@ cr4_cpuid_sync_test +platform_info_test set_sregs_test sync_regs_test vmx_tsc_adjust_test diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile index 03b0f551bedf..ec32dad3c3f0 100644 --- a/tools/testing/selftests/kvm/Makefile +++ b/tools/testing/selftests/kvm/Makefile @@ -6,7 +6,8 @@ UNAME_M := $(shell uname -m) LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c LIBKVM_x86_64 = lib/x86.c lib/vmx.c -TEST_GEN_PROGS_x86_64 = set_sregs_test +TEST_GEN_PROGS_x86_64 = platform_info_test +TEST_GEN_PROGS_x86_64 += set_sregs_test TEST_GEN_PROGS_x86_64 += sync_regs_test TEST_GEN_PROGS_x86_64 += vmx_tsc_adjust_test TEST_GEN_PROGS_x86_64 += cr4_cpuid_sync_test @@ -20,7 +21,7 @@ INSTALL_HDR_PATH = $(top_srcdir)/usr LINUX_HDR_PATH = $(INSTALL_HDR_PATH)/include/ LINUX_TOOL_INCLUDE = $(top_srcdir)tools/include CFLAGS += -O2 -g -std=gnu99 -I$(LINUX_TOOL_INCLUDE) -I$(LINUX_HDR_PATH) -Iinclude -I$(<D) -I.. -LDFLAGS += -lpthread +LDFLAGS += -pthread # After inclusion, $(OUTPUT) is defined and # $(TEST_GEN_PROGS) starts with $(OUTPUT)/ @@ -37,9 +38,6 @@ $(LIBKVM_OBJ): $(OUTPUT)/%.o: %.c $(OUTPUT)/libkvm.a: $(LIBKVM_OBJ) $(AR) crs $@ $^ -$(LINUX_HDR_PATH): - make -C $(top_srcdir) headers_install - -all: $(STATIC_LIBS) $(LINUX_HDR_PATH) +all: $(STATIC_LIBS) $(TEST_GEN_PROGS): $(STATIC_LIBS) -$(TEST_GEN_PROGS) $(LIBKVM_OBJ): | $(LINUX_HDR_PATH) +$(STATIC_LIBS):| khdr diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h index bb5a25fb82c6..3acf9a91704c 100644 --- a/tools/testing/selftests/kvm/include/kvm_util.h +++ b/tools/testing/selftests/kvm/include/kvm_util.h @@ -50,6 +50,7 @@ enum vm_mem_backing_src_type { }; int kvm_check_cap(long cap); +int vm_enable_cap(struct kvm_vm *vm, struct kvm_enable_cap *cap); struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm); void kvm_vm_free(struct kvm_vm *vmp); @@ -108,6 +109,9 @@ void vcpu_events_get(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_vcpu_events *events); void vcpu_events_set(struct kvm_vm *vm, uint32_t vcpuid, struct kvm_vcpu_events *events); +uint64_t vcpu_get_msr(struct kvm_vm *vm, uint32_t vcpuid, uint64_t msr_index); +void vcpu_set_msr(struct kvm_vm *vm, uint32_t vcpuid, uint64_t msr_index, + uint64_t msr_value); const char *exit_reason_str(unsigned int exit_reason); diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c index e9ba389c48db..6fd8c089cafc 100644 --- a/tools/testing/selftests/kvm/lib/kvm_util.c +++ b/tools/testing/selftests/kvm/lib/kvm_util.c @@ -63,6 +63,29 @@ int kvm_check_cap(long cap) return ret; } +/* VM Enable Capability + * + * Input Args: + * vm - Virtual Machine + * cap - Capability + * + * Output Args: None + * + * Return: On success, 0. On failure a TEST_ASSERT failure is produced. + * + * Enables a capability (KVM_CAP_*) on the VM. + */ +int vm_enable_cap(struct kvm_vm *vm, struct kvm_enable_cap *cap) +{ + int ret; + + ret = ioctl(vm->fd, KVM_ENABLE_CAP, cap); + TEST_ASSERT(ret == 0, "KVM_ENABLE_CAP IOCTL failed,\n" + " rc: %i errno: %i", ret, errno); + + return ret; +} + static void vm_open(struct kvm_vm *vm, int perm) { vm->kvm_fd = open(KVM_DEV_PATH, perm); @@ -1220,6 +1243,72 @@ void vcpu_events_set(struct kvm_vm *vm, uint32_t vcpuid, ret, errno); } +/* VCPU Get MSR + * + * Input Args: + * vm - Virtual Machine + * vcpuid - VCPU ID + * msr_index - Index of MSR + * + * Output Args: None + * + * Return: On success, value of the MSR. On failure a TEST_ASSERT is produced. + * + * Get value of MSR for VCPU. + */ +uint64_t vcpu_get_msr(struct kvm_vm *vm, uint32_t vcpuid, uint64_t msr_index) +{ + struct vcpu *vcpu = vcpu_find(vm, vcpuid); + struct { + struct kvm_msrs header; + struct kvm_msr_entry entry; + } buffer = {}; + int r; + + TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid); + buffer.header.nmsrs = 1; + buffer.entry.index = msr_index; + r = ioctl(vcpu->fd, KVM_GET_MSRS, &buffer.header); + TEST_ASSERT(r == 1, "KVM_GET_MSRS IOCTL failed,\n" + " rc: %i errno: %i", r, errno); + + return buffer.entry.data; +} + +/* VCPU Set MSR + * + * Input Args: + * vm - Virtual Machine + * vcpuid - VCPU ID + * msr_index - Index of MSR + * msr_value - New value of MSR + * + * Output Args: None + * + * Return: On success, nothing. On failure a TEST_ASSERT is produced. + * + * Set value of MSR for VCPU. + */ +void vcpu_set_msr(struct kvm_vm *vm, uint32_t vcpuid, uint64_t msr_index, + uint64_t msr_value) +{ + struct vcpu *vcpu = vcpu_find(vm, vcpuid); + struct { + struct kvm_msrs header; + struct kvm_msr_entry entry; + } buffer = {}; + int r; + + TEST_ASSERT(vcpu != NULL, "vcpu not found, vcpuid: %u", vcpuid); + memset(&buffer, 0, sizeof(buffer)); + buffer.header.nmsrs = 1; + buffer.entry.index = msr_index; + buffer.entry.data = msr_value; + r = ioctl(vcpu->fd, KVM_SET_MSRS, &buffer.header); + TEST_ASSERT(r == 1, "KVM_SET_MSRS IOCTL failed,\n" + " rc: %i errno: %i", r, errno); +} + /* VM VCPU Args Set * * Input Args: diff --git a/tools/testing/selftests/kvm/platform_info_test.c b/tools/testing/selftests/kvm/platform_info_test.c new file mode 100644 index 000000000000..3764e7121265 --- /dev/null +++ b/tools/testing/selftests/kvm/platform_info_test.c @@ -0,0 +1,110 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Test for x86 KVM_CAP_MSR_PLATFORM_INFO + * + * Copyright (C) 2018, Google LLC. + * + * This work is licensed under the terms of the GNU GPL, version 2. + * + * Verifies expected behavior of controlling guest access to + * MSR_PLATFORM_INFO. + */ + +#define _GNU_SOURCE /* for program_invocation_short_name */ +#include <fcntl.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <sys/ioctl.h> + +#include "test_util.h" +#include "kvm_util.h" +#include "x86.h" + +#define VCPU_ID 0 +#define MSR_PLATFORM_INFO_MAX_TURBO_RATIO 0xff00 + +static void guest_code(void) +{ + uint64_t msr_platform_info; + + for (;;) { + msr_platform_info = rdmsr(MSR_PLATFORM_INFO); + GUEST_SYNC(msr_platform_info); + asm volatile ("inc %r11"); + } +} + +static void set_msr_platform_info_enabled(struct kvm_vm *vm, bool enable) +{ + struct kvm_enable_cap cap = {}; + + cap.cap = KVM_CAP_MSR_PLATFORM_INFO; + cap.flags = 0; + cap.args[0] = (int)enable; + vm_enable_cap(vm, &cap); +} + +static void test_msr_platform_info_enabled(struct kvm_vm *vm) +{ + struct kvm_run *run = vcpu_state(vm, VCPU_ID); + struct guest_args args; + + set_msr_platform_info_enabled(vm, true); + vcpu_run(vm, VCPU_ID); + TEST_ASSERT(run->exit_reason == KVM_EXIT_IO, + "Exit_reason other than KVM_EXIT_IO: %u (%s),\n", + run->exit_reason, + exit_reason_str(run->exit_reason)); + guest_args_read(vm, VCPU_ID, &args); + TEST_ASSERT(args.port == GUEST_PORT_SYNC, + "Received IO from port other than PORT_HOST_SYNC: %u\n", + run->io.port); + TEST_ASSERT((args.arg1 & MSR_PLATFORM_INFO_MAX_TURBO_RATIO) == + MSR_PLATFORM_INFO_MAX_TURBO_RATIO, + "Expected MSR_PLATFORM_INFO to have max turbo ratio mask: %i.", + MSR_PLATFORM_INFO_MAX_TURBO_RATIO); +} + +static void test_msr_platform_info_disabled(struct kvm_vm *vm) +{ + struct kvm_run *run = vcpu_state(vm, VCPU_ID); + + set_msr_platform_info_enabled(vm, false); + vcpu_run(vm, VCPU_ID); + TEST_ASSERT(run->exit_reason == KVM_EXIT_SHUTDOWN, + "Exit_reason other than KVM_EXIT_SHUTDOWN: %u (%s)\n", + run->exit_reason, + exit_reason_str(run->exit_reason)); +} + +int main(int argc, char *argv[]) +{ + struct kvm_vm *vm; + struct kvm_run *state; + int rv; + uint64_t msr_platform_info; + + /* Tell stdout not to buffer its content */ + setbuf(stdout, NULL); + + rv = kvm_check_cap(KVM_CAP_MSR_PLATFORM_INFO); + if (!rv) { + fprintf(stderr, + "KVM_CAP_MSR_PLATFORM_INFO not supported, skip test\n"); + exit(KSFT_SKIP); + } + + vm = vm_create_default(VCPU_ID, 0, guest_code); + + msr_platform_info = vcpu_get_msr(vm, VCPU_ID, MSR_PLATFORM_INFO); + vcpu_set_msr(vm, VCPU_ID, MSR_PLATFORM_INFO, + msr_platform_info | MSR_PLATFORM_INFO_MAX_TURBO_RATIO); + test_msr_platform_info_disabled(vm); + test_msr_platform_info_enabled(vm); + vcpu_set_msr(vm, VCPU_ID, MSR_PLATFORM_INFO, msr_platform_info); + + kvm_vm_free(vm); + + return 0; +} diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk index 17ab36605a8e..0a8e75886224 100644 --- a/tools/testing/selftests/lib.mk +++ b/tools/testing/selftests/lib.mk @@ -16,8 +16,20 @@ TEST_GEN_PROGS := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS)) TEST_GEN_PROGS_EXTENDED := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_PROGS_EXTENDED)) TEST_GEN_FILES := $(patsubst %,$(OUTPUT)/%,$(TEST_GEN_FILES)) +top_srcdir ?= ../../../.. +include $(top_srcdir)/scripts/subarch.include +ARCH ?= $(SUBARCH) + all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES) +.PHONY: khdr +khdr: + make ARCH=$(ARCH) -C $(top_srcdir) headers_install + +ifdef KSFT_KHDR_INSTALL +$(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES):| khdr +endif + .ONESHELL: define RUN_TEST_PRINT_RESULT TEST_HDR_MSG="selftests: "`basename $$PWD`:" $$BASENAME_TEST"; \ diff --git a/tools/testing/selftests/memory-hotplug/config b/tools/testing/selftests/memory-hotplug/config index 2fde30191a47..a7e8cd5bb265 100644 --- a/tools/testing/selftests/memory-hotplug/config +++ b/tools/testing/selftests/memory-hotplug/config @@ -2,3 +2,4 @@ CONFIG_MEMORY_HOTPLUG=y CONFIG_MEMORY_HOTPLUG_SPARSE=y CONFIG_NOTIFIER_ERROR_INJECTION=y CONFIG_MEMORY_NOTIFIER_ERROR_INJECT=m +CONFIG_MEMORY_HOTREMOVE=y diff --git a/tools/testing/selftests/net/Makefile b/tools/testing/selftests/net/Makefile index 9cca68e440a0..919aa2ac00af 100644 --- a/tools/testing/selftests/net/Makefile +++ b/tools/testing/selftests/net/Makefile @@ -15,6 +15,7 @@ TEST_GEN_FILES += udpgso udpgso_bench_tx udpgso_bench_rx TEST_GEN_PROGS = reuseport_bpf reuseport_bpf_cpu reuseport_bpf_numa TEST_GEN_PROGS += reuseport_dualstack reuseaddr_conflict tls +KSFT_KHDR_INSTALL := 1 include ../lib.mk $(OUTPUT)/reuseport_bpf_numa: LDFLAGS += -lnuma diff --git a/tools/testing/selftests/net/pmtu.sh b/tools/testing/selftests/net/pmtu.sh index f8cc38afffa2..0ab9423d009f 100755 --- a/tools/testing/selftests/net/pmtu.sh +++ b/tools/testing/selftests/net/pmtu.sh @@ -46,6 +46,9 @@ # Kselftest framework requirement - SKIP code is 4. ksft_skip=4 +# Some systems don't have a ping6 binary anymore +which ping6 > /dev/null 2>&1 && ping6=$(which ping6) || ping6=$(which ping) + tests=" pmtu_vti6_exception vti6: PMTU exceptions pmtu_vti4_exception vti4: PMTU exceptions @@ -175,8 +178,8 @@ setup() { cleanup() { [ ${cleanup_done} -eq 1 ] && return - ip netns del ${NS_A} 2 > /dev/null - ip netns del ${NS_B} 2 > /dev/null + ip netns del ${NS_A} 2> /dev/null + ip netns del ${NS_B} 2> /dev/null cleanup_done=1 } @@ -274,7 +277,7 @@ test_pmtu_vti6_exception() { mtu "${ns_b}" veth_b 4000 mtu "${ns_a}" vti6_a 5000 mtu "${ns_b}" vti6_b 5000 - ${ns_a} ping6 -q -i 0.1 -w 2 -s 60000 ${vti6_b_addr} > /dev/null + ${ns_a} ${ping6} -q -i 0.1 -w 2 -s 60000 ${vti6_b_addr} > /dev/null # Check that exception was created if [ "$(route_get_dst_pmtu_from_exception "${ns_a}" ${vti6_b_addr})" = "" ]; then @@ -334,7 +337,7 @@ test_pmtu_vti4_link_add_mtu() { fail=0 min=68 - max=$((65528 - 20)) + max=$((65535 - 20)) # Check invalid values first for v in $((min - 1)) $((max + 1)); do ${ns_a} ip link add vti4_a mtu ${v} type vti local ${veth4_a_addr} remote ${veth4_b_addr} key 10 2>/dev/null diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c index b3ebf2646e52..8fdfeafaf8c0 100644 --- a/tools/testing/selftests/net/tls.c +++ b/tools/testing/selftests/net/tls.c @@ -502,6 +502,55 @@ TEST_F(tls, recv_peek_multiple) EXPECT_EQ(memcmp(test_str, buf, send_len), 0); } +TEST_F(tls, recv_peek_multiple_records) +{ + char const *test_str = "test_read_peek_mult_recs"; + char const *test_str_first = "test_read_peek"; + char const *test_str_second = "_mult_recs"; + int len; + char buf[64]; + + len = strlen(test_str_first); + EXPECT_EQ(send(self->fd, test_str_first, len, 0), len); + + len = strlen(test_str_second) + 1; + EXPECT_EQ(send(self->fd, test_str_second, len, 0), len); + + len = sizeof(buf); + memset(buf, 0, len); + EXPECT_NE(recv(self->cfd, buf, len, MSG_PEEK), -1); + + /* MSG_PEEK can only peek into the current record. */ + len = strlen(test_str_first) + 1; + EXPECT_EQ(memcmp(test_str_first, buf, len), 0); + + len = sizeof(buf); + memset(buf, 0, len); + EXPECT_NE(recv(self->cfd, buf, len, 0), -1); + + /* Non-MSG_PEEK will advance strparser (and therefore record) + * however. + */ + len = strlen(test_str) + 1; + EXPECT_EQ(memcmp(test_str, buf, len), 0); + + /* MSG_MORE will hold current record open, so later MSG_PEEK + * will see everything. + */ + len = strlen(test_str_first); + EXPECT_EQ(send(self->fd, test_str_first, len, MSG_MORE), len); + + len = strlen(test_str_second) + 1; + EXPECT_EQ(send(self->fd, test_str_second, len, 0), len); + + len = sizeof(buf); + memset(buf, 0, len); + EXPECT_NE(recv(self->cfd, buf, len, MSG_PEEK), -1); + + len = strlen(test_str) + 1; + EXPECT_EQ(memcmp(test_str, buf, len), 0); +} + TEST_F(tls, pollin) { char const *test_str = "test_poll"; diff --git a/tools/testing/selftests/networking/timestamping/Makefile b/tools/testing/selftests/networking/timestamping/Makefile index a728040edbe1..14cfcf006936 100644 --- a/tools/testing/selftests/networking/timestamping/Makefile +++ b/tools/testing/selftests/networking/timestamping/Makefile @@ -5,6 +5,7 @@ TEST_PROGS := hwtstamp_config rxtimestamp timestamping txtimestamp all: $(TEST_PROGS) +top_srcdir = ../../../../.. include ../../lib.mk clean: diff --git a/tools/testing/selftests/powerpc/alignment/Makefile b/tools/testing/selftests/powerpc/alignment/Makefile index 93baacab7693..d056486f49de 100644 --- a/tools/testing/selftests/powerpc/alignment/Makefile +++ b/tools/testing/selftests/powerpc/alignment/Makefile @@ -1,5 +1,6 @@ TEST_GEN_PROGS := copy_first_unaligned alignment_handler +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c ../utils.c diff --git a/tools/testing/selftests/powerpc/benchmarks/Makefile b/tools/testing/selftests/powerpc/benchmarks/Makefile index b4d7432a0ecd..d40300a65b42 100644 --- a/tools/testing/selftests/powerpc/benchmarks/Makefile +++ b/tools/testing/selftests/powerpc/benchmarks/Makefile @@ -4,6 +4,7 @@ TEST_GEN_FILES := exec_target CFLAGS += -O2 +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c diff --git a/tools/testing/selftests/powerpc/cache_shape/Makefile b/tools/testing/selftests/powerpc/cache_shape/Makefile index 1be547434a49..ede4d3dae750 100644 --- a/tools/testing/selftests/powerpc/cache_shape/Makefile +++ b/tools/testing/selftests/powerpc/cache_shape/Makefile @@ -5,6 +5,7 @@ all: $(TEST_PROGS) $(TEST_PROGS): ../harness.c ../utils.c +top_srcdir = ../../../../.. include ../../lib.mk clean: diff --git a/tools/testing/selftests/powerpc/copyloops/Makefile b/tools/testing/selftests/powerpc/copyloops/Makefile index 1cf89a34d97c..44574f3818b3 100644 --- a/tools/testing/selftests/powerpc/copyloops/Makefile +++ b/tools/testing/selftests/powerpc/copyloops/Makefile @@ -17,6 +17,7 @@ TEST_GEN_PROGS := copyuser_64_t0 copyuser_64_t1 copyuser_64_t2 \ EXTRA_SOURCES := validate.c ../harness.c stubs.S +top_srcdir = ../../../../.. include ../../lib.mk $(OUTPUT)/copyuser_64_t%: copyuser_64.S $(EXTRA_SOURCES) diff --git a/tools/testing/selftests/powerpc/dscr/Makefile b/tools/testing/selftests/powerpc/dscr/Makefile index 55d7db7a616b..5df476364b4d 100644 --- a/tools/testing/selftests/powerpc/dscr/Makefile +++ b/tools/testing/selftests/powerpc/dscr/Makefile @@ -3,6 +3,7 @@ TEST_GEN_PROGS := dscr_default_test dscr_explicit_test dscr_user_test \ dscr_inherit_test dscr_inherit_exec_test dscr_sysfs_test \ dscr_sysfs_thread_test +top_srcdir = ../../../../.. include ../../lib.mk $(OUTPUT)/dscr_default_test: LDLIBS += -lpthread diff --git a/tools/testing/selftests/powerpc/math/Makefile b/tools/testing/selftests/powerpc/math/Makefile index 0dd3a01fdab9..11a10d7a2bbd 100644 --- a/tools/testing/selftests/powerpc/math/Makefile +++ b/tools/testing/selftests/powerpc/math/Makefile @@ -1,6 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 TEST_GEN_PROGS := fpu_syscall fpu_preempt fpu_signal vmx_syscall vmx_preempt vmx_signal vsx_preempt +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c diff --git a/tools/testing/selftests/powerpc/mm/Makefile b/tools/testing/selftests/powerpc/mm/Makefile index 8ebbe96d80a8..33ced6e0ad25 100644 --- a/tools/testing/selftests/powerpc/mm/Makefile +++ b/tools/testing/selftests/powerpc/mm/Makefile @@ -5,6 +5,7 @@ noarg: TEST_GEN_PROGS := hugetlb_vs_thp_test subpage_prot prot_sao segv_errors TEST_GEN_FILES := tempfile +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c diff --git a/tools/testing/selftests/powerpc/pmu/Makefile b/tools/testing/selftests/powerpc/pmu/Makefile index 6e1629bf5b09..19046db995fe 100644 --- a/tools/testing/selftests/powerpc/pmu/Makefile +++ b/tools/testing/selftests/powerpc/pmu/Makefile @@ -5,6 +5,7 @@ noarg: TEST_GEN_PROGS := count_instructions l3_bank_test per_event_excludes EXTRA_SOURCES := ../harness.c event.c lib.c ../utils.c +top_srcdir = ../../../../.. include ../../lib.mk all: $(TEST_GEN_PROGS) ebb diff --git a/tools/testing/selftests/powerpc/pmu/ebb/Makefile b/tools/testing/selftests/powerpc/pmu/ebb/Makefile index c4e64bc2e265..bd5dfa509272 100644 --- a/tools/testing/selftests/powerpc/pmu/ebb/Makefile +++ b/tools/testing/selftests/powerpc/pmu/ebb/Makefile @@ -17,6 +17,7 @@ TEST_GEN_PROGS := reg_access_test event_attributes_test cycles_test \ lost_exception_test no_handler_test \ cycles_with_mmcr2_test +top_srcdir = ../../../../../.. include ../../../lib.mk $(TEST_GEN_PROGS): ../../harness.c ../../utils.c ../event.c ../lib.c \ diff --git a/tools/testing/selftests/powerpc/primitives/Makefile b/tools/testing/selftests/powerpc/primitives/Makefile index 175366db7be8..ea2b7bd09e36 100644 --- a/tools/testing/selftests/powerpc/primitives/Makefile +++ b/tools/testing/selftests/powerpc/primitives/Makefile @@ -2,6 +2,7 @@ CFLAGS += -I$(CURDIR) TEST_GEN_PROGS := load_unaligned_zeropad +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c diff --git a/tools/testing/selftests/powerpc/ptrace/Makefile b/tools/testing/selftests/powerpc/ptrace/Makefile index 28f5b781a553..923d531265f8 100644 --- a/tools/testing/selftests/powerpc/ptrace/Makefile +++ b/tools/testing/selftests/powerpc/ptrace/Makefile @@ -4,6 +4,7 @@ TEST_PROGS := ptrace-gpr ptrace-tm-gpr ptrace-tm-spd-gpr \ ptrace-tm-spd-vsx ptrace-tm-spr ptrace-hwbreak ptrace-pkey core-pkey \ perf-hwbreak +top_srcdir = ../../../../.. include ../../lib.mk all: $(TEST_PROGS) diff --git a/tools/testing/selftests/powerpc/signal/Makefile b/tools/testing/selftests/powerpc/signal/Makefile index a7cbd5082e27..1fca25c6ace0 100644 --- a/tools/testing/selftests/powerpc/signal/Makefile +++ b/tools/testing/selftests/powerpc/signal/Makefile @@ -8,6 +8,7 @@ $(TEST_PROGS): ../harness.c ../utils.c signal.S CFLAGS += -maltivec signal_tm: CFLAGS += -mhtm +top_srcdir = ../../../../.. include ../../lib.mk clean: diff --git a/tools/testing/selftests/powerpc/stringloops/Makefile b/tools/testing/selftests/powerpc/stringloops/Makefile index 10b35c87a4f4..7fc0623d85c3 100644 --- a/tools/testing/selftests/powerpc/stringloops/Makefile +++ b/tools/testing/selftests/powerpc/stringloops/Makefile @@ -29,6 +29,7 @@ endif ASFLAGS = $(CFLAGS) +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): $(EXTRA_SOURCES) diff --git a/tools/testing/selftests/powerpc/switch_endian/Makefile b/tools/testing/selftests/powerpc/switch_endian/Makefile index 30b8ff8fb82e..fcd2dcb8972b 100644 --- a/tools/testing/selftests/powerpc/switch_endian/Makefile +++ b/tools/testing/selftests/powerpc/switch_endian/Makefile @@ -5,6 +5,7 @@ ASFLAGS += -O2 -Wall -g -nostdlib -m64 EXTRA_CLEAN = $(OUTPUT)/*.o $(OUTPUT)/check-reversed.S +top_srcdir = ../../../../.. include ../../lib.mk $(OUTPUT)/switch_endian_test: $(OUTPUT)/check-reversed.S diff --git a/tools/testing/selftests/powerpc/syscalls/Makefile b/tools/testing/selftests/powerpc/syscalls/Makefile index da22ca7c38c1..161b8846336f 100644 --- a/tools/testing/selftests/powerpc/syscalls/Makefile +++ b/tools/testing/selftests/powerpc/syscalls/Makefile @@ -2,6 +2,7 @@ TEST_GEN_PROGS := ipc_unmuxed CFLAGS += -I../../../../../usr/include +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c diff --git a/tools/testing/selftests/powerpc/tm/Makefile b/tools/testing/selftests/powerpc/tm/Makefile index c0e45d2dde25..9fc2cf6fbc92 100644 --- a/tools/testing/selftests/powerpc/tm/Makefile +++ b/tools/testing/selftests/powerpc/tm/Makefile @@ -6,6 +6,7 @@ TEST_GEN_PROGS := tm-resched-dscr tm-syscall tm-signal-msr-resv tm-signal-stack tm-vmxcopy tm-fork tm-tar tm-tmspr tm-vmx-unavail tm-unavailable tm-trap \ $(SIGNAL_CONTEXT_CHK_TESTS) tm-sigreturn +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c ../utils.c diff --git a/tools/testing/selftests/powerpc/vphn/Makefile b/tools/testing/selftests/powerpc/vphn/Makefile index f8ced26748f8..fb82068c9fda 100644 --- a/tools/testing/selftests/powerpc/vphn/Makefile +++ b/tools/testing/selftests/powerpc/vphn/Makefile @@ -2,6 +2,7 @@ TEST_GEN_PROGS := test-vphn CFLAGS += -m64 +top_srcdir = ../../../../.. include ../../lib.mk $(TEST_GEN_PROGS): ../harness.c diff --git a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json index f03763d81617..30f9b54bd666 100644 --- a/tools/testing/selftests/tc-testing/tc-tests/actions/police.json +++ b/tools/testing/selftests/tc-testing/tc-tests/actions/police.json @@ -313,6 +313,54 @@ ] }, { + "id": "6aaf", + "name": "Add police actions with conform-exceed control pass/pipe [with numeric values]", + "category": [ + "actions", + "police" + ], + "setup": [ + [ + "$TC actions flush action police", + 0, + 1, + 255 + ] + ], + "cmdUnderTest": "$TC actions add action police rate 3mbit burst 250k conform-exceed 0/3 index 1", + "expExitCode": "0", + "verifyCmd": "$TC actions get action police index 1", + "matchPattern": "action order [0-9]*: police 0x1 rate 3Mbit burst 250Kb mtu 2Kb action pass/pipe", + "matchCount": "1", + "teardown": [ + "$TC actions flush action police" + ] + }, + { + "id": "29b1", + "name": "Add police actions with conform-exceed control <invalid>/drop", + "category": [ + "actions", + "police" + ], + "setup": [ + [ + "$TC actions flush action police", + 0, + 1, + 255 + ] + ], + "cmdUnderTest": "$TC actions add action police rate 3mbit burst 250k conform-exceed 10/drop index 1", + "expExitCode": "255", + "verifyCmd": "$TC actions ls action police", + "matchPattern": "action order [0-9]*: police 0x1 rate 3Mbit burst 250Kb mtu 2Kb action ", + "matchCount": "0", + "teardown": [ + "$TC actions flush action police" + ] + }, + { "id": "c26f", "name": "Add police action with invalid peakrate value", "category": [ diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm/Makefile index 9881876d2aa0..e94b7b14bcb2 100644 --- a/tools/testing/selftests/vm/Makefile +++ b/tools/testing/selftests/vm/Makefile @@ -26,10 +26,6 @@ TEST_PROGS := run_vmtests include ../lib.mk -$(OUTPUT)/userfaultfd: ../../../../usr/include/linux/kernel.h $(OUTPUT)/userfaultfd: LDLIBS += -lpthread $(OUTPUT)/mlock-random-test: LDLIBS += -lcap - -../../../../usr/include/linux/kernel.h: - make -C ../../../.. headers_install diff --git a/tools/vm/page-types.c b/tools/vm/page-types.c index 30cb0a0713ff..37908a83ddc2 100644 --- a/tools/vm/page-types.c +++ b/tools/vm/page-types.c @@ -159,12 +159,6 @@ static const char * const page_flag_names[] = { }; -static const char * const debugfs_known_mountpoints[] = { - "/sys/kernel/debug", - "/debug", - 0, -}; - /* * data structures */ diff --git a/tools/vm/slabinfo.c b/tools/vm/slabinfo.c index f82c2eaa859d..334b16db0ebb 100644 --- a/tools/vm/slabinfo.c +++ b/tools/vm/slabinfo.c @@ -30,8 +30,8 @@ struct slabinfo { int alias; int refs; int aliases, align, cache_dma, cpu_slabs, destroy_by_rcu; - int hwcache_align, object_size, objs_per_slab; - int sanity_checks, slab_size, store_user, trace; + unsigned int hwcache_align, object_size, objs_per_slab; + unsigned int sanity_checks, slab_size, store_user, trace; int order, poison, reclaim_account, red_zone; unsigned long partial, objects, slabs, objects_partial, objects_total; unsigned long alloc_fastpath, alloc_slowpath; diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c index 91aaf73b00df..ed162a6c57c5 100644 --- a/virt/kvm/arm/mmu.c +++ b/virt/kvm/arm/mmu.c @@ -1817,18 +1817,6 @@ static int kvm_unmap_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *dat return 0; } -int kvm_unmap_hva(struct kvm *kvm, unsigned long hva) -{ - unsigned long end = hva + PAGE_SIZE; - - if (!kvm->arch.pgd) - return 0; - - trace_kvm_unmap_hva(hva); - handle_hva_to_gpa(kvm, hva, end, &kvm_unmap_hva_handler, NULL); - return 0; -} - int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end) { @@ -1860,13 +1848,20 @@ static int kvm_set_spte_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *data void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte) { unsigned long end = hva + PAGE_SIZE; + kvm_pfn_t pfn = pte_pfn(pte); pte_t stage2_pte; if (!kvm->arch.pgd) return; trace_kvm_set_spte_hva(hva); - stage2_pte = pfn_pte(pte_pfn(pte), PAGE_S2); + + /* + * We've moved a page around, probably through CoW, so let's treat it + * just like a translation fault and clean the cache to the PoC. + */ + clean_dcache_guest_page(pfn, PAGE_SIZE); + stage2_pte = pfn_pte(pfn, PAGE_S2); handle_hva_to_gpa(kvm, hva, end, &kvm_set_spte_handler, &stage2_pte); } diff --git a/virt/kvm/arm/trace.h b/virt/kvm/arm/trace.h index e53b596f483b..57b3edebbb40 100644 --- a/virt/kvm/arm/trace.h +++ b/virt/kvm/arm/trace.h @@ -134,21 +134,6 @@ TRACE_EVENT(kvm_mmio_emulate, __entry->vcpu_pc, __entry->instr, __entry->cpsr) ); -TRACE_EVENT(kvm_unmap_hva, - TP_PROTO(unsigned long hva), - TP_ARGS(hva), - - TP_STRUCT__entry( - __field( unsigned long, hva ) - ), - - TP_fast_assign( - __entry->hva = hva; - ), - - TP_printk("mmu notifier unmap hva: %#08lx", __entry->hva) -); - TRACE_EVENT(kvm_unmap_hva_range, TP_PROTO(unsigned long start, unsigned long end), TP_ARGS(start, end), |