* Re: [PATCH v3 01/12] fpga: dfl: fme: support 512bit data width PR
From: Wu Hao @ 2019-07-24 14:22 UTC (permalink / raw)
To: Greg KH
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Ananda Ravuri, Xu Yilun
In-Reply-To: <20190724093532.GB29532@kroah.com>
On Wed, Jul 24, 2019 at 11:35:32AM +0200, Greg KH wrote:
> On Tue, Jul 23, 2019 at 12:51:24PM +0800, Wu Hao wrote:
> > In early partial reconfiguration private feature, it only
> > supports 32bit data width when writing data to hardware for
> > PR. 512bit data width PR support is an important optimization
> > for some specific solutions (e.g. XEON with FPGA integrated),
> > it allows driver to use AVX512 instruction to improve the
> > performance of partial reconfiguration. e.g. programming one
> > 100MB bitstream image via this 512bit data width PR hardware
> > only takes ~300ms, but 32bit revision requires ~3s per test
> > result.
> >
> > Please note now this optimization is only done on revision 2
> > of this PR private feature which is only used in integrated
> > solution that AVX512 is always supported. This revision 2
> > hardware doesn't support 32bit PR.
> >
> > Signed-off-by: Ananda Ravuri <ananda.ravuri@intel.com>
> > Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> > Acked-by: Alan Tull <atull@kernel.org>
> > Signed-off-by: Moritz Fischer <mdf@kernel.org>
> > ---
> > v2: remove DRV/MODULE_VERSION modifications
> > ---
> > drivers/fpga/dfl-fme-mgr.c | 110 ++++++++++++++++++++++++++++++++++++++-------
> > drivers/fpga/dfl-fme-pr.c | 43 +++++++++++-------
> > drivers/fpga/dfl-fme.h | 2 +
> > drivers/fpga/dfl.h | 5 +++
> > 4 files changed, 129 insertions(+), 31 deletions(-)
> >
> > diff --git a/drivers/fpga/dfl-fme-mgr.c b/drivers/fpga/dfl-fme-mgr.c
> > index b3f7eee..46e17f0 100644
> > --- a/drivers/fpga/dfl-fme-mgr.c
> > +++ b/drivers/fpga/dfl-fme-mgr.c
> > @@ -22,6 +22,7 @@
> > #include <linux/io-64-nonatomic-lo-hi.h>
> > #include <linux/fpga/fpga-mgr.h>
> >
> > +#include "dfl.h"
> > #include "dfl-fme-pr.h"
> >
> > /* FME Partial Reconfiguration Sub Feature Register Set */
> > @@ -30,6 +31,7 @@
> > #define FME_PR_STS 0x10
> > #define FME_PR_DATA 0x18
> > #define FME_PR_ERR 0x20
> > +#define FME_PR_512_DATA 0x40 /* Data Register for 512bit datawidth PR */
> > #define FME_PR_INTFC_ID_L 0xA8
> > #define FME_PR_INTFC_ID_H 0xB0
> >
> > @@ -67,8 +69,43 @@
> > #define PR_WAIT_TIMEOUT 8000000
> > #define PR_HOST_STATUS_IDLE 0
> >
> > +#if defined(CONFIG_X86) && defined(CONFIG_AS_AVX512)
> > +
> > +#include <linux/cpufeature.h>
> > +#include <asm/fpu/api.h>
> > +
> > +static inline int is_cpu_avx512_enabled(void)
> > +{
> > + return cpu_feature_enabled(X86_FEATURE_AVX512F);
> > +}
>
> That's a very arch specific function, why would a driver ever care about
> this?
Yes, this is only applied to a specific FPGA solution, which FPGA
has been integrated with XEON. Hardware indicates this using register
to software. As it's cpu integrated solution, so CPU always has this
AVX512 capability. The only check we do, is make sure this is not
manually disabled by kernel.
With this hardware, software could use AVX512 to accelerate the FPGA
partial reconfiguration as mentioned in the patch commit message.
It brings performance benifits to people who uses it. This is only one
optimization (512 vs 32bit data write to hw) for a specific hardware.
For other discrete solutions, e.g. FPGA PCIe Card, this is not used
at all as driver does check hardware register to avoid any AVX512 code.
>
> > +
> > +static inline void copy512(const void *src, void __iomem *dst)
> > +{
> > + kernel_fpu_begin();
> > +
> > + asm volatile("vmovdqu64 (%0), %%zmm0;"
> > + "vmovntdq %%zmm0, (%1);"
> > + :
> > + : "r"(src), "r"(dst)
> > + : "memory");
> > +
> > + kernel_fpu_end();
> > +}
>
> Shouldn't this be an arch-specific function somewhere? Burying this in
> a random driver is not ok. Please make this generic for all systems.
If more people need the same avx operation like this in kernel, then maybe
this can be moved to some arch-specific lib code somewhere as some common
functions to everybody, but i am not very sure if this is the case. Let me
think about this more.
>
> > +#else
> > +static inline int is_cpu_avx512_enabled(void)
> > +{
> > + return 0;
> > +}
> > +
> > +static inline void copy512(const void *src, void __iomem *dst)
> > +{
> > + WARN_ON_ONCE(1);
>
> Are you trying to get reports from syzbot? :)
Oh.. no.. I will remove it. :)
Thank you very much!
Hao
>
> Please fix this all up.
>
> greg k-h
^ permalink raw reply
* Re: [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing
From: Joel Fernandes @ 2019-07-24 14:10 UTC (permalink / raw)
To: Minchan Kim
Cc: linux-kernel, vdavydov.dev, Brendan Gregg, kernel-team,
Alexey Dobriyan, Al Viro, Andrew Morton, carmenjackson,
Christian Hansen, Colin Ian King, dancol, David Howells, fmayer,
joaodias, Jonathan Corbet, Kees Cook, Kirill Tkhai,
Konstantin Khlebnikov, linux-doc, linux-fsdevel, linux-mm,
Michal Hocko, Mike Rapoport, namhyung, sspatil, surenb,
Thomas Gleixner, timmurray, tkjos, Vlastimil Babka, wvw
In-Reply-To: <20190724042842.GA39273@google.com>
On Wed, Jul 24, 2019 at 01:28:42PM +0900, Minchan Kim wrote:
> On Tue, Jul 23, 2019 at 10:20:49AM -0400, Joel Fernandes wrote:
> > On Tue, Jul 23, 2019 at 03:13:58PM +0900, Minchan Kim wrote:
> > > Hi Joel,
> > >
> > > On Mon, Jul 22, 2019 at 05:32:04PM -0400, Joel Fernandes (Google) wrote:
> > > > The page_idle tracking feature currently requires looking up the pagemap
> > > > for a process followed by interacting with /sys/kernel/mm/page_idle.
> > > > This is quite cumbersome and can be error-prone too. If between
> > >
> > > cumbersome: That's the fair tradeoff between idle page tracking and
> > > clear_refs because idle page tracking could check even though the page
> > > is not mapped.
> >
> > It is fair tradeoff, but could be made simpler. The userspace code got
> > reduced by a good amount as well.
> >
> > > error-prone: What's the error?
> >
> > We see in normal Android usage, that some of the times pages appear not to be
> > idle even when they really are idle. Reproducing this is a bit unpredictable
> > and happens at random occasions. With this new interface, we are seeing this
> > happen much much lesser.
>
> I don't know how you did test. Maybe that could be contributed by
> swapping out or shared pages touched by other processes or some kernel
> behavior not to keep access bit of their operation.
It could be something along these lines is my thinking as well. So we know
its already has issues due to what you mentioned, I am not sure what else
needs investigation?
> Please investigate more what's the root cause. That would be important
> point to justify for the patch motivation.
The motivation is security. I am dropping the 'accuracy' factor I mentioned
from the patch description since it created a lot of confusion.
> > > > More over looking up PFN from pagemap in Android devices is not
> > > > supported by unprivileged process and requires SYS_ADMIN and gives 0 for
> > > > the PFN.
> > > >
> > > > This patch adds support to directly interact with page_idle tracking at
> > > > the PID level by introducing a /proc/<pid>/page_idle file. This
> > > > eliminates the need for userspace to calculate the mapping of the page.
> > > > It follows the exact same semantics as the global
> > > > /sys/kernel/mm/page_idle, however it is easier to use for some usecases
> > > > where looking up PFN is not needed and also does not require SYS_ADMIN.
> > >
> > > Ah, so the primary goal is to provide convinience interface and it would
> > > help accurary, too. IOW, accuracy is not your main goal?
> >
> > There are a couple of primary goals: Security, conveience and also solving
> > the accuracy/reliability problem we are seeing. Do keep in mind looking up
> > PFN has security implications. The PFN field in pagemap is zeroed if the user
> > does not have CAP_SYS_ADMIN.
>
> Myaybe you don't need PFN. is it?
With the traditional idle tracking, PFN is needed which has the mentioned
security issues. This patch solves it. And the interface is identical and
familiar to the existing page_idle bitmap interface.
> > > > In Android, we are using this for the heap profiler (heapprofd) which
> > > > profiles and pin points code paths which allocates and leaves memory
> > > > idle for long periods of time.
> > >
> > > So the goal is to detect idle pages with idle memory tracking?
> >
> > Isn't that what idle memory tracking does?
>
> To me, it's rather misleading. Please read motivation section in document.
> The feature would be good to detect workingset pages, not idle pages
> because workingset pages are never freed, swapped out and even we could
> count on newly allocated pages.
>
> Motivation
> ==========
>
> The idle page tracking feature allows to track which memory pages are being
> accessed by a workload and which are idle. This information can be useful for
> estimating the workload's working set size, which, in turn, can be taken into
> account when configuring the workload parameters, setting memory cgroup limits,
> or deciding where to place the workload within a compute cluster.
As we discussed by chat, we could collect additional metadata to check if
pages were swapped or freed ever since the time we marked them as idle.
However this can be incremental improvement.
> > > It couldn't work well because such idle pages could finally swap out and
> > > lose every flags of the page descriptor which is working mechanism of
> > > idle page tracking. It should have named "workingset page tracking",
> > > not "idle page tracking".
> >
> > The heap profiler that uses page-idle tracking is not to measure working set,
> > but to look for pages that are idle for long periods of time.
>
> It's important part. Please include it in the description so that people
> understands what's the usecase. As I said above, if it aims for finding
> idle pages durting the period, current idle page tracking feature is not
> good ironically.
Ok, I will mention.
> > Thanks for bringing up the swapping corner case.. Perhaps we can improve
> > the heap profiler to detect this by looking at bits 0-4 in pagemap. While it
>
> Yeb, that could work but it could add overhead again what you want to remove?
> Even, userspace should keep metadata to identify that page was already swapped
> in last period or newly swapped in new period.
Yep.
thanks,
- Joel
^ permalink raw reply
* Re: [PATCH v3 02/12] fpga: dfl: fme: add DFL_FPGA_FME_PORT_RELEASE/ASSIGN ioctl support.
From: Wu Hao @ 2019-07-24 13:47 UTC (permalink / raw)
To: Greg KH
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Zhang Yi Z, Xu Yilun
In-Reply-To: <20190724093357.GA29532@kroah.com>
On Wed, Jul 24, 2019 at 11:33:57AM +0200, Greg KH wrote:
> On Tue, Jul 23, 2019 at 12:51:25PM +0800, Wu Hao wrote:
> > +/**
> > + * dfl_fpga_cdev_config_port - configure a port feature dev
> > + * @cdev: parent container device.
> > + * @port_id: id of the port feature device.
> > + * @release: release port or assign port back.
> > + *
> > + * This function allows user to release port platform device or assign it back.
> > + * e.g. to safely turn one port from PF into VF for PCI device SRIOV support,
> > + * release port platform device is one necessary step.
> > + */
> > +int dfl_fpga_cdev_config_port(struct dfl_fpga_cdev *cdev, int port_id,
> > + bool release)
> > +{
> > + return release ? detach_port_dev(cdev, port_id) :
> > + attach_port_dev(cdev, port_id);
> > +}
> > +EXPORT_SYMBOL_GPL(dfl_fpga_cdev_config_port);
>
> That's a horrible api. Every time you see this call in code, you have
> to go and look up what "bool" means here. There's no reason for it.
>
> Just have 2 different functions, one that attaches a port, and one that
> detaches it. That way when you read the code that calls this function,
> you know what it does instantly without having to go look up some api
> function somewhere else.
>
> Write code for people to read first. And you are saving nothing here by
> trying to do two different things in the same exact function.
I see, you're right, it saves everybody's time on reading, very important.
I will fix this and keep it in mind. Thank you.
Hao
>
> thanks,
>
> greg k-h
^ permalink raw reply
* Re: [PATCH v3 03/12] fpga: dfl: pci: enable SRIOV support.
From: Wu Hao @ 2019-07-24 13:37 UTC (permalink / raw)
To: Greg KH
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Zhang Yi Z, Xu Yilun
In-Reply-To: <20190724093744.GC29532@kroah.com>
On Wed, Jul 24, 2019 at 11:37:44AM +0200, Greg KH wrote:
> On Tue, Jul 23, 2019 at 12:51:26PM +0800, Wu Hao wrote:
> > This patch enables the standard sriov support. It allows user to
> > enable SRIOV (and VFs), then user could pass through accelerators
> > (VFs) into virtual machine or use VFs directly in host.
> >
> > Signed-off-by: Zhang Yi Z <yi.z.zhang@intel.com>
> > Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> > Acked-by: Alan Tull <atull@kernel.org>
> > Acked-by: Moritz Fischer <mdf@kernel.org>
> > Signed-off-by: Moritz Fischer <mdf@kernel.org>
> > ---
> > v2: remove DRV/MODULE_VERSION modifications.
> > ---
> > drivers/fpga/dfl-pci.c | 39 +++++++++++++++++++++++++++++++++++++++
> > drivers/fpga/dfl.c | 41 +++++++++++++++++++++++++++++++++++++++++
> > drivers/fpga/dfl.h | 1 +
> > 3 files changed, 81 insertions(+)
> >
> > diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c
> > index 66b5720..0e65d81 100644
> > --- a/drivers/fpga/dfl-pci.c
> > +++ b/drivers/fpga/dfl-pci.c
> > @@ -223,8 +223,46 @@ int cci_pci_probe(struct pci_dev *pcidev, const struct pci_device_id *pcidevid)
> > return ret;
> > }
> >
> > +static int cci_pci_sriov_configure(struct pci_dev *pcidev, int num_vfs)
> > +{
> > + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
> > + struct dfl_fpga_cdev *cdev = drvdata->cdev;
> > + int ret = 0;
> > +
> > + mutex_lock(&cdev->lock);
> > +
> > + if (!num_vfs) {
> > + /*
> > + * disable SRIOV and then put released ports back to default
> > + * PF access mode.
> > + */
> > + pci_disable_sriov(pcidev);
> > +
> > + __dfl_fpga_cdev_config_port_vf(cdev, false);
> > +
> > + } else if (cdev->released_port_num == num_vfs) {
> > + /*
> > + * only enable SRIOV if cdev has matched released ports, put
> > + * released ports into VF access mode firstly.
> > + */
> > + __dfl_fpga_cdev_config_port_vf(cdev, true);
> > +
> > + ret = pci_enable_sriov(pcidev, num_vfs);
> > + if (ret)
> > + __dfl_fpga_cdev_config_port_vf(cdev, false);
> > + } else {
> > + ret = -EINVAL;
> > + }
> > +
> > + mutex_unlock(&cdev->lock);
> > + return ret;
> > +}
> > +
> > static void cci_pci_remove(struct pci_dev *pcidev)
> > {
> > + if (dev_is_pf(&pcidev->dev))
> > + cci_pci_sriov_configure(pcidev, 0);
> > +
> > cci_remove_feature_devs(pcidev);
> > pci_disable_pcie_error_reporting(pcidev);
> > }
> > @@ -234,6 +272,7 @@ static void cci_pci_remove(struct pci_dev *pcidev)
> > .id_table = cci_pcie_id_tbl,
> > .probe = cci_pci_probe,
> > .remove = cci_pci_remove,
> > + .sriov_configure = cci_pci_sriov_configure,
> > };
> >
> > module_pci_driver(cci_pci_driver);
> > diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c
> > index e04ed45..c3a8e1d 100644
> > --- a/drivers/fpga/dfl.c
> > +++ b/drivers/fpga/dfl.c
> > @@ -1112,6 +1112,47 @@ int dfl_fpga_cdev_config_port(struct dfl_fpga_cdev *cdev, int port_id,
> > }
> > EXPORT_SYMBOL_GPL(dfl_fpga_cdev_config_port);
> >
> > +static void config_port_vf(struct device *fme_dev, int port_id, bool is_vf)
> > +{
> > + void __iomem *base;
> > + u64 v;
> > +
> > + base = dfl_get_feature_ioaddr_by_id(fme_dev, FME_FEATURE_ID_HEADER);
> > +
> > + v = readq(base + FME_HDR_PORT_OFST(port_id));
> > +
> > + v &= ~FME_PORT_OFST_ACC_CTRL;
> > + v |= FIELD_PREP(FME_PORT_OFST_ACC_CTRL,
> > + is_vf ? FME_PORT_OFST_ACC_VF : FME_PORT_OFST_ACC_PF);
> > +
> > + writeq(v, base + FME_HDR_PORT_OFST(port_id));
> > +}
> > +
> > +/**
> > + * __dfl_fpga_cdev_config_port_vf - configure port to VF access mode
> > + *
> > + * @cdev: parent container device.
> > + * @if_vf: true for VF access mode, and false for PF access mode
> > + *
> > + * Return: 0 on success, negative error code otherwise.
> > + *
> > + * This function is needed in sriov configuration routine. It could be used to
> > + * configures the released ports access mode to VF or PF.
> > + * The caller needs to hold lock for protection.
> > + */
> > +void __dfl_fpga_cdev_config_port_vf(struct dfl_fpga_cdev *cdev, bool is_vf)
> > +{
> > + struct dfl_feature_platform_data *pdata;
> > +
> > + list_for_each_entry(pdata, &cdev->port_dev_list, node) {
> > + if (device_is_registered(&pdata->dev->dev))
> > + continue;
> > +
> > + config_port_vf(cdev->fme_dev, pdata->id, is_vf);
> > + }
> > +}
> > +EXPORT_SYMBOL_GPL(__dfl_fpga_cdev_config_port_vf);
>
> Why are you exporting a function with a leading __?
>
> You are expecting someone else, in who knows what code, to do locking
> correctly? If so, and the caller always has to have a local lock, then
> it's not a big deal, just drop the '__', otherwise if you have to have a
> specific lock for a specific device, then you have a really complex and
> probably broken api here :(
Yes, I just want to remind the user of this API, caller needs to hold the
lock to protect the list. I fully agree, it does make sense to make the
APIs easy to use. I will try to improve this, maybe move the lock inside
this function, then API user doesn't need to know the details of locking.
Thanks a lot for the comments, it really helps.
Hao
>
> thanks,
>
> greg k-h
^ permalink raw reply
* Re: [PATCH v3 04/12] fpga: dfl: afu: add AFU state related sysfs interfaces
From: Wu Hao @ 2019-07-24 13:29 UTC (permalink / raw)
To: Greg KH
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Ananda Ravuri, Xu Yilun
In-Reply-To: <20190724094110.GD29532@kroah.com>
On Wed, Jul 24, 2019 at 11:41:10AM +0200, Greg KH wrote:
> On Tue, Jul 23, 2019 at 12:51:27PM +0800, Wu Hao wrote:
> > This patch introduces more sysfs interfaces for Accelerated
> > Function Unit (AFU). These interfaces allow users to read
> > current AFU Power State (APx), read / clear AFU Power (APx)
> > events which are sticky to identify transient APx state,
> > and manage AFU's LTR (latency tolerance reporting).
> >
> > Signed-off-by: Ananda Ravuri <ananda.ravuri@intel.com>
> > Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> > Acked-by: Alan Tull <atull@kernel.org>
> > Signed-off-by: Moritz Fischer <mdf@kernel.org>
> > ---
> > v2: rebased, and remove DRV/MODULE_VERSION modifications
> > v3: update kernel version and date in sysfs doc
> > ---
> > Documentation/ABI/testing/sysfs-platform-dfl-port | 30 +++++
> > drivers/fpga/dfl-afu-main.c | 137 ++++++++++++++++++++++
> > drivers/fpga/dfl.h | 11 ++
> > 3 files changed, 178 insertions(+)
> >
> > diff --git a/Documentation/ABI/testing/sysfs-platform-dfl-port b/Documentation/ABI/testing/sysfs-platform-dfl-port
> > index 6a92dda..5961fb2 100644
> > --- a/Documentation/ABI/testing/sysfs-platform-dfl-port
> > +++ b/Documentation/ABI/testing/sysfs-platform-dfl-port
> > @@ -14,3 +14,33 @@ Description: Read-only. User can program different PR bitstreams to FPGA
> > Accelerator Function Unit (AFU) for different functions. It
> > returns uuid which could be used to identify which PR bitstream
> > is programmed in this AFU.
> > +
> > +What: /sys/bus/platform/devices/dfl-port.0/power_state
> > +Date: July 2019
> > +KernelVersion: 5.4
> > +Contact: Wu Hao <hao.wu@intel.com>
> > +Description: Read-only. It reports the APx (AFU Power) state, different APx
> > + means different throttling level. When reading this file, it
> > + returns "0" - Normal / "1" - AP1 / "2" - AP2 / "6" - AP6.
> > +
> > +What: /sys/bus/platform/devices/dfl-port.0/ap1_event
> > +Date: July 2019
> > +KernelVersion: 5.4
> > +Contact: Wu Hao <hao.wu@intel.com>
> > +Description: Read-write. Read or set 1 to clear AP1 (AFU Power State 1)
> > + event. It's used to indicate transient AP1 state.
>
> So reading the value changes the state of the system? That's almost
> always never a good idea.
>
> Force userspace to write the value to change something. Otherwise all
> libraries that use sysfs will be accidentally changing the state of your
> system without you ever knowing it.
Oh.. I think the description makes some misunderstanding here, will fix it
in the next version. This AP1/AP2 event will only be cleared by write 1 to
it, read will not change the state.
>
> > +
> > +What: /sys/bus/platform/devices/dfl-port.0/ap2_event
> > +Date: July 2019
> > +KernelVersion: 5.4
> > +Contact: Wu Hao <hao.wu@intel.com>
> > +Description: Read-write. Read or set 1 to clear AP2 (AFU Power State 2)
> > + event. It's used to indicate transient AP2 state.
> > +
> > +What: /sys/bus/platform/devices/dfl-port.0/ltr
> > +Date: July 2019
> > +KernelVersion: 5.4
> > +Contact: Wu Hao <hao.wu@intel.com>
> > +Description: Read-write. Read and set AFU latency tolerance reporting value.
> > + Set ltr to 1 if the AFU can tolerate latency >= 40us or set it
> > + to 0 if it is latency sensitive.
> > diff --git a/drivers/fpga/dfl-afu-main.c b/drivers/fpga/dfl-afu-main.c
> > index 68b4d08..cb3f73e 100644
> > --- a/drivers/fpga/dfl-afu-main.c
> > +++ b/drivers/fpga/dfl-afu-main.c
> > @@ -141,8 +141,145 @@ static int port_get_id(struct platform_device *pdev)
> > }
> > static DEVICE_ATTR_RO(id);
> >
> > +static ssize_t
> > +ltr_show(struct device *dev, struct device_attribute *attr, char *buf)
> > +{
> > + struct dfl_feature_platform_data *pdata = dev_get_platdata(dev);
> > + void __iomem *base;
> > + u64 v;
> > +
> > + base = dfl_get_feature_ioaddr_by_id(dev, PORT_FEATURE_ID_HEADER);
> > +
> > + mutex_lock(&pdata->lock);
> > + v = readq(base + PORT_HDR_CTRL);
> > + mutex_unlock(&pdata->lock);
>
> Why do you need a lock to call readq()? What are you protecting here?
If this code is running on 32bit machine, readq will be replaced with 2
readl operation. If that is the case, should we protect the code against
it?
>
>
> > +
> > + return sprintf(buf, "%x\n", (u8)FIELD_GET(PORT_CTRL_LATENCY, v));
> > +}
> > +
> > +static ssize_t
> > +ltr_store(struct device *dev, struct device_attribute *attr,
> > + const char *buf, size_t count)
> > +{
> > + struct dfl_feature_platform_data *pdata = dev_get_platdata(dev);
> > + void __iomem *base;
> > + u8 ltr;
> > + u64 v;
> > +
> > + if (kstrtou8(buf, 0, <r) || ltr > 1)
> > + return -EINVAL;
>
> Are you doing anything with this value? If not, how about just using
> the sysfs boolean read function and if it is 1, then do the clearing?
>
> Same for all other show/store functions in here.
Sure, will fix this in the next version.
Thanks a lot for the comments.
Hao
>
> thanks,
>
> greg k-h
^ permalink raw reply
* Re: [PATCH v3 09/12] fpga: dfl: afu: add STP (SignalTap) support
From: Wu Hao @ 2019-07-24 13:03 UTC (permalink / raw)
To: Greg KH
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Xu Yilun
In-Reply-To: <20190724101109.GE29532@kroah.com>
On Wed, Jul 24, 2019 at 12:11:09PM +0200, Greg KH wrote:
> On Tue, Jul 23, 2019 at 12:51:32PM +0800, Wu Hao wrote:
> > STP (SignalTap) is one of the private features under the port for
> > debugging. This patch adds private feature driver support for it
> > to allow userspace applications to mmap related mmio region and
> > provide STP service.
> >
> > Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> > Signed-off-by: Wu Hao <hao.wu@intel.com>
> > Acked-by: Moritz Fischer <mdf@kernel.org>
> > Acked-by: Alan Tull <atull@kernel.org>
> > Signed-off-by: Moritz Fischer <mdf@kernel.org>
> > ---
> > drivers/fpga/dfl-afu-main.c | 34 ++++++++++++++++++++++++++++++++++
> > 1 file changed, 34 insertions(+)
> >
> > diff --git a/drivers/fpga/dfl-afu-main.c b/drivers/fpga/dfl-afu-main.c
> > index 15dd4cb..395f96e 100644
> > --- a/drivers/fpga/dfl-afu-main.c
> > +++ b/drivers/fpga/dfl-afu-main.c
> > @@ -514,6 +514,36 @@ static void port_afu_uinit(struct platform_device *pdev,
> > .uinit = port_afu_uinit,
> > };
> >
> > +static int port_stp_init(struct platform_device *pdev,
> > + struct dfl_feature *feature)
> > +{
> > + struct resource *res = &pdev->resource[feature->resource_index];
> > +
> > + dev_dbg(&pdev->dev, "PORT STP Init.\n");
>
> ftrace is your friend, no need to do a lot of "look I am here!"
> messages.
Hi Greg,
Thanks for the code review!
Sure, let me remove them.
>
> > +
> > + return afu_mmio_region_add(dev_get_platdata(&pdev->dev),
> > + DFL_PORT_REGION_INDEX_STP,
> > + resource_size(res), res->start,
> > + DFL_PORT_REGION_MMAP | DFL_PORT_REGION_READ |
> > + DFL_PORT_REGION_WRITE);
> > +}
> > +
> > +static void port_stp_uinit(struct platform_device *pdev,
> > + struct dfl_feature *feature)
> > +{
> > + dev_dbg(&pdev->dev, "PORT STP UInit.\n");
>
> Same here.
>
> Why have this function at all if it does not do anything?
Let me remove them in the next version. actually uinit callback is
always required in current code, i will add one more patch to change
it, and remove all uinit functions who do nothing, it does save code.
Thanks for the comments.
Hao
>
>
> thanks,
>
> greg k-h
^ permalink raw reply
* Re: [PATCH v3 09/12] fpga: dfl: afu: add STP (SignalTap) support
From: Greg KH @ 2019-07-24 10:11 UTC (permalink / raw)
To: Wu Hao; +Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Xu Yilun
In-Reply-To: <1563857495-26692-10-git-send-email-hao.wu@intel.com>
On Tue, Jul 23, 2019 at 12:51:32PM +0800, Wu Hao wrote:
> STP (SignalTap) is one of the private features under the port for
> debugging. This patch adds private feature driver support for it
> to allow userspace applications to mmap related mmio region and
> provide STP service.
>
> Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> Signed-off-by: Wu Hao <hao.wu@intel.com>
> Acked-by: Moritz Fischer <mdf@kernel.org>
> Acked-by: Alan Tull <atull@kernel.org>
> Signed-off-by: Moritz Fischer <mdf@kernel.org>
> ---
> drivers/fpga/dfl-afu-main.c | 34 ++++++++++++++++++++++++++++++++++
> 1 file changed, 34 insertions(+)
>
> diff --git a/drivers/fpga/dfl-afu-main.c b/drivers/fpga/dfl-afu-main.c
> index 15dd4cb..395f96e 100644
> --- a/drivers/fpga/dfl-afu-main.c
> +++ b/drivers/fpga/dfl-afu-main.c
> @@ -514,6 +514,36 @@ static void port_afu_uinit(struct platform_device *pdev,
> .uinit = port_afu_uinit,
> };
>
> +static int port_stp_init(struct platform_device *pdev,
> + struct dfl_feature *feature)
> +{
> + struct resource *res = &pdev->resource[feature->resource_index];
> +
> + dev_dbg(&pdev->dev, "PORT STP Init.\n");
ftrace is your friend, no need to do a lot of "look I am here!"
messages.
> +
> + return afu_mmio_region_add(dev_get_platdata(&pdev->dev),
> + DFL_PORT_REGION_INDEX_STP,
> + resource_size(res), res->start,
> + DFL_PORT_REGION_MMAP | DFL_PORT_REGION_READ |
> + DFL_PORT_REGION_WRITE);
> +}
> +
> +static void port_stp_uinit(struct platform_device *pdev,
> + struct dfl_feature *feature)
> +{
> + dev_dbg(&pdev->dev, "PORT STP UInit.\n");
Same here.
Why have this function at all if it does not do anything?
thanks,
greg k-h
^ permalink raw reply
* doc: mds: nitpicking and typo fix
From: Pavel Machek @ 2019-07-24 10:01 UTC (permalink / raw)
To: tglx, mingo, bp, hpa, x86, corbet, linux-kernel, linux-doc
[-- Attachment #1: Type: text/plain, Size: 1308 bytes --]
Consistently end sentences, fix typo.
Signed-off-by: Pavel Machek <pavel@denx.de>
commit 310cb17613f46db97cebbd9dc11187961e4b1c69
Author: Pavel <pavel@ucw.cz>
Date: Mon May 20 10:46:35 2019 +0200
doc: typo fix, consistency in mds.
diff --git a/Documentation/x86/mds.rst b/Documentation/x86/mds.rst
index 5d4330b..9983b50 100644
--- a/Documentation/x86/mds.rst
+++ b/Documentation/x86/mds.rst
@@ -54,13 +54,13 @@ needed for exploiting MDS requires:
- to control the load to trigger a fault or assist
- to have a disclosure gadget which exposes the speculatively accessed
- data for consumption through a side channel.
+ data for consumption through a side channel
- to control the pointer through which the disclosure gadget exposes the
data
The existence of such a construct in the kernel cannot be excluded with
-100% certainty, but the complexity involved makes it extremly unlikely.
+100% certainty, but the complexity involved makes it extremely unlikely.
There is one exception, which is untrusted BPF. The functionality of
untrusted BPF is limited, but it needs to be thoroughly investigated
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]
^ permalink raw reply related
* Re: [PATCH v3 04/12] fpga: dfl: afu: add AFU state related sysfs interfaces
From: Greg KH @ 2019-07-24 9:41 UTC (permalink / raw)
To: Wu Hao
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Ananda Ravuri, Xu Yilun
In-Reply-To: <1563857495-26692-5-git-send-email-hao.wu@intel.com>
On Tue, Jul 23, 2019 at 12:51:27PM +0800, Wu Hao wrote:
> This patch introduces more sysfs interfaces for Accelerated
> Function Unit (AFU). These interfaces allow users to read
> current AFU Power State (APx), read / clear AFU Power (APx)
> events which are sticky to identify transient APx state,
> and manage AFU's LTR (latency tolerance reporting).
>
> Signed-off-by: Ananda Ravuri <ananda.ravuri@intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> Signed-off-by: Wu Hao <hao.wu@intel.com>
> Acked-by: Alan Tull <atull@kernel.org>
> Signed-off-by: Moritz Fischer <mdf@kernel.org>
> ---
> v2: rebased, and remove DRV/MODULE_VERSION modifications
> v3: update kernel version and date in sysfs doc
> ---
> Documentation/ABI/testing/sysfs-platform-dfl-port | 30 +++++
> drivers/fpga/dfl-afu-main.c | 137 ++++++++++++++++++++++
> drivers/fpga/dfl.h | 11 ++
> 3 files changed, 178 insertions(+)
>
> diff --git a/Documentation/ABI/testing/sysfs-platform-dfl-port b/Documentation/ABI/testing/sysfs-platform-dfl-port
> index 6a92dda..5961fb2 100644
> --- a/Documentation/ABI/testing/sysfs-platform-dfl-port
> +++ b/Documentation/ABI/testing/sysfs-platform-dfl-port
> @@ -14,3 +14,33 @@ Description: Read-only. User can program different PR bitstreams to FPGA
> Accelerator Function Unit (AFU) for different functions. It
> returns uuid which could be used to identify which PR bitstream
> is programmed in this AFU.
> +
> +What: /sys/bus/platform/devices/dfl-port.0/power_state
> +Date: July 2019
> +KernelVersion: 5.4
> +Contact: Wu Hao <hao.wu@intel.com>
> +Description: Read-only. It reports the APx (AFU Power) state, different APx
> + means different throttling level. When reading this file, it
> + returns "0" - Normal / "1" - AP1 / "2" - AP2 / "6" - AP6.
> +
> +What: /sys/bus/platform/devices/dfl-port.0/ap1_event
> +Date: July 2019
> +KernelVersion: 5.4
> +Contact: Wu Hao <hao.wu@intel.com>
> +Description: Read-write. Read or set 1 to clear AP1 (AFU Power State 1)
> + event. It's used to indicate transient AP1 state.
So reading the value changes the state of the system? That's almost
always never a good idea.
Force userspace to write the value to change something. Otherwise all
libraries that use sysfs will be accidentally changing the state of your
system without you ever knowing it.
> +
> +What: /sys/bus/platform/devices/dfl-port.0/ap2_event
> +Date: July 2019
> +KernelVersion: 5.4
> +Contact: Wu Hao <hao.wu@intel.com>
> +Description: Read-write. Read or set 1 to clear AP2 (AFU Power State 2)
> + event. It's used to indicate transient AP2 state.
> +
> +What: /sys/bus/platform/devices/dfl-port.0/ltr
> +Date: July 2019
> +KernelVersion: 5.4
> +Contact: Wu Hao <hao.wu@intel.com>
> +Description: Read-write. Read and set AFU latency tolerance reporting value.
> + Set ltr to 1 if the AFU can tolerate latency >= 40us or set it
> + to 0 if it is latency sensitive.
> diff --git a/drivers/fpga/dfl-afu-main.c b/drivers/fpga/dfl-afu-main.c
> index 68b4d08..cb3f73e 100644
> --- a/drivers/fpga/dfl-afu-main.c
> +++ b/drivers/fpga/dfl-afu-main.c
> @@ -141,8 +141,145 @@ static int port_get_id(struct platform_device *pdev)
> }
> static DEVICE_ATTR_RO(id);
>
> +static ssize_t
> +ltr_show(struct device *dev, struct device_attribute *attr, char *buf)
> +{
> + struct dfl_feature_platform_data *pdata = dev_get_platdata(dev);
> + void __iomem *base;
> + u64 v;
> +
> + base = dfl_get_feature_ioaddr_by_id(dev, PORT_FEATURE_ID_HEADER);
> +
> + mutex_lock(&pdata->lock);
> + v = readq(base + PORT_HDR_CTRL);
> + mutex_unlock(&pdata->lock);
Why do you need a lock to call readq()? What are you protecting here?
> +
> + return sprintf(buf, "%x\n", (u8)FIELD_GET(PORT_CTRL_LATENCY, v));
> +}
> +
> +static ssize_t
> +ltr_store(struct device *dev, struct device_attribute *attr,
> + const char *buf, size_t count)
> +{
> + struct dfl_feature_platform_data *pdata = dev_get_platdata(dev);
> + void __iomem *base;
> + u8 ltr;
> + u64 v;
> +
> + if (kstrtou8(buf, 0, <r) || ltr > 1)
> + return -EINVAL;
Are you doing anything with this value? If not, how about just using
the sysfs boolean read function and if it is 1, then do the clearing?
Same for all other show/store functions in here.
thanks,
greg k-h
^ permalink raw reply
* Re: [PATCH v3 03/12] fpga: dfl: pci: enable SRIOV support.
From: Greg KH @ 2019-07-24 9:37 UTC (permalink / raw)
To: Wu Hao
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Zhang Yi Z, Xu Yilun
In-Reply-To: <1563857495-26692-4-git-send-email-hao.wu@intel.com>
On Tue, Jul 23, 2019 at 12:51:26PM +0800, Wu Hao wrote:
> This patch enables the standard sriov support. It allows user to
> enable SRIOV (and VFs), then user could pass through accelerators
> (VFs) into virtual machine or use VFs directly in host.
>
> Signed-off-by: Zhang Yi Z <yi.z.zhang@intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> Signed-off-by: Wu Hao <hao.wu@intel.com>
> Acked-by: Alan Tull <atull@kernel.org>
> Acked-by: Moritz Fischer <mdf@kernel.org>
> Signed-off-by: Moritz Fischer <mdf@kernel.org>
> ---
> v2: remove DRV/MODULE_VERSION modifications.
> ---
> drivers/fpga/dfl-pci.c | 39 +++++++++++++++++++++++++++++++++++++++
> drivers/fpga/dfl.c | 41 +++++++++++++++++++++++++++++++++++++++++
> drivers/fpga/dfl.h | 1 +
> 3 files changed, 81 insertions(+)
>
> diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c
> index 66b5720..0e65d81 100644
> --- a/drivers/fpga/dfl-pci.c
> +++ b/drivers/fpga/dfl-pci.c
> @@ -223,8 +223,46 @@ int cci_pci_probe(struct pci_dev *pcidev, const struct pci_device_id *pcidevid)
> return ret;
> }
>
> +static int cci_pci_sriov_configure(struct pci_dev *pcidev, int num_vfs)
> +{
> + struct cci_drvdata *drvdata = pci_get_drvdata(pcidev);
> + struct dfl_fpga_cdev *cdev = drvdata->cdev;
> + int ret = 0;
> +
> + mutex_lock(&cdev->lock);
> +
> + if (!num_vfs) {
> + /*
> + * disable SRIOV and then put released ports back to default
> + * PF access mode.
> + */
> + pci_disable_sriov(pcidev);
> +
> + __dfl_fpga_cdev_config_port_vf(cdev, false);
> +
> + } else if (cdev->released_port_num == num_vfs) {
> + /*
> + * only enable SRIOV if cdev has matched released ports, put
> + * released ports into VF access mode firstly.
> + */
> + __dfl_fpga_cdev_config_port_vf(cdev, true);
> +
> + ret = pci_enable_sriov(pcidev, num_vfs);
> + if (ret)
> + __dfl_fpga_cdev_config_port_vf(cdev, false);
> + } else {
> + ret = -EINVAL;
> + }
> +
> + mutex_unlock(&cdev->lock);
> + return ret;
> +}
> +
> static void cci_pci_remove(struct pci_dev *pcidev)
> {
> + if (dev_is_pf(&pcidev->dev))
> + cci_pci_sriov_configure(pcidev, 0);
> +
> cci_remove_feature_devs(pcidev);
> pci_disable_pcie_error_reporting(pcidev);
> }
> @@ -234,6 +272,7 @@ static void cci_pci_remove(struct pci_dev *pcidev)
> .id_table = cci_pcie_id_tbl,
> .probe = cci_pci_probe,
> .remove = cci_pci_remove,
> + .sriov_configure = cci_pci_sriov_configure,
> };
>
> module_pci_driver(cci_pci_driver);
> diff --git a/drivers/fpga/dfl.c b/drivers/fpga/dfl.c
> index e04ed45..c3a8e1d 100644
> --- a/drivers/fpga/dfl.c
> +++ b/drivers/fpga/dfl.c
> @@ -1112,6 +1112,47 @@ int dfl_fpga_cdev_config_port(struct dfl_fpga_cdev *cdev, int port_id,
> }
> EXPORT_SYMBOL_GPL(dfl_fpga_cdev_config_port);
>
> +static void config_port_vf(struct device *fme_dev, int port_id, bool is_vf)
> +{
> + void __iomem *base;
> + u64 v;
> +
> + base = dfl_get_feature_ioaddr_by_id(fme_dev, FME_FEATURE_ID_HEADER);
> +
> + v = readq(base + FME_HDR_PORT_OFST(port_id));
> +
> + v &= ~FME_PORT_OFST_ACC_CTRL;
> + v |= FIELD_PREP(FME_PORT_OFST_ACC_CTRL,
> + is_vf ? FME_PORT_OFST_ACC_VF : FME_PORT_OFST_ACC_PF);
> +
> + writeq(v, base + FME_HDR_PORT_OFST(port_id));
> +}
> +
> +/**
> + * __dfl_fpga_cdev_config_port_vf - configure port to VF access mode
> + *
> + * @cdev: parent container device.
> + * @if_vf: true for VF access mode, and false for PF access mode
> + *
> + * Return: 0 on success, negative error code otherwise.
> + *
> + * This function is needed in sriov configuration routine. It could be used to
> + * configures the released ports access mode to VF or PF.
> + * The caller needs to hold lock for protection.
> + */
> +void __dfl_fpga_cdev_config_port_vf(struct dfl_fpga_cdev *cdev, bool is_vf)
> +{
> + struct dfl_feature_platform_data *pdata;
> +
> + list_for_each_entry(pdata, &cdev->port_dev_list, node) {
> + if (device_is_registered(&pdata->dev->dev))
> + continue;
> +
> + config_port_vf(cdev->fme_dev, pdata->id, is_vf);
> + }
> +}
> +EXPORT_SYMBOL_GPL(__dfl_fpga_cdev_config_port_vf);
Why are you exporting a function with a leading __?
You are expecting someone else, in who knows what code, to do locking
correctly? If so, and the caller always has to have a local lock, then
it's not a big deal, just drop the '__', otherwise if you have to have a
specific lock for a specific device, then you have a really complex and
probably broken api here :(
thanks,
greg k-h
^ permalink raw reply
* Re: [PATCH v3 01/12] fpga: dfl: fme: support 512bit data width PR
From: Greg KH @ 2019-07-24 9:35 UTC (permalink / raw)
To: Wu Hao
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Ananda Ravuri, Xu Yilun
In-Reply-To: <1563857495-26692-2-git-send-email-hao.wu@intel.com>
On Tue, Jul 23, 2019 at 12:51:24PM +0800, Wu Hao wrote:
> In early partial reconfiguration private feature, it only
> supports 32bit data width when writing data to hardware for
> PR. 512bit data width PR support is an important optimization
> for some specific solutions (e.g. XEON with FPGA integrated),
> it allows driver to use AVX512 instruction to improve the
> performance of partial reconfiguration. e.g. programming one
> 100MB bitstream image via this 512bit data width PR hardware
> only takes ~300ms, but 32bit revision requires ~3s per test
> result.
>
> Please note now this optimization is only done on revision 2
> of this PR private feature which is only used in integrated
> solution that AVX512 is always supported. This revision 2
> hardware doesn't support 32bit PR.
>
> Signed-off-by: Ananda Ravuri <ananda.ravuri@intel.com>
> Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> Signed-off-by: Wu Hao <hao.wu@intel.com>
> Acked-by: Alan Tull <atull@kernel.org>
> Signed-off-by: Moritz Fischer <mdf@kernel.org>
> ---
> v2: remove DRV/MODULE_VERSION modifications
> ---
> drivers/fpga/dfl-fme-mgr.c | 110 ++++++++++++++++++++++++++++++++++++++-------
> drivers/fpga/dfl-fme-pr.c | 43 +++++++++++-------
> drivers/fpga/dfl-fme.h | 2 +
> drivers/fpga/dfl.h | 5 +++
> 4 files changed, 129 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/fpga/dfl-fme-mgr.c b/drivers/fpga/dfl-fme-mgr.c
> index b3f7eee..46e17f0 100644
> --- a/drivers/fpga/dfl-fme-mgr.c
> +++ b/drivers/fpga/dfl-fme-mgr.c
> @@ -22,6 +22,7 @@
> #include <linux/io-64-nonatomic-lo-hi.h>
> #include <linux/fpga/fpga-mgr.h>
>
> +#include "dfl.h"
> #include "dfl-fme-pr.h"
>
> /* FME Partial Reconfiguration Sub Feature Register Set */
> @@ -30,6 +31,7 @@
> #define FME_PR_STS 0x10
> #define FME_PR_DATA 0x18
> #define FME_PR_ERR 0x20
> +#define FME_PR_512_DATA 0x40 /* Data Register for 512bit datawidth PR */
> #define FME_PR_INTFC_ID_L 0xA8
> #define FME_PR_INTFC_ID_H 0xB0
>
> @@ -67,8 +69,43 @@
> #define PR_WAIT_TIMEOUT 8000000
> #define PR_HOST_STATUS_IDLE 0
>
> +#if defined(CONFIG_X86) && defined(CONFIG_AS_AVX512)
> +
> +#include <linux/cpufeature.h>
> +#include <asm/fpu/api.h>
> +
> +static inline int is_cpu_avx512_enabled(void)
> +{
> + return cpu_feature_enabled(X86_FEATURE_AVX512F);
> +}
That's a very arch specific function, why would a driver ever care about
this?
> +
> +static inline void copy512(const void *src, void __iomem *dst)
> +{
> + kernel_fpu_begin();
> +
> + asm volatile("vmovdqu64 (%0), %%zmm0;"
> + "vmovntdq %%zmm0, (%1);"
> + :
> + : "r"(src), "r"(dst)
> + : "memory");
> +
> + kernel_fpu_end();
> +}
Shouldn't this be an arch-specific function somewhere? Burying this in
a random driver is not ok. Please make this generic for all systems.
> +#else
> +static inline int is_cpu_avx512_enabled(void)
> +{
> + return 0;
> +}
> +
> +static inline void copy512(const void *src, void __iomem *dst)
> +{
> + WARN_ON_ONCE(1);
Are you trying to get reports from syzbot? :)
Please fix this all up.
greg k-h
^ permalink raw reply
* Re: [PATCH v3 02/12] fpga: dfl: fme: add DFL_FPGA_FME_PORT_RELEASE/ASSIGN ioctl support.
From: Greg KH @ 2019-07-24 9:33 UTC (permalink / raw)
To: Wu Hao
Cc: mdf, linux-fpga, linux-kernel, linux-api, linux-doc, atull,
Zhang Yi Z, Xu Yilun
In-Reply-To: <1563857495-26692-3-git-send-email-hao.wu@intel.com>
On Tue, Jul 23, 2019 at 12:51:25PM +0800, Wu Hao wrote:
> +/**
> + * dfl_fpga_cdev_config_port - configure a port feature dev
> + * @cdev: parent container device.
> + * @port_id: id of the port feature device.
> + * @release: release port or assign port back.
> + *
> + * This function allows user to release port platform device or assign it back.
> + * e.g. to safely turn one port from PF into VF for PCI device SRIOV support,
> + * release port platform device is one necessary step.
> + */
> +int dfl_fpga_cdev_config_port(struct dfl_fpga_cdev *cdev, int port_id,
> + bool release)
> +{
> + return release ? detach_port_dev(cdev, port_id) :
> + attach_port_dev(cdev, port_id);
> +}
> +EXPORT_SYMBOL_GPL(dfl_fpga_cdev_config_port);
That's a horrible api. Every time you see this call in code, you have
to go and look up what "bool" means here. There's no reason for it.
Just have 2 different functions, one that attaches a port, and one that
detaches it. That way when you read the code that calls this function,
you know what it does instantly without having to go look up some api
function somewhere else.
Write code for people to read first. And you are saving nothing here by
trying to do two different things in the same exact function.
thanks,
greg k-h
^ permalink raw reply
* Re: [PATCH] Documentation: move Documentation/virtual to Documentation/virt
From: Paolo Bonzini @ 2019-07-24 8:51 UTC (permalink / raw)
To: Christoph Hellwig, corbet
Cc: rkrcmar, jdike, richard, anton.ivanov, linux-doc, kvm, linux-um,
linux-kernel
In-Reply-To: <20190724072449.19599-1-hch@lst.de>
On 24/07/19 09:24, Christoph Hellwig wrote:
> Renaming docs seems to be en vogue at the moment, so fix on of the
> grossly misnamed directories. We usually never use "virtual" as
> a shortcut for virtualization in the kernel, but always virt,
> as seen in the virt/ top-level directory. Fix up the documentation
> to match that.
>
> Fixes: ed16648eb5b8 ("Move kvm, uml, and lguest subdirectories under a common "virtual" directory, I.E:")
> Signed-off-by: Christoph Hellwig <hch@lst.de>
Queued, thanks. I can't count how many times I said "I really should
rename that directory".
Paolo
> ---
> Documentation/admin-guide/kernel-parameters.txt | 2 +-
> Documentation/{virtual => virt}/index.rst | 0
> .../{virtual => virt}/kvm/amd-memory-encryption.rst | 0
> Documentation/{virtual => virt}/kvm/api.txt | 2 +-
> Documentation/{virtual => virt}/kvm/arm/hyp-abi.txt | 0
> Documentation/{virtual => virt}/kvm/arm/psci.txt | 0
> Documentation/{virtual => virt}/kvm/cpuid.rst | 0
> Documentation/{virtual => virt}/kvm/devices/README | 0
> .../{virtual => virt}/kvm/devices/arm-vgic-its.txt | 0
> Documentation/{virtual => virt}/kvm/devices/arm-vgic-v3.txt | 0
> Documentation/{virtual => virt}/kvm/devices/arm-vgic.txt | 0
> Documentation/{virtual => virt}/kvm/devices/mpic.txt | 0
> Documentation/{virtual => virt}/kvm/devices/s390_flic.txt | 0
> Documentation/{virtual => virt}/kvm/devices/vcpu.txt | 0
> Documentation/{virtual => virt}/kvm/devices/vfio.txt | 0
> Documentation/{virtual => virt}/kvm/devices/vm.txt | 0
> Documentation/{virtual => virt}/kvm/devices/xics.txt | 0
> Documentation/{virtual => virt}/kvm/devices/xive.txt | 0
> Documentation/{virtual => virt}/kvm/halt-polling.txt | 0
> Documentation/{virtual => virt}/kvm/hypercalls.txt | 4 ++--
> Documentation/{virtual => virt}/kvm/index.rst | 0
> Documentation/{virtual => virt}/kvm/locking.txt | 0
> Documentation/{virtual => virt}/kvm/mmu.txt | 2 +-
> Documentation/{virtual => virt}/kvm/msr.txt | 0
> Documentation/{virtual => virt}/kvm/nested-vmx.txt | 0
> Documentation/{virtual => virt}/kvm/ppc-pv.txt | 0
> Documentation/{virtual => virt}/kvm/review-checklist.txt | 2 +-
> Documentation/{virtual => virt}/kvm/s390-diag.txt | 0
> Documentation/{virtual => virt}/kvm/timekeeping.txt | 0
> Documentation/{virtual => virt}/kvm/vcpu-requests.rst | 0
> Documentation/{virtual => virt}/paravirt_ops.rst | 0
> Documentation/{virtual => virt}/uml/UserModeLinux-HOWTO.txt | 0
> MAINTAINERS | 6 +++---
> arch/powerpc/include/uapi/asm/kvm_para.h | 2 +-
> arch/x86/kvm/mmu.c | 2 +-
> include/uapi/linux/kvm.h | 4 ++--
> tools/include/uapi/linux/kvm.h | 4 ++--
> virt/kvm/arm/arm.c | 2 +-
> virt/kvm/arm/vgic/vgic-mmio-v3.c | 2 +-
> virt/kvm/arm/vgic/vgic.h | 4 ++--
> 40 files changed, 19 insertions(+), 19 deletions(-)
> rename Documentation/{virtual => virt}/index.rst (100%)
> rename Documentation/{virtual => virt}/kvm/amd-memory-encryption.rst (100%)
> rename Documentation/{virtual => virt}/kvm/api.txt (99%)
> rename Documentation/{virtual => virt}/kvm/arm/hyp-abi.txt (100%)
> rename Documentation/{virtual => virt}/kvm/arm/psci.txt (100%)
> rename Documentation/{virtual => virt}/kvm/cpuid.rst (100%)
> rename Documentation/{virtual => virt}/kvm/devices/README (100%)
> rename Documentation/{virtual => virt}/kvm/devices/arm-vgic-its.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/arm-vgic-v3.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/arm-vgic.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/mpic.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/s390_flic.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/vcpu.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/vfio.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/vm.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/xics.txt (100%)
> rename Documentation/{virtual => virt}/kvm/devices/xive.txt (100%)
> rename Documentation/{virtual => virt}/kvm/halt-polling.txt (100%)
> rename Documentation/{virtual => virt}/kvm/hypercalls.txt (97%)
> rename Documentation/{virtual => virt}/kvm/index.rst (100%)
> rename Documentation/{virtual => virt}/kvm/locking.txt (100%)
> rename Documentation/{virtual => virt}/kvm/mmu.txt (99%)
> rename Documentation/{virtual => virt}/kvm/msr.txt (100%)
> rename Documentation/{virtual => virt}/kvm/nested-vmx.txt (100%)
> rename Documentation/{virtual => virt}/kvm/ppc-pv.txt (100%)
> rename Documentation/{virtual => virt}/kvm/review-checklist.txt (95%)
> rename Documentation/{virtual => virt}/kvm/s390-diag.txt (100%)
> rename Documentation/{virtual => virt}/kvm/timekeeping.txt (100%)
> rename Documentation/{virtual => virt}/kvm/vcpu-requests.rst (100%)
> rename Documentation/{virtual => virt}/paravirt_ops.rst (100%)
> rename Documentation/{virtual => virt}/uml/UserModeLinux-HOWTO.txt (100%)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index 46b826fcb5ad..7ccd158b3894 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -2545,7 +2545,7 @@
> mem_encrypt=on: Activate SME
> mem_encrypt=off: Do not activate SME
>
> - Refer to Documentation/virtual/kvm/amd-memory-encryption.rst
> + Refer to Documentation/virt/kvm/amd-memory-encryption.rst
> for details on when memory encryption can be activated.
>
> mem_sleep_default= [SUSPEND] Default system suspend mode:
> diff --git a/Documentation/virtual/index.rst b/Documentation/virt/index.rst
> similarity index 100%
> rename from Documentation/virtual/index.rst
> rename to Documentation/virt/index.rst
> diff --git a/Documentation/virtual/kvm/amd-memory-encryption.rst b/Documentation/virt/kvm/amd-memory-encryption.rst
> similarity index 100%
> rename from Documentation/virtual/kvm/amd-memory-encryption.rst
> rename to Documentation/virt/kvm/amd-memory-encryption.rst
> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virt/kvm/api.txt
> similarity index 99%
> rename from Documentation/virtual/kvm/api.txt
> rename to Documentation/virt/kvm/api.txt
> index e54a3f51ddc5..2d067767b617 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virt/kvm/api.txt
> @@ -3781,7 +3781,7 @@ encrypted VMs.
>
> Currently, this ioctl is used for issuing Secure Encrypted Virtualization
> (SEV) commands on AMD Processors. The SEV commands are defined in
> -Documentation/virtual/kvm/amd-memory-encryption.rst.
> +Documentation/virt/kvm/amd-memory-encryption.rst.
>
> 4.111 KVM_MEMORY_ENCRYPT_REG_REGION
>
> diff --git a/Documentation/virtual/kvm/arm/hyp-abi.txt b/Documentation/virt/kvm/arm/hyp-abi.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/arm/hyp-abi.txt
> rename to Documentation/virt/kvm/arm/hyp-abi.txt
> diff --git a/Documentation/virtual/kvm/arm/psci.txt b/Documentation/virt/kvm/arm/psci.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/arm/psci.txt
> rename to Documentation/virt/kvm/arm/psci.txt
> diff --git a/Documentation/virtual/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst
> similarity index 100%
> rename from Documentation/virtual/kvm/cpuid.rst
> rename to Documentation/virt/kvm/cpuid.rst
> diff --git a/Documentation/virtual/kvm/devices/README b/Documentation/virt/kvm/devices/README
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/README
> rename to Documentation/virt/kvm/devices/README
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virt/kvm/devices/arm-vgic-its.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/arm-vgic-its.txt
> rename to Documentation/virt/kvm/devices/arm-vgic-its.txt
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic-v3.txt b/Documentation/virt/kvm/devices/arm-vgic-v3.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> rename to Documentation/virt/kvm/devices/arm-vgic-v3.txt
> diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virt/kvm/devices/arm-vgic.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/arm-vgic.txt
> rename to Documentation/virt/kvm/devices/arm-vgic.txt
> diff --git a/Documentation/virtual/kvm/devices/mpic.txt b/Documentation/virt/kvm/devices/mpic.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/mpic.txt
> rename to Documentation/virt/kvm/devices/mpic.txt
> diff --git a/Documentation/virtual/kvm/devices/s390_flic.txt b/Documentation/virt/kvm/devices/s390_flic.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/s390_flic.txt
> rename to Documentation/virt/kvm/devices/s390_flic.txt
> diff --git a/Documentation/virtual/kvm/devices/vcpu.txt b/Documentation/virt/kvm/devices/vcpu.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/vcpu.txt
> rename to Documentation/virt/kvm/devices/vcpu.txt
> diff --git a/Documentation/virtual/kvm/devices/vfio.txt b/Documentation/virt/kvm/devices/vfio.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/vfio.txt
> rename to Documentation/virt/kvm/devices/vfio.txt
> diff --git a/Documentation/virtual/kvm/devices/vm.txt b/Documentation/virt/kvm/devices/vm.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/vm.txt
> rename to Documentation/virt/kvm/devices/vm.txt
> diff --git a/Documentation/virtual/kvm/devices/xics.txt b/Documentation/virt/kvm/devices/xics.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/xics.txt
> rename to Documentation/virt/kvm/devices/xics.txt
> diff --git a/Documentation/virtual/kvm/devices/xive.txt b/Documentation/virt/kvm/devices/xive.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/devices/xive.txt
> rename to Documentation/virt/kvm/devices/xive.txt
> diff --git a/Documentation/virtual/kvm/halt-polling.txt b/Documentation/virt/kvm/halt-polling.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/halt-polling.txt
> rename to Documentation/virt/kvm/halt-polling.txt
> diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virt/kvm/hypercalls.txt
> similarity index 97%
> rename from Documentation/virtual/kvm/hypercalls.txt
> rename to Documentation/virt/kvm/hypercalls.txt
> index da210651f714..5f6d291bd004 100644
> --- a/Documentation/virtual/kvm/hypercalls.txt
> +++ b/Documentation/virt/kvm/hypercalls.txt
> @@ -18,7 +18,7 @@ S390:
> number in R1.
>
> For further information on the S390 diagnose call as supported by KVM,
> - refer to Documentation/virtual/kvm/s390-diag.txt.
> + refer to Documentation/virt/kvm/s390-diag.txt.
>
> PowerPC:
> It uses R3-R10 and hypercall number in R11. R4-R11 are used as output registers.
> @@ -26,7 +26,7 @@ S390:
>
> KVM hypercalls uses 4 byte opcode, that are patched with 'hypercall-instructions'
> property inside the device tree's /hypervisor node.
> - For more information refer to Documentation/virtual/kvm/ppc-pv.txt
> + For more information refer to Documentation/virt/kvm/ppc-pv.txt
>
> MIPS:
> KVM hypercalls use the HYPCALL instruction with code 0 and the hypercall
> diff --git a/Documentation/virtual/kvm/index.rst b/Documentation/virt/kvm/index.rst
> similarity index 100%
> rename from Documentation/virtual/kvm/index.rst
> rename to Documentation/virt/kvm/index.rst
> diff --git a/Documentation/virtual/kvm/locking.txt b/Documentation/virt/kvm/locking.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/locking.txt
> rename to Documentation/virt/kvm/locking.txt
> diff --git a/Documentation/virtual/kvm/mmu.txt b/Documentation/virt/kvm/mmu.txt
> similarity index 99%
> rename from Documentation/virtual/kvm/mmu.txt
> rename to Documentation/virt/kvm/mmu.txt
> index 2efe0efc516e..1b9880dfba0a 100644
> --- a/Documentation/virtual/kvm/mmu.txt
> +++ b/Documentation/virt/kvm/mmu.txt
> @@ -298,7 +298,7 @@ Handling a page fault is performed as follows:
> vcpu->arch.mmio_gfn, and call the emulator
> - If both P bit and R/W bit of error code are set, this could possibly
> be handled as a "fast page fault" (fixed without taking the MMU lock). See
> - the description in Documentation/virtual/kvm/locking.txt.
> + the description in Documentation/virt/kvm/locking.txt.
> - if needed, walk the guest page tables to determine the guest translation
> (gva->gpa or ngpa->gpa)
> - if permissions are insufficient, reflect the fault back to the guest
> diff --git a/Documentation/virtual/kvm/msr.txt b/Documentation/virt/kvm/msr.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/msr.txt
> rename to Documentation/virt/kvm/msr.txt
> diff --git a/Documentation/virtual/kvm/nested-vmx.txt b/Documentation/virt/kvm/nested-vmx.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/nested-vmx.txt
> rename to Documentation/virt/kvm/nested-vmx.txt
> diff --git a/Documentation/virtual/kvm/ppc-pv.txt b/Documentation/virt/kvm/ppc-pv.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/ppc-pv.txt
> rename to Documentation/virt/kvm/ppc-pv.txt
> diff --git a/Documentation/virtual/kvm/review-checklist.txt b/Documentation/virt/kvm/review-checklist.txt
> similarity index 95%
> rename from Documentation/virtual/kvm/review-checklist.txt
> rename to Documentation/virt/kvm/review-checklist.txt
> index a83b27635fdd..499af499e296 100644
> --- a/Documentation/virtual/kvm/review-checklist.txt
> +++ b/Documentation/virt/kvm/review-checklist.txt
> @@ -7,7 +7,7 @@ Review checklist for kvm patches
> 2. Patches should be against kvm.git master branch.
>
> 3. If the patch introduces or modifies a new userspace API:
> - - the API must be documented in Documentation/virtual/kvm/api.txt
> + - the API must be documented in Documentation/virt/kvm/api.txt
> - the API must be discoverable using KVM_CHECK_EXTENSION
>
> 4. New state must include support for save/restore.
> diff --git a/Documentation/virtual/kvm/s390-diag.txt b/Documentation/virt/kvm/s390-diag.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/s390-diag.txt
> rename to Documentation/virt/kvm/s390-diag.txt
> diff --git a/Documentation/virtual/kvm/timekeeping.txt b/Documentation/virt/kvm/timekeeping.txt
> similarity index 100%
> rename from Documentation/virtual/kvm/timekeeping.txt
> rename to Documentation/virt/kvm/timekeeping.txt
> diff --git a/Documentation/virtual/kvm/vcpu-requests.rst b/Documentation/virt/kvm/vcpu-requests.rst
> similarity index 100%
> rename from Documentation/virtual/kvm/vcpu-requests.rst
> rename to Documentation/virt/kvm/vcpu-requests.rst
> diff --git a/Documentation/virtual/paravirt_ops.rst b/Documentation/virt/paravirt_ops.rst
> similarity index 100%
> rename from Documentation/virtual/paravirt_ops.rst
> rename to Documentation/virt/paravirt_ops.rst
> diff --git a/Documentation/virtual/uml/UserModeLinux-HOWTO.txt b/Documentation/virt/uml/UserModeLinux-HOWTO.txt
> similarity index 100%
> rename from Documentation/virtual/uml/UserModeLinux-HOWTO.txt
> rename to Documentation/virt/uml/UserModeLinux-HOWTO.txt
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 783569e3c4b4..5e1f9ee8f86f 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -8808,7 +8808,7 @@ L: kvm@vger.kernel.org
> W: http://www.linux-kvm.org
> T: git git://git.kernel.org/pub/scm/virt/kvm/kvm.git
> S: Supported
> -F: Documentation/virtual/kvm/
> +F: Documentation/virt/kvm/
> F: include/trace/events/kvm.h
> F: include/uapi/asm-generic/kvm*
> F: include/uapi/linux/kvm*
> @@ -12137,7 +12137,7 @@ M: Thomas Hellstrom <thellstrom@vmware.com>
> M: "VMware, Inc." <pv-drivers@vmware.com>
> L: virtualization@lists.linux-foundation.org
> S: Supported
> -F: Documentation/virtual/paravirt_ops.txt
> +F: Documentation/virt/paravirt_ops.txt
> F: arch/*/kernel/paravirt*
> F: arch/*/include/asm/paravirt*.h
> F: include/linux/hypervisor.h
> @@ -16854,7 +16854,7 @@ W: http://user-mode-linux.sourceforge.net
> Q: https://patchwork.ozlabs.org/project/linux-um/list/
> T: git git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git
> S: Maintained
> -F: Documentation/virtual/uml/
> +F: Documentation/virt/uml/
> F: arch/um/
> F: arch/x86/um/
> F: fs/hostfs/
> diff --git a/arch/powerpc/include/uapi/asm/kvm_para.h b/arch/powerpc/include/uapi/asm/kvm_para.h
> index 01555c6ae0f5..be48c2215fa2 100644
> --- a/arch/powerpc/include/uapi/asm/kvm_para.h
> +++ b/arch/powerpc/include/uapi/asm/kvm_para.h
> @@ -31,7 +31,7 @@
> * Struct fields are always 32 or 64 bit aligned, depending on them being 32
> * or 64 bit wide respectively.
> *
> - * See Documentation/virtual/kvm/ppc-pv.txt
> + * See Documentation/virt/kvm/ppc-pv.txt
> */
> struct kvm_vcpu_arch_shared {
> __u64 scratch1;
> diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
> index 8f72526e2f68..24843cf49579 100644
> --- a/arch/x86/kvm/mmu.c
> +++ b/arch/x86/kvm/mmu.c
> @@ -3466,7 +3466,7 @@ static bool fast_page_fault(struct kvm_vcpu *vcpu, gva_t gva, int level,
> /*
> * Currently, fast page fault only works for direct mapping
> * since the gfn is not stable for indirect shadow page. See
> - * Documentation/virtual/kvm/locking.txt to get more detail.
> + * Documentation/virt/kvm/locking.txt to get more detail.
> */
> fault_handled = fast_pf_fix_direct_spte(vcpu, sp,
> iterator.sptep, spte,
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index a7c19540ce21..5e3f12d5359e 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -116,7 +116,7 @@ struct kvm_irq_level {
> * ACPI gsi notion of irq.
> * For IA-64 (APIC model) IOAPIC0: irq 0-23; IOAPIC1: irq 24-47..
> * For X86 (standard AT mode) PIC0/1: irq 0-15. IOAPIC0: 0-23..
> - * For ARM: See Documentation/virtual/kvm/api.txt
> + * For ARM: See Documentation/virt/kvm/api.txt
> */
> union {
> __u32 irq;
> @@ -1086,7 +1086,7 @@ struct kvm_xen_hvm_config {
> *
> * KVM_IRQFD_FLAG_RESAMPLE indicates resamplefd is valid and specifies
> * the irqfd to operate in resampling mode for level triggered interrupt
> - * emulation. See Documentation/virtual/kvm/api.txt.
> + * emulation. See Documentation/virt/kvm/api.txt.
> */
> #define KVM_IRQFD_FLAG_RESAMPLE (1 << 1)
>
> diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
> index c2152f3dd02d..e7c67be7c15f 100644
> --- a/tools/include/uapi/linux/kvm.h
> +++ b/tools/include/uapi/linux/kvm.h
> @@ -116,7 +116,7 @@ struct kvm_irq_level {
> * ACPI gsi notion of irq.
> * For IA-64 (APIC model) IOAPIC0: irq 0-23; IOAPIC1: irq 24-47..
> * For X86 (standard AT mode) PIC0/1: irq 0-15. IOAPIC0: 0-23..
> - * For ARM: See Documentation/virtual/kvm/api.txt
> + * For ARM: See Documentation/virt/kvm/api.txt
> */
> union {
> __u32 irq;
> @@ -1085,7 +1085,7 @@ struct kvm_xen_hvm_config {
> *
> * KVM_IRQFD_FLAG_RESAMPLE indicates resamplefd is valid and specifies
> * the irqfd to operate in resampling mode for level triggered interrupt
> - * emulation. See Documentation/virtual/kvm/api.txt.
> + * emulation. See Documentation/virt/kvm/api.txt.
> */
> #define KVM_IRQFD_FLAG_RESAMPLE (1 << 1)
>
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index f645c0fbf7ec..acc43242a310 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -727,7 +727,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> * Ensure we set mode to IN_GUEST_MODE after we disable
> * interrupts and before the final VCPU requests check.
> * See the comment in kvm_vcpu_exiting_guest_mode() and
> - * Documentation/virtual/kvm/vcpu-requests.rst
> + * Documentation/virt/kvm/vcpu-requests.rst
> */
> smp_store_mb(vcpu->mode, IN_GUEST_MODE);
>
> diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> index 936962abc38d..c45e2d7e942f 100644
> --- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
> +++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
> @@ -250,7 +250,7 @@ static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
> * pending state of interrupt is latched in pending_latch variable.
> * Userspace will save and restore pending state and line_level
> * separately.
> - * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
> + * Refer to Documentation/virt/kvm/devices/arm-vgic-v3.txt
> * for handling of ISPENDR and ICPENDR.
> */
> for (i = 0; i < len * 8; i++) {
> diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
> index 57205beaa981..3b7525deec80 100644
> --- a/virt/kvm/arm/vgic/vgic.h
> +++ b/virt/kvm/arm/vgic/vgic.h
> @@ -42,7 +42,7 @@
> VGIC_AFFINITY_LEVEL(val, 3))
>
> /*
> - * As per Documentation/virtual/kvm/devices/arm-vgic-v3.txt,
> + * As per Documentation/virt/kvm/devices/arm-vgic-v3.txt,
> * below macros are defined for CPUREG encoding.
> */
> #define KVM_REG_ARM_VGIC_SYSREG_OP0_MASK 0x000000000000c000
> @@ -63,7 +63,7 @@
> KVM_REG_ARM_VGIC_SYSREG_OP2_MASK)
>
> /*
> - * As per Documentation/virtual/kvm/devices/arm-vgic-its.txt,
> + * As per Documentation/virt/kvm/devices/arm-vgic-its.txt,
> * below macros are defined for ITS table entry encoding.
> */
> #define KVM_ITS_CTE_VALID_SHIFT 63
>
^ permalink raw reply
* Re: [PATCH v9 04/18] kunit: test: add kunit_stream a std::stream like logger
From: Petr Mladek @ 2019-07-24 7:31 UTC (permalink / raw)
To: Stephen Boyd
Cc: Brendan Higgins, Jeff Dike, Kevin Hilman, Logan Gunthorpe,
Michael Ellerman, Daniel Vetter, Amir Goldstein, Frank Rowand,
Steven Rostedt, Kees Cook, David Rientjes, kunit-dev,
Kieran Bingham, Peter Zijlstra, Randy Dunlap, Joel Stanley,
Luis Chamberlain, Rob Herring, shuah, wfg, Greg KH, Julia Lawall,
linux-nvdimm, dri-devel, linux-um, Sasha Levin, Theodore Ts'o,
Richard Weinberger, Dan Carpenter, Knut Omang, Josh Poimboeuf,
Masahiro Yamada, Timothy Bird, devicetree,
open list:DOCUMENTATION, linux-fsdevel, linux-kbuild,
Linux Kernel Mailing List, open list:KERNEL SELFTEST FRAMEWORK
In-Reply-To: <20190722235411.06C1320840@mail.kernel.org>
On Mon 2019-07-22 16:54:10, Stephen Boyd wrote:
> Quoting Brendan Higgins (2019-07-22 15:30:49)
> > On Mon, Jul 22, 2019 at 1:03 PM Stephen Boyd <sboyd@kernel.org> wrote:
> > >
> > >
> > > What's the calling context of the assertions and expectations? I still
> > > don't like the fact that string stream needs to allocate buffers and
> > > throw them into a list somewhere because the calling context matters
> > > there.
> >
> > The calling context is the same as before, which is anywhere.
>
> Ok. That's concerning then.
>
> >
> > > I'd prefer we just wrote directly to the console/log via printk
> > > instead. That way things are simple because we use the existing
> > > buffering path of printk, but maybe there's some benefit to the string
> > > stream that I don't see? Right now it looks like it builds a string and
> > > then dumps it to printk so I'm sort of lost what the benefit is over
> > > just writing directly with printk.
> >
> > It's just buffering it so the whole string gets printed uninterrupted.
> > If we were to print out piecemeal to printk, couldn't we have another
> > call to printk come in causing it to garble the KUnit message we are
> > in the middle of printing?
>
> Yes, printing piecemeal by calling printk many times could lead to
> interleaving of messages if something else comes in such as an interrupt
> printing something. Printk has some support to hold "records" but I'm
> not sure how that would work here because KERN_CONT talks about only
> being used early on in boot code. I haven't looked at printk in detail
> though so maybe I'm all wrong and KERN_CONT just works?
KERN_CONT does not guarantee that the message will get printed
together. The pieces get interleaved with messages printed in
parallel.
Note that KERN_CONT was originally really meant to be used only during
boot. It was later used more widely and ended in the best effort category.
There were several attempts to make it more reliable. But it was
always either too complicated or error prone or both.
You need to use your own buffering if you rely want perfect output.
The question is if it is really worth the complexity. Also note that
any buffering reduces the chance that the messages will reach
the console.
BTW: There is a work in progress on a lockless printk ring buffer.
It will make printk() more secure regarding deadlocks. But it might
make transparent handling of continuous lines even more tricky.
I guess that local buffering, before calling printk(), will be
even more important then. Well, it might really force us to create
an API for it.
> Can printk be called once with whatever is in the struct? Otherwise if
> this is about making printk into a structured log then maybe printk
> isn't the proper solution anyway. Maybe a dev interface should be used
> instead that can handle starting and stopping tests (via ioctl) in
> addition to reading test results, records, etc. with read() and a
> clearing of the records. Then the seqfile API works naturally. All of
> this is a bit premature, but it looks like you're going down the path of
> making something akin to ftrace that stores binary formatted
> assertion/expectation records in a lockless ring buffer that then
> formats those records when the user asks for them.
IMHO, ftrace postpones the text formatting primary because it does not
not want to slow down the traced code more than necessary. It is yet
another layer and there should be some strong reason for it.
> I can imagine someone wanting to write unit tests that check conditions
> from a simulated hardirq context via irq works (a driver mock
> framework?), so this doesn't seem far off.
Note that stroring the messages into the printk log is basically safe in any
context. It uses temporary per-CPU buffers for recursive messages and
in NMI. The only problem is panic() when some CPU gets stuck with the
lock taken. This will get solved by the lockless ringbuffer. Also
the temporary buffers will not be necessary any longer.
Much bigger problems are with consoles. There are many of them. It
means a lot of code and more locks involved, including scheduler
locks. Note that console lock is a semaphore.
Best Regards,
Petr
^ permalink raw reply
* [PATCH] Documentation: move Documentation/virtual to Documentation/virt
From: Christoph Hellwig @ 2019-07-24 7:24 UTC (permalink / raw)
To: corbet
Cc: pbonzini, rkrcmar, jdike, richard, anton.ivanov, linux-doc, kvm,
linux-um, linux-kernel
Renaming docs seems to be en vogue at the moment, so fix on of the
grossly misnamed directories. We usually never use "virtual" as
a shortcut for virtualization in the kernel, but always virt,
as seen in the virt/ top-level directory. Fix up the documentation
to match that.
Fixes: ed16648eb5b8 ("Move kvm, uml, and lguest subdirectories under a common "virtual" directory, I.E:")
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
Documentation/admin-guide/kernel-parameters.txt | 2 +-
Documentation/{virtual => virt}/index.rst | 0
.../{virtual => virt}/kvm/amd-memory-encryption.rst | 0
Documentation/{virtual => virt}/kvm/api.txt | 2 +-
Documentation/{virtual => virt}/kvm/arm/hyp-abi.txt | 0
Documentation/{virtual => virt}/kvm/arm/psci.txt | 0
Documentation/{virtual => virt}/kvm/cpuid.rst | 0
Documentation/{virtual => virt}/kvm/devices/README | 0
.../{virtual => virt}/kvm/devices/arm-vgic-its.txt | 0
Documentation/{virtual => virt}/kvm/devices/arm-vgic-v3.txt | 0
Documentation/{virtual => virt}/kvm/devices/arm-vgic.txt | 0
Documentation/{virtual => virt}/kvm/devices/mpic.txt | 0
Documentation/{virtual => virt}/kvm/devices/s390_flic.txt | 0
Documentation/{virtual => virt}/kvm/devices/vcpu.txt | 0
Documentation/{virtual => virt}/kvm/devices/vfio.txt | 0
Documentation/{virtual => virt}/kvm/devices/vm.txt | 0
Documentation/{virtual => virt}/kvm/devices/xics.txt | 0
Documentation/{virtual => virt}/kvm/devices/xive.txt | 0
Documentation/{virtual => virt}/kvm/halt-polling.txt | 0
Documentation/{virtual => virt}/kvm/hypercalls.txt | 4 ++--
Documentation/{virtual => virt}/kvm/index.rst | 0
Documentation/{virtual => virt}/kvm/locking.txt | 0
Documentation/{virtual => virt}/kvm/mmu.txt | 2 +-
Documentation/{virtual => virt}/kvm/msr.txt | 0
Documentation/{virtual => virt}/kvm/nested-vmx.txt | 0
Documentation/{virtual => virt}/kvm/ppc-pv.txt | 0
Documentation/{virtual => virt}/kvm/review-checklist.txt | 2 +-
Documentation/{virtual => virt}/kvm/s390-diag.txt | 0
Documentation/{virtual => virt}/kvm/timekeeping.txt | 0
Documentation/{virtual => virt}/kvm/vcpu-requests.rst | 0
Documentation/{virtual => virt}/paravirt_ops.rst | 0
Documentation/{virtual => virt}/uml/UserModeLinux-HOWTO.txt | 0
MAINTAINERS | 6 +++---
arch/powerpc/include/uapi/asm/kvm_para.h | 2 +-
arch/x86/kvm/mmu.c | 2 +-
include/uapi/linux/kvm.h | 4 ++--
tools/include/uapi/linux/kvm.h | 4 ++--
virt/kvm/arm/arm.c | 2 +-
virt/kvm/arm/vgic/vgic-mmio-v3.c | 2 +-
virt/kvm/arm/vgic/vgic.h | 4 ++--
40 files changed, 19 insertions(+), 19 deletions(-)
rename Documentation/{virtual => virt}/index.rst (100%)
rename Documentation/{virtual => virt}/kvm/amd-memory-encryption.rst (100%)
rename Documentation/{virtual => virt}/kvm/api.txt (99%)
rename Documentation/{virtual => virt}/kvm/arm/hyp-abi.txt (100%)
rename Documentation/{virtual => virt}/kvm/arm/psci.txt (100%)
rename Documentation/{virtual => virt}/kvm/cpuid.rst (100%)
rename Documentation/{virtual => virt}/kvm/devices/README (100%)
rename Documentation/{virtual => virt}/kvm/devices/arm-vgic-its.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/arm-vgic-v3.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/arm-vgic.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/mpic.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/s390_flic.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/vcpu.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/vfio.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/vm.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/xics.txt (100%)
rename Documentation/{virtual => virt}/kvm/devices/xive.txt (100%)
rename Documentation/{virtual => virt}/kvm/halt-polling.txt (100%)
rename Documentation/{virtual => virt}/kvm/hypercalls.txt (97%)
rename Documentation/{virtual => virt}/kvm/index.rst (100%)
rename Documentation/{virtual => virt}/kvm/locking.txt (100%)
rename Documentation/{virtual => virt}/kvm/mmu.txt (99%)
rename Documentation/{virtual => virt}/kvm/msr.txt (100%)
rename Documentation/{virtual => virt}/kvm/nested-vmx.txt (100%)
rename Documentation/{virtual => virt}/kvm/ppc-pv.txt (100%)
rename Documentation/{virtual => virt}/kvm/review-checklist.txt (95%)
rename Documentation/{virtual => virt}/kvm/s390-diag.txt (100%)
rename Documentation/{virtual => virt}/kvm/timekeeping.txt (100%)
rename Documentation/{virtual => virt}/kvm/vcpu-requests.rst (100%)
rename Documentation/{virtual => virt}/paravirt_ops.rst (100%)
rename Documentation/{virtual => virt}/uml/UserModeLinux-HOWTO.txt (100%)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 46b826fcb5ad..7ccd158b3894 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2545,7 +2545,7 @@
mem_encrypt=on: Activate SME
mem_encrypt=off: Do not activate SME
- Refer to Documentation/virtual/kvm/amd-memory-encryption.rst
+ Refer to Documentation/virt/kvm/amd-memory-encryption.rst
for details on when memory encryption can be activated.
mem_sleep_default= [SUSPEND] Default system suspend mode:
diff --git a/Documentation/virtual/index.rst b/Documentation/virt/index.rst
similarity index 100%
rename from Documentation/virtual/index.rst
rename to Documentation/virt/index.rst
diff --git a/Documentation/virtual/kvm/amd-memory-encryption.rst b/Documentation/virt/kvm/amd-memory-encryption.rst
similarity index 100%
rename from Documentation/virtual/kvm/amd-memory-encryption.rst
rename to Documentation/virt/kvm/amd-memory-encryption.rst
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virt/kvm/api.txt
similarity index 99%
rename from Documentation/virtual/kvm/api.txt
rename to Documentation/virt/kvm/api.txt
index e54a3f51ddc5..2d067767b617 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virt/kvm/api.txt
@@ -3781,7 +3781,7 @@ encrypted VMs.
Currently, this ioctl is used for issuing Secure Encrypted Virtualization
(SEV) commands on AMD Processors. The SEV commands are defined in
-Documentation/virtual/kvm/amd-memory-encryption.rst.
+Documentation/virt/kvm/amd-memory-encryption.rst.
4.111 KVM_MEMORY_ENCRYPT_REG_REGION
diff --git a/Documentation/virtual/kvm/arm/hyp-abi.txt b/Documentation/virt/kvm/arm/hyp-abi.txt
similarity index 100%
rename from Documentation/virtual/kvm/arm/hyp-abi.txt
rename to Documentation/virt/kvm/arm/hyp-abi.txt
diff --git a/Documentation/virtual/kvm/arm/psci.txt b/Documentation/virt/kvm/arm/psci.txt
similarity index 100%
rename from Documentation/virtual/kvm/arm/psci.txt
rename to Documentation/virt/kvm/arm/psci.txt
diff --git a/Documentation/virtual/kvm/cpuid.rst b/Documentation/virt/kvm/cpuid.rst
similarity index 100%
rename from Documentation/virtual/kvm/cpuid.rst
rename to Documentation/virt/kvm/cpuid.rst
diff --git a/Documentation/virtual/kvm/devices/README b/Documentation/virt/kvm/devices/README
similarity index 100%
rename from Documentation/virtual/kvm/devices/README
rename to Documentation/virt/kvm/devices/README
diff --git a/Documentation/virtual/kvm/devices/arm-vgic-its.txt b/Documentation/virt/kvm/devices/arm-vgic-its.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/arm-vgic-its.txt
rename to Documentation/virt/kvm/devices/arm-vgic-its.txt
diff --git a/Documentation/virtual/kvm/devices/arm-vgic-v3.txt b/Documentation/virt/kvm/devices/arm-vgic-v3.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/arm-vgic-v3.txt
rename to Documentation/virt/kvm/devices/arm-vgic-v3.txt
diff --git a/Documentation/virtual/kvm/devices/arm-vgic.txt b/Documentation/virt/kvm/devices/arm-vgic.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/arm-vgic.txt
rename to Documentation/virt/kvm/devices/arm-vgic.txt
diff --git a/Documentation/virtual/kvm/devices/mpic.txt b/Documentation/virt/kvm/devices/mpic.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/mpic.txt
rename to Documentation/virt/kvm/devices/mpic.txt
diff --git a/Documentation/virtual/kvm/devices/s390_flic.txt b/Documentation/virt/kvm/devices/s390_flic.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/s390_flic.txt
rename to Documentation/virt/kvm/devices/s390_flic.txt
diff --git a/Documentation/virtual/kvm/devices/vcpu.txt b/Documentation/virt/kvm/devices/vcpu.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/vcpu.txt
rename to Documentation/virt/kvm/devices/vcpu.txt
diff --git a/Documentation/virtual/kvm/devices/vfio.txt b/Documentation/virt/kvm/devices/vfio.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/vfio.txt
rename to Documentation/virt/kvm/devices/vfio.txt
diff --git a/Documentation/virtual/kvm/devices/vm.txt b/Documentation/virt/kvm/devices/vm.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/vm.txt
rename to Documentation/virt/kvm/devices/vm.txt
diff --git a/Documentation/virtual/kvm/devices/xics.txt b/Documentation/virt/kvm/devices/xics.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/xics.txt
rename to Documentation/virt/kvm/devices/xics.txt
diff --git a/Documentation/virtual/kvm/devices/xive.txt b/Documentation/virt/kvm/devices/xive.txt
similarity index 100%
rename from Documentation/virtual/kvm/devices/xive.txt
rename to Documentation/virt/kvm/devices/xive.txt
diff --git a/Documentation/virtual/kvm/halt-polling.txt b/Documentation/virt/kvm/halt-polling.txt
similarity index 100%
rename from Documentation/virtual/kvm/halt-polling.txt
rename to Documentation/virt/kvm/halt-polling.txt
diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virt/kvm/hypercalls.txt
similarity index 97%
rename from Documentation/virtual/kvm/hypercalls.txt
rename to Documentation/virt/kvm/hypercalls.txt
index da210651f714..5f6d291bd004 100644
--- a/Documentation/virtual/kvm/hypercalls.txt
+++ b/Documentation/virt/kvm/hypercalls.txt
@@ -18,7 +18,7 @@ S390:
number in R1.
For further information on the S390 diagnose call as supported by KVM,
- refer to Documentation/virtual/kvm/s390-diag.txt.
+ refer to Documentation/virt/kvm/s390-diag.txt.
PowerPC:
It uses R3-R10 and hypercall number in R11. R4-R11 are used as output registers.
@@ -26,7 +26,7 @@ S390:
KVM hypercalls uses 4 byte opcode, that are patched with 'hypercall-instructions'
property inside the device tree's /hypervisor node.
- For more information refer to Documentation/virtual/kvm/ppc-pv.txt
+ For more information refer to Documentation/virt/kvm/ppc-pv.txt
MIPS:
KVM hypercalls use the HYPCALL instruction with code 0 and the hypercall
diff --git a/Documentation/virtual/kvm/index.rst b/Documentation/virt/kvm/index.rst
similarity index 100%
rename from Documentation/virtual/kvm/index.rst
rename to Documentation/virt/kvm/index.rst
diff --git a/Documentation/virtual/kvm/locking.txt b/Documentation/virt/kvm/locking.txt
similarity index 100%
rename from Documentation/virtual/kvm/locking.txt
rename to Documentation/virt/kvm/locking.txt
diff --git a/Documentation/virtual/kvm/mmu.txt b/Documentation/virt/kvm/mmu.txt
similarity index 99%
rename from Documentation/virtual/kvm/mmu.txt
rename to Documentation/virt/kvm/mmu.txt
index 2efe0efc516e..1b9880dfba0a 100644
--- a/Documentation/virtual/kvm/mmu.txt
+++ b/Documentation/virt/kvm/mmu.txt
@@ -298,7 +298,7 @@ Handling a page fault is performed as follows:
vcpu->arch.mmio_gfn, and call the emulator
- If both P bit and R/W bit of error code are set, this could possibly
be handled as a "fast page fault" (fixed without taking the MMU lock). See
- the description in Documentation/virtual/kvm/locking.txt.
+ the description in Documentation/virt/kvm/locking.txt.
- if needed, walk the guest page tables to determine the guest translation
(gva->gpa or ngpa->gpa)
- if permissions are insufficient, reflect the fault back to the guest
diff --git a/Documentation/virtual/kvm/msr.txt b/Documentation/virt/kvm/msr.txt
similarity index 100%
rename from Documentation/virtual/kvm/msr.txt
rename to Documentation/virt/kvm/msr.txt
diff --git a/Documentation/virtual/kvm/nested-vmx.txt b/Documentation/virt/kvm/nested-vmx.txt
similarity index 100%
rename from Documentation/virtual/kvm/nested-vmx.txt
rename to Documentation/virt/kvm/nested-vmx.txt
diff --git a/Documentation/virtual/kvm/ppc-pv.txt b/Documentation/virt/kvm/ppc-pv.txt
similarity index 100%
rename from Documentation/virtual/kvm/ppc-pv.txt
rename to Documentation/virt/kvm/ppc-pv.txt
diff --git a/Documentation/virtual/kvm/review-checklist.txt b/Documentation/virt/kvm/review-checklist.txt
similarity index 95%
rename from Documentation/virtual/kvm/review-checklist.txt
rename to Documentation/virt/kvm/review-checklist.txt
index a83b27635fdd..499af499e296 100644
--- a/Documentation/virtual/kvm/review-checklist.txt
+++ b/Documentation/virt/kvm/review-checklist.txt
@@ -7,7 +7,7 @@ Review checklist for kvm patches
2. Patches should be against kvm.git master branch.
3. If the patch introduces or modifies a new userspace API:
- - the API must be documented in Documentation/virtual/kvm/api.txt
+ - the API must be documented in Documentation/virt/kvm/api.txt
- the API must be discoverable using KVM_CHECK_EXTENSION
4. New state must include support for save/restore.
diff --git a/Documentation/virtual/kvm/s390-diag.txt b/Documentation/virt/kvm/s390-diag.txt
similarity index 100%
rename from Documentation/virtual/kvm/s390-diag.txt
rename to Documentation/virt/kvm/s390-diag.txt
diff --git a/Documentation/virtual/kvm/timekeeping.txt b/Documentation/virt/kvm/timekeeping.txt
similarity index 100%
rename from Documentation/virtual/kvm/timekeeping.txt
rename to Documentation/virt/kvm/timekeeping.txt
diff --git a/Documentation/virtual/kvm/vcpu-requests.rst b/Documentation/virt/kvm/vcpu-requests.rst
similarity index 100%
rename from Documentation/virtual/kvm/vcpu-requests.rst
rename to Documentation/virt/kvm/vcpu-requests.rst
diff --git a/Documentation/virtual/paravirt_ops.rst b/Documentation/virt/paravirt_ops.rst
similarity index 100%
rename from Documentation/virtual/paravirt_ops.rst
rename to Documentation/virt/paravirt_ops.rst
diff --git a/Documentation/virtual/uml/UserModeLinux-HOWTO.txt b/Documentation/virt/uml/UserModeLinux-HOWTO.txt
similarity index 100%
rename from Documentation/virtual/uml/UserModeLinux-HOWTO.txt
rename to Documentation/virt/uml/UserModeLinux-HOWTO.txt
diff --git a/MAINTAINERS b/MAINTAINERS
index 783569e3c4b4..5e1f9ee8f86f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8808,7 +8808,7 @@ L: kvm@vger.kernel.org
W: http://www.linux-kvm.org
T: git git://git.kernel.org/pub/scm/virt/kvm/kvm.git
S: Supported
-F: Documentation/virtual/kvm/
+F: Documentation/virt/kvm/
F: include/trace/events/kvm.h
F: include/uapi/asm-generic/kvm*
F: include/uapi/linux/kvm*
@@ -12137,7 +12137,7 @@ M: Thomas Hellstrom <thellstrom@vmware.com>
M: "VMware, Inc." <pv-drivers@vmware.com>
L: virtualization@lists.linux-foundation.org
S: Supported
-F: Documentation/virtual/paravirt_ops.txt
+F: Documentation/virt/paravirt_ops.txt
F: arch/*/kernel/paravirt*
F: arch/*/include/asm/paravirt*.h
F: include/linux/hypervisor.h
@@ -16854,7 +16854,7 @@ W: http://user-mode-linux.sourceforge.net
Q: https://patchwork.ozlabs.org/project/linux-um/list/
T: git git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git
S: Maintained
-F: Documentation/virtual/uml/
+F: Documentation/virt/uml/
F: arch/um/
F: arch/x86/um/
F: fs/hostfs/
diff --git a/arch/powerpc/include/uapi/asm/kvm_para.h b/arch/powerpc/include/uapi/asm/kvm_para.h
index 01555c6ae0f5..be48c2215fa2 100644
--- a/arch/powerpc/include/uapi/asm/kvm_para.h
+++ b/arch/powerpc/include/uapi/asm/kvm_para.h
@@ -31,7 +31,7 @@
* Struct fields are always 32 or 64 bit aligned, depending on them being 32
* or 64 bit wide respectively.
*
- * See Documentation/virtual/kvm/ppc-pv.txt
+ * See Documentation/virt/kvm/ppc-pv.txt
*/
struct kvm_vcpu_arch_shared {
__u64 scratch1;
diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c
index 8f72526e2f68..24843cf49579 100644
--- a/arch/x86/kvm/mmu.c
+++ b/arch/x86/kvm/mmu.c
@@ -3466,7 +3466,7 @@ static bool fast_page_fault(struct kvm_vcpu *vcpu, gva_t gva, int level,
/*
* Currently, fast page fault only works for direct mapping
* since the gfn is not stable for indirect shadow page. See
- * Documentation/virtual/kvm/locking.txt to get more detail.
+ * Documentation/virt/kvm/locking.txt to get more detail.
*/
fault_handled = fast_pf_fix_direct_spte(vcpu, sp,
iterator.sptep, spte,
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index a7c19540ce21..5e3f12d5359e 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -116,7 +116,7 @@ struct kvm_irq_level {
* ACPI gsi notion of irq.
* For IA-64 (APIC model) IOAPIC0: irq 0-23; IOAPIC1: irq 24-47..
* For X86 (standard AT mode) PIC0/1: irq 0-15. IOAPIC0: 0-23..
- * For ARM: See Documentation/virtual/kvm/api.txt
+ * For ARM: See Documentation/virt/kvm/api.txt
*/
union {
__u32 irq;
@@ -1086,7 +1086,7 @@ struct kvm_xen_hvm_config {
*
* KVM_IRQFD_FLAG_RESAMPLE indicates resamplefd is valid and specifies
* the irqfd to operate in resampling mode for level triggered interrupt
- * emulation. See Documentation/virtual/kvm/api.txt.
+ * emulation. See Documentation/virt/kvm/api.txt.
*/
#define KVM_IRQFD_FLAG_RESAMPLE (1 << 1)
diff --git a/tools/include/uapi/linux/kvm.h b/tools/include/uapi/linux/kvm.h
index c2152f3dd02d..e7c67be7c15f 100644
--- a/tools/include/uapi/linux/kvm.h
+++ b/tools/include/uapi/linux/kvm.h
@@ -116,7 +116,7 @@ struct kvm_irq_level {
* ACPI gsi notion of irq.
* For IA-64 (APIC model) IOAPIC0: irq 0-23; IOAPIC1: irq 24-47..
* For X86 (standard AT mode) PIC0/1: irq 0-15. IOAPIC0: 0-23..
- * For ARM: See Documentation/virtual/kvm/api.txt
+ * For ARM: See Documentation/virt/kvm/api.txt
*/
union {
__u32 irq;
@@ -1085,7 +1085,7 @@ struct kvm_xen_hvm_config {
*
* KVM_IRQFD_FLAG_RESAMPLE indicates resamplefd is valid and specifies
* the irqfd to operate in resampling mode for level triggered interrupt
- * emulation. See Documentation/virtual/kvm/api.txt.
+ * emulation. See Documentation/virt/kvm/api.txt.
*/
#define KVM_IRQFD_FLAG_RESAMPLE (1 << 1)
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index f645c0fbf7ec..acc43242a310 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -727,7 +727,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
* Ensure we set mode to IN_GUEST_MODE after we disable
* interrupts and before the final VCPU requests check.
* See the comment in kvm_vcpu_exiting_guest_mode() and
- * Documentation/virtual/kvm/vcpu-requests.rst
+ * Documentation/virt/kvm/vcpu-requests.rst
*/
smp_store_mb(vcpu->mode, IN_GUEST_MODE);
diff --git a/virt/kvm/arm/vgic/vgic-mmio-v3.c b/virt/kvm/arm/vgic/vgic-mmio-v3.c
index 936962abc38d..c45e2d7e942f 100644
--- a/virt/kvm/arm/vgic/vgic-mmio-v3.c
+++ b/virt/kvm/arm/vgic/vgic-mmio-v3.c
@@ -250,7 +250,7 @@ static unsigned long vgic_v3_uaccess_read_pending(struct kvm_vcpu *vcpu,
* pending state of interrupt is latched in pending_latch variable.
* Userspace will save and restore pending state and line_level
* separately.
- * Refer to Documentation/virtual/kvm/devices/arm-vgic-v3.txt
+ * Refer to Documentation/virt/kvm/devices/arm-vgic-v3.txt
* for handling of ISPENDR and ICPENDR.
*/
for (i = 0; i < len * 8; i++) {
diff --git a/virt/kvm/arm/vgic/vgic.h b/virt/kvm/arm/vgic/vgic.h
index 57205beaa981..3b7525deec80 100644
--- a/virt/kvm/arm/vgic/vgic.h
+++ b/virt/kvm/arm/vgic/vgic.h
@@ -42,7 +42,7 @@
VGIC_AFFINITY_LEVEL(val, 3))
/*
- * As per Documentation/virtual/kvm/devices/arm-vgic-v3.txt,
+ * As per Documentation/virt/kvm/devices/arm-vgic-v3.txt,
* below macros are defined for CPUREG encoding.
*/
#define KVM_REG_ARM_VGIC_SYSREG_OP0_MASK 0x000000000000c000
@@ -63,7 +63,7 @@
KVM_REG_ARM_VGIC_SYSREG_OP2_MASK)
/*
- * As per Documentation/virtual/kvm/devices/arm-vgic-its.txt,
+ * As per Documentation/virt/kvm/devices/arm-vgic-its.txt,
* below macros are defined for ITS table entry encoding.
*/
#define KVM_ITS_CTE_VALID_SHIFT 63
--
2.20.1
^ permalink raw reply related
* Re: [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing
From: Minchan Kim @ 2019-07-24 4:28 UTC (permalink / raw)
To: Joel Fernandes
Cc: linux-kernel, vdavydov.dev, Brendan Gregg, kernel-team,
Alexey Dobriyan, Al Viro, Andrew Morton, carmenjackson,
Christian Hansen, Colin Ian King, dancol, David Howells, fmayer,
joaodias, Jonathan Corbet, Kees Cook, Kirill Tkhai,
Konstantin Khlebnikov, linux-doc, linux-fsdevel, linux-mm,
Michal Hocko, Mike Rapoport, namhyung, sspatil, surenb,
Thomas Gleixner, timmurray, tkjos, Vlastimil Babka, wvw
In-Reply-To: <20190723142049.GC104199@google.com>
On Tue, Jul 23, 2019 at 10:20:49AM -0400, Joel Fernandes wrote:
> On Tue, Jul 23, 2019 at 03:13:58PM +0900, Minchan Kim wrote:
> > Hi Joel,
> >
> > On Mon, Jul 22, 2019 at 05:32:04PM -0400, Joel Fernandes (Google) wrote:
> > > The page_idle tracking feature currently requires looking up the pagemap
> > > for a process followed by interacting with /sys/kernel/mm/page_idle.
> > > This is quite cumbersome and can be error-prone too. If between
> >
> > cumbersome: That's the fair tradeoff between idle page tracking and
> > clear_refs because idle page tracking could check even though the page
> > is not mapped.
>
> It is fair tradeoff, but could be made simpler. The userspace code got
> reduced by a good amount as well.
>
> > error-prone: What's the error?
>
> We see in normal Android usage, that some of the times pages appear not to be
> idle even when they really are idle. Reproducing this is a bit unpredictable
> and happens at random occasions. With this new interface, we are seeing this
> happen much much lesser.
I don't know how you did test. Maybe that could be contributed by
swapping out or shared pages touched by other processes or some kernel
behavior not to keep access bit of their operation.
Please investigate more what's the root cause. That would be important
point to justify for the patch motivation.
>
> > > accessing the per-PID pagemap and the global page_idle bitmap, if
> > > something changes with the page then the information is not accurate.
> >
> > What you mean with error is this timing issue?
> > Why do you need to be accurate? IOW, accurate is always good but what's
> > the scale of the accuracy?
>
> There is a time window between looking up pagemap and checking if page is
> idle. Anyway, see below for the primary goals as you asked:
>
> > > More over looking up PFN from pagemap in Android devices is not
> > > supported by unprivileged process and requires SYS_ADMIN and gives 0 for
> > > the PFN.
> > >
> > > This patch adds support to directly interact with page_idle tracking at
> > > the PID level by introducing a /proc/<pid>/page_idle file. This
> > > eliminates the need for userspace to calculate the mapping of the page.
> > > It follows the exact same semantics as the global
> > > /sys/kernel/mm/page_idle, however it is easier to use for some usecases
> > > where looking up PFN is not needed and also does not require SYS_ADMIN.
> >
> > Ah, so the primary goal is to provide convinience interface and it would
> > help accurary, too. IOW, accuracy is not your main goal?
>
> There are a couple of primary goals: Security, conveience and also solving
> the accuracy/reliability problem we are seeing. Do keep in mind looking up
> PFN has security implications. The PFN field in pagemap is zeroed if the user
> does not have CAP_SYS_ADMIN.
Myaybe you don't need PFN. is it?
>
> > > In Android, we are using this for the heap profiler (heapprofd) which
> > > profiles and pin points code paths which allocates and leaves memory
> > > idle for long periods of time.
> >
> > So the goal is to detect idle pages with idle memory tracking?
>
> Isn't that what idle memory tracking does?
To me, it's rather misleading. Please read motivation section in document.
The feature would be good to detect workingset pages, not idle pages
because workingset pages are never freed, swapped out and even we could
count on newly allocated pages.
Motivation
==========
The idle page tracking feature allows to track which memory pages are being
accessed by a workload and which are idle. This information can be useful for
estimating the workload's working set size, which, in turn, can be taken into
account when configuring the workload parameters, setting memory cgroup limits,
or deciding where to place the workload within a compute cluster.
>
> > It couldn't work well because such idle pages could finally swap out and
> > lose every flags of the page descriptor which is working mechanism of
> > idle page tracking. It should have named "workingset page tracking",
> > not "idle page tracking".
>
> The heap profiler that uses page-idle tracking is not to measure working set,
> but to look for pages that are idle for long periods of time.
It's important part. Please include it in the description so that people
understands what's the usecase. As I said above, if it aims for finding
idle pages durting the period, current idle page tracking feature is not
good ironically.
>
> Thanks for bringing up the swapping corner case.. Perhaps we can improve
> the heap profiler to detect this by looking at bits 0-4 in pagemap. While it
Yeb, that could work but it could add overhead again what you want to remove?
Even, userspace should keep metadata to identify that page was already swapped
in last period or newly swapped in new period.
> is true that we would lose access information during the window, there is a
> high likelihood that the page was not accessed which is why it was swapped.
> Thoughts?
It depends on system memory size, workingset size and your sampling period.
It would be never corner case for small memory system as they want to use
memory more efficiently.
>
> thanks,
>
> - Joel
>
>
>
> > > Documentation material:
> > > The idle page tracking API for virtual address indexing using virtual page
> > > frame numbers (VFN) is located at /proc/<pid>/page_idle. It is a bitmap
> > > that follows the same semantics as /sys/kernel/mm/page_idle/bitmap
> > > except that it uses virtual instead of physical frame numbers.
> > >
> > > This idle page tracking API can be simpler to use than physical address
> > > indexing, since the pagemap for a process does not need to be looked up
> > > to mark or read a page's idle bit. It is also more accurate than
> > > physical address indexing since in physical address indexing, address
> > > space changes can occur between reading the pagemap and reading the
> > > bitmap. In virtual address indexing, the process's mmap_sem is held for
> > > the duration of the access.
> > >
> > > Cc: vdavydov.dev@gmail.com
> > > Cc: Brendan Gregg <bgregg@netflix.com>
> > > Cc: kernel-team@android.com
> > > Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
> > >
> > > ---
> > > Internal review -> v1:
> > > Fixes from Suren.
> > > Corrections to change log, docs (Florian, Sandeep)
> > >
> > > fs/proc/base.c | 3 +
> > > fs/proc/internal.h | 1 +
> > > fs/proc/task_mmu.c | 57 +++++++
> > > include/linux/page_idle.h | 4 +
> > > mm/page_idle.c | 305 +++++++++++++++++++++++++++++++++-----
> > > 5 files changed, 330 insertions(+), 40 deletions(-)
> > >
> > > diff --git a/fs/proc/base.c b/fs/proc/base.c
> > > index 77eb628ecc7f..a58dd74606e9 100644
> > > --- a/fs/proc/base.c
> > > +++ b/fs/proc/base.c
> > > @@ -3021,6 +3021,9 @@ static const struct pid_entry tgid_base_stuff[] = {
> > > REG("smaps", S_IRUGO, proc_pid_smaps_operations),
> > > REG("smaps_rollup", S_IRUGO, proc_pid_smaps_rollup_operations),
> > > REG("pagemap", S_IRUSR, proc_pagemap_operations),
> > > +#ifdef CONFIG_IDLE_PAGE_TRACKING
> > > + REG("page_idle", S_IRUSR|S_IWUSR, proc_page_idle_operations),
> > > +#endif
> > > #endif
> > > #ifdef CONFIG_SECURITY
> > > DIR("attr", S_IRUGO|S_IXUGO, proc_attr_dir_inode_operations, proc_attr_dir_operations),
> > > diff --git a/fs/proc/internal.h b/fs/proc/internal.h
> > > index cd0c8d5ce9a1..bc9371880c63 100644
> > > --- a/fs/proc/internal.h
> > > +++ b/fs/proc/internal.h
> > > @@ -293,6 +293,7 @@ extern const struct file_operations proc_pid_smaps_operations;
> > > extern const struct file_operations proc_pid_smaps_rollup_operations;
> > > extern const struct file_operations proc_clear_refs_operations;
> > > extern const struct file_operations proc_pagemap_operations;
> > > +extern const struct file_operations proc_page_idle_operations;
> > >
> > > extern unsigned long task_vsize(struct mm_struct *);
> > > extern unsigned long task_statm(struct mm_struct *,
> > > diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> > > index 4d2b860dbc3f..11ccc53da38e 100644
> > > --- a/fs/proc/task_mmu.c
> > > +++ b/fs/proc/task_mmu.c
> > > @@ -1642,6 +1642,63 @@ const struct file_operations proc_pagemap_operations = {
> > > .open = pagemap_open,
> > > .release = pagemap_release,
> > > };
> > > +
> > > +#ifdef CONFIG_IDLE_PAGE_TRACKING
> > > +static ssize_t proc_page_idle_read(struct file *file, char __user *buf,
> > > + size_t count, loff_t *ppos)
> > > +{
> > > + int ret;
> > > + struct task_struct *tsk = get_proc_task(file_inode(file));
> > > +
> > > + if (!tsk)
> > > + return -EINVAL;
> > > + ret = page_idle_proc_read(file, buf, count, ppos, tsk);
> > > + put_task_struct(tsk);
> > > + return ret;
> > > +}
> > > +
> > > +static ssize_t proc_page_idle_write(struct file *file, const char __user *buf,
> > > + size_t count, loff_t *ppos)
> > > +{
> > > + int ret;
> > > + struct task_struct *tsk = get_proc_task(file_inode(file));
> > > +
> > > + if (!tsk)
> > > + return -EINVAL;
> > > + ret = page_idle_proc_write(file, (char __user *)buf, count, ppos, tsk);
> > > + put_task_struct(tsk);
> > > + return ret;
> > > +}
> > > +
> > > +static int proc_page_idle_open(struct inode *inode, struct file *file)
> > > +{
> > > + struct mm_struct *mm;
> > > +
> > > + mm = proc_mem_open(inode, PTRACE_MODE_READ);
> > > + if (IS_ERR(mm))
> > > + return PTR_ERR(mm);
> > > + file->private_data = mm;
> > > + return 0;
> > > +}
> > > +
> > > +static int proc_page_idle_release(struct inode *inode, struct file *file)
> > > +{
> > > + struct mm_struct *mm = file->private_data;
> > > +
> > > + if (mm)
> > > + mmdrop(mm);
> > > + return 0;
> > > +}
> > > +
> > > +const struct file_operations proc_page_idle_operations = {
> > > + .llseek = mem_lseek, /* borrow this */
> > > + .read = proc_page_idle_read,
> > > + .write = proc_page_idle_write,
> > > + .open = proc_page_idle_open,
> > > + .release = proc_page_idle_release,
> > > +};
> > > +#endif /* CONFIG_IDLE_PAGE_TRACKING */
> > > +
> > > #endif /* CONFIG_PROC_PAGE_MONITOR */
> > >
> > > #ifdef CONFIG_NUMA
> > > diff --git a/include/linux/page_idle.h b/include/linux/page_idle.h
> > > index 1e894d34bdce..f1bc2640d85e 100644
> > > --- a/include/linux/page_idle.h
> > > +++ b/include/linux/page_idle.h
> > > @@ -106,6 +106,10 @@ static inline void clear_page_idle(struct page *page)
> > > }
> > > #endif /* CONFIG_64BIT */
> > >
> > > +ssize_t page_idle_proc_write(struct file *file,
> > > + char __user *buf, size_t count, loff_t *ppos, struct task_struct *tsk);
> > > +ssize_t page_idle_proc_read(struct file *file,
> > > + char __user *buf, size_t count, loff_t *ppos, struct task_struct *tsk);
> > > #else /* !CONFIG_IDLE_PAGE_TRACKING */
> > >
> > > static inline bool page_is_young(struct page *page)
> > > diff --git a/mm/page_idle.c b/mm/page_idle.c
> > > index 295512465065..874a60c41fef 100644
> > > --- a/mm/page_idle.c
> > > +++ b/mm/page_idle.c
> > > @@ -11,6 +11,7 @@
> > > #include <linux/mmu_notifier.h>
> > > #include <linux/page_ext.h>
> > > #include <linux/page_idle.h>
> > > +#include <linux/sched/mm.h>
> > >
> > > #define BITMAP_CHUNK_SIZE sizeof(u64)
> > > #define BITMAP_CHUNK_BITS (BITMAP_CHUNK_SIZE * BITS_PER_BYTE)
> > > @@ -28,15 +29,12 @@
> > > *
> > > * This function tries to get a user memory page by pfn as described above.
> > > */
> > > -static struct page *page_idle_get_page(unsigned long pfn)
> > > +static struct page *page_idle_get_page(struct page *page_in)
> > > {
> > > struct page *page;
> > > pg_data_t *pgdat;
> > >
> > > - if (!pfn_valid(pfn))
> > > - return NULL;
> > > -
> > > - page = pfn_to_page(pfn);
> > > + page = page_in;
> > > if (!page || !PageLRU(page) ||
> > > !get_page_unless_zero(page))
> > > return NULL;
> > > @@ -51,6 +49,15 @@ static struct page *page_idle_get_page(unsigned long pfn)
> > > return page;
> > > }
> > >
> > > +static struct page *page_idle_get_page_pfn(unsigned long pfn)
> > > +{
> > > +
> > > + if (!pfn_valid(pfn))
> > > + return NULL;
> > > +
> > > + return page_idle_get_page(pfn_to_page(pfn));
> > > +}
> > > +
> > > static bool page_idle_clear_pte_refs_one(struct page *page,
> > > struct vm_area_struct *vma,
> > > unsigned long addr, void *arg)
> > > @@ -118,6 +125,47 @@ static void page_idle_clear_pte_refs(struct page *page)
> > > unlock_page(page);
> > > }
> > >
> > > +/* Helper to get the start and end frame given a pos and count */
> > > +static int page_idle_get_frames(loff_t pos, size_t count, struct mm_struct *mm,
> > > + unsigned long *start, unsigned long *end)
> > > +{
> > > + unsigned long max_frame;
> > > +
> > > + /* If an mm is not given, assume we want physical frames */
> > > + max_frame = mm ? (mm->task_size >> PAGE_SHIFT) : max_pfn;
> > > +
> > > + if (pos % BITMAP_CHUNK_SIZE || count % BITMAP_CHUNK_SIZE)
> > > + return -EINVAL;
> > > +
> > > + *start = pos * BITS_PER_BYTE;
> > > + if (*start >= max_frame)
> > > + return -ENXIO;
> > > +
> > > + *end = *start + count * BITS_PER_BYTE;
> > > + if (*end > max_frame)
> > > + *end = max_frame;
> > > + return 0;
> > > +}
> > > +
> > > +static bool page_really_idle(struct page *page)
> > > +{
> > > + if (!page)
> > > + return false;
> > > +
> > > + if (page_is_idle(page)) {
> > > + /*
> > > + * The page might have been referenced via a
> > > + * pte, in which case it is not idle. Clear
> > > + * refs and recheck.
> > > + */
> > > + page_idle_clear_pte_refs(page);
> > > + if (page_is_idle(page))
> > > + return true;
> > > + }
> > > +
> > > + return false;
> > > +}
> > > +
> > > static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj,
> > > struct bin_attribute *attr, char *buf,
> > > loff_t pos, size_t count)
> > > @@ -125,35 +173,21 @@ static ssize_t page_idle_bitmap_read(struct file *file, struct kobject *kobj,
> > > u64 *out = (u64 *)buf;
> > > struct page *page;
> > > unsigned long pfn, end_pfn;
> > > - int bit;
> > > -
> > > - if (pos % BITMAP_CHUNK_SIZE || count % BITMAP_CHUNK_SIZE)
> > > - return -EINVAL;
> > > -
> > > - pfn = pos * BITS_PER_BYTE;
> > > - if (pfn >= max_pfn)
> > > - return 0;
> > > + int bit, ret;
> > >
> > > - end_pfn = pfn + count * BITS_PER_BYTE;
> > > - if (end_pfn > max_pfn)
> > > - end_pfn = max_pfn;
> > > + ret = page_idle_get_frames(pos, count, NULL, &pfn, &end_pfn);
> > > + if (ret == -ENXIO)
> > > + return 0; /* Reads beyond max_pfn do nothing */
> > > + else if (ret)
> > > + return ret;
> > >
> > > for (; pfn < end_pfn; pfn++) {
> > > bit = pfn % BITMAP_CHUNK_BITS;
> > > if (!bit)
> > > *out = 0ULL;
> > > - page = page_idle_get_page(pfn);
> > > - if (page) {
> > > - if (page_is_idle(page)) {
> > > - /*
> > > - * The page might have been referenced via a
> > > - * pte, in which case it is not idle. Clear
> > > - * refs and recheck.
> > > - */
> > > - page_idle_clear_pte_refs(page);
> > > - if (page_is_idle(page))
> > > - *out |= 1ULL << bit;
> > > - }
> > > + page = page_idle_get_page_pfn(pfn);
> > > + if (page && page_really_idle(page)) {
> > > + *out |= 1ULL << bit;
> > > put_page(page);
> > > }
> > > if (bit == BITMAP_CHUNK_BITS - 1)
> > > @@ -170,23 +204,16 @@ static ssize_t page_idle_bitmap_write(struct file *file, struct kobject *kobj,
> > > const u64 *in = (u64 *)buf;
> > > struct page *page;
> > > unsigned long pfn, end_pfn;
> > > - int bit;
> > > + int bit, ret;
> > >
> > > - if (pos % BITMAP_CHUNK_SIZE || count % BITMAP_CHUNK_SIZE)
> > > - return -EINVAL;
> > > -
> > > - pfn = pos * BITS_PER_BYTE;
> > > - if (pfn >= max_pfn)
> > > - return -ENXIO;
> > > -
> > > - end_pfn = pfn + count * BITS_PER_BYTE;
> > > - if (end_pfn > max_pfn)
> > > - end_pfn = max_pfn;
> > > + ret = page_idle_get_frames(pos, count, NULL, &pfn, &end_pfn);
> > > + if (ret)
> > > + return ret;
> > >
> > > for (; pfn < end_pfn; pfn++) {
> > > bit = pfn % BITMAP_CHUNK_BITS;
> > > if ((*in >> bit) & 1) {
> > > - page = page_idle_get_page(pfn);
> > > + page = page_idle_get_page_pfn(pfn);
> > > if (page) {
> > > page_idle_clear_pte_refs(page);
> > > set_page_idle(page);
> > > @@ -224,10 +251,208 @@ struct page_ext_operations page_idle_ops = {
> > > };
> > > #endif
> > >
> > > +/* page_idle tracking for /proc/<pid>/page_idle */
> > > +
> > > +static DEFINE_SPINLOCK(idle_page_list_lock);
> > > +struct list_head idle_page_list;
> > > +
> > > +struct page_node {
> > > + struct page *page;
> > > + unsigned long addr;
> > > + struct list_head list;
> > > +};
> > > +
> > > +struct page_idle_proc_priv {
> > > + unsigned long start_addr;
> > > + char *buffer;
> > > + int write;
> > > +};
> > > +
> > > +static void add_page_idle_list(struct page *page,
> > > + unsigned long addr, struct mm_walk *walk)
> > > +{
> > > + struct page *page_get;
> > > + struct page_node *pn;
> > > + int bit;
> > > + unsigned long frames;
> > > + struct page_idle_proc_priv *priv = walk->private;
> > > + u64 *chunk = (u64 *)priv->buffer;
> > > +
> > > + if (priv->write) {
> > > + /* Find whether this page was asked to be marked */
> > > + frames = (addr - priv->start_addr) >> PAGE_SHIFT;
> > > + bit = frames % BITMAP_CHUNK_BITS;
> > > + chunk = &chunk[frames / BITMAP_CHUNK_BITS];
> > > + if (((*chunk >> bit) & 1) == 0)
> > > + return;
> > > + }
> > > +
> > > + page_get = page_idle_get_page(page);
> > > + if (!page_get)
> > > + return;
> > > +
> > > + pn = kmalloc(sizeof(*pn), GFP_ATOMIC);
> > > + if (!pn)
> > > + return;
> > > +
> > > + pn->page = page_get;
> > > + pn->addr = addr;
> > > + list_add(&pn->list, &idle_page_list);
> > > +}
> > > +
> > > +static int pte_page_idle_proc_range(pmd_t *pmd, unsigned long addr,
> > > + unsigned long end,
> > > + struct mm_walk *walk)
> > > +{
> > > + struct vm_area_struct *vma = walk->vma;
> > > + pte_t *pte;
> > > + spinlock_t *ptl;
> > > + struct page *page;
> > > +
> > > + ptl = pmd_trans_huge_lock(pmd, vma);
> > > + if (ptl) {
> > > + if (pmd_present(*pmd)) {
> > > + page = follow_trans_huge_pmd(vma, addr, pmd,
> > > + FOLL_DUMP|FOLL_WRITE);
> > > + if (!IS_ERR_OR_NULL(page))
> > > + add_page_idle_list(page, addr, walk);
> > > + }
> > > + spin_unlock(ptl);
> > > + return 0;
> > > + }
> > > +
> > > + if (pmd_trans_unstable(pmd))
> > > + return 0;
> > > +
> > > + pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl);
> > > + for (; addr != end; pte++, addr += PAGE_SIZE) {
> > > + if (!pte_present(*pte))
> > > + continue;
> > > +
> > > + page = vm_normal_page(vma, addr, *pte);
> > > + if (page)
> > > + add_page_idle_list(page, addr, walk);
> > > + }
> > > +
> > > + pte_unmap_unlock(pte - 1, ptl);
> > > + return 0;
> > > +}
> > > +
> > > +ssize_t page_idle_proc_generic(struct file *file, char __user *ubuff,
> > > + size_t count, loff_t *pos,
> > > + struct task_struct *tsk, int write)
> > > +{
> > > + int ret;
> > > + char *buffer;
> > > + u64 *out;
> > > + unsigned long start_addr, end_addr, start_frame, end_frame;
> > > + struct mm_struct *mm = file->private_data;
> > > + struct mm_walk walk = { .pmd_entry = pte_page_idle_proc_range, };
> > > + struct page_node *cur, *next;
> > > + struct page_idle_proc_priv priv;
> > > + bool walk_error = false;
> > > +
> > > + if (!mm || !mmget_not_zero(mm))
> > > + return -EINVAL;
> > > +
> > > + if (count > PAGE_SIZE)
> > > + count = PAGE_SIZE;
> > > +
> > > + buffer = kzalloc(PAGE_SIZE, GFP_KERNEL);
> > > + if (!buffer) {
> > > + ret = -ENOMEM;
> > > + goto out_mmput;
> > > + }
> > > + out = (u64 *)buffer;
> > > +
> > > + if (write && copy_from_user(buffer, ubuff, count)) {
> > > + ret = -EFAULT;
> > > + goto out;
> > > + }
> > > +
> > > + ret = page_idle_get_frames(*pos, count, mm, &start_frame, &end_frame);
> > > + if (ret)
> > > + goto out;
> > > +
> > > + start_addr = (start_frame << PAGE_SHIFT);
> > > + end_addr = (end_frame << PAGE_SHIFT);
> > > + priv.buffer = buffer;
> > > + priv.start_addr = start_addr;
> > > + priv.write = write;
> > > + walk.private = &priv;
> > > + walk.mm = mm;
> > > +
> > > + down_read(&mm->mmap_sem);
> > > +
> > > + /*
> > > + * Protects the idle_page_list which is needed because
> > > + * walk_page_vma() holds ptlock which deadlocks with
> > > + * page_idle_clear_pte_refs(). So we have to collect all
> > > + * pages first, and then call page_idle_clear_pte_refs().
> > > + */
> > > + spin_lock(&idle_page_list_lock);
> > > + ret = walk_page_range(start_addr, end_addr, &walk);
> > > + if (ret)
> > > + walk_error = true;
> > > +
> > > + list_for_each_entry_safe(cur, next, &idle_page_list, list) {
> > > + int bit, index;
> > > + unsigned long off;
> > > + struct page *page = cur->page;
> > > +
> > > + if (unlikely(walk_error))
> > > + goto remove_page;
> > > +
> > > + if (write) {
> > > + page_idle_clear_pte_refs(page);
> > > + set_page_idle(page);
> > > + } else {
> > > + if (page_really_idle(page)) {
> > > + off = ((cur->addr) >> PAGE_SHIFT) - start_frame;
> > > + bit = off % BITMAP_CHUNK_BITS;
> > > + index = off / BITMAP_CHUNK_BITS;
> > > + out[index] |= 1ULL << bit;
> > > + }
> > > + }
> > > +remove_page:
> > > + put_page(page);
> > > + list_del(&cur->list);
> > > + kfree(cur);
> > > + }
> > > + spin_unlock(&idle_page_list_lock);
> > > +
> > > + if (!write && !walk_error)
> > > + ret = copy_to_user(ubuff, buffer, count);
> > > +
> > > + up_read(&mm->mmap_sem);
> > > +out:
> > > + kfree(buffer);
> > > +out_mmput:
> > > + mmput(mm);
> > > + if (!ret)
> > > + ret = count;
> > > + return ret;
> > > +
> > > +}
> > > +
> > > +ssize_t page_idle_proc_read(struct file *file, char __user *ubuff,
> > > + size_t count, loff_t *pos, struct task_struct *tsk)
> > > +{
> > > + return page_idle_proc_generic(file, ubuff, count, pos, tsk, 0);
> > > +}
> > > +
> > > +ssize_t page_idle_proc_write(struct file *file, char __user *ubuff,
> > > + size_t count, loff_t *pos, struct task_struct *tsk)
> > > +{
> > > + return page_idle_proc_generic(file, ubuff, count, pos, tsk, 1);
> > > +}
> > > +
> > > static int __init page_idle_init(void)
> > > {
> > > int err;
> > >
> > > + INIT_LIST_HEAD(&idle_page_list);
> > > +
> > > err = sysfs_create_group(mm_kobj, &page_idle_attr_group);
> > > if (err) {
> > > pr_err("page_idle: register sysfs failed\n");
> > > --
> > > 2.22.0.657.g960e92d24f-goog
^ permalink raw reply
* RE: [PATCH V15 1/5] dt-bindings: fsl: scu: add thermal binding
From: Anson Huang @ 2019-07-24 3:16 UTC (permalink / raw)
To: Anson Huang, robh+dt@kernel.org, mark.rutland@arm.com,
corbet@lwn.net, shawnguo@kernel.org, s.hauer@pengutronix.de,
kernel@pengutronix.de, festevam@gmail.com,
catalin.marinas@arm.com, will.deacon@arm.com, rui.zhang@intel.com,
edubezval@gmail.com, daniel.lezcano@linaro.org, Aisheng Dong,
ulf.hansson@linaro.org, Peng Fan, mchehab+samsung@kernel.org,
linux@roeck-us.net, Daniel Baluta, maxime.ripard@bootlin.com,
horms+renesas@verge.net.au, olof@lixom.net,
jagan@amarulasolutions.com, bjorn.andersson@linaro.org,
Leonard Crestez, dinguyen@kernel.org,
enric.balletbo@collabora.com, devicetree@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
linux-arm-kernel@lists.infradead.org, linux-pm@vger.kernel.org
Cc: dl-linux-imx
In-Reply-To: <DB3PR0402MB39162C5B5AF828B127DD871EF5E00@DB3PR0402MB3916.eurprd04.prod.outlook.com>
Ping...
> Hi, Daniel/Rui/Eduardo
> Could you please take a look at this patch series?
>
> Anson
>
> > From: Anson Huang <Anson.Huang@nxp.com>
> >
> > NXP i.MX8QXP is an ARMv8 SoC with a Cortex-M4 core inside as system
> > controller, the system controller is in charge of system power, clock
> > and thermal sensors etc. management, Linux kernel has to communicate
> > with system controller via MU (message unit) IPC to get temperature
> > from thermal sensors, this patch adds binding doc for i.MX system
> > controller thermal driver.
> >
> > Signed-off-by: Anson Huang <Anson.Huang@nxp.com>
> > Reviewed-by: Rob Herring <robh@kernel.org>
> > Reviewed-by: Dong Aisheng <aisheng.dong@nxp.com>
> > ---
> > No change.
> > ---
> > .../devicetree/bindings/arm/freescale/fsl,scu.txt | 16
> ++++++++++++++++
> > 1 file changed, 16 insertions(+)
> >
> > diff --git
> > a/Documentation/devicetree/bindings/arm/freescale/fsl,scu.txt
> > b/Documentation/devicetree/bindings/arm/freescale/fsl,scu.txt
> > index a575e42..fc3844e 100644
> > --- a/Documentation/devicetree/bindings/arm/freescale/fsl,scu.txt
> > +++ b/Documentation/devicetree/bindings/arm/freescale/fsl,scu.txt
> > @@ -155,6 +155,17 @@ Required properties:
> > Optional properties:
> > - timeout-sec: contains the watchdog timeout in seconds.
> >
> > +Thermal bindings based on SCU Message Protocol
> > +------------------------------------------------------------
> > +
> > +Required properties:
> > +- compatible: Should be :
> > + "fsl,imx8qxp-sc-thermal"
> > + followed by "fsl,imx-sc-thermal";
> > +
> > +- #thermal-sensor-cells: See
> > Documentation/devicetree/bindings/thermal/thermal.txt
> > + for a description.
> > +
> > Example (imx8qxp):
> > -------------
> > aliases {
> > @@ -222,6 +233,11 @@ firmware {
> > compatible = "fsl,imx8qxp-sc-wdt", "fsl,imx-sc-wdt";
> > timeout-sec = <60>;
> > };
> > +
> > + tsens: thermal-sensor {
> > + compatible = "fsl,imx8qxp-sc-thermal", "fsl,imx-sc-
> > thermal";
> > + #thermal-sensor-cells = <1>;
> > + };
> > };
> > };
> >
> > --
> > 2.7.4
^ permalink raw reply
* [PATCH v7 3/7] of/platform: Add functional dependency link from DT bindings
From: Saravana Kannan @ 2019-07-24 0:10 UTC (permalink / raw)
To: Rob Herring, Mark Rutland, Greg Kroah-Hartman, Rafael J. Wysocki,
Frank Rowand, Jonathan Corbet
Cc: Saravana Kannan, devicetree, linux-kernel, David Collins,
kernel-team, linux-doc
In-Reply-To: <20190724001100.133423-1-saravanak@google.com>
Add device-links after the devices are created (but before they are
probed) by looking at common DT bindings like clocks and
interconnects.
Automatically adding device-links for functional dependencies at the
framework level provides the following benefits:
- Optimizes device probe order and avoids the useless work of
attempting probes of devices that will not probe successfully
(because their suppliers aren't present or haven't probed yet).
For example, in a commonly available mobile SoC, registering just
one consumer device's driver at an initcall level earlier than the
supplier device's driver causes 11 failed probe attempts before the
consumer device probes successfully. This was with a kernel with all
the drivers statically compiled in. This problem gets a lot worse if
all the drivers are loaded as modules without direct symbol
dependencies.
- Supplier devices like clock providers, interconnect providers, etc
need to keep the resources they provide active and at a particular
state(s) during boot up even if their current set of consumers don't
request the resource to be active. This is because the rest of the
consumers might not have probed yet and turning off the resource
before all the consumers have probed could lead to a hang or
undesired user experience.
Some frameworks (Eg: regulator) handle this today by turning off
"unused" resources at late_initcall_sync and hoping all the devices
have probed by then. This is not a valid assumption for systems with
loadable modules. Other frameworks (Eg: clock) just don't handle
this due to the lack of a clear signal for when they can turn off
resources. This leads to downstream hacks to handle cases like this
that can easily be solved in the upstream kernel.
By linking devices before they are probed, we give suppliers a clear
count of the number of dependent consumers. Once all of the
consumers are active, the suppliers can turn off the unused
resources without making assumptions about the number of consumers.
By default we just add device-links to track "driver presence" (probe
succeeded) of the supplier device. If any other functionality provided
by device-links are needed, it is left to the consumer/supplier
devices to change the link when they probe.
Signed-off-by: Saravana Kannan <saravanak@google.com>
---
.../admin-guide/kernel-parameters.txt | 5 +
drivers/of/platform.c | 165 ++++++++++++++++++
2 files changed, 170 insertions(+)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 46b826fcb5ad..12937349d79d 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3170,6 +3170,11 @@
This can be set from sysctl after boot.
See Documentation/admin-guide/sysctl/vm.rst for details.
+ of_devlink [KNL] Make device links from common DT bindings. Useful
+ for optimizing probe order and making sure resources
+ aren't turned off before the consumer devices have
+ probed.
+
ohci1394_dma=early [HW] enable debugging via the ohci1394 driver.
See Documentation/debugging-via-ohci1394.txt for more
info.
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 7801e25e6895..4344419a26fc 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -508,6 +508,170 @@ int of_platform_default_populate(struct device_node *root,
}
EXPORT_SYMBOL_GPL(of_platform_default_populate);
+bool of_link_is_valid(struct device_node *con, struct device_node *sup)
+{
+ of_node_get(sup);
+ /*
+ * Don't allow linking a device node as a consumer of one of its
+ * descendant nodes. By definition, a child node can't be a functional
+ * dependency for the parent node.
+ */
+ while (sup) {
+ if (sup == con) {
+ of_node_put(sup);
+ return false;
+ }
+ sup = of_get_next_parent(sup);
+ }
+ return true;
+}
+
+static int of_link_to_phandle(struct device *dev, struct device_node *sup_np)
+{
+ struct platform_device *sup_dev;
+ u32 dl_flags = DL_FLAG_AUTOPROBE_CONSUMER;
+ int ret = 0;
+
+ /*
+ * Since we are trying to create device links, we need to find
+ * the actual device node that owns this supplier phandle.
+ * Often times it's the same node, but sometimes it can be one
+ * of the parents. So walk up the parent till you find a
+ * device.
+ */
+ while (sup_np && !of_find_property(sup_np, "compatible", NULL))
+ sup_np = of_get_next_parent(sup_np);
+ if (!sup_np)
+ return 0;
+
+ if (!of_link_is_valid(dev->of_node, sup_np)) {
+ of_node_put(sup_np);
+ return 0;
+ }
+ sup_dev = of_find_device_by_node(sup_np);
+ of_node_put(sup_np);
+ if (!sup_dev)
+ return -ENODEV;
+ if (!device_link_add(dev, &sup_dev->dev, dl_flags))
+ ret = -ENODEV;
+ put_device(&sup_dev->dev);
+ return ret;
+}
+
+static struct device_node *parse_prop_cells(struct device_node *np,
+ const char *prop, int index,
+ const char *binding,
+ const char *cell)
+{
+ struct of_phandle_args sup_args;
+
+ /* Don't need to check property name for every index. */
+ if (!index && strcmp(prop, binding))
+ return NULL;
+
+ if (of_parse_phandle_with_args(np, binding, cell, index, &sup_args))
+ return NULL;
+
+ return sup_args.np;
+}
+
+static struct device_node *parse_clocks(struct device_node *np,
+ const char *prop, int index)
+{
+ return parse_prop_cells(np, prop, index, "clocks", "#clock-cells");
+}
+
+static struct device_node *parse_interconnects(struct device_node *np,
+ const char *prop, int index)
+{
+ return parse_prop_cells(np, prop, index, "interconnects",
+ "#interconnect-cells");
+}
+
+static int strcmp_suffix(const char *str, const char *suffix)
+{
+ unsigned int len, suffix_len;
+
+ len = strlen(str);
+ suffix_len = strlen(suffix);
+ if (len <= suffix_len)
+ return -1;
+ return strcmp(str + len - suffix_len, suffix);
+}
+
+static struct device_node *parse_regulators(struct device_node *np,
+ const char *prop, int index)
+{
+ if (index || strcmp_suffix(prop, "-supply"))
+ return NULL;
+
+ return of_parse_phandle(np, prop, 0);
+}
+
+/**
+ * struct supplier_bindings - Information for parsing supplier DT binding
+ *
+ * @parse_prop: If the function cannot parse the property, return NULL.
+ * Otherwise, return the phandle listed in the property
+ * that corresponds to the index.
+ */
+struct supplier_bindings {
+ struct device_node *(*parse_prop)(struct device_node *np,
+ const char *name, int index);
+};
+
+static const struct supplier_bindings bindings[] = {
+ { .parse_prop = parse_clocks, },
+ { .parse_prop = parse_interconnects, },
+ { .parse_prop = parse_regulators, },
+ { },
+};
+
+static bool of_link_property(struct device *dev, struct device_node *con_np,
+ const char *prop)
+{
+ struct device_node *phandle;
+ struct supplier_bindings *s = bindings;
+ unsigned int i = 0;
+ bool done = true, matched = false;
+
+ while (!matched && s->parse_prop) {
+ while ((phandle = s->parse_prop(con_np, prop, i))) {
+ matched = true;
+ i++;
+ if (of_link_to_phandle(dev, phandle))
+ /*
+ * Don't stop at the first failure. See
+ * Documentation for bus_type.add_links for
+ * more details.
+ */
+ done = false;
+ }
+ s++;
+ }
+ return done ? 0 : -ENODEV;
+}
+
+static bool of_devlink;
+core_param(of_devlink, of_devlink, bool, 0);
+
+static int of_link_to_suppliers(struct device *dev)
+{
+ struct property *p;
+ bool done = true;
+
+ if (!of_devlink)
+ return 0;
+ if (unlikely(!dev->of_node))
+ return 0;
+
+ for_each_property_of_node(dev->of_node, p)
+ if (of_link_property(dev, dev->of_node, p->name))
+ done = false;
+
+ return done ? 0 : -ENODEV;
+}
+
#ifndef CONFIG_PPC
static const struct of_device_id reserved_mem_matches[] = {
{ .compatible = "qcom,rmtfs-mem" },
@@ -523,6 +687,7 @@ static int __init of_platform_default_populate_init(void)
if (!of_have_populated_dt())
return -ENODEV;
+ platform_bus_type.add_links = of_link_to_suppliers;
/*
* Handle certain compatibles explicitly, since we don't want to create
* platform_devices for every node in /reserved-memory with a
--
2.22.0.709.g102302147b-goog
^ permalink raw reply related
* Re: [PATCH v6 3/7] of/platform: Add functional dependency link from DT bindings
From: Saravana Kannan @ 2019-07-23 23:57 UTC (permalink / raw)
To: Rob Herring
Cc: Mark Rutland, Greg Kroah-Hartman, Rafael J. Wysocki, Frank Rowand,
Jonathan Corbet,
open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS,
linux-kernel@vger.kernel.org, David Collins, Android Kernel Team,
Linux Doc Mailing List
In-Reply-To: <CAL_JsqJC-xvj5OJeFRPPNaYJj=-RqmXFJepZ5Q2+z36-7qyPgQ@mail.gmail.com>
On Tue, Jul 23, 2019 at 3:18 PM Rob Herring <robh+dt@kernel.org> wrote:
>
> On Tue, Jul 23, 2019 at 2:49 PM Saravana Kannan <saravanak@google.com> wrote:
> >
> > On Tue, Jul 23, 2019 at 11:06 AM Rob Herring <robh+dt@kernel.org> wrote:
> > >
> > > On Sat, Jul 20, 2019 at 12:17 AM Saravana Kannan <saravanak@google.com> wrote:
> > > >
> > > > Add device-links after the devices are created (but before they are
> > > > probed) by looking at common DT bindings like clocks and
> > > > interconnects.
> > >
> > > The structure now looks a lot better to me. A few minor things below.
> >
> > Thanks.
> >
> > > >
> > > > Automatically adding device-links for functional dependencies at the
> > > > framework level provides the following benefits:
> > > >
> > > > - Optimizes device probe order and avoids the useless work of
> > > > attempting probes of devices that will not probe successfully
> > > > (because their suppliers aren't present or haven't probed yet).
> > > >
> > > > For example, in a commonly available mobile SoC, registering just
> > > > one consumer device's driver at an initcall level earlier than the
> > > > supplier device's driver causes 11 failed probe attempts before the
> > > > consumer device probes successfully. This was with a kernel with all
> > > > the drivers statically compiled in. This problem gets a lot worse if
> > > > all the drivers are loaded as modules without direct symbol
> > > > dependencies.
> > > >
> > > > - Supplier devices like clock providers, interconnect providers, etc
> > > > need to keep the resources they provide active and at a particular
> > > > state(s) during boot up even if their current set of consumers don't
> > > > request the resource to be active. This is because the rest of the
> > > > consumers might not have probed yet and turning off the resource
> > > > before all the consumers have probed could lead to a hang or
> > > > undesired user experience.
> > > >
> > > > Some frameworks (Eg: regulator) handle this today by turning off
> > > > "unused" resources at late_initcall_sync and hoping all the devices
> > > > have probed by then. This is not a valid assumption for systems with
> > > > loadable modules. Other frameworks (Eg: clock) just don't handle
> > > > this due to the lack of a clear signal for when they can turn off
> > > > resources. This leads to downstream hacks to handle cases like this
> > > > that can easily be solved in the upstream kernel.
> > > >
> > > > By linking devices before they are probed, we give suppliers a clear
> > > > count of the number of dependent consumers. Once all of the
> > > > consumers are active, the suppliers can turn off the unused
> > > > resources without making assumptions about the number of consumers.
> > > >
> > > > By default we just add device-links to track "driver presence" (probe
> > > > succeeded) of the supplier device. If any other functionality provided
> > > > by device-links are needed, it is left to the consumer/supplier
> > > > devices to change the link when they probe.
> > > >
> > > > Signed-off-by: Saravana Kannan <saravanak@google.com>
> > > > ---
> > > > .../admin-guide/kernel-parameters.txt | 5 +
> > > > drivers/of/platform.c | 158 ++++++++++++++++++
> > > > 2 files changed, 163 insertions(+)
> > > >
> > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> > > > index 138f6664b2e2..109b4310844f 100644
> > > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > > @@ -3141,6 +3141,11 @@
> > > > This can be set from sysctl after boot.
> > > > See Documentation/sysctl/vm.txt for details.
> > > >
> > > > + of_devlink [KNL] Make device links from common DT bindings. Useful
> > > > + for optimizing probe order and making sure resources
> > > > + aren't turned off before the consumer devices have
> > > > + probed.
> > > > +
> > > > ohci1394_dma=early [HW] enable debugging via the ohci1394 driver.
> > > > See Documentation/debugging-via-ohci1394.txt for more
> > > > info.
> > > > diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> > > > index 04ad312fd85b..88a2086e26fa 100644
> > > > --- a/drivers/of/platform.c
> > > > +++ b/drivers/of/platform.c
> > > > @@ -509,6 +509,163 @@ int of_platform_default_populate(struct device_node *root,
> > > > }
> > > > EXPORT_SYMBOL_GPL(of_platform_default_populate);
> > > >
> > > > +bool of_link_is_valid(struct device_node *con, struct device_node *sup)
> > > > +{
> > > > + of_node_get(sup);
> > > > + /*
> > > > + * Don't allow linking a device node as a consumer of one of its
> > > > + * descendant nodes. By definition, a child node can't be a functional
> > > > + * dependency for the parent node.
> > > > + */
> > > > + while (sup) {
> > > > + if (sup == con) {
> > > > + of_node_put(sup);
> > > > + return false;
> > > > + }
> > > > + sup = of_get_next_parent(sup);
> > > > + }
> > > > + return true;
> > > > +}
> > > > +
> > > > +static int of_link_to_phandle(struct device *dev, struct device_node *sup_np)
> > > > +{
> > > > + struct platform_device *sup_dev;
> > > > + u32 dl_flags = DL_FLAG_AUTOPROBE_CONSUMER;
> > > > + int ret = 0;
> > > > +
> > > > + /*
> > > > + * Since we are trying to create device links, we need to find
> > > > + * the actual device node that owns this supplier phandle.
> > > > + * Often times it's the same node, but sometimes it can be one
> > > > + * of the parents. So walk up the parent till you find a
> > > > + * device.
> > > > + */
> > > > + while (sup_np && !of_find_property(sup_np, "compatible", NULL))
> > > > + sup_np = of_get_next_parent(sup_np);
> > > > + if (!sup_np)
> > > > + return 0;
> > > > +
> > > > + if (!of_link_is_valid(dev->of_node, sup_np)) {
> > > > + of_node_put(sup_np);
> > > > + return 0;
> > > > + }
> > > > + sup_dev = of_find_device_by_node(sup_np);
> > > > + of_node_put(sup_np);
> > > > + if (!sup_dev)
> > > > + return -ENODEV;
> > > > + if (!device_link_add(dev, &sup_dev->dev, dl_flags))
> > > > + ret = -ENODEV;
> > > > + put_device(&sup_dev->dev);
> > > > + return ret;
> > > > +}
> > > > +
> > > > +static struct device_node *parse_prop_cells(struct device_node *np,
> > > > + const char *prop, int i,
> > >
> > > I like 'i' for for loops, but less so for function params. Perhaps
> > > 'index' instead like of_parse_phandle_with_args.
> >
> > Sounds good.
> >
> > >
> > > > + const char *binding,
> > > > + const char *cell)
> > > > +{
> > > > + struct of_phandle_args sup_args;
> > > > +
> > > > + if (!i && strcmp(prop, binding))
> > >
> > > Why the '!i' test?
> >
> > To avoid a string comparison for every index. It's kinda wasteful once
> > the first index passes.
>
> That's not very obvious and pretty fragile though this is a static
> function. Perhaps we should split to match() and parse() functions.
Yeah, I did think about doing this. That's why I made it a struct for
supplier_bindings instead of just an array of function pointers. But
having a parse function just for a strcmp() was creating a lot of code
noise. So went ahead and did it this way. We can keep it this way and
if we later see the need for a separate parse function, it should be
easy to do so (because it's already a struct for each binding).
> At
> least put a comment here as to what we're doing.
Done.
> >
> > > > + return NULL;
> > > > +
> > > > + if (of_parse_phandle_with_args(np, binding, cell, i, &sup_args))
> > > > + return NULL;
> > > > +
> > > > + return sup_args.np;
> > > > +}
> > > > +
> > > > +static struct device_node *parse_clocks(struct device_node *np,
> > > > + const char *prop, int i)
> > > > +{
> > > > + return parse_prop_cells(np, prop, i, "clocks", "#clock-cells");
> > > > +}
> > > > +
> > > > +static struct device_node *parse_interconnects(struct device_node *np,
> > > > + const char *prop, int i)
> > > > +{
> > > > + return parse_prop_cells(np, prop, i, "interconnects",
> > > > + "#interconnect-cells");
> > > > +}
> > > > +
> > > > +static int strcmp_suffix(const char *str, const char *suffix)
> > > > +{
> > > > + unsigned int len, suffix_len;
> > > > +
> > > > + len = strlen(str);
> > > > + suffix_len = strlen(suffix);
> > > > + if (len <= suffix_len)
> > > > + return -1;
> > > > + return strcmp(str + len - suffix_len, suffix);
> > > > +}
> > > > +
> > > > +static struct device_node *parse_regulators(struct device_node *np,
> > > > + const char *prop, int i)
> > > > +{
> > > > + if (i || strcmp_suffix(prop, "-supply"))
> > > > + return NULL;
> > > > +
> > > > + return of_parse_phandle(np, prop, 0);
> > > > +}
> > > > +
> > > > +/**
> > > > + * struct supplier_bindings - Information for parsing supplier DT binding
> > > > + *
> > > > + * @parse_prop: If the function cannot parse the property, return NULL.
> > > > + * Otherwise, return the phandle listed in the property
> > > > + * that corresponds to index i.
> > > > + */
> > > > +struct supplier_bindings {
> > > > + struct device_node *(*parse_prop)(struct device_node *np,
> > > > + const char *name, int i);
> > > > +};
> > > > +
> > > > +struct supplier_bindings bindings[] = {
> > >
> > > static const
> >
> > Will do.
> >
> > >
> > > > + { .parse_prop = parse_clocks, },
> > > > + { .parse_prop = parse_interconnects, },
> > > > + { .parse_prop = parse_regulators, },
> > > > + { },
> > > > +};
> > > > +
> > > > +static bool of_link_property(struct device *dev, struct device_node *con_np,
> > > > + const char *prop)
> > > > +{
> > > > + struct device_node *phandle;
> > > > + struct supplier_bindings *s = bindings;
> > > > + unsigned int i = 0;
> > > > + bool done = true;
> > > > +
> > > > + while (!i && s->parse_prop) {
> > >
> > > Using 'i' is a little odd. Perhaps a 'matched' bool would be easier to read.
> >
> > That's how I wrote it first (locally) and then redid it this way
> > because the bool felt very superfluous. I don't think this is that
> > hard to understand.
>
> Alright...
I like the name "matched" over "found" that I had used locally. So, I
actually went ahead and did this.
-Saravana
> > > > + while ((phandle = s->parse_prop(con_np, prop, i))) {
> > > > + i++;
> > > > + if (of_link_to_phandle(dev, phandle))
> > > > + done = false;
> > >
> > > Just return here. No point in continuing as 'done' is never set back to true.
> >
> > Actually, there is a point for this. Say Device-C depends on suppliers
> > Device-S1 and Device-S2 and they are listed in DT in that order.
> >
> > Say, S1 gets populated after late_initcall_sync but S2 is probes way
> > before that. If I don't continue past a "failed linking" to S1 and
> > also link up to S2, then S2 will get a sync_state() callback before C
> > is probed. So I have to go through all possible suppliers and as many
> > as possible.
> >
> > Let me add a comment about this somewhere in the code (probably the
> > header that defines the add_links() ops).
>
> Okay, makes sense.
>
> Rob
^ permalink raw reply
* Re: [PATCH v6 1/1] sched/fair: Fix low cpu usage with high throttling by removing expiration of cpu-local slices
From: Phil Auld @ 2019-07-23 23:26 UTC (permalink / raw)
To: Dave Chiluk
Cc: Ben Segall, Peter Oskolkov, Peter Zijlstra, Ingo Molnar, cgroups,
Linux Kernel Mailing List, Brendan Gregg, Kyle Anderson,
Gabriel Munos, John Hammond, Cong Wang, Jonathan Corbet,
linux-doc
In-Reply-To: <CAC=E7cUUOfShgOWAhajVJsqxxMmyEPThd__xWqy2vdpF2xz4UA@mail.gmail.com>
On Tue, Jul 23, 2019 at 05:12:18PM -0500 Dave Chiluk wrote:
> Thanks for all the help and testing you provided. It's good to know
> these changes have passed at least some scheduler regression tests.
> If it comes to a v7 I'll add the Reviewed-by, otherwise I'll just let
> Peter add it.
>
Sounds good.
> Will you be handling the backport into the RHEL 8 kernels? I'll
> submit this to Ubuntu and linux-stable once it gets accepted.
>
Yep.
> Thanks again,
Thank you :)
Cheers,
Phil
>
>
> On Tue, Jul 23, 2019 at 12:13 PM Phil Auld <pauld@redhat.com> wrote:
> >
> > Hi Dave,
> >
> > On Tue, Jul 23, 2019 at 11:44:26AM -0500 Dave Chiluk wrote:
> > > It has been observed, that highly-threaded, non-cpu-bound applications
> > > running under cpu.cfs_quota_us constraints can hit a high percentage of
> > > periods throttled while simultaneously not consuming the allocated
> > > amount of quota. This use case is typical of user-interactive non-cpu
> > > bound applications, such as those running in kubernetes or mesos when
> > > run on multiple cpu cores.
> > >
> > > This has been root caused to cpu-local run queue being allocated per cpu
> > > bandwidth slices, and then not fully using that slice within the period.
> > > At which point the slice and quota expires. This expiration of unused
> > > slice results in applications not being able to utilize the quota for
> > > which they are allocated.
> > >
> > > The non-expiration of per-cpu slices was recently fixed by
> > > 'commit 512ac999d275 ("sched/fair: Fix bandwidth timer clock drift
> > > condition")'. Prior to that it appears that this had been broken since
> > > at least 'commit 51f2176d74ac ("sched/fair: Fix unlocked reads of some
> > > cfs_b->quota/period")' which was introduced in v3.16-rc1 in 2014. That
> > > added the following conditional which resulted in slices never being
> > > expired.
> > >
> > > if (cfs_rq->runtime_expires != cfs_b->runtime_expires) {
> > > /* extend local deadline, drift is bounded above by 2 ticks */
> > > cfs_rq->runtime_expires += TICK_NSEC;
> > >
> > > Because this was broken for nearly 5 years, and has recently been fixed
> > > and is now being noticed by many users running kubernetes
> > > (https://github.com/kubernetes/kubernetes/issues/67577) it is my opinion
> > > that the mechanisms around expiring runtime should be removed
> > > altogether.
> > >
> > > This allows quota already allocated to per-cpu run-queues to live longer
> > > than the period boundary. This allows threads on runqueues that do not
> > > use much CPU to continue to use their remaining slice over a longer
> > > period of time than cpu.cfs_period_us. However, this helps prevent the
> > > above condition of hitting throttling while also not fully utilizing
> > > your cpu quota.
> > >
> > > This theoretically allows a machine to use slightly more than its
> > > allotted quota in some periods. This overflow would be bounded by the
> > > remaining quota left on each per-cpu runqueueu. This is typically no
> > > more than min_cfs_rq_runtime=1ms per cpu. For CPU bound tasks this will
> > > change nothing, as they should theoretically fully utilize all of their
> > > quota in each period. For user-interactive tasks as described above this
> > > provides a much better user/application experience as their cpu
> > > utilization will more closely match the amount they requested when they
> > > hit throttling. This means that cpu limits no longer strictly apply per
> > > period for non-cpu bound applications, but that they are still accurate
> > > over longer timeframes.
> > >
> > > This greatly improves performance of high-thread-count, non-cpu bound
> > > applications with low cfs_quota_us allocation on high-core-count
> > > machines. In the case of an artificial testcase (10ms/100ms of quota on
> > > 80 CPU machine), this commit resulted in almost 30x performance
> > > improvement, while still maintaining correct cpu quota restrictions.
> > > That testcase is available at https://github.com/indeedeng/fibtest.
> > >
> > > Fixes: 512ac999d275 ("sched/fair: Fix bandwidth timer clock drift condition")
> > > Signed-off-by: Dave Chiluk <chiluk+linux@indeed.com>
> > > Reviewed-by: Ben Segall <bsegall@google.com>
> >
> > This still works for me. The documentation reads pretty well, too. Good job.
> >
> > Feel free to add my Acked-by: or Reviewed-by: Phil Auld <pauld@redhat.com>.
> >
> > I'll run it through some more tests when I have time. The code is the same
> > as the earlier one I tested from what I can see.
> >
> > Cheers,
> > Phil
> >
> > > ---
> > > Documentation/scheduler/sched-bwc.rst | 74 ++++++++++++++++++++++++++++-------
> > > kernel/sched/fair.c | 72 ++++------------------------------
> > > kernel/sched/sched.h | 4 --
> > > 3 files changed, 67 insertions(+), 83 deletions(-)
> > >
> > > diff --git a/Documentation/scheduler/sched-bwc.rst b/Documentation/scheduler/sched-bwc.rst
> > > index 3a90642..9801d6b 100644
> > > --- a/Documentation/scheduler/sched-bwc.rst
> > > +++ b/Documentation/scheduler/sched-bwc.rst
> > > @@ -9,15 +9,16 @@ CFS bandwidth control is a CONFIG_FAIR_GROUP_SCHED extension which allows the
> > > specification of the maximum CPU bandwidth available to a group or hierarchy.
> > >
> > > The bandwidth allowed for a group is specified using a quota and period. Within
> > > -each given "period" (microseconds), a group is allowed to consume only up to
> > > -"quota" microseconds of CPU time. When the CPU bandwidth consumption of a
> > > -group exceeds this limit (for that period), the tasks belonging to its
> > > -hierarchy will be throttled and are not allowed to run again until the next
> > > -period.
> > > -
> > > -A group's unused runtime is globally tracked, being refreshed with quota units
> > > -above at each period boundary. As threads consume this bandwidth it is
> > > -transferred to cpu-local "silos" on a demand basis. The amount transferred
> > > +each given "period" (microseconds), a task group is allocated up to "quota"
> > > +microseconds of CPU time. That quota is assigned to per-cpu run queues in
> > > +slices as threads in the cgroup become runnable. Once all quota has been
> > > +assigned any additional requests for quota will result in those threads being
> > > +throttled. Throttled threads will not be able to run again until the next
> > > +period when the quota is replenished.
> > > +
> > > +A group's unassigned quota is globally tracked, being refreshed back to
> > > +cfs_quota units at each period boundary. As threads consume this bandwidth it
> > > +is transferred to cpu-local "silos" on a demand basis. The amount transferred
> > > within each of these updates is tunable and described as the "slice".
> > >
> > > Management
> > > @@ -35,12 +36,12 @@ The default values are::
> > >
> > > A value of -1 for cpu.cfs_quota_us indicates that the group does not have any
> > > bandwidth restriction in place, such a group is described as an unconstrained
> > > -bandwidth group. This represents the traditional work-conserving behavior for
> > > +bandwidth group. This represents the traditional work-conserving behavior for
> > > CFS.
> > >
> > > Writing any (valid) positive value(s) will enact the specified bandwidth limit.
> > > -The minimum quota allowed for the quota or period is 1ms. There is also an
> > > -upper bound on the period length of 1s. Additional restrictions exist when
> > > +The minimum quota allowed for the quota or period is 1ms. There is also an
> > > +upper bound on the period length of 1s. Additional restrictions exist when
> > > bandwidth limits are used in a hierarchical fashion, these are explained in
> > > more detail below.
> > >
> > > @@ -53,8 +54,8 @@ unthrottled if it is in a constrained state.
> > > System wide settings
> > > --------------------
> > > For efficiency run-time is transferred between the global pool and CPU local
> > > -"silos" in a batch fashion. This greatly reduces global accounting pressure
> > > -on large systems. The amount transferred each time such an update is required
> > > +"silos" in a batch fashion. This greatly reduces global accounting pressure
> > > +on large systems. The amount transferred each time such an update is required
> > > is described as the "slice".
> > >
> > > This is tunable via procfs::
> > > @@ -97,6 +98,51 @@ There are two ways in which a group may become throttled:
> > > In case b) above, even though the child may have runtime remaining it will not
> > > be allowed to until the parent's runtime is refreshed.
> > >
> > > +CFS Bandwidth Quota Caveats
> > > +---------------------------
> > > +Once a slice is assigned to a cpu it does not expire. However all but 1ms of
> > > +the slice may be returned to the global pool if all threads on that cpu become
> > > +unrunnable. This is configured at compile time by the min_cfs_rq_runtime
> > > +variable. This is a performance tweak that helps prevent added contention on
> > > +the global lock.
> > > +
> > > +The fact that cpu-local slices do not expire results in some interesting corner
> > > +cases that should be understood.
> > > +
> > > +For cgroup cpu constrained applications that are cpu limited this is a
> > > +relatively moot point because they will naturally consume the entirety of their
> > > +quota as well as the entirety of each cpu-local slice in each period. As a
> > > +result it is expected that nr_periods roughly equal nr_throttled, and that
> > > +cpuacct.usage will increase roughly equal to cfs_quota_us in each period.
> > > +
> > > +For highly-threaded, non-cpu bound applications this non-expiration nuance
> > > +allows applications to briefly burst past their quota limits by the amount of
> > > +unused slice on each cpu that the task group is running on (typically at most
> > > +1ms per cpu or as defined by min_cfs_rq_runtime). This slight burst only
> > > +applies if quota had been assigned to a cpu and then not fully used or returned
> > > +in previous periods. This burst amount will not be transferred between cores.
> > > +As a result, this mechanism still strictly limits the task group to quota
> > > +average usage, albeit over a longer time window than a single period. This
> > > +also limits the burst ability to no more than 1ms per cpu. This provides
> > > +better more predictable user experience for highly threaded applications with
> > > +small quota limits on high core count machines. It also eliminates the
> > > +propensity to throttle these applications while simultanously using less than
> > > +quota amounts of cpu. Another way to say this, is that by allowing the unused
> > > +portion of a slice to remain valid across periods we have decreased the
> > > +possibility of wastefully expiring quota on cpu-local silos that don't need a
> > > +full slice's amount of cpu time.
> > > +
> > > +The interaction between cpu-bound and non-cpu-bound-interactive applications
> > > +should also be considered, especially when single core usage hits 100%. If you
> > > +gave each of these applications half of a cpu-core and they both got scheduled
> > > +on the same CPU it is theoretically possible that the non-cpu bound application
> > > +will use up to 1ms additional quota in some periods, thereby preventing the
> > > +cpu-bound application from fully using its quota by that same amount. In these
> > > +instances it will be up to the CFS algorithm (see sched-design-CFS.rst) to
> > > +decide which application is chosen to run, as they will both be runnable and
> > > +have remaining quota. This runtime discrepancy will be made up in the following
> > > +periods when the interactive application idles.
> > > +
> > > Examples
> > > --------
> > > 1. Limit a group to 1 CPU worth of runtime::
> > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > index 036be95..00b68f0 100644
> > > --- a/kernel/sched/fair.c
> > > +++ b/kernel/sched/fair.c
> > > @@ -4316,8 +4316,6 @@ void __refill_cfs_bandwidth_runtime(struct cfs_bandwidth *cfs_b)
> > >
> > > now = sched_clock_cpu(smp_processor_id());
> > > cfs_b->runtime = cfs_b->quota;
> > > - cfs_b->runtime_expires = now + ktime_to_ns(cfs_b->period);
> > > - cfs_b->expires_seq++;
> > > }
> > >
> > > static inline struct cfs_bandwidth *tg_cfs_bandwidth(struct task_group *tg)
> > > @@ -4339,8 +4337,7 @@ static int assign_cfs_rq_runtime(struct cfs_rq *cfs_rq)
> > > {
> > > struct task_group *tg = cfs_rq->tg;
> > > struct cfs_bandwidth *cfs_b = tg_cfs_bandwidth(tg);
> > > - u64 amount = 0, min_amount, expires;
> > > - int expires_seq;
> > > + u64 amount = 0, min_amount;
> > >
> > > /* note: this is a positive sum as runtime_remaining <= 0 */
> > > min_amount = sched_cfs_bandwidth_slice() - cfs_rq->runtime_remaining;
> > > @@ -4357,61 +4354,17 @@ static int assign_cfs_rq_runtime(struct cfs_rq *cfs_rq)
> > > cfs_b->idle = 0;
> > > }
> > > }
> > > - expires_seq = cfs_b->expires_seq;
> > > - expires = cfs_b->runtime_expires;
> > > raw_spin_unlock(&cfs_b->lock);
> > >
> > > cfs_rq->runtime_remaining += amount;
> > > - /*
> > > - * we may have advanced our local expiration to account for allowed
> > > - * spread between our sched_clock and the one on which runtime was
> > > - * issued.
> > > - */
> > > - if (cfs_rq->expires_seq != expires_seq) {
> > > - cfs_rq->expires_seq = expires_seq;
> > > - cfs_rq->runtime_expires = expires;
> > > - }
> > >
> > > return cfs_rq->runtime_remaining > 0;
> > > }
> > >
> > > -/*
> > > - * Note: This depends on the synchronization provided by sched_clock and the
> > > - * fact that rq->clock snapshots this value.
> > > - */
> > > -static void expire_cfs_rq_runtime(struct cfs_rq *cfs_rq)
> > > -{
> > > - struct cfs_bandwidth *cfs_b = tg_cfs_bandwidth(cfs_rq->tg);
> > > -
> > > - /* if the deadline is ahead of our clock, nothing to do */
> > > - if (likely((s64)(rq_clock(rq_of(cfs_rq)) - cfs_rq->runtime_expires) < 0))
> > > - return;
> > > -
> > > - if (cfs_rq->runtime_remaining < 0)
> > > - return;
> > > -
> > > - /*
> > > - * If the local deadline has passed we have to consider the
> > > - * possibility that our sched_clock is 'fast' and the global deadline
> > > - * has not truly expired.
> > > - *
> > > - * Fortunately we can check determine whether this the case by checking
> > > - * whether the global deadline(cfs_b->expires_seq) has advanced.
> > > - */
> > > - if (cfs_rq->expires_seq == cfs_b->expires_seq) {
> > > - /* extend local deadline, drift is bounded above by 2 ticks */
> > > - cfs_rq->runtime_expires += TICK_NSEC;
> > > - } else {
> > > - /* global deadline is ahead, expiration has passed */
> > > - cfs_rq->runtime_remaining = 0;
> > > - }
> > > -}
> > > -
> > > static void __account_cfs_rq_runtime(struct cfs_rq *cfs_rq, u64 delta_exec)
> > > {
> > > /* dock delta_exec before expiring quota (as it could span periods) */
> > > cfs_rq->runtime_remaining -= delta_exec;
> > > - expire_cfs_rq_runtime(cfs_rq);
> > >
> > > if (likely(cfs_rq->runtime_remaining > 0))
> > > return;
> > > @@ -4602,8 +4555,7 @@ void unthrottle_cfs_rq(struct cfs_rq *cfs_rq)
> > > resched_curr(rq);
> > > }
> > >
> > > -static u64 distribute_cfs_runtime(struct cfs_bandwidth *cfs_b,
> > > - u64 remaining, u64 expires)
> > > +static u64 distribute_cfs_runtime(struct cfs_bandwidth *cfs_b, u64 remaining)
> > > {
> > > struct cfs_rq *cfs_rq;
> > > u64 runtime;
> > > @@ -4625,7 +4577,6 @@ static u64 distribute_cfs_runtime(struct cfs_bandwidth *cfs_b,
> > > remaining -= runtime;
> > >
> > > cfs_rq->runtime_remaining += runtime;
> > > - cfs_rq->runtime_expires = expires;
> > >
> > > /* we check whether we're throttled above */
> > > if (cfs_rq->runtime_remaining > 0)
> > > @@ -4650,7 +4601,7 @@ static u64 distribute_cfs_runtime(struct cfs_bandwidth *cfs_b,
> > > */
> > > static int do_sched_cfs_period_timer(struct cfs_bandwidth *cfs_b, int overrun, unsigned long flags)
> > > {
> > > - u64 runtime, runtime_expires;
> > > + u64 runtime;
> > > int throttled;
> > >
> > > /* no need to continue the timer with no bandwidth constraint */
> > > @@ -4678,8 +4629,6 @@ static int do_sched_cfs_period_timer(struct cfs_bandwidth *cfs_b, int overrun, u
> > > /* account preceding periods in which throttling occurred */
> > > cfs_b->nr_throttled += overrun;
> > >
> > > - runtime_expires = cfs_b->runtime_expires;
> > > -
> > > /*
> > > * This check is repeated as we are holding onto the new bandwidth while
> > > * we unthrottle. This can potentially race with an unthrottled group
> > > @@ -4692,8 +4641,7 @@ static int do_sched_cfs_period_timer(struct cfs_bandwidth *cfs_b, int overrun, u
> > > cfs_b->distribute_running = 1;
> > > raw_spin_unlock_irqrestore(&cfs_b->lock, flags);
> > > /* we can't nest cfs_b->lock while distributing bandwidth */
> > > - runtime = distribute_cfs_runtime(cfs_b, runtime,
> > > - runtime_expires);
> > > + runtime = distribute_cfs_runtime(cfs_b, runtime);
> > > raw_spin_lock_irqsave(&cfs_b->lock, flags);
> > >
> > > cfs_b->distribute_running = 0;
> > > @@ -4775,8 +4723,7 @@ static void __return_cfs_rq_runtime(struct cfs_rq *cfs_rq)
> > > return;
> > >
> > > raw_spin_lock(&cfs_b->lock);
> > > - if (cfs_b->quota != RUNTIME_INF &&
> > > - cfs_rq->runtime_expires == cfs_b->runtime_expires) {
> > > + if (cfs_b->quota != RUNTIME_INF) {
> > > cfs_b->runtime += slack_runtime;
> > >
> > > /* we are under rq->lock, defer unthrottling using a timer */
> > > @@ -4809,7 +4756,6 @@ static void do_sched_cfs_slack_timer(struct cfs_bandwidth *cfs_b)
> > > {
> > > u64 runtime = 0, slice = sched_cfs_bandwidth_slice();
> > > unsigned long flags;
> > > - u64 expires;
> > >
> > > /* confirm we're still not at a refresh boundary */
> > > raw_spin_lock_irqsave(&cfs_b->lock, flags);
> > > @@ -4827,7 +4773,6 @@ static void do_sched_cfs_slack_timer(struct cfs_bandwidth *cfs_b)
> > > if (cfs_b->quota != RUNTIME_INF && cfs_b->runtime > slice)
> > > runtime = cfs_b->runtime;
> > >
> > > - expires = cfs_b->runtime_expires;
> > > if (runtime)
> > > cfs_b->distribute_running = 1;
> > >
> > > @@ -4836,11 +4781,10 @@ static void do_sched_cfs_slack_timer(struct cfs_bandwidth *cfs_b)
> > > if (!runtime)
> > > return;
> > >
> > > - runtime = distribute_cfs_runtime(cfs_b, runtime, expires);
> > > + runtime = distribute_cfs_runtime(cfs_b, runtime);
> > >
> > > raw_spin_lock_irqsave(&cfs_b->lock, flags);
> > > - if (expires == cfs_b->runtime_expires)
> > > - lsub_positive(&cfs_b->runtime, runtime);
> > > + lsub_positive(&cfs_b->runtime, runtime);
> > > cfs_b->distribute_running = 0;
> > > raw_spin_unlock_irqrestore(&cfs_b->lock, flags);
> > > }
> > > @@ -4997,8 +4941,6 @@ void start_cfs_bandwidth(struct cfs_bandwidth *cfs_b)
> > >
> > > cfs_b->period_active = 1;
> > > overrun = hrtimer_forward_now(&cfs_b->period_timer, cfs_b->period);
> > > - cfs_b->runtime_expires += (overrun + 1) * ktime_to_ns(cfs_b->period);
> > > - cfs_b->expires_seq++;
> > > hrtimer_start_expires(&cfs_b->period_timer, HRTIMER_MODE_ABS_PINNED);
> > > }
> > >
> > > diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
> > > index 802b1f3..28c16e9 100644
> > > --- a/kernel/sched/sched.h
> > > +++ b/kernel/sched/sched.h
> > > @@ -335,8 +335,6 @@ struct cfs_bandwidth {
> > > u64 quota;
> > > u64 runtime;
> > > s64 hierarchical_quota;
> > > - u64 runtime_expires;
> > > - int expires_seq;
> > >
> > > u8 idle;
> > > u8 period_active;
> > > @@ -556,8 +554,6 @@ struct cfs_rq {
> > >
> > > #ifdef CONFIG_CFS_BANDWIDTH
> > > int runtime_enabled;
> > > - int expires_seq;
> > > - u64 runtime_expires;
> > > s64 runtime_remaining;
> > >
> > > u64 throttled_clock;
> > > --
> > > 1.8.3.1
> > >
> >
> > --
--
^ permalink raw reply
* [PATCH v4 0/3] Casefolding in F2FS
From: Daniel Rosenberg @ 2019-07-23 23:05 UTC (permalink / raw)
To: Jaegeuk Kim, Chao Yu, Jonathan Corbet, linux-f2fs-devel
Cc: linux-kernel, linux-doc, linux-fsdevel, linux-api, kernel-team,
Daniel Rosenberg
These patches are largely based on the casefolding patches for ext4
v4: Added FS_CASEFOLD_FL flag, added documentation that escaped the last
format-patch, moved setting dentry ops to f2fs_setup_casefold
v3: Addressed feedback, apart from F2FS_CASEFOLD_FL/FS_CASEFOLD_FL
Added sysfs file "encoding" to see the encoding set on a filesystem
v2: Rebased patches again master, changed f2fs_msg to f2fs_info/f2fs_err
Daniel Rosenberg (3):
fs: Reserve flag for casefolding
f2fs: include charset encoding information in the superblock
f2fs: Support case-insensitive file name lookups
Documentation/ABI/testing/sysfs-fs-f2fs | 7 ++
Documentation/filesystems/f2fs.txt | 3 +
fs/f2fs/dir.c | 126 ++++++++++++++++++++++--
fs/f2fs/f2fs.h | 21 +++-
fs/f2fs/file.c | 16 ++-
fs/f2fs/hash.c | 35 ++++++-
fs/f2fs/inline.c | 4 +-
fs/f2fs/inode.c | 4 +-
fs/f2fs/namei.c | 21 ++++
fs/f2fs/super.c | 96 ++++++++++++++++++
fs/f2fs/sysfs.c | 23 +++++
include/linux/f2fs_fs.h | 9 +-
include/uapi/linux/fs.h | 1 +
tools/include/uapi/linux/fs.h | 1 +
14 files changed, 347 insertions(+), 20 deletions(-)
--
2.22.0.657.g960e92d24f-goog
^ permalink raw reply
* [PATCH v4 3/3] f2fs: Support case-insensitive file name lookups
From: Daniel Rosenberg @ 2019-07-23 23:05 UTC (permalink / raw)
To: Jaegeuk Kim, Chao Yu, Jonathan Corbet, linux-f2fs-devel
Cc: linux-kernel, linux-doc, linux-fsdevel, linux-api, kernel-team,
Daniel Rosenberg
In-Reply-To: <20190723230529.251659-1-drosen@google.com>
Modeled after commit b886ee3e778e ("ext4: Support case-insensitive file
name lookups")
"""
This patch implements the actual support for case-insensitive file name
lookups in f2fs, based on the feature bit and the encoding stored in the
superblock.
A filesystem that has the casefold feature set is able to configure
directories with the +F (F2FS_CASEFOLD_FL) attribute, enabling lookups
to succeed in that directory in a case-insensitive fashion, i.e: match
a directory entry even if the name used by userspace is not a byte per
byte match with the disk name, but is an equivalent case-insensitive
version of the Unicode string. This operation is called a
case-insensitive file name lookup.
The feature is configured as an inode attribute applied to directories
and inherited by its children. This attribute can only be enabled on
empty directories for filesystems that support the encoding feature,
thus preventing collision of file names that only differ by case.
* dcache handling:
For a +F directory, F2Fs only stores the first equivalent name dentry
used in the dcache. This is done to prevent unintentional duplication of
dentries in the dcache, while also allowing the VFS code to quickly find
the right entry in the cache despite which equivalent string was used in
a previous lookup, without having to resort to ->lookup().
d_hash() of casefolded directories is implemented as the hash of the
casefolded string, such that we always have a well-known bucket for all
the equivalencies of the same string. d_compare() uses the
utf8_strncasecmp() infrastructure, which handles the comparison of
equivalent, same case, names as well.
For now, negative lookups are not inserted in the dcache, since they
would need to be invalidated anyway, because we can't trust missing file
dentries. This is bad for performance but requires some leveraging of
the vfs layer to fix. We can live without that for now, and so does
everyone else.
* on-disk data:
Despite using a specific version of the name as the internal
representation within the dcache, the name stored and fetched from the
disk is a byte-per-byte match with what the user requested, making this
implementation 'name-preserving'. i.e. no actual information is lost
when writing to storage.
DX is supported by modifying the hashes used in +F directories to make
them case/encoding-aware. The new disk hashes are calculated as the
hash of the full casefolded string, instead of the string directly.
This allows us to efficiently search for file names in the htree without
requiring the user to provide an exact name.
* Dealing with invalid sequences:
By default, when a invalid UTF-8 sequence is identified, ext4 will treat
it as an opaque byte sequence, ignoring the encoding and reverting to
the old behavior for that unique file. This means that case-insensitive
file name lookup will not work only for that file. An optional bit can
be set in the superblock telling the filesystem code and userspace tools
to enforce the encoding. When that optional bit is set, any attempt to
create a file name using an invalid UTF-8 sequence will fail and return
an error to userspace.
* Normalization algorithm:
The UTF-8 algorithms used to compare strings in f2fs is implemented
in fs/unicode, and is based on a previous version developed by
SGI. It implements the Canonical decomposition (NFD) algorithm
described by the Unicode specification 12.1, or higher, combined with
the elimination of ignorable code points (NFDi) and full
case-folding (CF) as documented in fs/unicode/utf8_norm.c.
NFD seems to be the best normalization method for F2FS because:
- It has a lower cost than NFC/NFKC (which requires
decomposing to NFD as an intermediary step)
- It doesn't eliminate important semantic meaning like
compatibility decompositions.
Although:
- This implementation is not completely linguistic accurate, because
different languages have conflicting rules, which would require the
specialization of the filesystem to a given locale, which brings all
sorts of problems for removable media and for users who use more than
one language.
"""
Signed-off-by: Daniel Rosenberg <drosen@google.com>
---
fs/f2fs/dir.c | 126 +++++++++++++++++++++++++++++++++++++++++++----
fs/f2fs/f2fs.h | 15 ++++--
fs/f2fs/file.c | 16 +++++-
fs/f2fs/hash.c | 35 ++++++++++++-
fs/f2fs/inline.c | 4 +-
fs/f2fs/inode.c | 4 +-
fs/f2fs/namei.c | 21 ++++++++
fs/f2fs/super.c | 1 +
8 files changed, 203 insertions(+), 19 deletions(-)
diff --git a/fs/f2fs/dir.c b/fs/f2fs/dir.c
index 85a1528f319f2..2913483473f30 100644
--- a/fs/f2fs/dir.c
+++ b/fs/f2fs/dir.c
@@ -8,6 +8,7 @@
#include <linux/fs.h>
#include <linux/f2fs_fs.h>
#include <linux/sched/signal.h>
+#include <linux/unicode.h>
#include "f2fs.h"
#include "node.h"
#include "acl.h"
@@ -81,7 +82,8 @@ static unsigned long dir_block_index(unsigned int level,
return bidx;
}
-static struct f2fs_dir_entry *find_in_block(struct page *dentry_page,
+static struct f2fs_dir_entry *find_in_block(struct inode *dir,
+ struct page *dentry_page,
struct fscrypt_name *fname,
f2fs_hash_t namehash,
int *max_slots,
@@ -93,7 +95,7 @@ static struct f2fs_dir_entry *find_in_block(struct page *dentry_page,
dentry_blk = (struct f2fs_dentry_block *)page_address(dentry_page);
- make_dentry_ptr_block(NULL, &d, dentry_blk);
+ make_dentry_ptr_block(dir, &d, dentry_blk);
de = f2fs_find_target_dentry(fname, namehash, max_slots, &d);
if (de)
*res_page = dentry_page;
@@ -101,6 +103,39 @@ static struct f2fs_dir_entry *find_in_block(struct page *dentry_page,
return de;
}
+#ifdef CONFIG_UNICODE
+/*
+ * Test whether a case-insensitive directory entry matches the filename
+ * being searched for.
+ *
+ * Returns: 0 if the directory entry matches, more than 0 if it
+ * doesn't match or less than zero on error.
+ */
+int f2fs_ci_compare(const struct inode *parent, const struct qstr *name,
+ const struct qstr *entry)
+{
+ const struct f2fs_sb_info *sbi = F2FS_SB(parent->i_sb);
+ const struct unicode_map *um = sbi->s_encoding;
+ int ret;
+
+ ret = utf8_strncasecmp(um, name, entry);
+ if (ret < 0) {
+ /* Handle invalid character sequence as either an error
+ * or as an opaque byte sequence.
+ */
+ if (f2fs_has_strict_mode(sbi))
+ return -EINVAL;
+
+ if (name->len != entry->len)
+ return 1;
+
+ return !!memcmp(name->name, entry->name, name->len);
+ }
+
+ return ret;
+}
+#endif
+
struct f2fs_dir_entry *f2fs_find_target_dentry(struct fscrypt_name *fname,
f2fs_hash_t namehash, int *max_slots,
struct f2fs_dentry_ptr *d)
@@ -108,6 +143,9 @@ struct f2fs_dir_entry *f2fs_find_target_dentry(struct fscrypt_name *fname,
struct f2fs_dir_entry *de;
unsigned long bit_pos = 0;
int max_len = 0;
+#ifdef CONFIG_UNICODE
+ struct qstr entry;
+#endif
if (max_slots)
*max_slots = 0;
@@ -119,16 +157,28 @@ struct f2fs_dir_entry *f2fs_find_target_dentry(struct fscrypt_name *fname,
}
de = &d->dentry[bit_pos];
+#ifdef CONFIG_UNICODE
+ entry.name = d->filename[bit_pos];
+ entry.len = de->name_len;
+#endif
if (unlikely(!de->name_len)) {
bit_pos++;
continue;
}
+ if (de->hash_code == namehash) {
+#ifdef CONFIG_UNICODE
+ if (F2FS_SB(d->inode->i_sb)->s_encoding &&
+ IS_CASEFOLDED(d->inode) &&
+ !f2fs_ci_compare(d->inode,
+ fname->usr_fname, &entry))
+ goto found;
- if (de->hash_code == namehash &&
- fscrypt_match_name(fname, d->filename[bit_pos],
- le16_to_cpu(de->name_len)))
- goto found;
+#endif
+ if (fscrypt_match_name(fname, d->filename[bit_pos],
+ le16_to_cpu(de->name_len)))
+ goto found;
+ }
if (max_slots && max_len > *max_slots)
*max_slots = max_len;
@@ -157,7 +207,7 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
struct f2fs_dir_entry *de = NULL;
bool room = false;
int max_slots;
- f2fs_hash_t namehash = f2fs_dentry_hash(&name, fname);
+ f2fs_hash_t namehash = f2fs_dentry_hash(dir, &name, fname);
nbucket = dir_buckets(level, F2FS_I(dir)->i_dir_level);
nblock = bucket_blocks(level);
@@ -179,8 +229,8 @@ static struct f2fs_dir_entry *find_in_level(struct inode *dir,
}
}
- de = find_in_block(dentry_page, fname, namehash, &max_slots,
- res_page);
+ de = find_in_block(dir, dentry_page, fname, namehash,
+ &max_slots, res_page);
if (de)
break;
@@ -250,6 +300,14 @@ struct f2fs_dir_entry *f2fs_find_entry(struct inode *dir,
struct fscrypt_name fname;
int err;
+#ifdef CONFIG_UNICODE
+ if (f2fs_has_strict_mode(F2FS_I_SB(dir)) && IS_CASEFOLDED(dir) &&
+ utf8_validate(F2FS_I_SB(dir)->s_encoding, child)) {
+ *res_page = ERR_PTR(-EINVAL);
+ return NULL;
+ }
+#endif
+
err = fscrypt_setup_filename(dir, child, 1, &fname);
if (err) {
if (err == -ENOENT)
@@ -504,7 +562,7 @@ int f2fs_add_regular_entry(struct inode *dir, const struct qstr *new_name,
level = 0;
slots = GET_DENTRY_SLOTS(new_name->len);
- dentry_hash = f2fs_dentry_hash(new_name, NULL);
+ dentry_hash = f2fs_dentry_hash(dir, new_name, NULL);
current_depth = F2FS_I(dir)->i_current_depth;
if (F2FS_I(dir)->chash == dentry_hash) {
@@ -943,3 +1001,51 @@ const struct file_operations f2fs_dir_operations = {
.compat_ioctl = f2fs_compat_ioctl,
#endif
};
+
+#ifdef CONFIG_UNICODE
+static int f2fs_d_compare(const struct dentry *dentry, unsigned int len,
+ const char *str, const struct qstr *name)
+{
+ struct qstr qstr = {.name = str, .len = len };
+
+ if (!IS_CASEFOLDED(dentry->d_parent->d_inode)) {
+ if (len != name->len)
+ return -1;
+ return memcmp(str, name, len);
+ }
+
+ return f2fs_ci_compare(dentry->d_parent->d_inode, name, &qstr);
+}
+
+static int f2fs_d_hash(const struct dentry *dentry, struct qstr *str)
+{
+ struct f2fs_sb_info *sbi = F2FS_SB(dentry->d_sb);
+ const struct unicode_map *um = sbi->s_encoding;
+ unsigned char *norm;
+ int len, ret = 0;
+
+ if (!IS_CASEFOLDED(dentry->d_inode))
+ return 0;
+
+ norm = f2fs_kmalloc(sbi, PATH_MAX, GFP_ATOMIC);
+ if (!norm)
+ return -ENOMEM;
+
+ len = utf8_casefold(um, str, norm, PATH_MAX);
+ if (len < 0) {
+ if (f2fs_has_strict_mode(sbi))
+ ret = -EINVAL;
+ goto out;
+ }
+ str->hash = full_name_hash(dentry, norm, len);
+out:
+ kvfree(norm);
+ return ret;
+}
+
+const struct dentry_operations f2fs_dentry_ops = {
+ .d_hash = f2fs_d_hash,
+ .d_compare = f2fs_d_compare,
+};
+#endif
+
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index c6c7904572d0d..31fd2a268ba14 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -2364,10 +2364,12 @@ static inline void f2fs_change_bit(unsigned int nr, char *addr)
#define F2FS_INDEX_FL 0x00001000 /* hash-indexed directory */
#define F2FS_DIRSYNC_FL 0x00010000 /* dirsync behaviour (directories only) */
#define F2FS_PROJINHERIT_FL 0x20000000 /* Create with parents projid */
+#define F2FS_CASEFOLD_FL 0x40000000 /* Casefolded file */
/* Flags that should be inherited by new inodes from their parent. */
#define F2FS_FL_INHERITED (F2FS_SYNC_FL | F2FS_NODUMP_FL | F2FS_NOATIME_FL | \
- F2FS_DIRSYNC_FL | F2FS_PROJINHERIT_FL)
+ F2FS_DIRSYNC_FL | F2FS_PROJINHERIT_FL | \
+ F2FS_CASEFOLD_FL)
/* Flags that are appropriate for regular files (all but dir-specific ones). */
#define F2FS_REG_FLMASK (~(F2FS_DIRSYNC_FL | F2FS_PROJINHERIT_FL))
@@ -2930,6 +2932,10 @@ int f2fs_update_extension_list(struct f2fs_sb_info *sbi, const char *name,
bool hot, bool set);
struct dentry *f2fs_get_parent(struct dentry *child);
+extern int f2fs_ci_compare(const struct inode *parent,
+ const struct qstr *name,
+ const struct qstr *entry);
+
/*
* dir.c
*/
@@ -2993,8 +2999,8 @@ int f2fs_sanity_check_ckpt(struct f2fs_sb_info *sbi);
/*
* hash.c
*/
-f2fs_hash_t f2fs_dentry_hash(const struct qstr *name_info,
- struct fscrypt_name *fname);
+f2fs_hash_t f2fs_dentry_hash(const struct inode *dir,
+ const struct qstr *name_info, struct fscrypt_name *fname);
/*
* node.c
@@ -3437,6 +3443,9 @@ static inline void f2fs_destroy_root_stats(void) { }
#endif
extern const struct file_operations f2fs_dir_operations;
+#ifdef CONFIG_UNICODE
+extern const struct dentry_operations f2fs_dentry_ops;
+#endif
extern const struct file_operations f2fs_file_operations;
extern const struct inode_operations f2fs_file_inode_operations;
extern const struct address_space_operations f2fs_dblock_aops;
diff --git a/fs/f2fs/file.c b/fs/f2fs/file.c
index f8d46df8fa9ee..7ab2e6fea5b82 100644
--- a/fs/f2fs/file.c
+++ b/fs/f2fs/file.c
@@ -1660,7 +1660,16 @@ static int f2fs_setflags_common(struct inode *inode, u32 iflags, u32 mask)
return -EPERM;
oldflags = fi->i_flags;
+ if ((iflags ^ oldflags) & F2FS_CASEFOLD_FL) {
+ if (!f2fs_sb_has_casefold(F2FS_I_SB(inode)))
+ return -EOPNOTSUPP;
+
+ if (!S_ISDIR(inode->i_mode))
+ return -ENOTDIR;
+ if (!f2fs_empty_dir(inode))
+ return -ENOTEMPTY;
+ }
if ((iflags ^ oldflags) & (F2FS_APPEND_FL | F2FS_IMMUTABLE_FL))
if (!capable(CAP_LINUX_IMMUTABLE))
return -EPERM;
@@ -1699,6 +1708,7 @@ static const struct {
{ F2FS_INDEX_FL, FS_INDEX_FL },
{ F2FS_DIRSYNC_FL, FS_DIRSYNC_FL },
{ F2FS_PROJINHERIT_FL, FS_PROJINHERIT_FL },
+ { F2FS_CASEFOLD_FL, FS_CASEFOLD_FL },
};
#define F2FS_GETTABLE_FS_FL ( \
@@ -1712,7 +1722,8 @@ static const struct {
FS_PROJINHERIT_FL | \
FS_ENCRYPT_FL | \
FS_INLINE_DATA_FL | \
- FS_NOCOW_FL)
+ FS_NOCOW_FL | \
+ FS_CASEFOLD_FL)
#define F2FS_SETTABLE_FS_FL ( \
FS_SYNC_FL | \
@@ -1721,7 +1732,8 @@ static const struct {
FS_NODUMP_FL | \
FS_NOATIME_FL | \
FS_DIRSYNC_FL | \
- FS_PROJINHERIT_FL)
+ FS_PROJINHERIT_FL | \
+ FS_CASEFOLD_FL)
/* Convert f2fs on-disk i_flags to FS_IOC_{GET,SET}FLAGS flags */
static inline u32 f2fs_iflags_to_fsflags(u32 iflags)
diff --git a/fs/f2fs/hash.c b/fs/f2fs/hash.c
index cc82f142f811f..99e79934f5088 100644
--- a/fs/f2fs/hash.c
+++ b/fs/f2fs/hash.c
@@ -14,6 +14,7 @@
#include <linux/f2fs_fs.h>
#include <linux/cryptohash.h>
#include <linux/pagemap.h>
+#include <linux/unicode.h>
#include "f2fs.h"
@@ -67,7 +68,7 @@ static void str2hashbuf(const unsigned char *msg, size_t len,
*buf++ = pad;
}
-f2fs_hash_t f2fs_dentry_hash(const struct qstr *name_info,
+static f2fs_hash_t __f2fs_dentry_hash(const struct qstr *name_info,
struct fscrypt_name *fname)
{
__u32 hash;
@@ -103,3 +104,35 @@ f2fs_hash_t f2fs_dentry_hash(const struct qstr *name_info,
f2fs_hash = cpu_to_le32(hash & ~F2FS_HASH_COL_BIT);
return f2fs_hash;
}
+
+f2fs_hash_t f2fs_dentry_hash(const struct inode *dir,
+ const struct qstr *name_info, struct fscrypt_name *fname)
+{
+#ifdef CONFIG_UNICODE
+ struct f2fs_sb_info *sbi = F2FS_SB(dir->i_sb);
+ const struct unicode_map *um = sbi->s_encoding;
+ int r, dlen;
+ unsigned char *buff;
+ struct qstr *folded;
+
+ if (name_info->len && IS_CASEFOLDED(dir)) {
+ buff = f2fs_kzalloc(sbi, sizeof(char) * PATH_MAX, GFP_KERNEL);
+ if (!buff)
+ return -ENOMEM;
+
+ dlen = utf8_casefold(um, name_info, buff, PATH_MAX);
+ if (dlen < 0) {
+ kvfree(buff);
+ goto opaque_seq;
+ }
+ folded->name = buff;
+ folded->len = dlen;
+ r = __f2fs_dentry_hash(folded, fname);
+
+ kvfree(buff);
+ return r;
+ }
+opaque_seq:
+#endif
+ return __f2fs_dentry_hash(name_info, fname);
+}
diff --git a/fs/f2fs/inline.c b/fs/f2fs/inline.c
index 3613efca8c00c..354f71cf9e6ba 100644
--- a/fs/f2fs/inline.c
+++ b/fs/f2fs/inline.c
@@ -320,7 +320,7 @@ struct f2fs_dir_entry *f2fs_find_in_inline_dir(struct inode *dir,
return NULL;
}
- namehash = f2fs_dentry_hash(&name, fname);
+ namehash = f2fs_dentry_hash(dir, &name, fname);
inline_dentry = inline_data_addr(dir, ipage);
@@ -580,7 +580,7 @@ int f2fs_add_inline_entry(struct inode *dir, const struct qstr *new_name,
f2fs_wait_on_page_writeback(ipage, NODE, true, true);
- name_hash = f2fs_dentry_hash(new_name, NULL);
+ name_hash = f2fs_dentry_hash(dir, new_name, NULL);
f2fs_update_dentry(ino, mode, &d, new_name, name_hash, bit_pos);
set_page_dirty(ipage);
diff --git a/fs/f2fs/inode.c b/fs/f2fs/inode.c
index a33d7a849b2df..9a1f0d6616577 100644
--- a/fs/f2fs/inode.c
+++ b/fs/f2fs/inode.c
@@ -46,9 +46,11 @@ void f2fs_set_inode_flags(struct inode *inode)
new_fl |= S_DIRSYNC;
if (file_is_encrypt(inode))
new_fl |= S_ENCRYPTED;
+ if (flags & F2FS_CASEFOLD_FL)
+ new_fl |= S_CASEFOLD;
inode_set_flags(inode, new_fl,
S_SYNC|S_APPEND|S_IMMUTABLE|S_NOATIME|S_DIRSYNC|
- S_ENCRYPTED);
+ S_ENCRYPTED|S_CASEFOLD);
}
static void __get_inode_rdev(struct inode *inode, struct f2fs_inode *ri)
diff --git a/fs/f2fs/namei.c b/fs/f2fs/namei.c
index c5b99042e6f2b..727de2f8620f2 100644
--- a/fs/f2fs/namei.c
+++ b/fs/f2fs/namei.c
@@ -489,6 +489,17 @@ static struct dentry *f2fs_lookup(struct inode *dir, struct dentry *dentry,
goto out_iput;
}
out_splice:
+#ifdef CONFIG_UNICODE
+ if (!inode && IS_CASEFOLDED(dir)) {
+ /* Eventually we want to call d_add_ci(dentry, NULL)
+ * for negative dentries in the encoding case as
+ * well. For now, prevent the negative dentry
+ * from being cached.
+ */
+ trace_f2fs_lookup_end(dir, dentry, ino, err);
+ return NULL;
+ }
+#endif
new = d_splice_alias(inode, dentry);
err = PTR_ERR_OR_ZERO(new);
trace_f2fs_lookup_end(dir, dentry, ino, err);
@@ -537,6 +548,16 @@ static int f2fs_unlink(struct inode *dir, struct dentry *dentry)
goto fail;
}
f2fs_delete_entry(de, page, dir, inode);
+#ifdef CONFIG_UNICODE
+ /* VFS negative dentries are incompatible with Encoding and
+ * Case-insensitiveness. Eventually we'll want avoid
+ * invalidating the dentries here, alongside with returning the
+ * negative dentries at f2fs_lookup(), when it is better
+ * supported by the VFS for the CI case.
+ */
+ if (IS_CASEFOLDED(dir))
+ d_invalidate(dentry);
+#endif
f2fs_unlock_op(sbi);
if (IS_DIRSYNC(dir))
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 068db78a1e03e..20bf5fc3c5333 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -3108,6 +3108,7 @@ static int f2fs_setup_casefold(struct f2fs_sb_info *sbi)
sbi->s_encoding = encoding;
sbi->s_encoding_flags = encoding_flags;
+ sbi->sb->s_d_op = &f2fs_dentry_ops;
}
#else
if (f2fs_sb_has_casefold(sbi)) {
--
2.22.0.657.g960e92d24f-goog
^ permalink raw reply related
* [PATCH v4 2/3] f2fs: include charset encoding information in the superblock
From: Daniel Rosenberg @ 2019-07-23 23:05 UTC (permalink / raw)
To: Jaegeuk Kim, Chao Yu, Jonathan Corbet, linux-f2fs-devel
Cc: linux-kernel, linux-doc, linux-fsdevel, linux-api, kernel-team,
Daniel Rosenberg
In-Reply-To: <20190723230529.251659-1-drosen@google.com>
Add charset encoding to f2fs to support casefolding. It is modeled after
the same feature introduced in commit c83ad55eaa91 ("ext4: include charset
encoding information in the superblock")
Currently this is not compatible with encryption, similar to the current
ext4 imlpementation. This will change in the future.
From the ext4 patch:
"""
The s_encoding field stores a magic number indicating the encoding
format and version used globally by file and directory names in the
filesystem. The s_encoding_flags defines policies for using the charset
encoding, like how to handle invalid sequences. The magic number is
mapped to the exact charset table, but the mapping is specific to ext4.
Since we don't have any commitment to support old encodings, the only
encoding I am supporting right now is utf8-12.1.0.
The current implementation prevents the user from enabling encoding and
per-directory encryption on the same filesystem at the same time. The
incompatibility between these features lies in how we do efficient
directory searches when we cannot be sure the encryption of the user
provided fname will match the actual hash stored in the disk without
decrypting every directory entry, because of normalization cases. My
quickest solution is to simply block the concurrent use of these
features for now, and enable it later, once we have a better solution.
"""
Signed-off-by: Daniel Rosenberg <drosen@google.com>
---
Documentation/ABI/testing/sysfs-fs-f2fs | 7 ++
Documentation/filesystems/f2fs.txt | 3 +
fs/f2fs/f2fs.h | 6 ++
fs/f2fs/super.c | 95 +++++++++++++++++++++++++
fs/f2fs/sysfs.c | 23 ++++++
include/linux/f2fs_fs.h | 9 ++-
6 files changed, 142 insertions(+), 1 deletion(-)
diff --git a/Documentation/ABI/testing/sysfs-fs-f2fs b/Documentation/ABI/testing/sysfs-fs-f2fs
index dca326e0ee3e1..7ab2b1b5e255f 100644
--- a/Documentation/ABI/testing/sysfs-fs-f2fs
+++ b/Documentation/ABI/testing/sysfs-fs-f2fs
@@ -251,3 +251,10 @@ Description:
If checkpoint=disable, it displays the number of blocks that are unusable.
If checkpoint=enable it displays the enumber of blocks that would be unusable
if checkpoint=disable were to be set.
+
+What: /sys/fs/f2fs/<disk>/encoding
+Date July 2019
+Contact: "Daniel Rosenberg" <drosen@google.com>
+Description:
+ Displays name and version of the encoding set for the filesystem.
+ If no encoding is set, displays (none)
diff --git a/Documentation/filesystems/f2fs.txt b/Documentation/filesystems/f2fs.txt
index 496fa28b24925..5fa38ab373cab 100644
--- a/Documentation/filesystems/f2fs.txt
+++ b/Documentation/filesystems/f2fs.txt
@@ -413,6 +413,9 @@ Files in /sys/fs/f2fs/<devname>
that would be unusable if checkpoint=disable were
to be set.
+encoding This shows the encoding used for casefolding.
+ If casefolding is not enabled, returns (none)
+
================================================================================
USAGE
================================================================================
diff --git a/fs/f2fs/f2fs.h b/fs/f2fs/f2fs.h
index 17382da7f0bd9..c6c7904572d0d 100644
--- a/fs/f2fs/f2fs.h
+++ b/fs/f2fs/f2fs.h
@@ -153,6 +153,7 @@ struct f2fs_mount_info {
#define F2FS_FEATURE_LOST_FOUND 0x0200
#define F2FS_FEATURE_VERITY 0x0400 /* reserved */
#define F2FS_FEATURE_SB_CHKSUM 0x0800
+#define F2FS_FEATURE_CASEFOLD 0x1000
#define __F2FS_HAS_FEATURE(raw_super, mask) \
((raw_super->feature & cpu_to_le32(mask)) != 0)
@@ -1169,6 +1170,10 @@ struct f2fs_sb_info {
int valid_super_block; /* valid super block no */
unsigned long s_flag; /* flags for sbi */
struct mutex writepages; /* mutex for writepages() */
+#ifdef CONFIG_UNICODE
+ struct unicode_map *s_encoding;
+ __u16 s_encoding_flags;
+#endif
#ifdef CONFIG_BLK_DEV_ZONED
unsigned int blocks_per_blkz; /* F2FS blocks per zone */
@@ -3562,6 +3567,7 @@ F2FS_FEATURE_FUNCS(quota_ino, QUOTA_INO);
F2FS_FEATURE_FUNCS(inode_crtime, INODE_CRTIME);
F2FS_FEATURE_FUNCS(lost_found, LOST_FOUND);
F2FS_FEATURE_FUNCS(sb_chksum, SB_CHKSUM);
+F2FS_FEATURE_FUNCS(casefold, CASEFOLD);
#ifdef CONFIG_BLK_DEV_ZONED
static inline bool f2fs_blkz_is_seq(struct f2fs_sb_info *sbi, int devi,
diff --git a/fs/f2fs/super.c b/fs/f2fs/super.c
index 6de6cda440315..068db78a1e03e 100644
--- a/fs/f2fs/super.c
+++ b/fs/f2fs/super.c
@@ -23,6 +23,7 @@
#include <linux/f2fs_fs.h>
#include <linux/sysfs.h>
#include <linux/quota.h>
+#include <linux/unicode.h>
#include "f2fs.h"
#include "node.h"
@@ -222,6 +223,36 @@ void f2fs_printk(struct f2fs_sb_info *sbi, const char *fmt, ...)
va_end(args);
}
+#ifdef CONFIG_UNICODE
+static const struct f2fs_sb_encodings {
+ __u16 magic;
+ char *name;
+ char *version;
+} f2fs_sb_encoding_map[] = {
+ {F2FS_ENC_UTF8_12_1, "utf8", "12.1.0"},
+};
+
+static int f2fs_sb_read_encoding(const struct f2fs_super_block *sb,
+ const struct f2fs_sb_encodings **encoding,
+ __u16 *flags)
+{
+ __u16 magic = le16_to_cpu(sb->s_encoding);
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(f2fs_sb_encoding_map); i++)
+ if (magic == f2fs_sb_encoding_map[i].magic)
+ break;
+
+ if (i >= ARRAY_SIZE(f2fs_sb_encoding_map))
+ return -EINVAL;
+
+ *encoding = &f2fs_sb_encoding_map[i];
+ *flags = le16_to_cpu(sb->s_encoding_flags);
+
+ return 0;
+}
+#endif
+
static inline void limit_reserve_root(struct f2fs_sb_info *sbi)
{
block_t limit = min((sbi->user_block_count << 1) / 1000,
@@ -798,6 +829,13 @@ static int parse_options(struct super_block *sb, char *options)
return -EINVAL;
}
#endif
+#ifndef CONFIG_UNICODE
+ if (f2fs_sb_has_casefold(sbi)) {
+ f2fs_err(sbi,
+ "Filesystem with casefold feature cannot be mounted without CONFIG_UNICODE");
+ return -EINVAL;
+ }
+#endif
if (F2FS_IO_SIZE_BITS(sbi) && !test_opt(sbi, LFS)) {
f2fs_err(sbi, "Should set mode=lfs with %uKB-sized IO",
@@ -1089,6 +1127,9 @@ static void f2fs_put_super(struct super_block *sb)
destroy_percpu_info(sbi);
for (i = 0; i < NR_PAGE_TYPE; i++)
kvfree(sbi->write_io[i]);
+#ifdef CONFIG_UNICODE
+ utf8_unload(sbi->s_encoding);
+#endif
kvfree(sbi);
}
@@ -3031,6 +3072,52 @@ static int f2fs_scan_devices(struct f2fs_sb_info *sbi)
return 0;
}
+static int f2fs_setup_casefold(struct f2fs_sb_info *sbi)
+{
+#ifdef CONFIG_UNICODE
+ if (f2fs_sb_has_casefold(sbi) && !sbi->s_encoding) {
+ const struct f2fs_sb_encodings *encoding_info;
+ struct unicode_map *encoding;
+ __u16 encoding_flags;
+
+ if (f2fs_sb_has_encrypt(sbi)) {
+ f2fs_err(sbi,
+ "Can't mount with encoding and encryption");
+ return -EINVAL;
+ }
+
+ if (f2fs_sb_read_encoding(sbi->raw_super, &encoding_info,
+ &encoding_flags)) {
+ f2fs_err(sbi,
+ "Encoding requested by superblock is unknown");
+ return -EINVAL;
+ }
+
+ encoding = utf8_load(encoding_info->version);
+ if (IS_ERR(encoding)) {
+ f2fs_err(sbi,
+ "can't mount with superblock charset: %s-%s "
+ "not supported by the kernel. flags: 0x%x.",
+ encoding_info->name, encoding_info->version,
+ encoding_flags);
+ return PTR_ERR(encoding);
+ }
+ f2fs_info(sbi, "Using encoding defined by superblock: "
+ "%s-%s with flags 0x%hx", encoding_info->name,
+ encoding_info->version?:"\b", encoding_flags);
+
+ sbi->s_encoding = encoding;
+ sbi->s_encoding_flags = encoding_flags;
+ }
+#else
+ if (f2fs_sb_has_casefold(sbi)) {
+ f2fs_err(sbi, "Filesystem with casefold feature cannot be mounted without CONFIG_UNICODE");
+ return -EINVAL;
+ }
+#endif
+ return 0;
+}
+
static void f2fs_tuning_parameters(struct f2fs_sb_info *sbi)
{
struct f2fs_sm_info *sm_i = SM_I(sbi);
@@ -3127,6 +3214,10 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent)
le32_to_cpu(raw_super->log_blocksize);
sb->s_max_links = F2FS_LINK_MAX;
+ err = f2fs_setup_casefold(sbi);
+ if (err)
+ goto free_options;
+
#ifdef CONFIG_QUOTA
sb->dq_op = &f2fs_quota_operations;
sb->s_qcop = &f2fs_quotactl_ops;
@@ -3477,6 +3568,10 @@ static int f2fs_fill_super(struct super_block *sb, void *data, int silent)
free_bio_info:
for (i = 0; i < NR_PAGE_TYPE; i++)
kvfree(sbi->write_io[i]);
+
+#ifdef CONFIG_UNICODE
+ utf8_unload(sbi->s_encoding);
+#endif
free_options:
#ifdef CONFIG_QUOTA
for (i = 0; i < MAXQUOTAS; i++)
diff --git a/fs/f2fs/sysfs.c b/fs/f2fs/sysfs.c
index 3aeacd0aacfd2..f9fcca695db9f 100644
--- a/fs/f2fs/sysfs.c
+++ b/fs/f2fs/sysfs.c
@@ -10,6 +10,7 @@
#include <linux/proc_fs.h>
#include <linux/f2fs_fs.h>
#include <linux/seq_file.h>
+#include <linux/unicode.h>
#include "f2fs.h"
#include "segment.h"
@@ -81,6 +82,19 @@ static ssize_t unusable_show(struct f2fs_attr *a,
(unsigned long long)unusable);
}
+static ssize_t encoding_show(struct f2fs_attr *a,
+ struct f2fs_sb_info *sbi, char *buf)
+{
+#ifdef CONFIG_UNICODE
+ if (f2fs_sb_has_casefold(sbi))
+ return snprintf(buf, PAGE_SIZE, "%s (%d.%d.%d)\n",
+ sbi->s_encoding->charset,
+ (sbi->s_encoding->version >> 16) & 0xff,
+ (sbi->s_encoding->version >> 8) & 0xff,
+ sbi->s_encoding->version & 0xff);
+#endif
+ return snprintf(buf, PAGE_SIZE, "(none)");
+}
static ssize_t lifetime_write_kbytes_show(struct f2fs_attr *a,
struct f2fs_sb_info *sbi, char *buf)
@@ -134,6 +148,9 @@ static ssize_t features_show(struct f2fs_attr *a,
if (f2fs_sb_has_sb_chksum(sbi))
len += snprintf(buf + len, PAGE_SIZE - len, "%s%s",
len ? ", " : "", "sb_checksum");
+ if (f2fs_sb_has_casefold(sbi))
+ len += snprintf(buf + len, PAGE_SIZE - len, "%s%s",
+ len ? ", " : "", "casefold");
len += snprintf(buf + len, PAGE_SIZE - len, "\n");
return len;
}
@@ -365,6 +382,7 @@ enum feat_id {
FEAT_INODE_CRTIME,
FEAT_LOST_FOUND,
FEAT_SB_CHECKSUM,
+ FEAT_CASEFOLD,
};
static ssize_t f2fs_feature_show(struct f2fs_attr *a,
@@ -382,6 +400,7 @@ static ssize_t f2fs_feature_show(struct f2fs_attr *a,
case FEAT_INODE_CRTIME:
case FEAT_LOST_FOUND:
case FEAT_SB_CHECKSUM:
+ case FEAT_CASEFOLD:
return snprintf(buf, PAGE_SIZE, "supported\n");
}
return 0;
@@ -455,6 +474,7 @@ F2FS_GENERAL_RO_ATTR(lifetime_write_kbytes);
F2FS_GENERAL_RO_ATTR(features);
F2FS_GENERAL_RO_ATTR(current_reserved_blocks);
F2FS_GENERAL_RO_ATTR(unusable);
+F2FS_GENERAL_RO_ATTR(encoding);
#ifdef CONFIG_FS_ENCRYPTION
F2FS_FEATURE_RO_ATTR(encryption, FEAT_CRYPTO);
@@ -471,6 +491,7 @@ F2FS_FEATURE_RO_ATTR(quota_ino, FEAT_QUOTA_INO);
F2FS_FEATURE_RO_ATTR(inode_crtime, FEAT_INODE_CRTIME);
F2FS_FEATURE_RO_ATTR(lost_found, FEAT_LOST_FOUND);
F2FS_FEATURE_RO_ATTR(sb_checksum, FEAT_SB_CHECKSUM);
+F2FS_FEATURE_RO_ATTR(casefold, FEAT_CASEFOLD);
#define ATTR_LIST(name) (&f2fs_attr_##name.attr)
static struct attribute *f2fs_attrs[] = {
@@ -515,6 +536,7 @@ static struct attribute *f2fs_attrs[] = {
ATTR_LIST(features),
ATTR_LIST(reserved_blocks),
ATTR_LIST(current_reserved_blocks),
+ ATTR_LIST(encoding),
NULL,
};
ATTRIBUTE_GROUPS(f2fs);
@@ -535,6 +557,7 @@ static struct attribute *f2fs_feat_attrs[] = {
ATTR_LIST(inode_crtime),
ATTR_LIST(lost_found),
ATTR_LIST(sb_checksum),
+ ATTR_LIST(casefold),
NULL,
};
ATTRIBUTE_GROUPS(f2fs_feat);
diff --git a/include/linux/f2fs_fs.h b/include/linux/f2fs_fs.h
index 65559900d4d76..b7c9c7f721339 100644
--- a/include/linux/f2fs_fs.h
+++ b/include/linux/f2fs_fs.h
@@ -36,6 +36,11 @@
#define F2FS_MAX_QUOTAS 3
+#define F2FS_ENC_UTF8_12_1 1
+#define F2FS_ENC_STRICT_MODE_FL (1 << 0)
+#define f2fs_has_strict_mode(sbi) \
+ (sbi->s_encoding_flags & F2FS_ENC_STRICT_MODE_FL)
+
#define F2FS_IO_SIZE(sbi) (1 << F2FS_OPTION(sbi).write_io_size_bits) /* Blocks */
#define F2FS_IO_SIZE_KB(sbi) (1 << (F2FS_OPTION(sbi).write_io_size_bits + 2)) /* KB */
#define F2FS_IO_SIZE_BYTES(sbi) (1 << (F2FS_OPTION(sbi).write_io_size_bits + 12)) /* B */
@@ -109,7 +114,9 @@ struct f2fs_super_block {
struct f2fs_device devs[MAX_DEVICES]; /* device list */
__le32 qf_ino[F2FS_MAX_QUOTAS]; /* quota inode numbers */
__u8 hot_ext_count; /* # of hot file extension */
- __u8 reserved[310]; /* valid reserved region */
+ __le16 s_encoding; /* Filename charset encoding */
+ __le16 s_encoding_flags; /* Filename charset encoding flags */
+ __u8 reserved[306]; /* valid reserved region */
__le32 crc; /* checksum of superblock */
} __packed;
--
2.22.0.657.g960e92d24f-goog
^ permalink raw reply related
* [PATCH v4 1/3] fs: Reserve flag for casefolding
From: Daniel Rosenberg @ 2019-07-23 23:05 UTC (permalink / raw)
To: Jaegeuk Kim, Chao Yu, Jonathan Corbet, linux-f2fs-devel
Cc: linux-kernel, linux-doc, linux-fsdevel, linux-api, kernel-team,
Daniel Rosenberg
In-Reply-To: <20190723230529.251659-1-drosen@google.com>
In preparation for including the casefold feature within f2fs, elevate
the EXT4_CASEFOLD_FL flag to FS_CASEFOLD_FL.
Signed-off-by: Daniel Rosenberg <drosen@google.com>
---
include/uapi/linux/fs.h | 1 +
tools/include/uapi/linux/fs.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index 59c71fa8c553a..2a616aa3f6862 100644
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -311,6 +311,7 @@ struct fscrypt_key {
#define FS_NOCOW_FL 0x00800000 /* Do not cow file */
#define FS_INLINE_DATA_FL 0x10000000 /* Reserved for ext4 */
#define FS_PROJINHERIT_FL 0x20000000 /* Create with parents projid */
+#define FS_CASEFOLD_FL 0x40000000 /* Folder is case insensitive */
#define FS_RESERVED_FL 0x80000000 /* reserved for ext2 lib */
#define FS_FL_USER_VISIBLE 0x0003DFFF /* User visible flags */
diff --git a/tools/include/uapi/linux/fs.h b/tools/include/uapi/linux/fs.h
index 59c71fa8c553a..2a616aa3f6862 100644
--- a/tools/include/uapi/linux/fs.h
+++ b/tools/include/uapi/linux/fs.h
@@ -311,6 +311,7 @@ struct fscrypt_key {
#define FS_NOCOW_FL 0x00800000 /* Do not cow file */
#define FS_INLINE_DATA_FL 0x10000000 /* Reserved for ext4 */
#define FS_PROJINHERIT_FL 0x20000000 /* Create with parents projid */
+#define FS_CASEFOLD_FL 0x40000000 /* Folder is case insensitive */
#define FS_RESERVED_FL 0x80000000 /* reserved for ext2 lib */
#define FS_FL_USER_VISIBLE 0x0003DFFF /* User visible flags */
--
2.22.0.657.g960e92d24f-goog
^ permalink raw reply related
* Re: [PATCH] Documentation: filesystem: fix "Removed Sysctls" table
From: Dave Chinner @ 2019-07-23 22:21 UTC (permalink / raw)
To: Sheriff Esseson
Cc: Jonathan Corbet, Darrick J. Wong, supporter:XFS FILESYSTEM,
open list:DOCUMENTATION, open list, skhan, linux-kernel-mentees
In-Reply-To: <20190723145201.GA20658@localhost>
On Tue, Jul 23, 2019 at 03:52:01PM +0100, Sheriff Esseson wrote:
> On Tue, Jul 23, 2019 at 07:42:18AM -0600, Jonathan Corbet wrote:
> > On Tue, 23 Jul 2019 12:48:13 +0100
> > Sheriff Esseson <sheriffesseson@gmail.com> wrote:
> >
> > > the "Removed Sysctls" section is a table - bring it alive with ReST.
> > >
> > > Signed-off-by: Sheriff Esseson <sheriffesseson@gmail.com>
> >
> > So this appears to be identical to the patch you sent three days ago; is
> > there a reason why you are sending it again now?
> >
> > Thanks,
> >
> > jon
>
> Sorry, I was think the patch went unnoticed during the merge window - I could
> not find a response.
The correct thing to do in that case is to reply to the original
patch and ask if it has been looked at. The usual way of doing this
is quoting the commit message and replying with a "Ping?" comment
to bump it back to the top of everyone's mail stacks.
But, again, 3 days is not a long time, people tend to be extremely
busy and might take a few days to get to reviewing non-critical
changes, and people may not even review patches during the merge
window. I'd suggest waiting a week before pinging a patch you've
sent if there's been no response....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox