* [PATCH RESEND v6 13/16] mm: Allow VM_FAULT_RETRY for multiple times
From: Peter Xu @ 2020-02-20 16:02 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Peter Xu, Martin Cracauer, Mike Rapoport, Hugh Dickins,
Jerome Glisse, Kirill A . Shutemov, Matthew Wilcox,
Pavel Emelyanov, Brian Geffon, Maya Gokhale, Denis Plotnikov,
Andrea Arcangeli, Johannes Weiner, Dr . David Alan Gilbert,
Linus Torvalds, Mike Kravetz, Marty McFadden, David Hildenbrand,
Bobby Powers, Mel Gorman
In-Reply-To: <20200220155353.8676-1-peterx@redhat.com>
The idea comes from a discussion between Linus and Andrea [1].
Before this patch we only allow a page fault to retry once. We
achieved this by clearing the FAULT_FLAG_ALLOW_RETRY flag when doing
handle_mm_fault() the second time. This was majorly used to avoid
unexpected starvation of the system by looping over forever to handle
the page fault on a single page. However that should hardly happen,
and after all for each code path to return a VM_FAULT_RETRY we'll
first wait for a condition (during which time we should possibly yield
the cpu) to happen before VM_FAULT_RETRY is really returned.
This patch removes the restriction by keeping the
FAULT_FLAG_ALLOW_RETRY flag when we receive VM_FAULT_RETRY. It means
that the page fault handler now can retry the page fault for multiple
times if necessary without the need to generate another page fault
event. Meanwhile we still keep the FAULT_FLAG_TRIED flag so page
fault handler can still identify whether a page fault is the first
attempt or not.
Then we'll have these combinations of fault flags (only considering
ALLOW_RETRY flag and TRIED flag):
- ALLOW_RETRY and !TRIED: this means the page fault allows to
retry, and this is the first try
- ALLOW_RETRY and TRIED: this means the page fault allows to
retry, and this is not the first try
- !ALLOW_RETRY and !TRIED: this means the page fault does not allow
to retry at all
- !ALLOW_RETRY and TRIED: this is forbidden and should never be used
In existing code we have multiple places that has taken special care
of the first condition above by checking against (fault_flags &
FAULT_FLAG_ALLOW_RETRY). This patch introduces a simple helper to
detect the first retry of a page fault by checking against
both (fault_flags & FAULT_FLAG_ALLOW_RETRY) and !(fault_flag &
FAULT_FLAG_TRIED) because now even the 2nd try will have the
ALLOW_RETRY set, then use that helper in all existing special paths.
One example is in __lock_page_or_retry(), now we'll drop the mmap_sem
only in the first attempt of page fault and we'll keep it in follow up
retries, so old locking behavior will be retained.
This will be a nice enhancement for current code [2] at the same time
a supporting material for the future userfaultfd-writeprotect work,
since in that work there will always be an explicit userfault
writeprotect retry for protected pages, and if that cannot resolve the
page fault (e.g., when userfaultfd-writeprotect is used in conjunction
with swapped pages) then we'll possibly need a 3rd retry of the page
fault. It might also benefit other potential users who will have
similar requirement like userfault write-protection.
GUP code is not touched yet and will be covered in follow up patch.
Please read the thread below for more information.
[1] https://lore.kernel.org/lkml/20171102193644.GB22686@redhat.com/
[2] https://lore.kernel.org/lkml/20181230154648.GB9832@redhat.com/
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
arch/alpha/mm/fault.c | 2 +-
arch/arc/mm/fault.c | 1 -
arch/arm/mm/fault.c | 3 ---
arch/arm64/mm/fault.c | 5 -----
arch/hexagon/mm/vm_fault.c | 1 -
arch/ia64/mm/fault.c | 1 -
arch/m68k/mm/fault.c | 3 ---
arch/microblaze/mm/fault.c | 1 -
arch/mips/mm/fault.c | 1 -
arch/nds32/mm/fault.c | 1 -
arch/nios2/mm/fault.c | 3 ---
arch/openrisc/mm/fault.c | 1 -
arch/parisc/mm/fault.c | 4 +---
arch/powerpc/mm/fault.c | 6 ------
arch/riscv/mm/fault.c | 5 -----
arch/s390/mm/fault.c | 5 +----
arch/sh/mm/fault.c | 1 -
arch/sparc/mm/fault_32.c | 1 -
arch/sparc/mm/fault_64.c | 1 -
arch/um/kernel/trap.c | 1 -
arch/unicore32/mm/fault.c | 4 +---
arch/x86/mm/fault.c | 2 --
arch/xtensa/mm/fault.c | 1 -
drivers/gpu/drm/ttm/ttm_bo_vm.c | 12 ++++++++---
include/linux/mm.h | 37 +++++++++++++++++++++++++++++++++
mm/filemap.c | 2 +-
mm/internal.h | 6 +++---
27 files changed, 54 insertions(+), 57 deletions(-)
diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c
index fcfa229cc1e7..c2d7b6d7bac7 100644
--- a/arch/alpha/mm/fault.c
+++ b/arch/alpha/mm/fault.c
@@ -169,7 +169,7 @@ do_page_fault(unsigned long address, unsigned long mmcsr,
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
+ flags |= FAULT_FLAG_TRIED;
/* No need to up_read(&mm->mmap_sem) as we would
* have already released it in __lock_page_or_retry
diff --git a/arch/arc/mm/fault.c b/arch/arc/mm/fault.c
index 643fad774071..92b339c7adba 100644
--- a/arch/arc/mm/fault.c
+++ b/arch/arc/mm/fault.c
@@ -145,7 +145,6 @@ void do_page_fault(unsigned long address, struct pt_regs *regs)
*/
if (unlikely((fault & VM_FAULT_RETRY) &&
(flags & FAULT_FLAG_ALLOW_RETRY))) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
goto retry;
}
diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
index 18ef0b143ac2..b598e6978b29 100644
--- a/arch/arm/mm/fault.c
+++ b/arch/arm/mm/fault.c
@@ -319,9 +319,6 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
regs, addr);
}
if (fault & VM_FAULT_RETRY) {
- /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
- * of starvation. */
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
goto retry;
}
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index cbb29a43aa7f..1027851d469a 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -521,12 +521,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr,
}
if (fault & VM_FAULT_RETRY) {
- /*
- * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk of
- * starvation.
- */
if (mm_flags & FAULT_FLAG_ALLOW_RETRY) {
- mm_flags &= ~FAULT_FLAG_ALLOW_RETRY;
mm_flags |= FAULT_FLAG_TRIED;
goto retry;
}
diff --git a/arch/hexagon/mm/vm_fault.c b/arch/hexagon/mm/vm_fault.c
index d9e15d941bdb..72334b26317a 100644
--- a/arch/hexagon/mm/vm_fault.c
+++ b/arch/hexagon/mm/vm_fault.c
@@ -102,7 +102,6 @@ void do_page_fault(unsigned long address, long cause, struct pt_regs *regs)
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
goto retry;
}
diff --git a/arch/ia64/mm/fault.c b/arch/ia64/mm/fault.c
index b5aa4e80c762..30d0c1fca99e 100644
--- a/arch/ia64/mm/fault.c
+++ b/arch/ia64/mm/fault.c
@@ -167,7 +167,6 @@ ia64_do_page_fault (unsigned long address, unsigned long isr, struct pt_regs *re
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/* No need to up_read(&mm->mmap_sem) as we would
diff --git a/arch/m68k/mm/fault.c b/arch/m68k/mm/fault.c
index 182799fd9987..f7afb9897966 100644
--- a/arch/m68k/mm/fault.c
+++ b/arch/m68k/mm/fault.c
@@ -162,9 +162,6 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
- * of starvation. */
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/*
diff --git a/arch/microblaze/mm/fault.c b/arch/microblaze/mm/fault.c
index 32da02778a63..3248141f8ed5 100644
--- a/arch/microblaze/mm/fault.c
+++ b/arch/microblaze/mm/fault.c
@@ -236,7 +236,6 @@ void do_page_fault(struct pt_regs *regs, unsigned long address,
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/*
diff --git a/arch/mips/mm/fault.c b/arch/mips/mm/fault.c
index f177da67d940..fd64b135fd7b 100644
--- a/arch/mips/mm/fault.c
+++ b/arch/mips/mm/fault.c
@@ -178,7 +178,6 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write,
tsk->min_flt++;
}
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/*
diff --git a/arch/nds32/mm/fault.c b/arch/nds32/mm/fault.c
index 2810a4e5ab27..0cf0c08c7da2 100644
--- a/arch/nds32/mm/fault.c
+++ b/arch/nds32/mm/fault.c
@@ -246,7 +246,6 @@ void do_page_fault(unsigned long entry, unsigned long addr,
1, regs, addr);
}
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/* No need to up_read(&mm->mmap_sem) as we would
diff --git a/arch/nios2/mm/fault.c b/arch/nios2/mm/fault.c
index c38bea4220fb..ec9d8a9c426f 100644
--- a/arch/nios2/mm/fault.c
+++ b/arch/nios2/mm/fault.c
@@ -157,9 +157,6 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long cause,
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
- * of starvation. */
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/*
diff --git a/arch/openrisc/mm/fault.c b/arch/openrisc/mm/fault.c
index 30d5c51e9d40..8af1cc78c4fb 100644
--- a/arch/openrisc/mm/fault.c
+++ b/arch/openrisc/mm/fault.c
@@ -181,7 +181,6 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long address,
else
tsk->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/* No need to up_read(&mm->mmap_sem) as we would
diff --git a/arch/parisc/mm/fault.c b/arch/parisc/mm/fault.c
index 8e88e5c5f26a..86e8c848f3d7 100644
--- a/arch/parisc/mm/fault.c
+++ b/arch/parisc/mm/fault.c
@@ -328,14 +328,12 @@ void do_page_fault(struct pt_regs *regs, unsigned long code,
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
-
/*
* No need to up_read(&mm->mmap_sem) as we would
* have already released it in __lock_page_or_retry
* in mm/filemap.c.
*/
-
+ flags |= FAULT_FLAG_TRIED;
goto retry;
}
}
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index d7e1f8dc7e4c..d15f0f0ee806 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -590,13 +590,7 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address,
* case.
*/
if (unlikely(fault & VM_FAULT_RETRY)) {
- /* We retry only once */
if (flags & FAULT_FLAG_ALLOW_RETRY) {
- /*
- * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
- * of starvation.
- */
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
goto retry;
}
diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
index a252d9e38561..be84e32adc4c 100644
--- a/arch/riscv/mm/fault.c
+++ b/arch/riscv/mm/fault.c
@@ -144,11 +144,6 @@ asmlinkage void do_page_fault(struct pt_regs *regs)
1, regs, addr);
}
if (fault & VM_FAULT_RETRY) {
- /*
- * Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
- * of starvation.
- */
- flags &= ~(FAULT_FLAG_ALLOW_RETRY);
flags |= FAULT_FLAG_TRIED;
/*
diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c
index 551ac311bd35..aeccdb30899a 100644
--- a/arch/s390/mm/fault.c
+++ b/arch/s390/mm/fault.c
@@ -513,10 +513,7 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access)
fault = VM_FAULT_PFAULT;
goto out_up;
}
- /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
- * of starvation. */
- flags &= ~(FAULT_FLAG_ALLOW_RETRY |
- FAULT_FLAG_RETRY_NOWAIT);
+ flags &= ~FAULT_FLAG_RETRY_NOWAIT;
flags |= FAULT_FLAG_TRIED;
down_read(&mm->mmap_sem);
goto retry;
diff --git a/arch/sh/mm/fault.c b/arch/sh/mm/fault.c
index d9c8f2d00a54..13ee4d20e622 100644
--- a/arch/sh/mm/fault.c
+++ b/arch/sh/mm/fault.c
@@ -481,7 +481,6 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs,
regs, address);
}
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/*
diff --git a/arch/sparc/mm/fault_32.c b/arch/sparc/mm/fault_32.c
index a91b0c2d84f8..f6e0e601f857 100644
--- a/arch/sparc/mm/fault_32.c
+++ b/arch/sparc/mm/fault_32.c
@@ -261,7 +261,6 @@ asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write,
1, regs, address);
}
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/* No need to up_read(&mm->mmap_sem) as we would
diff --git a/arch/sparc/mm/fault_64.c b/arch/sparc/mm/fault_64.c
index 30653418a672..c0c0dd471b6b 100644
--- a/arch/sparc/mm/fault_64.c
+++ b/arch/sparc/mm/fault_64.c
@@ -449,7 +449,6 @@ asmlinkage void __kprobes do_sparc64_fault(struct pt_regs *regs)
1, regs, address);
}
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/* No need to up_read(&mm->mmap_sem) as we would
diff --git a/arch/um/kernel/trap.c b/arch/um/kernel/trap.c
index c59ad37eacda..8f18cf56b3dd 100644
--- a/arch/um/kernel/trap.c
+++ b/arch/um/kernel/trap.c
@@ -97,7 +97,6 @@ int handle_page_fault(unsigned long address, unsigned long ip,
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
goto retry;
diff --git a/arch/unicore32/mm/fault.c b/arch/unicore32/mm/fault.c
index 34a90453ca18..a9bd08fbe588 100644
--- a/arch/unicore32/mm/fault.c
+++ b/arch/unicore32/mm/fault.c
@@ -259,9 +259,7 @@ static int do_pf(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
else
tsk->min_flt++;
if (fault & VM_FAULT_RETRY) {
- /* Clear FAULT_FLAG_ALLOW_RETRY to avoid any risk
- * of starvation. */
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
+ flags |= FAULT_FLAG_TRIED;
goto retry;
}
}
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 7b6f65333355..4ce647bbe546 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1457,8 +1457,6 @@ void do_user_addr_fault(struct pt_regs *regs,
*/
if (unlikely((fault & VM_FAULT_RETRY) &&
(flags & FAULT_FLAG_ALLOW_RETRY))) {
- /* Retry at most once */
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
goto retry;
}
diff --git a/arch/xtensa/mm/fault.c b/arch/xtensa/mm/fault.c
index 7d196dc951e8..e7172bd53ced 100644
--- a/arch/xtensa/mm/fault.c
+++ b/arch/xtensa/mm/fault.c
@@ -128,7 +128,6 @@ void do_page_fault(struct pt_regs *regs)
else
current->min_flt++;
if (fault & VM_FAULT_RETRY) {
- flags &= ~FAULT_FLAG_ALLOW_RETRY;
flags |= FAULT_FLAG_TRIED;
/* No need to up_read(&mm->mmap_sem) as we would
diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c
index 389128b8c4dd..cb8829ca6c7f 100644
--- a/drivers/gpu/drm/ttm/ttm_bo_vm.c
+++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c
@@ -59,9 +59,10 @@ static vm_fault_t ttm_bo_vm_fault_idle(struct ttm_buffer_object *bo,
/*
* If possible, avoid waiting for GPU with mmap_sem
- * held.
+ * held. We only do this if the fault allows retry and this
+ * is the first attempt.
*/
- if (vmf->flags & FAULT_FLAG_ALLOW_RETRY) {
+ if (fault_flag_allow_retry_first(vmf->flags)) {
ret = VM_FAULT_RETRY;
if (vmf->flags & FAULT_FLAG_RETRY_NOWAIT)
goto out_unlock;
@@ -135,7 +136,12 @@ vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo,
* for the buffer to become unreserved.
*/
if (unlikely(!dma_resv_trylock(bo->base.resv))) {
- if (vmf->flags & FAULT_FLAG_ALLOW_RETRY) {
+ /*
+ * If the fault allows retry and this is the first
+ * fault attempt, we try to release the mmap_sem
+ * before waiting
+ */
+ if (fault_flag_allow_retry_first(vmf->flags)) {
if (!(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) {
ttm_bo_get(bo);
up_read(&vmf->vma->vm_mm->mmap_sem);
diff --git a/include/linux/mm.h b/include/linux/mm.h
index ff653f9136dd..51a886d50758 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -391,6 +391,25 @@ extern pgprot_t protection_map[16];
* @FAULT_FLAG_REMOTE: The fault is not for current task/mm.
* @FAULT_FLAG_INSTRUCTION: The fault was during an instruction fetch.
* @FAULT_FLAG_INTERRUPTIBLE: The fault can be interrupted by non-fatal signals.
+ *
+ * About @FAULT_FLAG_ALLOW_RETRY and @FAULT_FLAG_TRIED: we can specify
+ * whether we would allow page faults to retry by specifying these two
+ * fault flags correctly. Currently there can be three legal combinations:
+ *
+ * (a) ALLOW_RETRY and !TRIED: this means the page fault allows retry, and
+ * this is the first try
+ *
+ * (b) ALLOW_RETRY and TRIED: this means the page fault allows retry, and
+ * we've already tried at least once
+ *
+ * (c) !ALLOW_RETRY and !TRIED: this means the page fault does not allow retry
+ *
+ * The unlisted combination (!ALLOW_RETRY && TRIED) is illegal and should never
+ * be used. Note that page faults can be allowed to retry for multiple times,
+ * in which case we'll have an initial fault with flags (a) then later on
+ * continuous faults with flags (b). We should always try to detect pending
+ * signals before a retry to make sure the continuous page faults can still be
+ * interrupted if necessary.
*/
#define FAULT_FLAG_WRITE 0x01
#define FAULT_FLAG_MKWRITE 0x02
@@ -411,6 +430,24 @@ extern pgprot_t protection_map[16];
FAULT_FLAG_KILLABLE | \
FAULT_FLAG_INTERRUPTIBLE)
+/**
+ * fault_flag_allow_retry_first - check ALLOW_RETRY the first time
+ *
+ * This is mostly used for places where we want to try to avoid taking
+ * the mmap_sem for too long a time when waiting for another condition
+ * to change, in which case we can try to be polite to release the
+ * mmap_sem in the first round to avoid potential starvation of other
+ * processes that would also want the mmap_sem.
+ *
+ * Return: true if the page fault allows retry and this is the first
+ * attempt of the fault handling; false otherwise.
+ */
+static inline bool fault_flag_allow_retry_first(unsigned int flags)
+{
+ return (flags & FAULT_FLAG_ALLOW_RETRY) &&
+ (!(flags & FAULT_FLAG_TRIED));
+}
+
#define FAULT_FLAG_TRACE \
{ FAULT_FLAG_WRITE, "WRITE" }, \
{ FAULT_FLAG_MKWRITE, "MKWRITE" }, \
diff --git a/mm/filemap.c b/mm/filemap.c
index 1784478270e1..590ec3a9f5da 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1386,7 +1386,7 @@ EXPORT_SYMBOL_GPL(__lock_page_killable);
int __lock_page_or_retry(struct page *page, struct mm_struct *mm,
unsigned int flags)
{
- if (flags & FAULT_FLAG_ALLOW_RETRY) {
+ if (fault_flag_allow_retry_first(flags)) {
/*
* CAUTION! In this case, mmap_sem is not released
* even though return 0.
diff --git a/mm/internal.h b/mm/internal.h
index 3cf20ab3ca01..5958cfe50a0c 100644
--- a/mm/internal.h
+++ b/mm/internal.h
@@ -377,10 +377,10 @@ static inline struct file *maybe_unlock_mmap_for_io(struct vm_fault *vmf,
/*
* FAULT_FLAG_RETRY_NOWAIT means we don't want to wait on page locks or
* anything, so we only pin the file and drop the mmap_sem if only
- * FAULT_FLAG_ALLOW_RETRY is set.
+ * FAULT_FLAG_ALLOW_RETRY is set, while this is the first attempt.
*/
- if ((flags & (FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_RETRY_NOWAIT)) ==
- FAULT_FLAG_ALLOW_RETRY) {
+ if (fault_flag_allow_retry_first(flags) &&
+ !(flags & FAULT_FLAG_RETRY_NOWAIT)) {
fpin = get_file(vmf->vma->vm_file);
up_read(&vmf->vma->vm_mm->mmap_sem);
}
--
2.24.1
^ permalink raw reply related
* Re: [PATCH v2] xfs: fix iclog release error check race with shutdown
From: Brian Foster @ 2020-02-20 16:02 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Darrick J. Wong, Dave Chinner, linux-xfs
In-Reply-To: <20200220154317.GB6870@infradead.org>
On Thu, Feb 20, 2020 at 07:43:17AM -0800, Christoph Hellwig wrote:
> On Thu, Feb 20, 2020 at 07:41:44AM -0500, Brian Foster wrote:
> > I wasn't planning on a v3. The discussion to this point has been
> > centered around the xfs_force_shutdown() call in the associated function
> > (which is orthogonal to the bug). v1 is technically correct, but
> > Christoph suggested to restore historical behavior wrt to the shutdown
> > call. v2 does that, but is a bit superfluous in that the iclog error
> > state with the lock held implies shutdown has already occurred. This is
> > harmless (unless we're worried about shutdown performance or
> > something..), but I think Dave indicated he preferred v1 based on that
> > reasoning.
> >
> > Functionally I don't think it matters either way and at this point I
> > have no preference between v1 or v2. They fix the same problem. Do note
> > that v2 does have the Fixed: tag I missed with v1 (as well as a R-b)...
>
> I'm fine with v1 after all this discussion, and volunteer to clean up
> all the ioerr handling for the log code after this fix goes in.
>
Ok.
> That being said as noted in one of my replies I think we also need to
> add the same check in the other caller of __xlog_state_release_iclog.
>
That seems reasonable as a cleanup, but I'm not sure how critical it is
otherwise. We've already handled the iclog at that point, so we're
basically just changing the function to return an error code regardless
of the fact that we ran xlog_sync() (which doesn't care about iclog
state until we attempt to write, where it checks state again before bio
submission) and that most callers have their own IOERROR state checks up
the chain anyways because there is no other indication of I/O submission
failure..
Brian
^ permalink raw reply
* [yocto] Lots of basehash related errors
From: Dmitri Toubelis @ 2020-02-20 16:02 UTC (permalink / raw)
To: yocto
[-- Attachment #1: Type: text/plain, Size: 2916 bytes --]
Hi,
I'm migrating from yocto morty to zeus and I'm receiving a whole lot of errors like this:
>
> ERROR: When reparsing
> /srv/yocto/poky/meta-loopedge/meta-loopedge-dist/recipes-core/images/loopedge-std.bb:do_image_wic,
> the basehash value changed from
> 8dd96e09d0b7defa552e586e626933ca37ace5180918ea65addbfcb6c1247b1c to
> e38fae3f400b3e3de114fcd668d46a1f7c3eec436ec72962e78df906714a6fb0. The
> metadata is not deterministic and this needs to be fixed.
> ERROR: The following commands may help:
> ERROR: $ bitbake loopedge-std -cdo_image_wic -Snone
> ERROR: Then:
> ERROR: $ bitbake loopedge-std -cdo_image_wic -Sprintdiff
>
> ERROR: When reparsing
> /srv/yocto/poky/meta-loopedge/meta-loopedge-dist/recipes-core/images/loopedge-std.bb:do_image_ext4,
> the basehash value changed from
> a8209ab35324ce59bb193b80871c12c492f69f42fd97b03801165bd4a12670f6 to
> 1ab2d25ef217fe87b4cce1106d122acd4286043b04dcd74d98df30a01aa6a0b9. The
> metadata is not deterministic and this needs to be fixed.
> ERROR: The following commands may help:
> ERROR: $ bitbake loopedge-std -cdo_image_ext4 -Snone
> ERROR: Then:
> ERROR: $ bitbake loopedge-std -cdo_image_ext4 -Sprintdiff
>
> ERROR: When reparsing
> /srv/yocto/poky/meta-loopedge/meta-loopedge-dist/recipes-core/images/loopedge-std.bb:do_image_tar,
> the basehash value changed from
> c5ab62cac832e502a338d59124efc690e66560a4e877bc4ba3487c3a734c2497 to
> bb7ca72863614cb5c9915eb502259b1ffa8b98992f7ad3280d1e049a1824b930. The
> metadata is not deterministic and this needs to be fixed.
> ERROR: The following commands may help:
> ERROR: $ bitbake loopedge-std -cdo_image_tar -Snone
> ERROR: Then:
> ERROR: $ bitbake loopedge-std -cdo_image_tar -Sprintdiff
>
I search around for answers and there are here are reasons and solutions for this that I found:
- to make sure any date related variables are excluded from basehash via `do_task_name[vardepsexclude] = "DATE DATETIME"`
- clears state cache with `bitbake image -c cleansstate`
- delete tmp directory and build from scratch
Here is my observation and interpretation:
- this messages occur when running with pristine build directory, i.e. it only contains 2 files in `conf` dir - `local.conf` and `bblatyers.conf`, so I can rule out contamination from a previous run.
- same messages reapeat over and over totalling ~900 errors at the end of the run
- I have few custom classes and I removed them from the image to rule out contamination from my own code.
- Tasks that give this error are coming from image.bbclass from poky and none of them have been altered in any way.
- The image build runs through the end but because bitbake exits with non-zero exit code it breaks lots of our tools, so just ignoring them is a bad option.
So, am I doing something obviously wrong? It is a known issue in zeus? And if so is there a known workaround?
Thanks in advance.
[-- Attachment #2: Type: text/html, Size: 3152 bytes --]
^ permalink raw reply
* [PATCH RESEND v6 11/16] mm: Introduce FAULT_FLAG_DEFAULT
From: Peter Xu @ 2020-02-20 16:02 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Peter Xu, Martin Cracauer, Mike Rapoport, Hugh Dickins,
Jerome Glisse, Kirill A . Shutemov, Matthew Wilcox,
Pavel Emelyanov, Brian Geffon, Maya Gokhale, Denis Plotnikov,
Andrea Arcangeli, Johannes Weiner, Dr . David Alan Gilbert,
Linus Torvalds, Mike Kravetz, Marty McFadden, David Hildenbrand,
Bobby Powers, Mel Gorman
In-Reply-To: <20200220155353.8676-1-peterx@redhat.com>
Although there're tons of arch-specific page fault handlers, most of
them are still sharing the same initial value of the page fault flags.
Say, merely all of the page fault handlers would allow the fault to be
retried, and they also allow the fault to respond to SIGKILL.
Let's define a default value for the fault flags to replace those
initial page fault flags that were copied over. With this, it'll be
far easier to introduce new fault flag that can be used by all the
architectures instead of touching all the archs.
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
arch/alpha/mm/fault.c | 2 +-
arch/arc/mm/fault.c | 2 +-
arch/arm/mm/fault.c | 2 +-
arch/arm64/mm/fault.c | 2 +-
arch/hexagon/mm/vm_fault.c | 2 +-
arch/ia64/mm/fault.c | 2 +-
arch/m68k/mm/fault.c | 2 +-
arch/microblaze/mm/fault.c | 2 +-
arch/mips/mm/fault.c | 2 +-
arch/nds32/mm/fault.c | 2 +-
arch/nios2/mm/fault.c | 2 +-
arch/openrisc/mm/fault.c | 2 +-
arch/parisc/mm/fault.c | 2 +-
arch/powerpc/mm/fault.c | 2 +-
arch/riscv/mm/fault.c | 2 +-
arch/s390/mm/fault.c | 2 +-
arch/sh/mm/fault.c | 2 +-
arch/sparc/mm/fault_32.c | 2 +-
arch/sparc/mm/fault_64.c | 2 +-
arch/um/kernel/trap.c | 2 +-
arch/unicore32/mm/fault.c | 2 +-
arch/x86/mm/fault.c | 2 +-
arch/xtensa/mm/fault.c | 2 +-
include/linux/mm.h | 7 +++++++
24 files changed, 30 insertions(+), 23 deletions(-)
diff --git a/arch/alpha/mm/fault.c b/arch/alpha/mm/fault.c
index aea33b599037..fcfa229cc1e7 100644
--- a/arch/alpha/mm/fault.c
+++ b/arch/alpha/mm/fault.c
@@ -89,7 +89,7 @@ do_page_fault(unsigned long address, unsigned long mmcsr,
const struct exception_table_entry *fixup;
int si_code = SEGV_MAPERR;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
/* As of EV6, a load into $31/$f31 is a prefetch, and never faults
(or is suppressed by the PALcode). Support that for older CPUs
diff --git a/arch/arc/mm/fault.c b/arch/arc/mm/fault.c
index 6eb821a59b49..643fad774071 100644
--- a/arch/arc/mm/fault.c
+++ b/arch/arc/mm/fault.c
@@ -100,7 +100,7 @@ void do_page_fault(unsigned long address, struct pt_regs *regs)
(regs->ecr_cause == ECR_C_PROTV_INST_FETCH))
exec = 1;
- flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ flags = FAULT_FLAG_DEFAULT;
if (user_mode(regs))
flags |= FAULT_FLAG_USER;
if (write)
diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
index 937b81ff8649..18ef0b143ac2 100644
--- a/arch/arm/mm/fault.c
+++ b/arch/arm/mm/fault.c
@@ -241,7 +241,7 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
struct mm_struct *mm;
int sig, code;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
if (kprobe_page_fault(regs, fsr))
return 0;
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 6f4b69d712b1..cbb29a43aa7f 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -446,7 +446,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr,
struct mm_struct *mm = current->mm;
vm_fault_t fault, major = 0;
unsigned long vm_flags = VM_READ | VM_WRITE | VM_EXEC;
- unsigned int mm_flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int mm_flags = FAULT_FLAG_DEFAULT;
if (kprobe_page_fault(regs, esr))
return 0;
diff --git a/arch/hexagon/mm/vm_fault.c b/arch/hexagon/mm/vm_fault.c
index d19beaf11b4c..d9e15d941bdb 100644
--- a/arch/hexagon/mm/vm_fault.c
+++ b/arch/hexagon/mm/vm_fault.c
@@ -41,7 +41,7 @@ void do_page_fault(unsigned long address, long cause, struct pt_regs *regs)
int si_code = SEGV_MAPERR;
vm_fault_t fault;
const struct exception_table_entry *fixup;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
/*
* If we're in an interrupt or have no user context,
diff --git a/arch/ia64/mm/fault.c b/arch/ia64/mm/fault.c
index 211b4f439384..b5aa4e80c762 100644
--- a/arch/ia64/mm/fault.c
+++ b/arch/ia64/mm/fault.c
@@ -65,7 +65,7 @@ ia64_do_page_fault (unsigned long address, unsigned long isr, struct pt_regs *re
struct mm_struct *mm = current->mm;
unsigned long mask;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
mask = ((((isr >> IA64_ISR_X_BIT) & 1UL) << VM_EXEC_BIT)
| (((isr >> IA64_ISR_W_BIT) & 1UL) << VM_WRITE_BIT));
diff --git a/arch/m68k/mm/fault.c b/arch/m68k/mm/fault.c
index a455e202691b..182799fd9987 100644
--- a/arch/m68k/mm/fault.c
+++ b/arch/m68k/mm/fault.c
@@ -71,7 +71,7 @@ int do_page_fault(struct pt_regs *regs, unsigned long address,
struct mm_struct *mm = current->mm;
struct vm_area_struct * vma;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
pr_debug("do page fault:\nregs->sr=%#x, regs->pc=%#lx, address=%#lx, %ld, %p\n",
regs->sr, regs->pc, address, error_code, mm ? mm->pgd : NULL);
diff --git a/arch/microblaze/mm/fault.c b/arch/microblaze/mm/fault.c
index cdde01dcdfc3..32da02778a63 100644
--- a/arch/microblaze/mm/fault.c
+++ b/arch/microblaze/mm/fault.c
@@ -91,7 +91,7 @@ void do_page_fault(struct pt_regs *regs, unsigned long address,
int code = SEGV_MAPERR;
int is_write = error_code & ESR_S;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
regs->ear = address;
regs->esr = error_code;
diff --git a/arch/mips/mm/fault.c b/arch/mips/mm/fault.c
index 0ee6fafc57bc..f177da67d940 100644
--- a/arch/mips/mm/fault.c
+++ b/arch/mips/mm/fault.c
@@ -44,7 +44,7 @@ static void __kprobes __do_page_fault(struct pt_regs *regs, unsigned long write,
const int field = sizeof(unsigned long) * 2;
int si_code;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
static DEFINE_RATELIMIT_STATE(ratelimit_state, 5 * HZ, 10);
diff --git a/arch/nds32/mm/fault.c b/arch/nds32/mm/fault.c
index 0e63f81eff5b..2810a4e5ab27 100644
--- a/arch/nds32/mm/fault.c
+++ b/arch/nds32/mm/fault.c
@@ -80,7 +80,7 @@ void do_page_fault(unsigned long entry, unsigned long addr,
int si_code;
vm_fault_t fault;
unsigned int mask = VM_READ | VM_WRITE | VM_EXEC;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
error_code = error_code & (ITYPE_mskINST | ITYPE_mskETYPE);
tsk = current;
diff --git a/arch/nios2/mm/fault.c b/arch/nios2/mm/fault.c
index 704ace8ca0ee..c38bea4220fb 100644
--- a/arch/nios2/mm/fault.c
+++ b/arch/nios2/mm/fault.c
@@ -47,7 +47,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long cause,
struct mm_struct *mm = tsk->mm;
int code = SEGV_MAPERR;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
cause >>= 2;
diff --git a/arch/openrisc/mm/fault.c b/arch/openrisc/mm/fault.c
index 85c7eb0c0186..30d5c51e9d40 100644
--- a/arch/openrisc/mm/fault.c
+++ b/arch/openrisc/mm/fault.c
@@ -50,7 +50,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs, unsigned long address,
struct vm_area_struct *vma;
int si_code;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
tsk = current;
diff --git a/arch/parisc/mm/fault.c b/arch/parisc/mm/fault.c
index f9be1d1cb43f..8e88e5c5f26a 100644
--- a/arch/parisc/mm/fault.c
+++ b/arch/parisc/mm/fault.c
@@ -274,7 +274,7 @@ void do_page_fault(struct pt_regs *regs, unsigned long code,
if (!mm)
goto no_context;
- flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ flags = FAULT_FLAG_DEFAULT;
if (user_mode(regs))
flags |= FAULT_FLAG_USER;
diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c
index 0868172ce4e3..d7e1f8dc7e4c 100644
--- a/arch/powerpc/mm/fault.c
+++ b/arch/powerpc/mm/fault.c
@@ -434,7 +434,7 @@ static int __do_page_fault(struct pt_regs *regs, unsigned long address,
{
struct vm_area_struct * vma;
struct mm_struct *mm = current->mm;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
int is_exec = TRAP(regs) == 0x400;
int is_user = user_mode(regs);
int is_write = page_fault_is_write(error_code);
diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
index 1d3869e9ddef..a252d9e38561 100644
--- a/arch/riscv/mm/fault.c
+++ b/arch/riscv/mm/fault.c
@@ -30,7 +30,7 @@ asmlinkage void do_page_fault(struct pt_regs *regs)
struct vm_area_struct *vma;
struct mm_struct *mm;
unsigned long addr, cause;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
int code = SEGV_MAPERR;
vm_fault_t fault;
diff --git a/arch/s390/mm/fault.c b/arch/s390/mm/fault.c
index 179cf92a56e5..551ac311bd35 100644
--- a/arch/s390/mm/fault.c
+++ b/arch/s390/mm/fault.c
@@ -429,7 +429,7 @@ static inline vm_fault_t do_exception(struct pt_regs *regs, int access)
address = trans_exc_code & __FAIL_ADDR_MASK;
perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
- flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ flags = FAULT_FLAG_DEFAULT;
if (user_mode(regs))
flags |= FAULT_FLAG_USER;
if (access == VM_WRITE || (trans_exc_code & store_indication) == 0x400)
diff --git a/arch/sh/mm/fault.c b/arch/sh/mm/fault.c
index eb4048ad0b38..d9c8f2d00a54 100644
--- a/arch/sh/mm/fault.c
+++ b/arch/sh/mm/fault.c
@@ -380,7 +380,7 @@ asmlinkage void __kprobes do_page_fault(struct pt_regs *regs,
struct mm_struct *mm;
struct vm_area_struct * vma;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
tsk = current;
mm = tsk->mm;
diff --git a/arch/sparc/mm/fault_32.c b/arch/sparc/mm/fault_32.c
index 6efbeb227644..a91b0c2d84f8 100644
--- a/arch/sparc/mm/fault_32.c
+++ b/arch/sparc/mm/fault_32.c
@@ -168,7 +168,7 @@ asmlinkage void do_sparc_fault(struct pt_regs *regs, int text_fault, int write,
int from_user = !(regs->psr & PSR_PS);
int code;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
if (text_fault)
address = regs->pc;
diff --git a/arch/sparc/mm/fault_64.c b/arch/sparc/mm/fault_64.c
index dd1ed6555831..30653418a672 100644
--- a/arch/sparc/mm/fault_64.c
+++ b/arch/sparc/mm/fault_64.c
@@ -271,7 +271,7 @@ asmlinkage void __kprobes do_sparc64_fault(struct pt_regs *regs)
int si_code, fault_code;
vm_fault_t fault;
unsigned long address, mm_rss;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
fault_code = get_thread_fault_code();
diff --git a/arch/um/kernel/trap.c b/arch/um/kernel/trap.c
index 818553064f04..c59ad37eacda 100644
--- a/arch/um/kernel/trap.c
+++ b/arch/um/kernel/trap.c
@@ -33,7 +33,7 @@ int handle_page_fault(unsigned long address, unsigned long ip,
pmd_t *pmd;
pte_t *pte;
int err = -EFAULT;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
*code_out = SEGV_MAPERR;
diff --git a/arch/unicore32/mm/fault.c b/arch/unicore32/mm/fault.c
index 59d0e6ec2cfc..34a90453ca18 100644
--- a/arch/unicore32/mm/fault.c
+++ b/arch/unicore32/mm/fault.c
@@ -202,7 +202,7 @@ static int do_pf(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
struct mm_struct *mm;
int sig, code;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
tsk = current;
mm = tsk->mm;
diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
index 6a00bc8d047f..7b6f65333355 100644
--- a/arch/x86/mm/fault.c
+++ b/arch/x86/mm/fault.c
@@ -1288,7 +1288,7 @@ void do_user_addr_fault(struct pt_regs *regs,
struct task_struct *tsk;
struct mm_struct *mm;
vm_fault_t fault, major = 0;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
tsk = current;
mm = tsk->mm;
diff --git a/arch/xtensa/mm/fault.c b/arch/xtensa/mm/fault.c
index 59515905d4ad..7d196dc951e8 100644
--- a/arch/xtensa/mm/fault.c
+++ b/arch/xtensa/mm/fault.c
@@ -43,7 +43,7 @@ void do_page_fault(struct pt_regs *regs)
int is_write, is_exec;
vm_fault_t fault;
- unsigned int flags = FAULT_FLAG_ALLOW_RETRY | FAULT_FLAG_KILLABLE;
+ unsigned int flags = FAULT_FLAG_DEFAULT;
code = SEGV_MAPERR;
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 52269e56c514..08ca35d3c341 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -388,6 +388,13 @@ extern pgprot_t protection_map[16];
#define FAULT_FLAG_REMOTE 0x80 /* faulting for non current tsk/mm */
#define FAULT_FLAG_INSTRUCTION 0x100 /* The fault was during an instruction fetch */
+/*
+ * The default fault flags that should be used by most of the
+ * arch-specific page fault handlers.
+ */
+#define FAULT_FLAG_DEFAULT (FAULT_FLAG_ALLOW_RETRY | \
+ FAULT_FLAG_KILLABLE)
+
#define FAULT_FLAG_TRACE \
{ FAULT_FLAG_WRITE, "WRITE" }, \
{ FAULT_FLAG_MKWRITE, "MKWRITE" }, \
--
2.24.1
^ permalink raw reply related
* Re: [PATCH] console: make QMP screendump use coroutine
From: Markus Armbruster @ 2020-02-20 16:01 UTC (permalink / raw)
To: Marc-André Lureau
Cc: Kevin Wolf, QEMU, David Alan Gilbert, Gerd Hoffmann
In-Reply-To: <CAJ+F1C+M3yPreBLOHXkt16b5aghesT7qYkEPbS_3Dm7vGTaMKA@mail.gmail.com>
Cc: David for questions regarding the HMP core. David, please look for
"Is HMP blocking the main loop a problem?"
Marc-André Lureau <marcandre.lureau@gmail.com> writes:
> Hi
>
> On Thu, Feb 20, 2020 at 8:49 AM Markus Armbruster <armbru@redhat.com> wrote:
>>
>> Marc-André Lureau <marcandre.lureau@redhat.com> writes:
>>
>> > Thanks to the QMP coroutine support, the screendump handler can
>> > trigger a graphic_hw_update(), yield and let the main loop run until
>> > update is done. Then the handler is resumed, and the ppm_save() will
>> > write the screen image to disk in the coroutine context (thus
>> > non-blocking).
>> >
>> > For now, HMP doesn't have coroutine support, so it remains potentially
>> > outdated or glitched.
>> >
>> > Fixes:
>> > https://bugzilla.redhat.com/show_bug.cgi?id=1230527
>> >
>> > Based-on: <20200109183545.27452-2-kwolf@redhat.com>
>> >
>> > Cc: Kevin Wolf <kwolf@redhat.com>
>> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> > ---
>> > qapi/ui.json | 3 ++-
>> > ui/console.c | 35 +++++++++++++++++++++++++++--------
>> > ui/trace-events | 2 +-
>> > 3 files changed, 30 insertions(+), 10 deletions(-)
>> >
>> > diff --git a/qapi/ui.json b/qapi/ui.json
>> > index e04525d8b4..d941202f34 100644
>> > --- a/qapi/ui.json
>> > +++ b/qapi/ui.json
>> > @@ -96,7 +96,8 @@
>> > #
>> > ##
>> > { 'command': 'screendump',
>> > - 'data': {'filename': 'str', '*device': 'str', '*head': 'int'} }
>> > + 'data': {'filename': 'str', '*device': 'str', '*head': 'int'},
>> > + 'coroutine': true }
>> >
>> > ##
>> > # == Spice
>> > diff --git a/ui/console.c b/ui/console.c
>> > index ac79d679f5..db184b473f 100644
>> > --- a/ui/console.c
>> > +++ b/ui/console.c
>> > @@ -167,6 +167,7 @@ struct QemuConsole {
>> > QEMUFIFO out_fifo;
>> > uint8_t out_fifo_buf[16];
>> > QEMUTimer *kbd_timer;
>> > + Coroutine *screendump_co;
>> >
>> > QTAILQ_ENTRY(QemuConsole) next;
>> > };
>> > @@ -194,7 +195,6 @@ static void dpy_refresh(DisplayState *s);
>> > static DisplayState *get_alloc_displaystate(void);
>> > static void text_console_update_cursor_timer(void);
>> > static void text_console_update_cursor(void *opaque);
>> > -static bool ppm_save(int fd, DisplaySurface *ds, Error **errp);
>> >
>> > static void gui_update(void *opaque)
>> > {
>> > @@ -263,6 +263,9 @@ static void gui_setup_refresh(DisplayState *ds)
>> >
>> > void graphic_hw_update_done(QemuConsole *con)
>> > {
>> > + if (con && con->screendump_co) {
>>
>> How can !con happen?
>
> I don't think it can happen anymore (the patch evolved over several
> years, this is probably a left-over). In any case, it doesn't hurt.
I hate such dead checks, because they make me assume they can actually
happen. Incorrect assumptions breed bugs.
But I'm willing to defer to the maintainer here. Gerd?
>> > + aio_co_wake(con->screendump_co);
>> > + }
>> > }
>> >
>> > void graphic_hw_update(QemuConsole *con)
>> > @@ -310,16 +313,16 @@ void graphic_hw_invalidate(QemuConsole *con)
>> > }
>> > }
>> >
>> > -static bool ppm_save(int fd, DisplaySurface *ds, Error **errp)
>> > +static bool ppm_save(int fd, pixman_image_t *image, Error **errp)
>> > {
>> > - int width = pixman_image_get_width(ds->image);
>> > - int height = pixman_image_get_height(ds->image);
>> > + int width = pixman_image_get_width(image);
>> > + int height = pixman_image_get_height(image);
>> > g_autoptr(Object) ioc = OBJECT(qio_channel_file_new_fd(fd));
>> > g_autofree char *header = NULL;
>> > g_autoptr(pixman_image_t) linebuf = NULL;
>> > int y;
>> >
>> > - trace_ppm_save(fd, ds);
>> > + trace_ppm_save(fd, image);
>> >
>> > header = g_strdup_printf("P6\n%d %d\n%d\n", width, height, 255);
>> > if (qio_channel_write_all(QIO_CHANNEL(ioc),
>> > @@ -329,7 +332,7 @@ static bool ppm_save(int fd, DisplaySurface *ds, Error **errp)
>> >
>> > linebuf = qemu_pixman_linebuf_create(PIXMAN_BE_r8g8b8, width);
>> > for (y = 0; y < height; y++) {
>> > - qemu_pixman_linebuf_fill(linebuf, ds->image, width, 0, y);
>> > + qemu_pixman_linebuf_fill(linebuf, image, width, 0, y);
>> > if (qio_channel_write_all(QIO_CHANNEL(ioc),
>> > (char *)pixman_image_get_data(linebuf),
>> > pixman_image_get_stride(linebuf), errp) < 0) {
>>
>> Looks like an unrelated optimization / simplification. If I was
>> maintainer, I'd ask for a separate patch.
>
> I can be split, but it's related. We should pass a reference to
> pixman_image_t, rather than a pointer to DisplaySurface, as the
> underlying image may change over time, and would result in corrupted
> coroutine save or worse.
Work that into your commit message, please. Might be easier if you
split, but that's between you and the maintainer :)
>> > @@ -340,11 +343,18 @@ static bool ppm_save(int fd, DisplaySurface *ds, Error **errp)
>> > return true;
>> > }
>> >
>> > +static void graphic_hw_update_bh(void *con)
>> > +{
>> > + graphic_hw_update(con);
>> > +}
>> > +
>> > +/* may be called in coroutine context or not */
>>
>> Hmm.
>>
>> Even though the QMP core always calls in coroutine context, the comment
>> is correct: hmp_screendump() calls it outside coroutine context.
>> Because of that...
>>
>> > void qmp_screendump(const char *filename, bool has_device, const char *device,
>> > bool has_head, int64_t head, Error **errp)
>> > {
>> > QemuConsole *con;
>> > DisplaySurface *surface;
>> > + g_autoptr(pixman_image_t) image = NULL;
>> > int fd;
>> >
>> > if (has_device) {
>> > @@ -365,7 +375,15 @@ void qmp_screendump(const char *filename, bool has_device, const char *device,
>> > }
>> > }
>> >
>> > - graphic_hw_update(con);
>> > + if (qemu_in_coroutine()) {
>> > + assert(!con->screendump_co);
>>
>> What if multiple QMP monitors simultaneously screendump? Hmm, it works
>> because all execute one after another in the same coroutine
>> qmp_dispatcher_co. Implicit mutual exclusion.
>>
>> Executing them one after another is bad, because it lets an ill-behaved
>> QMP command starve *all* QMP monitors. We do it only out of
>> (reasonable!) fear of implicit mutual exclusion requirements like the
>> one you add.
>>
>> Let's not add more if we can help it.
>
> The situation is not worse than the current blocking handling.
Really?
What makes executing multiple qmp_screendump() concurrently (in separate
threads) or interleaved (in separate coroutines in the same thread)
unsafe before this patch?
>> Your screendump_co is per QemuConsole instead of per QMP monitor only
>> because you need to find the coroutine in graphic_hw_update_done(). Can
>> we somehow pass it via function arguments?
>
> I think it could be done later, so I suggest a TODO.
We should avoid making our dependence on implicit mutual exclusion
worse. When we do it anyway, a big, fat, ugly comment is definitely
called for.
>> In case avoiding the mutual exclusion is impractical: please explain it
>> in a comment to make it somewhat less implicit.
It is anything but: see appended patch.
>> > + con->screendump_co = qemu_coroutine_self();
>> > + aio_bh_schedule_oneshot(qemu_get_aio_context(),
>> > + graphic_hw_update_bh, con);
>> > + qemu_coroutine_yield();
>> > + con->screendump_co = NULL;
>> > + }
>> > +
>>
>> ... the command handler needs extra code to cope with either. Is this
>> really what we want for coroutine QMP command handlers? We'll acquire
>> more of them, and I'd hate to make each one run both in and outside
>> coroutine context. Shouldn't we let the HMP core take care of this? Or
>> at least have some common infrastructure these handlers can use?
>
> We have several functions that have this dual support, for ex QIO.
>
> Changing both QMP & HMP commands to run in coroutine is likely
> additional work that we may not care at this point.
If it wasn't for non-QMP calls (typically HMP, but also CLI), then
handlers for QMP commands with 'coroutine': true could be coroutine_fn.
So far, coroutine_fn is merely documentation. Perhaps it can guide a
checker for "do coroutine stuff only in coroutines" some day. Would be
nice, because the coroutine calls are often buried deep, and far away
from the code that ensures they run in a coroutine.
My point is: marking functions coroutine_fn is good. We should do it
more. We should try to avoid stuff that hinders doing it more.
> I propose to leave a TODO, once we have several similar QMP & HMP mix
> cases we can try to find a common HMP solution to make the code
> simpler in QMP handler.
Collecting several users before building infrastructure makes sense when
the design of the infrastructure isn't obvious, or when the need for it
is in doubt.
Neither is the case for running QMP handlers in a coroutine: QMP
commands blocking the main loop is without doubt a problem we need to
solve, and the way to solve it was obvious enough for Kevin to do it
with one user: block_resize. A second one quickly followed: screendump.
The only part that's different for HMP, I think, is "need".
Is HMP blocking the main loop a problem?
If yes, is it serious enough to justify solving it?
If yes, then putting workarounds into QMP handlers now so we can put off
solving it some more is taking on technical debt.
> I don't know if this is going to be a common pattern, we may end up
> with conversions that can run both without explicit handling (like the
> ppm_save() function, thanks to QIO).
Yes, such handlers may exist. Running them out of coroutine context
would throw away their capability not to block the event loop, though,
wouldn't it?
>> Why is it okay not to call graphic_hw_update() anymore when
>> !qemu_in_coroutine()?
>
> You could call it, but then you should wait for completion by
> reentering the main loop (that was the point of my earlier qapi-async
> series)
Possibly stupid question: why is it necessary before this patch
(assuming it is, since we call it), and why is it no longer necessary
after?
Oh, see below.
>>
>> If qemu_in_coroutine(), we now run graphic_hw_update() in a bottom half,
>> then yield until the update completes (see graphic_hw_update_done()
>> above). Can you explain the need for the bottom half?
>
> At least spice rendering is done in a separate thread, completion is async.
When I ask a stupid question like this one, I'm really after the "for
dummies" explanation. I may be able to figure it out myself with some
effort, but having to put that kind of effort into patch review makes me
grumpy, and once I'm sufficiently grumpy, I don't want to review patches
anymore, let alone merge them.
Oh well, let me try. We're in the main loop. We want to trigger a
"graphics update" (whatever that is, doesn't matter) and wait for it to
complete without blocking the main loop.
"Without blocking the main loop" means the QMP coroutine yields. I'd
naively expect
QMP coroutine: schedule the job; yield
whatever gets scheduled: complete the job; wake up QMP coroutine
Now let's examine the "graphics update" interface.
GraphicHwOps callback gfx_update() comes in two flavours:
* synchronous: complete the job, return
* asynchronous: start the job, return immediately,
graphic_hw_update_done() will get called on job completion
graphic_hw_update() partly hides the difference:
* synchronous: complete the job, call graphic_hw_update_done()
* asynchronous: start the job, return immediately,
graphic_hw_update_done() will get called on job completion
This lets you treat the synchronous case more like the asynchronous
case.
You use graphic_hw_update_done() to wake up the QMP coroutine.
I think I can now answer my question "why is it okay not to call
graphic_hw_update() anymore when !qemu_in_coroutine()?"
Before the patch, both QMP and HMP:
* with synchronous gfx_update(): we update before we write out the
screendump. The screendump is up-to-date. Both update and write out
block the main loop.
* with asynchronous gfx_update(): we start updating, but don't wait for
it to complete before we write out. This is wrong. Write out blocks
the main loop, but update does not.
After the patch:
* QMP with either gfx_update(): we update before we write out the
screendump. The screendump is up-to-date. Neither update nor write
out block the main loop. Improvement.
* HMP with either gfx_update(): we don't update before we write out.
Similarly wrong for asynchronous gfx_update(), regression for
synchronous gfx_update(). Write out blocks the main loop as before.
Why is the regression okay?
Back to the bottom half. The way graphic_hw_update() works, the QMP
coroutine can't schedule then yield. It *has* to yield before
graphic_hw_update() runs. That means we need a bottom half.
Alright, I'm officially grumpy now.
Please explain the need for a bottom half in a comment.
>> > surface = qemu_console_surface(con);
>> > if (!surface) {
>> > error_setg(errp, "no surface");
>> > @@ -379,7 +397,8 @@ void qmp_screendump(const char *filename, bool has_device, const char *device,
>> > return;
>> > }
>> >
>> > - if (!ppm_save(fd, surface, errp)) {
>> > + image = pixman_image_ref(surface->image);
>> > + if (!ppm_save(fd, image, errp)) {
>> > qemu_unlink(filename);
>> > }
>> > }
>> > diff --git a/ui/trace-events b/ui/trace-events
>> > index 0dcda393c1..e8726fc969 100644
>> > --- a/ui/trace-events
>> > +++ b/ui/trace-events
>> > @@ -15,7 +15,7 @@ displaysurface_create_pixman(void *display_surface) "surface=%p"
>> > displaysurface_free(void *display_surface) "surface=%p"
>> > displaychangelistener_register(void *dcl, const char *name) "%p [ %s ]"
>> > displaychangelistener_unregister(void *dcl, const char *name) "%p [ %s ]"
>> > -ppm_save(int fd, void *display_surface) "fd=%d surface=%p"
>> > +ppm_save(int fd, void *image) "fd=%d image=%p"
>> >
>> > # gtk.c
>> > # gtk-gl-area.c
diff --git a/ui/console.c b/ui/console.c
index 57df3a5439..c5aabf5a5f 100644
--- a/ui/console.c
+++ b/ui/console.c
@@ -167,7 +167,6 @@ struct QemuConsole {
QEMUFIFO out_fifo;
uint8_t out_fifo_buf[16];
QEMUTimer *kbd_timer;
- Coroutine *screendump_co;
QTAILQ_ENTRY(QemuConsole) next;
};
@@ -261,14 +260,20 @@ static void gui_setup_refresh(DisplayState *ds)
ds->have_text = have_text;
}
-void graphic_hw_update_done(QemuConsole *con)
+static void graphic_hw_update_done_with_kick(QemuConsole *con,
+ Coroutine *kick_me)
{
- if (con && con->screendump_co) {
- aio_co_wake(con->screendump_co);
+ if (kick_me) {
+ aio_co_wake(kick_me);
}
}
-void graphic_hw_update(QemuConsole *con)
+void graphic_hw_update_done(QemuConsole *con)
+{
+ graphic_hw_update_done_with_kick(con, NULL);
+}
+
+static void graphic_hw_update_with_kick(QemuConsole *con, Coroutine *kick_me)
{
bool async = false;
if (!con) {
@@ -279,10 +284,15 @@ void graphic_hw_update(QemuConsole *con)
async = con->hw_ops->gfx_update_async;
}
if (!async) {
- graphic_hw_update_done(con);
+ graphic_hw_update_done_with_kick(con, kick_me);
}
}
+void graphic_hw_update(QemuConsole *con)
+{
+ graphic_hw_update_with_kick(con, NULL);
+}
+
void graphic_hw_gl_block(QemuConsole *con, bool block)
{
assert(con != NULL);
@@ -343,9 +353,16 @@ static bool ppm_save(int fd, pixman_image_t *image, Error **errp)
return true;
}
-static void graphic_hw_update_bh(void *con)
+typedef struct {
+ QemuConsole *con;
+ Coroutine *kick_me;
+} UpdateBHargs;
+
+static void graphic_hw_update_bh(void *p)
{
- graphic_hw_update(con);
+ UpdateBHargs *args = p;
+
+ graphic_hw_update_with_kick(args->con, args->kick_me);
}
/* may be called in coroutine context or not */
@@ -376,12 +393,13 @@ void qmp_screendump(const char *filename, bool has_device, const char *device,
}
if (qemu_in_coroutine()) {
- assert(!con->screendump_co);
- con->screendump_co = qemu_coroutine_self();
+ UpdateBHargs args = {
+ .con = con,
+ .kick_me = qemu_coroutine_self(),
+ };
aio_bh_schedule_oneshot(qemu_get_aio_context(),
- graphic_hw_update_bh, con);
+ graphic_hw_update_bh, &args);
qemu_coroutine_yield();
- con->screendump_co = NULL;
}
surface = qemu_console_surface(con);
^ permalink raw reply related
* [PATCH RESEND v6 10/16] userfaultfd: Don't retake mmap_sem to emulate NOPAGE
From: Peter Xu @ 2020-02-20 16:02 UTC (permalink / raw)
To: linux-mm, linux-kernel
Cc: Peter Xu, Martin Cracauer, Mike Rapoport, Hugh Dickins,
Jerome Glisse, Kirill A . Shutemov, Matthew Wilcox,
Pavel Emelyanov, Brian Geffon, Maya Gokhale, Denis Plotnikov,
Andrea Arcangeli, Johannes Weiner, Dr . David Alan Gilbert,
Linus Torvalds, Mike Kravetz, Marty McFadden, David Hildenbrand,
Bobby Powers, Mel Gorman
In-Reply-To: <20200220155353.8676-1-peterx@redhat.com>
This patch removes the risk path in handle_userfault() then we will be
sure that the callers of handle_mm_fault() will know that the VMAs
might have changed. Meanwhile with previous patch we don't lose
responsiveness as well since the core mm code now can handle the
nonfatal userspace signals even if we return VM_FAULT_RETRY.
Suggested-by: Andrea Arcangeli <aarcange@redhat.com>
Suggested-by: Linus Torvalds <torvalds@linux-foundation.org>
Reviewed-by: Jerome Glisse <jglisse@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
---
fs/userfaultfd.c | 24 ------------------------
1 file changed, 24 deletions(-)
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 37df7c9eedb1..888272621f38 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -524,30 +524,6 @@ vm_fault_t handle_userfault(struct vm_fault *vmf, unsigned long reason)
__set_current_state(TASK_RUNNING);
- if (return_to_userland) {
- if (signal_pending(current) &&
- !fatal_signal_pending(current)) {
- /*
- * If we got a SIGSTOP or SIGCONT and this is
- * a normal userland page fault, just let
- * userland return so the signal will be
- * handled and gdb debugging works. The page
- * fault code immediately after we return from
- * this function is going to release the
- * mmap_sem and it's not depending on it
- * (unlike gup would if we were not to return
- * VM_FAULT_RETRY).
- *
- * If a fatal signal is pending we still take
- * the streamlined VM_FAULT_RETRY failure path
- * and there's no need to retake the mmap_sem
- * in such case.
- */
- down_read(&mm->mmap_sem);
- ret = VM_FAULT_NOPAGE;
- }
- }
-
/*
* Here we race with the list_del; list_add in
* userfaultfd_ctx_read(), however because we don't ever run
--
2.24.1
^ permalink raw reply related
* Re: [PATCH v2 1/2] s390x: fix memleaks in cpu_finalize
From: Cornelia Huck @ 2020-02-20 15:59 UTC (permalink / raw)
To: pannengyuan
Cc: peter.maydell, zhang.zhanghailiang, alistair, qemu-devel,
mav2-rk.cave-ayland, qemu-arm, qemu-ppc, euler.robot,
edgar.iglesias, Richard Henderson, david
In-Reply-To: <20200217032127.46508-2-pannengyuan@huawei.com>
On Mon, 17 Feb 2020 11:21:26 +0800
<pannengyuan@huawei.com> wrote:
> From: Pan Nengyuan <pannengyuan@huawei.com>
>
> This patch fix memleaks when we call tests/qtest/cpu-plug-test on s390x. The leak stack is as follow:
>
> Direct leak of 48 byte(s) in 1 object(s) allocated from:
> #0 0x7fb43c7cd970 in __interceptor_calloc (/lib64/libasan.so.5+0xef970)
> #1 0x7fb43be2149d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5249d)
> #2 0x558ba96da716 in timer_new_full /mnt/sdb/qemu-new/qemu/include/qemu/timer.h:530
> #3 0x558ba96da716 in timer_new /mnt/sdb/qemu-new/qemu/include/qemu/timer.h:551
> #4 0x558ba96da716 in timer_new_ns /mnt/sdb/qemu-new/qemu/include/qemu/timer.h:569
> #5 0x558ba96da716 in s390_cpu_initfn /mnt/sdb/qemu-new/qemu/target/s390x/cpu.c:285
> #6 0x558ba9c969ab in object_init_with_type /mnt/sdb/qemu-new/qemu/qom/object.c:372
> #7 0x558ba9c9eb5f in object_initialize_with_type /mnt/sdb/qemu-new/qemu/qom/object.c:516
> #8 0x558ba9c9f053 in object_new_with_type /mnt/sdb/qemu-new/qemu/qom/object.c:684
> #9 0x558ba967ede6 in s390x_new_cpu /mnt/sdb/qemu-new/qemu/hw/s390x/s390-virtio-ccw.c:64
> #10 0x558ba99764b3 in hmp_cpu_add /mnt/sdb/qemu-new/qemu/hw/core/machine-hmp-cmds.c:57
> #11 0x558ba9b1c27f in handle_hmp_command /mnt/sdb/qemu-new/qemu/monitor/hmp.c:1082
> #12 0x558ba96c1b02 in qmp_human_monitor_command /mnt/sdb/qemu-new/qemu/monitor/misc.c:142
>
> Reported-by: Euler Robot <euler.robot@huawei.com>
> Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
> Cc: Richard Henderson <rth@twiddle.net>
> Cc: Cornelia Huck <cohuck@redhat.com>
> ---
> Changes v2 to v1:
> - Similarly to other cleanups, move timer_new into realize, then do
> timer_del in unrealize.
> ---
> target/s390x/cpu.c | 22 ++++++++++++++++++----
> 1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index cf84d307c6..f18dbc6fe4 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -170,7 +170,12 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)
> S390CPUClass *scc = S390_CPU_GET_CLASS(dev);
> #if !defined(CONFIG_USER_ONLY)
> S390CPU *cpu = S390_CPU(dev);
> + cpu->env.tod_timer =
> + timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu);
> + cpu->env.cpu_timer =
> + timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu);
I may be missing something, but what cleans up the timers if we fail
further down in this function? I don't think the unrealize callback is
invoked by the core in case of error?
Also, as a matter of personal preference, I think it would be better to
initialize the timers in the !CONFIG_USER_ONLY section further below,
rather than in the variable declaration section.
> #endif
> +
> Error *err = NULL;
>
> /* the model has to be realized before qemu_init_vcpu() due to kvm */
^ permalink raw reply
* ioctl seems to change errno behaviour in 5.6.0rc2
From: Antonio Larrosa @ 2020-02-20 16:02 UTC (permalink / raw)
To: Doug Gilbert, Jens Axboe; +Cc: linux-scsi
[-- Attachment #1: Type: text/plain, Size: 1110 bytes --]
Hello,
I noticed cdparanoia stopped working with kernel 5.6.0rc2 while it worked fine
with 5.5.2 .
Running as root `cdparanoia -v -d /dev/sr0 [0]` with 5.6.0rc2, gives the
following errors:
Testing /dev/sr0 for SCSI/MMC interface
no SG_IO support for device: /dev/sr0
Error trying to open /dev/sga exclusively (No such file or directory).
I checked that the sg module is loaded with both kernels and also did a diff of
the lsmod output with both kernels and didn't find anything suspicious.
After some tests, I did a small c application using code from cdparanoia where
it can be seen that the ioctl(fd, SG_IO, &hdr) call returns EINVAL in errno with
the 5.5.2 kernel but returns EFAULT with 5.6.0rc2 .
The code is attached and can be built just with `gcc test.c -o test` (note it's
hardcoded in main to use /dev/sr0, so it doesn't have any parameter).
Note that I'm not a cdparanoia developer (in fact, it seems to have been
unmaintained for many years), but I thought it might be interesting to report an
ioctl that changes the behaviour in different kernels.
Greetings,
--
Antonio Larrosa
[-- Attachment #2: test.c --]
[-- Type: text/x-csrc, Size: 1939 bytes --]
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <fcntl.h>
#include <string.h>
#include <sys/ioctl.h>
#include <scsi/sg.h>
int check_sgio(const char *device){
int fd;
struct sg_io_hdr hdr;
if (!device) return 0;
/* we don't really care what type of device it is -- if it can do
* SG_IO, then we'll put it through the normal mmc/atapi/etc tests
* later, but it's good enough for now. */
fd = open(device, O_RDWR|O_NONBLOCK);
if (fd < 0){
printf("Could not access device %s to test for SG_IO support\n",device);
return 0;
}
memset(&hdr, 0, sizeof (struct sg_io_hdr));
/* First try with interface_id = 'A'; for all but the sg case,
* that'll get us a -EINVAL if it supports SG_IO, and some other
* error for all other cases. */
hdr.interface_id = 'A';
if (ioctl(fd, SG_IO, &hdr)) {
int err=errno;
printf("errno: %d\n", err);
switch (err) {
case EINVAL: /* sr and ata give us EINVAL when SG_IO is
* supported but interface_id is bad. */
case ENOSYS: /* sg gives us ENOSYS when SG_IO is supported but
* interface_id is bad. IMHO, this is wrong and
* needs fixing in the kernel. */
close(fd);
return 1;
default: /* everything else gives ENOTTY, I think. I'm just
* going to be paranoid and reject everything else. */
close(fd);
return 0;
}
}
/* if we get here, something is dreadfuly wrong. ioctl(fd,SG_IO,&hdr)
* handled SG_IO, but took hdr.interface_id = 'A' as valid, and an empty
* command as good. Don't trust it. */
printf("closing\n");
close(fd);
return 0;
}
int main(int argc, char**argv)
{
const char *specialized_device = "/dev/sr0";
if(check_sgio(specialized_device)){
printf("SG_IO device: %s\n",specialized_device);
}else{
printf("no SG_IO support for device: %s\n",specialized_device);
}
return 0;
};
^ permalink raw reply
* Re: [dpdk-dev] [PATCH 4/4] ci: reorganise Travis jobs
From: David Marchand @ 2020-02-20 16:01 UTC (permalink / raw)
To: ci; +Cc: Thomas Monjalon, dev, Michael Santana, Aaron Conole,
Bruce Richardson
In-Reply-To: <f7tv9o19viw.fsf@dhcp-25.97.bos.redhat.com>
On Thu, Feb 20, 2020 at 3:35 PM Aaron Conole <aconole@redhat.com> wrote:
>
> David Marchand <david.marchand@redhat.com> writes:
>
> > On Thu, Feb 20, 2020 at 11:42 AM Thomas Monjalon <thomas@monjalon.net> wrote:
> >>
> >> 19/02/2020 22:39, Aaron Conole:
> >> > David Marchand <david.marchand@redhat.com> writes:
> >> >
> >> > > Let's prune the jobs list to limit the amount of time spent by the robot
> >> > > in Travis.
> >> > >
> >> > > Since meson enables automatically the relevant components, there is not
> >> > > much gain in testing with extra_packages vs required_packages only.
> >> > >
> >> > > For a given arch/compiler/env combination, compilation is first tested
> >> > > in all jobs that run tests or build the docs or run the ABI checks.
> >> > > In the same context, for jobs that accumulates running tests, building
> >> > > the docs etc..., those steps are independent and can be split to save
> >> > > some cpu on Travis.
> >> > >
> >> > > With this, we go down from 21 to 15 jobs.
> >> > >
> >> > > Note: this patch requires a flush of the existing caches in Travis.
> >> > >
> >> > > Signed-off-by: David Marchand <david.marchand@redhat.com>
> >> > > ---
> >> >
> >> > In general, I think the idea with required vs. extra was to have a build
> >> > that did the minimum required, and one that did all the packages (to
> >> > allow a minimum vs. full DPDK).
> >> >
> >> > At least, that's from
> >> > http://mails.dpdk.org/archives/dev/2019-January/124007.html
> >>
> >> I think the benefit of a minimum build is to have a quick report,
> >> and easy to setup.
> >
> > Yes, Travis serves as a first gate when submitting patches.
> > But since Travis is best effort/free, we can't have a full coverage.
> >
> >
> >> > Not sure if that's still something anyone cares about.
> >>
> >> Given that Travis knows how to satisfy the dependencies,
> >> and that we must wait for all jobs to finish,
> >> I don't see any benefit of a minimal setup.
> >
> > This minimal setup also tests that dpdk dependencies are correct.
> > If a change makes something rely on libX and libX is in the packages
> > always installed in Travis, the missing dependency would not get
> > caught.
> >
> > But here, this adds too many jobs.
> >
> > UNH, Intel and other CIs should step in and fill this kind of gap.
>
> Okay, makes sense to me. Are one of these CI providers offering to
> cover this?
Maybe it is already covered, the best is to ask, so sending to ci@dpdk.org.
For the CI guys, which packages are installed on the systems/vms that
do compilation tests?
Is it possible to have a summary of the different setups?
Thanks.
--
David Marchand
^ permalink raw reply
* Re: [PATCH v4 5/6] driver core: Rename deferred_probe_timeout and make it global
From: Bjorn Andersson @ 2020-02-20 16:00 UTC (permalink / raw)
To: John Stultz
Cc: lkml, Rob Herring, Rafael J. Wysocki, Kevin Hilman, Ulf Hansson,
Pavel Machek, Len Brown, Todd Kjos, Liam Girdwood, Mark Brown,
Thierry Reding, Linus Walleij, Greg Kroah-Hartman, linux-pm
In-Reply-To: <20200220050440.45878-6-john.stultz@linaro.org>
On Wed 19 Feb 21:04 PST 2020, John Stultz wrote:
> Since other subsystems (like regulator) have similar arbitrary
> timeouts for how long they try to resolve driver dependencies,
> rename deferred_probe_timeout to driver_deferred_probe_timeout
> and set it as global, so it can be shared.
>
> Cc: Rob Herring <robh@kernel.org>
> Cc: "Rafael J. Wysocki" <rjw@rjwysocki.net>
> Cc: Kevin Hilman <khilman@kernel.org>
> Cc: Ulf Hansson <ulf.hansson@linaro.org>
> Cc: Pavel Machek <pavel@ucw.cz>
> Cc: Len Brown <len.brown@intel.com>
> Cc: Todd Kjos <tkjos@google.com>
> Cc: Bjorn Andersson <bjorn.andersson@linaro.org>
> Cc: Liam Girdwood <lgirdwood@gmail.com>
> Cc: Mark Brown <broonie@kernel.org>
> Cc: Thierry Reding <treding@nvidia.com>
> Cc: Linus Walleij <linus.walleij@linaro.org>
> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
> Cc: linux-pm@vger.kernel.org
> Signed-off-by: John Stultz <john.stultz@linaro.org>
> Change-Id: I92ee3b392004ecc9217c5337b54eda48c2d7f3ee
Change-Id...
Reviewed-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Regards,
Bjorn
> ---
> v4:
> * Split out into its own patch as suggested by Mark
> * Renamed deferred_probe_timeout as suggested by Greg
> ---
> drivers/base/dd.c | 18 ++++++++++--------
> include/linux/device/driver.h | 1 +
> 2 files changed, 11 insertions(+), 8 deletions(-)
>
> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
> index 408e4da081da..39f1ce6d4f1c 100644
> --- a/drivers/base/dd.c
> +++ b/drivers/base/dd.c
> @@ -229,17 +229,19 @@ DEFINE_SHOW_ATTRIBUTE(deferred_devs);
> * In the case of modules, set the default probe timeout to
> * 30 seconds to give userland some time to load needed modules
> */
> -static int deferred_probe_timeout = 30;
> +int driver_deferred_probe_timeout = 30;
> #else
> /* In the case of !modules, no probe timeout needed */
> -static int deferred_probe_timeout = -1;
> +int driver_deferred_probe_timeout = -1;
> #endif
> +EXPORT_SYMBOL_GPL(driver_deferred_probe_timeout);
> +
> static int __init deferred_probe_timeout_setup(char *str)
> {
> int timeout;
>
> if (!kstrtoint(str, 10, &timeout))
> - deferred_probe_timeout = timeout;
> + driver_deferred_probe_timeout = timeout;
> return 1;
> }
> __setup("deferred_probe_timeout=", deferred_probe_timeout_setup);
> @@ -259,10 +261,10 @@ __setup("deferred_probe_timeout=", deferred_probe_timeout_setup);
> */
> int driver_deferred_probe_check_state(struct device *dev)
> {
> - if (!initcalls_done || deferred_probe_timeout > 0)
> + if (!initcalls_done || driver_deferred_probe_timeout > 0)
> return -EPROBE_DEFER;
>
> - if (!deferred_probe_timeout) {
> + if (!driver_deferred_probe_timeout) {
> dev_WARN(dev, "deferred probe timeout, ignoring dependency");
> return -ETIMEDOUT;
> }
> @@ -276,7 +278,7 @@ static void deferred_probe_timeout_work_func(struct work_struct *work)
> {
> struct device_private *private, *p;
>
> - deferred_probe_timeout = 0;
> + driver_deferred_probe_timeout = 0;
> driver_deferred_probe_trigger();
> flush_work(&deferred_probe_work);
>
> @@ -310,9 +312,9 @@ static int deferred_probe_initcall(void)
> driver_deferred_probe_trigger();
> flush_work(&deferred_probe_work);
>
> - if (deferred_probe_timeout > 0) {
> + if (driver_deferred_probe_timeout > 0) {
> schedule_delayed_work(&deferred_probe_timeout_work,
> - deferred_probe_timeout * HZ);
> + driver_deferred_probe_timeout * HZ);
> }
> return 0;
> }
> diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h
> index 5242afabfaba..ee7ba5b5417e 100644
> --- a/include/linux/device/driver.h
> +++ b/include/linux/device/driver.h
> @@ -236,6 +236,7 @@ driver_find_device_by_acpi_dev(struct device_driver *drv, const void *adev)
> }
> #endif
>
> +extern int driver_deferred_probe_timeout;
> void driver_deferred_probe_add(struct device *dev);
> int driver_deferred_probe_check_state(struct device *dev);
> void driver_init(void);
> --
> 2.17.1
>
^ permalink raw reply
* Re: [PATCH v2 1/2] s390x: fix memleaks in cpu_finalize
From: Cornelia Huck @ 2020-02-20 15:59 UTC (permalink / raw)
To: pannengyuan
Cc: peter.maydell, zhang.zhanghailiang, alistair, qemu-devel,
mav2-rk.cave-ayland, qemu-arm, qemu-ppc, euler.robot,
Richard Henderson, balrogg, david
In-Reply-To: <20200217032127.46508-2-pannengyuan@huawei.com>
On Mon, 17 Feb 2020 11:21:26 +0800
<pannengyuan@huawei.com> wrote:
> From: Pan Nengyuan <pannengyuan@huawei.com>
>
> This patch fix memleaks when we call tests/qtest/cpu-plug-test on s390x. The leak stack is as follow:
>
> Direct leak of 48 byte(s) in 1 object(s) allocated from:
> #0 0x7fb43c7cd970 in __interceptor_calloc (/lib64/libasan.so.5+0xef970)
> #1 0x7fb43be2149d in g_malloc0 (/lib64/libglib-2.0.so.0+0x5249d)
> #2 0x558ba96da716 in timer_new_full /mnt/sdb/qemu-new/qemu/include/qemu/timer.h:530
> #3 0x558ba96da716 in timer_new /mnt/sdb/qemu-new/qemu/include/qemu/timer.h:551
> #4 0x558ba96da716 in timer_new_ns /mnt/sdb/qemu-new/qemu/include/qemu/timer.h:569
> #5 0x558ba96da716 in s390_cpu_initfn /mnt/sdb/qemu-new/qemu/target/s390x/cpu.c:285
> #6 0x558ba9c969ab in object_init_with_type /mnt/sdb/qemu-new/qemu/qom/object.c:372
> #7 0x558ba9c9eb5f in object_initialize_with_type /mnt/sdb/qemu-new/qemu/qom/object.c:516
> #8 0x558ba9c9f053 in object_new_with_type /mnt/sdb/qemu-new/qemu/qom/object.c:684
> #9 0x558ba967ede6 in s390x_new_cpu /mnt/sdb/qemu-new/qemu/hw/s390x/s390-virtio-ccw.c:64
> #10 0x558ba99764b3 in hmp_cpu_add /mnt/sdb/qemu-new/qemu/hw/core/machine-hmp-cmds.c:57
> #11 0x558ba9b1c27f in handle_hmp_command /mnt/sdb/qemu-new/qemu/monitor/hmp.c:1082
> #12 0x558ba96c1b02 in qmp_human_monitor_command /mnt/sdb/qemu-new/qemu/monitor/misc.c:142
>
> Reported-by: Euler Robot <euler.robot@huawei.com>
> Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
> Cc: Richard Henderson <rth@twiddle.net>
> Cc: Cornelia Huck <cohuck@redhat.com>
> ---
> Changes v2 to v1:
> - Similarly to other cleanups, move timer_new into realize, then do
> timer_del in unrealize.
> ---
> target/s390x/cpu.c | 22 ++++++++++++++++++----
> 1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
> index cf84d307c6..f18dbc6fe4 100644
> --- a/target/s390x/cpu.c
> +++ b/target/s390x/cpu.c
> @@ -170,7 +170,12 @@ static void s390_cpu_realizefn(DeviceState *dev, Error **errp)
> S390CPUClass *scc = S390_CPU_GET_CLASS(dev);
> #if !defined(CONFIG_USER_ONLY)
> S390CPU *cpu = S390_CPU(dev);
> + cpu->env.tod_timer =
> + timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_tod_timer, cpu);
> + cpu->env.cpu_timer =
> + timer_new_ns(QEMU_CLOCK_VIRTUAL, s390x_cpu_timer, cpu);
I may be missing something, but what cleans up the timers if we fail
further down in this function? I don't think the unrealize callback is
invoked by the core in case of error?
Also, as a matter of personal preference, I think it would be better to
initialize the timers in the !CONFIG_USER_ONLY section further below,
rather than in the variable declaration section.
> #endif
> +
> Error *err = NULL;
>
> /* the model has to be realized before qemu_init_vcpu() due to kvm */
^ permalink raw reply
* [Buildroot] [PATCH v3 10/10] docs/manual/adding-packages-cargo: Update for legal-info
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
Added a bit of explanations regarding the vendored licenses.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
docs/manual/adding-packages-cargo.txt | 45 ++++++++++++++++++++-------
1 file changed, 34 insertions(+), 11 deletions(-)
diff --git a/docs/manual/adding-packages-cargo.txt b/docs/manual/adding-packages-cargo.txt
index e7d03eb7b5..e92b8bc353 100644
--- a/docs/manual/adding-packages-cargo.txt
+++ b/docs/manual/adding-packages-cargo.txt
@@ -38,19 +38,25 @@ with an example:
04: #
05: ################################################################################
06:
-07: FOO_VERSION = 1.0
-08: FOO_SOURCE = foo-$(FOO_VERSION).tar.gz
-09: FOO_SITE = http://www.foosoftware.org/download
-10: FOO_LICENSE = GPL-3.0+
-11: FOO_LICENSE_FILES = COPYING
-12:
-13: $(eval $(cargo-package))
+07: include package/foo/vendor.inc
+08:
+09: FOO_VERSION = 1.0
+10: FOO_SOURCE = foo-$(FOO_VERSION).tar.gz
+11: FOO_SITE = http://www.foosoftware.org/download
+12: FOO_LICENSE = GPL-3.0+
+13: FOO_LICENSE_FILES = COPYING $(FOO_VENDOR_LICENSE_FILES)
+14:
+15: $(eval $(cargo-package))
--------------------------------
-The Makefile starts with the definition of the standard variables for package
-declaration (lines 7 to 11).
+The Makefile starts with the inclusion of a second file that
+contains the vendored licenses, more info on that topic later.
+Then it continues with the definition of the standard variables
+for package declaration (lines 9 to 13).
-As seen in line 13, it is based on the cargo-package infrastructure. Cargo will
+FOO_VENDOR_LICENSE_FILES is defined in the previously included file vendor.inc
+
+As seen in line 15, it is based on the cargo-package infrastructure. Cargo will
be invoked automatically by this infrastructure. The required dependencies of the
crate will be downloaded and the buildroot dependencies will also be set.
@@ -58,7 +64,20 @@ It is still possible to define custom build commands or install commands (i.e.
with FOO_BUILD_CMDS and FOO_INSTALL_TARGET_CMDS).
Those will then replace the commands from the cargo infrastructure.
-==== About Dependencies Management
+The file vendor.inc contains only one variable :
+
+------------------------------
+01: FOO_VENDOR_LICENSES_FILES = \
+02: VENDOR/bar/COPYING \
+03: VENDOR/blah/LICENSE-MIT \
+04: VENDOR/bleh/COPYING \
+05:
+--------------------------------
+
+It is recommended to generate the list via scripting.
+A script example can be found in the ripgrep directory.
+
+==== About Dependencies Management and Licenses
A crate can depend on other libraries from crates.io or git repositories, listed
in its Cargo.toml file. Before starting a build, the +cargo vendor+ command
@@ -66,3 +85,7 @@ will be run and will take care of downloading those dependencies.
The final source tarball will contain all the required dependencies, effectively
permitting us to do an offline build.
+
+In order for the +legal-info+ target to work as expected, it is necessary to list
+the licenses of the vendored dependencies. Each of those entries should be added
+to the +.mk+ and +.hash+ files.
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 09/10] package/ripgrep: add legal-info for dependencies
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
As the dependencies are now vendored in the tarball, we can
extract them easily. Added a helper script to list them.
Updated the .hash & .mk files with those new licenses.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
package/ripgrep/find_vendor_licenses.sh | 6 +
package/ripgrep/ripgrep.hash | 167 +++++++++++++++++++++++
package/ripgrep/ripgrep.mk | 4 +-
package/ripgrep/vendor.inc | 169 ++++++++++++++++++++++++
4 files changed, 345 insertions(+), 1 deletion(-)
create mode 100755 package/ripgrep/find_vendor_licenses.sh
create mode 100644 package/ripgrep/vendor.inc
diff --git a/package/ripgrep/find_vendor_licenses.sh b/package/ripgrep/find_vendor_licenses.sh
new file mode 100755
index 0000000000..fd8638e399
--- /dev/null
+++ b/package/ripgrep/find_vendor_licenses.sh
@@ -0,0 +1,6 @@
+#!/bin/sh
+# to run from the extracted final sources
+find VENDOR/ -type f \
+ \( -iname '*license*' -o -iname 'licence*' -o -iname 'copying*' -o -iname 'copyright*' \) \
+ -a -not -name '*.a' | \
+ sort
diff --git a/package/ripgrep/ripgrep.hash b/package/ripgrep/ripgrep.hash
index 1e70cbae5a..cad9fdc865 100644
--- a/package/ripgrep/ripgrep.hash
+++ b/package/ripgrep/ripgrep.hash
@@ -1,3 +1,170 @@
# Locally calculated
sha256 c02fe3077c95872175b098d002f872a619195436569a7dff443bbb605bc27474 ripgrep-11.0.1.tar.gz
sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f LICENSE-MIT
+# Locally calculated with:
+# for i in $(find_vendor_licenses.sh) ; do echo -n "sha256 " ; sha256sum $i ; done | sort --key=3
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/aho-corasick/COPYING
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/aho-corasick/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/aho-corasick/UNLICENSE
+sha256 f3f8d32084848316048c5a1e125a3c5003eb32145a5f5f2a0d5586377324f9ba VENDOR/atty/LICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/autocfg/LICENSE-APACHE
+sha256 27995d58ad5c1145c1a8cd86244ce844886958a35eb2b78c6b772748669999ac VENDOR/autocfg/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/base64/LICENSE-APACHE
+sha256 0dd882e53de11566d50f8e8e2d5a651bcf3fabee4987d70f306233cf39094ba7 VENDOR/base64/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/bitflags/LICENSE-APACHE
+sha256 6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb VENDOR/bitflags/LICENSE-MIT
+sha256 13361f93b3db60ddb43f8fdc97ba469461019bbf7589f6bde833eb3d995d2497 VENDOR/bstr/COPYING
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/bstr/LICENSE-APACHE
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/bstr/LICENSE-MIT
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 VENDOR/bytecount/LICENSE.Apache2
+sha256 a5dea80c1f383cb5f80a6bb0da5e55a2beb9f24adb123ce6300af2cbaaa3bf65 VENDOR/bytecount/LICENSE.MIT
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/byteorder/COPYING
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/byteorder/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/byteorder/UNLICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/cc/LICENSE-APACHE
+sha256 378f5840b258e2779c39418f3f2d7b2ba96f1c7917dd6be0713f88305dbda397 VENDOR/cc/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/cfg-if/LICENSE-APACHE
+sha256 378f5840b258e2779c39418f3f2d7b2ba96f1c7917dd6be0713f88305dbda397 VENDOR/cfg-if/LICENSE-MIT
+sha256 6725d1437fc6c77301f2ff0e7d52914cf4f9509213e1078dc77d9356dbe6eac5 VENDOR/clap/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/crossbeam-channel/LICENSE-APACHE
+sha256 23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 VENDOR/crossbeam-channel/LICENSE-MIT
+sha256 924a49392dc8304def57586be4ebd69aaf51e16fd245b55b4b69ad2cce6b715a VENDOR/crossbeam-channel/LICENSE-THIRD-PARTY
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/crossbeam-utils/LICENSE-APACHE
+sha256 23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 VENDOR/crossbeam-utils/LICENSE-MIT
+sha256 8b98376eb373dcf81950474efe34b5576a8171460dff500cc58a1ed8d160cd57 VENDOR/encoding_rs/COPYRIGHT
+sha256 13361f93b3db60ddb43f8fdc97ba469461019bbf7589f6bde833eb3d995d2497 VENDOR/encoding_rs_io/COPYING
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/encoding_rs_io/LICENSE-APACHE
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/encoding_rs_io/LICENSE-MIT
+sha256 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30 VENDOR/encoding_rs/LICENSE-APACHE
+sha256 f2ad48641d9c997d9ae3b95d93d1cd6e1ab12ab4c44de89937c7bfabbd076a4a VENDOR/encoding_rs/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/fnv/LICENSE-APACHE
+sha256 65fdb6c76cd61612070c066eec9ecdb30ee74fb27859d0d9af58b9f499fd0c3e VENDOR/fnv/LICENSE-MIT
+sha256 03b114f53e6587a398931762ee11e2395bfdba252a329940e2c8c9e81813845b VENDOR/fuchsia-cprng/LICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/glob/LICENSE-APACHE
+sha256 6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb VENDOR/glob/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/itoa/LICENSE-APACHE
+sha256 23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 VENDOR/itoa/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/lazy_static/LICENSE-APACHE
+sha256 0621878e61f0d0fda054bcbe02df75192c28bde1ecc8289cbd86aeba2dd72720 VENDOR/lazy_static/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/libc/LICENSE-APACHE
+sha256 6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb VENDOR/libc/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/log/LICENSE-APACHE
+sha256 6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb VENDOR/log/LICENSE-MIT
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/memchr/COPYING
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/memchr/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/memchr/UNLICENSE
+sha256 04ea4849dba9dcae07113850c6f1b1a69052c625210639914eee352023f750ad VENDOR/memmap/LICENSE-APACHE
+sha256 bd1d8f06a6ce1f7645991e347b95864b970eed43624305eb4bb78c09aef8692d VENDOR/memmap/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/num_cpus/LICENSE-APACHE
+sha256 0593d22d122d4bfec6407115e3907546312976f75473417aaa4c57ecd2095ae6 VENDOR/num_cpus/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/packed_simd/LICENSE-APACHE
+sha256 6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb VENDOR/packed_simd/LICENSE-MIT
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/pcre2/COPYING
+sha256 cb3c929a05e6cbc9de9ab06a4c57eeb60ca8c724bef6c138c87d3a577e27aa14 VENDOR/pcre2/LICENSE-MIT
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/pcre2-sys/COPYING
+sha256 cb3c929a05e6cbc9de9ab06a4c57eeb60ca8c724bef6c138c87d3a577e27aa14 VENDOR/pcre2-sys/LICENSE-MIT
+sha256 46cde7dc11e64c78d650b4851b88f6704b4665ff60f22a1caf68ceb15e217e5b VENDOR/pcre2-sys/pcre2/cmake/COPYING-CMAKE-SCRIPTS
+sha256 99272c55f3dcfa07a8a7e15a5c1a33096e4727de74241d65fa049fccfdd59507 VENDOR/pcre2-sys/pcre2/COPYING
+sha256 c4a8b89cd38d6a7501d5b11a472fa15e71a051b66d6331c6cda364101389d6ee VENDOR/pcre2-sys/pcre2/LICENCE
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/pcre2-sys/UNLICENSE
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/pcre2/UNLICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/pkg-config/LICENSE-APACHE
+sha256 378f5840b258e2779c39418f3f2d7b2ba96f1c7917dd6be0713f88305dbda397 VENDOR/pkg-config/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/proc-macro2/LICENSE-APACHE
+sha256 378f5840b258e2779c39418f3f2d7b2ba96f1c7917dd6be0713f88305dbda397 VENDOR/proc-macro2/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/quote/LICENSE-APACHE
+sha256 c9a75f18b9ab2927829a208fc6aa2cf4e63b8420887ba29cdb265d6619ae82d5 VENDOR/quote/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_chacha/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_chacha/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand_chacha/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand/COPYRIGHT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_core-0.3.1/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_core-0.3.1/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand_core-0.3.1/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_core/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_core/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand_core/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_hc/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_hc/LICENSE-APACHE
+sha256 a771e4354f6b3ad4c92da1a5c9a239b6c291527db869632ecea4f20e24ca1135 VENDOR/rand_hc/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_isaac/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_isaac/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand_isaac/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_jitter/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_jitter/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand_jitter/LICENSE-MIT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_os/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_os/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand_os/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_pcg/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_pcg/LICENSE-APACHE
+sha256 2234e3cefee876aeb686ad89e978bdb07bf118a1186ab1cf161bcdf69d4b4f57 VENDOR/rand_pcg/LICENSE-MIT
+sha256 90eb64f0279b0d9432accfa6023ff803bc4965212383697eee27a0f426d5f8d5 VENDOR/rand_xorshift/COPYRIGHT
+sha256 aaff376532ea30a0cd5330b9502ad4a4c8bf769c539c87ffe78819d188a18ebf VENDOR/rand_xorshift/LICENSE-APACHE
+sha256 209fbbe0ad52d9235e37badf9cadfe4dbdc87203179c0899e738b39ade42177b VENDOR/rand_xorshift/LICENSE-MIT
+sha256 00d7b0c8bf95ea93162fccc84da96b906b15add708eade04f7ee6141f7b53141 VENDOR/rdrand/LICENSE
+sha256 efcfee7981ff72431fffb06925cad00a23dce079ed4354f61030ad5abdb78829 VENDOR/redox_syscall/LICENSE
+sha256 cb46b697c3fd9d27d7bfe1b1ad48f8a58a284984504c6eb215ae2164538df7cb VENDOR/redox_termios/LICENSE
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/regex-automata/COPYING
+sha256 58cf078acc03da3e280a938c2bd9943f554fc9b6ced89ad93ba35ca436872899 VENDOR/regex-automata/data/fowler-tests/LICENSE
+sha256 58cf078acc03da3e280a938c2bd9943f554fc9b6ced89ad93ba35ca436872899 VENDOR/regex-automata/data/tests/fowler/LICENSE
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/regex-automata/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/regex-automata/UNLICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/regex/LICENSE-APACHE
+sha256 6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb VENDOR/regex/LICENSE-MIT
+sha256 58cf078acc03da3e280a938c2bd9943f554fc9b6ced89ad93ba35ca436872899 VENDOR/regex/src/testdata/LICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/regex-syntax/LICENSE-APACHE
+sha256 6485b8ed310d3f0340bf1ad1f47645069ce4069dcc6bb46c7d5c6faf41de1fdb VENDOR/regex-syntax/LICENSE-MIT
+sha256 74db5baf44a41b1000312c673544b3374e4198af5605c7f9080a402cec42cfa3 VENDOR/regex-syntax/src/unicode_tables/LICENSE-UNICODE
+sha256 c6c8c9dbe29fb4d68d829c7a402f9f6baae3472ecf107cc2a57c75a9a8d1b85c VENDOR/remove_dir_all/LICENCE-APACHE
+sha256 db264505cb1856383e255c8373da9e5aeadc1cd92b570fcc94fd1fb7d892db78 VENDOR/remove_dir_all/LICENCE-MIT
+sha256 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 VENDOR/ryu/LICENSE-APACHE
+sha256 c9bff75738922193e67fa726fa225535870d2aa1059f91452c411736284ad566 VENDOR/ryu/LICENSE-BOOST
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/same-file/COPYING
+sha256 cb3c929a05e6cbc9de9ab06a4c57eeb60ca8c724bef6c138c87d3a577e27aa14 VENDOR/same-file/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/same-file/UNLICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/serde_derive/LICENSE-APACHE
+sha256 23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 VENDOR/serde_derive/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/serde_json/LICENSE-APACHE
+sha256 23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 VENDOR/serde_json/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/serde/LICENSE-APACHE
+sha256 23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 VENDOR/serde/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/smallvec/LICENSE-APACHE
+sha256 0b28172679e0009b655da42797c03fd163a3379d5cfa67ba1f1655e974a2a1a9 VENDOR/smallvec/LICENSE-MIT
+sha256 1738b51502ae831fb59ffbeb22ebdd90bf17e5c72fe57c00b47552415f133fd8 VENDOR/strsim/LICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/syn/LICENSE-APACHE
+sha256 23f18e03dc49df91622fe2a76176497404e46ced8a715d9d2b67a7446571cca3 VENDOR/syn/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/tempfile/LICENSE-APACHE
+sha256 8b427f5bc501764575e52ba4f9d95673cf8f6d80a86d0d06599852e1a9a20a36 VENDOR/tempfile/LICENSE-MIT
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/termcolor/COPYING
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/termcolor/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/termcolor/UNLICENSE
+sha256 6252f0c8d4a0df9b2dc0c6464cb2489dbe8859b0eb727e19c14e6af1ee432394 VENDOR/termion/LICENSE
+sha256 ce93600c49fbb3e14df32efe752264644f6a2f8e08a735ba981725799e5309ef VENDOR/textwrap/LICENSE
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/thread_local/LICENSE-APACHE
+sha256 c9a75f18b9ab2927829a208fc6aa2cf4e63b8420887ba29cdb265d6619ae82d5 VENDOR/thread_local/LICENSE-MIT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/ucd-util/LICENSE-APACHE
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/ucd-util/LICENSE-MIT
+sha256 ea8df489e83b7674d5c9d50c320bec3b15d2e2eeb7a3a8879a666d91e5640c16 VENDOR/ucd-util/LICENSE-UNICODE
+sha256 23860c2a7b5d96b21569afedf033469bab9fe14a1b24a35068b8641c578ce24d VENDOR/unicode-width/COPYRIGHT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/unicode-width/LICENSE-APACHE
+sha256 7b63ecd5f1902af1b63729947373683c32745c16a10e8e6292e2e2dcd7e90ae0 VENDOR/unicode-width/LICENSE-MIT
+sha256 23860c2a7b5d96b21569afedf033469bab9fe14a1b24a35068b8641c578ce24d VENDOR/unicode-xid/COPYRIGHT
+sha256 a60eea817514531668d7e00765731449fe14d059d3249e0bc93b36de45f759f2 VENDOR/unicode-xid/LICENSE-APACHE
+sha256 7b63ecd5f1902af1b63729947373683c32745c16a10e8e6292e2e2dcd7e90ae0 VENDOR/unicode-xid/LICENSE-MIT
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/utf8-ranges/COPYING
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/utf8-ranges/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/utf8-ranges/UNLICENSE
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/walkdir/COPYING
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/walkdir/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/walkdir/UNLICENSE
+sha256 b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1 VENDOR/winapi/LICENSE-APACHE
+sha256 ce7bc3499fee93d5022ef430d5e4201e79a6d9154f3974e42f41349f0569e09b VENDOR/winapi/LICENSE-MIT
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/winapi-util/COPYING
+sha256 cb3c929a05e6cbc9de9ab06a4c57eeb60ca8c724bef6c138c87d3a577e27aa14 VENDOR/winapi-util/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/winapi-util/UNLICENSE
+sha256 01c266bced4a434da0051174d6bee16a4c82cf634e2679b6155d40d75012390f VENDOR/wincolor/COPYING
+sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f VENDOR/wincolor/LICENSE-MIT
+sha256 7e12e5df4bae12cb21581ba157ced20e1986a0508dd10d0e8a4ab9a4cf94e85c VENDOR/wincolor/UNLICENSE
diff --git a/package/ripgrep/ripgrep.mk b/package/ripgrep/ripgrep.mk
index 1023b4bad8..3be143e15c 100644
--- a/package/ripgrep/ripgrep.mk
+++ b/package/ripgrep/ripgrep.mk
@@ -4,9 +4,11 @@
#
################################################################################
+include package/ripgrep/vendor.inc
+
RIPGREP_VERSION = 11.0.1
RIPGREP_SITE = $(call github,burntsushi,ripgrep,$(RIPGREP_VERSION))
RIPGREP_LICENSE = MIT
-RIPGREP_LICENSE_FILES = LICENSE-MIT
+RIPGREP_LICENSE_FILES = LICENSE-MIT $(RIPGREP_VENDOR_LICENSE_FILES)
$(eval $(cargo-package))
diff --git a/package/ripgrep/vendor.inc b/package/ripgrep/vendor.inc
new file mode 100644
index 0000000000..df131ac533
--- /dev/null
+++ b/package/ripgrep/vendor.inc
@@ -0,0 +1,169 @@
+# Generated with:
+# (echo "RIPGREP_VENDOR_LICENSE_FILES = \\" ; \
+# find_vendor_licenses.sh | sed 's%^\(.*\)$%\t\1 \\%')
+RIPGREP_VENDOR_LICENSE_FILES = \
+ VENDOR/aho-corasick/COPYING \
+ VENDOR/aho-corasick/LICENSE-MIT \
+ VENDOR/aho-corasick/UNLICENSE \
+ VENDOR/atty/LICENSE \
+ VENDOR/autocfg/LICENSE-APACHE \
+ VENDOR/autocfg/LICENSE-MIT \
+ VENDOR/base64/LICENSE-APACHE \
+ VENDOR/base64/LICENSE-MIT \
+ VENDOR/bitflags/LICENSE-APACHE \
+ VENDOR/bitflags/LICENSE-MIT \
+ VENDOR/bstr/COPYING \
+ VENDOR/bstr/LICENSE-APACHE \
+ VENDOR/bstr/LICENSE-MIT \
+ VENDOR/bytecount/LICENSE.Apache2 \
+ VENDOR/bytecount/LICENSE.MIT \
+ VENDOR/byteorder/COPYING \
+ VENDOR/byteorder/LICENSE-MIT \
+ VENDOR/byteorder/UNLICENSE \
+ VENDOR/cc/LICENSE-APACHE \
+ VENDOR/cc/LICENSE-MIT \
+ VENDOR/cfg-if/LICENSE-APACHE \
+ VENDOR/cfg-if/LICENSE-MIT \
+ VENDOR/clap/LICENSE-MIT \
+ VENDOR/crossbeam-channel/LICENSE-APACHE \
+ VENDOR/crossbeam-channel/LICENSE-MIT \
+ VENDOR/crossbeam-channel/LICENSE-THIRD-PARTY \
+ VENDOR/crossbeam-utils/LICENSE-APACHE \
+ VENDOR/crossbeam-utils/LICENSE-MIT \
+ VENDOR/encoding_rs/COPYRIGHT \
+ VENDOR/encoding_rs_io/COPYING \
+ VENDOR/encoding_rs_io/LICENSE-APACHE \
+ VENDOR/encoding_rs_io/LICENSE-MIT \
+ VENDOR/encoding_rs/LICENSE-APACHE \
+ VENDOR/encoding_rs/LICENSE-MIT \
+ VENDOR/fnv/LICENSE-APACHE \
+ VENDOR/fnv/LICENSE-MIT \
+ VENDOR/fuchsia-cprng/LICENSE \
+ VENDOR/glob/LICENSE-APACHE \
+ VENDOR/glob/LICENSE-MIT \
+ VENDOR/itoa/LICENSE-APACHE \
+ VENDOR/itoa/LICENSE-MIT \
+ VENDOR/lazy_static/LICENSE-APACHE \
+ VENDOR/lazy_static/LICENSE-MIT \
+ VENDOR/libc/LICENSE-APACHE \
+ VENDOR/libc/LICENSE-MIT \
+ VENDOR/log/LICENSE-APACHE \
+ VENDOR/log/LICENSE-MIT \
+ VENDOR/memchr/COPYING \
+ VENDOR/memchr/LICENSE-MIT \
+ VENDOR/memchr/UNLICENSE \
+ VENDOR/memmap/LICENSE-APACHE \
+ VENDOR/memmap/LICENSE-MIT \
+ VENDOR/num_cpus/LICENSE-APACHE \
+ VENDOR/num_cpus/LICENSE-MIT \
+ VENDOR/packed_simd/LICENSE-APACHE \
+ VENDOR/packed_simd/LICENSE-MIT \
+ VENDOR/pcre2/COPYING \
+ VENDOR/pcre2/LICENSE-MIT \
+ VENDOR/pcre2-sys/COPYING \
+ VENDOR/pcre2-sys/LICENSE-MIT \
+ VENDOR/pcre2-sys/pcre2/cmake/COPYING-CMAKE-SCRIPTS \
+ VENDOR/pcre2-sys/pcre2/COPYING \
+ VENDOR/pcre2-sys/pcre2/LICENCE \
+ VENDOR/pcre2-sys/UNLICENSE \
+ VENDOR/pcre2/UNLICENSE \
+ VENDOR/pkg-config/LICENSE-APACHE \
+ VENDOR/pkg-config/LICENSE-MIT \
+ VENDOR/proc-macro2/LICENSE-APACHE \
+ VENDOR/proc-macro2/LICENSE-MIT \
+ VENDOR/quote/LICENSE-APACHE \
+ VENDOR/quote/LICENSE-MIT \
+ VENDOR/rand_chacha/COPYRIGHT \
+ VENDOR/rand_chacha/LICENSE-APACHE \
+ VENDOR/rand_chacha/LICENSE-MIT \
+ VENDOR/rand/COPYRIGHT \
+ VENDOR/rand_core-0.3.1/COPYRIGHT \
+ VENDOR/rand_core-0.3.1/LICENSE-APACHE \
+ VENDOR/rand_core-0.3.1/LICENSE-MIT \
+ VENDOR/rand_core/COPYRIGHT \
+ VENDOR/rand_core/LICENSE-APACHE \
+ VENDOR/rand_core/LICENSE-MIT \
+ VENDOR/rand_hc/COPYRIGHT \
+ VENDOR/rand_hc/LICENSE-APACHE \
+ VENDOR/rand_hc/LICENSE-MIT \
+ VENDOR/rand_isaac/COPYRIGHT \
+ VENDOR/rand_isaac/LICENSE-APACHE \
+ VENDOR/rand_isaac/LICENSE-MIT \
+ VENDOR/rand_jitter/COPYRIGHT \
+ VENDOR/rand_jitter/LICENSE-APACHE \
+ VENDOR/rand_jitter/LICENSE-MIT \
+ VENDOR/rand/LICENSE-APACHE \
+ VENDOR/rand/LICENSE-MIT \
+ VENDOR/rand_os/COPYRIGHT \
+ VENDOR/rand_os/LICENSE-APACHE \
+ VENDOR/rand_os/LICENSE-MIT \
+ VENDOR/rand_pcg/COPYRIGHT \
+ VENDOR/rand_pcg/LICENSE-APACHE \
+ VENDOR/rand_pcg/LICENSE-MIT \
+ VENDOR/rand_xorshift/COPYRIGHT \
+ VENDOR/rand_xorshift/LICENSE-APACHE \
+ VENDOR/rand_xorshift/LICENSE-MIT \
+ VENDOR/rdrand/LICENSE \
+ VENDOR/redox_syscall/LICENSE \
+ VENDOR/redox_termios/LICENSE \
+ VENDOR/regex-automata/COPYING \
+ VENDOR/regex-automata/data/fowler-tests/LICENSE \
+ VENDOR/regex-automata/data/tests/fowler/LICENSE \
+ VENDOR/regex-automata/LICENSE-MIT \
+ VENDOR/regex-automata/UNLICENSE \
+ VENDOR/regex/LICENSE-APACHE \
+ VENDOR/regex/LICENSE-MIT \
+ VENDOR/regex/src/testdata/LICENSE \
+ VENDOR/regex-syntax/LICENSE-APACHE \
+ VENDOR/regex-syntax/LICENSE-MIT \
+ VENDOR/regex-syntax/src/unicode_tables/LICENSE-UNICODE \
+ VENDOR/remove_dir_all/LICENCE-APACHE \
+ VENDOR/remove_dir_all/LICENCE-MIT \
+ VENDOR/ryu/LICENSE-APACHE \
+ VENDOR/ryu/LICENSE-BOOST \
+ VENDOR/same-file/COPYING \
+ VENDOR/same-file/LICENSE-MIT \
+ VENDOR/same-file/UNLICENSE \
+ VENDOR/serde_derive/LICENSE-APACHE \
+ VENDOR/serde_derive/LICENSE-MIT \
+ VENDOR/serde_json/LICENSE-APACHE \
+ VENDOR/serde_json/LICENSE-MIT \
+ VENDOR/serde/LICENSE-APACHE \
+ VENDOR/serde/LICENSE-MIT \
+ VENDOR/smallvec/LICENSE-APACHE \
+ VENDOR/smallvec/LICENSE-MIT \
+ VENDOR/strsim/LICENSE \
+ VENDOR/syn/LICENSE-APACHE \
+ VENDOR/syn/LICENSE-MIT \
+ VENDOR/tempfile/LICENSE-APACHE \
+ VENDOR/tempfile/LICENSE-MIT \
+ VENDOR/termcolor/COPYING \
+ VENDOR/termcolor/LICENSE-MIT \
+ VENDOR/termcolor/UNLICENSE \
+ VENDOR/termion/LICENSE \
+ VENDOR/textwrap/LICENSE \
+ VENDOR/thread_local/LICENSE-APACHE \
+ VENDOR/thread_local/LICENSE-MIT \
+ VENDOR/ucd-util/LICENSE-APACHE \
+ VENDOR/ucd-util/LICENSE-MIT \
+ VENDOR/ucd-util/LICENSE-UNICODE \
+ VENDOR/unicode-width/COPYRIGHT \
+ VENDOR/unicode-width/LICENSE-APACHE \
+ VENDOR/unicode-width/LICENSE-MIT \
+ VENDOR/unicode-xid/COPYRIGHT \
+ VENDOR/unicode-xid/LICENSE-APACHE \
+ VENDOR/unicode-xid/LICENSE-MIT \
+ VENDOR/utf8-ranges/COPYING \
+ VENDOR/utf8-ranges/LICENSE-MIT \
+ VENDOR/utf8-ranges/UNLICENSE \
+ VENDOR/walkdir/COPYING \
+ VENDOR/walkdir/LICENSE-MIT \
+ VENDOR/walkdir/UNLICENSE \
+ VENDOR/winapi/LICENSE-APACHE \
+ VENDOR/winapi/LICENSE-MIT \
+ VENDOR/winapi-util/COPYING \
+ VENDOR/winapi-util/LICENSE-MIT \
+ VENDOR/winapi-util/UNLICENSE \
+ VENDOR/wincolor/COPYING \
+ VENDOR/wincolor/LICENSE-MIT \
+ VENDOR/wincolor/UNLICENSE \
--
2.17.1
^ permalink raw reply related
* Re: [Xen-devel] [PATCH] MAINTAINERS: make Roger VPCI maintainer
From: Jan Beulich @ 2020-02-20 16:01 UTC (permalink / raw)
To: Wei Liu
Cc: Stefano Stabellini, Julien Grall, Konrad Rzeszutek Wilk,
George Dunlap, Andrew Cooper, Ian Jackson, Xen Development List,
Roger Pau Monné
In-Reply-To: <20200220155843.1795-1-wl@xen.org>
On 20.02.2020 16:58, Wei Liu wrote:
> Roger has kindly agreed to take on the burden.
>
> Signed-off-by: Wei Liu <wl@xen.org>
Acked-by: Jan Beulich <jbeulich@suse.com>
with ...
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -500,6 +500,12 @@ F: xen/include/*/vm_event.h
> F: xen/include/asm-x86/hvm/monitor.h
> F: xen/include/asm-x86/hvm/vm_event.h
>
> +VPCI
> +M: Roger Pau Monné <roger.pau@citrix.com>
> +S: Supported
> +F: xen/drivers/vpci
... a trailing slash added here to indicate it's a directory.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply
* [Buildroot] [PATCH v3 08/10] package/ripgrep: bump to version 11.0.1
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
This bump is needed because the old cached archive of the
tarball of ripgrep is still using the old format and can't
be used anymore (as the expected content of the final archive has
changed with the recent infrastructure changes for cargo packages).
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
package/ripgrep/ripgrep.hash | 2 +-
package/ripgrep/ripgrep.mk | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/package/ripgrep/ripgrep.hash b/package/ripgrep/ripgrep.hash
index 8c48458cd8..1e70cbae5a 100644
--- a/package/ripgrep/ripgrep.hash
+++ b/package/ripgrep/ripgrep.hash
@@ -1,3 +1,3 @@
# Locally calculated
-sha256 cb895cff182740c219fefbbaaf903e60f005a4ac4688cac1e43e5fbbbccc5a94 ripgrep-0.8.1.tar.gz
+sha256 c02fe3077c95872175b098d002f872a619195436569a7dff443bbb605bc27474 ripgrep-11.0.1.tar.gz
sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f LICENSE-MIT
diff --git a/package/ripgrep/ripgrep.mk b/package/ripgrep/ripgrep.mk
index 97e9e2ce5f..1023b4bad8 100644
--- a/package/ripgrep/ripgrep.mk
+++ b/package/ripgrep/ripgrep.mk
@@ -4,7 +4,7 @@
#
################################################################################
-RIPGREP_VERSION = 0.8.1
+RIPGREP_VERSION = 11.0.1
RIPGREP_SITE = $(call github,burntsushi,ripgrep,$(RIPGREP_VERSION))
RIPGREP_LICENSE = MIT
RIPGREP_LICENSE_FILES = LICENSE-MIT
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 07/10] docs/manual/adding-packages-cargo: update doc for new infra
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
docs/manual/adding-packages-cargo.txt | 16 ++++------------
1 file changed, 4 insertions(+), 12 deletions(-)
diff --git a/docs/manual/adding-packages-cargo.txt b/docs/manual/adding-packages-cargo.txt
index 9a496224e3..e7d03eb7b5 100644
--- a/docs/manual/adding-packages-cargo.txt
+++ b/docs/manual/adding-packages-cargo.txt
@@ -61,16 +61,8 @@ Those will then replace the commands from the cargo infrastructure.
==== About Dependencies Management
A crate can depend on other libraries from crates.io or git repositories, listed
-in its Cargo.toml file. Before starting a build, Cargo usually downloads
-automatically them. This step can also be performed independently, via the
-+cargo fetch+ command.
+in its Cargo.toml file. Before starting a build, the +cargo vendor+ command
+will be run and will take care of downloading those dependencies.
-Cargo maintains a local cache of the registry index and of git checkouts of the
-crates, whose location is given by +$CARGO_HOME+. As seen in the package
-Makefile example at line 15, this environment variable is set to
-+$(HOST_DIR)/share/cargo+.
-
-This dependency download mechanism is not convenient when performing an offline
-build, as Cargo will fail to fetch the dependencies. In that case, it is advised
-to generate a tarball of the dependencies using the +cargo vendor+ and add it to
-+FOO_EXTRA_DOWNLOADS+.
+The final source tarball will contain all the required dependencies, effectively
+permitting us to do an offline build.
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 06/10] package/pkg-cargo.mk: Introduce the cargo dl backend
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
Cargo is now a fully supported PKGMGR, automatically set for any
package using the cargo infrastructure.
This effectively splits the download phase and the build phase.
The cargo backend will set CARGO_HOME inside DL_DIR during download.
The cached files in CARGO_HOME permits to download only once the
crates index and each dependencies.
During download phase, it will call cargo vendor to copy the
dependencies inside the VENDOR directory inside the source archive.
A local cargo config is also inserted inside the archive in order
to use the VENDOR dir during the build phase.
The build phase is forced to not query the online repository anymore
and thus will be using the vendored dependencies from the tarball.
This also permits to have offline builds.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
package/pkg-cargo.mk | 10 ++++--
package/ripgrep/ripgrep.hash | 2 +-
support/download/cargo | 65 ++++++++++++++++++++++++++++++++++++
support/download/dl-wrapper | 2 +-
4 files changed, 75 insertions(+), 4 deletions(-)
create mode 100755 support/download/cargo
diff --git a/package/pkg-cargo.mk b/package/pkg-cargo.mk
index 35f7c15ad9..c084f7b35e 100644
--- a/package/pkg-cargo.mk
+++ b/package/pkg-cargo.mk
@@ -39,6 +39,8 @@ define inner-cargo-package
# We need host-rustc to run cargo
$(2)_DEPENDENCIES += host-rustc
+$(2)_PKGMGR = cargo\|
+
$(2)_CARGO_ENV = \
CARGO_HOME=$(HOST_DIR)/share/cargo \
$(TARGET_CONFIGURE_OPTS)
@@ -55,11 +57,13 @@ endif
#
ifndef $(2)_BUILD_CMDS
define $(2)_BUILD_CMDS
+ cd $$(@D) && \
$(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
cargo build \
--$$($(2)_CARGO_MODE) \
$$($(2)_CARGO_TARGET_OPT) \
- --manifest-path $$(@D)/Cargo.toml
+ --manifest-path Cargo.toml \
+ --offline
endef
endif
@@ -69,12 +73,14 @@ endif
#
ifndef $(2)_INSTALL_TARGET_CMDS
define $(2)_INSTALL_TARGET_CMDS
+ cd $$(@D) && \
$(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
cargo install \
--root $(TARGET_DIR)/usr/ \
--bins \
- --path $$(@D) \
+ --path ./ \
$$($(2)_CARGO_TARGET_OPT) \
+ --offline \
--force
endef
endif
diff --git a/package/ripgrep/ripgrep.hash b/package/ripgrep/ripgrep.hash
index 0841c0185c..8c48458cd8 100644
--- a/package/ripgrep/ripgrep.hash
+++ b/package/ripgrep/ripgrep.hash
@@ -1,3 +1,3 @@
# Locally calculated
-sha256 7035379fce0c1e32552e8ee528b92c3d01b8d3935ea31d26c51a73287be74bb3 ripgrep-0.8.1.tar.gz
+sha256 cb895cff182740c219fefbbaaf903e60f005a4ac4688cac1e43e5fbbbccc5a94 ripgrep-0.8.1.tar.gz
sha256 0f96a83840e146e43c0ec96a22ec1f392e0680e6c1226e6f3ba87e0740af850f LICENSE-MIT
diff --git a/support/download/cargo b/support/download/cargo
new file mode 100755
index 0000000000..0ce94cf16b
--- /dev/null
+++ b/support/download/cargo
@@ -0,0 +1,65 @@
+#!/usr/bin/env bash
+
+# We want to catch any unexpected failure, and exit immediately
+set -e
+
+# Download helper for cargo package. It will populate the VENDOR directory
+# inside the archive with the dependencies of the package
+#
+# Arguments are in this order:
+# $1 mandatory fullpath to an already downloaded source file of a cargo package
+# $2 mandatory fullpath to a temp dir
+# $3 mandatory fullpath to the DL_DIR
+#
+# Environment:
+# cargo in PATH
+
+tmpd=""
+
+do_clean() {
+ if [ -n "${tmpd}" ]; then
+ rm -rf "${tmpd}"
+ fi
+}
+
+trap do_clean EXIT
+
+if [ $# -le 2 ] ; then
+ echo 'Need at least 3 arguments' >&2
+ exit 1
+fi;
+
+tmpd="$(mktemp -d -p "$2")"
+cd "${tmpd}"
+
+tar xf "${1}"
+cd ./*/
+echo "Running cargo vendor."
+CARGO_HOME="${3}"/cargo_home cargo vendor -q --locked VENDOR
+# Create the local .cargo/config with vendor info
+mkdir -p .cargo/
+cat <<EOF >.cargo/config
+[source.crates-io]
+replace-with = "vendored-sources"
+
+[source.vendored-sources]
+directory = "VENDOR"
+EOF
+
+cd ..
+
+# Generate the archive, sort with the C locale so that it is reproducible.
+find "$(basename "$OLDPWD")" -not -type d -print0 >files.list
+LC_ALL=C sort -z <files.list >files.list.sorted
+# let's use a fixed hardcoded date to be reproducible
+date="2020-02-06 01:02:03"
+
+# Create GNU-format tarballs, since that's the format of the tarballs on
+# sources.buildroot.org and used in the *.hash files
+echo "Creating final archive."
+tar cf new.tar --null --verbatim-files-from --numeric-owner --format=gnu \
+ --owner=0 --group=0 --mtime="${date}" -T files.list.sorted
+gzip -6 -n <new.tar >new.tar.gz
+mv "${1}" "${1}".old
+mv new.tar.gz "${1}"
+rm "${1}".old
diff --git a/support/download/dl-wrapper b/support/download/dl-wrapper
index 3f613bb622..5e52b3e60f 100755
--- a/support/download/dl-wrapper
+++ b/support/download/dl-wrapper
@@ -98,7 +98,7 @@ main() {
case "${b}" in
urlencode) urlencode="${b}" ;;
git|svn|cvs|bzr|file|scp|hg) backend="${b}" ;;
- # insert here supported second backends) backend2="${b}" ;;
+ cargo) backend2="${b}" ;;
esac
done
uri=${uri#*+}
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 05/10] docs/manual/adding-packages-generic: update for new FOO_PKGMGR value
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
docs/manual/adding-packages-generic.txt | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index baa052e31c..7fd58ed9a7 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -265,6 +265,13 @@ not and can not work as people would expect it should:
+LIBFOO_SITE=/opt/software/libfoo.tar.gz+ +
+LIBFOO_SITE=$(TOPDIR)/../src/libfoo+
+* +LIBFOO_PKGMGR+ may contain the name of the second backend used
+ to complete the sources tarball. This second backend is called after
+ the first archive has been generated (e.g. git) or simply downloaded
+ (wget). It can be useful in the case where some other files are
+ needed for the build and require to be downloaded by a specific tool.
+ The value of this variable must end with a '|' (serves as a separator).
+
* +LIBFOO_DL_OPTS+ is a space-separated list of additional options to
pass to the downloader. Useful for retrieving documents with
server-side checking for user logins and passwords, or to use a proxy.
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 04/10] support/download/dl-wrapper: rework backend parsing
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
The point of this modification is to be more generic in the way
the backends/options are parsed from the urls.
The '+' char serves as delimiter between the backends and the
actual url.
The '|' char serves as separator between the different
options or backends.
This will be useful later when more options or backends would be
needed.
pkg-generic.mk now passes the optional FOO_PKGMGR value in case a
package needs to use a second backend.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
package/pkg-generic.mk | 2 +-
support/download/dl-wrapper | 34 ++++++++++++++++++++++------------
2 files changed, 23 insertions(+), 13 deletions(-)
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 268d999efb..a5038ebff4 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -541,7 +541,7 @@ ifndef $(2)_PATCH
endif
$(2)_ALL_DOWNLOADS = \
- $$(if $$($(2)_SOURCE),$$($(2)_SITE_METHOD)+$$($(2)_SITE)/$$($(2)_SOURCE)) \
+ $$(if $$($(2)_SOURCE),$$($(2)_PKGMGR)$$($(2)_SITE_METHOD)+$$($(2)_SITE)/$$($(2)_SOURCE)) \
$$(foreach p,$$($(2)_PATCH) $$($(2)_EXTRA_DOWNLOADS),\
$$(if $$(findstring ://,$$(p)),$$(p),\
$$($(2)_SITE_METHOD)+$$($(2)_SITE)/$$(p)))
diff --git a/support/download/dl-wrapper b/support/download/dl-wrapper
index 3315bd410e..3f613bb622 100755
--- a/support/download/dl-wrapper
+++ b/support/download/dl-wrapper
@@ -21,8 +21,8 @@ export BR_BACKEND_DL_GETOPTS=":hc:d:o:n:N:H:ru:qf:e"
main() {
local OPT OPTARG
- local backend output hfile recurse quiet rc
- local -a uris
+ local backend backends_str output hfile recurse quiet rc backend2
+ local -a uris backends
# Parse our options; anything after '--' is for the backend
while getopts ":c:d:D:o:n:N:H:rf:u:q" OPT; do
@@ -85,18 +85,24 @@ main() {
download_and_check=0
rc=1
for uri in "${uris[@]}"; do
- backend_urlencode="${uri%%+*}"
- backend="${backend_urlencode%|*}"
- case "${backend}" in
- git|svn|cvs|bzr|file|scp|hg) ;;
- *) backend="wget" ;;
- esac
+ urlencode=""
+ backend="wget"
+ backend2=""
+
+ # Extract the backends list, separated by '+' from the actual url
+ backends_str="${uri%%+*}"
+ # make an array out of it, backends are '|' separated
+ IFS='|' read -r -a backends <<< "${backends_str}"
+ # parse each of them, set flags and set the real backend
+ for b in "${backends[@]}" ; do
+ case "${b}" in
+ urlencode) urlencode="${b}" ;;
+ git|svn|cvs|bzr|file|scp|hg) backend="${b}" ;;
+ # insert here supported second backends) backend2="${b}" ;;
+ esac
+ done
uri=${uri#*+}
- urlencode=${backend_urlencode#*|}
- # urlencode must be "urlencode"
- [ "${urlencode}" != "urlencode" ] && urlencode=""
-
# tmpd is a temporary directory in which backends may store
# intermediate by-products of the download.
# tmpf is the file in which the backends should put the downloaded
@@ -138,6 +144,10 @@ main() {
# cd back to free the temp-dir, so we can remove it later
cd "${OLDPWD}"
+ if [ -n "${backend2}" ] ; then
+ support/download/${backend2} "${tmpf}" "${tmpd}" "${old_dl_dir}"
+ fi
+
# Check if the downloaded file is sane, and matches the stored hashes
# for that file
if support/download/check-hash ${quiet} "${hfile}" "${tmpf}" "${output##*/}"; then
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 03/10] package/ripgrep: convert to cargo infrastructure
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
package/ripgrep/ripgrep.mk | 23 +----------------------
1 file changed, 1 insertion(+), 22 deletions(-)
diff --git a/package/ripgrep/ripgrep.mk b/package/ripgrep/ripgrep.mk
index 832c076f26..97e9e2ce5f 100644
--- a/package/ripgrep/ripgrep.mk
+++ b/package/ripgrep/ripgrep.mk
@@ -9,25 +9,4 @@ RIPGREP_SITE = $(call github,burntsushi,ripgrep,$(RIPGREP_VERSION))
RIPGREP_LICENSE = MIT
RIPGREP_LICENSE_FILES = LICENSE-MIT
-RIPGREP_DEPENDENCIES = host-rustc
-RIPGREP_CARGO_ENV = CARGO_HOME=$(HOST_DIR)/share/cargo
-RIPGREP_CARGO_MODE = $(if $(BR2_ENABLE_DEBUG),debug,release)
-
-RIPGREP_BIN_DIR = target/$(RUSTC_TARGET_NAME)/$(RIPGREP_CARGO_MODE)
-
-RIPGREP_CARGO_OPTS = \
- --$(RIPGREP_CARGO_MODE) \
- --target=$(RUSTC_TARGET_NAME) \
- --manifest-path=$(@D)/Cargo.toml
-
-define RIPGREP_BUILD_CMDS
- $(TARGET_MAKE_ENV) $(RIPGREP_CARGO_ENV) \
- cargo build $(RIPGREP_CARGO_OPTS)
-endef
-
-define RIPGREP_INSTALL_TARGET_CMDS
- $(INSTALL) -D -m 0755 $(@D)/$(RIPGREP_BIN_DIR)/rg \
- $(TARGET_DIR)/usr/bin/rg
-endef
-
-$(eval $(generic-package))
+$(eval $(cargo-package))
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 02/10] docs/manual/cargo: Update manual for cargo packages.
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In-Reply-To: <20200220160119.3407-1-patrick.havelange@essensium.com>
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
docs/manual/adding-packages-cargo.txt | 54 +++++----------------------
1 file changed, 10 insertions(+), 44 deletions(-)
diff --git a/docs/manual/adding-packages-cargo.txt b/docs/manual/adding-packages-cargo.txt
index bb078b6981..9a496224e3 100644
--- a/docs/manual/adding-packages-cargo.txt
+++ b/docs/manual/adding-packages-cargo.txt
@@ -27,9 +27,9 @@ The +Config.in+ file of Cargo-based package 'foo' should contain:
==== Cargo-based package's +.mk+ file
-Buildroot does not (yet) provide a dedicated package infrastructure for
-Cargo-based packages. So, we will explain how to write a +.mk+ file for such a
-package. Let's start with an example:
+Buildroot provides a dedicated package infrastructure for Cargo-based packages.
+So, we will explain how to write a +.mk+ file for such a package. Let's start
+with an example:
------------------------------
01: ################################################################################
@@ -44,53 +44,19 @@ package. Let's start with an example:
10: FOO_LICENSE = GPL-3.0+
11: FOO_LICENSE_FILES = COPYING
12:
-13: FOO_DEPENDENCIES = host-rustc
-14:
-15: FOO_CARGO_ENV = CARGO_HOME=$(HOST_DIR)/share/cargo
-16: FOO_CARGO_MODE = $(if $(BR2_ENABLE_DEBUG),debug,release)
-17:
-18: FOO_BIN_DIR = target/$(RUSTC_TARGET_NAME)/$(FOO_CARGO_MODE)
-19:
-20: FOO_CARGO_OPTS = \
-21: --$(FOO_CARGO_MODE) \
-22: --target=$(RUSTC_TARGET_NAME) \
-23: --manifest-path=$(@D)/Cargo.toml
-24:
-25: define FOO_BUILD_CMDS
-26: $(TARGET_MAKE_ENV) $(FOO_CARGO_ENV) \
-27: cargo build $(FOO_CARGO_OPTS)
-28: endef
-29:
-30: define FOO_INSTALL_TARGET_CMDS
-31: $(INSTALL) -D -m 0755 $(@D)/$(FOO_BIN_DIR)/foo \
-32: $(TARGET_DIR)/usr/bin/foo
-33: endef
-34:
-35: $(eval $(generic-package))
+13: $(eval $(cargo-package))
--------------------------------
The Makefile starts with the definition of the standard variables for package
declaration (lines 7 to 11).
-As seen in line 35, it is based on the
-xref:generic-package-tutorial[+generic-package+ infrastructure]. So, it defines
-the variables required by this particular infrastructure, where Cargo is
-invoked:
+As seen in line 13, it is based on the cargo-package infrastructure. Cargo will
+be invoked automatically by this infrastructure. The required dependencies of the
+crate will be downloaded and the buildroot dependencies will also be set.
-* +FOO_BUILD_CMDS+: Cargo is invoked to perform the build. The options required
- to configure the cross-compilation of the package are passed via
- +FOO_CONF_OPTS+.
-
-* +FOO_INSTALL_TARGET_CMDS+: The binary executable generated is installed on
- the target.
-
-In order to have Cargo available for the build, +FOO_DEPENDENCIES+ needs to
-contain +host-cargo+.
-
-To sum it up, to add a new Cargo-based package, the Makefile example can be
-copied verbatim then edited to replace all occurences of +FOO+ with the
-uppercase name of the new package and update the values of the standard
-variables.
+It is still possible to define custom build commands or install commands (i.e.
+with FOO_BUILD_CMDS and FOO_INSTALL_TARGET_CMDS).
+Those will then replace the commands from the cargo infrastructure.
==== About Dependencies Management
--
2.17.1
^ permalink raw reply related
* [Buildroot] [PATCH v3 01/10] package/pkg-cargo.mk: Introduce the cargo package infrastructure.
From: Patrick Havelange @ 2020-02-20 16:01 UTC (permalink / raw)
To: buildroot
In order to be package agnostic, the install phase is now using
cargo instead of install.
TARGET_CONFIGURE_OPTS is now also set when running cargo in order
to support cross compiling C code within cargo.
Signed-off-by: Patrick Havelange <patrick.havelange@essensium.com>
---
changes:
v2 -> v3 (Sam Voss)
- set TARGET_CONFIGURE_OPTS when running cargo
This is the V3 of the previously posted series :
http://patchwork.ozlabs.org/project/buildroot/list/?series=156166&state=%2A&archive=both
A lot of changes following the buildroot developer meeting's
decisions from February 2020.
Needs to be applied on top of the rust 1.40 bump
---
package/Makefile.in | 1 +
package/pkg-cargo.mk | 92 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 93 insertions(+)
create mode 100644 package/pkg-cargo.mk
diff --git a/package/Makefile.in b/package/Makefile.in
index 285e2837ef..650d7c166e 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -426,3 +426,4 @@ include package/pkg-kernel-module.mk
include package/pkg-waf.mk
include package/pkg-golang.mk
include package/pkg-meson.mk
+include package/pkg-cargo.mk
diff --git a/package/pkg-cargo.mk b/package/pkg-cargo.mk
new file mode 100644
index 0000000000..35f7c15ad9
--- /dev/null
+++ b/package/pkg-cargo.mk
@@ -0,0 +1,92 @@
+################################################################################
+# Cargo package infrastructure
+#
+# This file implements an infrastructure that eases development of package
+# .mk files for Cargo packages. It should be used for all packages that use
+# Cargo as their build system.
+#
+# See the Buildroot documentation for details on the usage of this
+# infrastructure
+#
+# In terms of implementation, this Cargo infrastructure requires the .mk file
+# to only specify metadata information about the package: name, version,
+# download URL, etc.
+#
+# We still allow the package .mk file to override what the different steps
+# are doing, if needed. For example, if <PKG>_BUILD_CMDS is already defined,
+# it is used as the list of commands to perform to build the package,
+# instead of the default Cargo behaviour. The package can also define some
+# post operation hooks.
+#
+################################################################################
+
+################################################################################
+# inner-cargo-package -- defines how the configuration, compilation and
+# installation of a cargo package should be done, implements a few hooks
+# to tune the build process for cargo specifities and calls the generic
+# package infrastructure to generate the necessary make targets
+#
+# argument 1 is the lowercase package name
+# argument 2 is the uppercase package name, including a HOST_ prefix
+# for host packages
+# argument 3 is the uppercase package name, without the HOST_ prefix
+# for host packages
+# argument 4 is the type (target or host)
+################################################################################
+
+define inner-cargo-package
+
+# We need host-rustc to run cargo
+$(2)_DEPENDENCIES += host-rustc
+
+$(2)_CARGO_ENV = \
+ CARGO_HOME=$(HOST_DIR)/share/cargo \
+ $(TARGET_CONFIGURE_OPTS)
+
+$(2)_CARGO_MODE = $(if $(BR2_ENABLE_DEBUG),debug,release)
+
+ifeq ($(4),target)
+ $(2)_CARGO_TARGET_OPT = --target $$(RUSTC_TARGET_NAME)
+endif
+
+#
+# Build step. Only define it if not already defined by the package .mk
+# file.
+#
+ifndef $(2)_BUILD_CMDS
+define $(2)_BUILD_CMDS
+ $(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
+ cargo build \
+ --$$($(2)_CARGO_MODE) \
+ $$($(2)_CARGO_TARGET_OPT) \
+ --manifest-path $$(@D)/Cargo.toml
+endef
+endif
+
+#
+# Target installation step. Only define it if not already defined by
+# the package .mk file.
+#
+ifndef $(2)_INSTALL_TARGET_CMDS
+define $(2)_INSTALL_TARGET_CMDS
+ $(TARGET_MAKE_ENV) $$($(2)_CARGO_ENV) \
+ cargo install \
+ --root $(TARGET_DIR)/usr/ \
+ --bins \
+ --path $$(@D) \
+ $$($(2)_CARGO_TARGET_OPT) \
+ --force
+endef
+endif
+
+# Call the generic package infrastructure to generate the necessary
+# make targets
+$(call inner-generic-package,$(1),$(2),$(3),$(4))
+
+endef
+
+################################################################################
+# cargo-package -- the target generator macro for Cargo packages
+################################################################################
+
+cargo-package = $(call inner-cargo-package,$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target)
--
2.17.1
^ permalink raw reply related
* Re: [PATCH V2 0/5] Add support to defer core initialization
From: Lorenzo Pieralisi @ 2020-02-20 16:00 UTC (permalink / raw)
To: Vidya Sagar
Cc: Kishon Vijay Abraham I, andrew.murray, Tom Joseph, Milind Parab,
jingoohan1, gustavo.pimentel, bhelgaas, thierry.reding,
Jisheng.Zhang, jonathanh, linux-pci, linux-kernel, kthota,
mmaddireddy, sagar.tv
In-Reply-To: <7e8dafcd-bc3f-4acc-7023-85e24bebdd94@nvidia.com>
On Wed, Feb 19, 2020 at 07:06:47PM +0530, Vidya Sagar wrote:
> Hi Lorenzo, Andrew,
> Kishon did rebase [1] mentioned below and removed dependencies.
> New patch series is available
> @ http://patchwork.ozlabs.org/project/linux-pci/list/?series=158088
>
> I rebased my patches on top of this and is available for review
> @ http://patchwork.ozlabs.org/project/linux-pci/list/?series=158959
>
> Please let us know the way forward towards merging these patches.
Hi Vidya,
I shall have a look shortly, I have planned to start queueing patches
from next week.
Thanks,
Lorenzo
> Thanks,
> Vidya Sagar
>
> On 2/5/2020 12:07 PM, Kishon Vijay Abraham I wrote:
> > External email: Use caution opening links or attachments
> >
> >
> > +Tom, Milind
> >
> > Hi,
> >
> > On 23/01/20 3:25 PM, Kishon Vijay Abraham I wrote:
> > > Hi Vidya Sagar,
> > >
> > > On 23/01/20 2:54 pm, Vidya Sagar wrote:
> > > > Hi Kishon,
> > > > Apologies for pinging again. Could you please review this series?
> > > >
> > > > Thanks,
> > > > Vidya Sagar
> > > >
> > > > On 1/11/2020 5:18 PM, Vidya Sagar wrote:
> > > > > Hi Kishon,
> > > > > Could you please review this series?
> > > > >
> > > > > Also, this series depends on the following change of yours
> > > > > http://patchwork.ozlabs.org/patch/1109884/
> > > > > Whats the plan to get this merged?
> > >
> > > I've posted the endpoint improvements as a separate series
> > > http://lore.kernel.org/r/20191231100331.6316-1-kishon@ti.com
> > >
> > > I'd prefer this series gets tested by others. I'm also planning to test
> > > this series. Sorry for the delay. I'll test review and test this series
> > > early next week.
> >
> > I tested this series with DRA7 configured in EP mode. So for the series
> > itself
> >
> > Acked-by: Kishon Vijay Abraham I <kishon@ti.com>
> >
> > Tom, Can you test this series in Cadence platform?
> >
> > Lorenzo, Andrew,
> >
> > How do you want to go about merging this series? This series depends on
> > [1] which in turn is dependent on two other series. If required, I can
> > rebase [1] on mainline kernel and remove it's dependencies with the
> > other series. That way this series and [1] could be merged. And the
> > other series could be worked later. Kindly let me know.
> >
> > Thanks
> > Kishon
> >
> > [1] ->
> > https://lore.kernel.org/linux-pci/20191231100331.6316-1-kishon@ti.com/
> > >
> > > Thanks
> > > Kishon
> > >
> > > > >
> > > > > Thanks,
> > > > > Vidya Sagar
> > > > >
> > > > > On 1/3/20 3:37 PM, Vidya Sagar wrote:
> > > > > > EPC/DesignWare core endpoint subsystems assume that the core
> > > > > > registers are
> > > > > > available always for SW to initialize. But, that may not be the case
> > > > > > always.
> > > > > > For example, Tegra194 hardware has the core running on a clock that
> > > > > > is derived
> > > > > > from reference clock that is coming into the endpoint system from host.
> > > > > > Hence core is made available asynchronously based on when host system
> > > > > > is going
> > > > > > for enumeration of devices. To accommodate this kind of hardwares,
> > > > > > support is
> > > > > > required to defer the core initialization until the respective
> > > > > > platform driver
> > > > > > informs the EPC/DWC endpoint sub-systems that the core is indeed
> > > > > > available for
> > > > > > initiaization. This patch series is attempting to add precisely that.
> > > > > > This series is based on Kishon's patch that adds notification mechanism
> > > > > > support from EPC to EPF @ http://patchwork.ozlabs.org/patch/1109884/
> > > > > >
> > > > > > Vidya Sagar (5):
> > > > > > PCI: endpoint: Add core init notifying feature
> > > > > > PCI: dwc: Refactor core initialization code for EP mode
> > > > > > PCI: endpoint: Add notification for core init completion
> > > > > > PCI: dwc: Add API to notify core initialization completion
> > > > > > PCI: pci-epf-test: Add support to defer core initialization
> > > > > >
> > > > > > .../pci/controller/dwc/pcie-designware-ep.c | 79 +++++++-----
> > > > > > drivers/pci/controller/dwc/pcie-designware.h | 11 ++
> > > > > > drivers/pci/endpoint/functions/pci-epf-test.c | 118 ++++++++++++------
> > > > > > drivers/pci/endpoint/pci-epc-core.c | 19 ++-
> > > > > > include/linux/pci-epc.h | 2 +
> > > > > > include/linux/pci-epf.h | 5 +
> > > > > > 6 files changed, 164 insertions(+), 70 deletions(-)
> > > > > >
^ permalink raw reply
* Re: [PATCH 1/4] parse-options: convert "command mode" to a flag
From: Johannes Schindelin @ 2020-02-20 16:00 UTC (permalink / raw)
To: Eric Sunshine; +Cc: Paolo Bonzini, Git List, bfields
In-Reply-To: <CAPig+cQkBKJLW3-W4SS0KX9+Gs2fT-Z-DrvVpcVOLZpFmVBoQA@mail.gmail.com>
Hi Eric,
On Wed, 19 Feb 2020, Eric Sunshine wrote:
> On Wed, Feb 19, 2020 at 11:15 AM <pbonzini@redhat.com> wrote:
> > OPTION_CMDMODE is essentially OPTION_SET_INT plus the extra check
> > that the variable had not set before. In order to allow custom
> > processing, change it to OPTION_SET_INT plus a new flag that takes
> > care of the check. This works as long as the option value points
> > to an int.
> >
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> > diff --git a/t/t0040-parse-options.sh b/t/t0040-parse-options.sh
> > @@ -324,6 +326,22 @@ test_expect_success 'OPT_NEGBIT() works' '
> > +test_expect_success 'OPT_CMDMODE() detects incompatibility' '
> > + test_must_fail test-tool parse-options --mode1 --mode2 >output 2>output.err &&
> > + test_must_be_empty output &&
> > + grep "incompatible with --mode" output.err
> > +'
>
> The error message may have been localized, so use test_i18ngrep()
> instead of 'grep':
>
> test_i18ngrep "incompatible with --mode" output.err
The error message _is_ localized, causing the GETTEXT_POISON job to fail:
https://dev.azure.com/gitgitgadget/git/_build/results?buildId=31113&view=ms.vss-test-web.build-test-results-tab&runId=97704&resultId=102357&paneView=debug
So yes. It needs to be changed to `test_i18ngrep`.
Thanks,
Johannes
>
> > +
> > +test_expect_success 'OPT_CMDMODE() detects incompatibility with something else' '
> > + test_must_fail test-tool parse-options --set23 --mode2 >output 2>output.err &&
> > + test_must_be_empty output &&
> > + grep "incompatible with something else" output.err
> > +'
>
> Ditto.
>
>
^ permalink raw reply
* Re: [Intel-gfx] [igt-dev] [PATCH i-g-t 9/9] i915: Exercise I915_CONTEXT_PARAM_RINGSIZE
From: Chris Wilson @ 2020-02-20 16:00 UTC (permalink / raw)
To: Janusz Krzysztofik; +Cc: igt-dev, intel-gfx
In-Reply-To: <9103010.GsyPYnKGBp@jkrzyszt-desk.ger.corp.intel.com>
Quoting Janusz Krzysztofik (2020-02-20 15:57:24)
> Hi Chris,
>
> On Monday, December 2, 2019 3:59:19 PM CET Chris Wilson wrote:
> > Quoting Janusz Krzysztofik (2019-12-02 14:42:58)
> > > Hi Chris,
> > >
> > > I have a few questions rather than comments. I hope they are worth spending
> > > your time.
> > >
> > > On Wednesday, November 13, 2019 1:52:40 PM CET Chris Wilson wrote:
> > > > I915_CONTEXT_PARAM_RINGSIZE specifies how large to create the command
> > > > ringbuffer for logical ring contects. This directly affects the number
> > >
> > > s/contects/contexts/
> > >
> > > > of batches userspace can submit before blocking waiting for space.
> > > >
> > > > Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
>
> Have you got this patch still queued somewhere? As UMD has accepted the
> solution and are ready with changes on their side, I think we need to merge it
> soon, and the kernel side as well.
Link? That's all I need to merge the kernel...
-Chris
_______________________________________________
Intel-gfx mailing list
Intel-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.