* Re: [RFC PATCH] hwclock: Remove ioperm declare as it causes nested extern declare warning
From: Karel Zak @ 2024-12-09 10:43 UTC (permalink / raw)
To: Zhaoming Luo; +Cc: util-linux, samuel.thibault
In-Reply-To: <20241209021154.255340-1-zhmingluo@163.com>
On Mon, Dec 09, 2024 at 10:11:54AM GMT, Zhaoming Luo wrote:
> sys-utils/hwclock-cmos.c | 1 -
> 1 file changed, 1 deletion(-)
Applied, thanks.
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
* Re: [PATCH] hwclock: Support GNU Hurd
From: Samuel Thibault @ 2024-12-09 10:36 UTC (permalink / raw)
To: Karel Zak; +Cc: Zhaoming Luo, util-linux, bug-hurd
In-Reply-To: <kmz3eghhmo6rqlnenuwvcj3aa45elxajfdhwqiht52thxnb4qh@f2z34wslmajb>
Karel Zak, le lun. 09 déc. 2024 11:32:33 +0100, a ecrit:
> On Mon, Dec 09, 2024 at 10:58:15AM GMT, Zhaoming Luo wrote:
> > if (ctl->rtc_dev_name) {
> > rtc_dev_name = ctl->rtc_dev_name;
> > - rtc_dev_fd = open(rtc_dev_name, O_RDONLY);
> > + rtc_dev_fd = open(rtc_dev_name, O_RDWR);
>
> Why do you need O_RDWR on HURD?
For setting the time, it makes sense to request WR?
> Maybe it would be better to add an
> #ifdef, as it is unnecessary for Linux.
I have to say I'm surprised that Linux doesn't require it for setting
the time.
Samuel
^ permalink raw reply
* Re: [PATCH] hwclock: Support GNU Hurd
From: Karel Zak @ 2024-12-09 10:32 UTC (permalink / raw)
To: Zhaoming Luo; +Cc: util-linux, bug-hurd
In-Reply-To: <20241209025815.7299-1-zhmingluo@163.com>
On Mon, Dec 09, 2024 at 10:58:15AM GMT, Zhaoming Luo wrote:
> if (ctl->rtc_dev_name) {
> rtc_dev_name = ctl->rtc_dev_name;
> - rtc_dev_fd = open(rtc_dev_name, O_RDONLY);
> + rtc_dev_fd = open(rtc_dev_name, O_RDWR);
Why do you need O_RDWR on HURD? Maybe it would be better to add an
#ifdef, as it is unnecessary for Linux.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
* Re: Ensuring that mount(8) will always interpret a filesystem correctly
From: Karel Zak @ 2024-12-09 10:26 UTC (permalink / raw)
To: Demi Marie Obenour; +Cc: util-linux
In-Reply-To: <6f939e2c-7d86-4b3d-885b-2c171c944079@gmail.com>
Hi Demi,
On Sat, Dec 07, 2024 at 08:45:32PM GMT, Demi Marie Obenour wrote:
> Is there a guarantee that if all data before the filesystem superblock is
> zero, and that the filesystem never writes to this region, libblkid (and
> thus, presumably, mount(8)) will always mount the filesystem with the
> correct filesystem type, even if e.g. someone writes a file containing
> a superblock of a different filesystem and the filesystem happens to put
> it where that superblock is valid?
the libblkid library offers multiple modes, with "safe mode" being the
default for detecting filesystems. In this mode, the library checks
for any additional valid superblocks on the device. There are
exceptions for certain filesystems on CD/DVD media (such as udf and
iso), but for regular filesystems, sharing the same device is not
allowed.
There is also an option to specify that a superblock is only valid if
no other area is using it (using blkid_probe_set_wiper() and
blkid_probe_use_wiper()). However, this is only used for LVM and
bcache.
The library does not require that there are zeros before the
superblock, as not all mkfs-like programs zero out all areas.
In recent years, there have been no reports of collisions. In the
entire history of the library, the only collisions I can recall are
with swap areas and luks, and occasionally with poorly detected FAT
filesystems (due to the messy design of FAT).
> The motivation for this message is that systemd-gpt-generator generates
> mountpoints based on Discoverable Partition Specification GUIDs. These
> indicate the mountpoint of the partition but not the filesystem type.
Filesystem auto-detection is a common feature. The situation is
similar to having an "auto" fstype in fstab. The systemd-gpt-generator
simply identifies the partition as "/usr" (or any other mountpoint)
and the rest is usual scenario.
> If a correctly-produced filesystem image will always continue to be
> recognized as the correct type, this is fine. Otherwise, an unlucky
> combination of writes to the filesystem and filesystem allocation decisions
> could cause the filesystem to start being mounted as the wrong type, which
> would be very bad. According to https://github.com/util-linux/util-linux/issues/1305,
> libblkid can indeed probe for subsequent superblocks after the first one it
> finds.
I believe the situation would be the same even without the
Discoverable Partition Specification. The kernel always divides the
whole disk into partitions, and libblkid/mount utilizes these
partitions. Therefore, the filesystems are automatically separated by
the partition table.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
* [PATCH] hwclock: Support GNU Hurd
From: Zhaoming Luo @ 2024-12-09 2:58 UTC (permalink / raw)
To: util-linux; +Cc: bug-hurd, Zhaoming Luo
* configure.ac: add HURD so it can be used in conditional in am files
* sys-utils/Makemodule.am: compile hwclock for GNU Hurd
* sys-utils/hwclock-rtc.c: compile for GNU Hurd
* sys-utils/hwclock.c: compile for GNU Hurd
* sys-utils/hwclock.h: compile for GNU Hurd
Signed-off-by: Zhaoming Luo <zhmingluo@163.com>
---
configure.ac | 6 +++++-
sys-utils/Makemodule.am | 5 +++++
sys-utils/hwclock-rtc.c | 15 ++++++++++++---
sys-utils/hwclock.c | 2 +-
sys-utils/hwclock.h | 4 +++-
5 files changed, 26 insertions(+), 6 deletions(-)
diff --git a/configure.ac b/configure.ac
index 698da36..8de6af6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -263,16 +263,20 @@ AC_PATH_PROG([XSLTPROC], [xsltproc])
linux_os=no
bsd_os=no
+gnu_os=no
AS_CASE([${host_os}],
[*linux*],
[linux_os=yes],
[*darwin*],
[darwin_os=yes],
[*bsd*],
- [bsd_os=yes])
+ [bsd_os=yes],
+ [gnu*],
+ [gnu_os=yes])
AM_CONDITIONAL([LINUX], [test "x$linux_os" = xyes])
AM_CONDITIONAL([DARWIN], [test "x$darwin_os" = xyes])
AM_CONDITIONAL([BSD], [test "x$bsd_os" = xyes])
+AM_CONDITIONAL([HURD], [test "x$gnu_os" = xyes])
AS_IF([test "x$darwin_os" = xyes], [
AC_DEFINE([_DARWIN_C_SOURCE], [1], [Enable MAP_ANON in sys/mman.h on Mac OS X])
diff --git a/sys-utils/Makemodule.am b/sys-utils/Makemodule.am
index 209b656..8cc69be 100644
--- a/sys-utils/Makemodule.am
+++ b/sys-utils/Makemodule.am
@@ -570,6 +570,11 @@ hwclock_SOURCES += \
lib/monotonic.c
hwclock_LDADD += $(REALTIME_LIBS)
endif
+if HURD
+hwclock_SOURCES += \
+ sys-utils/hwclock-rtc.c \
+ lib/monotonic.c
+endif
if HAVE_AUDIT
hwclock_LDADD += -laudit
endif
diff --git a/sys-utils/hwclock-rtc.c b/sys-utils/hwclock-rtc.c
index 2796f2e..7162ae9 100644
--- a/sys-utils/hwclock-rtc.c
+++ b/sys-utils/hwclock-rtc.c
@@ -8,10 +8,15 @@
*
* rtc.c - Use /dev/rtc for clock access
*/
+#ifdef __GNU__
+#include <sys/ioctl.h>
+#include <hurd/rtc.h>
+#else
#include <asm/ioctl.h>
-#include <errno.h>
#include <linux/rtc.h>
#include <linux/types.h>
+#endif /* __GNU__ */
+#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
@@ -28,6 +33,7 @@
#include "hwclock.h"
+#ifndef __GNU__
#ifndef RTC_PARAM_GET
struct rtc_param {
__u64 param;
@@ -60,6 +66,7 @@ const struct hwclock_param *get_hwclock_params(void)
{
return hwclock_params;
}
+#endif /* __GNU__ */
/*
* /dev/rtc is conventionally chardev 10/135
@@ -99,12 +106,12 @@ static int open_rtc(const struct hwclock_control *ctl)
/* --rtc option has been given */
if (ctl->rtc_dev_name) {
rtc_dev_name = ctl->rtc_dev_name;
- rtc_dev_fd = open(rtc_dev_name, O_RDONLY);
+ rtc_dev_fd = open(rtc_dev_name, O_RDWR);
} else {
for (i = 0; i < ARRAY_SIZE(fls); i++) {
if (ctl->verbose)
printf(_("Trying to open: %s\n"), fls[i]);
- rtc_dev_fd = open(fls[i], O_RDONLY);
+ rtc_dev_fd = open(fls[i], O_RDWR);
if (rtc_dev_fd < 0) {
if (errno == ENOENT || errno == ENODEV)
@@ -411,6 +418,7 @@ int set_epoch_rtc(const struct hwclock_control *ctl)
+#ifndef __GNU__
static int resolve_rtc_param_alias(const char *alias, __u64 *value)
{
const struct hwclock_param *param = &hwclock_params[0];
@@ -609,3 +617,4 @@ int rtc_vl_clear(const struct hwclock_control *ctl)
return 0;
}
+#endif /* __GNU__ */
diff --git a/sys-utils/hwclock.c b/sys-utils/hwclock.c
index 2b33dfb..2714775 100644
--- a/sys-utils/hwclock.c
+++ b/sys-utils/hwclock.c
@@ -989,7 +989,7 @@ static void determine_clock_access_method(const struct hwclock_control *ctl)
if (ctl->directisa)
ur = probe_for_cmos_clock();
#endif
-#ifdef __linux__
+#if defined(__linux__) || defined(__GNU__)
if (!ur)
ur = probe_for_rtc_clock(ctl);
#endif
diff --git a/sys-utils/hwclock.h b/sys-utils/hwclock.h
index 2522d6c..4cbbff9 100644
--- a/sys-utils/hwclock.h
+++ b/sys-utils/hwclock.h
@@ -36,8 +36,10 @@ struct hwclock_control {
#if defined(__linux__) && defined(__alpha__)
char *epoch_option;
#endif
-#ifdef __linux__
+#if defined(__linux__) || defined(__GNU__)
char *rtc_dev_name;
+#endif
+#ifdef __linux__
uint32_t param_idx; /* --param-index <n> */
#endif
char *param_get_option;
--
2.45.2
^ permalink raw reply related
* [RFC PATCH] hwclock: Remove ioperm declare as it causes nested extern declare warning
From: Zhaoming Luo @ 2024-12-09 2:11 UTC (permalink / raw)
To: util-linux; +Cc: samuel.thibault
The extern declaration causes the following warning:
```
CC sys-utils/hwclock-hwclock-cmos.o
sys-utils/hwclock-cmos.c: In function 'i386_iopl':
sys-utils/hwclock-cmos.c:356:20: warning: nested extern declaration of 'ioperm' [-Wnested-externs]
356 | extern int ioperm(unsigned long from, unsigned long num, int turn_on);
| ^~~~~~
sys-utils/hwclock-cmos.c:356:20: warning: redundant redeclaration of 'ioperm' [-Wredundant-decls]
In file included from sys-utils/hwclock-cmos.c:68:
/usr/include/i386-gnu/sys/io.h:29:12: note: previous declaration of 'ioperm' with type 'int(long unsigned int, long unsigned int, int)'
29 | extern int ioperm (unsigned long int __from, unsigned long int __num,
| ^~~~~~
CCLD hwclock
```
---
sys-utils/hwclock-cmos.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/sys-utils/hwclock-cmos.c b/sys-utils/hwclock-cmos.c
index 7e5eecc7d..c7f82437d 100644
--- a/sys-utils/hwclock-cmos.c
+++ b/sys-utils/hwclock-cmos.c
@@ -352,7 +352,6 @@ static int i386_iopl(const int level)
# else
static int i386_iopl(const int level __attribute__ ((__unused__)))
{
- extern int ioperm(unsigned long from, unsigned long num, int turn_on);
return ioperm(clock_ctl_addr, 2, 1);
}
# endif
--
2.47.1
^ permalink raw reply related
* Ensuring that mount(8) will always interpret a filesystem correctly
From: Demi Marie Obenour @ 2024-12-08 1:45 UTC (permalink / raw)
To: util-linux
Is there a guarantee that if all data before the filesystem superblock is
zero, and that the filesystem never writes to this region, libblkid (and
thus, presumably, mount(8)) will always mount the filesystem with the
correct filesystem type, even if e.g. someone writes a file containing
a superblock of a different filesystem and the filesystem happens to put
it where that superblock is valid?
The motivation for this message is that systemd-gpt-generator generates
mountpoints based on Discoverable Partition Specification GUIDs. These
indicate the mountpoint of the partition but not the filesystem type.
If a correctly-produced filesystem image will always continue to be
recognized as the correct type, this is fine. Otherwise, an unlucky
combination of writes to the filesystem and filesystem allocation decisions
could cause the filesystem to start being mounted as the wrong type, which
would be very bad. According to https://github.com/util-linux/util-linux/issues/1305,
libblkid can indeed probe for subsequent superblocks after the first one it
finds.
--
Sincerely,
Demi Marie Obenour (she/her/hers)
^ permalink raw reply
* Re: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Krister Johansen @ 2024-11-22 8:54 UTC (permalink / raw)
To: Karel Zak
Cc: Theodore Ts'o, util-linux@vger.kernel.org,
systemd-devel@lists.freedesktop.org, David Reaver
In-Reply-To: <cu2l7alrme6y7mm3kjmdnszyc7ka4a6zh5lw3ps3vj7igzjpkv@t7pgmn4aztkj>
On Thu, Nov 21, 2024 at 11:44:14AM +0100, Karel Zak wrote:
> On Tue, Nov 19, 2024 at 09:49:57AM GMT, Theodore Ts'o wrote:
> > The O_DIRECT read isn't a magic bullet.
>
> Hmm... the nice thing about this patch is that it only uses O_DIRECT
> when necessary (if the ordinary method fails). This means that it does
> not add complexity or overhead to regular FS probing.
Let me know if there are additional changes you'd like me to make to
this patch. I hope that it's not too unappealing. It's resolved the
problem in our environment where we went from seeing this about 20ish
times a day to none.
-K
^ permalink raw reply
* Re: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Theodore Ts'o @ 2024-11-21 15:55 UTC (permalink / raw)
To: Karel Zak
Cc: Windl, Ulrich, Krister Johansen, util-linux@vger.kernel.org,
systemd-devel@lists.freedesktop.org, David Reaver
In-Reply-To: <cu2l7alrme6y7mm3kjmdnszyc7ka4a6zh5lw3ps3vj7igzjpkv@t7pgmn4aztkj>
On Thu, Nov 21, 2024 at 11:44:14AM +0100, Karel Zak wrote:
> I doubt it will be helpful for us.
>
> I believe that EXT4_IOC_GET_SUPERBLOCK will be used with a mountpoint
> file descriptor, but libblkid works directly with the block device
> (e.g. open(/dev/sda1)) where it searches for valid filesystems.
Yeah, that's why I haven't prioritized implementing it. Higher
priority is to implement ioctls so that tune2fs will no longer need to
modify the superblock while the file system is mounted, so we can
allow prohibiting read/write access to the block device while it is
mounted. (Well, after waiting a decent interval so that distros
everywhere can update to a sufficiently new version of e2fsprogs.)
I had guessed that libblkid wouldn't be excited about trying to
determine the mountpoint and using an ioctl that required an open fd
on the mountpoint, but if I had been wrong, I would have been happy to
priotize EXT4_IOC_GET_SUPERBLOCK higher on my todo list.
> Another issue is that libblkid does not check if the device is
> mounted, so the FS prober can be triggered in all cases. It simply
> calls seek()+read() and tries to be smart.
Well, what I had been proposing was something that could be used in
by the ext[234] specific probe code.
> Ideally, we would have a generic ioctl (for block devices) to ask the
> kernel if a superblock at a specific location is valid.
>
> ioctl(fd, BLKVERIFYFS, { .fsname="ext4", .offset=123456 })
I wouldn't have thought this to be that useful since there are plenty
of file systems known by libblkid that the kernel doesn't support ---
and even if the source code exists in the kernel, there is no
guarantee that it is actually compiled into a particular kernel image.
(Exhibit 1: Reiserfs)
So would it really simplify libblkid all that much?
Cheers,
- Ted
^ permalink raw reply
* Re: lsblk vs blkid to fetch UUID
From: Karel Zak @ 2024-11-21 10:59 UTC (permalink / raw)
To: anchal agarwal; +Cc: util-linux
In-Reply-To: <CAGppiHZvn-vgiQczdTyG+F3-jwzSAMB=Kwf+jFa7UWmF_68-wQ@mail.gmail.com>
On Tue, Nov 19, 2024 at 12:15:02PM GMT, anchal agarwal wrote:
> Hi,
> I have a use case which involves imaging of a system using custom
> scripts. Imaging involves installing an operating system and
> formatting all the disks and setting up a File System.
> At times I would like to enable LUKS on certain disk partitions, so I
> format the partition using cryptsetup to setup luks format.
> However, I had been seeing issues with fetching UUID post luksFormat
> using lsblk. I have seen the same issue with blkid but its
> occurrences are exponentially less.
> I want to use lsblk because its faster but seems like blkid is more
> reliable here.
> From what I understood from the code, blkid opens a FD for the device
> and reads directly from super block; however, going through the code
> of lsblk.c I see lsblk_device_get_properties() gets it from various
> methods.
>
> I am slightly confused as at that point in time the device database is
> not set up so is lsblk fetching the UUID the same way as blkid does
> using libblkid? Any information on this will be helpful.
> Please correct me if my understanding is incorrect.
blkid (libblkid) is the primary and usually only way to read UUID from
not yet mounted filesystems. It reads it directly from block devices,
so the caller needs root permission to open the block device.
The usual setup is to call blkid from udevd when a new device or a
write change on a device is detected. Udevd gathers this information
and keeps it in the udev DB. This DB should be used as the source for
other system tools because it is (or should be) up to date and does
not require root permissions.
lsblk primarily reads information from the udev DB. If it is not
available, then it tries to read it directly from the device, which is
useless for non-root users.
If you get incomplete information from lsblk, it is usually because
the udevd prober is not done. The typical case when this happens is:
mkfs.foo /dev/sda1
lsblk /dev/sda1
In this case, lsblk reads data at a time when udevd is still running.
All you need to do is call "udevadm settle" before calling lsblk.
mkfs.foo /dev/sda1
udevadm settle
lsblk /dev/sda1
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
* Re: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Karel Zak @ 2024-11-21 10:44 UTC (permalink / raw)
To: Theodore Ts'o
Cc: Windl, Ulrich, Krister Johansen, util-linux@vger.kernel.org,
systemd-devel@lists.freedesktop.org, David Reaver
In-Reply-To: <20241119174957.GA3484088@mit.edu>
On Tue, Nov 19, 2024 at 09:49:57AM GMT, Theodore Ts'o wrote:
> The O_DIRECT read isn't a magic bullet.
Hmm... the nice thing about this patch is that it only uses O_DIRECT
when necessary (if the ordinary method fails). This means that it does
not add complexity or overhead to regular FS probing.
> For example, if you have a
> scratch file system which is guaranteed not to survive a Kubernetes or
> Borg container getting aborted, you might decide to format the file
> system without a jbd2 journal, since that would be more efficient, and
> by definition you don't care about the contents of the file system
> after a crash. So there are millions of ext4 file systems in
> hyperscale computing environments that are created without a journal;
> and in that case, O_DIRECT will not be sufficient for guaranteeing a
> consistent read of the superblock.
>
> In the long term, I'll probably be adding an ioctl which will allow
> userspace to read the superblock consistently for a mounted file
> system. We actually already have ioctls, EXT4_IOC_GETFSUUID and
> FS_IOC_GETFSLABEL which will allow userspace to fetch the UUID and
> Label for a mounted file system. So eventually, I'll probably end
> up adding EXT4_IOC_GET_SUPERBLOCK. Let me know if this is something
> that util-linux would very much want.
I doubt it will be helpful for us.
I believe that EXT4_IOC_GET_SUPERBLOCK will be used with a mountpoint
file descriptor, but libblkid works directly with the block device
(e.g. open(/dev/sda1)) where it searches for valid filesystems.
Another issue is that libblkid does not check if the device is
mounted, so the FS prober can be triggered in all cases. It simply
calls seek()+read() and tries to be smart.
Ideally, we would have a generic ioctl (for block devices) to ask the
kernel if a superblock at a specific location is valid.
ioctl(fd, BLKVERIFYFS, { .fsname="ext4", .offset=123456 })
This would greatly simplify the complex processes currently in
userspace, and it would remove duplicity. Currently, we have many file
system checks in libblkid to verify that the on-disk data are not
random bytes. These same checks are also present in file system
drivers when reading the superblock.
Another userspace dream ... :-)
> Note: this does require figuring out (a) whether the file system is
> mounted, and (b) if so, where is it mounted. So if blkid wants to use
> this, it would need to have something like the function
> ext2fs_check_mount_point[1].
Yes, that's not good.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
* Re: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Theodore Ts'o @ 2024-11-20 6:07 UTC (permalink / raw)
To: Krister Johansen
Cc: Windl, Ulrich, util-linux@vger.kernel.org, Karel Zak,
systemd-devel@lists.freedesktop.org, David Reaver
In-Reply-To: <20241119235953.GA1865@templeofstupid.com>
On Tue, Nov 19, 2024 at 03:59:53PM -0800, Krister Johansen wrote:
>
> Thanks for the additional detail on jbd2's involvement. When I
> originally encountered this, it was on a 5.15 kernel where
> ext4_commit_super() was still using mark_buffer_dirty() prior to
> submitting the IO for the superblock write. I had managed to convince
> myself that ext4_commit_super() holding the BH_lock combined with
> O_DIRECT waiting for the dirty buffers associated with the superblock to
> get written was sufficient to get a consistent read of the superblock.
> I missed that this was changed as part of another bugfix[1].
Actually, even in 5.15, ext4_commit_super() only gets used (a) during
the mount paths before the journal has been initialized, (b) in the
umount code paths after the journal has been shutdown, (c) for file
systems without a journal, or (d) when the journal update has failed ---
for example, if the journal was aborted due to catastrophic failure.
Most of the time during normal operation, say, when the file system is
being resized, or the orphan list is being actively manipulated during
a huge number of unlinks or truncates, or changing the UUID using
EXT4_IOC_SETFSUUID, etc., the kernel updates the superblock using a
jbd2 transaction, and not ext4_commit_super(). So the change which
you cited in [1] doesn't make a change in practice unless the journal
can't be used for some reason.
If I remember correctly, the patch submitter for [1] hit the problem
they were trying to fix after a journal abort while they were doing
fault injection to test I/O error handling. (In other words, case (d)
above.)
- Ted
> [1] https://lore.kernel.org/all/20220520023216.3065073-1-yi.zhang@huawei.com/
^ permalink raw reply
* Re: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Krister Johansen @ 2024-11-19 23:59 UTC (permalink / raw)
To: Theodore Ts'o
Cc: Windl, Ulrich, util-linux@vger.kernel.org, Karel Zak,
systemd-devel@lists.freedesktop.org, David Reaver
In-Reply-To: <20241119174957.GA3484088@mit.edu>
On Tue, Nov 19, 2024 at 09:49:57AM -0800, Theodore Ts'o wrote:
> Yes, this can happen if the file system is mounted. The reason for
> this is that the kernel updates metadata blocks via the block buffer
> cache, with the jbd2 (journaled block layer v2) subsystem managing the
> atomic updates. The jbd2 layer will block buffer cache writebacks
> until the changes are committed in a jbd2 transaction. So the version
> on disk is guaranteed to be consistent.
>
> However, a buffer cache read does not have any consistency guarantees,
> and if the file system is being actively modified, it is possible that
> you could a superblock where the checksum hasn't yet been updated.
>
> The O_DIRECT read isn't a magic bullet. For example, if you have a
> scratch file system which is guaranteed not to survive a Kubernetes or
> Borg container getting aborted, you might decide to format the file
> system without a jbd2 journal, since that would be more efficient, and
> by definition you don't care about the contents of the file system
> after a crash. So there are millions of ext4 file systems in
> hyperscale computing environments that are created without a journal;
> and in that case, O_DIRECT will not be sufficient for guaranteeing a
> consistent read of the superblock.
Thanks for the additional detail on jbd2's involvement. When I
originally encountered this, it was on a 5.15 kernel where
ext4_commit_super() was still using mark_buffer_dirty() prior to
submitting the IO for the superblock write. I had managed to convince
myself that ext4_commit_super() holding the BH_lock combined with
O_DIRECT waiting for the dirty buffers associated with the superblock to
get written was sufficient to get a consistent read of the superblock.
I missed that this was changed as part of another bugfix[1].
The version of this fix that you applied for resize2fs has resulted in
no re-occurence of the problem in the environments where we had been
previously encountering the problem.
With libblkid, it's resulted in systemd-udevd removing
/dev/disk/by-label and /dev/disk/by-uuid links for devices when the
superblock checksum can't be read. This in turn has resulted in /boot
failing to mount (when it's on a separate filesystem), update-grub calls
failing because /boot isn't mounted, and we recently had a mkinitramfs
fail because the /dev/disk/by-uuid links were missing for the root
device.
The patch I sent has resolved the problems in our production
environments, and was also run through a battery of synthetic boot
tests. We've seen no re-occurence with it applied. I've also run the
change against the util-linux unit tests and observed no regressions.
I included systemd-devel on this in case other users were observing
disappearing /dev/disk/ links. I hoped I might save somebody else from
having to debug this a second time.
-K
[1] https://lore.kernel.org/all/20220520023216.3065073-1-yi.zhang@huawei.com/
^ permalink raw reply
* lsblk vs blkid to fetch UUID
From: anchal agarwal @ 2024-11-19 20:15 UTC (permalink / raw)
To: util-linux; +Cc: Karel Zak
Hi,
I have a use case which involves imaging of a system using custom
scripts. Imaging involves installing an operating system and
formatting all the disks and setting up a File System.
At times I would like to enable LUKS on certain disk partitions, so I
format the partition using cryptsetup to setup luks format.
However, I had been seeing issues with fetching UUID post luksFormat
using lsblk. I have seen the same issue with blkid but its
occurrences are exponentially less.
I want to use lsblk because its faster but seems like blkid is more
reliable here.
From what I understood from the code, blkid opens a FD for the device
and reads directly from super block; however, going through the code
of lsblk.c I see lsblk_device_get_properties() gets it from various
methods.
I am slightly confused as at that point in time the device database is
not set up so is lsblk fetching the UUID the same way as blkid does
using libblkid? Any information on this will be helpful.
Please correct me if my understanding is incorrect.
Thanks & Regards,
Anchal Agarwal
^ permalink raw reply
* Re: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Theodore Ts'o @ 2024-11-19 17:49 UTC (permalink / raw)
To: Windl, Ulrich
Cc: Krister Johansen, util-linux@vger.kernel.org, Karel Zak,
systemd-devel@lists.freedesktop.org, David Reaver
In-Reply-To: <7bc43689ca4249f18c60fa4b063822ea@ukr.de>
On Tue, Nov 19, 2024 at 08:15:29AM +0000, Windl, Ulrich wrote:
> > Reads of ext superblocks can race with updates. If libblkid observes a
> [Windl, Ulrich]
>
> I really wonder:
>
> Can one single block be inconsistent when read, considering that the
> block on disk is not inconsistent? That would mean that the block
> buffer you are reading is being modified by another process. AFAIK
> the basic UNIX semantic guarantee that a block is read atomically;
> if it's not, something is severely broken, and I don't think that
> O_DIRECT fixes that.
Yes, this can happen if the file system is mounted. The reason for
this is that the kernel updates metadata blocks via the block buffer
cache, with the jbd2 (journaled block layer v2) subsystem managing the
atomic updates. The jbd2 layer will block buffer cache writebacks
until the changes are committed in a jbd2 transaction. So the version
on disk is guaranteed to be consistent.
However, a buffer cache read does not have any consistency guarantees,
and if the file system is being actively modified, it is possible that
you could a superblock where the checksum hasn't yet been updated.
The O_DIRECT read isn't a magic bullet. For example, if you have a
scratch file system which is guaranteed not to survive a Kubernetes or
Borg container getting aborted, you might decide to format the file
system without a jbd2 journal, since that would be more efficient, and
by definition you don't care about the contents of the file system
after a crash. So there are millions of ext4 file systems in
hyperscale computing environments that are created without a journal;
and in that case, O_DIRECT will not be sufficient for guaranteeing a
consistent read of the superblock.
In the long term, I'll probably be adding an ioctl which will allow
userspace to read the superblock consistently for a mounted file
system. We actually already have ioctls, EXT4_IOC_GETFSUUID and
FS_IOC_GETFSLABEL which will allow userspace to fetch the UUID and
Label for a mounted file system. So eventually, I'll probably end
up adding EXT4_IOC_GET_SUPERBLOCK. Let me know if this is something
that util-linux would very much want.
Note: this does require figuring out (a) whether the file system is
mounted, and (b) if so, where is it mounted. So if blkid wants to use
this, it would need to have something like the function
ext2fs_check_mount_point[1].
Cheers,
- Ted
[1] https://github.com/tytso/e2fsprogs/blob/950a0d69c82b585aba30118f01bf80151deffe8c/lib/ext2fs/ismounted.c#L363
^ permalink raw reply
* RE: [EXT] Re: [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Windl, Ulrich @ 2024-11-19 8:19 UTC (permalink / raw)
To: Krister Johansen, Lennart Poettering
Cc: util-linux@vger.kernel.org, Karel Zak,
systemd-devel@lists.freedesktop.org, David Reaver,
Theodore Ts'o
In-Reply-To: <20241118231352.GC1885@templeofstupid.com>
> -----Original Message-----
> From: systemd-devel <systemd-devel-bounces@lists.freedesktop.org> On
> Behalf Of Krister Johansen
> Sent: Tuesday, November 19, 2024 12:14 AM
> To: Lennart Poettering <lennart@poettering.net>
> Cc: util-linux@vger.kernel.org; Karel Zak <kzak@redhat.com>; systemd-
> devel@lists.freedesktop.org; David Reaver <me@davidreaver.com>;
> Theodore Ts'o <tytso@mit.edu>
> Subject: [EXT] Re: [systemd-devel] [PATCH] libblkid: fix spurious ext
> superblock checksum mismatches
>
...
> I may have done a poor job of explaining this. This is ext writing its
> own superblock from the kernel, but reads seeing an potentially
> inconsistent view of that write. O_DIRECT causes us to seralize with
> the locks ext4 holds when it writes the superblock, which prevents the
> read from observing a partial update.
>
> It's not necessarily the partitioning tools causing this, but any
> filesystem level udpdate that modifies the contents of the superblock.
As I wrote before: I don't think the needless O_DIRECT fixes things some other code broke.
^ permalink raw reply
* RE: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Windl, Ulrich @ 2024-11-19 8:15 UTC (permalink / raw)
To: Krister Johansen, util-linux@vger.kernel.org
Cc: Karel Zak, systemd-devel@lists.freedesktop.org, David Reaver,
Theodore Ts'o
In-Reply-To: <6d16e6d83ab48d2ea4402db17c9c0ed5514933a7.1731961869.git.kjlx@templeofstupid.com>
> -----Original Message-----
> From: systemd-devel <systemd-devel-bounces@lists.freedesktop.org> On
> Behalf Of Krister Johansen
> Sent: Monday, November 18, 2024 9:35 PM
> To: util-linux@vger.kernel.org
> Cc: Karel Zak <kzak@redhat.com>; systemd-devel@lists.freedesktop.org;
> David Reaver <me@davidreaver.com>; Theodore Ts'o <tytso@mit.edu>
> Subject: [EXT] [systemd-devel] [PATCH] libblkid: fix spurious ext superblock
> checksum mismatches
>
> Reads of ext superblocks can race with updates. If libblkid observes a
[Windl, Ulrich]
I really wonder:
Can one single block be inconsistent when read, considering that the block on disk is not inconsistent?
That would mean that the block buffer you are reading is being modified by another process.
AFAIK the basic UNIX semantic guarantee that a block is read atomically; if it's not, something is severely broken, and I don't think that O_DIRECT fixes that.
^ permalink raw reply
* Re: [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Krister Johansen @ 2024-11-18 23:13 UTC (permalink / raw)
To: Lennart Poettering
Cc: util-linux, Karel Zak, systemd-devel, David Reaver,
Theodore Ts'o
In-Reply-To: <ZzvBgOP_skwId4ci@gardel-login>
On Mon, Nov 18, 2024 at 11:36:48PM +0100, Lennart Poettering wrote:
> On Mo, 18.11.24 12:35, Krister Johansen (kjlx@templeofstupid.com) wrote:
>
> > Reads of ext superblocks can race with updates. If libblkid observes a
> > checksum mismatch, re-read the superblock with O_DIRECT in order to get
> > a consistent view of its contents. Only if the O_DIRECT read fails the
> > checksum should it be reported to have failed.
> >
> > This fixes a problem where devices that were named by filesystem label
> > failed to be found when systemd attempted to mount them on boot. The
> > problem was caused by systemd-udevd using libblkid. If a read of a
> > superblock resulted in a checksum mismatch, udev will remove the
> > by-label links which result in the mount call failing to find the
> > device. The checksum mismatch that was triggering the problem was
> > spurious, and when we use O_DIRECT, or even perform a subsequent retry,
> > the superblock is correctly read. This resulted in a failure to mount
> > /boot in one out of every 2,000 or so attempts in our environment.
> >
> > e2fsprogs fixed[1] an identical version of this bug that afflicted
> > resize2fs during online grow operations when run from cloud-init. The
> > fix there was also to use O_DIRECT in order to read the superblock.
> > This patch uses a similar approach: read the superblock with O_DIRECT in
> > the case where a bad checksum is detected.
>
> Umpf. udev has a clearly defined protocol to comprehensively avoid
> such issues:
>
> https://systemd.io/BLOCK_DEVICE_LOCKING
>
> Partitioning tools should simply follow this logic, and udev and
> programs downstream from it will not even be tempted to operate with
> half-written superblocks, partition tables or such.
>
> Hence, I personally am not convinced of that O_DIRECT approach. First
> of all, it only works on superblocks that have a useful checksum
> covering enough relevant data, and it can never really catch scenarios
> where a disk is comprehensively repartitioned, i.e. one or more fs and
> partition metadata changed at the same time...
I may have done a poor job of explaining this. This is ext writing its
own superblock from the kernel, but reads seeing an potentially
inconsistent view of that write. O_DIRECT causes us to seralize with
the locks ext4 holds when it writes the superblock, which prevents the
read from observing a partial update.
It's not necessarily the partitioning tools causing this, but any
filesystem level udpdate that modifies the contents of the superblock.
-K
^ permalink raw reply
* Re: [systemd-devel] [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Lennart Poettering @ 2024-11-18 22:36 UTC (permalink / raw)
To: Krister Johansen
Cc: util-linux, Karel Zak, systemd-devel, David Reaver,
Theodore Ts'o
In-Reply-To: <6d16e6d83ab48d2ea4402db17c9c0ed5514933a7.1731961869.git.kjlx@templeofstupid.com>
On Mo, 18.11.24 12:35, Krister Johansen (kjlx@templeofstupid.com) wrote:
> Reads of ext superblocks can race with updates. If libblkid observes a
> checksum mismatch, re-read the superblock with O_DIRECT in order to get
> a consistent view of its contents. Only if the O_DIRECT read fails the
> checksum should it be reported to have failed.
>
> This fixes a problem where devices that were named by filesystem label
> failed to be found when systemd attempted to mount them on boot. The
> problem was caused by systemd-udevd using libblkid. If a read of a
> superblock resulted in a checksum mismatch, udev will remove the
> by-label links which result in the mount call failing to find the
> device. The checksum mismatch that was triggering the problem was
> spurious, and when we use O_DIRECT, or even perform a subsequent retry,
> the superblock is correctly read. This resulted in a failure to mount
> /boot in one out of every 2,000 or so attempts in our environment.
>
> e2fsprogs fixed[1] an identical version of this bug that afflicted
> resize2fs during online grow operations when run from cloud-init. The
> fix there was also to use O_DIRECT in order to read the superblock.
> This patch uses a similar approach: read the superblock with O_DIRECT in
> the case where a bad checksum is detected.
Umpf. udev has a clearly defined protocol to comprehensively avoid
such issues:
https://systemd.io/BLOCK_DEVICE_LOCKING
Partitioning tools should simply follow this logic, and udev and
programs downstream from it will not even be tempted to operate with
half-written superblocks, partition tables or such.
Hence, I personally am not convinced of that O_DIRECT approach. First
of all, it only works on superblocks that have a useful checksum
covering enough relevant data, and it can never really catch scenarios
where a disk is comprehensively repartitioned, i.e. one or more fs and
partition metadata changed at the same time...
Lennart
--
Lennart Poettering, Berlin
^ permalink raw reply
* [PATCH] libblkid: fix spurious ext superblock checksum mismatches
From: Krister Johansen @ 2024-11-18 20:35 UTC (permalink / raw)
To: util-linux; +Cc: Karel Zak, systemd-devel, David Reaver, Theodore Ts'o
Reads of ext superblocks can race with updates. If libblkid observes a
checksum mismatch, re-read the superblock with O_DIRECT in order to get
a consistent view of its contents. Only if the O_DIRECT read fails the
checksum should it be reported to have failed.
This fixes a problem where devices that were named by filesystem label
failed to be found when systemd attempted to mount them on boot. The
problem was caused by systemd-udevd using libblkid. If a read of a
superblock resulted in a checksum mismatch, udev will remove the
by-label links which result in the mount call failing to find the
device. The checksum mismatch that was triggering the problem was
spurious, and when we use O_DIRECT, or even perform a subsequent retry,
the superblock is correctly read. This resulted in a failure to mount
/boot in one out of every 2,000 or so attempts in our environment.
e2fsprogs fixed[1] an identical version of this bug that afflicted
resize2fs during online grow operations when run from cloud-init. The
fix there was also to use O_DIRECT in order to read the superblock.
This patch uses a similar approach: read the superblock with O_DIRECT in
the case where a bad checksum is detected.
[1] https://lore.kernel.org/linux-ext4/20230609042239.GA1436857@mit.edu/
Signed-off-by: Krister Johansen <kjlx@templeofstupid.com>
---
libblkid/src/blkidP.h | 5 +++++
libblkid/src/probe.c | 27 +++++++++++++++++++++++++++
libblkid/src/superblocks/ext.c | 22 ++++++++++++++++++++--
3 files changed, 52 insertions(+), 2 deletions(-)
diff --git a/libblkid/src/blkidP.h b/libblkid/src/blkidP.h
index f71e13cb3..fa2379c4d 100644
--- a/libblkid/src/blkidP.h
+++ b/libblkid/src/blkidP.h
@@ -421,6 +421,11 @@ extern const unsigned char *blkid_probe_get_buffer(blkid_probe pr,
__attribute__((nonnull))
__attribute__((warn_unused_result));
+extern const unsigned char *blkid_probe_get_buffer_direct(blkid_probe pr,
+ uint64_t off, uint64_t len)
+ __attribute__((nonnull))
+ __attribute__((warn_unused_result));
+
extern const unsigned char *blkid_probe_get_sector(blkid_probe pr, unsigned int sector)
__attribute__((nonnull))
__attribute__((warn_unused_result));
diff --git a/libblkid/src/probe.c b/libblkid/src/probe.c
index 5a17e873d..e029f5649 100644
--- a/libblkid/src/probe.c
+++ b/libblkid/src/probe.c
@@ -791,6 +791,33 @@ const unsigned char *blkid_probe_get_buffer(blkid_probe pr, uint64_t off, uint64
return real_off ? bf->data + (real_off - bf->off + bias) : bf->data + bias;
}
+/*
+ * This is blkid_probe_get_buffer with the read done as an O_DIRECT operation.
+ * Note that @off is offset within probing area, the probing area is defined by
+ * pr->off and pr->size.
+ */
+const unsigned char *blkid_probe_get_buffer_direct(blkid_probe pr, uint64_t off, uint64_t len)
+{
+ const unsigned char *ret = NULL;
+ int flags, rc, olderrno;
+
+ flags = fcntl(pr->fd, F_GETFL);
+ rc = fcntl(pr->fd, F_SETFL, flags | O_DIRECT);
+ if (rc) {
+ DBG(LOWPROBE, ul_debug("fcntl F_SETFL failed to set O_DIRECT"));
+ errno = 0;
+ return NULL;
+ }
+ ret = blkid_probe_get_buffer(pr, off, len);
+ olderrno = errno;
+ rc = fcntl(pr->fd, F_SETFL, flags);
+ if (rc) {
+ DBG(LOWPROBE, ul_debug("fcntl F_SETFL failed to clear O_DIRECT"));
+ errno = olderrno;
+ }
+ return ret;
+}
+
/**
* blkid_probe_reset_buffers:
* @pr: prober
diff --git a/libblkid/src/superblocks/ext.c b/libblkid/src/superblocks/ext.c
index a5f34810f..7a9f8c9b9 100644
--- a/libblkid/src/superblocks/ext.c
+++ b/libblkid/src/superblocks/ext.c
@@ -156,8 +156,26 @@ static struct ext2_super_block *ext_get_super(
return NULL;
if (le32_to_cpu(es->s_feature_ro_compat) & EXT4_FEATURE_RO_COMPAT_METADATA_CSUM) {
uint32_t csum = crc32c(~0, es, offsetof(struct ext2_super_block, s_checksum));
- if (!blkid_probe_verify_csum(pr, csum, le32_to_cpu(es->s_checksum)))
- return NULL;
+ /*
+ * A read of the superblock can race with other updates to the
+ * same superblock. In the unlikely event that this occurs and
+ * we see a checksum failure, re-read the superblock with
+ * O_DIRECT to ensure that it's consistent. If it _still_ fails
+ * then declare a checksum mismatch.
+ */
+ if (!blkid_probe_verify_csum(pr, csum, le32_to_cpu(es->s_checksum))) {
+ if (blkid_probe_reset_buffers(pr))
+ return NULL;
+
+ es = (struct ext2_super_block *)
+ blkid_probe_get_buffer_direct(pr, EXT_SB_OFF, sizeof(struct ext2_super_block));
+ if (!es)
+ return NULL;
+
+ csum = crc32c(~0, es, offsetof(struct ext2_super_block, s_checksum));
+ if (!blkid_probe_verify_csum(pr, csum, le32_to_cpu(es->s_checksum)))
+ return NULL;
+ }
}
if (fc)
*fc = le32_to_cpu(es->s_feature_compat);
--
2.25.1
^ permalink raw reply related
* Chef Knife For vger
From: Melissa Lazaro @ 2024-11-14 18:42 UTC (permalink / raw)
To: util-linux
Hey there
vger ,
I hope all is well with you! I am Melissa Lazaro representing SAKUTO KNIVES, We design and manufacture superior Damascus Chef Knives.
We've been staying updated on vger, for a considerable period, and The value and allure of your content always impress us.
With a desire to work together, We are eager to present a superior Damascus Kitchen Knife ( Charge at 199. 99) as a gesture of gratitude in exchange for a bac klink.
Here's a beneficial proposal: We'd be thrilled to send you a premium Damascus Chef Knife fre e of cost.
We create an intricately developed article ( human-written by our professional writers high-quality content ) that has completed five A I Assessment tools and Never published .
Should it catch your attention, could you please reply with your shipping address? and we'll gladly forward you( vger ) the knife and the content for your constructive review.
Best wishes ,
Melissa Lazaro
^ permalink raw reply
* lsblk.8: Some remarks about this man page
From: Bjarni Ingi Gislason @ 2024-11-12 1:10 UTC (permalink / raw)
To: util-linux
[-- Attachment #1: Type: text/plain, Size: 1871 bytes --]
Package: util-linux (Debian testing)
Version: 2.40.2-9
Severity: minor
* What led up to the situation?
Checking for defects with
test-[g|n]roff -mandoc -t -K utf8 -rF0 -rHY=0 -ww -b -z < "man page"
[Use "groff -e ' $' <file>" to find trailing spaces.]
["test-groff" is a script in the repository for "groff"; is not shipped]
(local copy and "troff" slightly changed by me).
[The fate of "test-nroff" was decided in groff bug #55941.]
* What was the outcome of this action?
troff: backtrace: file '<stdin>':276
troff:<stdin>:276: warning: [page 4, 3.6i]: cannot break line
* What outcome did you expect instead?
No output (no warnings).
-.-
General remarks and further material, if a diff-file exist, are in the
attachments.
-- System Information:
Debian Release: trixie/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: amd64 (x86_64)
Kernel: Linux 6.11.5-amd64 (SMP w/2 CPU threads; PREEMPT)
Locale: LANG=is_IS.iso88591, LC_CTYPE=is_IS.iso88591 (charmap=ISO-8859-1), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: sysvinit (via /sbin/init)
Versions of packages util-linux depends on:
ii libblkid1 2.40.2-9
ii libc6 2.40-3
ii libcap-ng0 0.8.5-3
ii libcrypt1 1:4.4.36-5
ii libmount1 2.40.2-9
ii libpam-modules 1.5.3-7+b1
ii libpam-runtime 1.5.3-7
ii libpam0g 1.5.3-7+b1
ii libselinux1 3.7-3
ii libsmartcols1 2.40.2-9
ii libsystemd0 256.7-3
ii libtinfo6 6.5-2+b1
ii libudev1 256.7-3
ii libuuid1 2.40.2-9
Versions of packages util-linux recommends:
ii sensible-utils 0.0.24
Versions of packages util-linux suggests:
pn dosfstools <none>
ii kbd 2.6.4-3
ii util-linux-extra 2.40.2-9
pn util-linux-locales <none>
ii wtmpdb 0.13.0-1
-- no debconf information
[-- Attachment #2: chk_man.err.lsblk.8 --]
[-- Type: text/plain, Size: 26444 bytes --]
Any program (person), that produces man pages, should check the output
for defects by using (both groff and nroff)
[gn]roff -mandoc -t -ww -b -z -K utf8 <man page>
The same goes for man pages that are used as an input.
For a style guide use
mandoc -T lint
-.-
So any 'generator' should check its products with the above mentioned
'groff', 'mandoc', and additionally with 'nroff ...'.
This is just a simple quality control measure.
The 'generator' may have to be corrected to get a better man page,
the source file may, and any additional file may.
Common defects:
Input text line longer than 80 bytes.
Not removing trailing spaces (in in- and output).
The reason for these trailing spaces should be found and eliminated.
Not beginning each input sentence on a new line.
Lines should thus be shorter.
See man-pages(7), item 'semantic newline'.
-.-
The difference between the formatted output of the original and patched file
can be seen with:
nroff -mandoc <file1> > <out1>
nroff -mandoc <file2> > <out2>
diff -u <out1> <out2>
and for groff, using
"printf '%s\n%s\n' '.kern 0' '.ss 12 0' | groff -mandoc -Z - "
instead of 'nroff -mandoc'
Add the option '-t', if the file contains a table.
Read the output of 'diff -u' with 'less -R' or similar.
-.-.
If 'man' (man-db) is used to check the manual for warnings,
the following must be set:
The option "-warnings=w"
The environmental variable:
export MAN_KEEP_STDERR=yes (or any non-empty value)
or
(produce only warnings):
export MANROFFOPT="-ww -b -z"
export MAN_KEEP_STDERR=yes (or any non-empty value)
-.-.
Output from "mandoc -T lint lsblk.8": (possibly shortened list)
mandoc: lsblk.8:28:4: ERROR: skipping unknown macro: LINKSTYLE blue R < >
mandoc: lsblk.8:33:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:36:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:39:305: STYLE: input text line longer than 80 bytes: By default, the comm...
mandoc: lsblk.8:41:253: STYLE: input text line longer than 80 bytes: The tree\-like outpu...
mandoc: lsblk.8:43:362: STYLE: input text line longer than 80 bytes: The default output, ...
mandoc: lsblk.8:47:243: STYLE: input text line longer than 80 bytes: Note that \fBlsblk\f...
mandoc: lsblk.8:49:475: STYLE: input text line longer than 80 bytes: The relationship bet...
mandoc: lsblk.8:51:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:59:82: STYLE: input text line longer than 80 bytes: Disable all built\-i...
mandoc: lsblk.8:75:111: STYLE: input text line longer than 80 bytes: List the available c...
mandoc: lsblk.8:80:82: STYLE: input text line longer than 80 bytes: Print information ab...
mandoc: lsblk.8:85:129: STYLE: input text line longer than 80 bytes: Do not print holder ...
mandoc: lsblk.8:90:233: STYLE: input text line longer than 80 bytes: Use \fIcolumn\fP as ...
mandoc: lsblk.8:92:106: STYLE: input text line longer than 80 bytes: The usual use case i...
mandoc: lsblk.8:97:334: STYLE: input text line longer than 80 bytes: Exclude the devices ...
mandoc: lsblk.8:102:227: STYLE: input text line longer than 80 bytes: Output info about fi...
mandoc: lsblk.8:107:241: STYLE: input text line longer than 80 bytes: Include devices spec...
mandoc: lsblk.8:117:196: STYLE: input text line longer than 80 bytes: Use JSON output form...
mandoc: lsblk.8:122:288: STYLE: input text line longer than 80 bytes: Produce output in th...
mandoc: lsblk.8:127:129: STYLE: input text line longer than 80 bytes: Group parents of sub...
mandoc: lsblk.8:132:114: STYLE: input text line longer than 80 bytes: Output info about de...
mandoc: lsblk.8:152:232: STYLE: input text line longer than 80 bytes: Specify which output...
mandoc: lsblk.8:154:127: STYLE: input text line longer than 80 bytes: The default list of ...
mandoc: lsblk.8:164:205: STYLE: input text line longer than 80 bytes: Produce output in th...
mandoc: lsblk.8:181:83: STYLE: input text line longer than 80 bytes: exclude sda and sdb...
mandoc: lsblk.8:207:100: STYLE: input text line longer than 80 bytes: The \fIname\fP is th...
mandoc: lsblk.8:212:86: STYLE: input text line longer than 80 bytes: If the \fIparam\fP i...
mandoc: lsblk.8:215:92: STYLE: input text line longer than 80 bytes: For example, \fB\-\-...
mandoc: lsblk.8:234:94: STYLE: input text line longer than 80 bytes: Define a restriction...
mandoc: lsblk.8:254:207: STYLE: input text line longer than 80 bytes: Produce output in ra...
mandoc: lsblk.8:259:91: STYLE: input text line longer than 80 bytes: Output info about SC...
mandoc: lsblk.8:264:128: STYLE: input text line longer than 80 bytes: Print dependencies i...
mandoc: lsblk.8:269:127: STYLE: input text line longer than 80 bytes: Force tree\-like out...
mandoc: lsblk.8:291:402: STYLE: input text line longer than 80 bytes: Specifies output wid...
mandoc: lsblk.8:296:229: STYLE: input text line longer than 80 bytes: Sort output lines by...
mandoc: lsblk.8:301:406: STYLE: input text line longer than 80 bytes: The column name will...
mandoc: lsblk.8:311:282: STYLE: input text line longer than 80 bytes: Gather data for a Li...
mandoc: lsblk.8:314:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:335:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:361:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:362:94: STYLE: input text line longer than 80 bytes: For partitions, some...
mandoc: lsblk.8:364:328: STYLE: input text line longer than 80 bytes: The \fBlsblk\fP comm...
mandoc: lsblk.8:366:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:370:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:376:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:380:2: WARNING: skipping paragraph macro: sp after SH
mandoc: lsblk.8:381:90: STYLE: input text line longer than 80 bytes: The \fBlsblk\fP comm...
-.-.
Lines containing '\c' (' \c' does not make sense):
377:For bug reports, use the issue tracker at \c
381:The \fBlsblk\fP command is part of the util\-linux package which can be downloaded from \c
-.-
Strings longer than 3/4 of a standard line length (80)
Use "\:" to split the string at the end of an output line, for example a
long URLs (web address)
276 \fB\-o NAME,ALIGNMENT,MIN\-IO,OPT\-IO,PHY\-SEC,LOG\-SEC,ROTA,SCHED,RQ\-SIZE,RA,WSAME\fP.
-.-.
Wrong distance between sentences in the input file.
Separate the sentences and subordinate clauses; each begins on a new
line. See man-pages(7) ("Conventions for source file layout") and
"info groff" ("Input Conventions").
The best procedure is to always start a new sentence on a new line,
at least, if you are typing on a computer.
Remember coding: Only one command ("sentence") on each (logical) line.
E-mail: Easier to quote exactly the relevant lines.
Generally: Easier to edit the sentence.
Patches: Less unaffected text.
Search for two adjacent words is easier, when they belong to the same line,
and the same phrase.
The amount of space between sentences in the output can then be
controlled with the ".ss" request.
37:\fBlsblk\fP lists information about all available or the specified block devices. The \fBlsblk\fP command reads the \fBsysfs\fP filesystem and \fBudev db\fP to gather information. If the udev db is not available or \fBlsblk\fP is compiled without udev support, then it tries to read LABELs, UUIDs and filesystem types from the block device. In this case root permissions are necessary.
39:By default, the command prints all block devices (except RAM disks) in a tree\-like format. The same device can be repeated in the tree if it relates to other devices. The \fB\-\-merge\fP option is recommended for more complicated setups to gather groups of devices and describe complex N:M relationships.
41:The tree\-like output (or \fBchildren[]\fP array in the JSON output) is enabled only if NAME column it present in the output or when \fB\-\-tree\fP command line option is used. See also \fB\-\-nodeps\fP and \fB\-\-list\fP to control the tree formatting.
43:The default output, as well as the default output from options like \fB\-\-fs\fP and \fB\-\-topology\fP, is subject to change. So whenever possible, you should avoid using default outputs in your scripts. Always explicitly define expected columns by using \fB\-\-output\fP \fIcolumns\-list\fP and \fB\-\-list\fP in environments where a stable output is required.
47:Note that \fBlsblk\fP might be executed in time when \fBudev\fP does not have all information about recently added or modified devices yet. In this case it is recommended to use \fBudevadm settle\fP before \fBlsblk\fP to synchronize with udev.
49:The relationship between block devices and filesystems is not always one\-to\-one. The filesystem may use more block devices, or the same filesystem may be accessible by more paths. This is the reason why \fBlsblk\fP provides MOUNTPOINT and MOUNTPOINTS (pl.) columns. The column MOUNTPOINT displays only one mount point (usually the last mounted instance of the filesystem), and the column MOUNTPOINTS displays by multi\-line cell all mount points associated with the device.
67:power of 2^10 (1024). Abbreviations of symbols are exhibited truncated in order
85:Do not print holder devices or slaves. For example, \fBlsblk \-\-nodeps /dev/sda\fP prints information about the sda device only.
90:Use \fIcolumn\fP as a de\-duplication key to de\-duplicate output tree. If the key is not available for the device, or the device is a partition and parental whole\-disk device provides the same key than the device is always printed.
97:Exclude the devices specified by the comma\-separated \fIlist\fP of major device numbers. Note that RAM disks (major=1) are excluded by default if \fB\-\-all\fP is not specified. The filter is applied to the top\-level devices only. This may be confusing for \fB\-\-list\fP output format where hierarchy of the devices is not obvious.
102:Output info about filesystems. This option is equivalent to \fB\-o NAME,FSTYPE,FSVER,LABEL,UUID,FSAVAIL,FSUSE%,MOUNTPOINTS\fP. The authoritative information about filesystems and raids is provided by the \fBblkid\fP(8) command.
107:Include devices specified by the comma\-separated \fIlist\fP of major device numbers. The filter is applied to the top\-level devices only. This may be confusing for \fB\-\-list\fP output format where hierarchy of the devices is not obvious.
117:Use JSON output format. It\(cqs strongly recommended to use \fB\-\-output\fP and also \fB\-\-tree\fP if necessary. Note that \fBchildren[]\fP is used only if NAME column or \fB\-\-tree\fP is used.
122:Produce output in the form of a list. The output does not provide information about relationships between devices and since version 2.34 every device is printed only once if \fB\-\-pairs\fP or \fB\-\-raw\fP not specified (the parsable outputs are maintained in backwardly compatible way).
127:Group parents of sub\-trees to provide more readable output for RAIDs and Multi\-path devices. The tree\-like output is required.
132:Output info about device owner, group and mode. This option is equivalent to \fB\-o NAME,SIZE,OWNER,GROUP,MODE\fP.
152:Specify which output columns to print. Use \fB\-\-list\-columns\fP to get a list of all supported columns. The columns may affect tree\-like output. The default is to use tree for the column \*(AqNAME\*(Aq (see also \fB\-\-tree\fP).
164:Produce output in the form of key="value" pairs. The output lines are still ordered by dependencies. All potentially unsafe value characters are hex\-escaped (\(rsx<code>). See also option \fB\-\-shell\fP.
174:Print only the devices that meet the conditions specified by the expr. The
175:filter is assessed prior to lsblk collecting data for all output columns. Only
177:the system. This approach can enhance performance when compared to
180:This feature is EXPERIMENTAL. See also \fBscols\-filter\fP(5). For example
201:This feature is EXPERIMENTAL. See also \fBscols\-filter\fP(5).
206:Define a custom counter. The counters are printed after the standard output.
209:the aggregation function, supported functions are: count, min, max, or sum. The
212:If the \fIparam\fP is not specified, then the counter counts the number of lines. This
213:feature is EXPERIMENTAL. See also \fB\-\-ct\-filter\fP.
234:Define a restriction for the next counter. This feature is EXPERIMENTAL. See also \fB\-\-ct\fP
235:and \fBscols\-filter\fP(5). For example, aggregate sizes by device type:
254:Produce output in raw format. The output lines are still ordered by dependencies. All potentially unsafe characters are hex\-escaped (\(rsx<code>) in the NAME, KNAME, LABEL, PARTLABEL and MOUNTPOINT columns.
259:Output info about SCSI devices only. All partitions, slaves and holder devices are ignored.
264:Print dependencies in inverse order. If the \fB\-\-list\fP output is requested then the lines are still ordered by dependencies.
269:Force tree\-like output format. If \fIcolumn\fP is specified, then a tree is printed in the column. The default is NAME column.
274:Output info about block\-device topology. This option is equivalent to
291:Specifies output width as a number of characters. The default is the number of the terminal columns, and if not executed on a terminal, then output width is not restricted at all by default. This option also forces \fBlsblk\fP to assume that terminal control characters and unsafe characters are not allowed. The expected use\-case is for example when \fBlsblk\fP is used by the \fBwatch\fP(1) command.
296:Sort output lines by \fIcolumn\fP. This option enables \fB\-\-list\fP output format by default. It is possible to use the option \fB\-\-tree\fP to force tree\-like output and than the tree branches are sorted by the \fIcolumn\fP.
301:The column name will be modified to contain only characters allowed for shell variable identifiers, for example, MIN_IO and FSUSE_PCT instead of MIN\-IO and FSUSE%. This is usable, for example, with \fB\-\-pairs\fP. Note that this feature has been automatically enabled for \fB\-\-pairs\fP in version 2.37, but due to compatibility issues, now it\(cqs necessary to request this behavior by \fB\-\-shell\fP.
311:Gather data for a Linux instance other than the instance from which the \fBlsblk\fP command is issued. The specified directory is the system root of the Linux instance to be inspected. The real device nodes in the target directory can be replaced by text files with udev attributes.
364:The \fBlsblk\fP command needs to be able to look up each block device by major:minor numbers, which is done by using \fI/sys/dev/block\fP. This sysfs block directory appeared in kernel 2.6.27 (October 2008). In case of problems with a new enough kernel, check that \fBCONFIG_SYSFS\fP was enabled at the time of the kernel build.
-.-.
Split lines longer than 80 characters into two or more lines.
Appropriate break points are the end of a sentence and a subordinate
clause; after punctuation marks.
Line 37, length 385
\fBlsblk\fP lists information about all available or the specified block devices. The \fBlsblk\fP command reads the \fBsysfs\fP filesystem and \fBudev db\fP to gather information. If the udev db is not available or \fBlsblk\fP is compiled without udev support, then it tries to read LABELs, UUIDs and filesystem types from the block device. In this case root permissions are necessary.
Line 39, length 305
By default, the command prints all block devices (except RAM disks) in a tree\-like format. The same device can be repeated in the tree if it relates to other devices. The \fB\-\-merge\fP option is recommended for more complicated setups to gather groups of devices and describe complex N:M relationships.
Line 41, length 253
The tree\-like output (or \fBchildren[]\fP array in the JSON output) is enabled only if NAME column it present in the output or when \fB\-\-tree\fP command line option is used. See also \fB\-\-nodeps\fP and \fB\-\-list\fP to control the tree formatting.
Line 43, length 362
The default output, as well as the default output from options like \fB\-\-fs\fP and \fB\-\-topology\fP, is subject to change. So whenever possible, you should avoid using default outputs in your scripts. Always explicitly define expected columns by using \fB\-\-output\fP \fIcolumns\-list\fP and \fB\-\-list\fP in environments where a stable output is required.
Line 47, length 243
Note that \fBlsblk\fP might be executed in time when \fBudev\fP does not have all information about recently added or modified devices yet. In this case it is recommended to use \fBudevadm settle\fP before \fBlsblk\fP to synchronize with udev.
Line 49, length 475
The relationship between block devices and filesystems is not always one\-to\-one. The filesystem may use more block devices, or the same filesystem may be accessible by more paths. This is the reason why \fBlsblk\fP provides MOUNTPOINT and MOUNTPOINTS (pl.) columns. The column MOUNTPOINT displays only one mount point (usually the last mounted instance of the filesystem), and the column MOUNTPOINTS displays by multi\-line cell all mount points associated with the device.
Line 59, length 82
Disable all built\-in filters and list all empty devices and RAM disk devices too.
Line 75, length 111
List the available columns, use with \fB\-\-json\fP or \fB\-\-raw\fP to get output in machine\-readable format.
Line 80, length 82
Print information about the discarding capabilities (TRIM, UNMAP) for each device.
Line 85, length 129
Do not print holder devices or slaves. For example, \fBlsblk \-\-nodeps /dev/sda\fP prints information about the sda device only.
Line 90, length 233
Use \fIcolumn\fP as a de\-duplication key to de\-duplicate output tree. If the key is not available for the device, or the device is a partition and parental whole\-disk device provides the same key than the device is always printed.
Line 92, length 106
The usual use case is to de\-duplicate output on system multi\-path devices, for example by \fB\-E WWN\fP.
Line 97, length 334
Exclude the devices specified by the comma\-separated \fIlist\fP of major device numbers. Note that RAM disks (major=1) are excluded by default if \fB\-\-all\fP is not specified. The filter is applied to the top\-level devices only. This may be confusing for \fB\-\-list\fP output format where hierarchy of the devices is not obvious.
Line 102, length 227
Output info about filesystems. This option is equivalent to \fB\-o NAME,FSTYPE,FSVER,LABEL,UUID,FSAVAIL,FSUSE%,MOUNTPOINTS\fP. The authoritative information about filesystems and raids is provided by the \fBblkid\fP(8) command.
Line 107, length 241
Include devices specified by the comma\-separated \fIlist\fP of major device numbers. The filter is applied to the top\-level devices only. This may be confusing for \fB\-\-list\fP output format where hierarchy of the devices is not obvious.
Line 117, length 196
Use JSON output format. It\(cqs strongly recommended to use \fB\-\-output\fP and also \fB\-\-tree\fP if necessary. Note that \fBchildren[]\fP is used only if NAME column or \fB\-\-tree\fP is used.
Line 122, length 288
Produce output in the form of a list. The output does not provide information about relationships between devices and since version 2.34 every device is printed only once if \fB\-\-pairs\fP or \fB\-\-raw\fP not specified (the parsable outputs are maintained in backwardly compatible way).
Line 127, length 129
Group parents of sub\-trees to provide more readable output for RAIDs and Multi\-path devices. The tree\-like output is required.
Line 132, length 114
Output info about device owner, group and mode. This option is equivalent to \fB\-o NAME,SIZE,OWNER,GROUP,MODE\fP.
Line 152, length 232
Specify which output columns to print. Use \fB\-\-list\-columns\fP to get a list of all supported columns. The columns may affect tree\-like output. The default is to use tree for the column \*(AqNAME\*(Aq (see also \fB\-\-tree\fP).
Line 154, length 127
The default list of columns may be extended if \fIlist\fP is specified in the format \fI+list\fP (e.g., \fBlsblk \-o +UUID\fP).
Line 164, length 205
Produce output in the form of key="value" pairs. The output lines are still ordered by dependencies. All potentially unsafe value characters are hex\-escaped (\(rsx<code>). See also option \fB\-\-shell\fP.
Line 181, length 83
exclude sda and sdb, but print everything else (\*(Aq!~\*(Aq is a negative regular
Line 207, length 100
The \fIname\fP is the custom name of the counter, the optional \fIparam\fP is the name of the column
Line 212, length 86
If the \fIparam\fP is not specified, then the counter counts the number of lines. This
Line 215, length 92
For example, \fB\-\-ct MyCounter:SIZE:sum\fP will count the summary for SIZE from all lines;
Line 224, length 91
lsblk \-\-ct\-filter \*(AqTYPE=="disk" && TRAN=="sata"\*(Aq \-\-ct "Number of SATA devices"
Line 234, length 94
Define a restriction for the next counter. This feature is EXPERIMENTAL. See also \fB\-\-ct\fP
Line 254, length 207
Produce output in raw format. The output lines are still ordered by dependencies. All potentially unsafe characters are hex\-escaped (\(rsx<code>) in the NAME, KNAME, LABEL, PARTLABEL and MOUNTPOINT columns.
Line 259, length 91
Output info about SCSI devices only. All partitions, slaves and holder devices are ignored.
Line 264, length 128
Print dependencies in inverse order. If the \fB\-\-list\fP output is requested then the lines are still ordered by dependencies.
Line 269, length 127
Force tree\-like output format. If \fIcolumn\fP is specified, then a tree is printed in the column. The default is NAME column.
Line 276, length 88
\fB\-o NAME,ALIGNMENT,MIN\-IO,OPT\-IO,PHY\-SEC,LOG\-SEC,ROTA,SCHED,RQ\-SIZE,RA,WSAME\fP.
Line 291, length 402
Specifies output width as a number of characters. The default is the number of the terminal columns, and if not executed on a terminal, then output width is not restricted at all by default. This option also forces \fBlsblk\fP to assume that terminal control characters and unsafe characters are not allowed. The expected use\-case is for example when \fBlsblk\fP is used by the \fBwatch\fP(1) command.
Line 296, length 229
Sort output lines by \fIcolumn\fP. This option enables \fB\-\-list\fP output format by default. It is possible to use the option \fB\-\-tree\fP to force tree\-like output and than the tree branches are sorted by the \fIcolumn\fP.
Line 301, length 406
The column name will be modified to contain only characters allowed for shell variable identifiers, for example, MIN_IO and FSUSE_PCT instead of MIN\-IO and FSUSE%. This is usable, for example, with \fB\-\-pairs\fP. Note that this feature has been automatically enabled for \fB\-\-pairs\fP in version 2.37, but due to compatibility issues, now it\(cqs necessary to request this behavior by \fB\-\-shell\fP.
Line 311, length 282
Gather data for a Linux instance other than the instance from which the \fBlsblk\fP command is issued. The specified directory is the system root of the Linux instance to be inspected. The real device nodes in the target directory can be replaced by text files with udev attributes.
Line 362, length 94
For partitions, some information (e.g., queue attributes) is inherited from the parent device.
Line 364, length 328
The \fBlsblk\fP command needs to be able to look up each block device by major:minor numbers, which is done by using \fI/sys/dev/block\fP. This sysfs block directory appeared in kernel 2.6.27 (October 2008). In case of problems with a new enough kernel, check that \fBCONFIG_SYSFS\fP was enabled at the time of the kernel build.
Line 381, length 90
The \fBlsblk\fP command is part of the util\-linux package which can be downloaded from \c
Line 382, length 85
.URL "https://www.kernel.org/pub/linux/utils/util\-linux/" "Linux Kernel Archive" "."
-.-.
Put a parenthetical sentence, phrase on a separate line,
if not part of a code.
See man-pages(7), item "semantic newline".
lsblk.8:39:By default, the command prints all block devices (except RAM disks) in a tree\-like format. The same device can be repeated in the tree if it relates to other devices. The \fB\-\-merge\fP option is recommended for more complicated setups to gather groups of devices and describe complex N:M relationships.
lsblk.8:41:The tree\-like output (or \fBchildren[]\fP array in the JSON output) is enabled only if NAME column it present in the output or when \fB\-\-tree\fP command line option is used. See also \fB\-\-nodeps\fP and \fB\-\-list\fP to control the tree formatting.
lsblk.8:49:The relationship between block devices and filesystems is not always one\-to\-one. The filesystem may use more block devices, or the same filesystem may be accessible by more paths. This is the reason why \fBlsblk\fP provides MOUNTPOINT and MOUNTPOINTS (pl.) columns. The column MOUNTPOINT displays only one mount point (usually the last mounted instance of the filesystem), and the column MOUNTPOINTS displays by multi\-line cell all mount points associated with the device.
lsblk.8:122:Produce output in the form of a list. The output does not provide information about relationships between devices and since version 2.34 every device is printed only once if \fB\-\-pairs\fP or \fB\-\-raw\fP not specified (the parsable outputs are maintained in backwardly compatible way).
lsblk.8:152:Specify which output columns to print. Use \fB\-\-list\-columns\fP to get a list of all supported columns. The columns may affect tree\-like output. The default is to use tree for the column \*(AqNAME\*(Aq (see also \fB\-\-tree\fP).
lsblk.8:154:The default list of columns may be extended if \fIlist\fP is specified in the format \fI+list\fP (e.g., \fBlsblk \-o +UUID\fP).
lsblk.8:362:For partitions, some information (e.g., queue attributes) is inherited from the parent device.
lsblk.8:364:The \fBlsblk\fP command needs to be able to look up each block device by major:minor numbers, which is done by using \fI/sys/dev/block\fP. This sysfs block directory appeared in kernel 2.6.27 (October 2008). In case of problems with a new enough kernel, check that \fBCONFIG_SYSFS\fP was enabled at the time of the kernel build.
-.-.
Output from "test-groff -mandoc -t -K utf8 -rF0 -rHY=0 -ww -b -z ":
troff: backtrace: file '<stdin>':276
troff:<stdin>:276: warning: [page 4, 3.6i]: cannot break line
-.-.
^ permalink raw reply
* Re: [PATCH v2] lscpu: fix incorrect number of sockets during hotplug
From: Karel Zak @ 2024-11-06 9:51 UTC (permalink / raw)
To: Anjali K; +Cc: util-linux, anushree.mathur
In-Reply-To: <20241104063226.172077-1-anjalik@linux.ibm.com>
On Mon, Nov 04, 2024 at 12:02:26PM GMT, Anjali K wrote:
> sys-utils/lscpu-topology.c | 71 ++++++++++++++++++++++++++++++++++----
> 1 file changed, 64 insertions(+), 7 deletions(-)
Applied, thanks.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
* Re: [PATCH 2/4] hardlink: add --list-duplicates and --zero
From: Karel Zak @ 2024-11-05 10:20 UTC (permalink / raw)
To: наб; +Cc: util-linux
In-Reply-To: <geq2cawx3udvbe3b3tjton2cuim7ccemndul2sfknvxdll22sg@tarta.nabijaczleweli.xyz>
On Thu, Oct 31, 2024 at 03:56:40PM GMT, наб wrote:
> On Thu, Oct 31, 2024 at 09:58:19AM +0100, Karel Zak wrote:
> > On Mon, Oct 28, 2024 at 07:19:30PM GMT, наб wrote:
> > > --list-duplicates codifies what everyone keeps re-implementing with
> > > find -exec b2sum or src:perforate's finddup or whatever.
> > >
> > > hardlink already knows this, so make the data available thusly,
> > > in a format well-suited for pipeline processing
> > > (fixed-width key for uniq/cut/&c.,
> > > tab delimiter for cut &a.,
> > > -z for correct filename handling).
> >
> > Why do we need a 16-byte discriminator? The list consists of absolute
> > paths, so it should be unique enough. This seems like an unusual
> > thing,
> Well, the point is to have a list of lists of files, right.
> hardlink(1) finds, within the given domain,
> a set of sets of "these files are identical"
> (or, the logical set of "these are the link names of this file"
> for all eligible files).
> The only way to flatten this is to a single-layer list is by having a
> list of filenames discriminated by the set in which they belong, so
> [[a, b], [c, d, e]]
> discriminated as
> 0 a
> 0 b
> 1 c
> 1 d
> 1 e
> which allows you to reconstuct the sets live while stream-processing
> (the implementation uses a unique ASLR-randomised discriminator
> because the order isn't stable anyway I think? but same difference).
>
> A list of just filenames is useless.
I see, thanks.
> On Thu, Oct 31, 2024 at 09:51:00AM +0100, Karel Zak wrote:
> > The new option should also be added to the "bash-completion/hardlink"
> > file. However, I can fix this after merging locally.
> I missed this. I'll include it in v2 if we get to v2 but if we don't,
> please do, thanks.
Merged and bash-completion updated.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
* Re: [PATCH v2] lscpu: fix incorrect number of sockets during hotplug
From: Karel Zak @ 2024-11-04 12:08 UTC (permalink / raw)
To: Anjali K; +Cc: util-linux, anushree.mathur
In-Reply-To: <20241104063226.172077-1-anjalik@linux.ibm.com>
Thank you, it looks good. I made some cosmetic changes to the patch
and created a pull request to run it through CI tests on GitHub.
https://github.com/util-linux/util-linux/pull/3265
Thanks!
On Mon, Nov 04, 2024 at 12:02:26PM GMT, Anjali K wrote:
> sys-utils/lscpu-topology.c | 71 ++++++++++++++++++++++++++++++++++----
> 1 file changed, 64 insertions(+), 7 deletions(-)
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox