* Re: How to package e2scrub
2019-05-29 23:59 ` Theodore Ts'o
@ 2019-05-30 1:54 ` Andreas Dilger
2019-05-30 9:59 ` Jan Kara
2019-06-03 12:32 ` Lukas Czerner
2 siblings, 0 replies; 17+ messages in thread
From: Andreas Dilger @ 2019-05-30 1:54 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Lukas Czerner, linux-ext4, Jan Kara
Rather than naming the packages "e2scrub-*" it makes more sense to me to use "e2fsprogs-scrub" so that it is clear this is a sub-package of e2fsprogs? Or is the thought that the scrub functionality might move out of e2fsprogs and xfsprogs at some point in the future.
Cheers, Andreas
PS: I'd agree with Darrick that "xfsprogs-scrub" is probably a better name.
> On May 29, 2019, at 17:59, Theodore Ts'o <tytso@mit.edu> wrote:
>
>> On Wed, May 29, 2019 at 02:06:03PM +0200, Lukas Czerner wrote:
>> Hi guys,
>>
>> I am about to release 1.45.2 for Fedora rawhide, but I was thinking
>> about how to package the e2scrub cron job/systemd service.
>>
>> I really do not like the idea of installing cron job and/or the service as
>> a part of regular e2fsprogs package. This can potentially really surprise
>> people in a bad way.
>>
>> Note that I've already heard some complaints from debian users about the
>> systemd service being installed on their system after the e2fsprogs
>> update.
>
> One of the reasons I deliberately decided to enable it for Debian
> Unstable was it was the best way to flush out the bugs. :-)
>
> Yeah, Debian Unstable users are my guinea pigs. :-) Doesn't it work
> that way with Fedora and RHEL? :-)
>
> BTW, The complaints were mostly from e2scrub_all not working correctly
> if certain packages weren't installed, or if the LVM package was
> installed, but there were no LVM volumes present, etc. The other
> complaint I got was when there was no free space for the snapshot.
> I'm kind of hopeful that I've gotten them all at this point, but we'll
> see....
>
>> What I am going to do is to split the systemd service into a separate
>> package and I'd like to come to some agreement about the name of the
>> package so that we can have the same name across distributions (at least
>> Fedora/Debian/Suse).
>
> Hmm.... what keeping the service as part of the e2fsprogs package, but
> then not enabling out of the box. That is, require that user run:
>
> systemctl enable e2scrub_all.timer
>
> in order to actually get the feature? (They can also disable it using
> "systemctl disable e2scrub_all.timer".)
>
> As far as the cron job is concerned, we could just leave the crontab
> entry commented out by default, and require that the user go in and
> edit the /etc/cron.d/e2scrub_all file if they want to enable it. Not
> packaging it also seems fine; Debian's support for non-systemd
> configurations is at best marginal at this point, and while I'm not a
> fan of systemd, I'm also a realist...
>
> What do folks think?
>
> - Ted
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: How to package e2scrub
2019-05-29 23:59 ` Theodore Ts'o
2019-05-30 1:54 ` Andreas Dilger
@ 2019-05-30 9:59 ` Jan Kara
2019-05-30 13:51 ` Theodore Ts'o
2019-06-03 12:32 ` Lukas Czerner
2 siblings, 1 reply; 17+ messages in thread
From: Jan Kara @ 2019-05-30 9:59 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Lukas Czerner, linux-ext4, Jan Kara
On Wed 29-05-19 19:59:48, Theodore Ts'o wrote:
> On Wed, May 29, 2019 at 02:06:03PM +0200, Lukas Czerner wrote:
> > What I am going to do is to split the systemd service into a separate
> > package and I'd like to come to some agreement about the name of the
> > package so that we can have the same name across distributions (at least
> > Fedora/Debian/Suse).
>
> Hmm.... what keeping the service as part of the e2fsprogs package, but
> then not enabling out of the box. That is, require that user run:
>
> systemctl enable e2scrub_all.timer
>
> in order to actually get the feature? (They can also disable it using
> "systemctl disable e2scrub_all.timer".)
>
> As far as the cron job is concerned, we could just leave the crontab
> entry commented out by default, and require that the user go in and
> edit the /etc/cron.d/e2scrub_all file if they want to enable it. Not
> packaging it also seems fine; Debian's support for non-systemd
> configurations is at best marginal at this point, and while I'm not a
> fan of systemd, I'm also a realist...
>
> What do folks think?
Yeah, my plan is to just not package cron bits at all since openSUSE / SLES
support only systemd init anyway these days (and in fact our distro people
want to deprecate cron in favor of systemd). I guess I'll split off the
scrub bits into a separate sub-package (likely e2fsprogs will suggest
installation of this sub-package) and the service will be disabled by
default.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: How to package e2scrub
2019-05-30 9:59 ` Jan Kara
@ 2019-05-30 13:51 ` Theodore Ts'o
2019-05-31 10:07 ` Jan Kara
0 siblings, 1 reply; 17+ messages in thread
From: Theodore Ts'o @ 2019-05-30 13:51 UTC (permalink / raw)
To: Jan Kara; +Cc: Lukas Czerner, linux-ext4, Jan Kara
On Thu, May 30, 2019 at 11:59:07AM +0200, Jan Kara wrote:
> Yeah, my plan is to just not package cron bits at all since openSUSE / SLES
> support only systemd init anyway these days (and in fact our distro people
> want to deprecate cron in favor of systemd). I guess I'll split off the
> scrub bits into a separate sub-package (likely e2fsprogs will suggest
> installation of this sub-package) and the service will be disabled by
> default.
I'm not super-fond of extra sub-packages for their own sake, and the
extra e2scrub bits are small enough (about 32k?) that I don't believe
it justifies an extra sub-package; but that's a distribution-level
packaging decision, so it's certainly fine if we're not completely aligned.
My plan is to change the Debian package to turn off the timer unit
file by default, probably with a NEWS.Debian file[1] which tells users
how to enable it if they want at package installation time --- but to
keep the e2scrub bits in the base e2fsprogs package.
[1] https://www.debian.org/doc/manuals/developers-reference/ch06.en.html#bpp-news-debian
Out of curiosity, were any of the complaints that you've heard gone
beyond people who ran into the various e2scrub/e2scrub_all bugs? I'm
curious what their concerns were.
- Ted
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: How to package e2scrub
2019-05-30 13:51 ` Theodore Ts'o
@ 2019-05-31 10:07 ` Jan Kara
2019-05-31 14:10 ` Theodore Ts'o
2019-05-31 15:43 ` Darrick J. Wong
0 siblings, 2 replies; 17+ messages in thread
From: Jan Kara @ 2019-05-31 10:07 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Jan Kara, Lukas Czerner, linux-ext4, Jan Kara
On Thu 30-05-19 09:51:55, Theodore Ts'o wrote:
> On Thu, May 30, 2019 at 11:59:07AM +0200, Jan Kara wrote:
> > Yeah, my plan is to just not package cron bits at all since openSUSE / SLES
> > support only systemd init anyway these days (and in fact our distro people
> > want to deprecate cron in favor of systemd). I guess I'll split off the
> > scrub bits into a separate sub-package (likely e2fsprogs will suggest
> > installation of this sub-package) and the service will be disabled by
> > default.
>
> I'm not super-fond of extra sub-packages for their own sake, and the
> extra e2scrub bits are small enough (about 32k?) that I don't believe
> it justifies an extra sub-package; but that's a distribution-level
> packaging decision, so it's certainly fine if we're not completely aligned.
Yes, size is not a big concern but the scrub bits require util-linux, lvm,
and mailer to work correctly and I don't want to add these dependencies to
stock e2fsprogs package because some minimal installations do not want e.g.
lvm at all. Granted these are just scripts so I could get away with not
requiring e.g. lvm at all but it seems user-unfriendly to leave it up to
user to determine that his systemd-service fails due to missing packages.
> Out of curiosity, were any of the complaints that you've heard gone
> beyond people who ran into the various e2scrub/e2scrub_all bugs? I'm
> curious what their concerns were.
I didn't hear any complaints so far. But I have my doubts anyone actually
run that code so far - openSUSE Tumbleweed has limited userbase, we do
installs to btrfs by default, we don't propose LVM by default, and I didn't
enable the service files to run by default.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: How to package e2scrub
2019-05-31 10:07 ` Jan Kara
@ 2019-05-31 14:10 ` Theodore Ts'o
2019-05-31 21:43 ` Andreas Dilger
2019-06-03 9:30 ` Jan Kara
2019-05-31 15:43 ` Darrick J. Wong
1 sibling, 2 replies; 17+ messages in thread
From: Theodore Ts'o @ 2019-05-31 14:10 UTC (permalink / raw)
To: Jan Kara; +Cc: Lukas Czerner, linux-ext4, Jan Kara
On Fri, May 31, 2019 at 12:07:13PM +0200, Jan Kara wrote:
> On Thu 30-05-19 09:51:55, Theodore Ts'o wrote:
> > On Thu, May 30, 2019 at 11:59:07AM +0200, Jan Kara wrote:
> > > Yeah, my plan is to just not package cron bits at all since openSUSE / SLES
> > > support only systemd init anyway these days (and in fact our distro people
> > > want to deprecate cron in favor of systemd). I guess I'll split off the
> > > scrub bits into a separate sub-package (likely e2fsprogs will suggest
> > > installation of this sub-package) and the service will be disabled by
> > > default.
> >
> > I'm not super-fond of extra sub-packages for their own sake, and the
> > extra e2scrub bits are small enough (about 32k?) that I don't believe
> > it justifies an extra sub-package; but that's a distribution-level
> > packaging decision, so it's certainly fine if we're not completely aligned.
>
> Yes, size is not a big concern but the scrub bits require util-linux, lvm,
> and mailer to work correctly and I don't want to add these dependencies to
> stock e2fsprogs package because some minimal installations do not want e.g.
> lvm at all. Granted these are just scripts so I could get away with not
> requiring e.g. lvm at all but it seems user-unfriendly to leave it up to
> user to determine that his systemd-service fails due to missing packages.
So you're using an extra package to force the installation of the
necessary prerequisite packages, instead of the current approach where
we don't require them, but we just skip running the scrub if lvm and
util-linux are not present. I think both approaches makes sense.
It's also a good point that we need to handle the case of a missing
sendmail intelligently. It looks like we currently skip sending mail
at all in the cron case, and in the case systemd case, we'll spew a
warning (which won't get mailed since there's no sendmail, but it does
mean some extra lines in the logfile). All of this being said, it's
not _completely_ useless to scrub without an mailer; we still mark the
file system as needing to be checked on the next boot. But it's
another argument that we shouldn't enable the service by default.
For that reason, I'm not sure I'd want to force the installation of a
mailer, since someone might want to run e2scrub by hand, and
e2scrub_all every week w/o isn't a completely insane thing. But we
certainly should handle that case gracefully.
- Ted
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: How to package e2scrub
2019-05-31 14:10 ` Theodore Ts'o
@ 2019-05-31 21:43 ` Andreas Dilger
2019-06-03 9:30 ` Jan Kara
1 sibling, 0 replies; 17+ messages in thread
From: Andreas Dilger @ 2019-05-31 21:43 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Jan Kara, Lukas Czerner, linux-ext4, Jan Kara
[-- Attachment #1: Type: text/plain, Size: 2924 bytes --]
On May 31, 2019, at 8:10 AM, Theodore Ts'o <tytso@mit.edu> wrote:
>
> On Fri, May 31, 2019 at 12:07:13PM +0200, Jan Kara wrote:
>> On Thu 30-05-19 09:51:55, Theodore Ts'o wrote:
>>> On Thu, May 30, 2019 at 11:59:07AM +0200, Jan Kara wrote:
>>>> Yeah, my plan is to just not package cron bits at all since openSUSE / SLES
>>>> support only systemd init anyway these days (and in fact our distro people
>>>> want to deprecate cron in favor of systemd). I guess I'll split off the
>>>> scrub bits into a separate sub-package (likely e2fsprogs will suggest
>>>> installation of this sub-package) and the service will be disabled by
>>>> default.
>>>
>>> I'm not super-fond of extra sub-packages for their own sake, and the
>>> extra e2scrub bits are small enough (about 32k?) that I don't believe
>>> it justifies an extra sub-package; but that's a distribution-level
>>> packaging decision, so it's certainly fine if we're not completely aligned.
>>
>> Yes, size is not a big concern but the scrub bits require util-linux, lvm,
>> and mailer to work correctly and I don't want to add these dependencies to
>> stock e2fsprogs package because some minimal installations do not want e.g.
>> lvm at all. Granted these are just scripts so I could get away with not
>> requiring e.g. lvm at all but it seems user-unfriendly to leave it up to
>> user to determine that his systemd-service fails due to missing packages.
>
> So you're using an extra package to force the installation of the
> necessary prerequisite packages, instead of the current approach where
> we don't require them, but we just skip running the scrub if lvm and
> util-linux are not present. I think both approaches makes sense.
>
> It's also a good point that we need to handle the case of a missing
> sendmail intelligently. It looks like we currently skip sending mail
> at all in the cron case, and in the case systemd case, we'll spew a
> warning (which won't get mailed since there's no sendmail, but it does
> mean some extra lines in the logfile). All of this being said, it's
> not _completely_ useless to scrub without an mailer; we still mark the
> file system as needing to be checked on the next boot. But it's
> another argument that we shouldn't enable the service by default.
>
> For that reason, I'm not sure I'd want to force the installation of a
> mailer, since someone might want to run e2scrub by hand, and
> e2scrub_all every week w/o isn't a completely insane thing. But we
> certainly should handle that case gracefully.
If sendmail is unavailable (and maybe even if it *is* available), e2scrub
can use logger to write a short message to syslog if there is an error.
Something like:
e2scrub: $device errors detected, needs offline e2fsck to correct
e2scrub: $device logs in /var/log/e2scrub....
in mark_corrupt() or from e2scrub_fail.
Cheers, Andreas
[-- Attachment #2: Message signed with OpenPGP --]
[-- Type: application/pgp-signature, Size: 873 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: How to package e2scrub
2019-05-31 14:10 ` Theodore Ts'o
2019-05-31 21:43 ` Andreas Dilger
@ 2019-06-03 9:30 ` Jan Kara
1 sibling, 0 replies; 17+ messages in thread
From: Jan Kara @ 2019-06-03 9:30 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: Jan Kara, Lukas Czerner, linux-ext4, Jan Kara
On Fri 31-05-19 10:10:19, Theodore Ts'o wrote:
> On Fri, May 31, 2019 at 12:07:13PM +0200, Jan Kara wrote:
> > On Thu 30-05-19 09:51:55, Theodore Ts'o wrote:
> > > On Thu, May 30, 2019 at 11:59:07AM +0200, Jan Kara wrote:
> > > > Yeah, my plan is to just not package cron bits at all since openSUSE / SLES
> > > > support only systemd init anyway these days (and in fact our distro people
> > > > want to deprecate cron in favor of systemd). I guess I'll split off the
> > > > scrub bits into a separate sub-package (likely e2fsprogs will suggest
> > > > installation of this sub-package) and the service will be disabled by
> > > > default.
> > >
> > > I'm not super-fond of extra sub-packages for their own sake, and the
> > > extra e2scrub bits are small enough (about 32k?) that I don't believe
> > > it justifies an extra sub-package; but that's a distribution-level
> > > packaging decision, so it's certainly fine if we're not completely aligned.
> >
> > Yes, size is not a big concern but the scrub bits require util-linux, lvm,
> > and mailer to work correctly and I don't want to add these dependencies to
> > stock e2fsprogs package because some minimal installations do not want e.g.
> > lvm at all. Granted these are just scripts so I could get away with not
> > requiring e.g. lvm at all but it seems user-unfriendly to leave it up to
> > user to determine that his systemd-service fails due to missing packages.
>
> So you're using an extra package to force the installation of the
> necessary prerequisite packages, instead of the current approach where
> we don't require them, but we just skip running the scrub if lvm and
> util-linux are not present. I think both approaches makes sense.
>
> It's also a good point that we need to handle the case of a missing
> sendmail intelligently. It looks like we currently skip sending mail
> at all in the cron case, and in the case systemd case, we'll spew a
> warning (which won't get mailed since there's no sendmail, but it does
> mean some extra lines in the logfile). All of this being said, it's
> not _completely_ useless to scrub without an mailer; we still mark the
> file system as needing to be checked on the next boot. But it's
> another argument that we shouldn't enable the service by default.
>
> For that reason, I'm not sure I'd want to force the installation of a
> mailer, since someone might want to run e2scrub by hand, and
> e2scrub_all every week w/o isn't a completely insane thing. But we
> certainly should handle that case gracefully.
Yeah, if the scripts can handle missing mailer and do something useful in
that case, I think I will switch the RPM dependency on postfix to just
Recommends and not Requires.
Honza
--
Jan Kara <jack@suse.com>
SUSE Labs, CR
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: How to package e2scrub
2019-05-31 10:07 ` Jan Kara
2019-05-31 14:10 ` Theodore Ts'o
@ 2019-05-31 15:43 ` Darrick J. Wong
1 sibling, 0 replies; 17+ messages in thread
From: Darrick J. Wong @ 2019-05-31 15:43 UTC (permalink / raw)
To: Jan Kara; +Cc: Theodore Ts'o, Lukas Czerner, linux-ext4, Jan Kara
On Fri, May 31, 2019 at 12:07:13PM +0200, Jan Kara wrote:
> On Thu 30-05-19 09:51:55, Theodore Ts'o wrote:
> > On Thu, May 30, 2019 at 11:59:07AM +0200, Jan Kara wrote:
> > > Yeah, my plan is to just not package cron bits at all since openSUSE / SLES
> > > support only systemd init anyway these days (and in fact our distro people
> > > want to deprecate cron in favor of systemd). I guess I'll split off the
> > > scrub bits into a separate sub-package (likely e2fsprogs will suggest
> > > installation of this sub-package) and the service will be disabled by
> > > default.
> >
> > I'm not super-fond of extra sub-packages for their own sake, and the
> > extra e2scrub bits are small enough (about 32k?) that I don't believe
> > it justifies an extra sub-package; but that's a distribution-level
> > packaging decision, so it's certainly fine if we're not completely aligned.
>
> Yes, size is not a big concern but the scrub bits require util-linux, lvm,
> and mailer to work correctly and I don't want to add these dependencies to
> stock e2fsprogs package because some minimal installations do not want e.g.
> lvm at all. Granted these are just scripts so I could get away with not
> requiring e.g. lvm at all but it seems user-unfriendly to leave it up to
> user to determine that his systemd-service fails due to missing packages.
All good reasons for a separate package, particularly considering that
on the RH side they've split out xfs_scrub because of its python 3
dependencies.
> > Out of curiosity, were any of the complaints that you've heard gone
> > beyond people who ran into the various e2scrub/e2scrub_all bugs? I'm
> > curious what their concerns were.
>
> I didn't hear any complaints so far. But I have my doubts anyone actually
> run that code so far - openSUSE Tumbleweed has limited userbase, we do
> installs to btrfs by default, we don't propose LVM by default, and I didn't
> enable the service files to run by default.
(I suspect it's only Debian Unstable users who are running it right
now...)
--D
>
> Honza
> --
> Jan Kara <jack@suse.com>
> SUSE Labs, CR
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: How to package e2scrub
2019-05-29 23:59 ` Theodore Ts'o
2019-05-30 1:54 ` Andreas Dilger
2019-05-30 9:59 ` Jan Kara
@ 2019-06-03 12:32 ` Lukas Czerner
2 siblings, 0 replies; 17+ messages in thread
From: Lukas Czerner @ 2019-06-03 12:32 UTC (permalink / raw)
To: Theodore Ts'o; +Cc: linux-ext4, Jan Kara
On Wed, May 29, 2019 at 07:59:48PM -0400, Theodore Ts'o wrote:
> On Wed, May 29, 2019 at 02:06:03PM +0200, Lukas Czerner wrote:
> > Hi guys,
> >
> > I am about to release 1.45.2 for Fedora rawhide, but I was thinking
> > about how to package the e2scrub cron job/systemd service.
> >
> > I really do not like the idea of installing cron job and/or the service as
> > a part of regular e2fsprogs package. This can potentially really surprise
> > people in a bad way.
> >
> > Note that I've already heard some complaints from debian users about the
> > systemd service being installed on their system after the e2fsprogs
> > update.
>
> One of the reasons I deliberately decided to enable it for Debian
> Unstable was it was the best way to flush out the bugs. :-)
>
> Yeah, Debian Unstable users are my guinea pigs. :-) Doesn't it work
> that way with Fedora and RHEL? :-)
>
> BTW, The complaints were mostly from e2scrub_all not working correctly
> if certain packages weren't installed, or if the LVM package was
> installed, but there were no LVM volumes present, etc. The other
> complaint I got was when there was no free space for the snapshot.
> I'm kind of hopeful that I've gotten them all at this point, but we'll
> see....
Yeah, I've heard from two people and it was all about the service being
enabled by default when installing/updating e2fsprogs which for them was
very much unexpected. They were the types of people what want to have as
much controll as they can so they were annoyed by that and immediatelly
disabled the service :)
>
> > What I am going to do is to split the systemd service into a separate
> > package and I'd like to come to some agreement about the name of the
> > package so that we can have the same name across distributions (at least
> > Fedora/Debian/Suse).
>
> Hmm.... what keeping the service as part of the e2fsprogs package, but
> then not enabling out of the box. That is, require that user run:
>
> systemctl enable e2scrub_all.timer
>
> in order to actually get the feature? (They can also disable it using
> "systemctl disable e2scrub_all.timer".)
That's the suggestion for rpm packages in fedora - not enabling services by
default. I am still not decided about this since installing separate service
package is strong enough of a hint that user probably want to enable it.
>
> As far as the cron job is concerned, we could just leave the crontab
> entry commented out by default, and require that the user go in and
> edit the /etc/cron.d/e2scrub_all file if they want to enable it. Not
> packaging it also seems fine; Debian's support for non-systemd
> configurations is at best marginal at this point, and while I'm not a
> fan of systemd, I'm also a realist...
Yeah, commenting out the crontab entry sounds like a good way to go
about it.
>
> What do folks think?
>
> - Ted
^ permalink raw reply [flat|nested] 17+ messages in thread