[master][PATCH 1/2] dnf: only write the log lock to root for native dnf

[master][PATCH 1/2] dnf: only write the log lock to root for native dnf

[changqing.li]

From: Changqing Li <changqing.li@windriver.com>

From commit 742a1b7124, log_lock.pid is writen to root, but following
file is not changed, which will make it never deleted, and an unexpected
file exist in root dir after boot target.

$ tail  -n 1 etc/tmpfiles.d/dnf.conf
r /var/log/log_lock.pid

Besides, root dir may be read-only, so it is better still
keep the log_lock.pid under /var/log, only write the log lock to root
for native dnf for fixing issue mentioned in 742a1b7124

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 meta/recipes-devtools/dnf/dnf_4.16.1.bb | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/recipes-devtools/dnf/dnf_4.16.1.bb b/meta/recipes-devtools/dnf/dnf_4.16.1.bb
index ff79701dc7..9134411fa9 100644
--- a/meta/recipes-devtools/dnf/dnf_4.16.1.bb
+++ b/meta/recipes-devtools/dnf/dnf_4.16.1.bb
@@ -15,9 +15,10 @@ SRC_URI = "git://github.com/rpm-software-management/dnf.git;branch=master;protoc
            file://0029-Do-not-set-PYTHON_INSTALL_DIR-by-running-python.patch \
            file://0030-Run-python-scripts-using-env.patch \
            file://0001-set-python-path-for-completion_helper.patch \
-           file://0001-dnf-write-the-log-lock-to-root.patch \
            "
 
+SRC_URI:append:class-native = "file://0001-dnf-write-the-log-lock-to-root.patch"
+
 SRCREV = "94b7cc7956580405b219329541d6b40db6499cf1"
 UPSTREAM_CHECK_GITTAGREGEX = "(?P<pver>\d+(\.\d+)+)"
 
-- 
2.25.1

[master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[changqing.li]

From: Changqing Li <changqing.li@windriver.com>

Remove log_lock.pid which maybe created during do_rootfs. In commit
[dnf: only write the log lock to root for native dnf],
native dnf changed to write log lock to root, and target dnf still
use /var/log, so log_lock.pid need to be removed post do_rootfs.

Signed-off-by: Changqing Li <changqing.li@windriver.com>
---
 meta/classes-recipe/rootfs-postcommands.bbclass | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass b/meta/classes-recipe/rootfs-postcommands.bbclass
index 4492c9c0aa..53b241413e 100644
--- a/meta/classes-recipe/rootfs-postcommands.bbclass
+++ b/meta/classes-recipe/rootfs-postcommands.bbclass
@@ -49,6 +49,8 @@ ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;'
 
 ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "overlayfs", "overlayfs_qa_check; overlayfs_postprocess;", "", d)}'
 
+ROOTFS_POSTPROCESS_COMMAND += 'remove_unused_dnf_log_lock;'
+
 inherit image-artifact-names
 
 # Sort the user and group entries in /etc by ID in order to make the content
@@ -361,6 +363,11 @@ empty_var_volatile () {
 	fi
 }
 
+remove_unused_dnf_log_lock() {
+	if [ -e ${IMAGE_ROOTFS}/log_lock.pid ]; then
+		rm -rf ${IMAGE_ROOTFS}/log_lock.pid
+	fi
+}
 # Turn any symbolic /sbin/init link into a file
 remove_init_link () {
 	if [ -h ${IMAGE_ROOTFS}/sbin/init ]; then
-- 
2.25.1

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Alexander Kanavin]

On Fri, 30 Jun 2023 at 11:14, Changqing Li
<changqing.li@eng.windriver.com> wrote:
> Remove log_lock.pid which maybe created during do_rootfs. In commit
> [dnf: only write the log lock to root for native dnf],
> native dnf changed to write log lock to root, and target dnf still
> use /var/log, so log_lock.pid need to be removed post do_rootfs.

This is not making clear why the file needs to be removed. What
problems occur if it is left in place? Is it supposed to be added,
then removed by dnf during do_rootfs, and if this doesn't happen, is
that a problem with dnf that needs to be fixed, rather than removing
the file manually after the fact?

Alex

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Ross Burton]

On 30 Jun 2023, at 16:07, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> 
> On Fri, 30 Jun 2023 at 11:14, Changqing Li
> <changqing.li@eng.windriver.com> wrote:
>> Remove log_lock.pid which maybe created during do_rootfs. In commit
>> [dnf: only write the log lock to root for native dnf],
>> native dnf changed to write log lock to root, and target dnf still
>> use /var/log, so log_lock.pid need to be removed post do_rootfs.
> 
> This is not making clear why the file needs to be removed. What
> problems occur if it is left in place? Is it supposed to be added,
> then removed by dnf during do_rootfs, and if this doesn't happen, is
> that a problem with dnf that needs to be fixed, rather than removing
> the file manually after the fact?

Absolutely.  If the dnf image creation is leaving lock files then we fix the dnf image creation.  Does dnf leave a daemon hanging around? Does it leave lock files when it shouldn’t?  Either way, this should be in dnf or the image creation code itself, not a generic rootfs postcommand.

Ross

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Changqing Li]

On 7/4/23 19:11, Ross Burton wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On 30 Jun 2023, at 16:07, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
>> On Fri, 30 Jun 2023 at 11:14, Changqing Li
>> <changqing.li@eng.windriver.com> wrote:
>>> Remove log_lock.pid which maybe created during do_rootfs. In commit
>>> [dnf: only write the log lock to root for native dnf],
>>> native dnf changed to write log lock to root, and target dnf still
>>> use /var/log, so log_lock.pid need to be removed post do_rootfs.
>> This is not making clear why the file needs to be removed. What
>> problems occur if it is left in place? Is it supposed to be added,
>> then removed by dnf during do_rootfs, and if this doesn't happen, is
>> that a problem with dnf that needs to be fixed, rather than removing
>> the file manually after the fact?
> Absolutely.  If the dnf image creation is leaving lock files then we fix the dnf image creation.  Does dnf leave a daemon hanging around? Does it leave lock files when it shouldn’t?  Either way, this should be in dnf or the image creation code itself, not a generic rootfs postcommand.

Alex and Ross,  There is no dnf daemon hanging around,  you are right,  
seems like an dnf bug, I will report this to dnf upstream.

And there is no functional problem if this file is not removed, only it 
may confuse user there is an useless file that is generated during 
do_rootfs, it should not exit in rootfs.


//Changqing

>
> Ross
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#183827): https://lists.openembedded.org/g/openembedded-core/message/183827
> Mute This Topic: https://lists.openembedded.org/mt/99869451/3616873
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [changqing.li@eng.windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Alexander Kanavin]

This merged to master and it should not have happened. I'm sending a revert.

Alex


On Tue, 11 Jul 2023 at 10:34, Changqing Li
<changqing.li@eng.windriver.com> wrote:
>
>
> On 7/4/23 19:11, Ross Burton wrote:
> > CAUTION: This email comes from a non Wind River email account!
> > Do not click links or open attachments unless you recognize the sender and know the content is safe.
> >
> > On 30 Jun 2023, at 16:07, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> >> On Fri, 30 Jun 2023 at 11:14, Changqing Li
> >> <changqing.li@eng.windriver.com> wrote:
> >>> Remove log_lock.pid which maybe created during do_rootfs. In commit
> >>> [dnf: only write the log lock to root for native dnf],
> >>> native dnf changed to write log lock to root, and target dnf still
> >>> use /var/log, so log_lock.pid need to be removed post do_rootfs.
> >> This is not making clear why the file needs to be removed. What
> >> problems occur if it is left in place? Is it supposed to be added,
> >> then removed by dnf during do_rootfs, and if this doesn't happen, is
> >> that a problem with dnf that needs to be fixed, rather than removing
> >> the file manually after the fact?
> > Absolutely.  If the dnf image creation is leaving lock files then we fix the dnf image creation.  Does dnf leave a daemon hanging around? Does it leave lock files when it shouldn’t?  Either way, this should be in dnf or the image creation code itself, not a generic rootfs postcommand.
>
> Alex and Ross,  There is no dnf daemon hanging around,  you are right,
> seems like an dnf bug, I will report this to dnf upstream.
>
> And there is no functional problem if this file is not removed, only it
> may confuse user there is an useless file that is generated during
> do_rootfs, it should not exit in rootfs.
>
>
> //Changqing
>
> >
> > Ross
> >
> >
> >
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#184123): https://lists.openembedded.org/g/openembedded-core/message/184123
> Mute This Topic: https://lists.openembedded.org/mt/99869451/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Changqing Li]

On 7/18/23 04:22, Alexander Kanavin wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> This merged to master and it should not have happened. I'm sending a revert.
>
> Alex

Hi, Alex

I know that following patch already reverted.

https://git.openembedded.org/openembedded-core/commit/?id=49bad18012a4079f0dbfe6c541a46ec508940f28

But after review it again,  I think maybe it is still needed.

dnf works like this,  it will create log file /var/log/log_lock.pid,  
and this lock file is designed to by removed after reboot, refer 
following link.

https://github.com/rpm-software-management/dnf/blob/master/etc/tmpfiles.d/dnf.conf

But following patch for fixing oe specific issue break above dnf design 
about removing lock file.

https://git.openembedded.org/openembedded-core/commit/?id=742a1b71249f4da1c8d8e13e270b0eb6128a3f66

So  I add this patch 
https://git.openembedded.org/openembedded-core/commit/?id=406a72a9a47c2735b7e18cefc682b1df00d5a9aa 
to remove the lock file in rootfs.

Since for dnf-native,  it will not have reboot process,  so it is better 
to remove it at post process of rootfs, to  make an clean rootfs.


Regards

//Changqing


>
>
> On Tue, 11 Jul 2023 at 10:34, Changqing Li
> <changqing.li@eng.windriver.com> wrote:
>>
>> On 7/4/23 19:11, Ross Burton wrote:
>>> CAUTION: This email comes from a non Wind River email account!
>>> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>>>
>>> On 30 Jun 2023, at 16:07, Alexander Kanavin via lists.openembedded.org <alex.kanavin=gmail.com@lists.openembedded.org> wrote:
>>>> On Fri, 30 Jun 2023 at 11:14, Changqing Li
>>>> <changqing.li@eng.windriver.com> wrote:
>>>>> Remove log_lock.pid which maybe created during do_rootfs. In commit
>>>>> [dnf: only write the log lock to root for native dnf],
>>>>> native dnf changed to write log lock to root, and target dnf still
>>>>> use /var/log, so log_lock.pid need to be removed post do_rootfs.
>>>> This is not making clear why the file needs to be removed. What
>>>> problems occur if it is left in place? Is it supposed to be added,
>>>> then removed by dnf during do_rootfs, and if this doesn't happen, is
>>>> that a problem with dnf that needs to be fixed, rather than removing
>>>> the file manually after the fact?
>>> Absolutely.  If the dnf image creation is leaving lock files then we fix the dnf image creation.  Does dnf leave a daemon hanging around? Does it leave lock files when it shouldn’t?  Either way, this should be in dnf or the image creation code itself, not a generic rootfs postcommand.
>> Alex and Ross,  There is no dnf daemon hanging around,  you are right,
>> seems like an dnf bug, I will report this to dnf upstream.
>>
>> And there is no functional problem if this file is not removed, only it
>> may confuse user there is an useless file that is generated during
>> do_rootfs, it should not exit in rootfs.
>>
>>
>> //Changqing
>>
>>> Ross
>>>
>>>
>>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#184123): https://lists.openembedded.org/g/openembedded-core/message/184123
>> Mute This Topic: https://lists.openembedded.org/mt/99869451/1686489
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Alexander Kanavin]

On Tue, 5 Dec 2023 at 06:46, Changqing Li
<changqing.li@eng.windriver.com> wrote:

> So  I add this patch
> https://git.openembedded.org/openembedded-core/commit/?id=406a72a9a47c2735b7e18cefc682b1df00d5a9aa
> to remove the lock file in rootfs.
>
> Since for dnf-native,  it will not have reboot process,  so it is better
> to remove it at post process of rootfs, to  make an clean rootfs.

The review comments that led to the revert are still valid, and this
message does not address them. The concern was that the removal via
rootfs postprocess is too late. If dnf leaves lock files around, they
should be removed just after the dnf execution.

Alex

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Changqing Li]

[-- Attachment #1: Type: text/plain, Size: 1290 bytes --]


On 12/5/23 17:55, Alexander Kanavin wrote:
> CAUTION: This email comes from a non Wind River email account!
> Do not click links or open attachments unless you recognize the sender and know the content is safe.
>
> On Tue, 5 Dec 2023 at 06:46, Changqing Li
> <changqing.li@eng.windriver.com>  wrote:
>
>> So  I add this patch
>> https://git.openembedded.org/openembedded-core/commit/?id=406a72a9a47c2735b7e18cefc682b1df00d5a9aa
>> to remove the lock file in rootfs.
>>
>> Since for dnf-native,  it will not have reboot process,  so it is better
>> to remove it at post process of rootfs, to  make an clean rootfs.
> The review comments that led to the revert are still valid, and this
> message does not address them. The concern was that the removal via
> rootfs postprocess is too late. If dnf leaves lock files around, they
> should be removed just after the dnf execution.

Hi, Alex

Thanks.  Agree that the lock file should be removed after dnf execution. 
This is a bug of dnf. I had raise a bug upstream in Jul:

https://github.com/rpm-software-management/dnf/issues/1963.

This patch more like an workaround before dnf fix this issue, and this 
log_lock.pid in rootfs

has no function impact, just a dirty file in rootfs.

Regards

Sandy

> Alex

[-- Attachment #2: Type: text/html, Size: 2610 bytes --]

Re: [OE-core] [master][PATCH 2/2] rootfs-postcommands.bbclass: add post func remove_unused_dnf_log_lock

[Alexander Kanavin]

On Wed, 6 Dec 2023 at 03:39, Changqing Li
<changqing.li@eng.windriver.com> wrote:
> The review comments that led to the revert are still valid, and this
> message does not address them. The concern was that the removal via
> rootfs postprocess is too late. If dnf leaves lock files around, they
> should be removed just after the dnf execution.
> Thanks.  Agree that the lock file should be removed after dnf execution.   This is a bug of dnf. I had raise a bug upstream in Jul:
>
> https://github.com/rpm-software-management/dnf/issues/1963.

Looks like it's not something upstream is going to fix quickly - low
priority, no comments.

> This patch more like an workaround before dnf fix this issue, and this log_lock.pid in rootfs
>
> has no function impact, just a dirty file in rootfs.

If you can rework the patch so that the stale files are removed
immediately after running dnf, I think no one is going to object. The
location is somewhere in meta/lib/oe/package_manager/rpm/.

Alex