All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@suse.com>
To: Baoquan He <bhe@redhat.com>
Cc: Donald Dutile <ddutile@redhat.com>, Jiri Bohac <jbohac@suse.cz>,
	Pingfan Liu <piliu@redhat.com>, Tao Liu <ltao@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>, Dave Young <dyoung@redhat.com>,
	kexec@lists.infradead.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 0/4] kdump: crashkernel reservation from CMA
Date: Thu, 30 Nov 2023 14:41:12 +0100	[thread overview]
Message-ID: <ZWiQ-II9CvGv8EWK@tiehlicka> (raw)
In-Reply-To: <ZWiAsJlLookvCI+h@MiWiFi-R3L-srv>

On Thu 30-11-23 20:31:44, Baoquan He wrote:
[...]
> > > which doesn't use the proper pinning API (which would migrate away from
> > > the CMA) then what is the worst case? We will get crash kernel corrupted
> > > potentially and fail to take a proper kernel crash, right? Is this
> > > worrisome? Yes. Is it a real roadblock? I do not think so. The problem
> 
> We may fail to take a proper kernel crash, why isn't it a roadblock?

It would be if the threat was practical. So far I only see very
theoretical what-if concerns. And I do not mean to downplay those at
all. As already explained proper CMA users shouldn't ever leak out any
writes across kernel reboot.

> We
> have stable way with a little more memory, why would we take risk to
> take another way, just for saving memory? Usually only high end server
> needs the big memory for crashkernel and the big end server usually have
> huge system ram. The big memory will be a very small percentage relative
> to huge system RAM.

Jiri will likely talk more specific about that but our experience tells
that proper crashkernel memory scaling has turned out a real
maintainability problem because existing setups tend to break with major
kernel version upgrades or non trivial changes.
 
> > > seems theoretical to me and it is not CMA usage at fault here IMHO. It
> > > is the said theoretical driver that needs fixing anyway.
> 
> Now, what we want to make clear is if it's a theoretical possibility, or
> very likely happen. We have met several on-flight DMA stomping into
> kexec kernel's initrd in the past two years because device driver didn't
> provide shutdown() methor properly. For kdump, once it happen, the pain
> is we don't know how to debug. For kexec reboot, customer allows to
> login their system to reproduce and figure out the stomping. For kdump,
> the system corruption rarely happend, and the stomping could rarely
> happen too.

yes, this is understood.
 
> The code change looks simple and the benefit is very attractive. I
> surely like it if finally people confirm there's no risk. As I said, we
> can't afford to take the risk if it possibly happen. But I don't object
> if other people would rather take risk, we can let it land in kernel.

I think it is fair to be cautious and I wouldn't impose the new method
as a default. Only time can tell how safe this really is. It is hard to
protect agains theoretical issues though. Bugs should be fixed.
I believe this option would allow to configure kdump much easier and
less fragile.
 
> My personal opinion, thanks for sharing your thought.

Thanks for sharing.

-- 
Michal Hocko
SUSE Labs

_______________________________________________
kexec mailing list
kexec@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/kexec

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@suse.com>
To: Baoquan He <bhe@redhat.com>
Cc: Donald Dutile <ddutile@redhat.com>, Jiri Bohac <jbohac@suse.cz>,
	Pingfan Liu <piliu@redhat.com>, Tao Liu <ltao@redhat.com>,
	Vivek Goyal <vgoyal@redhat.com>, Dave Young <dyoung@redhat.com>,
	kexec@lists.infradead.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 0/4] kdump: crashkernel reservation from CMA
Date: Thu, 30 Nov 2023 14:41:12 +0100	[thread overview]
Message-ID: <ZWiQ-II9CvGv8EWK@tiehlicka> (raw)
In-Reply-To: <ZWiAsJlLookvCI+h@MiWiFi-R3L-srv>

On Thu 30-11-23 20:31:44, Baoquan He wrote:
[...]
> > > which doesn't use the proper pinning API (which would migrate away from
> > > the CMA) then what is the worst case? We will get crash kernel corrupted
> > > potentially and fail to take a proper kernel crash, right? Is this
> > > worrisome? Yes. Is it a real roadblock? I do not think so. The problem
> 
> We may fail to take a proper kernel crash, why isn't it a roadblock?

It would be if the threat was practical. So far I only see very
theoretical what-if concerns. And I do not mean to downplay those at
all. As already explained proper CMA users shouldn't ever leak out any
writes across kernel reboot.

> We
> have stable way with a little more memory, why would we take risk to
> take another way, just for saving memory? Usually only high end server
> needs the big memory for crashkernel and the big end server usually have
> huge system ram. The big memory will be a very small percentage relative
> to huge system RAM.

Jiri will likely talk more specific about that but our experience tells
that proper crashkernel memory scaling has turned out a real
maintainability problem because existing setups tend to break with major
kernel version upgrades or non trivial changes.
 
> > > seems theoretical to me and it is not CMA usage at fault here IMHO. It
> > > is the said theoretical driver that needs fixing anyway.
> 
> Now, what we want to make clear is if it's a theoretical possibility, or
> very likely happen. We have met several on-flight DMA stomping into
> kexec kernel's initrd in the past two years because device driver didn't
> provide shutdown() methor properly. For kdump, once it happen, the pain
> is we don't know how to debug. For kexec reboot, customer allows to
> login their system to reproduce and figure out the stomping. For kdump,
> the system corruption rarely happend, and the stomping could rarely
> happen too.

yes, this is understood.
 
> The code change looks simple and the benefit is very attractive. I
> surely like it if finally people confirm there's no risk. As I said, we
> can't afford to take the risk if it possibly happen. But I don't object
> if other people would rather take risk, we can let it land in kernel.

I think it is fair to be cautious and I wouldn't impose the new method
as a default. Only time can tell how safe this really is. It is hard to
protect agains theoretical issues though. Bugs should be fixed.
I believe this option would allow to configure kdump much easier and
less fragile.
 
> My personal opinion, thanks for sharing your thought.

Thanks for sharing.

-- 
Michal Hocko
SUSE Labs

  reply	other threads:[~2023-11-30 13:41 UTC|newest]

Thread overview: 96+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-11-24 19:54 [PATCH 0/4] kdump: crashkernel reservation from CMA Jiri Bohac
2023-11-24 19:54 ` Jiri Bohac
2023-11-24 19:57 ` [PATCH 1/4] kdump: add crashkernel cma suffix Jiri Bohac
2023-11-24 19:57   ` Jiri Bohac
2023-11-25  7:24   ` kernel test robot
2023-11-25  7:24     ` kernel test robot
2023-11-24 19:58 ` [PATCH 2/4] kdump: implement reserve_crashkernel_cma Jiri Bohac
2023-11-24 19:58   ` Jiri Bohac
2023-11-24 19:58 ` [PATCH 3/4] kdump, x86: implement crashkernel CMA reservation Jiri Bohac
2023-11-24 19:58   ` Jiri Bohac
2023-11-24 19:58 ` [PATCH 4/4] kdump, documentation: describe craskernel " Jiri Bohac
2023-11-24 19:58   ` Jiri Bohac
2023-11-25  1:51 ` [PATCH 0/4] kdump: crashkernel reservation from CMA Tao Liu
2023-11-25  1:51   ` Tao Liu
2023-11-25 21:22   ` Jiri Bohac
2023-11-25 21:22     ` Jiri Bohac
2023-11-28  1:12     ` Tao Liu
2023-11-28  1:12       ` Tao Liu
2023-11-28  2:11       ` Baoquan He
2023-11-28  2:11         ` Baoquan He
2023-11-28  9:08         ` Michal Hocko
2023-11-28  9:08           ` Michal Hocko
2023-11-29  7:57           ` Baoquan He
2023-11-29  7:57             ` Baoquan He
2023-11-29  9:25             ` Michal Hocko
2023-11-29  9:25               ` Michal Hocko
2023-11-30  2:42               ` Baoquan He
2023-11-30  2:42                 ` Baoquan He
2023-11-29 10:51             ` Jiri Bohac
2023-11-29 10:51               ` Jiri Bohac
2023-11-30  4:01               ` Baoquan He
2023-11-30  4:01                 ` Baoquan He
2023-12-01 12:35                 ` Jiri Bohac
2023-12-01 12:35                   ` Jiri Bohac
2023-11-29  8:10           ` Baoquan He
2023-11-29  8:10             ` Baoquan He
2023-11-29 15:03             ` Donald Dutile
2023-11-29 15:03               ` Donald Dutile
2023-11-30  3:00               ` Baoquan He
2023-11-30  3:00                 ` Baoquan He
2023-11-30 10:16                 ` Michal Hocko
2023-11-30 10:16                   ` Michal Hocko
2023-11-30 12:04                   ` Baoquan He
2023-11-30 12:04                     ` Baoquan He
2023-11-30 12:31                     ` Baoquan He
2023-11-30 12:31                       ` Baoquan He
2023-11-30 13:41                       ` Michal Hocko [this message]
2023-11-30 13:41                         ` Michal Hocko
2023-12-01 11:33                         ` Philipp Rudo
2023-12-01 11:33                           ` Philipp Rudo
2023-12-01 11:55                           ` Michal Hocko
2023-12-01 11:55                             ` Michal Hocko
2023-12-01 15:51                             ` Philipp Rudo
2023-12-01 15:51                               ` Philipp Rudo
2023-12-01 16:59                               ` Michal Hocko
2023-12-01 16:59                                 ` Michal Hocko
2023-12-06 11:08                                 ` Philipp Rudo
2023-12-06 11:08                                   ` Philipp Rudo
2023-12-06 11:23                                   ` David Hildenbrand
2023-12-06 11:23                                     ` David Hildenbrand
2023-12-06 13:49                                   ` Michal Hocko
2023-12-06 13:49                                     ` Michal Hocko
2023-12-06 15:19                                     ` Michal Hocko
2023-12-06 15:19                                       ` Michal Hocko
2023-12-07  4:23                                       ` Baoquan He
2023-12-07  4:23                                         ` Baoquan He
2023-12-07  8:55                                         ` Michal Hocko
2023-12-07  8:55                                           ` Michal Hocko
2023-12-07 11:13                                           ` Philipp Rudo
2023-12-07 11:13                                             ` Philipp Rudo
2023-12-07 11:52                                             ` Michal Hocko
2023-12-07 11:52                                               ` Michal Hocko
2023-12-08  1:55                                               ` Baoquan He
2023-12-08  1:55                                                 ` Baoquan He
2023-12-08 10:04                                                 ` Michal Hocko
2023-12-08 10:04                                                   ` Michal Hocko
2023-12-08  2:10                                           ` Baoquan He
2023-12-08  2:10                                             ` Baoquan He
2023-12-07 11:13                                       ` Philipp Rudo
2023-12-07 11:13                                         ` Philipp Rudo
2023-11-30 13:29                     ` Michal Hocko
2023-11-30 13:29                       ` Michal Hocko
2023-11-30 13:33                       ` Pingfan Liu
2023-11-30 13:33                         ` Pingfan Liu
2023-11-30 13:43                         ` Michal Hocko
2023-11-30 13:43                           ` Michal Hocko
2023-12-01  0:54                           ` Pingfan Liu
2023-12-01  0:54                             ` Pingfan Liu
2023-12-01 10:37                             ` Michal Hocko
2023-12-01 10:37                               ` Michal Hocko
2023-11-28  2:07     ` Pingfan Liu
2023-11-28  2:07       ` Pingfan Liu
2023-11-28  8:58       ` Michal Hocko
2023-11-28  8:58         ` Michal Hocko
2023-12-01 11:34 ` Philipp Rudo
2023-12-01 11:34   ` Philipp Rudo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=ZWiQ-II9CvGv8EWK@tiehlicka \
    --to=mhocko@suse.com \
    --cc=bhe@redhat.com \
    --cc=ddutile@redhat.com \
    --cc=dyoung@redhat.com \
    --cc=jbohac@suse.cz \
    --cc=kexec@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=ltao@redhat.com \
    --cc=piliu@redhat.com \
    --cc=vgoyal@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.