From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751752AbdGZRUy (ORCPT ); Wed, 26 Jul 2017 13:20:54 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:51877 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751684AbdGZRUw (ORCPT ); Wed, 26 Jul 2017 13:20:52 -0400 Date: Wed, 26 Jul 2017 19:20:39 +0200 From: Gerald Schaefer To: Michal Hocko Cc: Heiko Carstens , linux-mm@kvack.org, Andrew Morton , Mel Gorman , Vlastimil Babka , Andrea Arcangeli , Jerome Glisse , Reza Arbab , Yasuaki Ishimatsu , qiuxishi@huawei.com, Kani Toshimitsu , slaoub@gmail.com, Joonsoo Kim , Andi Kleen , Daniel Kiper , Igor Mammedov , Vitaly Kuznetsov , LKML , Benjamin Herrenschmidt , Dan Williams , "H. Peter Anvin" , Ingo Molnar , Michael Ellerman , Paul Mackerras , Thomas Gleixner , gerald.schaefer@de.ibm.com, Martin Schwidefsky Subject: Re: [RFC PATCH 3/5] mm, memory_hotplug: allocate memmap from the added memory range for sparse-vmemmap In-Reply-To: <20170726123040.GO2981@dhcp22.suse.cz> References: <20170726083333.17754-1-mhocko@kernel.org> <20170726083333.17754-4-mhocko@kernel.org> <20170726114539.GG3218@osiris> <20170726123040.GO2981@dhcp22.suse.cz> Organization: IBM Deutschland Research & Development GmbH / Vorsitzende des Aufsichtsrats: Martina Koederitz / Geschaeftsfuehrung: Dirk Wittkopp / Sitz der Gesellschaft: Boeblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294 X-Mailer: Claws Mail 3.9.0 (GTK+ 2.24.23; x86_64-redhat-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 x-cbid: 17072617-0016-0000-0000-000004DA9BAC X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17072617-0017-0000-0000-000028111819 Message-Id: <20170726192039.48b81161@thinkpad> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-07-26_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1706020000 definitions=main-1707260252 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 26 Jul 2017 14:30:41 +0200 Michal Hocko wrote: > On Wed 26-07-17 13:45:39, Heiko Carstens wrote: > [...] > > In general I do like your idea, however if I understand your patches > > correctly we might have an ordering problem on s390: it is not possible to > > access hot-added memory on s390 before it is online (MEM_GOING_ONLINE > > succeeded). > > Could you point me to the code please? I cannot seem to find the > notifier which implements that. It is in drivers/s390/char/sclp_cmd.c: sclp_mem_notifier(). > > > On MEM_GOING_ONLINE we ask the hypervisor to back the potential available > > hot-added memory region with physical pages. Accessing those ranges before > > that will result in an exception. > > Can we make the range which backs the memmap range available? E.g from > s390 specific __vmemmap_populate path? No, only the complete range of a storage increment can be made available. The size of those increments may vary between z/VM and LPAR, but at least with LPAR it will always be minimum 256 MB, IIRC. > > > However with your approach the memory is still allocated when add_memory() > > is being called, correct? That wouldn't be a change to the current > > behaviour; except for the ordering problem outlined above. > > Could you be more specific please? I do not change when the memmap is > allocated. I guess this is about the difference between s390 and others, wrt when we call add_memory(). It is also in drivers/s390/char/sclp_cmd.c, early during memory detection, as opposed to other archs, where I guess this could be triggered by an ACPI event during runtime, at least for newly added and to-be-onlined memory. This probably means that any approach that tries to allocate memmap memory during add_memory(), out of the "to-be-onlined but still offline" memory, will be difficult for s390, because we call add_memory() only once during memory detection for the complete range of (hypervisor) defined online and offline memory. The offline parts are then made available in the MEM_GOING_ONLINE notifier called from online_pages(). Only after this point the memory would then be available to allocate a memmap in it. Nevertheless, we have great interest in such a "allocate memmap from the added memory range" solution. I guess we would need some way to separate the memmap allocation from add_memory(), which sounds odd, or provide some way to have add_memory() only allocate a memmap for online memory, and a mechanism to add the memmaps for offline memory blocks later when they are being set online. Regards, Gerald