From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: virtio-dev-return-6860-cohuck=redhat.com@lists.oasis-open.org Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [10.110.1.242]) by lists.oasis-open.org (Postfix) with ESMTP id 71CCC985CDE for ; Mon, 2 Mar 2020 13:52:52 +0000 (UTC) From: David Hildenbrand Date: Mon, 2 Mar 2020 14:49:37 +0100 Message-Id: <20200302134941.315212-8-david@redhat.com> In-Reply-To: <20200302134941.315212-1-david@redhat.com> References: <20200302134941.315212-1-david@redhat.com> MIME-Version: 1.0 Subject: [virtio-dev] [PATCH v1 07/11] virtio-mem: Allow to offline partially unplugged memory blocks Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtio-dev@lists.oasis-open.org, virtualization@lists.linux-foundation.org, kvm@vger.kernel.org, Michal Hocko , Andrew Morton , "Michael S . Tsirkin" , David Hildenbrand , Jason Wang , Oscar Salvador , Igor Mammedov , Dave Young , Dan Williams , Pavel Tatashin , Stefan Hajnoczi , Vlastimil Babka List-ID: Dropping the reference count of PageOffline() pages allows offlining code to skip them. However, we also have to convert PG_reserved to another flag - let's use PG_dirty - so has_unmovable_pages() will properly handle them. PG_reserved pages get detected as unmovable right away. We need the flag to see if we are onlining pages the first time, or if we allocated them via alloc_contig_range(). Properly take care of offlining code also modifying the stats and special handling in case the driver gets unloaded. Cc: "Michael S. Tsirkin" Cc: Jason Wang Cc: Oscar Salvador Cc: Michal Hocko Cc: Igor Mammedov Cc: Dave Young Cc: Andrew Morton Cc: Dan Williams Cc: Pavel Tatashin Cc: Stefan Hajnoczi Cc: Vlastimil Babka Signed-off-by: David Hildenbrand --- drivers/virtio/virtio_mem.c | 64 ++++++++++++++++++++++++++++++++++++- 1 file changed, 63 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_mem.c b/drivers/virtio/virtio_mem.c index 5b26d57be551..2916f8b970fa 100644 --- a/drivers/virtio/virtio_mem.c +++ b/drivers/virtio/virtio_mem.c @@ -570,6 +570,53 @@ static void virtio_mem_notify_online(struct virtio_mem= *vm, unsigned long mb_id, =09=09virtio_mem_retry(vm); } =20 +static void virtio_mem_notify_going_offline(struct virtio_mem *vm, +=09=09=09=09=09 unsigned long mb_id) +{ +=09const unsigned long nr_pages =3D PFN_DOWN(vm->subblock_size); +=09unsigned long pfn; +=09int sb_id, i; + +=09for (sb_id =3D 0; sb_id < vm->nb_sb_per_mb; sb_id++) { +=09=09if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) +=09=09=09continue; +=09=09/* +=09=09 * Drop our reference to the pages so the memory can get +=09=09 * offlined and add the unplugged pages to the managed +=09=09 * page counters (so offlining code can correctly subtract +=09=09 * them again). +=09=09 */ +=09=09pfn =3D PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) + +=09=09=09 sb_id * vm->subblock_size); +=09=09adjust_managed_page_count(pfn_to_page(pfn), nr_pages); +=09=09for (i =3D 0; i < nr_pages; i++) +=09=09=09page_ref_dec(pfn_to_page(pfn + i)); +=09} +} + +static void virtio_mem_notify_cancel_offline(struct virtio_mem *vm, +=09=09=09=09=09 unsigned long mb_id) +{ +=09const unsigned long nr_pages =3D PFN_DOWN(vm->subblock_size); +=09unsigned long pfn; +=09int sb_id, i; + +=09for (sb_id =3D 0; sb_id < vm->nb_sb_per_mb; sb_id++) { +=09=09if (virtio_mem_mb_test_sb_plugged(vm, mb_id, sb_id, 1)) +=09=09=09continue; +=09=09/* +=09=09 * Get the reference we dropped when going offline and +=09=09 * subtract the unplugged pages from the managed page +=09=09 * counters. +=09=09 */ +=09=09pfn =3D PFN_DOWN(virtio_mem_mb_id_to_phys(mb_id) + +=09=09=09 sb_id * vm->subblock_size); +=09=09adjust_managed_page_count(pfn_to_page(pfn), -nr_pages); +=09=09for (i =3D 0; i < nr_pages; i++) +=09=09=09page_ref_inc(pfn_to_page(pfn + i)); +=09} +} + /* * This callback will either be called synchronously from add_memory() or * asynchronously (e.g., triggered via user space). We have to be careful @@ -616,6 +663,7 @@ static int virtio_mem_memory_notifier_cb(struct notifie= r_block *nb, =09=09=09break; =09=09} =09=09vm->hotplug_active =3D true; +=09=09virtio_mem_notify_going_offline(vm, mb_id); =09=09break; =09case MEM_GOING_ONLINE: =09=09mutex_lock(&vm->hotplug_mutex); @@ -640,6 +688,12 @@ static int virtio_mem_memory_notifier_cb(struct notifi= er_block *nb, =09=09mutex_unlock(&vm->hotplug_mutex); =09=09break; =09case MEM_CANCEL_OFFLINE: +=09=09if (!vm->hotplug_active) +=09=09=09break; +=09=09virtio_mem_notify_cancel_offline(vm, mb_id); +=09=09vm->hotplug_active =3D false; +=09=09mutex_unlock(&vm->hotplug_mutex); +=09=09break; =09case MEM_CANCEL_ONLINE: =09=09if (!vm->hotplug_active) =09=09=09break; @@ -666,8 +720,11 @@ static void virtio_mem_set_fake_offline(unsigned long = pfn, =09=09struct page *page =3D pfn_to_page(pfn); =20 =09=09__SetPageOffline(page); -=09=09if (!onlined) +=09=09if (!onlined) { =09=09=09SetPageDirty(page); +=09=09=09/* FIXME: remove after cleanups */ +=09=09=09ClearPageReserved(page); +=09=09} =09} } =20 @@ -1717,6 +1774,11 @@ static void virtio_mem_remove(struct virtio_device *= vdev) =09=09rc =3D virtio_mem_mb_remove(vm, mb_id); =09=09BUG_ON(rc); =09} +=09/* +=09 * After we unregistered our callbacks, user space can no longer +=09 * offline partially plugged online memory blocks. No need to worry +=09 * about them. +=09 */ =20 =09/* unregister callbacks */ =09unregister_virtio_mem_device(vm); --=20 2.24.1 --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org