From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail138.messagelabs.com (mail138.messagelabs.com [216.82.249.35]) by kanga.kvack.org (Postfix) with SMTP id 1C5F16B004D for ; Tue, 18 Aug 2009 10:03:39 -0400 (EDT) Received: by yxe14 with SMTP id 14so4762262yxe.12 for ; Tue, 18 Aug 2009 07:03:38 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20090818111125.GA20217@localhost> References: <4A87829C.4090908@redhat.com> <20090816112910.GA3208@localhost> <28c262360908170733q4bc5ddb8ob2fc976b6a468d6e@mail.gmail.com> <20090818023438.GB7958@localhost> <20090818131734.3d5bceb2.minchan.kim@barrios-desktop> <20090818093119.GA12679@localhost> <20090818185247.a4516389.minchan.kim@barrios-desktop> <20090818100031.GC16298@localhost> <28c262360908180400q361ea322o8959fd5ea5ae3217@mail.gmail.com> <20090818111125.GA20217@localhost> Date: Tue, 18 Aug 2009 23:03:38 +0900 Message-ID: <28c262360908180703p61e9048el9981a595b72f470d@mail.gmail.com> Subject: Re: [RFC] respect the referenced bit of KVM guest pages? From: Minchan Kim Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org To: Wu Fengguang Cc: Lee Schermerhorn , Rik van Riel , Jeff Dike , Avi Kivity , Andrea Arcangeli , "Yu, Wilfred" , "Kleen, Andi" , Hugh Dickins , Andrew Morton , Christoph Lameter , KOSAKI Motohiro , Mel Gorman , LKML , linux-mm List-ID: On Tue, Aug 18, 2009 at 8:11 PM, Wu Fengguang wrote= : > On Tue, Aug 18, 2009 at 07:00:48PM +0800, Minchan Kim wrote: >> On Tue, Aug 18, 2009 at 7:00 PM, Wu Fengguang wr= ote: >> > On Tue, Aug 18, 2009 at 05:52:47PM +0800, Minchan Kim wrote: >> >> On Tue, 18 Aug 2009 17:31:19 +0800 >> >> Wu Fengguang wrote: >> >> >> >> > On Tue, Aug 18, 2009 at 12:17:34PM +0800, Minchan Kim wrote: >> >> > > On Tue, 18 Aug 2009 10:34:38 +0800 >> >> > > Wu Fengguang wrote: >> >> > > >> >> > > > Minchan, >> >> > > > >> >> > > > On Mon, Aug 17, 2009 at 10:33:54PM +0800, Minchan Kim wrote: >> >> > > > > On Sun, Aug 16, 2009 at 8:29 PM, Wu Fengguang wrote: >> >> > > > > > On Sun, Aug 16, 2009 at 01:15:02PM +0800, Wu Fengguang wrot= e: >> >> > > > > >> On Sun, Aug 16, 2009 at 11:53:00AM +0800, Rik van Riel wro= te: >> >> > > > > >> > Wu Fengguang wrote: >> >> > > > > >> > > On Fri, Aug 07, 2009 at 05:09:55AM +0800, Jeff Dike wr= ote: >> >> > > > > >> > >> Side question - >> >> > > > > >> > >> =C2=A0Is there a good reason for this to be in shrink= _active_list() >> >> > > > > >> > >> as opposed to __isolate_lru_page? >> >> > > > > >> > >> >> >> > > > > >> > >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (unlikely(!page_= evictable(page, NULL))) { >> >> > > > > >> > >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0putback_lru_page(page); >> >> > > > > >> > >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0continue; >> >> > > > > >> > >> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0} >> >> > > > > >> > >> >> >> > > > > >> > >> Maybe we want to minimize the amount of code under th= e lru lock or >> >> > > > > >> > >> avoid duplicate logic in the isolate_page functions. >> >> > > > > >> > > >> >> > > > > >> > > I guess the quick test means to avoid the expensive pa= ge_referenced() >> >> > > > > >> > > call that follows it. But that should be mostly one sh= ot cost - the >> >> > > > > >> > > unevictable pages are unlikely to cycle in active/inac= tive list again >> >> > > > > >> > > and again. >> >> > > > > >> > >> >> > > > > >> > Please read what putback_lru_page does. >> >> > > > > >> > >> >> > > > > >> > It moves the page onto the unevictable list, so that >> >> > > > > >> > it will not end up in this scan again. >> >> > > > > >> >> >> > > > > >> Yes it does. I said 'mostly' because there is a small hole= that an >> >> > > > > >> unevictable page may be scanned but still not moved to une= victable >> >> > > > > >> list: when a page is mapped in two places, the first pte h= as the >> >> > > > > >> referenced bit set, the _second_ VMA has VM_LOCKED bit set= , then >> >> > > > > >> page_referenced() will return 1 and shrink_page_list() wil= l move it >> >> > > > > >> into active list instead of unevictable list. Shall we fix= this rare >> >> > > > > >> case? >> >> > > > > >> >> > > > > I think it's not a big deal. >> >> > > > >> >> > > > Maybe, otherwise I should bring up this issue long time before = :) >> >> > > > >> >> > > > > As you mentioned, it's rare case so there would be few pages = in active >> >> > > > > list instead of unevictable list. >> >> > > > >> >> > > > Yes. >> >> > > > >> >> > > > > When next time to scan comes, we can try to move the pages in= to >> >> > > > > unevictable list, again. >> >> > > > >> >> > > > Will PG_mlocked be set by then? Otherwise the situation is not = likely >> >> > > > to change and the VM_LOCKED pages may circulate in active/inact= ive >> >> > > > list for countless times. >> >> > > >> >> > > PG_mlocked is not important in that case. >> >> > > Important thing is VM_LOCKED vma. >> >> > > I think below annotaion can help you to understand my point. :) >> >> > >> >> > Hmm, it looks like pages under VM_LOCKED vma is guaranteed to have >> >> > PG_mlocked set, and so will be caught by page_evictable(). Is it? >> >> >> >> No. I am sorry for making my point not clear. >> >> I meant following as. >> >> When the next time to scan, >> >> >> >> shrink_page_list >> > =C2=A0-> >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0referenced =3D = page_referenced(page, 1, >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0sc->mem_cgroup, &vm_flags); >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0/* In active us= e or really unfreeable? =C2=A0Activate it. */ >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (sc->order <= =3D PAGE_ALLOC_COSTLY_ORDER && >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0refere= nced && page_mapping_inuse(page)) >> > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0goto activate_locked; >> > >> >> -> try_to_unmap >> > =C2=A0 =C2=A0 ~~~~~~~~~~~~ this line won't be reached if page is found= to be >> > =C2=A0 =C2=A0 referenced in the above lines? >> >> Indeed! In fact, I was worry about that. >> It looks after live lock problem. >> But I think =C2=A0it's very small race window so =C2=A0there isn't any r= eport until now. >> Let's Cced Lee. >> >> If we have to fix it, how about this ? >> This version =C2=A0has small overhead than yours since >> there is less shrink_page_list call than page_referenced. > > Yeah, it looks better. However I still wonder if (VM_LOCKED && !PG_mlocke= d) > is possible and somehow persistent. Does anyone have the answer? Thanks! I think it's possible. munlock_vma_page pre-clears PG_mlocked of page. And then if isolate_lru_page fail, the page have no PG_mlocked and vma whic= h have VM_LOCKED. As munlock_vma_page's annotation said, we hope the page will be rescued by try_to_unmap. But As you pointed out, if the page have PG_referenced, it ca= n't reach try_to_unmap so that it will go into the active list. What are others' opinion ? > Thanks, > Fengguang --=20 Kind regards, Minchan Kim -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org