* arm L_PTE_XXX entry addition for Debugging purpose
@ 2012-07-30 9:39 Dhyan
2012-07-31 7:13 ` bill4carson
0 siblings, 1 reply; 13+ messages in thread
From: Dhyan @ 2012-07-30 9:39 UTC (permalink / raw)
To: kernelnewbies
Dear All,
>From linux(2.6.35) arm page table architecture i can see we have one
hardware page table and there is corresponding Linux page table Entry
(L_PTE_*).The "Linux" PTE definitions are as like below from
arch/arm/include/asm/pgtable.h.
#define L_PTE_PRESENT (1 << 0)
#define L_PTE_FILE (1 << 1)
#define L_PTE_YOUNG (1 << 1)
#define L_PTE_BUFFERABLE(1 << 2)
#define L_PTE_CACHEABLE (1 << 3)
#define L_PTE_USER (1 << 4)
#define L_PTE_WRITE (1 << 5)
#define L_PTE_EXEC (1 << 6)
#define L_PTE_DIRTY (1 << 7)
#define L_PTE_COHERENT (1 << 9)
#define L_PTE_SHARED (1 << 10)
So is it possible to add one more #define L_PTE_DEBUG (1 << 11) for my
debugging purpose (basically to trap all the write to that page and set
this bit when write happens and clear it off in another thread )? Or is
there any limitation like we can use only L_PTE till 10th bit ?
So could you please help
--
Thanks & Regards
Dhayn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120730/ff28b36d/attachment.html
^ permalink raw reply [flat|nested] 13+ messages in thread* arm L_PTE_XXX entry addition for Debugging purpose 2012-07-30 9:39 arm L_PTE_XXX entry addition for Debugging purpose Dhyan @ 2012-07-31 7:13 ` bill4carson 2012-07-31 9:20 ` Dhyan 0 siblings, 1 reply; 13+ messages in thread From: bill4carson @ 2012-07-31 7:13 UTC (permalink / raw) To: kernelnewbies On 2012?07?30? 17:39, Dhyan wrote: > Dear All, > > From linux(2.6.35) arm page table architecture i can see we have one > hardware page table and there is corresponding Linux page table Entry > (L_PTE_*).The "Linux" PTE definitions are as like below from > arch/arm/include/asm/pgtable.h. > > |#define L_PTE_PRESENT (1<< 0) > #define L_PTE_FILE (1<< 1) > #define L_PTE_YOUNG (1<< 1) > #define L_PTE_BUFFERABLE(1<< 2) > #define L_PTE_CACHEABLE (1<< 3) > #define L_PTE_USER (1<< 4) > #define L_PTE_WRITE (1<< 5) > #define L_PTE_EXEC (1<< 6) > #define L_PTE_DIRTY (1<< 7) > #define L_PTE_COHERENT (1<< 9) > #define L_PTE_SHARED (1<< 10) > | > > So is it possible to add one more #|define L_PTE_DEBUG (1 << 11)| for my > debugging purpose (basically to trap all the write to that page and set > this bit when write happens and clear it off in another thread )? Or > is there any limitation like we can use only L_PTE till 10th bit ? > No such limitation on bit 11, so you can use define L_PTE_DEBUG (1 << 11) However I don't follow why you want to do so? > So could you please help > > -- > > Thanks & Regards > > Dhayn > > > > _______________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.org > http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies -- Love each day! --bill ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-07-31 7:13 ` bill4carson @ 2012-07-31 9:20 ` Dhyan 2012-08-01 2:08 ` bill4carson 0 siblings, 1 reply; 13+ messages in thread From: Dhyan @ 2012-07-31 9:20 UTC (permalink / raw) To: kernelnewbies Thank You Bill !!! I dont know my approach is correct or not,But the actual purpose was to dump only written pages of a user process using a kernel module.I have a kernel thread which will dump user process memory in specific interval.So i thought of updating this flag (L_PTE_DEBUG) from handle_pte_fault and clear from my clear thread so that i can dump only the written pages after my last dump. if you have some suggestions could you please share wth me? -- Thanks Dhyan On Tue, Jul 31, 2012 at 12:43 PM, bill4carson <bill4carson@gmail.com> wrote: > > > On 2012?07?30? 17:39, Dhyan wrote: > >> Dear All, >> >> From linux(2.6.35) arm page table architecture i can see we have one >> hardware page table and there is corresponding Linux page table Entry >> (L_PTE_*).The "Linux" PTE definitions are as like below from >> arch/arm/include/asm/pgtable.**h. >> >> |#define L_PTE_PRESENT (1<< 0) >> #define L_PTE_FILE (1<< 1) >> #define L_PTE_YOUNG (1<< 1) >> #define L_PTE_BUFFERABLE(1<< 2) >> #define L_PTE_CACHEABLE (1<< 3) >> #define L_PTE_USER (1<< 4) >> #define L_PTE_WRITE (1<< 5) >> #define L_PTE_EXEC (1<< 6) >> #define L_PTE_DIRTY (1<< 7) >> #define L_PTE_COHERENT (1<< 9) >> #define L_PTE_SHARED (1<< 10) >> | >> >> So is it possible to add one more #|define L_PTE_DEBUG (1 << 11)| for my >> >> debugging purpose (basically to trap all the write to that page and set >> this bit when write happens and clear it off in another thread )? Or >> is there any limitation like we can use only L_PTE till 10th bit ? >> >> > No such limitation on bit 11, so you can use define L_PTE_DEBUG (1 << 11) > However I don't follow why you want to do so? > > > So could you please help >> >> -- >> >> Thanks & Regards >> >> Dhayn >> >> >> >> ______________________________**_________________ >> Kernelnewbies mailing list >> Kernelnewbies at kernelnewbies.**org <Kernelnewbies@kernelnewbies.org> >> http://lists.kernelnewbies.**org/mailman/listinfo/**kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies> >> > > -- > Love each day! > > --bill > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120731/f56cc46e/attachment.html ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-07-31 9:20 ` Dhyan @ 2012-08-01 2:08 ` bill4carson 2012-08-01 4:53 ` Dhyan 0 siblings, 1 reply; 13+ messages in thread From: bill4carson @ 2012-08-01 2:08 UTC (permalink / raw) To: kernelnewbies On 2012?07?31? 17:20, Dhyan wrote: > Thank You Bill !!! > > I dont know my approach is correct or not,But the actual purpose was to > dump only written pages of a user process using a kernel module.I have > a kernel thread which will dump user process memory in specific > interval.So i thought of updating this flag (L_PTE_DEBUG) from > handle_pte_fault and clear from my clear thread so that i can dump only > the written pages after my last dump. Yes, you can do that, only if your accounting memory doesn't get swapped out. If I understand correctly, when writing a page, you mark corresponding linux pte entry with L_PTE_DEBUG. Then your kernel module periodically loops all linux pte table, find pte entry marked with L_PTE_DEBUG..... I don't think it's wise to do so, you have 768 1st level pgd entries for user space, followed by 256 pte entries with each pgd entry. it's much slower to find out the right one. moreover, you probably need to remap those L_PTE_DEBUG physical pages into your own current process address space. IMHO, I don't follow such idea could be feasible. > > if you have some suggestions could you please share wth me? I understand how you plan to do this, could I ask why you need to dump the written pages? > > -- > Thanks > Dhyan > > On Tue, Jul 31, 2012 at 12:43 PM, bill4carson <bill4carson@gmail.com > <mailto:bill4carson@gmail.com>> wrote: > > > > On 2012?07?30? 17:39, Dhyan wrote: > > Dear All, > > From linux(2.6.35) arm page table architecture i can see we > have one > hardware page table and there is corresponding Linux page table > Entry > (L_PTE_*).The "Linux" PTE definitions are as like below from > arch/arm/include/asm/pgtable.__h. > > |#define L_PTE_PRESENT (1<< 0) > #define L_PTE_FILE (1<< 1) > #define L_PTE_YOUNG (1<< 1) > #define L_PTE_BUFFERABLE(1<< 2) > #define L_PTE_CACHEABLE (1<< 3) > #define L_PTE_USER (1<< 4) > #define L_PTE_WRITE (1<< 5) > #define L_PTE_EXEC (1<< 6) > #define L_PTE_DIRTY (1<< 7) > #define L_PTE_COHERENT (1<< 9) > #define L_PTE_SHARED (1<< 10) > | > > So is it possible to add one more #|define L_PTE_DEBUG (1 << > 11)| for my > > debugging purpose (basically to trap all the write to that page > and set > this bit when write happens and clear it off in another thread > )? Or > is there any limitation like we can use only L_PTE till 10th bit ? > > > No such limitation on bit 11, so you can use define L_PTE_DEBUG (1 > << 11) > However I don't follow why you want to do so? > > > So could you please help > > -- > > Thanks & Regards > > Dhayn > > > > _________________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.__org > <mailto:Kernelnewbies@kernelnewbies.org> > http://lists.kernelnewbies.__org/mailman/listinfo/__kernelnewbies <http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies> > > > -- > Love each day! > > --bill > > -- Love each day! --bill ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-01 2:08 ` bill4carson @ 2012-08-01 4:53 ` Dhyan 2012-08-01 5:57 ` Mulyadi Santosa 2012-08-02 8:42 ` bill4carson 0 siblings, 2 replies; 13+ messages in thread From: Dhyan @ 2012-08-01 4:53 UTC (permalink / raw) To: kernelnewbies Dear Bill, Thank you for spending your valuable time for understanding and answering my queries !!! I was trying to apply some garbage collection algorithm on this dumped pages,thats why i want only the written pages. Sorry to ask,but is there any other good way to find the written pages of a user process? -- Thanks Dhyan On Wed, Aug 1, 2012 at 7:38 AM, bill4carson <bill4carson@gmail.com> wrote: > > > On 2012?07?31? 17:20, Dhyan wrote: > >> Thank You Bill !!! >> >> I dont know my approach is correct or not,But the actual purpose was to >> dump only written pages of a user process using a kernel module.I have >> a kernel thread which will dump user process memory in specific >> interval.So i thought of updating this flag (L_PTE_DEBUG) from >> handle_pte_fault and clear from my clear thread so that i can dump only >> the written pages after my last dump. >> > > Yes, you can do that, only if your accounting memory doesn't get swapped > out. > > If I understand correctly, when writing a page, you mark corresponding > linux pte entry with L_PTE_DEBUG. Then your kernel module periodically > loops all linux pte table, find pte entry marked with L_PTE_DEBUG..... > > I don't think it's wise to do so, you have 768 1st level pgd entries > for user space, followed by 256 pte entries with each pgd entry. > it's much slower to find out the right one. > > moreover, you probably need to remap those L_PTE_DEBUG physical pages > into your own current process address space. IMHO, I don't follow > such idea could be feasible. > > > >> if you have some suggestions could you please share wth me? >> > > I understand how you plan to do this, could I ask why you need to dump > the written pages? > > >> -- >> Thanks >> Dhyan >> >> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson <bill4carson@gmail.com >> <mailto:bill4carson@gmail.com>**> wrote: >> >> >> >> On 2012?07?30? 17:39, Dhyan wrote: >> >> Dear All, >> >> From linux(2.6.35) arm page table architecture i can see we >> have one >> hardware page table and there is corresponding Linux page table >> Entry >> (L_PTE_*).The "Linux" PTE definitions are as like below from >> arch/arm/include/asm/pgtable._**_h. >> >> >> |#define L_PTE_PRESENT (1<< 0) >> #define L_PTE_FILE (1<< 1) >> #define L_PTE_YOUNG (1<< 1) >> #define L_PTE_BUFFERABLE(1<< 2) >> #define L_PTE_CACHEABLE (1<< 3) >> #define L_PTE_USER (1<< 4) >> #define L_PTE_WRITE (1<< 5) >> #define L_PTE_EXEC (1<< 6) >> #define L_PTE_DIRTY (1<< 7) >> #define L_PTE_COHERENT (1<< 9) >> #define L_PTE_SHARED (1<< 10) >> | >> >> So is it possible to add one more #|define L_PTE_DEBUG (1 << >> 11)| for my >> >> debugging purpose (basically to trap all the write to that page >> and set >> this bit when write happens and clear it off in another thread >> )? Or >> is there any limitation like we can use only L_PTE till 10th bit ? >> >> >> No such limitation on bit 11, so you can use define L_PTE_DEBUG (1 >> << 11) >> However I don't follow why you want to do so? >> >> >> So could you please help >> >> -- >> >> Thanks & Regards >> >> Dhayn >> >> >> >> ______________________________**___________________ >> Kernelnewbies mailing list >> Kernelnewbies at kernelnewbies.__**org >> <mailto:Kernelnewbies@**kernelnewbies.org<Kernelnewbies@kernelnewbies.org> >> > >> http://lists.kernelnewbies.__**org/mailman/listinfo/__**kernelnewbies >> <http://lists.kernelnewbies.**org/mailman/listinfo/**kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies> >> > >> >> >> >> -- >> Love each day! >> >> --bill >> >> >> > -- > Love each day! > > --bill > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120801/7e331355/attachment.html ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-01 4:53 ` Dhyan @ 2012-08-01 5:57 ` Mulyadi Santosa 2012-08-01 6:17 ` Dhyan 2012-08-02 8:42 ` bill4carson 1 sibling, 1 reply; 13+ messages in thread From: Mulyadi Santosa @ 2012-08-01 5:57 UTC (permalink / raw) To: kernelnewbies Hi... Sorry for crossing in... On Wed, Aug 1, 2012 at 11:53 AM, Dhyan <linuxdhyan@gmail.com> wrote: > Sorry to ask,but is there any other good way to find the written pages of a > user process? by "written", you mean "dirty" pages? if yes, IIRC there's a flag that marks so....but that is for x86 and I forgot the name... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-01 5:57 ` Mulyadi Santosa @ 2012-08-01 6:17 ` Dhyan 2012-08-01 8:42 ` Mulyadi Santosa 0 siblings, 1 reply; 13+ messages in thread From: Dhyan @ 2012-08-01 6:17 UTC (permalink / raw) To: kernelnewbies Dear mulyadi santosa, I think you are mentioning the about PG_dirty flag.But in my case i want to clear the written flag of the user process from my kernel thread after each dump. So that next time when kernel thread runs it will dump only the pages written after last dump. -- Thanks Dhyan On Wed, Aug 1, 2012 at 11:27 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com>wrote: > Hi... > > Sorry for crossing in... > > On Wed, Aug 1, 2012 at 11:53 AM, Dhyan <linuxdhyan@gmail.com> wrote: > > Sorry to ask,but is there any other good way to find the written pages > of a > > user process? > > by "written", you mean "dirty" pages? > > if yes, IIRC there's a flag that marks so....but that is for x86 and I > forgot the name... > > > -- > regards, > > Mulyadi Santosa > Freelance Linux trainer and consultant > > blog: the-hydra.blogspot.com > training: mulyaditraining.blogspot.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120801/17f4f1af/attachment.html ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-01 6:17 ` Dhyan @ 2012-08-01 8:42 ` Mulyadi Santosa 0 siblings, 0 replies; 13+ messages in thread From: Mulyadi Santosa @ 2012-08-01 8:42 UTC (permalink / raw) To: kernelnewbies Hi.... On Wed, Aug 1, 2012 at 1:17 PM, Dhyan <linuxdhyan@gmail.com> wrote: > Dear mulyadi santosa, > > I think you are mentioning the about PG_dirty flag.But in my case i want to > clear the written flag of the user process from my kernel thread after each > dump. Oh, sorry then, I misunderstood your goal... > So that next time when kernel thread runs it will dump only the pages > written after last dump. Sounds like what you need could be implemented using a time stamp. Could be named like "last_written" and "last_dump". Both stores jiffies values, consecutively when last time it is written out and last time it is dumped out. And since they are jiffies_t, they probably better attached to "page_t". CMIIW... -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-01 4:53 ` Dhyan 2012-08-01 5:57 ` Mulyadi Santosa @ 2012-08-02 8:42 ` bill4carson 2012-08-02 9:03 ` Dhyan 1 sibling, 1 reply; 13+ messages in thread From: bill4carson @ 2012-08-02 8:42 UTC (permalink / raw) To: kernelnewbies On 2012?08?01? 12:53, Dhyan wrote: > Dear Bill, > > Thank you for spending your valuable time for understanding and > answering my queries !!! > > I was trying to apply some garbage collection algorithm on this dumped > pages,thats why i want only the written pages. > > Sorry to ask,but is there any other good way to find the written pages > of a user process? Only the first wirte trigger page fault, which setup the page table by grabbing a physical page to backup virtual address page. After this, all write into that page goes like wind without any kernel interference anymore. But my hunch tell me what you want is to track every user space write operation. > > -- > Thanks > Dhyan > > On Wed, Aug 1, 2012 at 7:38 AM, bill4carson <bill4carson@gmail.com > <mailto:bill4carson@gmail.com>> wrote: > > > > On 2012?07?31? 17:20, Dhyan wrote: > > Thank You Bill !!! > > I dont know my approach is correct or not,But the actual purpose > was to > dump only written pages of a user process using a kernel > module.I have > a kernel thread which will dump user process memory in specific > interval.So i thought of updating this flag (L_PTE_DEBUG) from > handle_pte_fault and clear from my clear thread so that i can > dump only > the written pages after my last dump. > > > Yes, you can do that, only if your accounting memory doesn't get swapped > out. > > If I understand correctly, when writing a page, you mark corresponding > linux pte entry with L_PTE_DEBUG. Then your kernel module periodically > loops all linux pte table, find pte entry marked with L_PTE_DEBUG..... > > I don't think it's wise to do so, you have 768 1st level pgd entries > for user space, followed by 256 pte entries with each pgd entry. > it's much slower to find out the right one. > > moreover, you probably need to remap those L_PTE_DEBUG physical pages > into your own current process address space. IMHO, I don't follow > such idea could be feasible. > > > > if you have some suggestions could you please share wth me? > > > I understand how you plan to do this, could I ask why you need to dump > the written pages? > > > -- > Thanks > Dhyan > > On Tue, Jul 31, 2012 at 12:43 PM, bill4carson > <bill4carson at gmail.com <mailto:bill4carson@gmail.com> > <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>>__> > wrote: > > > > On 2012?07?30? 17:39, Dhyan wrote: > > Dear All, > > From linux(2.6.35) arm page table architecture i can > see we > have one > hardware page table and there is corresponding Linux > page table > Entry > (L_PTE_*).The "Linux" PTE definitions are as like below > from > arch/arm/include/asm/pgtable.____h. > > > |#define L_PTE_PRESENT (1<< 0) > #define L_PTE_FILE (1<< 1) > #define L_PTE_YOUNG (1<< 1) > #define L_PTE_BUFFERABLE(1<< 2) > #define L_PTE_CACHEABLE (1<< 3) > #define L_PTE_USER (1<< 4) > #define L_PTE_WRITE (1<< 5) > #define L_PTE_EXEC (1<< 6) > #define L_PTE_DIRTY (1<< 7) > #define L_PTE_COHERENT (1<< 9) > #define L_PTE_SHARED (1<< 10) > | > > So is it possible to add one more #|define L_PTE_DEBUG > (1 << > 11)| for my > > debugging purpose (basically to trap all the write to > that page > and set > this bit when write happens and clear it off in another > thread > )? Or > is there any limitation like we can use only L_PTE till > 10th bit ? > > > No such limitation on bit 11, so you can use define > L_PTE_DEBUG (1 > << 11) > However I don't follow why you want to do so? > > > So could you please help > > -- > > Thanks & Regards > > Dhayn > > > > ___________________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.____org > <mailto:Kernelnewbies@__kernelnewbies.org > <mailto:Kernelnewbies@kernelnewbies.org>> > http://lists.kernelnewbies.____org/mailman/listinfo/____kernelnewbies > <http://lists.kernelnewbies.__org/mailman/listinfo/__kernelnewbies > <http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>> > > > > -- > Love each day! > > --bill > > > > -- > Love each day! > > --bill > > -- Love each day! --bill ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-02 8:42 ` bill4carson @ 2012-08-02 9:03 ` Dhyan 2012-08-02 9:28 ` bill4carson 0 siblings, 1 reply; 13+ messages in thread From: Dhyan @ 2012-08-02 9:03 UTC (permalink / raw) To: kernelnewbies Dear Bill, What i found from experimenting on arm Linux kernel is ,after every access if i clear the _PG_ACCESS bit of the pte (using /proc/<pid>/clear_refs),the next write also will come to kernel (handle_pte_fault).But I dont know whether my clearing action causing any bad impact on any other system. -- Thanks Dhyan On Thu, Aug 2, 2012 at 2:12 PM, bill4carson <bill4carson@gmail.com> wrote: > > > On 2012?08?01? 12:53, Dhyan wrote: > >> Dear Bill, >> >> Thank you for spending your valuable time for understanding and >> answering my queries !!! >> >> I was trying to apply some garbage collection algorithm on this dumped >> pages,thats why i want only the written pages. >> >> Sorry to ask,but is there any other good way to find the written pages >> of a user process? >> > > Only the first wirte trigger page fault, which setup the page table by > grabbing a physical page to backup virtual address page. After this, > all write into that page goes like wind without any kernel interference > anymore. > > But my hunch tell me what you want is to track every user space write > operation. > > > >> -- >> Thanks >> Dhyan >> >> On Wed, Aug 1, 2012 at 7:38 AM, bill4carson <bill4carson@gmail.com >> <mailto:bill4carson@gmail.com>**> wrote: >> >> >> >> On 2012?07?31? 17:20, Dhyan wrote: >> >> Thank You Bill !!! >> >> I dont know my approach is correct or not,But the actual purpose >> was to >> dump only written pages of a user process using a kernel >> module.I have >> a kernel thread which will dump user process memory in specific >> interval.So i thought of updating this flag (L_PTE_DEBUG) from >> handle_pte_fault and clear from my clear thread so that i can >> dump only >> the written pages after my last dump. >> >> >> Yes, you can do that, only if your accounting memory doesn't get >> swapped >> out. >> >> If I understand correctly, when writing a page, you mark corresponding >> linux pte entry with L_PTE_DEBUG. Then your kernel module periodically >> loops all linux pte table, find pte entry marked with L_PTE_DEBUG..... >> >> I don't think it's wise to do so, you have 768 1st level pgd entries >> for user space, followed by 256 pte entries with each pgd entry. >> it's much slower to find out the right one. >> >> moreover, you probably need to remap those L_PTE_DEBUG physical pages >> into your own current process address space. IMHO, I don't follow >> such idea could be feasible. >> >> >> >> if you have some suggestions could you please share wth me? >> >> >> I understand how you plan to do this, could I ask why you need to dump >> the written pages? >> >> >> -- >> Thanks >> Dhyan >> >> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson >> <bill4carson at gmail.com <mailto:bill4carson@gmail.com> >> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>** >> >__> >> >> wrote: >> >> >> >> On 2012?07?30? 17:39, Dhyan wrote: >> >> Dear All, >> >> From linux(2.6.35) arm page table architecture i can >> see we >> have one >> hardware page table and there is corresponding Linux >> page table >> Entry >> (L_PTE_*).The "Linux" PTE definitions are as like below >> from >> arch/arm/include/asm/pgtable._**___h. >> >> >> >> |#define L_PTE_PRESENT (1<< 0) >> #define L_PTE_FILE (1<< 1) >> #define L_PTE_YOUNG (1<< 1) >> #define L_PTE_BUFFERABLE(1<< 2) >> #define L_PTE_CACHEABLE (1<< 3) >> #define L_PTE_USER (1<< 4) >> #define L_PTE_WRITE (1<< 5) >> #define L_PTE_EXEC (1<< 6) >> #define L_PTE_DIRTY (1<< 7) >> #define L_PTE_COHERENT (1<< 9) >> #define L_PTE_SHARED (1<< 10) >> | >> >> So is it possible to add one more #|define L_PTE_DEBUG >> (1 << >> 11)| for my >> >> debugging purpose (basically to trap all the write to >> that page >> and set >> this bit when write happens and clear it off in another >> thread >> )? Or >> is there any limitation like we can use only L_PTE till >> 10th bit ? >> >> >> No such limitation on bit 11, so you can use define >> L_PTE_DEBUG (1 >> << 11) >> However I don't follow why you want to do so? >> >> >> So could you please help >> >> -- >> >> Thanks & Regards >> >> Dhayn >> >> >> >> ______________________________**_____________________ >> Kernelnewbies mailing list >> Kernelnewbies at kernelnewbies.__**__org >> <mailto:Kernelnewbies@__kernel**newbies.org<http://kernelnewbies.org> >> <mailto:Kernelnewbies@**kernelnewbies.org<Kernelnewbies@kernelnewbies.org> >> >> >> http://lists.kernelnewbies.___**_org/mailman/listinfo/____** >> kernelnewbies >> >> <http://lists.kernelnewbies.__**org/mailman/listinfo/__** >> kernelnewbies >> <http://lists.kernelnewbies.**org/mailman/listinfo/** >> kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies> >> >> >> >> >> >> -- >> Love each day! >> >> --bill >> >> >> >> -- >> Love each day! >> >> --bill >> >> >> > -- > Love each day! > > --bill > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120802/572c9b86/attachment-0001.html ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-02 9:03 ` Dhyan @ 2012-08-02 9:28 ` bill4carson 2012-08-02 9:33 ` Dhyan 0 siblings, 1 reply; 13+ messages in thread From: bill4carson @ 2012-08-02 9:28 UTC (permalink / raw) To: kernelnewbies On 2012?08?02? 17:03, Dhyan wrote: > Dear Bill, > > What i found from experimenting on arm Linux kernel is ,after every > access if i clear the _PG_ACCESS bit of the pte (using > /proc/<pid>/clear_refs),the next write also will come to kernel > (handle_pte_fault).But I dont know whether my clearing action causing > any bad impact on any other system. > 592 static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr, 593 unsigned long end, struct mm_walk *walk) 594 { 595 struct vm_area_struct *vma = walk->private; 596 pte_t *pte, ptent; 597 spinlock_t *ptl; 598 struct page *page; 599 600 split_huge_page_pmd(walk->mm, pmd); 601 if (pmd_trans_unstable(pmd)) 602 return 0; 603 604 pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl); 605 for (; addr != end; pte++, addr += PAGE_SIZE) { 606 ptent = *pte; 607 if (!pte_present(ptent)) 608 continue; 609 610 page = vm_normal_page(vma, addr, ptent); 611 if (!page) 612 continue; 613 614 /* Clear accessed and referenced bits. */ 615 ptep_test_and_clear_young(vma, addr, pte); ^^^^^^^^^^^^ For arm, it clear all the corresponding pte entries. that's why you got second page fault. Again, I don't understand your original purpose. > -- > Thanks > Dhyan > > On Thu, Aug 2, 2012 at 2:12 PM, bill4carson <bill4carson@gmail.com > <mailto:bill4carson@gmail.com>> wrote: > > > > On 2012?08?01? 12:53, Dhyan wrote: > > Dear Bill, > > Thank you for spending your valuable time for understanding and > answering my queries !!! > > I was trying to apply some garbage collection algorithm on this > dumped > pages,thats why i want only the written pages. > > Sorry to ask,but is there any other good way to find the written > pages > of a user process? > > > Only the first wirte trigger page fault, which setup the page table by > grabbing a physical page to backup virtual address page. After this, > all write into that page goes like wind without any kernel interference > anymore. > > > But my hunch tell me what you want is to track every user space write > operation. > > > > -- > Thanks > Dhyan > > On Wed, Aug 1, 2012 at 7:38 AM, bill4carson > <bill4carson at gmail.com <mailto:bill4carson@gmail.com> > <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>>__> > wrote: > > > > On 2012?07?31? 17:20, Dhyan wrote: > > Thank You Bill !!! > > I dont know my approach is correct or not,But the > actual purpose > was to > dump only written pages of a user process using a kernel > module.I have > a kernel thread which will dump user process memory in > specific > interval.So i thought of updating this flag > (L_PTE_DEBUG) from > handle_pte_fault and clear from my clear thread so that > i can > dump only > the written pages after my last dump. > > > Yes, you can do that, only if your accounting memory > doesn't get swapped > out. > > If I understand correctly, when writing a page, you mark > corresponding > linux pte entry with L_PTE_DEBUG. Then your kernel module > periodically > loops all linux pte table, find pte entry marked with > L_PTE_DEBUG..... > > I don't think it's wise to do so, you have 768 1st level > pgd entries > for user space, followed by 256 pte entries with each pgd > entry. > it's much slower to find out the right one. > > moreover, you probably need to remap those L_PTE_DEBUG > physical pages > into your own current process address space. IMHO, I don't > follow > such idea could be feasible. > > > > if you have some suggestions could you please share > wth me? > > > I understand how you plan to do this, could I ask why you > need to dump > the written pages? > > > -- > Thanks > Dhyan > > On Tue, Jul 31, 2012 at 12:43 PM, bill4carson > <bill4carson at gmail.com <mailto:bill4carson@gmail.com> > <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>> > <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com> > <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>>__>__> > > wrote: > > > > On 2012?07?30? 17:39, Dhyan wrote: > > Dear All, > > From linux(2.6.35) arm page table > architecture i can > see we > have one > hardware page table and there is > corresponding Linux > page table > Entry > (L_PTE_*).The "Linux" PTE definitions are as > like below > from > arch/arm/include/asm/pgtable.______h. > > > > |#define L_PTE_PRESENT (1<< 0) > #define L_PTE_FILE (1<< 1) > #define L_PTE_YOUNG (1<< 1) > #define L_PTE_BUFFERABLE(1<< 2) > #define L_PTE_CACHEABLE (1<< 3) > #define L_PTE_USER (1<< 4) > #define L_PTE_WRITE (1<< 5) > #define L_PTE_EXEC (1<< 6) > #define L_PTE_DIRTY (1<< 7) > #define L_PTE_COHERENT (1<< 9) > #define L_PTE_SHARED (1<< 10) > | > > So is it possible to add one more #|define > L_PTE_DEBUG > (1 << > 11)| for my > > debugging purpose (basically to trap all the > write to > that page > and set > this bit when write happens and clear it off > in another > thread > )? Or > is there any limitation like we can use only > L_PTE till > 10th bit ? > > > No such limitation on bit 11, so you can use define > L_PTE_DEBUG (1 > << 11) > However I don't follow why you want to do so? > > > So could you please help > > -- > > Thanks & Regards > > Dhayn > > > > > _____________________________________________________ > Kernelnewbies mailing list > Kernelnewbies at kernelnewbies.______org > <mailto:Kernelnewbies@ > <mailto:Kernelnewbies@>__kernel__newbies.org > <http://kernelnewbies.org> > <mailto:Kernelnewbies@__kernelnewbies.org > <mailto:Kernelnewbies@kernelnewbies.org>>> > http://lists.kernelnewbies.______org/mailman/listinfo/______kernelnewbies > > <http://lists.kernelnewbies.____org/mailman/listinfo/____kernelnewbies > <http://lists.kernelnewbies.__org/mailman/listinfo/__kernelnewbies > <http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>>> > > > > -- > Love each day! > > --bill > > > > -- > Love each day! > > --bill > > > > -- > Love each day! > > --bill > > -- Love each day! --bill ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose 2012-08-02 9:28 ` bill4carson @ 2012-08-02 9:33 ` Dhyan 0 siblings, 0 replies; 13+ messages in thread From: Dhyan @ 2012-08-02 9:33 UTC (permalink / raw) To: kernelnewbies I will dump all the written pages of the application and run a garbage collection algorithm over the dumped contents. Thanks Dhyan On Thu, Aug 2, 2012 at 2:58 PM, bill4carson <bill4carson@gmail.com> wrote: > > > On 2012?08?02? 17:03, Dhyan wrote: > >> Dear Bill, >> >> What i found from experimenting on arm Linux kernel is ,after every >> access if i clear the _PG_ACCESS bit of the pte (using >> /proc/<pid>/clear_refs),the next write also will come to kernel >> (handle_pte_fault).But I dont know whether my clearing action causing >> any bad impact on any other system. >> >> > 592 static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr, > 593 unsigned long end, struct mm_walk *walk) > 594 { > 595 struct vm_area_struct *vma = walk->private; > 596 pte_t *pte, ptent; > 597 spinlock_t *ptl; > 598 struct page *page; > 599 > 600 split_huge_page_pmd(walk->mm, pmd); > 601 if (pmd_trans_unstable(pmd)) > 602 return 0; > 603 > 604 pte = pte_offset_map_lock(vma->vm_**mm, pmd, addr, &ptl); > 605 for (; addr != end; pte++, addr += PAGE_SIZE) { > 606 ptent = *pte; > 607 if (!pte_present(ptent)) > 608 continue; > 609 > 610 page = vm_normal_page(vma, addr, ptent); > 611 if (!page) > 612 continue; > 613 > 614 /* Clear accessed and referenced bits. */ > 615 ptep_test_and_clear_young(vma, addr, pte); > ^^^^^^^^^^^^ > For arm, it clear all the corresponding pte entries. > that's why you got second page fault. > > Again, I don't understand your original purpose > >> -- >> Thanks >> Dhyan >> >> On Thu, Aug 2, 2012 at 2:12 PM, bill4carson <bill4carson@gmail.com >> <mailto:bill4carson@gmail.com>**> wrote: >> >> >> >> On 2012?08?01? 12:53, Dhyan wrote: >> >> Dear Bill, >> >> Thank you for spending your valuable time for understanding and >> answering my queries !!! >> >> I was trying to apply some garbage collection algorithm on this >> dumped >> pages,thats why i want only the written pages. >> >> Sorry to ask,but is there any other good way to find the written >> pages >> of a user process? >> >> >> Only the first wirte trigger page fault, which setup the page table by >> grabbing a physical page to backup virtual address page. After this, >> all write into that page goes like wind without any kernel >> interference >> anymore. >> >> >> But my hunch tell me what you want is to track every user space write >> operation. >> >> >> >> -- >> Thanks >> Dhyan >> >> On Wed, Aug 1, 2012 at 7:38 AM, bill4carson >> <bill4carson at gmail.com <mailto:bill4carson@gmail.com> >> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>** >> >__> >> wrote: >> >> >> >> On 2012?07?31? 17:20, Dhyan wrote: >> >> Thank You Bill !!! >> >> I dont know my approach is correct or not,But the >> actual purpose >> was to >> dump only written pages of a user process using a kernel >> module.I have >> a kernel thread which will dump user process memory in >> specific >> interval.So i thought of updating this flag >> (L_PTE_DEBUG) from >> handle_pte_fault and clear from my clear thread so that >> i can >> dump only >> the written pages after my last dump. >> >> >> Yes, you can do that, only if your accounting memory >> doesn't get swapped >> out. >> >> If I understand correctly, when writing a page, you mark >> corresponding >> linux pte entry with L_PTE_DEBUG. Then your kernel module >> periodically >> loops all linux pte table, find pte entry marked with >> L_PTE_DEBUG..... >> >> I don't think it's wise to do so, you have 768 1st level >> pgd entries >> for user space, followed by 256 pte entries with each pgd >> entry. >> it's much slower to find out the right one. >> >> moreover, you probably need to remap those L_PTE_DEBUG >> physical pages >> into your own current process address space. IMHO, I don't >> follow >> such idea could be feasible. >> >> >> >> if you have some suggestions could you please share >> wth me? >> >> >> I understand how you plan to do this, could I ask why you >> need to dump >> the written pages? >> >> >> -- >> Thanks >> Dhyan >> >> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson >> <bill4carson at gmail.com <mailto:bill4carson@gmail.com> >> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>**> >> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com> >> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>** >> >__>__> >> >> >> wrote: >> >> >> >> On 2012?07?30? 17:39, Dhyan wrote: >> >> Dear All, >> >> From linux(2.6.35) arm page table >> architecture i can >> see we >> have one >> hardware page table and there is >> corresponding Linux >> page table >> Entry >> (L_PTE_*).The "Linux" PTE definitions are as >> like below >> from >> arch/arm/include/asm/pgtable._**_____h. >> >> >> >> >> |#define L_PTE_PRESENT (1<< 0) >> #define L_PTE_FILE (1<< 1) >> #define L_PTE_YOUNG (1<< 1) >> #define L_PTE_BUFFERABLE(1<< 2) >> #define L_PTE_CACHEABLE (1<< 3) >> #define L_PTE_USER (1<< 4) >> #define L_PTE_WRITE (1<< 5) >> #define L_PTE_EXEC (1<< 6) >> #define L_PTE_DIRTY (1<< 7) >> #define L_PTE_COHERENT (1<< 9) >> #define L_PTE_SHARED (1<< 10) >> | >> >> So is it possible to add one more #|define >> L_PTE_DEBUG >> (1 << >> 11)| for my >> >> debugging purpose (basically to trap all the >> write to >> that page >> and set >> this bit when write happens and clear it off >> in another >> thread >> )? Or >> is there any limitation like we can use only >> L_PTE till >> 10th bit ? >> >> >> No such limitation on bit 11, so you can use define >> L_PTE_DEBUG (1 >> << 11) >> However I don't follow why you want to do so? >> >> >> So could you please help >> >> -- >> >> Thanks & Regards >> >> Dhayn >> >> >> >> >> ______________________________**_______________________ >> Kernelnewbies mailing list >> Kernelnewbies at kernelnewbies.__**____org >> <mailto:Kernelnewbies@ >> <mailto:Kernelnewbies@>__kerne**l__newbies.org<http://kernel__newbies.org> >> <http://kernelnewbies.org> >> <mailto:Kernelnewbies@__kernel**newbies.org<http://kernelnewbies.org> >> <mailto:Kernelnewbies@**kernelnewbies.org<Kernelnewbies@kernelnewbies.org> >> >>> >> http://lists.kernelnewbies.___**___org/mailman/listinfo/______** >> kernelnewbies >> >> >> <http://lists.kernelnewbies.__**__org/mailman/listinfo/____** >> kernelnewbies >> <http://lists.kernelnewbies.__**org/mailman/listinfo/__** >> kernelnewbies >> <http://lists.kernelnewbies.**org/mailman/listinfo/** >> kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies> >> >>> >> >> >> >> -- >> Love each day! >> >> --bill >> >> >> >> -- >> Love each day! >> >> --bill >> >> >> >> -- >> Love each day! >> >> --bill >> >> >> > -- > Love each day! > > --bill > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120802/e505efe7/attachment-0001.html ^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
@ 2012-08-01 6:54 Vladimir Murzin
0 siblings, 0 replies; 13+ messages in thread
From: Vladimir Murzin @ 2012-08-01 6:54 UTC (permalink / raw)
To: kernelnewbies
Hi!
AFAIK it's PG_dirty page flag.
Hope this helps!
Vladimir Murzin
------Original Message------
From: Mulyadi Santosa
Sender: kernelnewbies-bounces at kernelnewbies.org
To: Dhyan
Cc: bill4carson
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: arm L_PTE_XXX entry addition for Debugging purpose
Sent: 1 Aug 2012 09:57
Hi...
Sorry for crossing in...
On Wed, Aug 1, 2012 at 11:53 AM, Dhyan <linuxdhyan@gmail.com> wrote:
> Sorry to ask,but is there any other good way to find the written pages of a
> user process?
by "written", you mean "dirty" pages?
if yes, IIRC there's a flag that marks so....but that is for x86 and I
forgot the name...
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
^ permalink raw reply [flat|nested] 13+ messages in threadend of thread, other threads:[~2012-08-02 9:33 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-07-30 9:39 arm L_PTE_XXX entry addition for Debugging purpose Dhyan 2012-07-31 7:13 ` bill4carson 2012-07-31 9:20 ` Dhyan 2012-08-01 2:08 ` bill4carson 2012-08-01 4:53 ` Dhyan 2012-08-01 5:57 ` Mulyadi Santosa 2012-08-01 6:17 ` Dhyan 2012-08-01 8:42 ` Mulyadi Santosa 2012-08-02 8:42 ` bill4carson 2012-08-02 9:03 ` Dhyan 2012-08-02 9:28 ` bill4carson 2012-08-02 9:33 ` Dhyan -- strict thread matches above, loose matches on Subject: below -- 2012-08-01 6:54 Vladimir Murzin
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).