* arm L_PTE_XXX entry addition for Debugging purpose
@ 2012-07-30 9:39 Dhyan
2012-07-31 7:13 ` bill4carson
0 siblings, 1 reply; 13+ messages in thread
From: Dhyan @ 2012-07-30 9:39 UTC (permalink / raw)
To: kernelnewbies
Dear All,
>From linux(2.6.35) arm page table architecture i can see we have one
hardware page table and there is corresponding Linux page table Entry
(L_PTE_*).The "Linux" PTE definitions are as like below from
arch/arm/include/asm/pgtable.h.
#define L_PTE_PRESENT (1 << 0)
#define L_PTE_FILE (1 << 1)
#define L_PTE_YOUNG (1 << 1)
#define L_PTE_BUFFERABLE(1 << 2)
#define L_PTE_CACHEABLE (1 << 3)
#define L_PTE_USER (1 << 4)
#define L_PTE_WRITE (1 << 5)
#define L_PTE_EXEC (1 << 6)
#define L_PTE_DIRTY (1 << 7)
#define L_PTE_COHERENT (1 << 9)
#define L_PTE_SHARED (1 << 10)
So is it possible to add one more #define L_PTE_DEBUG (1 << 11) for my
debugging purpose (basically to trap all the write to that page and set
this bit when write happens and clear it off in another thread )? Or is
there any limitation like we can use only L_PTE till 10th bit ?
So could you please help
--
Thanks & Regards
Dhayn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120730/ff28b36d/attachment.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-07-30 9:39 arm L_PTE_XXX entry addition for Debugging purpose Dhyan
@ 2012-07-31 7:13 ` bill4carson
2012-07-31 9:20 ` Dhyan
0 siblings, 1 reply; 13+ messages in thread
From: bill4carson @ 2012-07-31 7:13 UTC (permalink / raw)
To: kernelnewbies
On 2012?07?30? 17:39, Dhyan wrote:
> Dear All,
>
> From linux(2.6.35) arm page table architecture i can see we have one
> hardware page table and there is corresponding Linux page table Entry
> (L_PTE_*).The "Linux" PTE definitions are as like below from
> arch/arm/include/asm/pgtable.h.
>
> |#define L_PTE_PRESENT (1<< 0)
> #define L_PTE_FILE (1<< 1)
> #define L_PTE_YOUNG (1<< 1)
> #define L_PTE_BUFFERABLE(1<< 2)
> #define L_PTE_CACHEABLE (1<< 3)
> #define L_PTE_USER (1<< 4)
> #define L_PTE_WRITE (1<< 5)
> #define L_PTE_EXEC (1<< 6)
> #define L_PTE_DIRTY (1<< 7)
> #define L_PTE_COHERENT (1<< 9)
> #define L_PTE_SHARED (1<< 10)
> |
>
> So is it possible to add one more #|define L_PTE_DEBUG (1 << 11)| for my
> debugging purpose (basically to trap all the write to that page and set
> this bit when write happens and clear it off in another thread )? Or
> is there any limitation like we can use only L_PTE till 10th bit ?
>
No such limitation on bit 11, so you can use define L_PTE_DEBUG (1 << 11)
However I don't follow why you want to do so?
> So could you please help
>
> --
>
> Thanks & Regards
>
> Dhayn
>
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
Love each day!
--bill
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-07-31 7:13 ` bill4carson
@ 2012-07-31 9:20 ` Dhyan
2012-08-01 2:08 ` bill4carson
0 siblings, 1 reply; 13+ messages in thread
From: Dhyan @ 2012-07-31 9:20 UTC (permalink / raw)
To: kernelnewbies
Thank You Bill !!!
I dont know my approach is correct or not,But the actual purpose was to
dump only written pages of a user process using a kernel module.I have a
kernel thread which will dump user process memory in specific interval.So i
thought of updating this flag (L_PTE_DEBUG) from handle_pte_fault and clear
from my clear thread so that i can dump only the written pages after my
last dump.
if you have some suggestions could you please share wth me?
--
Thanks
Dhyan
On Tue, Jul 31, 2012 at 12:43 PM, bill4carson <bill4carson@gmail.com> wrote:
>
>
> On 2012?07?30? 17:39, Dhyan wrote:
>
>> Dear All,
>>
>> From linux(2.6.35) arm page table architecture i can see we have one
>> hardware page table and there is corresponding Linux page table Entry
>> (L_PTE_*).The "Linux" PTE definitions are as like below from
>> arch/arm/include/asm/pgtable.**h.
>>
>> |#define L_PTE_PRESENT (1<< 0)
>> #define L_PTE_FILE (1<< 1)
>> #define L_PTE_YOUNG (1<< 1)
>> #define L_PTE_BUFFERABLE(1<< 2)
>> #define L_PTE_CACHEABLE (1<< 3)
>> #define L_PTE_USER (1<< 4)
>> #define L_PTE_WRITE (1<< 5)
>> #define L_PTE_EXEC (1<< 6)
>> #define L_PTE_DIRTY (1<< 7)
>> #define L_PTE_COHERENT (1<< 9)
>> #define L_PTE_SHARED (1<< 10)
>> |
>>
>> So is it possible to add one more #|define L_PTE_DEBUG (1 << 11)| for my
>>
>> debugging purpose (basically to trap all the write to that page and set
>> this bit when write happens and clear it off in another thread )? Or
>> is there any limitation like we can use only L_PTE till 10th bit ?
>>
>>
> No such limitation on bit 11, so you can use define L_PTE_DEBUG (1 << 11)
> However I don't follow why you want to do so?
>
>
> So could you please help
>>
>> --
>>
>> Thanks & Regards
>>
>> Dhayn
>>
>>
>>
>> ______________________________**_________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.**org <Kernelnewbies@kernelnewbies.org>
>> http://lists.kernelnewbies.**org/mailman/listinfo/**kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
>>
>
> --
> Love each day!
>
> --bill
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120731/f56cc46e/attachment.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-07-31 9:20 ` Dhyan
@ 2012-08-01 2:08 ` bill4carson
2012-08-01 4:53 ` Dhyan
0 siblings, 1 reply; 13+ messages in thread
From: bill4carson @ 2012-08-01 2:08 UTC (permalink / raw)
To: kernelnewbies
On 2012?07?31? 17:20, Dhyan wrote:
> Thank You Bill !!!
>
> I dont know my approach is correct or not,But the actual purpose was to
> dump only written pages of a user process using a kernel module.I have
> a kernel thread which will dump user process memory in specific
> interval.So i thought of updating this flag (L_PTE_DEBUG) from
> handle_pte_fault and clear from my clear thread so that i can dump only
> the written pages after my last dump.
Yes, you can do that, only if your accounting memory doesn't get swapped
out.
If I understand correctly, when writing a page, you mark corresponding
linux pte entry with L_PTE_DEBUG. Then your kernel module periodically
loops all linux pte table, find pte entry marked with L_PTE_DEBUG.....
I don't think it's wise to do so, you have 768 1st level pgd entries
for user space, followed by 256 pte entries with each pgd entry.
it's much slower to find out the right one.
moreover, you probably need to remap those L_PTE_DEBUG physical pages
into your own current process address space. IMHO, I don't follow
such idea could be feasible.
>
> if you have some suggestions could you please share wth me?
I understand how you plan to do this, could I ask why you need to dump
the written pages?
>
> --
> Thanks
> Dhyan
>
> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson <bill4carson@gmail.com
> <mailto:bill4carson@gmail.com>> wrote:
>
>
>
> On 2012?07?30? 17:39, Dhyan wrote:
>
> Dear All,
>
> From linux(2.6.35) arm page table architecture i can see we
> have one
> hardware page table and there is corresponding Linux page table
> Entry
> (L_PTE_*).The "Linux" PTE definitions are as like below from
> arch/arm/include/asm/pgtable.__h.
>
> |#define L_PTE_PRESENT (1<< 0)
> #define L_PTE_FILE (1<< 1)
> #define L_PTE_YOUNG (1<< 1)
> #define L_PTE_BUFFERABLE(1<< 2)
> #define L_PTE_CACHEABLE (1<< 3)
> #define L_PTE_USER (1<< 4)
> #define L_PTE_WRITE (1<< 5)
> #define L_PTE_EXEC (1<< 6)
> #define L_PTE_DIRTY (1<< 7)
> #define L_PTE_COHERENT (1<< 9)
> #define L_PTE_SHARED (1<< 10)
> |
>
> So is it possible to add one more #|define L_PTE_DEBUG (1 <<
> 11)| for my
>
> debugging purpose (basically to trap all the write to that page
> and set
> this bit when write happens and clear it off in another thread
> )? Or
> is there any limitation like we can use only L_PTE till 10th bit ?
>
>
> No such limitation on bit 11, so you can use define L_PTE_DEBUG (1
> << 11)
> However I don't follow why you want to do so?
>
>
> So could you please help
>
> --
>
> Thanks & Regards
>
> Dhayn
>
>
>
> _________________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.__org
> <mailto:Kernelnewbies@kernelnewbies.org>
> http://lists.kernelnewbies.__org/mailman/listinfo/__kernelnewbies <http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
>
>
> --
> Love each day!
>
> --bill
>
>
--
Love each day!
--bill
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-01 2:08 ` bill4carson
@ 2012-08-01 4:53 ` Dhyan
2012-08-01 5:57 ` Mulyadi Santosa
2012-08-02 8:42 ` bill4carson
0 siblings, 2 replies; 13+ messages in thread
From: Dhyan @ 2012-08-01 4:53 UTC (permalink / raw)
To: kernelnewbies
Dear Bill,
Thank you for spending your valuable time for understanding and answering
my queries !!!
I was trying to apply some garbage collection algorithm on this dumped
pages,thats why i want only the written pages.
Sorry to ask,but is there any other good way to find the written pages of a
user process?
--
Thanks
Dhyan
On Wed, Aug 1, 2012 at 7:38 AM, bill4carson <bill4carson@gmail.com> wrote:
>
>
> On 2012?07?31? 17:20, Dhyan wrote:
>
>> Thank You Bill !!!
>>
>> I dont know my approach is correct or not,But the actual purpose was to
>> dump only written pages of a user process using a kernel module.I have
>> a kernel thread which will dump user process memory in specific
>> interval.So i thought of updating this flag (L_PTE_DEBUG) from
>> handle_pte_fault and clear from my clear thread so that i can dump only
>> the written pages after my last dump.
>>
>
> Yes, you can do that, only if your accounting memory doesn't get swapped
> out.
>
> If I understand correctly, when writing a page, you mark corresponding
> linux pte entry with L_PTE_DEBUG. Then your kernel module periodically
> loops all linux pte table, find pte entry marked with L_PTE_DEBUG.....
>
> I don't think it's wise to do so, you have 768 1st level pgd entries
> for user space, followed by 256 pte entries with each pgd entry.
> it's much slower to find out the right one.
>
> moreover, you probably need to remap those L_PTE_DEBUG physical pages
> into your own current process address space. IMHO, I don't follow
> such idea could be feasible.
>
>
>
>> if you have some suggestions could you please share wth me?
>>
>
> I understand how you plan to do this, could I ask why you need to dump
> the written pages?
>
>
>> --
>> Thanks
>> Dhyan
>>
>> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson <bill4carson@gmail.com
>> <mailto:bill4carson@gmail.com>**> wrote:
>>
>>
>>
>> On 2012?07?30? 17:39, Dhyan wrote:
>>
>> Dear All,
>>
>> From linux(2.6.35) arm page table architecture i can see we
>> have one
>> hardware page table and there is corresponding Linux page table
>> Entry
>> (L_PTE_*).The "Linux" PTE definitions are as like below from
>> arch/arm/include/asm/pgtable._**_h.
>>
>>
>> |#define L_PTE_PRESENT (1<< 0)
>> #define L_PTE_FILE (1<< 1)
>> #define L_PTE_YOUNG (1<< 1)
>> #define L_PTE_BUFFERABLE(1<< 2)
>> #define L_PTE_CACHEABLE (1<< 3)
>> #define L_PTE_USER (1<< 4)
>> #define L_PTE_WRITE (1<< 5)
>> #define L_PTE_EXEC (1<< 6)
>> #define L_PTE_DIRTY (1<< 7)
>> #define L_PTE_COHERENT (1<< 9)
>> #define L_PTE_SHARED (1<< 10)
>> |
>>
>> So is it possible to add one more #|define L_PTE_DEBUG (1 <<
>> 11)| for my
>>
>> debugging purpose (basically to trap all the write to that page
>> and set
>> this bit when write happens and clear it off in another thread
>> )? Or
>> is there any limitation like we can use only L_PTE till 10th bit ?
>>
>>
>> No such limitation on bit 11, so you can use define L_PTE_DEBUG (1
>> << 11)
>> However I don't follow why you want to do so?
>>
>>
>> So could you please help
>>
>> --
>>
>> Thanks & Regards
>>
>> Dhayn
>>
>>
>>
>> ______________________________**___________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.__**org
>> <mailto:Kernelnewbies@**kernelnewbies.org<Kernelnewbies@kernelnewbies.org>
>> >
>> http://lists.kernelnewbies.__**org/mailman/listinfo/__**kernelnewbies
>> <http://lists.kernelnewbies.**org/mailman/listinfo/**kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
>> >
>>
>>
>>
>> --
>> Love each day!
>>
>> --bill
>>
>>
>>
> --
> Love each day!
>
> --bill
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120801/7e331355/attachment.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-01 4:53 ` Dhyan
@ 2012-08-01 5:57 ` Mulyadi Santosa
2012-08-01 6:17 ` Dhyan
2012-08-02 8:42 ` bill4carson
1 sibling, 1 reply; 13+ messages in thread
From: Mulyadi Santosa @ 2012-08-01 5:57 UTC (permalink / raw)
To: kernelnewbies
Hi...
Sorry for crossing in...
On Wed, Aug 1, 2012 at 11:53 AM, Dhyan <linuxdhyan@gmail.com> wrote:
> Sorry to ask,but is there any other good way to find the written pages of a
> user process?
by "written", you mean "dirty" pages?
if yes, IIRC there's a flag that marks so....but that is for x86 and I
forgot the name...
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-01 5:57 ` Mulyadi Santosa
@ 2012-08-01 6:17 ` Dhyan
2012-08-01 8:42 ` Mulyadi Santosa
0 siblings, 1 reply; 13+ messages in thread
From: Dhyan @ 2012-08-01 6:17 UTC (permalink / raw)
To: kernelnewbies
Dear mulyadi santosa,
I think you are mentioning the about PG_dirty flag.But in my case i want
to clear the written flag of the user process from my kernel thread after
each dump.
So that next time when kernel thread runs it will dump only the pages
written after last dump.
--
Thanks
Dhyan
On Wed, Aug 1, 2012 at 11:27 AM, Mulyadi Santosa
<mulyadi.santosa@gmail.com>wrote:
> Hi...
>
> Sorry for crossing in...
>
> On Wed, Aug 1, 2012 at 11:53 AM, Dhyan <linuxdhyan@gmail.com> wrote:
> > Sorry to ask,but is there any other good way to find the written pages
> of a
> > user process?
>
> by "written", you mean "dirty" pages?
>
> if yes, IIRC there's a flag that marks so....but that is for x86 and I
> forgot the name...
>
>
> --
> regards,
>
> Mulyadi Santosa
> Freelance Linux trainer and consultant
>
> blog: the-hydra.blogspot.com
> training: mulyaditraining.blogspot.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120801/17f4f1af/attachment.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
@ 2012-08-01 6:54 Vladimir Murzin
0 siblings, 0 replies; 13+ messages in thread
From: Vladimir Murzin @ 2012-08-01 6:54 UTC (permalink / raw)
To: kernelnewbies
Hi!
AFAIK it's PG_dirty page flag.
Hope this helps!
Vladimir Murzin
------Original Message------
From: Mulyadi Santosa
Sender: kernelnewbies-bounces at kernelnewbies.org
To: Dhyan
Cc: bill4carson
Cc: kernelnewbies at kernelnewbies.org
Subject: Re: arm L_PTE_XXX entry addition for Debugging purpose
Sent: 1 Aug 2012 09:57
Hi...
Sorry for crossing in...
On Wed, Aug 1, 2012 at 11:53 AM, Dhyan <linuxdhyan@gmail.com> wrote:
> Sorry to ask,but is there any other good way to find the written pages of a
> user process?
by "written", you mean "dirty" pages?
if yes, IIRC there's a flag that marks so....but that is for x86 and I
forgot the name...
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com
_______________________________________________
Kernelnewbies mailing list
Kernelnewbies at kernelnewbies.org
http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
--
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-01 6:17 ` Dhyan
@ 2012-08-01 8:42 ` Mulyadi Santosa
0 siblings, 0 replies; 13+ messages in thread
From: Mulyadi Santosa @ 2012-08-01 8:42 UTC (permalink / raw)
To: kernelnewbies
Hi....
On Wed, Aug 1, 2012 at 1:17 PM, Dhyan <linuxdhyan@gmail.com> wrote:
> Dear mulyadi santosa,
>
> I think you are mentioning the about PG_dirty flag.But in my case i want to
> clear the written flag of the user process from my kernel thread after each
> dump.
Oh, sorry then, I misunderstood your goal...
> So that next time when kernel thread runs it will dump only the pages
> written after last dump.
Sounds like what you need could be implemented using a time stamp.
Could be named like "last_written" and "last_dump". Both stores
jiffies values, consecutively when last time it is written out and
last time it is dumped out.
And since they are jiffies_t, they probably better attached to "page_t".
CMIIW...
--
regards,
Mulyadi Santosa
Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-01 4:53 ` Dhyan
2012-08-01 5:57 ` Mulyadi Santosa
@ 2012-08-02 8:42 ` bill4carson
2012-08-02 9:03 ` Dhyan
1 sibling, 1 reply; 13+ messages in thread
From: bill4carson @ 2012-08-02 8:42 UTC (permalink / raw)
To: kernelnewbies
On 2012?08?01? 12:53, Dhyan wrote:
> Dear Bill,
>
> Thank you for spending your valuable time for understanding and
> answering my queries !!!
>
> I was trying to apply some garbage collection algorithm on this dumped
> pages,thats why i want only the written pages.
>
> Sorry to ask,but is there any other good way to find the written pages
> of a user process?
Only the first wirte trigger page fault, which setup the page table by
grabbing a physical page to backup virtual address page. After this,
all write into that page goes like wind without any kernel interference
anymore.
But my hunch tell me what you want is to track every user space write
operation.
>
> --
> Thanks
> Dhyan
>
> On Wed, Aug 1, 2012 at 7:38 AM, bill4carson <bill4carson@gmail.com
> <mailto:bill4carson@gmail.com>> wrote:
>
>
>
> On 2012?07?31? 17:20, Dhyan wrote:
>
> Thank You Bill !!!
>
> I dont know my approach is correct or not,But the actual purpose
> was to
> dump only written pages of a user process using a kernel
> module.I have
> a kernel thread which will dump user process memory in specific
> interval.So i thought of updating this flag (L_PTE_DEBUG) from
> handle_pte_fault and clear from my clear thread so that i can
> dump only
> the written pages after my last dump.
>
>
> Yes, you can do that, only if your accounting memory doesn't get swapped
> out.
>
> If I understand correctly, when writing a page, you mark corresponding
> linux pte entry with L_PTE_DEBUG. Then your kernel module periodically
> loops all linux pte table, find pte entry marked with L_PTE_DEBUG.....
>
> I don't think it's wise to do so, you have 768 1st level pgd entries
> for user space, followed by 256 pte entries with each pgd entry.
> it's much slower to find out the right one.
>
> moreover, you probably need to remap those L_PTE_DEBUG physical pages
> into your own current process address space. IMHO, I don't follow
> such idea could be feasible.
>
>
>
> if you have some suggestions could you please share wth me?
>
>
> I understand how you plan to do this, could I ask why you need to dump
> the written pages?
>
>
> --
> Thanks
> Dhyan
>
> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson
> <bill4carson at gmail.com <mailto:bill4carson@gmail.com>
> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>>__>
> wrote:
>
>
>
> On 2012?07?30? 17:39, Dhyan wrote:
>
> Dear All,
>
> From linux(2.6.35) arm page table architecture i can
> see we
> have one
> hardware page table and there is corresponding Linux
> page table
> Entry
> (L_PTE_*).The "Linux" PTE definitions are as like below
> from
> arch/arm/include/asm/pgtable.____h.
>
>
> |#define L_PTE_PRESENT (1<< 0)
> #define L_PTE_FILE (1<< 1)
> #define L_PTE_YOUNG (1<< 1)
> #define L_PTE_BUFFERABLE(1<< 2)
> #define L_PTE_CACHEABLE (1<< 3)
> #define L_PTE_USER (1<< 4)
> #define L_PTE_WRITE (1<< 5)
> #define L_PTE_EXEC (1<< 6)
> #define L_PTE_DIRTY (1<< 7)
> #define L_PTE_COHERENT (1<< 9)
> #define L_PTE_SHARED (1<< 10)
> |
>
> So is it possible to add one more #|define L_PTE_DEBUG
> (1 <<
> 11)| for my
>
> debugging purpose (basically to trap all the write to
> that page
> and set
> this bit when write happens and clear it off in another
> thread
> )? Or
> is there any limitation like we can use only L_PTE till
> 10th bit ?
>
>
> No such limitation on bit 11, so you can use define
> L_PTE_DEBUG (1
> << 11)
> However I don't follow why you want to do so?
>
>
> So could you please help
>
> --
>
> Thanks & Regards
>
> Dhayn
>
>
>
> ___________________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.____org
> <mailto:Kernelnewbies@__kernelnewbies.org
> <mailto:Kernelnewbies@kernelnewbies.org>>
> http://lists.kernelnewbies.____org/mailman/listinfo/____kernelnewbies
> <http://lists.kernelnewbies.__org/mailman/listinfo/__kernelnewbies
> <http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>>
>
>
>
> --
> Love each day!
>
> --bill
>
>
>
> --
> Love each day!
>
> --bill
>
>
--
Love each day!
--bill
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-02 8:42 ` bill4carson
@ 2012-08-02 9:03 ` Dhyan
2012-08-02 9:28 ` bill4carson
0 siblings, 1 reply; 13+ messages in thread
From: Dhyan @ 2012-08-02 9:03 UTC (permalink / raw)
To: kernelnewbies
Dear Bill,
What i found from experimenting on arm Linux kernel is ,after every access
if i clear the _PG_ACCESS bit of the pte (using /proc/<pid>/clear_refs),the
next write also will come to kernel (handle_pte_fault).But I dont know
whether my clearing action causing any bad impact on any other system.
--
Thanks
Dhyan
On Thu, Aug 2, 2012 at 2:12 PM, bill4carson <bill4carson@gmail.com> wrote:
>
>
> On 2012?08?01? 12:53, Dhyan wrote:
>
>> Dear Bill,
>>
>> Thank you for spending your valuable time for understanding and
>> answering my queries !!!
>>
>> I was trying to apply some garbage collection algorithm on this dumped
>> pages,thats why i want only the written pages.
>>
>> Sorry to ask,but is there any other good way to find the written pages
>> of a user process?
>>
>
> Only the first wirte trigger page fault, which setup the page table by
> grabbing a physical page to backup virtual address page. After this,
> all write into that page goes like wind without any kernel interference
> anymore.
>
>
But my hunch tell me what you want is to track every user space write
> operation.
>
>
>
>> --
>> Thanks
>> Dhyan
>>
>> On Wed, Aug 1, 2012 at 7:38 AM, bill4carson <bill4carson@gmail.com
>> <mailto:bill4carson@gmail.com>**> wrote:
>>
>>
>>
>> On 2012?07?31? 17:20, Dhyan wrote:
>>
>> Thank You Bill !!!
>>
>> I dont know my approach is correct or not,But the actual purpose
>> was to
>> dump only written pages of a user process using a kernel
>> module.I have
>> a kernel thread which will dump user process memory in specific
>> interval.So i thought of updating this flag (L_PTE_DEBUG) from
>> handle_pte_fault and clear from my clear thread so that i can
>> dump only
>> the written pages after my last dump.
>>
>>
>> Yes, you can do that, only if your accounting memory doesn't get
>> swapped
>> out.
>>
>> If I understand correctly, when writing a page, you mark corresponding
>> linux pte entry with L_PTE_DEBUG. Then your kernel module periodically
>> loops all linux pte table, find pte entry marked with L_PTE_DEBUG.....
>>
>> I don't think it's wise to do so, you have 768 1st level pgd entries
>> for user space, followed by 256 pte entries with each pgd entry.
>> it's much slower to find out the right one.
>>
>> moreover, you probably need to remap those L_PTE_DEBUG physical pages
>> into your own current process address space. IMHO, I don't follow
>> such idea could be feasible.
>>
>>
>>
>> if you have some suggestions could you please share wth me?
>>
>>
>> I understand how you plan to do this, could I ask why you need to dump
>> the written pages?
>>
>>
>> --
>> Thanks
>> Dhyan
>>
>> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson
>> <bill4carson at gmail.com <mailto:bill4carson@gmail.com>
>> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>**
>> >__>
>>
>> wrote:
>>
>>
>>
>> On 2012?07?30? 17:39, Dhyan wrote:
>>
>> Dear All,
>>
>> From linux(2.6.35) arm page table architecture i can
>> see we
>> have one
>> hardware page table and there is corresponding Linux
>> page table
>> Entry
>> (L_PTE_*).The "Linux" PTE definitions are as like below
>> from
>> arch/arm/include/asm/pgtable._**___h.
>>
>>
>>
>> |#define L_PTE_PRESENT (1<< 0)
>> #define L_PTE_FILE (1<< 1)
>> #define L_PTE_YOUNG (1<< 1)
>> #define L_PTE_BUFFERABLE(1<< 2)
>> #define L_PTE_CACHEABLE (1<< 3)
>> #define L_PTE_USER (1<< 4)
>> #define L_PTE_WRITE (1<< 5)
>> #define L_PTE_EXEC (1<< 6)
>> #define L_PTE_DIRTY (1<< 7)
>> #define L_PTE_COHERENT (1<< 9)
>> #define L_PTE_SHARED (1<< 10)
>> |
>>
>> So is it possible to add one more #|define L_PTE_DEBUG
>> (1 <<
>> 11)| for my
>>
>> debugging purpose (basically to trap all the write to
>> that page
>> and set
>> this bit when write happens and clear it off in another
>> thread
>> )? Or
>> is there any limitation like we can use only L_PTE till
>> 10th bit ?
>>
>>
>> No such limitation on bit 11, so you can use define
>> L_PTE_DEBUG (1
>> << 11)
>> However I don't follow why you want to do so?
>>
>>
>> So could you please help
>>
>> --
>>
>> Thanks & Regards
>>
>> Dhayn
>>
>>
>>
>> ______________________________**_____________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.__**__org
>> <mailto:Kernelnewbies@__kernel**newbies.org<http://kernelnewbies.org>
>> <mailto:Kernelnewbies@**kernelnewbies.org<Kernelnewbies@kernelnewbies.org>
>> >>
>> http://lists.kernelnewbies.___**_org/mailman/listinfo/____**
>> kernelnewbies
>>
>> <http://lists.kernelnewbies.__**org/mailman/listinfo/__**
>> kernelnewbies
>> <http://lists.kernelnewbies.**org/mailman/listinfo/**
>> kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
>> >>
>>
>>
>>
>> --
>> Love each day!
>>
>> --bill
>>
>>
>>
>> --
>> Love each day!
>>
>> --bill
>>
>>
>>
> --
> Love each day!
>
> --bill
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120802/572c9b86/attachment-0001.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-02 9:03 ` Dhyan
@ 2012-08-02 9:28 ` bill4carson
2012-08-02 9:33 ` Dhyan
0 siblings, 1 reply; 13+ messages in thread
From: bill4carson @ 2012-08-02 9:28 UTC (permalink / raw)
To: kernelnewbies
On 2012?08?02? 17:03, Dhyan wrote:
> Dear Bill,
>
> What i found from experimenting on arm Linux kernel is ,after every
> access if i clear the _PG_ACCESS bit of the pte (using
> /proc/<pid>/clear_refs),the next write also will come to kernel
> (handle_pte_fault).But I dont know whether my clearing action causing
> any bad impact on any other system.
>
592 static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr,
593 unsigned long end, struct mm_walk *walk)
594 {
595 struct vm_area_struct *vma = walk->private;
596 pte_t *pte, ptent;
597 spinlock_t *ptl;
598 struct page *page;
599
600 split_huge_page_pmd(walk->mm, pmd);
601 if (pmd_trans_unstable(pmd))
602 return 0;
603
604 pte = pte_offset_map_lock(vma->vm_mm, pmd, addr, &ptl);
605 for (; addr != end; pte++, addr += PAGE_SIZE) {
606 ptent = *pte;
607 if (!pte_present(ptent))
608 continue;
609
610 page = vm_normal_page(vma, addr, ptent);
611 if (!page)
612 continue;
613
614 /* Clear accessed and referenced bits. */
615 ptep_test_and_clear_young(vma, addr, pte);
^^^^^^^^^^^^
For arm, it clear all the corresponding pte entries.
that's why you got second page fault.
Again, I don't understand your original purpose.
> --
> Thanks
> Dhyan
>
> On Thu, Aug 2, 2012 at 2:12 PM, bill4carson <bill4carson@gmail.com
> <mailto:bill4carson@gmail.com>> wrote:
>
>
>
> On 2012?08?01? 12:53, Dhyan wrote:
>
> Dear Bill,
>
> Thank you for spending your valuable time for understanding and
> answering my queries !!!
>
> I was trying to apply some garbage collection algorithm on this
> dumped
> pages,thats why i want only the written pages.
>
> Sorry to ask,but is there any other good way to find the written
> pages
> of a user process?
>
>
> Only the first wirte trigger page fault, which setup the page table by
> grabbing a physical page to backup virtual address page. After this,
> all write into that page goes like wind without any kernel interference
> anymore.
>
>
> But my hunch tell me what you want is to track every user space write
> operation.
>
>
>
> --
> Thanks
> Dhyan
>
> On Wed, Aug 1, 2012 at 7:38 AM, bill4carson
> <bill4carson at gmail.com <mailto:bill4carson@gmail.com>
> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>>__>
> wrote:
>
>
>
> On 2012?07?31? 17:20, Dhyan wrote:
>
> Thank You Bill !!!
>
> I dont know my approach is correct or not,But the
> actual purpose
> was to
> dump only written pages of a user process using a kernel
> module.I have
> a kernel thread which will dump user process memory in
> specific
> interval.So i thought of updating this flag
> (L_PTE_DEBUG) from
> handle_pte_fault and clear from my clear thread so that
> i can
> dump only
> the written pages after my last dump.
>
>
> Yes, you can do that, only if your accounting memory
> doesn't get swapped
> out.
>
> If I understand correctly, when writing a page, you mark
> corresponding
> linux pte entry with L_PTE_DEBUG. Then your kernel module
> periodically
> loops all linux pte table, find pte entry marked with
> L_PTE_DEBUG.....
>
> I don't think it's wise to do so, you have 768 1st level
> pgd entries
> for user space, followed by 256 pte entries with each pgd
> entry.
> it's much slower to find out the right one.
>
> moreover, you probably need to remap those L_PTE_DEBUG
> physical pages
> into your own current process address space. IMHO, I don't
> follow
> such idea could be feasible.
>
>
>
> if you have some suggestions could you please share
> wth me?
>
>
> I understand how you plan to do this, could I ask why you
> need to dump
> the written pages?
>
>
> --
> Thanks
> Dhyan
>
> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson
> <bill4carson at gmail.com <mailto:bill4carson@gmail.com>
> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>>
> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>
> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>>__>__>
>
> wrote:
>
>
>
> On 2012?07?30? 17:39, Dhyan wrote:
>
> Dear All,
>
> From linux(2.6.35) arm page table
> architecture i can
> see we
> have one
> hardware page table and there is
> corresponding Linux
> page table
> Entry
> (L_PTE_*).The "Linux" PTE definitions are as
> like below
> from
> arch/arm/include/asm/pgtable.______h.
>
>
>
> |#define L_PTE_PRESENT (1<< 0)
> #define L_PTE_FILE (1<< 1)
> #define L_PTE_YOUNG (1<< 1)
> #define L_PTE_BUFFERABLE(1<< 2)
> #define L_PTE_CACHEABLE (1<< 3)
> #define L_PTE_USER (1<< 4)
> #define L_PTE_WRITE (1<< 5)
> #define L_PTE_EXEC (1<< 6)
> #define L_PTE_DIRTY (1<< 7)
> #define L_PTE_COHERENT (1<< 9)
> #define L_PTE_SHARED (1<< 10)
> |
>
> So is it possible to add one more #|define
> L_PTE_DEBUG
> (1 <<
> 11)| for my
>
> debugging purpose (basically to trap all the
> write to
> that page
> and set
> this bit when write happens and clear it off
> in another
> thread
> )? Or
> is there any limitation like we can use only
> L_PTE till
> 10th bit ?
>
>
> No such limitation on bit 11, so you can use define
> L_PTE_DEBUG (1
> << 11)
> However I don't follow why you want to do so?
>
>
> So could you please help
>
> --
>
> Thanks & Regards
>
> Dhayn
>
>
>
>
> _____________________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.______org
> <mailto:Kernelnewbies@
> <mailto:Kernelnewbies@>__kernel__newbies.org
> <http://kernelnewbies.org>
> <mailto:Kernelnewbies@__kernelnewbies.org
> <mailto:Kernelnewbies@kernelnewbies.org>>>
> http://lists.kernelnewbies.______org/mailman/listinfo/______kernelnewbies
>
> <http://lists.kernelnewbies.____org/mailman/listinfo/____kernelnewbies
> <http://lists.kernelnewbies.__org/mailman/listinfo/__kernelnewbies
> <http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>>>
>
>
>
> --
> Love each day!
>
> --bill
>
>
>
> --
> Love each day!
>
> --bill
>
>
>
> --
> Love each day!
>
> --bill
>
>
--
Love each day!
--bill
^ permalink raw reply [flat|nested] 13+ messages in thread
* arm L_PTE_XXX entry addition for Debugging purpose
2012-08-02 9:28 ` bill4carson
@ 2012-08-02 9:33 ` Dhyan
0 siblings, 0 replies; 13+ messages in thread
From: Dhyan @ 2012-08-02 9:33 UTC (permalink / raw)
To: kernelnewbies
I will dump all the written pages of the application and run a garbage
collection algorithm over the dumped contents.
Thanks
Dhyan
On Thu, Aug 2, 2012 at 2:58 PM, bill4carson <bill4carson@gmail.com> wrote:
>
>
> On 2012?08?02? 17:03, Dhyan wrote:
>
>> Dear Bill,
>>
>> What i found from experimenting on arm Linux kernel is ,after every
>> access if i clear the _PG_ACCESS bit of the pte (using
>> /proc/<pid>/clear_refs),the next write also will come to kernel
>> (handle_pte_fault).But I dont know whether my clearing action causing
>> any bad impact on any other system.
>>
>>
> 592 static int clear_refs_pte_range(pmd_t *pmd, unsigned long addr,
> 593 unsigned long end, struct mm_walk *walk)
> 594 {
> 595 struct vm_area_struct *vma = walk->private;
> 596 pte_t *pte, ptent;
> 597 spinlock_t *ptl;
> 598 struct page *page;
> 599
> 600 split_huge_page_pmd(walk->mm, pmd);
> 601 if (pmd_trans_unstable(pmd))
> 602 return 0;
> 603
> 604 pte = pte_offset_map_lock(vma->vm_**mm, pmd, addr, &ptl);
> 605 for (; addr != end; pte++, addr += PAGE_SIZE) {
> 606 ptent = *pte;
> 607 if (!pte_present(ptent))
> 608 continue;
> 609
> 610 page = vm_normal_page(vma, addr, ptent);
> 611 if (!page)
> 612 continue;
> 613
> 614 /* Clear accessed and referenced bits. */
> 615 ptep_test_and_clear_young(vma, addr, pte);
> ^^^^^^^^^^^^
> For arm, it clear all the corresponding pte entries.
> that's why you got second page fault.
>
> Again, I don't understand your original purpose
>
>> --
>> Thanks
>> Dhyan
>>
>> On Thu, Aug 2, 2012 at 2:12 PM, bill4carson <bill4carson@gmail.com
>> <mailto:bill4carson@gmail.com>**> wrote:
>>
>>
>>
>> On 2012?08?01? 12:53, Dhyan wrote:
>>
>> Dear Bill,
>>
>> Thank you for spending your valuable time for understanding and
>> answering my queries !!!
>>
>> I was trying to apply some garbage collection algorithm on this
>> dumped
>> pages,thats why i want only the written pages.
>>
>> Sorry to ask,but is there any other good way to find the written
>> pages
>> of a user process?
>>
>>
>> Only the first wirte trigger page fault, which setup the page table by
>> grabbing a physical page to backup virtual address page. After this,
>> all write into that page goes like wind without any kernel
>> interference
>> anymore.
>>
>>
>> But my hunch tell me what you want is to track every user space write
>> operation.
>>
>>
>>
>> --
>> Thanks
>> Dhyan
>>
>> On Wed, Aug 1, 2012 at 7:38 AM, bill4carson
>> <bill4carson at gmail.com <mailto:bill4carson@gmail.com>
>> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>**
>> >__>
>> wrote:
>>
>>
>>
>> On 2012?07?31? 17:20, Dhyan wrote:
>>
>> Thank You Bill !!!
>>
>> I dont know my approach is correct or not,But the
>> actual purpose
>> was to
>> dump only written pages of a user process using a kernel
>> module.I have
>> a kernel thread which will dump user process memory in
>> specific
>> interval.So i thought of updating this flag
>> (L_PTE_DEBUG) from
>> handle_pte_fault and clear from my clear thread so that
>> i can
>> dump only
>> the written pages after my last dump.
>>
>>
>> Yes, you can do that, only if your accounting memory
>> doesn't get swapped
>> out.
>>
>> If I understand correctly, when writing a page, you mark
>> corresponding
>> linux pte entry with L_PTE_DEBUG. Then your kernel module
>> periodically
>> loops all linux pte table, find pte entry marked with
>> L_PTE_DEBUG.....
>>
>> I don't think it's wise to do so, you have 768 1st level
>> pgd entries
>> for user space, followed by 256 pte entries with each pgd
>> entry.
>> it's much slower to find out the right one.
>>
>> moreover, you probably need to remap those L_PTE_DEBUG
>> physical pages
>> into your own current process address space. IMHO, I don't
>> follow
>> such idea could be feasible.
>>
>>
>>
>> if you have some suggestions could you please share
>> wth me?
>>
>>
>> I understand how you plan to do this, could I ask why you
>> need to dump
>> the written pages?
>>
>>
>> --
>> Thanks
>> Dhyan
>>
>> On Tue, Jul 31, 2012 at 12:43 PM, bill4carson
>> <bill4carson at gmail.com <mailto:bill4carson@gmail.com>
>> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>**>
>> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>
>> <mailto:bill4carson at gmail.com <mailto:bill4carson@gmail.com>**
>> >__>__>
>>
>>
>> wrote:
>>
>>
>>
>> On 2012?07?30? 17:39, Dhyan wrote:
>>
>> Dear All,
>>
>> From linux(2.6.35) arm page table
>> architecture i can
>> see we
>> have one
>> hardware page table and there is
>> corresponding Linux
>> page table
>> Entry
>> (L_PTE_*).The "Linux" PTE definitions are as
>> like below
>> from
>> arch/arm/include/asm/pgtable._**_____h.
>>
>>
>>
>>
>> |#define L_PTE_PRESENT (1<< 0)
>> #define L_PTE_FILE (1<< 1)
>> #define L_PTE_YOUNG (1<< 1)
>> #define L_PTE_BUFFERABLE(1<< 2)
>> #define L_PTE_CACHEABLE (1<< 3)
>> #define L_PTE_USER (1<< 4)
>> #define L_PTE_WRITE (1<< 5)
>> #define L_PTE_EXEC (1<< 6)
>> #define L_PTE_DIRTY (1<< 7)
>> #define L_PTE_COHERENT (1<< 9)
>> #define L_PTE_SHARED (1<< 10)
>> |
>>
>> So is it possible to add one more #|define
>> L_PTE_DEBUG
>> (1 <<
>> 11)| for my
>>
>> debugging purpose (basically to trap all the
>> write to
>> that page
>> and set
>> this bit when write happens and clear it off
>> in another
>> thread
>> )? Or
>> is there any limitation like we can use only
>> L_PTE till
>> 10th bit ?
>>
>>
>> No such limitation on bit 11, so you can use define
>> L_PTE_DEBUG (1
>> << 11)
>> However I don't follow why you want to do so?
>>
>>
>> So could you please help
>>
>> --
>>
>> Thanks & Regards
>>
>> Dhayn
>>
>>
>>
>>
>> ______________________________**_______________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.__**____org
>> <mailto:Kernelnewbies@
>> <mailto:Kernelnewbies@>__kerne**l__newbies.org<http://kernel__newbies.org>
>> <http://kernelnewbies.org>
>> <mailto:Kernelnewbies@__kernel**newbies.org<http://kernelnewbies.org>
>> <mailto:Kernelnewbies@**kernelnewbies.org<Kernelnewbies@kernelnewbies.org>
>> >>>
>> http://lists.kernelnewbies.___**___org/mailman/listinfo/______**
>> kernelnewbies
>>
>>
>> <http://lists.kernelnewbies.__**__org/mailman/listinfo/____**
>> kernelnewbies
>> <http://lists.kernelnewbies.__**org/mailman/listinfo/__**
>> kernelnewbies
>> <http://lists.kernelnewbies.**org/mailman/listinfo/**
>> kernelnewbies<http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies>
>> >>>
>>
>>
>>
>> --
>> Love each day!
>>
>> --bill
>>
>>
>>
>> --
>> Love each day!
>>
>> --bill
>>
>>
>>
>> --
>> Love each day!
>>
>> --bill
>>
>>
>>
> --
> Love each day!
>
> --bill
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20120802/e505efe7/attachment-0001.html
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2012-08-02 9:33 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-07-30 9:39 arm L_PTE_XXX entry addition for Debugging purpose Dhyan
2012-07-31 7:13 ` bill4carson
2012-07-31 9:20 ` Dhyan
2012-08-01 2:08 ` bill4carson
2012-08-01 4:53 ` Dhyan
2012-08-01 5:57 ` Mulyadi Santosa
2012-08-01 6:17 ` Dhyan
2012-08-01 8:42 ` Mulyadi Santosa
2012-08-02 8:42 ` bill4carson
2012-08-02 9:03 ` Dhyan
2012-08-02 9:28 ` bill4carson
2012-08-02 9:33 ` Dhyan
-- strict thread matches above, loose matches on Subject: below --
2012-08-01 6:54 Vladimir Murzin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).