Build scatterlist covering a process' text segment?

Build scatterlist covering a process' text segment?

[Arvid Brodin]

Hi,

How do I build a scatterlist covering the text segment of a process? I have
found (struct task_struct).mm->start_code and ->end_code, that gives the address
(in some - unknown to me - address space) of the text segment. Since my try to
convert these addresses to an sglist:

	sg_init_one(&sg, task->mm->start_code, <length>);

failed miserably, I'm guessing these addresses need to be converted to some
address space compatible with sg_init_one()/sg_set_buf() before use? (I probably
need to search for pages containing these addresses?)


Thanks,
Arvid Brodin
Enea Services Stockholm AB

Build scatterlist covering a process' text segment?

[Arvid Brodin]

Arvid Brodin wrote:
> Hi,
> 
> How do I build a scatterlist covering the text segment of a process? I have
> found (struct task_struct).mm->start_code and ->end_code, that gives the address
> (in some - unknown to me - address space) of the text segment. Since my try to
> convert these addresses to an sglist:
> 
> 	sg_init_one(&sg, task->mm->start_code, <length>);
> 
> failed miserably, I'm guessing these addresses need to be converted to some
> address space compatible with sg_init_one()/sg_set_buf() before use? (I probably
> need to search for pages containing these addresses?)
> 

Am I correct in that the addresses in (struct task_struct).mm->start_code and
->end_code belong to the address space of the process whose task_struct I'm looking
at?

-- 
Arvid Brodin
Enea Services Stockholm AB

Build scatterlist covering a process' text segment?

[Mulyadi Santosa]

On 26/05/2011, Arvid Brodin <arvid.brodin@enea.com> wrote:
> Am I correct in that the addresses in (struct task_struct).mm->start_code
> and
> ->end_code belong to the address space of the process whose task_struct I'm
> looking
> at?


I believe yes....

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Build scatterlist covering a process' text segment?

[Arvid Brodin]

Mulyadi Santosa wrote:
> On 26/05/2011, Arvid Brodin <arvid.brodin@enea.com> wrote:
>> Am I correct in that the addresses in (struct task_struct).mm->start_code
>> and
>> ->end_code belong to the address space of the process whose task_struct I'm
>> looking
>> at?
> 
> 
> I believe yes....
> 

Ok. And looking at e.g. sg_set_buf(), the scatterlist expects a kernel virtual
address (it uses virt_to_page() on its "buf" parameter internally, which
requires a kernel virtual adress, if I understand correctly).

There seems to be no way to map process adresses to kernel addresses. (Well I 
guess one could follow the page tables to get the physical page, and then map
back to kernel space, but this only works as long as the memory is paged in.)
Please correct me if I'm wrong.

Perhaps a better idea is to do_mmap() the "exe_file", (struct mm_struct).exe_file?
I'm still confused about this though:

1) do_mmap() returns unsigned long - not the void * expected by sg_set_buf().
   Is this a sign that I'm again mixing different address spaces, or is it ok
   just to cast this?

2) I believe do_mmap() maps a file into the address space of the "current process",
   and that this address space is also the one used in kernel space at any moment.
   Is this correct, or is there an additional, kernel-only memory space that I
   should map the file to instead? If so, how do I do that?


Thanks,
Arvid Brodin
Enea Services Stockholm AB

Build scatterlist covering a process' text segment?

[Mulyadi Santosa]

Hi...

On Sat, May 28, 2011 at 04:58, Arvid Brodin <arvid.brodin@enea.com> wrote:
> Ok. And looking at e.g. sg_set_buf(), the scatterlist expects a kernel virtual
> address (it uses virt_to_page() on its "buf" parameter internally, which
> requires a kernel virtual adress, if I understand correctly).
>
> There seems to be no way to map process adresses to kernel addresses. (Well I
> guess one could follow the page tables to get the physical page, and then map
> back to kernel space, but this only works as long as the memory is paged in.)
> Please correct me if I'm wrong.

I am not good at it, but I think at the first place, you might use
get_user_pages() (take a look here
:http://lxr.linux.no/#linux+v2.6.39/mm/memory.c#L1703)

then once you get the pointer to the  pages (and making sure they're
pinned by get_user_pages), I think you just need to use kmap().

I suggest to really observe that scatter gather function and see if
address in kernel address space is really needed.... if not, you can
avoid using kmap() completely.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Build scatterlist covering a process' text segment?

[Arvid Brodin]

Mulyadi Santosa wrote:
> Hi...
> 
> On Sat, May 28, 2011 at 04:58, Arvid Brodin <arvid.brodin@enea.com> wrote:
>> Ok. And looking at e.g. sg_set_buf(), the scatterlist expects a kernel virtual
>> address (it uses virt_to_page() on its "buf" parameter internally, which
>> requires a kernel virtual adress, if I understand correctly).
>>
>> There seems to be no way to map process adresses to kernel addresses. (Well I
>> guess one could follow the page tables to get the physical page, and then map
>> back to kernel space, but this only works as long as the memory is paged in.)
>> Please correct me if I'm wrong.
> 
> I am not good at it, but I think at the first place, you might use
> get_user_pages() (take a look here
> :http://lxr.linux.no/#linux+v2.6.39/mm/memory.c#L1703)
> 
> then once you get the pointer to the  pages (and making sure they're
> pinned by get_user_pages), I think you just need to use kmap().
> 
> I suggest to really observe that scatter gather function and see if
> address in kernel address space is really needed.... if not, you can
> avoid using kmap() completely.
> 

Many thanks for that tip! I've been reading LDD3 ("Linux Device Drivers 3rd
edition") and "Understanding the Linux Kernel" as well as searching google and
of course reading code for many hours, and nowhere has there been a mention of
get_user_pages()! Obviously I haven't been using the right search terms...

(Also, for some reason your mail didn't reach my inbox, so I only saw it a few
days ago when looking at a mail list archive.) 

Anyway, the code looks like this now (below), and seems to work on 2.7.37.6. It
will probably have problems on systems with > 896 MiB and high memory enabled
though; I'm thinking the scatterlist functions won't handle virtual kernel
addresses?

Any ideas for improvements are welcome; especially if I'm doing something
stupid that risks a kernel panic.


static int proc_pid_text_checksum(struct seq_file *m, struct pid_namespace *ns,
				struct pid *pid, struct task_struct *task)
{
	int retval;

	int text_size;
	int nr_pages, nr_pages_mapped;
	int i;
	struct page **pages;

	struct scatterlist *sgl, *sg;

	u8 result[MD5_DIGEST_SIZE + 2];
	struct crypto_hash *tfm;
	struct hash_desc desc;

	retval = 0;

	if (!task->mm)
		return -EINVAL;

	text_size = task->mm->end_code - task->mm->start_code;
	nr_pages = (text_size + PAGE_SIZE - 1) >> PAGE_SHIFT;


	/**** User page code ****/

	pages = kmalloc(nr_pages * sizeof(*pages), GFP_KERNEL);
	if (!pages) {
		retval = -ENOMEM;
		goto err_pages;
	}

	down_read(&task->mm->mmap_sem);
	nr_pages_mapped = get_user_pages(current, task->mm,
			task->mm->start_code, nr_pages, 0, 0, pages, NULL);
	up_read(&task->mm->mmap_sem);

	if (nr_pages_mapped < nr_pages) {
		retval = -EBUSY; /* Weird error code for this,
		                    couldn't find any better */
		goto err_mapped;
	}


	/**** Scatterlist code ****/

	sgl = kmalloc(nr_pages_mapped * sizeof(*sgl), GFP_KERNEL);
	if (!sgl) {
		retval = -ENOMEM;
		goto err_sg;
	}

	sg_init_table(sgl, nr_pages_mapped);

	for_each_sg(sgl, sg, nr_pages_mapped, i)
		sg_set_page(sg, pages[i], (i < nr_pages_mapped) ? PAGE_SIZE :
						text_size & ~PAGE_MASK, 0);


	/**** Crypto code ****/

	tfm = crypto_alloc_hash("md5", 0, CRYPTO_ALG_ASYNC);
	if (IS_ERR(tfm)) {
		retval = -ENOMEM;
		goto err_crypto;
	}

	desc.tfm = tfm;
	desc.flags = CRYPTO_TFM_REQ_MAY_SLEEP;

	memset(result, 0, MD5_DIGEST_SIZE + 2);
	retval = crypto_hash_digest(&desc, sgl, text_size, result);
	if (retval)
		goto err_digest;

	for (i = 0; i < MD5_DIGEST_SIZE; i++)
		seq_printf(m, "%02x", result[i]);
	seq_printf(m, "\n");


err_digest:
	crypto_free_hash(tfm);

err_crypto:
	kfree(sgl);
	for (i = 0; i < nr_pages_mapped; i++)
		put_page(pages[i]);

err_sg:
err_mapped:
	kfree(pages);

err_pages:
	return retval;
}


-- 
Arvid Brodin
Enea Services Stockholm AB

Build scatterlist covering a process' text segment?

[Jonathan Neuschäfer]

On Fri, Jun 17, 2011 at 02:47:25AM +0200, Arvid Brodin wrote:
[...]
> 	/**** User page code ****/
> 
> 	pages = kmalloc(nr_pages * sizeof(*pages), GFP_KERNEL);

kcalloc?

You don't need the zeroing but the overflow check might be useful.

> 	if (!pages) {
> 		retval = -ENOMEM;
> 		goto err_pages;
> 	}

Greetings,
	Jonathan Neusch?fer

Build scatterlist covering a process' text segment?

[Metin KAYA]

Jonathan, it's kmalloc(), not kcalloc().

On 17 Haz 2011, at 18:10, Jonathan Neusch?fer <j.neuschaefer@gmx.net> wrote:

> On Fri, Jun 17, 2011 at 02:47:25AM +0200, Arvid Brodin wrote:
> [...]
>>    /**** User page code ****/
>> 
>>    pages = kmalloc(nr_pages * sizeof(*pages), GFP_KERNEL);
> 
> kcalloc?
> 
> You don't need the zeroing but the overflow check might be useful.
> 
>>    if (!pages) {
>>        retval = -ENOMEM;
>>        goto err_pages;
>>    }
> 
> Greetings,
>    Jonathan Neusch?fer
> 
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Build scatterlist covering a process' text segment?

[Dave Hylands]

Hi,

On Fri, Jun 17, 2011 at 8:18 AM, Metin KAYA <kayameti@gmail.com> wrote:
> Jonathan, it's kmalloc(), not kcalloc().

The kernel's kmalloc doesn't initialize the memory. There is a
function called kzalloc which will allocate and zero the memory, which
is sort of similar to the user-space calloc function, although calloc
takes num-of-items and size-of-one-item, whereas kzalloc takes
"size-of-allocation".

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Build scatterlist covering a process' text segment?

[Jonathan Neuschäfer]

On Fri, Jun 17, 2011 at 06:18:06PM +0300, Metin KAYA wrote:
> Jonathan, it's kmalloc(), not kcalloc().

No, I was trying to suggest using kcalloc(), which behaves roughly like
the userspace calloc, and implements overflow checking, which you may
need and zeroing, which you don't seem to need.

(see include/linux/slab.h, line 223 for reference)

> 
> On 17 Haz 2011, at 18:10, Jonathan Neusch?fer <j.neuschaefer@gmx.net> wrote:
> 
> > On Fri, Jun 17, 2011 at 02:47:25AM +0200, Arvid Brodin wrote:
> > [...]
> >>    /**** User page code ****/
> >> 
> >>    pages = kmalloc(nr_pages * sizeof(*pages), GFP_KERNEL);
> > 
> > kcalloc?
> > 
> > You don't need the zeroing but the overflow check might be useful.
> > 
> >>    if (!pages) {
> >>        retval = -ENOMEM;
> >>        goto err_pages;
> >>    }
> > 
> > Greetings,
> >    Jonathan Neusch?fer

HTH,
	Jonathan Neusch?fer

Build scatterlist covering a process' text segment?

[Dave Hylands]

Hey Jonathan,

On Fri, Jun 17, 2011 at 11:46 AM, Jonathan Neusch?fer
<j.neuschaefer@gmx.net> wrote:
> On Fri, Jun 17, 2011 at 06:18:06PM +0300, Metin KAYA wrote:
>> Jonathan, it's kmalloc(), not kcalloc().
>
> No, I was trying to suggest using kcalloc(), which behaves roughly like
> the userspace calloc, and implements overflow checking, which you may
> need and zeroing, which you don't seem to need.

Cool - I missed that somehow. So we now have kmalloc, kcalloc, and
kzalloc, and I see krealloc too.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Build scatterlist covering a process' text segment?

[Jonathan Neuschäfer]

On Fri, Jun 17, 2011 at 08:46:35PM +0200, Jonathan Neusch?fer wrote:
> On Fri, Jun 17, 2011 at 06:18:06PM +0300, Metin KAYA wrote:
> > Jonathan, it's kmalloc(), not kcalloc().
> 
> No, I was trying to suggest using kcalloc(), which behaves roughly like
> the userspace calloc, and implements overflow checking, which you may
> need and zeroing, which you don't seem to need.

Arvid, not you, sorry.

Jonathan Neusch?fer