ckpt-v19-rc2

ckpt-v19-rc2

[Oren Laadan]

I put together ckpt-v19-rc2 (kernel and user) that includes:

* eclone
* INET sockets
* timerfd, signalfd
* rebase to 2.6.32-rc8
* c/r thread/cpu state earlier
* misc fixes

Hoping to give it some testing to clean remaining rough edges, and
in particular net-c/r patches whose folding wasn't trivial, as well
as the other archs.

On the todo-list are:

* assorted cleanup/fixes
* more cleanups on ckpt_err/ckpt_msg
* support for x86-64
* unlinked files (if ready ?)

Now back to work...

Oren.

[PATCH] checkpoint: remove clone_with_pids remnants from arch/powerpc

[Nathan Lynch]

We have eclone now so this stuff can go (and it doesn't build anyway).

Signed-off-by: Nathan Lynch <ntl-e+AXbWqSrlAAvxtiuMwx3w@public.gmane.org>
---
 arch/powerpc/include/asm/systbl.h |    1 -
 arch/powerpc/kernel/process.c     |   18 ------------------
 2 files changed, 0 insertions(+), 19 deletions(-)

diff --git a/arch/powerpc/include/asm/systbl.h b/arch/powerpc/include/asm/systbl.h
index 280a2b5..c374cb7 100644
--- a/arch/powerpc/include/asm/systbl.h
+++ b/arch/powerpc/include/asm/systbl.h
@@ -329,4 +329,3 @@ COMPAT_SYS(rt_tgsigqueueinfo)
 PPC_SYS(eclone)
 SYSCALL(checkpoint)
 SYSCALL(restart)
-PPC_SYS(clone_with_pids)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index b9278d8..c1900b3 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -900,24 +900,6 @@ int sys_clone(unsigned long clone_flags, unsigned long usp,
 	return do_fork(clone_flags, usp, regs, 0, parent_tidp, child_tidp);
 }
 
-int sys_clone_with_pids(unsigned long clone_flags, unsigned long usp,
-	      int __user *parent_tidp, void __user *child_threadptr,
-	      int __user *child_tidp, void __user *upid_setp,
-	      struct pt_regs *regs)
-{
-	CHECK_FULL_REGS(regs);
-	if (usp == 0)
-		usp = regs->gpr[1];
-#ifdef CONFIG_PPC64
-	if (test_thread_flag(TIF_32BIT)) {
-		parent_tidp = TRUNC_PTR(parent_tidp);
-		child_tidp = TRUNC_PTR(child_tidp);
-	}
-#endif
-	return do_fork_with_pids(clone_flags, usp, regs, 0,
-				 parent_tidp, child_tidp, upid_setp);
-}
-
 int sys_eclone(unsigned long clone_flags_low,
 	       struct clone_args __user *uclone_args,
 	       size_t size,
-- 
1.6.0.6

[PATCH] checkpoint: fix net/checkpoint.c for 64-bit

[Nathan Lynch]

When NET_SKBUFF_DATA_USES_OFFSET is defined:

net/checkpoint.c:100: error: invalid operands to binary - (have
'sk_buff_data_t' and 'unsigned char *')

net/checkpoint.c:106: error: 'struct sk_buff' has no member named
'transport_hdr'

Signed-off-by: Nathan Lynch <ntl-e+AXbWqSrlAAvxtiuMwx3w@public.gmane.org>
---
 net/checkpoint.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/net/checkpoint.c b/net/checkpoint.c
index 56bb026..c1c3e29 100644
--- a/net/checkpoint.c
+++ b/net/checkpoint.c
@@ -97,13 +97,13 @@ static void sock_record_header_info(struct sk_buff *skb,
 	h->mac_len = skb->mac_len;
 	h->skb_len = skb->len;
 	h->hdr_len = skb->data - skb->head;
-	h->lin_len = (skb->tail - skb->head);
+	h->lin_len = ((unsigned long)skb->tail - (unsigned long)skb->head);
 	h->frg_len = skb->data_len;
 
 	h->data_offset = (skb->data - skb->head);
 
 #ifdef NET_SKBUFF_DATA_USES_OFFSET
-	h->transport_header = skb->transport_hdr;
+	h->transport_header = skb->transport_header;
 	h->network_header = skb->network_header;
 	h->mac_header = skb->mac_header;
 #else
-- 
1.6.0.6

Re: ckpt-v19-rc2

[Nathan Lynch]

[-- Attachment #1: Type: text/plain, Size: 1214 bytes --]

On Wed, 2009-12-02 at 00:23 -0500, Oren Laadan wrote:
> I put together ckpt-v19-rc2 (kernel and user)

I'm not sure yet whether this is a regression, but checkpoint seems to
be unable to handle file mappings that extend past the end of the file.
I noticed this with Fedora 11 userspace on powerpc (ld.so sometimes maps
libraries this way).  I did not see this failure with a v19-rc1-based
kernel I tested earlier this week, but I haven't retested with that yet.

Here's an example - 8K mapping of a 4K file:

# stat -c%s /tmp/myfile 
4096

# grep myfile /proc/5164/maps
f7e2b000-f7e2d000 r--p 00000000 08:06 5103709                            /tmp/myfile

# checkpoint 5164 > /tmp/mmap.ckpt
checkpoint: Bad address

When we try to follow the mapping past the end of the file, we get
VM_FAULT_SIGBUS from handle_mm_fault(); the stack trace from debugging
code I added is:

.__get_dirty_page+0x4c/0x164 (unreliable)
.checkpoint_memory_contents+0x134/0x5a4
.private_vma_checkpoint+0xf4/0x120
.filemap_checkpoint+0x198/0x1d0
.checkpoint_mm+0x3c4/0x4fc
.checkpoint_obj+0x17c/0x1d0
.checkpoint_obj_mm+0x50/0x88
.checkpoint_task+0x710/0xaa0
.do_checkpoint+0x9c0/0xb24
.SyS_checkpoint+0xd0/0x11c

Attached is a testcase.


[-- Attachment #2: mmap.c --]
[-- Type: text/x-csrc, Size: 1635 bytes --]

#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>

static size_t page_sz;	/* system page size */
static size_t file_sz;	/* size of the file */
static size_t map_sz;	/* size of the mapping to establish */

static const char *filepath = "myfile";

static void setup(void)
{
	page_sz = sysconf(_SC_PAGESIZE);
	file_sz = page_sz;
	map_sz  = file_sz + page_sz;
}

static void bail_perror(const char *msg)
{
	perror(msg);
	exit(1);
}

static void bail_msg(const char *msg)
{
	fprintf(stderr, "%s\n", msg);
	exit(1);
}

int create_file(size_t sz)
{
	const void *buf;
	ssize_t written;
	struct stat sb;
	int fd;
	int rc;

	rc = unlink(filepath);
	if (rc < 0 && errno != ENOENT)
		bail_perror("unlink");

	fd = open(filepath, (O_RDWR | O_CREAT | O_EXCL), S_IRUSR);
	if (fd < 0)
		bail_perror("creat");

	buf = calloc(1, sz);
	if (!buf)
		bail_perror("calloc");

	written = 0;
	while (written < sz) {
		ssize_t res;
		/* unsigned char zero = '\0'; */

		res = write(fd, buf + written, sz - written);

		if (res < 0)
			bail_perror("write");

		written += res;
	}

	/* check size */
	rc = fstat(fd, &sb);
	if (rc < 0)
		bail_perror("fstat");

	if (sb.st_size != sz)
		bail_msg("unexpected file size");

	return fd;
}

void map_file(int fd, size_t map_sz)
{
	void *addr;

	addr = mmap(NULL, map_sz, PROT_READ, MAP_PRIVATE, fd, 0);
	if (addr == MAP_FAILED)
		bail_perror("mmap");
}

int main(int argc, char **argv)
{
	int fd;

	setup();

	fd = create_file(file_sz);
	map_file(fd, map_sz);

	printf("ready\n");
	pause();

	return 0;
}

[-- Attachment #3: Type: text/plain, Size: 206 bytes --]

_______________________________________________
Containers mailing list
Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

Re: ckpt-v19-rc2

[Oren Laadan]

Oh ... maybe that's what why checkpoint fails on my experimental
x86-64 port: it worked well on 2.6.31 but fails with "bad address"
on 2.6.32.

It may be related to a change between 2.6.31 and 2.6.32 in
the arguments to follow_page(), see commit:

	mm: FOLL_DUMP replace FOLL_ANON
	8e4b9a60718970bbc02dfd3abd0b956ab65af231

?

Oren.

Nathan Lynch wrote:
> On Wed, 2009-12-02 at 00:23 -0500, Oren Laadan wrote:
>> I put together ckpt-v19-rc2 (kernel and user)
> 
> I'm not sure yet whether this is a regression, but checkpoint seems to
> be unable to handle file mappings that extend past the end of the file.
> I noticed this with Fedora 11 userspace on powerpc (ld.so sometimes maps
> libraries this way).  I did not see this failure with a v19-rc1-based
> kernel I tested earlier this week, but I haven't retested with that yet.
> 
> Here's an example - 8K mapping of a 4K file:
> 
> # stat -c%s /tmp/myfile 
> 4096
> 
> # grep myfile /proc/5164/maps
> f7e2b000-f7e2d000 r--p 00000000 08:06 5103709                            /tmp/myfile
> 
> # checkpoint 5164 > /tmp/mmap.ckpt
> checkpoint: Bad address
> 
> When we try to follow the mapping past the end of the file, we get
> VM_FAULT_SIGBUS from handle_mm_fault(); the stack trace from debugging
> code I added is:
> 
> .__get_dirty_page+0x4c/0x164 (unreliable)
> .checkpoint_memory_contents+0x134/0x5a4
> .private_vma_checkpoint+0xf4/0x120
> .filemap_checkpoint+0x198/0x1d0
> .checkpoint_mm+0x3c4/0x4fc
> .checkpoint_obj+0x17c/0x1d0
> .checkpoint_obj_mm+0x50/0x88
> .checkpoint_task+0x710/0xaa0
> .do_checkpoint+0x9c0/0xb24
> .SyS_checkpoint+0xd0/0x11c
> 
> Attached is a testcase.
> 
> 

Re: ckpt-v19-rc2

[Oren Laadan]

Thinking about it further, here are two possible scenarios:

A) Task maps a file beyond it's limit, never touches those
extra page (if it did, it would get EFAULT)

B) Task maps a file and writes the last page, then the file gets
truncated (by at least a page).  The task may continue to access
that extra page since it already became anonymous.

In kernel 2.3.31, FOLL_ANON flags would make follow_page() return
the zero-page for case A. For case B, the actual page is returned.

In kernel 2.3.32, FOLL_ANON is gone. Instead FOLL_DUMP makes
follow_page() return -EFAULT in case A, which we can use to
conclude that the page is not interesting (unmodified). Case B
is handled the same as before.

So checkpoint should work.

Now restart: case A is easy, because mmap() works as before, and
those pages that were never touched will not be restored either,
they will remain untouched.

For case B it may be problematic, since we will start with a map
that goes beyond the current file size (originally it wasn't),
and we will attempt to modify a page in the "forbidden" zone.
I wonder if get_user_page() will respond favorably when restart
requests such a page ?   let's see ...

Oren.


Oren Laadan wrote:
> Oh ... maybe that's what why checkpoint fails on my experimental
> x86-64 port: it worked well on 2.6.31 but fails with "bad address"
> on 2.6.32.
> 
> It may be related to a change between 2.6.31 and 2.6.32 in
> the arguments to follow_page(), see commit:
> 
> 	mm: FOLL_DUMP replace FOLL_ANON
> 	8e4b9a60718970bbc02dfd3abd0b956ab65af231
> 
> ?
> 
> Oren.
> 
> Nathan Lynch wrote:
>> On Wed, 2009-12-02 at 00:23 -0500, Oren Laadan wrote:
>>> I put together ckpt-v19-rc2 (kernel and user)
>> I'm not sure yet whether this is a regression, but checkpoint seems to
>> be unable to handle file mappings that extend past the end of the file.
>> I noticed this with Fedora 11 userspace on powerpc (ld.so sometimes maps
>> libraries this way).  I did not see this failure with a v19-rc1-based
>> kernel I tested earlier this week, but I haven't retested with that yet.
>>
>> Here's an example - 8K mapping of a 4K file:
>>
>> # stat -c%s /tmp/myfile 
>> 4096
>>
>> # grep myfile /proc/5164/maps
>> f7e2b000-f7e2d000 r--p 00000000 08:06 5103709                            /tmp/myfile
>>
>> # checkpoint 5164 > /tmp/mmap.ckpt
>> checkpoint: Bad address
>>
>> When we try to follow the mapping past the end of the file, we get
>> VM_FAULT_SIGBUS from handle_mm_fault(); the stack trace from debugging
>> code I added is:
>>
>> .__get_dirty_page+0x4c/0x164 (unreliable)
>> .checkpoint_memory_contents+0x134/0x5a4
>> .private_vma_checkpoint+0xf4/0x120
>> .filemap_checkpoint+0x198/0x1d0
>> .checkpoint_mm+0x3c4/0x4fc
>> .checkpoint_obj+0x17c/0x1d0
>> .checkpoint_obj_mm+0x50/0x88
>> .checkpoint_task+0x710/0xaa0
>> .do_checkpoint+0x9c0/0xb24
>> .SyS_checkpoint+0xd0/0x11c
>>
>> Attached is a testcase.
>>
>>
> _______________________________________________
> Containers mailing list
> Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linux-foundation.org/mailman/listinfo/containers
> 

Re: ckpt-v19-rc2

[Serge E. Hallyn]

Quoting Oren Laadan (orenl-eQaUEPhvms7ENvBUuze7eA@public.gmane.org):
> 
> I put together ckpt-v19-rc2 (kernel and user) that includes:
> 
> * eclone
> * INET sockets
> * timerfd, signalfd
> * rebase to 2.6.32-rc8
> * c/r thread/cpu state earlier
> * misc fixes
> 
> Hoping to give it some testing to clean remaining rough edges, and
> in particular net-c/r patches whose folding wasn't trivial, as well
> as the other archs.
> 
> On the todo-list are:
> 
> * assorted cleanup/fixes
> * more cleanups on ckpt_err/ckpt_msg
> * support for x86-64
> * unlinked files (if ready ?)

Ok, I only just realized, through a segfault on restart, that
current->fs->pwd isn't being restored.  Were you thinking of
supporting that in v19?

If we don't want to do 'proper' current->fs c/r, I htink it's
worthwhile doing a cheap c/r of current->fs->pwd as a filename.

(I'd send the cheap patch, but am mostly out from now until wed)

thanks,
-serge