Wiki for LXC with Checkpoint/Restart

Wiki for LXC with Checkpoint/Restart

[Sukadev Bhattiprolu]

Hi,

We created following wiki to describe the process to build/test LXC
with checkpoint/restart.

	https://ckpt.wiki.kernel.org/index.php/Link-LXC-USERCR

The USERCR and LXC patches that are needed are in:

        http://lxc.sourceforge.net/patches/lxc+usercr/usercr-patches/
        http://lxc.sourceforge.net/patches/lxc+usercr/lxc-patches/

(and also referred to in the above wiki).

For now, the wiki identifies specific "commit-id"s to identify the
commit with which the instructions in the wiki were tested. We hope
to create more meaningful tags to the commits and if so, we will update
the wiki with the tags.

Please let me know if you have any comments on the wiki or run into
problems. 

Thanks,

Sukadev

Re: Wiki for LXC with Checkpoint/Restart

[Nathan Lynch]

On Mon, 2010-04-12 at 15:27 -0700, Sukadev Bhattiprolu wrote:
> Hi,
> 
> We created following wiki to describe the process to build/test LXC
> with checkpoint/restart.
> 
> 	https://ckpt.wiki.kernel.org/index.php/Link-LXC-USERCR
> 
> The USERCR and LXC patches that are needed are in:
> 
>         http://lxc.sourceforge.net/patches/lxc+usercr/usercr-patches/
>         http://lxc.sourceforge.net/patches/lxc+usercr/lxc-patches/
> 
> (and also referred to in the above wiki).
> 
> For now, the wiki identifies specific "commit-id"s to identify the
> commit with which the instructions in the wiki were tested. We hope
> to create more meaningful tags to the commits and if so, we will update
> the wiki with the tags.

Commit IDs are ugly but unambiguous.  I didn't have a problem just
copy/pasting the commands.

> Please let me know if you have any comments on the wiki or run into
> problems. 

I took the liberty of cleaning up a few stray '|' characters in the last
code snippet and fixed up the deprecated old-style git invocations (e.g.
"git-clone" becomes "git clone").  Not trying to be picky about the git
thing, but the hyphenated commands don't work on newer distributions.

I ran into an issue trying to follow the instructions on a SLES11 x86_64
system.  When building lxc I get:

gcc -fPIC -DPIC -I../../src -DLIBCR -I/home/nathanl/user-cr
-I/home/nathanl/user-cr/include -g -O2 -Wall
-shared /home/nathanl/user-cr/checkpoint.o /home/nathanl/user-cr/restart.o /home/nathanl/user-cr/libeclone.a -Wl,-soname,liblxc.so.0  -o liblxc.so liblxc_so-arguments.o liblxc_so-commands.o liblxc_so-start.o liblxc_so-stop.o liblxc_so-monitor.o liblxc_so-console.o liblxc_so-freezer.o liblxc_so-checkpoint.o liblxc_so-restart.o liblxc_so-error.o liblxc_so-parse.o liblxc_so-cgroup.o liblxc_so-utils.o liblxc_so-namespace.o liblxc_so-conf.o liblxc_so-confile.o liblxc_so-state.o liblxc_so-log.o liblxc_so-network.o liblxc_so-nl.o liblxc_so-rtnl.o liblxc_so-genl.o liblxc_so-mainloop.o liblxc_so-af_unix.o -lutil
/usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: /home/nathanl/user-cr/checkpoint.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC

Not sure what to do about it immediately; user-cr/checkpoint.o should be
built with -fPIC I guess?

Re: Wiki for LXC with Checkpoint/Restart

[Nathan Lynch]

On Mon, 2010-04-12 at 15:27 -0700, Sukadev Bhattiprolu wrote:
> Hi,
> 
> We created following wiki to describe the process to build/test LXC
> with checkpoint/restart.
> 
> 	https://ckpt.wiki.kernel.org/index.php/Link-LXC-USERCR
> 
> The USERCR and LXC patches that are needed are in:
> 
>         http://lxc.sourceforge.net/patches/lxc+usercr/usercr-patches/
>         http://lxc.sourceforge.net/patches/lxc+usercr/lxc-patches/
> 
> (and also referred to in the above wiki).
> 
> For now, the wiki identifies specific "commit-id"s to identify the
> commit with which the instructions in the wiki were tested. We hope
> to create more meaningful tags to the commits and if so, we will update
> the wiki with the tags.
> 
> Please let me know if you have any comments on the wiki or run into
> problems.

When building the kernel with the commit id specified in the wiki
(0fdca57) I get:

ERROR: "ckpt_netdev_base" [net/ipv6/sit.ko] undefined!
ERROR: "ckpt_netdev_in_init_netns" [drivers/net/veth.ko] undefined!
ERROR: "ckpt_netdev_base" [drivers/net/veth.ko] undefined!
ERROR: "ckpt_netdev_base" [drivers/net/macvlan.ko] undefined!
make[1]: *** [__modpost] Error 1
make: *** [modules] Error 2

Re: Wiki for LXC with Checkpoint/Restart

[Dan Smith]

NL> When building the kernel with the commit id specified in the wiki
NL> (0fdca57) I get:

NL> ERROR: "ckpt_netdev_base" [net/ipv6/sit.ko] undefined!
NL> ERROR: "ckpt_netdev_in_init_netns" [drivers/net/veth.ko] undefined!
NL> ERROR: "ckpt_netdev_base" [drivers/net/veth.ko] undefined!
NL> ERROR: "ckpt_netdev_base" [drivers/net/macvlan.ko] undefined!
NL> make[1]: *** [__modpost] Error 1
NL> make: *** [modules] Error 2

Right and the fixes in my latest set should eliminate that :)

-- 
Dan Smith
IBM Linux Technology Center
email: danms-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org

Re: Wiki for LXC with Checkpoint/Restart

[Nathan Lynch]

On Wed, 2010-04-14 at 13:00 -0700, Dan Smith wrote:
> NL> When building the kernel with the commit id specified in the wiki
> NL> (0fdca57) I get:
> 
> NL> ERROR: "ckpt_netdev_base" [net/ipv6/sit.ko] undefined!
> NL> ERROR: "ckpt_netdev_in_init_netns" [drivers/net/veth.ko] undefined!
> NL> ERROR: "ckpt_netdev_base" [drivers/net/veth.ko] undefined!
> NL> ERROR: "ckpt_netdev_base" [drivers/net/macvlan.ko] undefined!
> NL> make[1]: *** [__modpost] Error 1
> NL> make: *** [modules] Error 2
> 
> Right and the fixes in my latest set should eliminate that :)
> 

I expected as much, but the question is how should we address this kind
of problem in the wiki?  Are the patches in a publicly available branch?

Re: Wiki for LXC with Checkpoint/Restart

[Dan Smith]

NL> Are the patches in a publicly available branch?

Not that I'm aware of, because they're wrapped in with some functional
changes and cleanups to the way the stuff is called.  I suppose I
could hack in some #ifdef's to the public bits to fix that, if
needed.

-- 
Dan Smith
IBM Linux Technology Center
email: danms-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org

Re: Wiki for LXC with Checkpoint/Restart

[Dan Smith]

SB> Do we run into the compile error only if some config tokens are selected ?
SB> Given that the instructions in the wiki don't use networking heavily,
SB> if we have a simple hack for the commit in the wiki, we could post a
SB> kernel patch to the lxc.sf.net directory.

Yeah, if you don't do CONFIG_IPV6=m you should be fine.

-- 
Dan Smith
IBM Linux Technology Center
email: danms-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org

Re: Wiki for LXC with Checkpoint/Restart

[Sukadev Bhattiprolu]

Nathan Lynch [ntl-e+AXbWqSrlAAvxtiuMwx3w@public.gmane.org] wrote:
| On Mon, 2010-04-12 at 15:27 -0700, Sukadev Bhattiprolu wrote:
| > Hi,
| > 
| > We created following wiki to describe the process to build/test LXC
| > with checkpoint/restart.
| > 
| > 	https://ckpt.wiki.kernel.org/index.php/Link-LXC-USERCR
| > 
| > The USERCR and LXC patches that are needed are in:
| > 
| >         http://lxc.sourceforge.net/patches/lxc+usercr/usercr-patches/
| >         http://lxc.sourceforge.net/patches/lxc+usercr/lxc-patches/
| > 
| > (and also referred to in the above wiki).
| > 
| > For now, the wiki identifies specific "commit-id"s to identify the
| > commit with which the instructions in the wiki were tested. We hope
| > to create more meaningful tags to the commits and if so, we will update
| > the wiki with the tags.
| 
| Commit IDs are ugly but unambiguous.  I didn't have a problem just
| copy/pasting the commands.
| 
| > Please let me know if you have any comments on the wiki or run into
| > problems. 
| 
| I took the liberty of cleaning up a few stray '|' characters in the last
| code snippet and fixed up the deprecated old-style git invocations (e.g.
| "git-clone" becomes "git clone").  Not trying to be picky about the git
| thing, but the hyphenated commands don't work on newer distributions.
| 
| I ran into an issue trying to follow the instructions on a SLES11 x86_64
| system.  When building lxc I get:
| 
| gcc -fPIC -DPIC -I../../src -DLIBCR -I/home/nathanl/user-cr
| -I/home/nathanl/user-cr/include -g -O2 -Wall
| -shared /home/nathanl/user-cr/checkpoint.o /home/nathanl/user-cr/restart.o /home/nathanl/user-cr/libeclone.a -Wl,-soname,liblxc.so.0  -o liblxc.so liblxc_so-arguments.o liblxc_so-commands.o liblxc_so-start.o liblxc_so-stop.o liblxc_so-monitor.o liblxc_so-console.o liblxc_so-freezer.o liblxc_so-checkpoint.o liblxc_so-restart.o liblxc_so-error.o liblxc_so-parse.o liblxc_so-cgroup.o liblxc_so-utils.o liblxc_so-namespace.o liblxc_so-conf.o liblxc_so-confile.o liblxc_so-state.o liblxc_so-log.o liblxc_so-network.o liblxc_so-nl.o liblxc_so-rtnl.o liblxc_so-genl.o liblxc_so-mainloop.o liblxc_so-af_unix.o -lutil
| /usr/lib64/gcc/x86_64-suse-linux/4.3/../../../../x86_64-suse-linux/bin/ld: /home/nathanl/user-cr/checkpoint.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
| 
| Not sure what to do about it immediately; user-cr/checkpoint.o should be
| built with -fPIC I guess?

Yes, I think thats what Michel or Daniel ran into. For some reason, I
did not run into on Fedora11. Can you add -fPIC to the usercr CFLAGS
and try ? If it works for you, I will try on Fedora11 too and we can
add a patch to lxc.sf.net/patches/lxc+usercr/.

We may need to do something similar for any kernel patches that are
needed on top of each "release" of the wiki.

Sukadev

Re: Wiki for LXC with Checkpoint/Restart

[Sukadev Bhattiprolu]

Dan Smith [danms-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org] wrote:
| NL> Are the patches in a publicly available branch?
| 
| Not that I'm aware of, because they're wrapped in with some functional
| changes and cleanups to the way the stuff is called.  I suppose I
| could hack in some #ifdef's to the public bits to fix that, if
| needed.

Do we run into the compile error only if some config tokens are selected ?
Given that the instructions in the wiki don't use networking heavily,
if we have a simple hack for the commit in the wiki, we could post a
kernel patch to the lxc.sf.net directory.
| 
| -- 
| Dan Smith
| IBM Linux Technology Center
| email: danms-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org

Re: Wiki for LXC with Checkpoint/Restart

[Nathan Lynch]

On Wed, 2010-04-14 at 13:40 -0700, Sukadev Bhattiprolu wrote:
> 
> Yes, I think thats what Michel or Daniel ran into. For some reason, I
> did not run into on Fedora11. Can you add -fPIC to the usercr CFLAGS
> and try ? If it works for you, I will try on Fedora11 too and we can
> add a patch to lxc.sf.net/patches/lxc+usercr/.

Yes, adding -fPIC to CFLAGS in the user-cr Makefile and rebuilding
allows the lxc build to succeed.

Re: Wiki for LXC with Checkpoint/Restart

[Nathan Lynch]

On Wed, 2010-04-14 at 15:55 -0500, Nathan Lynch wrote:
> On Wed, 2010-04-14 at 13:40 -0700, Sukadev Bhattiprolu wrote:
> > 
> > Yes, I think thats what Michel or Daniel ran into. For some reason, I
> > did not run into on Fedora11. Can you add -fPIC to the usercr CFLAGS
> > and try ? If it works for you, I will try on Fedora11 too and we can
> > add a patch to lxc.sf.net/patches/lxc+usercr/.
> 
> Yes, adding -fPIC to CFLAGS in the user-cr Makefile and rebuilding
> allows the lxc build to succeed.

Okay, so after getting the kernel build issues worked around and
documented, I've gotten to part 4 "Checkpoint/restart a simple LXC
container" in the wiki.  It works!  A couple of minor issues with the
instructions:

> $ lxc-execute --name foo --rcfile lxc-no-netns.conf -- /bin/sleep 1000

The full path to the rcfile needs to be supplied or lxc-execute errors
out.  Also, it seems that the above command needs its own terminal or it
should be run in the background.


> $ lxc-checkpoint --name foo --statefile /root/lxc-foo.ckpt

Is it intentional that this fails if /root/lxc-foo.ckpt already exists?