liblxc: lxc-debian

liblxc: lxc-debian

[Serge E. Hallyn]

Hi Daniel,

to create a debian-based container using lxc-debian on fedora 10,
I needed to just a couple of things:

	1. iptables -F   :)  Grrr.

	2. Right above the debootstrap command, I had to fool
	   chage (used during openssh configuration) into thinking
	   selinux was disabled.  So after the line:
	   mkdir -p "$CACHE/rootfs-$ARCH"
	   I added
	   mkdir -p "$CACHE/rootfs-$ARCH/selinux"
	   echo 0 > "$CACHE/rootfs-$ARCH/selinux/enforce"

	3. For the actual debootstrap command I had to do
	   debootstrap --arch $ARCH etc $CACHE/rootfs-$ARCH
	   Then apt-get install openssh-server and apache
	   worked fine.  But your debootstrap command failed
	   (the last time i tried) on chroot - no idea why.

Now it seems to work.  This shouldn't have taken me 2 hours to
figure out, but the symptoms were deceptive :)

thanks,
-serge

Re: liblxc: lxc-debian

[Daniel Lezcano]

Serge E. Hallyn wrote:
> Hi Daniel,
> 
> to create a debian-based container using lxc-debian on fedora 10,
> I needed to just a couple of things:
> 
> 	1. iptables -F   :)  Grrr.
> 
> 	2. Right above the debootstrap command, I had to fool
> 	   chage (used during openssh configuration) into thinking
> 	   selinux was disabled.  So after the line:
> 	   mkdir -p "$CACHE/rootfs-$ARCH"
> 	   I added
> 	   mkdir -p "$CACHE/rootfs-$ARCH/selinux"
> 	   echo 0 > "$CACHE/rootfs-$ARCH/selinux/enforce"

Good catch ! :)

> 	3. For the actual debootstrap command I had to do
> 	   debootstrap --arch $ARCH etc $CACHE/rootfs-$ARCH
> 	   Then apt-get install openssh-server and apache
> 	   worked fine.  But your debootstrap command failed
> 	   (the last time i tried) on chroot - no idea why.

Ok, I will try to figure out what is happening.

> Now it seems to work.  This shouldn't have taken me 2 hours to
> figure out, but the symptoms were deceptive :)

I have a some bugs reported I will fix with this one. I'll release a 
0.5.1 version soon.

Thanks a lot for taking the time to investigate :)

   -- Daniel

Re: liblxc: lxc-debian

[Serge E. Hallyn]

Quoting Daniel Lezcano (dlezcano-NmTC/0ZBporQT0dZR+AlfA@public.gmane.org):
> Serge E. Hallyn wrote:
>> Hi Daniel,
>>
>> to create a debian-based container using lxc-debian on fedora 10,
>> I needed to just a couple of things:
>>
>> 	1. iptables -F   :)  Grrr.
>>
>> 	2. Right above the debootstrap command, I had to fool
>> 	   chage (used during openssh configuration) into thinking
>> 	   selinux was disabled.  So after the line:
>> 	   mkdir -p "$CACHE/rootfs-$ARCH"
>> 	   I added
>> 	   mkdir -p "$CACHE/rootfs-$ARCH/selinux"
>> 	   echo 0 > "$CACHE/rootfs-$ARCH/selinux/enforce"
>
> Good catch ! :)

Are you going to put those lines into the 'official' lxc-debian?

>> 	3. For the actual debootstrap command I had to do
>> 	   debootstrap --arch $ARCH etc $CACHE/rootfs-$ARCH
>> 	   Then apt-get install openssh-server and apache
>> 	   worked fine.  But your debootstrap command failed
>> 	   (the last time i tried) on chroot - no idea why.
>
> Ok, I will try to figure out what is happening.

Great, thanks.

>> Now it seems to work.  This shouldn't have taken me 2 hours to
>> figure out, but the symptoms were deceptive :)
>
> I have a some bugs reported I will fix with this one. I'll release a  
> 0.5.1 version soon.
>
> Thanks a lot for taking the time to investigate :)

One more thing that would be helpful - can you think of an
easy way to specify devices whitelist rules for lxc-debian?
I don't want to complicate the creation process, but as it
is it's not trivial to define them.  Perhaps specifying a
default that should work for most everyone would be ok?  I
find the following to be plenty flexible:

(this is the code i inserted into the old lxc-debian command,
haven't checked if i need to change it for the new one)

    echo "lxc.cgroup.devices.deny = a" >> $CONFFILE
    # /dev/null and zero
    echo "lxc.cgroup.devices.allow = c 1:3 rwm" >> $CONFFILE
    echo "lxc.cgroup.devices.allow = c 1:5 rwm" >> $CONFFILE
    # consoles
    echo "lxc.cgroup.devices.allow = c 5:1 rwm" >> $CONFFILE
    echo "lxc.cgroup.devices.allow = c 5:0 rwm" >> $CONFFILE
    echo "lxc.cgroup.devices.allow = c 4:0 rwm" >> $CONFFILE
    echo "lxc.cgroup.devices.allow = c 4:1 rwm" >> $CONFFILE
    # /dev/{,u}random
    echo "lxc.cgroup.devices.allow = c 1:9 rwm" >> $CONFFILE
    echo "lxc.cgroup.devices.allow = c 1:8 rwm" >> $CONFFILE
    # /dev/pts/* - pts namespaces are "coming soon"
    echo "lxc.cgroup.devices.allow = c 136:* rwm" >> $CONFFILE
    # rtc
    echo "lxc.cgroup.devices.allow = c 254:0 rwm" >> $CONFFILE

thanks,
-serge

Re: liblxc: lxc-debian

[Matt Helsley]

On Fri, 2008-12-05 at 18:11 -0600, Serge E. Hallyn wrote:
> Quoting Daniel Lezcano (dlezcano-NmTC/0ZBporQT0dZR+AlfA@public.gmane.org):
> > Serge E. Hallyn wrote:
> >> Hi Daniel,
> >>
> >> to create a debian-based container using lxc-debian on fedora 10,
> >> I needed to just a couple of things:
> >>
> >> 	1. iptables -F   :)  Grrr.
> >>
> >> 	2. Right above the debootstrap command, I had to fool
> >> 	   chage (used during openssh configuration) into thinking
> >> 	   selinux was disabled.  So after the line:
> >> 	   mkdir -p "$CACHE/rootfs-$ARCH"
> >> 	   I added
> >> 	   mkdir -p "$CACHE/rootfs-$ARCH/selinux"
> >> 	   echo 0 > "$CACHE/rootfs-$ARCH/selinux/enforce"
> >
> > Good catch ! :)
> 
> Are you going to put those lines into the 'official' lxc-debian?
> 
> >> 	3. For the actual debootstrap command I had to do
> >> 	   debootstrap --arch $ARCH etc $CACHE/rootfs-$ARCH
> >> 	   Then apt-get install openssh-server and apache
> >> 	   worked fine.  But your debootstrap command failed
> >> 	   (the last time i tried) on chroot - no idea why.
> >
> > Ok, I will try to figure out what is happening.
> 
> Great, thanks.
> 
> >> Now it seems to work.  This shouldn't have taken me 2 hours to
> >> figure out, but the symptoms were deceptive :)
> >
> > I have a some bugs reported I will fix with this one. I'll release a  
> > 0.5.1 version soon.
> >
> > Thanks a lot for taking the time to investigate :)
> 
> One more thing that would be helpful - can you think of an
> easy way to specify devices whitelist rules for lxc-debian?
> I don't want to complicate the creation process, but as it
> is it's not trivial to define them.  Perhaps specifying a
> default that should work for most everyone would be ok?  I
> find the following to be plenty flexible:
> 
> (this is the code i inserted into the old lxc-debian command,
> haven't checked if i need to change it for the new one)
> 
>     echo "lxc.cgroup.devices.deny = a" >> $CONFFILE
>     # /dev/null and zero
>     echo "lxc.cgroup.devices.allow = c 1:3 rwm" >> $CONFFILE
>     echo "lxc.cgroup.devices.allow = c 1:5 rwm" >> $CONFFILE
>     # consoles
>     echo "lxc.cgroup.devices.allow = c 5:1 rwm" >> $CONFFILE
>     echo "lxc.cgroup.devices.allow = c 5:0 rwm" >> $CONFFILE
>     echo "lxc.cgroup.devices.allow = c 4:0 rwm" >> $CONFFILE
>     echo "lxc.cgroup.devices.allow = c 4:1 rwm" >> $CONFFILE
>     # /dev/{,u}random
>     echo "lxc.cgroup.devices.allow = c 1:9 rwm" >> $CONFFILE
>     echo "lxc.cgroup.devices.allow = c 1:8 rwm" >> $CONFFILE
>     # /dev/pts/* - pts namespaces are "coming soon"
>     echo "lxc.cgroup.devices.allow = c 136:* rwm" >> $CONFFILE
>     # rtc
>     echo "lxc.cgroup.devices.allow = c 254:0 rwm" >> $CONFFILE

This could be a tad prettier with a "here file" rather than all the
echoes and >> $CONFFILE:

        #
        # Write some reasonable default device whitelist rules
        #
        cat - >> $CONFFILE <<-"EOF"
        lxc.cgroup.devices.deny = a
        # /dev/null and zero
        lxc.cgroup.devices.allow = c 1:3 rwm
        lxc.cgroup.devices.allow = c 1:5 rwm
        # consoles
        lxc.cgroup.devices.allow = c 5:1 rwm
        lxc.cgroup.devices.allow = c 5:0 rwm
        lxc.cgroup.devices.allow = c 4:0 rwm
        lxc.cgroup.devices.allow = c 4:1 rwm
        # /dev/{,u}random
        lxc.cgroup.devices.allow = c 1:9 rwm
        lxc.cgroup.devices.allow = c 1:8 rwm
        # /dev/pts/* - pts namespaces are "coming soon"
        lxc.cgroup.devices.allow = c 136:* rwm
        # rtc 
        lxc.cgroup.devices.allow = c 254:0 rwm
        EOF

The quotes around EOF prevent bash from doing any substitution on the
file contents.

Cheers,
	-Matt

Re: liblxc: lxc-debian

[Serge E. Hallyn]

Quoting Matt Helsley (matthltc-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org):
> > (this is the code i inserted into the old lxc-debian command,
> > haven't checked if i need to change it for the new one)
> > 
> >     echo "lxc.cgroup.devices.deny = a" >> $CONFFILE
> >     # /dev/null and zero
> >     echo "lxc.cgroup.devices.allow = c 1:3 rwm" >> $CONFFILE
> >     echo "lxc.cgroup.devices.allow = c 1:5 rwm" >> $CONFFILE
> >     # consoles
> >     echo "lxc.cgroup.devices.allow = c 5:1 rwm" >> $CONFFILE
> >     echo "lxc.cgroup.devices.allow = c 5:0 rwm" >> $CONFFILE
> >     echo "lxc.cgroup.devices.allow = c 4:0 rwm" >> $CONFFILE
> >     echo "lxc.cgroup.devices.allow = c 4:1 rwm" >> $CONFFILE
> >     # /dev/{,u}random
> >     echo "lxc.cgroup.devices.allow = c 1:9 rwm" >> $CONFFILE
> >     echo "lxc.cgroup.devices.allow = c 1:8 rwm" >> $CONFFILE
> >     # /dev/pts/* - pts namespaces are "coming soon"
> >     echo "lxc.cgroup.devices.allow = c 136:* rwm" >> $CONFFILE
> >     # rtc
> >     echo "lxc.cgroup.devices.allow = c 254:0 rwm" >> $CONFFILE
> 
> This could be a tad prettier with a "here file" rather than all the
> echoes and >> $CONFFILE:

The new lxc-debian actually does

	cat > $CONFIGFILE << EOF

I pasted from my old version.

>         #
>         # Write some reasonable default device whitelist rules
>         #
>         cat - >> $CONFFILE <<-"EOF"
>         lxc.cgroup.devices.deny = a
>         # /dev/null and zero
>         lxc.cgroup.devices.allow = c 1:3 rwm
>         lxc.cgroup.devices.allow = c 1:5 rwm
>         # consoles
>         lxc.cgroup.devices.allow = c 5:1 rwm
>         lxc.cgroup.devices.allow = c 5:0 rwm
>         lxc.cgroup.devices.allow = c 4:0 rwm
>         lxc.cgroup.devices.allow = c 4:1 rwm
>         # /dev/{,u}random
>         lxc.cgroup.devices.allow = c 1:9 rwm
>         lxc.cgroup.devices.allow = c 1:8 rwm
>         # /dev/pts/* - pts namespaces are "coming soon"
>         lxc.cgroup.devices.allow = c 136:* rwm
>         # rtc 
>         lxc.cgroup.devices.allow = c 254:0 rwm
>         EOF
> 
> The quotes around EOF prevent bash from doing any substitution on the
> file contents.
> 
> Cheers,
> 	-Matt

Re: liblxc: lxc-debian

[Daniel Lezcano]

Serge E. Hallyn wrote:
> Quoting Matt Helsley (matthltc-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org):
>>> (this is the code i inserted into the old lxc-debian command,
>>> haven't checked if i need to change it for the new one)
>>>
>>>     echo "lxc.cgroup.devices.deny = a" >> $CONFFILE
>>>     # /dev/null and zero
>>>     echo "lxc.cgroup.devices.allow = c 1:3 rwm" >> $CONFFILE
>>>     echo "lxc.cgroup.devices.allow = c 1:5 rwm" >> $CONFFILE
>>>     # consoles
>>>     echo "lxc.cgroup.devices.allow = c 5:1 rwm" >> $CONFFILE
>>>     echo "lxc.cgroup.devices.allow = c 5:0 rwm" >> $CONFFILE
>>>     echo "lxc.cgroup.devices.allow = c 4:0 rwm" >> $CONFFILE
>>>     echo "lxc.cgroup.devices.allow = c 4:1 rwm" >> $CONFFILE
>>>     # /dev/{,u}random
>>>     echo "lxc.cgroup.devices.allow = c 1:9 rwm" >> $CONFFILE
>>>     echo "lxc.cgroup.devices.allow = c 1:8 rwm" >> $CONFFILE
>>>     # /dev/pts/* - pts namespaces are "coming soon"
>>>     echo "lxc.cgroup.devices.allow = c 136:* rwm" >> $CONFFILE
>>>     # rtc
>>>     echo "lxc.cgroup.devices.allow = c 254:0 rwm" >> $CONFFILE
>> This could be a tad prettier with a "here file" rather than all the
>> echoes and >> $CONFFILE:
> 
> The new lxc-debian actually does
> 
> 	cat > $CONFIGFILE << EOF
> 
> I pasted from my old version.
> 
>>         #
>>         # Write some reasonable default device whitelist rules
>>         #
>>         cat - >> $CONFFILE <<-"EOF"
>>         lxc.cgroup.devices.deny = a
>>         # /dev/null and zero
>>         lxc.cgroup.devices.allow = c 1:3 rwm
>>         lxc.cgroup.devices.allow = c 1:5 rwm
>>         # consoles
>>         lxc.cgroup.devices.allow = c 5:1 rwm
>>         lxc.cgroup.devices.allow = c 5:0 rwm
>>         lxc.cgroup.devices.allow = c 4:0 rwm
>>         lxc.cgroup.devices.allow = c 4:1 rwm
>>         # /dev/{,u}random
>>         lxc.cgroup.devices.allow = c 1:9 rwm
>>         lxc.cgroup.devices.allow = c 1:8 rwm
>>         # /dev/pts/* - pts namespaces are "coming soon"
>>         lxc.cgroup.devices.allow = c 136:* rwm
>>         # rtc 
>>         lxc.cgroup.devices.allow = c 254:0 rwm
>>         EOF
>>
>> The quotes around EOF prevent bash from doing any substitution on the
>> file contents.

I added these devices to the debian configuration file and fixed the 
cgroup list order, "lxc.cgroup.devices.deny = a" was the last entry :/

By default the debian has no root password, so the ssh connection will 
always fail until a password is set for root. I will look on how to 
change the root password to 'root' after debootstraping ...

I added "lxc.cgroup.devices.allow = c 5:2 rwm"
in order to use /dev/ptmx for the tty's ssh connection.

The container is no longer able to create /dev/initctl, so the poweroff 
command will fail. Serge do you know what is the syntax for the 
devices.allow for initctl ?

Thanks.
   -- Daniel

Re: liblxc: lxc-debian

[Serge E. Hallyn]

Quoting Daniel Lezcano (dlezcano-NmTC/0ZBporQT0dZR+AlfA@public.gmane.org):
> Serge E. Hallyn wrote:
>> Quoting Matt Helsley (matthltc-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org):
>>>         #
>>>         # Write some reasonable default device whitelist rules
>>>         #
>>>         cat - >> $CONFFILE <<-"EOF"
>>>         lxc.cgroup.devices.deny = a
>>>         # /dev/null and zero
>>>         lxc.cgroup.devices.allow = c 1:3 rwm
>>>         lxc.cgroup.devices.allow = c 1:5 rwm
>>>         # consoles
>>>         lxc.cgroup.devices.allow = c 5:1 rwm
>>>         lxc.cgroup.devices.allow = c 5:0 rwm
>>>         lxc.cgroup.devices.allow = c 4:0 rwm
>>>         lxc.cgroup.devices.allow = c 4:1 rwm
>>>         # /dev/{,u}random
>>>         lxc.cgroup.devices.allow = c 1:9 rwm
>>>         lxc.cgroup.devices.allow = c 1:8 rwm
>>>         # /dev/pts/* - pts namespaces are "coming soon"
>>>         lxc.cgroup.devices.allow = c 136:* rwm
>>>         # rtc         lxc.cgroup.devices.allow = c 254:0 rwm
>>>         EOF
>>>
>>> The quotes around EOF prevent bash from doing any substitution on the
>>> file contents.
>
> I added these devices to the debian configuration file and fixed the  
> cgroup list order, "lxc.cgroup.devices.deny = a" was the last entry :/

Weird.  It's the first now I hope :)

> By default the debian has no root password, so the ssh connection will  
> always fail until a password is set for root. I will look on how to  
> change the root password to 'root' after debootstraping ...
>
> I added "lxc.cgroup.devices.allow = c 5:2 rwm"
> in order to use /dev/ptmx for the tty's ssh connection.
>
> The container is no longer able to create /dev/initctl, so the poweroff  
> command will fail. Serge do you know what is the syntax for the  
> devices.allow for initctl ?

initctl isn't a device, it's a fifo.  At least on my laptop.

thanks,
-serge

Re: liblxc: lxc-debian

[Daniel Lezcano]

Serge E. Hallyn wrote:
> Quoting Daniel Lezcano (dlezcano-NmTC/0ZBporQT0dZR+AlfA@public.gmane.org):
>> Serge E. Hallyn wrote:
>>> Quoting Matt Helsley (matthltc-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org):
>>>>         #
>>>>         # Write some reasonable default device whitelist rules
>>>>         #
>>>>         cat - >> $CONFFILE <<-"EOF"
>>>>         lxc.cgroup.devices.deny = a
>>>>         # /dev/null and zero
>>>>         lxc.cgroup.devices.allow = c 1:3 rwm
>>>>         lxc.cgroup.devices.allow = c 1:5 rwm
>>>>         # consoles
>>>>         lxc.cgroup.devices.allow = c 5:1 rwm
>>>>         lxc.cgroup.devices.allow = c 5:0 rwm
>>>>         lxc.cgroup.devices.allow = c 4:0 rwm
>>>>         lxc.cgroup.devices.allow = c 4:1 rwm
>>>>         # /dev/{,u}random
>>>>         lxc.cgroup.devices.allow = c 1:9 rwm
>>>>         lxc.cgroup.devices.allow = c 1:8 rwm
>>>>         # /dev/pts/* - pts namespaces are "coming soon"
>>>>         lxc.cgroup.devices.allow = c 136:* rwm
>>>>         # rtc         lxc.cgroup.devices.allow = c 254:0 rwm
>>>>         EOF
>>>>
>>>> The quotes around EOF prevent bash from doing any substitution on the
>>>> file contents.
>> I added these devices to the debian configuration file and fixed the  
>> cgroup list order, "lxc.cgroup.devices.deny = a" was the last entry :/
> 
> Weird.  It's the first now I hope :)

I meant it was a bug of liblxc to store the cgroup in the wrong order :)

>> By default the debian has no root password, so the ssh connection will  
>> always fail until a password is set for root. I will look on how to  
>> change the root password to 'root' after debootstraping ...
>>
>> I added "lxc.cgroup.devices.allow = c 5:2 rwm"
>> in order to use /dev/ptmx for the tty's ssh connection.
>>
>> The container is no longer able to create /dev/initctl, so the poweroff  
>> command will fail. Serge do you know what is the syntax for the  
>> devices.allow for initctl ?
> 
> initctl isn't a device, it's a fifo.  At least on my laptop.

Yes, right.
"devices.deny = a" prohibits the creation of this fifo in /dev.

After doing "lxc-cgroup -n debian devices.allow a",
mknod -m 600 /dev/initctl p ,
succeed in the debian container.

(rm /dev/initctl)

But after doing "lxc-cgroup -n debian devices.deny a",
mknod -m 600 /dev/initctl p
mknod: `/dev/initctl': Operation not permitted

Is is a way to specify this fifo for devices.allow ?