loop-aes encrypted root on Fedora 15 using systemd

loop-aes encrypted root on Fedora 15 using systemd

[Frederick Gazerblezeebe]

I am trying to set up a Fedora 15 system (kernel 2.6.39) on which the
root filesystem is encrypted with loop-aes (v3.6c).

Can someone suggest the best location to place the initial losetup
commands? This new version uses the systemd boot stuff and
/etc/rc.d/rc.sysinit (where I placed the losetup commands before) no
longer exists. I was partially successful placing them in
/lib/systemd/fedora-readonly, meaning losetup successfully sets up the
loop device, but attempts to actually mount a file system on this loop
fail and the boot process terminates. I think I may be placing the
losetup too late in the sequence and the system is attempting to mount
on the loop before it is actually set up, but that's just a guess at
this point.

Any suggestions on how I might proceed would be greatly appreciated.

Thanks.

FG

Re: loop-aes encrypted root on Fedora 15 using systemd

[Jari Ruusu]

Frederick Gazerblezeebe wrote:
> I am trying to set up a Fedora 15 system (kernel 2.6.39) on which the
> root filesystem is encrypted with loop-aes (v3.6c).
> 
> Can someone suggest the best location to place the initial losetup
> commands? This new version uses the systemd boot stuff and
> /etc/rc.d/rc.sysinit (where I placed the losetup commands before) no
> longer exists. I was partially successful placing them in
> /lib/systemd/fedora-readonly, meaning losetup successfully sets up the
> loop device, but attempts to actually mount a file system on this loop
> fail and the boot process terminates. I think I may be placing the
> losetup too late in the sequence and the system is attempting to mount
> on the loop before it is actually set up, but that's just a guess at
> this point.

I am assuming you used loop-AES' build-initrd.sh script. I changed the
script so that it can be configured to set up more loop devices than the one
used by encrypted root partition. For example, if you want to set up
"no password entering required" /dev/loop6 and /dev/loop4, you can add these
lines to build-initrd.sh config:

EXTRACOMMANDRUN1=1
EXTRACOMMANDSTR1="/sbin/losetup -e AES128 -P /etc/cleartextkey-loop6.txt /dev/loop6 /dev/sda3"
EXTRACOMMANDRUN2=1
EXTRACOMMANDSTR2="/sbin/losetup -e AES128 -P /etc/cleartextkey-loop4.txt /dev/loop4 /dev/sdd12"

Limitations/rules:
1) Commands are run after switching to encrypted root but before starting
   /sbin/init
2) Encrypted root partition is mounted read-only. Other file systems are not
   mounted.
3) udev is not running yet, so dynamic device nodes on tmpfs (created and
   mounted by udev on top of /dev directory) are not available.
4) If commands need /dev/* device nodes, you must make sure that static
   device nodes exist on encrypted root partition on /dev directory. Use
   mknod program to create those device nodes. Above example would need 4
   nodes: /dev/loop6, /dev/sda3, /dev/loop4 and /dev/sdd12. The tricky part
   is putting them on the directory that is under udev mounted tmpfs file
   system.
5) /etc/cleartextkey-loop*.txt files (or whatever) on encrypted root
   partition are protected by root partition encryption. Each of these files
   contain 65 lines of key material that would normally be wrapped by and
   protected by gpg encryption.
6) Up to 8 extra commands can be configured. If you need more, make it run a
   shell script somewhere on encrypted root partition.

New version of the build-initrd.sh script is here:
http://loop-aes.sourceforge.net/updates/build-initrd.sh-20110531.bz2
http://loop-aes.sourceforge.net/updates/build-initrd.sh-20110531.bz2.sign

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

Re: loop-aes encrypted root on Fedora 15 using systemd

[Frederick Gazerblezeebe]

Hey Jari, thanks for the quick response and my apologies for taking so
long to get back to you.  Just as I started testing your new script my
hard drive failed, and I had to get a new one and reinstall fedora 15.
 So anyway, I am finally back to where I was when I first posted.

I've now built the initrd with your new script and I'm in the process
of encrypting the root partition as we speak.  I will post again once
it finishes and I've had a chance to test it.

Thanks again!

FG


On Tue, May 31, 2011 at 5:59 AM, Jari Ruusu
<jariruusu@users.sourceforge.net> wrote:
> Frederick Gazerblezeebe wrote:
>> I am trying to set up a Fedora 15 system (kernel 2.6.39) on which the
>> root filesystem is encrypted with loop-aes (v3.6c).
>>
>> Can someone suggest the best location to place the initial losetup
>> commands? This new version uses the systemd boot stuff and
>> /etc/rc.d/rc.sysinit (where I placed the losetup commands before) no
>> longer exists. I was partially successful placing them in
>> /lib/systemd/fedora-readonly, meaning losetup successfully sets up the
>> loop device, but attempts to actually mount a file system on this loop
>> fail and the boot process terminates. I think I may be placing the
>> losetup too late in the sequence and the system is attempting to mount
>> on the loop before it is actually set up, but that's just a guess at
>> this point.
>
> I am assuming you used loop-AES' build-initrd.sh script. I changed the
> script so that it can be configured to set up more loop devices than the one
> used by encrypted root partition. For example, if you want to set up
> "no password entering required" /dev/loop6 and /dev/loop4, you can add these
> lines to build-initrd.sh config:
>
> EXTRACOMMANDRUN1=1
> EXTRACOMMANDSTR1="/sbin/losetup -e AES128 -P /etc/cleartextkey-loop6.txt /dev/loop6 /dev/sda3"
> EXTRACOMMANDRUN2=1
> EXTRACOMMANDSTR2="/sbin/losetup -e AES128 -P /etc/cleartextkey-loop4.txt /dev/loop4 /dev/sdd12"
>
> Limitations/rules:
> 1) Commands are run after switching to encrypted root but before starting
>   /sbin/init
> 2) Encrypted root partition is mounted read-only. Other file systems are not
>   mounted.
> 3) udev is not running yet, so dynamic device nodes on tmpfs (created and
>   mounted by udev on top of /dev directory) are not available.
> 4) If commands need /dev/* device nodes, you must make sure that static
>   device nodes exist on encrypted root partition on /dev directory. Use
>   mknod program to create those device nodes. Above example would need 4
>   nodes: /dev/loop6, /dev/sda3, /dev/loop4 and /dev/sdd12. The tricky part
>   is putting them on the directory that is under udev mounted tmpfs file
>   system.
> 5) /etc/cleartextkey-loop*.txt files (or whatever) on encrypted root
>   partition are protected by root partition encryption. Each of these files
>   contain 65 lines of key material that would normally be wrapped by and
>   protected by gpg encryption.
> 6) Up to 8 extra commands can be configured. If you need more, make it run a
>   shell script somewhere on encrypted root partition.
>
> New version of the build-initrd.sh script is here:
> http://loop-aes.sourceforge.net/updates/build-initrd.sh-20110531.bz2
> http://loop-aes.sourceforge.net/updates/build-initrd.sh-20110531.bz2.sign
>
> --
> Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD
>

Re: loop-aes encrypted root on Fedora 15 using systemd

[Frederick Gazerblezeebe]

On Tue, May 31, 2011 at 5:59 AM, Jari Ruusu
<jariruusu@users.sourceforge.net> wrote:
> Frederick Gazerblezeebe wrote:
>> I am trying to set up a Fedora 15 system (kernel 2.6.39) on which the
>> root filesystem is encrypted with loop-aes (v3.6c).
>>
>> Can someone suggest the best location to place the initial losetup
>> commands? This new version uses the systemd boot stuff and
>> /etc/rc.d/rc.sysinit (where I placed the losetup commands before) no
>> longer exists. I was partially successful placing them in
>> /lib/systemd/fedora-readonly, meaning losetup successfully sets up the
>> loop device, but attempts to actually mount a file system on this loop
>> fail and the boot process terminates. I think I may be placing the
>> losetup too late in the sequence and the system is attempting to mount
>> on the loop before it is actually set up, but that's just a guess at
>> this point.
>
> I am assuming you used loop-AES' build-initrd.sh script. I changed the
> script so that it can be configured to set up more loop devices than the one
> used by encrypted root partition. For example, if you want to set up
> "no password entering required" /dev/loop6 and /dev/loop4, you can add these
> lines to build-initrd.sh config:
>
> EXTRACOMMANDRUN1=1
> EXTRACOMMANDSTR1="/sbin/losetup -e AES128 -P /etc/cleartextkey-loop6.txt /dev/loop6 /dev/sda3"
> EXTRACOMMANDRUN2=1
> EXTRACOMMANDSTR2="/sbin/losetup -e AES128 -P /etc/cleartextkey-loop4.txt /dev/loop4 /dev/sdd12"
>
> Limitations/rules:
> 1) Commands are run after switching to encrypted root but before starting
>   /sbin/init
> 2) Encrypted root partition is mounted read-only. Other file systems are not
>   mounted.
> 3) udev is not running yet, so dynamic device nodes on tmpfs (created and
>   mounted by udev on top of /dev directory) are not available.
> 4) If commands need /dev/* device nodes, you must make sure that static
>   device nodes exist on encrypted root partition on /dev directory. Use
>   mknod program to create those device nodes. Above example would need 4
>   nodes: /dev/loop6, /dev/sda3, /dev/loop4 and /dev/sdd12. The tricky part
>   is putting them on the directory that is under udev mounted tmpfs file
>   system.
> 5) /etc/cleartextkey-loop*.txt files (or whatever) on encrypted root
>   partition are protected by root partition encryption. Each of these files
>   contain 65 lines of key material that would normally be wrapped by and
>   protected by gpg encryption.
> 6) Up to 8 extra commands can be configured. If you need more, make it run a
>   shell script somewhere on encrypted root partition.
>
> New version of the build-initrd.sh script is here:
> http://loop-aes.sourceforge.net/updates/build-initrd.sh-20110531.bz2
> http://loop-aes.sourceforge.net/updates/build-initrd.sh-20110531.bz2.sign
>
> --
> Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD
>

I am currently up on the encrypted root with a couple of issues, which
may not be problems with loop-aes.

Using the EXTRACOMMANDRUN# options in your new script I am able to get
the loop device for /home initialized, as shown by the output of
losetup -a (loop2=/, loop3=home),

/dev/loop2: [0001]:5099 (/dev/sda2) encryption=AES128 multi-key-v3
/dev/loop3: [0702]:2104244 (/dev/sda3) encryption=AES128 multi-key-v3

but systemd is unable to mount it to /home as defined in fstab,

/dev/loop3      /home      ext4    defaults        0 2    #/dev/sda3

resulting in a failure to boot. At first I thought this was a selinux
problem, but further testing shows that this is not likely the case.
Next I need to go through the systemd scripts and find exactly where
it is failing and see what I can do about it.

One additional peculiarity is that although the swap is activated at
boot time, it is not  encrypted until I remove/add it again.  The
fstab entry is

/dev/sda5       swap       swap    sw,loop=/dev/loop5,encryption=AES128   0 0

Immediately after boot:

###[102]% losetup -a
/dev/loop2: [0001]:5099 (/dev/sda2) encryption=AES128 multi-key-v3
/dev/loop3: [0702]:2104244 (/dev/sda3) encryption=AES128 multi-key-v3
###[103]% swapon -s
Filename                                Type            Size    Used    Priority
/dev/sda5                               partition       6136824 0       0
###[104]% swapoff -a
###[105]% swapon -a
Setting up swapspace version 1, size = 6136820 KiB
no label, UUID=4f1b6b95-bd99-4ac5-aee6-b87d599b1f5c
###[107]% losetup -a
/dev/loop2: [0001]:5099 (/dev/sda2) encryption=AES128 multi-key-v3
/dev/loop3: [0702]:2104244 (/dev/sda3) encryption=AES128 multi-key-v3
/dev/loop5: [0005]:5244 (/dev/sda5) offset=4096 encryption=AES128 multi-key-v3
###[109]% swapon -s
Filename                                Type            Size    Used    Priority
/dev/loop5                              partition       6136820 0       -1

Once again I think this is a systemd issue as opposed to a loop-aes
problem and I am currently looking into it as well.

So thanks again for the quick script update, it really helped, and
I'll post back again when I've made some more progress.

FG

Re: loop-aes encrypted root on Fedora 15 using systemd

[Jari Ruusu]

Frederick Gazerblezeebe wrote:
> /dev/loop2: [0001]:5099 (/dev/sda2) encryption=AES128 multi-key-v3
> /dev/loop3: [0702]:2104244 (/dev/sda3) encryption=AES128 multi-key-v3
> 
> but systemd is unable to mount it to /home as defined in fstab,
> 
> /dev/loop3      /home      ext4    defaults        0 2    #/dev/sda3

Does it work if you remove #/dev/sda3 text at the end of the line? Strict
reading of fstab(5) man page says that lines that begin with # are comments.
Mount program fstab parser code seems to be happy with extra stuff at end of
line, but other fstab parser implementations are not necessarily ok with
that.

> One additional peculiarity is that although the swap is activated at
> boot time, it is not  encrypted until I remove/add it again.  The
> fstab entry is
> 
> /dev/sda5       swap       swap    sw,loop=/dev/loop5,encryption=AES128   0 0

That sounds like systemd is not using swapon program to enable swap. Does it
work if you set it up with non-changing encryption keys? As in,
build-initrd.sh sets it up, and /etc/fstab line is:

/dev/loop5  swap  swap  sw  0  0

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

Re: loop-aes encrypted root on Fedora 15 using systemd

[Frederick Gazerblezeebe]

On Fri, Jun 3, 2011 at 10:38 AM, Jari Ruusu
<jariruusu@users.sourceforge.net> wrote:
> Frederick Gazerblezeebe wrote:
>> /dev/loop2: [0001]:5099 (/dev/sda2) encryption=AES128 multi-key-v3
>> /dev/loop3: [0702]:2104244 (/dev/sda3) encryption=AES128 multi-key-v3
>>
>> but systemd is unable to mount it to /home as defined in fstab,
>>
>> /dev/loop3      /home      ext4    defaults        0 2    #/dev/sda3
>
> Does it work if you remove #/dev/sda3 text at the end of the line? Strict
> reading of fstab(5) man page says that lines that begin with # are comments.
> Mount program fstab parser code seems to be happy with extra stuff at end of
> line, but other fstab parser implementations are not necessarily ok with
> that.
>
Removing the comment at the end of the fstab entry had no effect;
booting is aborted leaving the system in rescue mode.  Looking at the
console output I found the following:

Starting /home aborted because a dependency failed.
systemd: job dev-loop3.device/start failed with result 'timeout'

I am investigating this right now...


>> One additional peculiarity is that although the swap is activated at
>> boot time, it is not  encrypted until I remove/add it again.  The
>> fstab entry is
>>
>> /dev/sda5       swap       swap    sw,loop=/dev/loop5,encryption=AES128   0 0
>
> That sounds like systemd is not using swapon program to enable swap. Does it
> work if you set it up with non-changing encryption keys? As in,
> build-initrd.sh sets it up, and /etc/fstab line is:
>
> /dev/loop5  swap  swap  sw  0  0
>
Changing the fstab entry per your suggestion fails to add any swap:

XXX[101]% swapon -s
Filename                                Type            Size    Used    Priority
XXX[102]% swapoff -a
XXX[103]% swapon -a
swapon: /dev/loop5: read swap header failed: Invalid argument

I then tried creating a static device node /dev/loop5 (I had to create
a static device node for /dev/loop3 (/home) before the
EXTRACOMMANDSTR1 for losetup would run), but the behavior was the
same.

I did find some error messages the syslog that are relevant:

Jun  3 13:17:49 mars systemd[1]: Unit systemd-tmpfiles-setup.service
entered failed state.
Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.device/start timed out.
Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.swap/start failed with
result 'dependency'.
Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.device/start failed
with result 'timeout'.

I'm looking in the systemd scripts right now to see if I can track
down the source of this error.  Unfortunately, a number of these
'scripts' are actually binaries and therefore essentially blackboxes
to me.  I'm hoping that the problem resides in one of the actual
scripts...

FG

Re: loop-aes encrypted root on Fedora 15 using systemd

[Jari Ruusu]

Frederick Gazerblezeebe wrote:
> Starting /home aborted because a dependency failed.
> systemd: job dev-loop3.device/start failed with result 'timeout'
[snip]
> Changing the fstab entry per your suggestion fails to add any swap:
> 
> XXX[101]% swapon -s
> Filename                                Type            Size    Used    Priority
> XXX[102]% swapoff -a
> XXX[103]% swapon -a
> swapon: /dev/loop5: read swap header failed: Invalid argument

If you use non-changing encryption keys for swap, you have to "format" the
device as swap (this needs to be done only once):

    swapoff -a
    mkswap /dev/loop5
    swapon -a

If /dev/loop5 encryption keys are ever changed, then mkswap has to be run
again. (When swapon program sets up random swap encryption keys, it runs
mkswap automatically)

> Jun  3 13:17:49 mars systemd[1]: Unit systemd-tmpfiles-setup.service
> entered failed state.
> Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.device/start timed out.
> Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.swap/start failed with
> result 'dependency'.
> Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.device/start failed
> with result 'timeout'.

That sounds like systemd is waiting for devices to be created. Does it help
if you configure udev to copy loop device nodes to /dev directory on boot?

    mknod -m 660 /lib/udev/devices/loop0 b 7 0
    mknod -m 660 /lib/udev/devices/loop1 b 7 1
    mknod -m 660 /lib/udev/devices/loop2 b 7 2
    mknod -m 660 /lib/udev/devices/loop3 b 7 3
    mknod -m 660 /lib/udev/devices/loop4 b 7 4
    mknod -m 660 /lib/udev/devices/loop5 b 7 5
    mknod -m 660 /lib/udev/devices/loop6 b 7 6
    mknod -m 660 /lib/udev/devices/loop7 b 7 7

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

Re: loop-aes encrypted root on Fedora 15 using systemd

[Frederick Gazerblezeebe]

On Sat, Jun 4, 2011 at 12:27 AM, Jari Ruusu
<jariruusu@users.sourceforge.net> wrote:
> Frederick Gazerblezeebe wrote:
>> Starting /home aborted because a dependency failed.
>> systemd: job dev-loop3.device/start failed with result 'timeout'
> [snip]
>> Changing the fstab entry per your suggestion fails to add any swap:
>>
>> XXX[101]% swapon -s
>> Filename                                Type            Size    Used    Priority
>> XXX[102]% swapoff -a
>> XXX[103]% swapon -a
>> swapon: /dev/loop5: read swap header failed: Invalid argument
>
> If you use non-changing encryption keys for swap, you have to "format" the
> device as swap (this needs to be done only once):
>
>    swapoff -a
>    mkswap /dev/loop5
>    swapon -a
>
> If /dev/loop5 encryption keys are ever changed, then mkswap has to be run
> again. (When swapon program sets up random swap encryption keys, it runs
> mkswap automatically)
>

Doh, I knew this; must have been tired when I tried it..  So,
initializing the swap correctly, the behavior remains the same: The
boot process stalls (but does not abort, resuming after a minute or so
delay) with the same error as before (Unit
systemd-tmpfiles-setup.service entered failed state; Job
dev-loop5.swap/start failed with result 'dependency';
 Job dev-loop5.device/start failed with result 'timeout'.)

>> Jun  3 13:17:49 mars systemd[1]: Unit systemd-tmpfiles-setup.service
>>  .
>> Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.device/start timed out.
>> Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.swap/start failed with
>> result 'dependency'.
>> Jun  3 13:19:02 mars systemd[1]: Job dev-loop5.device/start failed
>> with result 'timeout'.
>
> That sounds like systemd is waiting for devices to be created. Does it help
> if you configure udev to copy loop device nodes to /dev directory on boot?
>
>    mknod -m 660 /lib/udev/devices/loop0 b 7 0
>    mknod -m 660 /lib/udev/devices/loop1 b 7 1
>    mknod -m 660 /lib/udev/devices/loop2 b 7 2
>    mknod -m 660 /lib/udev/devices/loop3 b 7 3
>    mknod -m 660 /lib/udev/devices/loop4 b 7 4
>    mknod -m 660 /lib/udev/devices/loop5 b 7 5
>    mknod -m 660 /lib/udev/devices/loop6 b 7 6
>    mknod -m 660 /lib/udev/devices/loop7 b 7 7
>
The loop nodes are already present /lib/udev, so that is not the
source of the (mis)behavior. I had to create static links in /dev for
loop2 and sda2, loop3 and sda3, and loop5 and sda5 (root, home and
swap, respectively), in order to get them initialized and show up (via
losetup -a) after booting. At the present time, root is the only
partition whose loop device is successfully mounted during boot.  The
others (swap, home) have to be mounted to after booting has completed.

Next I will post a summary of what I have done and where I am,
including the behaviors we are discussing here, in reply to the
original topic.

...and again, thanks for all the help!

FG

Re: loop-aes encrypted root on Fedora 15 using systemd

[Frederick Gazerblezeebe]

I now have Fedora 15 (with selinux enabled) up and running from a
loop-aes-encrypted root partition. In case it can prove useful to
someone else running Fedora 15 (and/or some other systemd based linux)
I will describe what I have done and the status of my issues here.

The details (WARNING: LONG!!)...

Motherboard: Foxconn Mars (Intel p35 chipset)
Processor: Core2Duo E8400
OS: Fedora 15 with selinux enabled
kernel: linux-2.6.39 (kernel.org)
loop-aes-v3.6c
aespipe-v2.4c
util-linux-2.19.1 (kernel.org)
util-linux-2.19.1-20110510.diff
build-initrd.sh-20110531

After installing Fedora, the new kernel was downloaded and configured
per the loop-aes README. Installation of loop-aes and util-linux
proceeded without issue and I was able to mount and access my
(previously created) encrypted devices. The swap partition was also
encrypted at this point.

Moving on to encrypting the root partition...

Initially I was unable to boot into the encrypted root with the
build-initrd.sh found in the loop-aes-v3.6c archive, but this was
resolved very quickly with the modified script provided by Jari (thank
you Jari!). (In retrospect, I can't be certain that the original
script wouldn't have worked, because my hard disk was in the process
of failing and that may have been part, if not all, of the problem; I
have not gone back to check.)

The configuration options for build-initrd.sh-20110531 were:

USEDEVFS=0
USEPIVOT=2
BOOTDEV=/dev/sda1
BOOTTYPE=ext4
CRYPTROOT=/dev/sda2
ROOTTYPE=ext4
CIPHERTYPE=AES128
LOINIT="-I 0"
USEGPGKEY=1
GPGKEYFILE=key.gpg
EXTERNALGPGFILES=0
EXTERNALGPGDEV=/dev/fd0
EXTERNALGPGTYPE=ext2
USEMODULE=1
INITRDONLY=0
SOURCEROOT=
DESTINATIONROOT=
DESTINATIONPREFIX=/boot
INITRDGZNAME=initrd-loopaes.gz
ROOTLOOPINDEX=2
TEMPLOOPINDEX=7
LOOPMODPARAMS="max_loop=16"
UTF8KEYBMODE=0
LOADNATIONALKEYB=0
INITMD=1
INITMDDELAY=1
INITIALDELAY=0
MOUNTDELAY=0
TOOLSPROMPT=0
USEROOTSETUP=0
USEDIETLIBC=1
GCC=gcc
EXTRACOMMANDSTR1=1
EXTRACOMMANDSTR1="/sbin/losetup -e AES128 -P hkey.txt /dev/loop3 /dev/sda3"
EXTRACOMMANDRUN2=1
EXTRACOMMANDSTR2="/sbin/losetup -e AES128 -P skey.txt /dev/loop5 /dev/sda5"


The grub boot entry is:

title Fedora 15 encrypted (2.6.39)
        root (hd0,0)
        kernel /vmlinuz-2.6.39
        initrd /initrd-loopaes.gz

NOTE: The vmlinuz-2.6.39 had to be a file, not a symbolic link as set
up by the kernel 'make install' routine, or grub would fail.

The fstab entries are:

/dev/loop2      /          ext4    defaults        0 1
/dev/loop3      /home      ext4    defaults        0 2
/dev/sda5       swap       swap    sw,loop=/dev/loop5,encryption=AES128   0 0

In order to boot successfully I also had to create static nodes for a
number of devices in /dev:

mknod -m 660 /dev/tty c 5 0
mknod -m 660 /dev/console c 5 1
mknod -m 660 /dev/sda2 b 8 2       ## encrypted root
mknod -m 660 /dev/sda3 b 8 3       ## encrypted home
mknod -m 660 /dev/loop2 b 7 2
mknod -m 660 /dev/loop3 b 7 3


I also added these static nodes at some point, but can't recall when
or if they were actually required to boot successfully:

mknod -m 666 /mnt/dev/null c 1 3
mknod -m 666 /mnt/dev/zero c 1 5

I was able to boot successfully at this point, with the issue that the
swap partition was being added, but did not appear to be encrypted (as
shown by no entry for /dev/loop5 using losetup -a). This could be
corrected after boot by issuing the commands 'swapoff -a' and 'swapon
-a'.

In an attempt to address this I added static nodes to /dev

mknod -m 660 /dev/sda5 b 8 5       ## encrypted swap
mknod -m 660 /dev/loop5 b 7 5

...but this did not help. (My further attempts to troubleshoot this
with Jari's help are omitted here as they are covered in another part
of this discussion.)

I then attempted to encrypt and mount /home.  The following static
nodes were added to /dev:

mknod -m 660 /dev/sda3 b 8 3      ## encrypted /home
mknod -m 660 /dev/loop3 b 7 3

Like the swap partition, the loop device is properly losetup'd at
boot, but attempts to mount to it timeout during boot and are
unsuccessful.

So, the remaining issues I am attempting to resolve are:

1) Devices losetup'd at boot, even within the initrd built by Jari's
new script, do not get mounted by the Fedora init scripts. This
requires me to manually add swap and mount /home before logging in. I
would like to figure this out, but it is really only a minor annoyance
as I only reboot occassionally.

2) Once I have losetup'ed and mount'ed a loop device (defined in
/etc/fstab), umount'ing it fails to detach the loop device, and
attempts to 'losetup -d' it manually fail with the error 'ioctl:
LOOP_CLR_FD: Device or resource busy'.

3) Although selinux is otherwise running/behaving properly,
autorelabeling (of /home, /dev, /sys /run) fails at boot time; the
system does boot successfully though.

4) Rebooting my (external) router causes all mounted loop devices,
other than / and swap, to be umounted. I was able to fix this problem
in Fedora 13 (in which case the loop devices also became detached) by
disabling netfs, but that was kind of drastic and I'm hoping to avoid
it this time around.

So that is the current state of play and the issues I am currently
working to resolve. I am reasonably certain at this point that the
issues are related to Fedora 15 and the systemd init process, _not_
loop-aes. I will post back if/when I make some progress.

FG