Advanced dracut use help or tips needed.

Advanced dracut use help or tips needed.

[olivier.lahaye1-GANU6spQydw]

Hi, 

Please forgive me if this is the wrong list to post, but I didn't find any other places to ask what I'm looking for regarding advanced usage of dracut. 

I'm part of systemimager developpers, a tool to build and deploy linux images on computers (often used in clusters to deploy nodes). 

This perl software is building it's own initramfs from scratch (build kernel and all content), unfortunately, this is problematic as it needs a recent kernel to support latest hardware.(detect disk or net adapters) 
Unfortunateley, trying to build everything on an rhel6 distro for example will result in bad binaries. Indeed, if we build kernel 3.10 on a 2.6 kernel (it works), but then building lvm tools (bad binaries) as the tools will be built with kernel 2.6 system includes and run on a 3.10 kernel resulting at best in crashes and at worse in data loss. 
More over, it is based on udev only (no systemd or network manager). thus, now it's even more difficult to build it (difficult to build udev without systemd, ...) 

My aim would be to drop this old way of doing stuff and use dracut to build a bootable image using system installed binaries. Indeed, this image is only used to boot the node, create disk partition table, format partitions, init network download some post install scripts, start a custom deployment monitoring daemon and copy the image from server using preconfigured method (either rsync, nfs, bittorrent, ...) (and reboot) 
As linux vendors are backporting support for latest hardware, having the latest binaries is not needed anymore. using booted system materials will be sufficient. 

For example, here is the actual content of the initrd_template we are using: http://olivier.lahaye1.free.fr/OSCAR/initrd_template.txt 
It has no second stage and it is far from being optimized but it does more or less what we need. 

Despite my googleing, I've been unable to find examples other than creating a basic initial ramdisk for booting an installed OS. I was even unable to find how the fedora or redhat pxe boot image were created (and how they are built). 

what I'm looking for would be some sort of comand that would do: 

dracut --include-all-network-drivers --include-all-disk-drivers --include-nfsandother_drivers --put_what_is_needed_for_an_advanced_initramfs(udev+systemd+networkmanager+...) --put_buzybox --add_binaries_from_running_system <list_of_binaries.txt> --add_binaries_required_libs_from_system --add_other_files --start_inital_system_tasks_like_systemd_then_jump_to_our_rcS_script 

I'd greatly apreciate any help, tip on how to proceed or even pointer to pages describing how the distro installation initial pxe disks are build (for example how the code calling the second stage is put into the image and from when it is taken from). 

Best regards. 

-- 
  Olivier LAHAYE 

Re: Advanced dracut use help or tips needed.

[Tim Mohlmann]

I'm not sure if dracut is the correct weapon of choice for automatic
linux install deployment. Dracut is a event driven initramfs. Means it
waits for events to do all operations necessary to mount your root
partition.

Dracut uses udev to monitor for events to happen. Udev is either
supplied separately or together in the systemd package of your distro.
Also, dracut includes kernel modules present for the version you are
generating for. Means you need the kernel modules present on the same
system as where you are generating the dracut image.

It is possible to hook in your own scripts. Maybe binaries. But then
you need static ones, or include all the libs they depend on
(including glibc) making your initramfs a beast.

I would recommend to either use dracut to mount a nfs as root. On the
nfs host you just need to install a basic linux system with all the
tools necessary to partition and install your system.

Another way is to use busybox in a manual created initramfs. Busybox
at least provide you with some standard binaries: wget, httpd, gunzip,
mk2fs and a shell whit quite some functionality. But I guess this is a
bit of topic for this list, but I used to use it before I found
dracut. If you need a quickstart guide, send me a message directly.

Br, Tim

2014/1/11  <olivier.lahaye1-GANU6spQydw@public.gmane.org>:
>
> Hi,
>
> Please forgive me if this is the wrong list to post, but I didn't find any other places to ask what I'm looking for regarding advanced usage of dracut.
>
> I'm part of systemimager developpers, a tool to build and deploy linux images on computers (often used in clusters to deploy nodes).
>
> This perl software is building it's own initramfs from scratch (build kernel and all content), unfortunately, this is problematic as it needs a recent kernel to support latest hardware.(detect disk or net adapters)
> Unfortunateley, trying to build everything on an rhel6 distro for example will result in bad binaries. Indeed, if we build kernel 3.10 on a 2.6 kernel (it works), but then building lvm tools (bad binaries) as the tools will be built with kernel 2.6 system includes and run on a 3.10 kernel resulting at best in crashes and at worse in data loss.
> More over, it is based on udev only (no systemd or network manager). thus, now it's even more difficult to build it (difficult to build udev without systemd, ...)
>
> My aim would be to drop this old way of doing stuff and use dracut to build a bootable image using system installed binaries. Indeed, this image is only used to boot the node, create disk partition table, format partitions, init network download some post install scripts, start a custom deployment monitoring daemon and copy the image from server using preconfigured method (either rsync, nfs, bittorrent, ...) (and reboot)
> As linux vendors are backporting support for latest hardware, having the latest binaries is not needed anymore. using booted system materials will be sufficient.
>
> For example, here is the actual content of the initrd_template we are using: http://olivier.lahaye1.free.fr/OSCAR/initrd_template.txt
> It has no second stage and it is far from being optimized but it does more or less what we need.
>
> Despite my googleing, I've been unable to find examples other than creating a basic initial ramdisk for booting an installed OS. I was even unable to find how the fedora or redhat pxe boot image were created (and how they are built).
>
> what I'm looking for would be some sort of comand that would do:
>
> dracut --include-all-network-drivers --include-all-disk-drivers --include-nfsandother_drivers --put_what_is_needed_for_an_advanced_initramfs(udev+systemd+networkmanager+...) --put_buzybox --add_binaries_from_running_system <list_of_binaries.txt> --add_binaries_required_libs_from_system --add_other_files --start_inital_system_tasks_like_systemd_then_jump_to_our_rcS_script
>
> I'd greatly apreciate any help, tip on how to proceed or even pointer to pages describing how the distro installation initial pxe disks are build (for example how the code calling the second stage is put into the image and from when it is taken from).
>
> Best regards.
>
> --
>   Olivier LAHAYE
>
> --
> To unsubscribe from this list: send the line "unsubscribe initramfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Advanced dracut use help or tips needed.

[Harald Hoyer]

On 01/11/2014 01:27 PM, olivier.lahaye1-GANU6spQydw@public.gmane.org wrote:
> 
> Hi,
> 
> Please forgive me if this is the wrong list to post, but I didn't find any
> other places to ask what I'm looking for regarding advanced usage of dracut.
> 
> 
> I'm part of systemimager developpers, a tool to build and deploy linux images
> on computers (often used in clusters to deploy nodes).
> 
> This perl software is building it's own initramfs from scratch (build kernel
> and all content), unfortunately, this is problematic as it needs a recent
> kernel to support latest hardware.(detect disk or net adapters) 
> Unfortunateley, trying to build everything on an rhel6 distro for example
> will result in bad binaries. Indeed, if we build kernel 3.10 on a 2.6 kernel
> (it works), but then building lvm tools (bad binaries) as the tools will be
> built with kernel 2.6 system includes and run on a 3.10 kernel resulting at
> best in crashes and at worse in data loss. More over, it is based on udev
> only (no systemd or network manager). thus, now it's even more difficult to
> build it (difficult to build udev without systemd, ...)
> 
> My aim would be to drop this old way of doing stuff and use dracut to build a
> bootable image using system installed binaries. Indeed, this image is only
> used to boot the node, create disk partition table, format partitions, init
> network download some post install scripts, start a custom deployment
> monitoring daemon and copy the image from server using preconfigured method
> (either rsync, nfs, bittorrent, ...) (and reboot) As linux vendors are
> backporting support for latest hardware, having the latest binaries is not
> needed anymore. using booted system materials will be sufficient.
> 
> For example, here is the actual content of the initrd_template we are using:
> http://olivier.lahaye1.free.fr/OSCAR/initrd_template.txt It has no second
> stage and it is far from being optimized but it does more or less what we
> need.
> 
> Despite my googleing, I've been unable to find examples other than creating a
> basic initial ramdisk for booting an installed OS. I was even unable to find
> how the fedora or redhat pxe boot image were created (and how they are
> built).
> 
> what I'm looking for would be some sort of comand that would do:
> 
> dracut --include-all-network-drivers --include-all-disk-drivers
> --include-nfsandother_drivers
> --put_what_is_needed_for_an_advanced_initramfs(udev+systemd+networkmanager+...)
> --put_buzybox --add_binaries_from_running_system <list_of_binaries.txt>
> --add_binaries_required_libs_from_system --add_other_files
> --start_inital_system_tasks_like_systemd_then_jump_to_our_rcS_script
> 
> I'd greatly apreciate any help, tip on how to proceed or even pointer to
> pages describing how the distro installation initial pxe disks are build (for
> example how the code calling the second stage is put into the image and from
> when it is taken from).
> 
> Best regards.
> 

https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html

https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#NetworkBoot

Re: Advanced dracut use help or tips needed.

[olivier.lahaye1-GANU6spQydw]

Hi Harald,

Thanks for your answer, I know this doc; I've googled a lot, but I'm unable to find what I need.
All those docs explain how to boot an OS. either local or networked.
I mean, the initrd end with a swaproot and finish boot on the booted OS.

That's not what I want.

I'm looking for something that:
1/ detect all network and all hard drives
2/ start network
3/ run MY script
4/ reboots or shutdown
NO swaproot and NO os boot

In the doc, I've been unable to find the following infos:
1/ how do I tell to dracut to include ALL drivers including frimwares
   do I really need to find /lib/modules |xargs to create a full list and give drivers one y one to the dracut command line options? Really no way to specify all drivers or at least all network drivers + all hdisk controler + all filesystems? (for example, the pxeboot image for fedora install has ALL drivers + all firmwares)
2/ how to I tell to dracut that I don't want swaproot? (for example fedora pxeboot  image runs a secondstage loader, thus no swaproot)
3/ how do I tell dracut that instead I want to run /etc/rcS (or /bin/my_systemimager_scrit.sh)? (for example fedora pxeboot image runs a secondstage loader)

Assuming that all my specific software is located in /usr/lib/systemimager/template_img/
and knowing that I need to run /usr/bin/system_imager_client.sh
and knowing that I need parted+buzybox+lvmtools+*mkfs*+rsync_client+ssh_client
(and optionaly knowing that I need fr keyboard)

What would be a typical dracut command? (that would work the same on rhel-6 and on fedora-20 or debian-7)

My problems are:
1/ was unable to tell dracut no to swaproot
2/ was unable to tell to dracut to run my scrip instead
3/ was unable to tell "all drivers" (I can fallback to list them all but seems stupid)
4/ I'd like to use systemd when  possible

Sorry for asking things that may be evident to you, but I'm new in this world and 99.999% of google results only points to basic dracut usage explaining how to build/rebuild an OS boot image.

Best regards,

Olivier.

----- Mail original -----
> De: "Harald Hoyer" <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> À: "olivier lahaye1" <olivier.lahaye1-GANU6spQydw@public.gmane.org>, initramfs@vger.kernel.org
> Envoyé: Mercredi 22 Janvier 2014 13:17:23
> Objet: Re: Advanced dracut use help or tips needed.
> 
> On 01/11/2014 01:27 PM, olivier.lahaye1-GANU6spQydw@public.gmane.org wrote:
> > 
> > Hi,
> > 
> > Please forgive me if this is the wrong list to post, but I didn't
> > find any
> > other places to ask what I'm looking for regarding advanced usage
> > of dracut.
> > 
> > 
> > I'm part of systemimager developpers, a tool to build and deploy
> > linux images
> > on computers (often used in clusters to deploy nodes).
> > 
> > This perl software is building it's own initramfs from scratch
> > (build kernel
> > and all content), unfortunately, this is problematic as it needs a
> > recent
> > kernel to support latest hardware.(detect disk or net adapters)
> > Unfortunateley, trying to build everything on an rhel6 distro for
> > example
> > will result in bad binaries. Indeed, if we build kernel 3.10 on a
> > 2.6 kernel
> > (it works), but then building lvm tools (bad binaries) as the tools
> > will be
> > built with kernel 2.6 system includes and run on a 3.10 kernel
> > resulting at
> > best in crashes and at worse in data loss. More over, it is based
> > on udev
> > only (no systemd or network manager). thus, now it's even more
> > difficult to
> > build it (difficult to build udev without systemd, ...)
> > 
> > My aim would be to drop this old way of doing stuff and use dracut
> > to build a
> > bootable image using system installed binaries. Indeed, this image
> > is only
> > used to boot the node, create disk partition table, format
> > partitions, init
> > network download some post install scripts, start a custom
> > deployment
> > monitoring daemon and copy the image from server using
> > preconfigured method
> > (either rsync, nfs, bittorrent, ...) (and reboot) As linux vendors
> > are
> > backporting support for latest hardware, having the latest binaries
> > is not
> > needed anymore. using booted system materials will be sufficient.
> > 
> > For example, here is the actual content of the initrd_template we
> > are using:
> > http://olivier.lahaye1.free.fr/OSCAR/initrd_template.txt It has no
> > second
> > stage and it is far from being optimized but it does more or less
> > what we
> > need.
> > 
> > Despite my googleing, I've been unable to find examples other than
> > creating a
> > basic initial ramdisk for booting an installed OS. I was even
> > unable to find
> > how the fedora or redhat pxe boot image were created (and how they
> > are
> > built).
> > 
> > what I'm looking for would be some sort of comand that would do:
> > 
> > dracut --include-all-network-drivers --include-all-disk-drivers
> > --include-nfsandother_drivers
> > --put_what_is_needed_for_an_advanced_initramfs(udev+systemd+networkmanager+...)
> > --put_buzybox --add_binaries_from_running_system
> > <list_of_binaries.txt>
> > --add_binaries_required_libs_from_system --add_other_files
> > --start_inital_system_tasks_like_systemd_then_jump_to_our_rcS_script
> > 
> > I'd greatly apreciate any help, tip on how to proceed or even
> > pointer to
> > pages describing how the distro installation initial pxe disks are
> > build (for
> > example how the code calling the second stage is put into the image
> > and from
> > when it is taken from).
> > 
> > Best regards.
> > 
> 
> https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html
> 
> https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#NetworkBoot
> 

Re: Advanced dracut use help or tips needed.

[Harald Hoyer]

On 02/20/2014 08:52 AM, olivier.lahaye1-GANU6spQydw@public.gmane.org wrote:
> 
> Hi Harald,
> 
> Thanks for your answer, I know this doc; I've googled a lot, but I'm unable to find what I need.
> All those docs explain how to boot an OS. either local or networked.
> I mean, the initrd end with a swaproot and finish boot on the booted OS.
> 
> That's not what I want.
> 
> I'm looking for something that:
> 1/ detect all network and all hard drives

difficult to know, without specifying a timeout.
firmware and devices can take as long as they want to initialize

> 2/ start network

"rd.neednet=1 ip=dhcp" on the kernel command line

> 3/ run MY script

Hook a systemd service in

https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#dracutbootup7

Do not specify "root=..." on the kernel cmdline.

Your service should have

Before: basic.target
After: sysinit.target

Then in your service, you can wait as long as you want, while in the background
all drivers load.

you can do the same as dracut-initqueue.

> 4/ reboots or shutdown
> NO swaproot and NO os boot
> 
> In the doc, I've been unable to find the following infos:
> 1/ how do I tell to dracut to include ALL drivers including frimwares
>    do I really need to find /lib/modules |xargs to create a full list and give drivers one y one to the dracut command line options? Really no way to specify all drivers or at least all network drivers + all hdisk controler + all filesystems? (for example, the pxeboot image for fedora install has ALL drivers + all firmwares)

--no-hostonly for network + disk


> 2/ how to I tell to dracut that I don't want swaproot? (for example fedora pxeboot  image runs a secondstage loader, thus no swaproot)

do not specify "root="

> 3/ how do I tell dracut that instead I want to run /etc/rcS (or /bin/my_systemimager_scrit.sh)? (for example fedora pxeboot image runs a secondstage loader)

hook your service in the boot

> 
> Assuming that all my specific software is located in /usr/lib/systemimager/template_img/
> and knowing that I need to run /usr/bin/system_imager_client.sh
> and knowing that I need parted+buzybox+lvmtools+*mkfs*+rsync_client+ssh_client
> (and optionaly knowing that I need fr keyboard)
> 
> What would be a typical dracut command? (that would work the same on rhel-6 and on fedora-20 or debian-7)
> 
> My problems are:
> 1/ was unable to tell dracut no to swaproot
> 2/ was unable to tell to dracut to run my scrip instead
> 3/ was unable to tell "all drivers" (I can fallback to list them all but seems stupid)
> 4/ I'd like to use systemd when  possible

Re: Advanced dracut use help or tips needed.

[olivier.lahaye1-GANU6spQydw]

Hi Harald,

Thanks you for your detailed response, I'll start to dig my case using your tips.
I still have few more questions though.

>> 1/ detect all network and all hard drives
I just want to do the same as a linux installation disk. (as SystemImager is a sort of OS install disk).
I mean, how do all OS install disk wait that needed drivers have finished to initialize?
I just need that udev scan hardware and load needed drivers, nothing more. isn't there an udev target for that?
I just need to run my script once network is up and once hard drives harve been detected.
Can't I set a systemd dependacy on network and or disks? does a target matches that kind of need?

- Systemd compatibility with non systemd distros.
I see that you answered my restion about running my script using systemd, but many distro are not using systemd by default. do you think that using dracut with systemd on those distros is stable enough for my purpose? (I think about ubuntu, debian, rhel6, ...) does it work at least on rhel6? If not, are there alternatives? (I need to support: rhel-6+ fedora-17+ debian-6+ ubuntu-1204+ SuSE-11+)

- Kernel boot option.
how does the "rd.neednet=1 ip=dhcp" work when used? first device found that can get an ip wins? is there an order? if I have 2 ethernet devices and I want to create a bond later (thus both connected) is eth0 chosen over eth1? (for example if a dhcp servers gives addr to both)
Isn't there a dracut module for network setup using dhcp and udev (to detect hardware) that would add a systemd target?

- Systemd targets required befor I run my script
Can I add "After: network.target" to make sure I have network initialized?

- Supported filesystems
Is there a way to add all supported filesystem to the initramfs.img or should I list them all updating the list when new ones are added?

- Firmwares for drivers (net)
You told me to use "--no-hostonly for network + disk", but isn't there an option for include all modules present in /lib/modules/<kernel>?
At least, does --no-hostonly includes drivers firmwares if present when needed?
Can rd.neednet=1 ip=dhcp work with a NIC that requires a firmware to be loaded?

- dractut default modules: how do I know?
How do I knwow which modules are used by default and which ones are not (thus requiring me to use the -m option). For example, I've installed dracut-network which contains many modules. is there a sort of depend in it? any doc on that (no man in the fedora package and insid the modules. Do I need to reverse engineer the file to understand their function and usage?)

- dracut redudant options?
What is the difference between using --filesystems btrfs and --add-drivers btrfs??
What is the difference between using --install /usr/sbin/fsck.ext3 and --fscks ext3

- including modules: are dependancies for modules handled?
Does --modules includes depending modules. I mean does --module snd includes snd_hwdep,snd_timer,snd_hda_codec_idt,snd_pcm,snd_seq,snd_hda_codec,snd_hda_intel,snd_seq_device as well?

Sorry to bother whith such questions, but I'm completely new in this early OS boot world. As I said before, google is of no help for me as 99.99% of answers are about creating a simple initrd for OS boot.
So if you could bootstrap my knowledge, that would greatly help me in fixing SystemImager faster.

Best regards,

Olivier.

----- Mail original -----
> De: "Harald Hoyer" <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> À: "olivier lahaye1" <olivier.lahaye1-GANU6spQydw@public.gmane.org>
> Cc: initramfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Envoyé: Vendredi 21 Février 2014 16:39:09
> Objet: Re: Advanced dracut use help or tips needed.
> 
> On 02/20/2014 08:52 AM, olivier.lahaye1-GANU6spQydw@public.gmane.org wrote:
> > 
> > Hi Harald,
> > 
> > Thanks for your answer, I know this doc; I've googled a lot, but
> > I'm unable to find what I need.
> > All those docs explain how to boot an OS. either local or
> > networked.
> > I mean, the initrd end with a swaproot and finish boot on the
> > booted OS.
> > 
> > That's not what I want.
> > 
> > I'm looking for something that:
> > 1/ detect all network and all hard drives
> 
> difficult to know, without specifying a timeout.
> firmware and devices can take as long as they want to initialize
> 
> > 2/ start network
> 
> "rd.neednet=1 ip=dhcp" on the kernel command line
> 
> > 3/ run MY script
> 
> Hook a systemd service in
> 
> https://www.kernel.org/pub/linux/utils/boot/dracut/dracut.html#dracutbootup7
> 
> Do not specify "root=..." on the kernel cmdline.
> 
> Your service should have
> 
> Before: basic.target
> After: sysinit.target
> 
> Then in your service, you can wait as long as you want, while in the
> background
> all drivers load.
> 
> you can do the same as dracut-initqueue.
> 
> > 4/ reboots or shutdown
> > NO swaproot and NO os boot
> > 
> > In the doc, I've been unable to find the following infos:
> > 1/ how do I tell to dracut to include ALL drivers including
> > frimwares
> >    do I really need to find /lib/modules |xargs to create a full
> >    list and give drivers one y one to the dracut command line
> >    options? Really no way to specify all drivers or at least all
> >    network drivers + all hdisk controler + all filesystems? (for
> >    example, the pxeboot image for fedora install has ALL drivers +
> >    all firmwares)
> 
> --no-hostonly for network + disk
> 
> 
> > 2/ how to I tell to dracut that I don't want swaproot? (for example
> > fedora pxeboot  image runs a secondstage loader, thus no swaproot)
> 
> do not specify "root="
> 
> > 3/ how do I tell dracut that instead I want to run /etc/rcS (or
> > /bin/my_systemimager_scrit.sh)? (for example fedora pxeboot image
> > runs a secondstage loader)
> 
> hook your service in the boot
> 
> > 
> > Assuming that all my specific software is located in
> > /usr/lib/systemimager/template_img/
> > and knowing that I need to run /usr/bin/system_imager_client.sh
> > and knowing that I need
> > parted+buzybox+lvmtools+*mkfs*+rsync_client+ssh_client
> > (and optionaly knowing that I need fr keyboard)
> > 
> > What would be a typical dracut command? (that would work the same
> > on rhel-6 and on fedora-20 or debian-7)
> > 
> > My problems are:
> > 1/ was unable to tell dracut no to swaproot
> > 2/ was unable to tell to dracut to run my scrip instead
> > 3/ was unable to tell "all drivers" (I can fallback to list them
> > all but seems stupid)
> > 4/ I'd like to use systemd when  possible
> 
> 
> 
> 

Re: Advanced dracut use help or tips needed.

[Harald Hoyer]

On 02/22/2014 09:46 AM, olivier.lahaye1-GANU6spQydw@public.gmane.org wrote:
> Hi Harald,
> 
> Thanks you for your detailed response, I'll start to dig my case using your
> tips. I still have few more questions though.
> 
>>> 1/ detect all network and all hard drives
> I just want to do the same as a linux installation disk. (as SystemImager is
> a sort of OS install disk). I mean, how do all OS install disk wait that
> needed drivers have finished to initialize? I just need that udev scan
> hardware and load needed drivers, nothing more. isn't there an udev target
> for that? I just need to run my script once network is up and once hard
> drives harve been detected. Can't I set a systemd dependacy on network and
> or disks? does a target matches that kind of need?

Kind of. In dracut we use udev settle in the dracut initqueue loop to wait
until all needed devices show up.

Basically, you need to know what to wait for to do the waiting :-)

> 
> - Systemd compatibility with non systemd distros. I see that you answered my
> restion about running my script using systemd, but many distro are not using
> systemd by default. do you think that using dracut with systemd on those
> distros is stable enough for my purpose? (I think about ubuntu, debian,
> rhel6, ...) does it work at least on rhel6? If not, are there alternatives?
> (I need to support: rhel-6+ fedora-17+ debian-6+ ubuntu-1204+ SuSE-11+)

If you don't want to rely on systemd, you have to use the pure dracut
mechanisms. You create "end of wait" conditions with iniqueue/finished hooks,
where you can also call out to your custom script, and never return, if you want.

> 
> - Kernel boot option. how does the "rd.neednet=1 ip=dhcp" work when used?
> first device found that can get an ip wins? is there an order? if I have 2
> ethernet devices and I want to create a bond later (thus both connected) is
> eth0 chosen over eth1? (for example if a dhcp servers gives addr to both) 
> Isn't there a dracut module for network setup using dhcp and udev (to detect
> hardware) that would add a systemd target?

As of now, networking is not yet hooked in the systemd mechanism. This will
change in the future, but as of now, these are just custom dracut network
scripts. Setup of the network is driven by the kernel command line or injected
or static pseudo kernel command line options in /etc/cmdline.d/*.conf.

> 
> - Systemd targets required befor I run my script Can I add "After:
> network.target" to make sure I have network initialized?

Hmm, sadly not yet.

> 
> - Supported filesystems Is there a way to add all supported filesystem to
> the initramfs.img or should I list them all updating the list when new ones
> are added?

If you generate a non-hostonly aka generic initramfs, all available filesystems
are put in the initramfs.

> 
> - Firmwares for drivers (net) You told me to use "--no-hostonly for network
> + disk", but isn't there an option for include all modules present in
> /lib/modules/<kernel>? At least, does --no-hostonly includes drivers
> firmwares if present when needed? Can rd.neednet=1 ip=dhcp work with a NIC
> that requires a firmware to be loaded?

yes, firmwares are included for all drivers, which are in the initramfs image.

To include all /lib/modules/<kernel> you would have to write your own dracut
module, which does exactly that. Needed firmware would be included also
automatically.

> 
> - dractut default modules: how do I know? How do I knwow which modules are
> used by default and which ones are not (thus requiring me to use the -m
> option). For example, I've installed dracut-network which contains many
> modules. is there a sort of depend in it? any doc on that (no man in the
> fedora package and insid the modules. Do I need to reverse engineer the file
> to understand their function and usage?)

Reverse engineering + the documentation on the kernel command line parameters
is the fastest way to get an overview. Most shell scripts should be fairly easy
to understand.

All modules have a "check" function in module-setup.sh. If a module is included
is determined at runtime. Recent dracut modules output a warning, why the
module is not included in the initramfs, if a tool is missing. Of course
nothing prevents you from specifying the exact list.

> 
> - dracut redudant options? What is the difference between using
> --filesystems btrfs and --add-drivers btrfs?? What is the difference between
> using --install /usr/sbin/fsck.ext3 and --fscks ext3

* --filesystems btrfs
  would include the kernel driver btrfs and the fsck tool (if there were any)
* --add-drivers btrfs
  would only include the kernel driver
* --fscks ext3
  would only add the fsck.ext3
* --install <foo>
  just installs <foo> without any logic (except shared libs)

> 
> - including modules: are dependancies for modules handled? Does --modules
> includes depending modules. I mean does --module snd includes
> snd_hwdep,snd_timer,snd_hda_codec_idt,snd_pcm,snd_seq,snd_hda_codec,snd_hda_intel,snd_seq_device
> as well?

"--module" does not exist

"--modules" is for dracut modules, not kernel modules

what you probably meant is "--add-drivers snd"

This would add the kernel driver/module "snd" with all its dependencies
determined with:

$ modprobe "snd" --ignore-install --show-depends
insmod /lib/modules/3.14.0-0.rc2.git3.2.fc21.x86_64/kernel/sound/soundcore.ko
insmod /lib/modules/3.14.0-0.rc2.git3.2.fc21.x86_64/kernel/sound/core/snd.ko

So, "snd.ko soundcore.ko" ... nothing of snd_hda_intel.


> 
> Sorry to bother whith such questions, but I'm completely new in this early
> OS boot world. As I said before, google is of no help for me as 99.99% of
> answers are about creating a simple initrd for OS boot. So if you could
> bootstrap my knowledge, that would greatly help me in fixing SystemImager
> faster.
> 
> Best regards,
> 
> Olivier.
> 

No problem. What you can do right now is:
* ask me more questions
* clarify the documentation in the dracut project (asciidoc *.asc files)
* send patches for the documentation
-> win-win for all of us

Re: Advanced dracut use help or tips needed.

[Olivier LAHAYE]

Hi Harald.

Thanks a lot for your patience and detailed answer.

I'll do my best to help. I will certainly have more questions, especially for the active waiting loop for net as I'm not sure I've understood well.
As for systemd I like it very much. My question was about distros that don't like systemd. My wish is to write a dracut module for systemimager and I'd like to avoid managing difference between systemd and non systemd distros. I just hope that dracut systemd module exists on all distros I need to support.

Best regards

Le 24 févr. 2014 à 09:20, Harald Hoyer <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> a écrit :

> On 02/22/2014 09:46 AM, olivier.lahaye1-GANU6spQydw@public.gmane.org wrote:
>> Hi Harald,
>> 
>> Thanks you for your detailed response, I'll start to dig my case using your
>> tips. I still have few more questions though.
>> 
>>>> 1/ detect all network and all hard drives
>> I just want to do the same as a linux installation disk. (as SystemImager is
>> a sort of OS install disk). I mean, how do all OS install disk wait that
>> needed drivers have finished to initialize? I just need that udev scan
>> hardware and load needed drivers, nothing more. isn't there an udev target
>> for that? I just need to run my script once network is up and once hard
>> drives harve been detected. Can't I set a systemd dependacy on network and
>> or disks? does a target matches that kind of need?
> 
> Kind of. In dracut we use udev settle in the dracut initqueue loop to wait
> until all needed devices show up.
> 
> Basically, you need to know what to wait for to do the waiting :-)
> 
>> 
>> - Systemd compatibility with non systemd distros. I see that you answered my
>> restion about running my script using systemd, but many distro are not using
>> systemd by default. do you think that using dracut with systemd on those
>> distros is stable enough for my purpose? (I think about ubuntu, debian,
>> rhel6, ...) does it work at least on rhel6? If not, are there alternatives?
>> (I need to support: rhel-6+ fedora-17+ debian-6+ ubuntu-1204+ SuSE-11+)
> 
> If you don't want to rely on systemd, you have to use the pure dracut
> mechanisms. You create "end of wait" conditions with iniqueue/finished hooks,
> where you can also call out to your custom script, and never return, if you want.
> 
>> 
>> - Kernel boot option. how does the "rd.neednet=1 ip=dhcp" work when used?
>> first device found that can get an ip wins? is there an order? if I have 2
>> ethernet devices and I want to create a bond later (thus both connected) is
>> eth0 chosen over eth1? (for example if a dhcp servers gives addr to both) 
>> Isn't there a dracut module for network setup using dhcp and udev (to detect
>> hardware) that would add a systemd target?
> 
> As of now, networking is not yet hooked in the systemd mechanism. This will
> change in the future, but as of now, these are just custom dracut network
> scripts. Setup of the network is driven by the kernel command line or injected
> or static pseudo kernel command line options in /etc/cmdline.d/*.conf.
> 
>> 
>> - Systemd targets required befor I run my script Can I add "After:
>> network.target" to make sure I have network initialized?
> 
> Hmm, sadly not yet.
> 
>> 
>> - Supported filesystems Is there a way to add all supported filesystem to
>> the initramfs.img or should I list them all updating the list when new ones
>> are added?
> 
> If you generate a non-hostonly aka generic initramfs, all available filesystems
> are put in the initramfs.
> 
>> 
>> - Firmwares for drivers (net) You told me to use "--no-hostonly for network
>> + disk", but isn't there an option for include all modules present in
>> /lib/modules/<kernel>? At least, does --no-hostonly includes drivers
>> firmwares if present when needed? Can rd.neednet=1 ip=dhcp work with a NIC
>> that requires a firmware to be loaded?
> 
> yes, firmwares are included for all drivers, which are in the initramfs image.
> 
> To include all /lib/modules/<kernel> you would have to write your own dracut
> module, which does exactly that. Needed firmware would be included also
> automatically.
> 
>> 
>> - dractut default modules: how do I know? How do I knwow which modules are
>> used by default and which ones are not (thus requiring me to use the -m
>> option). For example, I've installed dracut-network which contains many
>> modules. is there a sort of depend in it? any doc on that (no man in the
>> fedora package and insid the modules. Do I need to reverse engineer the file
>> to understand their function and usage?)
> 
> Reverse engineering + the documentation on the kernel command line parameters
> is the fastest way to get an overview. Most shell scripts should be fairly easy
> to understand.
> 
> All modules have a "check" function in module-setup.sh. If a module is included
> is determined at runtime. Recent dracut modules output a warning, why the
> module is not included in the initramfs, if a tool is missing. Of course
> nothing prevents you from specifying the exact list.
> 
>> 
>> - dracut redudant options? What is the difference between using
>> --filesystems btrfs and --add-drivers btrfs?? What is the difference between
>> using --install /usr/sbin/fsck.ext3 and --fscks ext3
> 
> * --filesystems btrfs
>  would include the kernel driver btrfs and the fsck tool (if there were any)
> * --add-drivers btrfs
>  would only include the kernel driver
> * --fscks ext3
>  would only add the fsck.ext3
> * --install <foo>
>  just installs <foo> without any logic (except shared libs)
> 
>> 
>> - including modules: are dependancies for modules handled? Does --modules
>> includes depending modules. I mean does --module snd includes
>> snd_hwdep,snd_timer,snd_hda_codec_idt,snd_pcm,snd_seq,snd_hda_codec,snd_hda_intel,snd_seq_device
>> as well?
> 
> "--module" does not exist
> 
> "--modules" is for dracut modules, not kernel modules
> 
> what you probably meant is "--add-drivers snd"
> 
> This would add the kernel driver/module "snd" with all its dependencies
> determined with:
> 
> $ modprobe "snd" --ignore-install --show-depends
> insmod /lib/modules/3.14.0-0.rc2.git3.2.fc21.x86_64/kernel/sound/soundcore.ko
> insmod /lib/modules/3.14.0-0.rc2.git3.2.fc21.x86_64/kernel/sound/core/snd.ko
> 
> So, "snd.ko soundcore.ko" ... nothing of snd_hda_intel.
> 
> 
>> 
>> Sorry to bother whith such questions, but I'm completely new in this early
>> OS boot world. As I said before, google is of no help for me as 99.99% of
>> answers are about creating a simple initrd for OS boot. So if you could
>> bootstrap my knowledge, that would greatly help me in fixing SystemImager
>> faster.
>> 
>> Best regards,
>> 
>> Olivier.
> 
> No problem. What you can do right now is:
> * ask me more questions
> * clarify the documentation in the dracut project (asciidoc *.asc files)
> * send patches for the documentation
> -> win-win for all of us