grub-mkrescue and hard links

grub-mkrescue and hard links

[Carles Pina i Estany]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]


Hello,

grub-mkrescue is trying to execute cp -dpRl (l for link). If the origin
and destination directories are in different file systems this fails.

We could do a symbolic link (-s), detect if it's in the same device,
etc. but I think that just copying is enough and this optimization could
have other problems (like some utilites doesn't follow symbolic links,
etc.)

Small patch attached.

-- 
Carles Pina i Estany
	http://pinux.info

[-- Attachment #2: grub-mkrescue-nolinks.patch --]
[-- Type: text/x-diff, Size: 349 bytes --]

=== modified file 'util/grub-mkrescue.in'
--- util/grub-mkrescue.in	2009-11-14 18:38:11 +0000
+++ util/grub-mkrescue.in	2009-11-19 22:17:36 +0000
@@ -121,7 +121,7 @@
 if [ "${source}" != "" ] ; then
     for d in ${source}; do
       echo "Processing $d"
-      cp -dpRl "${d}" ${iso9660_dir}/
+      cp -dpR "${d}" ${iso9660_dir}/
     done
 fi
 


[-- Attachment #3: ChangeLog-grub-mkrescue-nolink --]
[-- Type: text/plain, Size: 120 bytes --]

2009-11-19  Carles Pina i Estany <carles@pina.cat>

	* util/grub-mkrescue.in: copies the files instead of hard linking

Re: grub-mkrescue and hard links

[Robert Millan]

On Thu, Nov 19, 2009 at 10:22:57PM +0000, Carles Pina i Estany wrote:
> 
> Hello,
> 
> grub-mkrescue is trying to execute cp -dpRl (l for link). If the origin
> and destination directories are in different file systems this fails.

One of the purposes of grub-mkrescue is to generate images that are bootable
using GRUB.  These images might be complete CDs or DVDs, making a complete
copy very undesireable.

I think I'd rather remove the copiing stage altogether and operate on the
original directory (this is not so strange, e.g. -boot-info-table in
mkisofs also modifies original data and is widely used).

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-mkrescue and hard links

[Carles Pina i Estany]

[-- Attachment #1: Type: text/plain, Size: 1577 bytes --]


Hi,

On Nov/20/2009, Robert Millan wrote:
> On Thu, Nov 19, 2009 at 10:22:57PM +0000, Carles Pina i Estany wrote:
> > 
> > Hello,
> > 
> > grub-mkrescue is trying to execute cp -dpRl (l for link). If the origin
> > and destination directories are in different file systems this fails.
> 
> One of the purposes of grub-mkrescue is to generate images that are
> bootable using GRUB.  These images might be complete CDs or DVDs,
> making a complete copy very undesireable.

I see...

> I think I'd rather remove the copiing stage altogether and operate on
> the original directory (this is not so strange, e.g. -boot-info-table
> in mkisofs also modifies original data and is widely used).

I'm not convinced about it, but I'm not familiar in these utilities
either.

Find attached a second patch that lets the user to specify where the
utilities are.

With this _and_ some manual things with the modules I've generated and
ISO image that boots using qemu.

Which approach do you have in mind for the modules? Which modules should
be copied and where? 

(I see some modules inside the core img, I guess that this is what
should be in the core image and then the modules that the user specifies
in the option --modules should be copied into boot/grub/ (or in 386-pc?)
or something by default?)

Yes, I know that could be better done (avoid copying, more options,
etc.) but I'm doing this to do some comfortable tests with qemu and
gettext :-)

If you tell what you wanted to do I can try to do tomorrow and carry on
with other things.

-- 
Carles Pina i Estany
	http://pinux.info

[-- Attachment #2: ChangeLog-grub-mkrescue.in.patch --]
[-- Type: text/x-diff, Size: 398 bytes --]

2009-11-21  Carles Pina i Estany <carles@pina.cat>

	* util/grub-mkrescue.in: copies the files instead of hard linking
	(removes option -l from cp)
	* adds the option --grub-utilities-directory to specify the
	directory of the grub utilities used by grub-mkrescue
	* adds the optoin --pc_dir to overwrite the detected by
	default
	* source from /boot/grub/grub.cfg to the specific by architecture


[-- Attachment #3: grub-mkrescue.in.patch --]
[-- Type: text/x-diff, Size: 3730 bytes --]

=== modified file 'util/grub-mkrescue.in'
--- util/grub-mkrescue.in	2009-11-14 18:38:11 +0000
+++ util/grub-mkrescue.in	2009-11-21 01:37:30 +0000
@@ -30,19 +30,22 @@
 
 coreboot_dir=${libdir}/$(echo ${PACKAGE_TARNAME} | sed ${transform})/${target_cpu}-coreboot
 pc_dir=${libdir}/$(echo ${PACKAGE_TARNAME} | sed ${transform})/${target_cpu}-pc
-grub_mkisofs="grub-mkisofs"
+grub_utilities_directory=""	# by default, from $PATH
 
 # Usage: usage
 # Print the usage.
 usage () {
     cat <<EOF
-Usage: $0 [OPTION] SOURCE...
+Usage: $0 [OPTION] SOURCE... 
 Make GRUB rescue image.
 
   -h, --help              print this message and exit
   -v, --version           print the version information and exit
   --modules=MODULES       pre-load specified modules MODULES
   --output=FILE           save output in FILE
+  --pc_dir=DIR            use images from directory DIR instead of ${pc_dir}
+  --grub-utilities-directory=DIR  use the grub utilities from DIR instead 
+                          of PATH (e.g. grub-mkisofs, grub-mkimage, ...)
 
 $0 generates a bootable rescue image with specified source files or directories.
 
@@ -63,6 +66,10 @@
 	modules=`echo "$option" | sed 's/--modules=//'` ;;
     --output=*)
 	output_image=`echo "$option" | sed 's/--output=//'` ;;
+    --grub-utilities-directory=*)
+        grub_utilities_directory=`echo "${option}/" | sed 's/--grub-utilities-directory=//'` ;;
+    --pc_dir=*)
+        pc_dir=`echo "$option" | sed 's/--pc_dir=//'` ;;
     -*)
 	echo "Unrecognized option \`$option'" 1>&2
 	usage
@@ -92,6 +99,7 @@
 
 # build coreboot core.img
 if test -e ${coreboot_dir} ; then
+    echo "Generates coreboot"
     memdisk_img=`mktemp`
     memdisk_dir=`mktemp -d`
     mkdir -p ${memdisk_dir}/boot/grub
@@ -111,26 +119,28 @@
 
     tar -C ${memdisk_dir} -cf ${memdisk_img} boot
     rm -rf ${memdisk_dir}
-    grub-mkelfimage -d ${coreboot_dir}/ -m ${memdisk_img} -o ${iso9660_dir}/boot/multiboot.img \
+    ${grub_utilities_directory}grub-mkelfimage -d ${coreboot_dir}/ -m ${memdisk_img} -o ${iso9660_dir}/boot/multiboot.img \
         memdisk tar search iso9660 configfile sh \
         ata at_keyboard
     rm -f ${memdisk_img}
-    grub_mkisofs="${grub_mkisofs} --modification-date=$(echo ${iso_uuid} | sed -e s/-//g)"
+    grub_mkisofs_arguments="--modification-date=$(echo ${iso_uuid} | sed -e s/-//g)"
 fi
 
 if [ "${source}" != "" ] ; then
     for d in ${source}; do
       echo "Processing $d"
-      cp -dpRl "${d}" ${iso9660_dir}/
+      cp -dpR "${d}" ${iso9660_dir}/
     done
 fi
 
 # build eltorito core.img
 if test -e ${pc_dir} ; then
+    echo "Generates eltorito"
     core_img=`mktemp`
-    grub-mkimage -d ${pc_dir}/ -o ${core_img} --prefix=/boot/grub/i386-pc \
+    ${grub_utilities_directory}grub-mkimage -d ${pc_dir}/ -o ${core_img} --prefix=/boot/grub/i386-pc \
         memdisk tar search iso9660 configfile sh \
         biosdisk
+    mkdir -p ${iso9660_dir}/boot/grub/i386-pc/
     cat ${pc_dir}/cdboot.img ${core_img} > ${iso9660_dir}/boot/grub/i386-pc/eltorito.img
     rm -f ${core_img}
 
@@ -138,14 +148,14 @@
     (for i in ${modules} ; do
         echo "insmod $i"
     done ; \
-    echo "source /boot/grub/grub.cfg") \
-    > ${iso9660_dir}/boot/grub/i386-pc/grub.cfg
+    echo "source /boot/grub/i386-pc/grub.cfg") \
+    > ${iso9660_dir}/boot/grub/grub.cfg
 
-    grub_mkisofs="${grub_mkisofs} -b boot/grub/i386-pc/eltorito.img -boot-info-table"
+    grub_mkisofs_arguments="-b boot/grub/i386-pc/eltorito.img -boot-info-table"
 fi
 
 # build iso image
-${grub_mkisofs} -o ${output_image} -r -J ${iso9660_dir}
+${grub_utilities_directory}grub-mkisofs ${grub_mkisofs_arguments} -o ${output_image} -r -J ${iso9660_dir}
 rm -rf ${iso9660_dir}
 
 exit 0

Re: grub-mkrescue and hard links

[Robert Millan]

On Sat, Nov 21, 2009 at 01:54:36AM +0000, Carles Pina i Estany wrote:
> 
> Find attached a second patch that lets the user to specify where the
> utilities are.

Committed, with some adjustments.  Notably, I unified both options into
a single one, and (intentionally) left it undocumented.

You can generate a test image from build tree with:

  ./grub-mkrescue --override-directory=`pwd` --output=/tmp/out.iso

> Which approach do you have in mind for the modules? Which modules should
> be copied and where? 

In principle, the idea is that all modules are copied to their corresponding
arch-specific dir, and additionally modules specified by user (via --modules)
are preloaded by grub-mkrescue (mechanisms for preloading might differ
between architectures).

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-mkrescue and hard links

[Vladimir 'φ-coder/phcoder' Serbinenko]

[-- Attachment #1: Type: text/plain, Size: 1526 bytes --]

Robert Millan wrote:
> On Thu, Nov 19, 2009 at 10:22:57PM +0000, Carles Pina i Estany wrote:
>   
>> Hello,
>>
>> grub-mkrescue is trying to execute cp -dpRl (l for link). If the origin
>> and destination directories are in different file systems this fails.
>>     
>
> One of the purposes of grub-mkrescue is to generate images that are bootable
> using GRUB.  These images might be complete CDs or DVDs, making a complete
> copy very undesireable.
>   
Why do you need copying at all? mkisofs supports taking source files
from multiple directories:
mkdir h1
mkdir h2
echo 4 > h1/hh
echo 5 > h1/hx
mkisofs -o h.iso h1 h2
isoinfo -l -i h.iso

Directory listing of /
d---------   0    0    0            2048 Nov 21 2009 [     23 02]  .
d---------   0    0    0            2048 Nov 21 2009 [     23 02]  ..
----------   0    0    0               2 Nov 21 2009 [     24 00]  HH.;1
----------   0    0    0               2 Nov 21 2009 [     25 00]  HX.;1

So you can just take grub files from temporary directory and overlay
from its original place

On a side note: when I did:
./grub-mkrescue -o grub.iso overlay
it added a directory named overlay to iso. E.g: I had a folder
overlay/boot instead of /boot on iso
> I think I'd rather remove the copiing stage altogether and operate on the
> original directory (this is not so strange, e.g. -boot-info-table in
> mkisofs also modifies original data and is widely used).
>
>   


-- 
Regards
Vladimir 'φ-coder/phcoder' Serbinenko



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 293 bytes --]

Re: grub-mkrescue and hard links

[Carles Pina i Estany]

Hello,

On Nov/21/2009, Vladimir '??-coder/phcoder' Serbinenko wrote:

> On a side note: when I did:
> ./grub-mkrescue -o grub.iso overlay
> it added a directory named overlay to iso. E.g: I had a folder
> overlay/boot instead of /boot on iso

for me too. Yesterday I did then:
./grub-mkrescue -o grub.iso overlay/*

I don't know if Robret did on purpose or not. I forgot to ask.

-- 
Carles Pina i Estany
	http://pinux.info

Re: grub-mkrescue and hard links

[Robert Millan]

On Sat, Nov 21, 2009 at 11:50:51AM +0000, Carles Pina i Estany wrote:
> 
> Hello,
> 
> On Nov/21/2009, Vladimir '??-coder/phcoder' Serbinenko wrote:
> 
> > On a side note: when I did:
> > ./grub-mkrescue -o grub.iso overlay
> > it added a directory named overlay to iso. E.g: I had a folder
> > overlay/boot instead of /boot on iso
> 
> for me too. Yesterday I did then:
> ./grub-mkrescue -o grub.iso overlay/*
> 
> I don't know if Robret did on purpose or not. I forgot to ask.

It wasn't on purpose.  But I think I fixed this on my last commit (ironically,
this wasn't on purpose either).

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-mkrescue and hard links

[Robert Millan]

On Sat, Nov 21, 2009 at 12:39:53PM +0100, Vladimir 'φ-coder/phcoder' Serbinenko wrote:
> Why do you need copying at all? mkisofs supports taking source files
> from multiple directories:

You're right.  I remove the hardlink hack then.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."