From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1Rrhhg-0004jI-AI for mharc-grub-devel@gnu.org; Sun, 29 Jan 2012 22:14:48 -0500 Received: from eggs.gnu.org ([140.186.70.92]:49278) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RrhhZ-0004b4-NK for grub-devel@gnu.org; Sun, 29 Jan 2012 22:14:43 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RrhhW-0007Zp-Eo for grub-devel@gnu.org; Sun, 29 Jan 2012 22:14:41 -0500 Received: from mail-we0-f169.google.com ([74.125.82.169]:58960) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RrhhW-0007ZO-5E for grub-devel@gnu.org; Sun, 29 Jan 2012 22:14:38 -0500 Received: by wera13 with SMTP id a13so3424532wer.0 for ; Sun, 29 Jan 2012 19:14:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type; bh=7PHW7iGPE9zMkdbH0Str2G6q4bAVaQ9kTNo5B5w8uLE=; b=wXxIfuTdrOSLsI5uq+MBDttYhXlw1GR0jy32Y7z7iOO9R4oeRczI3pChqBR+miWhMy pC+XTyCx7rkHLg+E1YYDwFLoj1z3qPNVVohL+6zBTET03WDjy4B4HxI3/OHlvhapwiQ7 yNYCUK4PE8jq8SzvBF/2Xzw6JR3R3lHiMsWKw= Received: by 10.216.139.87 with SMTP id b65mr6146075wej.59.1327893277302; Sun, 29 Jan 2012 19:14:37 -0800 (PST) Received: from fedora.x201.phnet (7-77.203-62.cust.bluewin.ch. [62.203.77.7]) by mx.google.com with ESMTPS id l8sm47964700wiy.5.2012.01.29.19.14.34 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 29 Jan 2012 19:14:36 -0800 (PST) Message-ID: <4F260B19.40605@gmail.com> Date: Mon, 30 Jan 2012 04:14:33 +0100 From: =?UTF-8?B?VmxhZGltaXIgJ8+GLWNvZGVyL3BoY29kZXInIFNlcmJpbmVua28=?= User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111222 Thunderbird/9.0 MIME-Version: 1.0 To: The development of GRUB 2 , Thomas Schmitt Subject: grub-install revamp and grub-mkrescue adjustments Content-Type: multipart/mixed; boundary="------------040504090704040606040302" X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 74.125.82.169 X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: The development of GNU GRUB List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 03:14:43 -0000 This is a multi-part message in MIME format. --------------040504090704040606040302 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit I was looking into Szymon Janc's patch to compress GRUB modules and found out that adding decompression modules makes it grow bigger than the embed zone. So I propose the following change: make boot sector point to eltotito.img and use diskboot.img at this location. This has given good results with using a very crude patch to xorriso (attached for illustration purposes). I've seen that isohybrid image make similar changes, what is the exact behaviour for isohybrid? PErhaps we can unify it with ours? Offsets aren't written in stone and I wanted to change boot.img for isos a bit first to allow it botting as a floppy even on broken BIOSes (we need few routines for those which are overwritten by partition table, I'll move them to the space reserved for BBP, it's no big deal since there is no BBP on iso9660). This also allows to use other embed sectors to create alternative filesystems. As for the roadmap for creating an iso bootable everywhere as both USB stick and CDROM we'll need following adjustments - i386-pc: detailed above. - *-efi: we currently support only CD-ROMs. Supporting USB sticks needs just to create a partition of type 0xEF pointing to EFI eltorito image. - Macs would need an APM and HFS(+) filesystem. No need to have HFS span over whole disk. It can just point to a small image. To have an APM we need to have bytes 4552 0200 (512 sectors). Rest of table goes to sectors 1-n (n is the number of partitions) - IBM PPC require PreP (msdos with type=42) partition. - ARC would require 0b e5 a9 41 at the beginning and so it's skipped - Put i386-ieee1275 images to the same 0xef partition as EFI images. - Have an ext2 partition for mipsel-loongson and i386-multiboot - Sun would use most of sector 0, so it's skipped. So the layout is: 4552 0200 55 aa Adjustments to xorriso: - patch address in boot sector to eltorito.img. Patch 2nd sector of eltorito.img to point to itself. - set addresses to partitions to point to efi.img, prep.img and to an EBR right after max (prep.img, efi.img, 4 x 512). Make EBR describe a logical partition spanning the rest of iso9660. efi.img or prep.img may be missing - generate APM to point to the hfs.img For supporting Sun (only, without any other one using sector 0): -generate sun disklabel and patch an address and size to boot image in sector 1. For supporting ARC (only, without any other one using sector 0): -generate DVH and set it to point to address of arc.img. What do you think about this how reasonable is it? -- Regards Vladimir 'φ-coder/phcoder' Serbinenko --------------040504090704040606040302 Content-Type: text/x-diff; name="compress.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="compress.diff" === modified file 'ChangeLog' --- ChangeLog 2012-01-30 00:18:02 +0000 +++ ChangeLog 2012-01-30 00:20:12 +0000 @@ -1,8 +1,18 @@ +<<<<<<< TREE 2012-01-30 Vladimir Serbinenko * grub-core/io/lzopio.c (uncompress_block): Fix use of incorrect variable. +======= +2012-01-29 Szymon Janc + + Put grub-mkrescue generated image on diet (modules compression and + xorriso options). + + * util/grub-mkrescue.in: New option --compress. + +>>>>>>> MERGE-SOURCE 2012-01-29 Vladimir Serbinenko * grub-core/lib/posix_wrap/string.h (memchr): New function. === modified file 'grub-core/boot/i386/pc/diskboot.S' --- grub-core/boot/i386/pc/diskboot.S 2011-07-06 18:05:26 +0000 +++ grub-core/boot/i386/pc/diskboot.S 2012-01-30 00:20:27 +0000 @@ -83,7 +83,7 @@ /* the maximum is limited to 0x7f because of Phoenix EDD */ xorl %eax, %eax - movb $0x7f, %al + movb $0x1, %al /* how many do we really want to read? */ cmpw %ax, 8(%di) /* compare against total number of sectors */ === modified file 'util/grub-mkrescue.in' --- util/grub-mkrescue.in 2012-01-24 12:17:36 +0000 +++ util/grub-mkrescue.in 2012-01-30 00:47:25 +0000 @@ -44,6 +44,7 @@ grub_mkimage="${bindir}/`echo grub-mkimage | sed ${transform}`" xorriso=xorriso +compress=no # Usage: usage # Print the usage. @@ -52,13 +53,14 @@ Usage: $self [OPTION] SOURCE... Make GRUB rescue image. - -h, --help print this message and exit - -v, --version print the version information and exit - -o, --output=FILE save output in FILE [required] - --modules=MODULES pre-load specified modules MODULES - --rom-directory=DIR save rom images in DIR [optional] - --xorriso=FILE use FILE as xorriso [optional] - --grub-mkimage=FILE use FILE as grub-mkimage + -h, --help print this message and exit + -v, --version print the version information and exit + -o, --output=FILE save output in FILE [required] + --modules=MODULES pre-load specified modules MODULES + --rom-directory=DIR save rom images in DIR [optional] + --xorriso=FILE use FILE as xorriso [optional] + --grub-mkimage=FILE use FILE as grub-mkimage + --compress[=no,nopad,xz,gz,lzo] apply size reducing measures [optional] $self generates a bootable rescue image with specified source files, source directories, or mkisofs options listed by: xorriso -as mkisofs -help @@ -133,6 +135,11 @@ --xorriso=*) xorriso=`echo "${option}" | sed 's/--xorriso=//'` ;; + --compress) + compress=`argument $option "$@"`; shift ;; + --compress=*) + compress=`echo "${option}" | sed 's/--compress=//'` ;; + *) source="${source} ${option} $@"; break ;; esac @@ -155,6 +162,26 @@ iso9660_dir=`mktemp -d "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"` || exit 1 mkdir -p ${iso9660_dir}/boot/grub +decmod="" +compressor="" +if [ x"${compress}" != xno ] && [ x"${compress}" != xnopad ]; then + case x"$compress" in + xgz) + compressor=`which gzip || true` + decmod="gzio" + compressor="$compressor --best --stdout";; + xxz) + compressor=`which xz || true` + decmod="xzio gcry_crc" + compressor="$compressor --lzma2=dict=128KiB --check=none --stdout";; + xlzo) + compressor=`which lzop || true` + decmod="lzopio adler32 gcry_crc" + compressor="$compressor -9 -c";; + esac +fi +export decmod + process_input_dir () { input_dir="$1" @@ -162,7 +189,12 @@ mkdir -p ${iso9660_dir}/boot/grub/${platform} for file in "${input_dir}/"*.mod "${input_dir}/"efiemu32.o "${input_dir}/"efiemu64.o; do if test -f "$file"; then + modname=`basename $file .mod` + if [ "$compressor" != "" ] ; then + `$compressor $file > ${iso9660_dir}/boot/grub/${platform}/$modname".mod"` + else cp -f "$file" ${iso9660_dir}/boot/grub/${platform}/ + fi fi done for file in ${pkglib_DATA}; do @@ -189,26 +221,26 @@ echo "Enabling $2 support ..." - memdisk_img=`mktemp "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"` || exit 1 - memdisk_dir=`mktemp -d "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"` || exit 1 - mkdir -p ${memdisk_dir}/boot/grub + memdisk_img="`mktemp "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"`" || exit 1 + memdisk_dir="`mktemp -d "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"`" || exit 1 + mkdir -p "${memdisk_dir}"/boot/grub - cat << EOF > ${memdisk_dir}/boot/grub/grub.cfg + cat << EOF > "${memdisk_dir}/boot/grub/grub.cfg" search --fs-uuid --set=root ${iso_uuid} set prefix=(\${root})/boot/grub/${platform} source \$prefix/grub.cfg EOF - (for i in $(cat ${source_directory}/partmap.lst) ${modules} ; do + (for i in $(cat "${source_directory}/partmap.lst") ${modules} ; do echo "insmod $i" done ; \ echo "source /boot/grub/grub.cfg") \ - > ${iso9660_dir}/boot/grub/${platform}/grub.cfg + > "${iso9660_dir}/boot/grub/${platform}/grub.cfg" (cd "${memdisk_dir}"; tar -cf - boot) > "${memdisk_img}" - rm -rf ${memdisk_dir} - $grub_mkimage -O ${platform} -d "${source_directory}" -m "${memdisk_img}" -o "$3" --prefix='(memdisk)/boot/grub' \ - search iso9660 configfile normal memdisk tar $4 - rm -rf ${memdisk_img} + rm -rf "${memdisk_dir}" + "$grub_mkimage" -O ${platform} -d "${source_directory}" -m "${memdisk_img}" -o "$3" --prefix='(memdisk)/boot/grub' \ + $decmod search iso9660 configfile normal memdisk tar $4 + rm -rf "${memdisk_img}" } if [ "${override_dir}" = "" ] ; then @@ -257,23 +289,20 @@ if test -e "${pc_dir}" ; then echo "Enabling BIOS support ..." core_img=`mktemp "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"` || exit 1 - $grub_mkimage -O i386-pc -d ${pc_dir}/ -o ${core_img} --prefix=/boot/grub/i386-pc \ - iso9660 biosdisk - cat ${pc_dir}/cdboot.img ${core_img} > ${iso9660_dir}/boot/grub/i386-pc/eltorito.img - - embed_img=`mktemp "${TMPDIR:-/tmp}/tmp.XXXXXXXXXX"` || exit 1 - cat ${pc_dir}/boot.img ${core_img} > ${embed_img} - - rm -f ${core_img} + "$grub_mkimage" -O i386-pc -d ${pc_dir}/ -o ${core_img} --prefix=/boot/grub/i386-pc \ + $decmod iso9660 biosdisk + cat "${pc_dir}/cdboot.img" "${core_img}" > "${iso9660_dir}/boot/grub/i386-pc/eltorito.img" + + rm -f "${core_img}" (for i in $(cat ${pc_dir}/partmap.lst) ${modules} ; do echo "insmod $i" done ; \ echo "source /boot/grub/grub.cfg") \ - > ${iso9660_dir}/boot/grub/i386-pc/grub.cfg + > "${iso9660_dir}/boot/grub/i386-pc/grub.cfg" grub_mkisofs_arguments="${grub_mkisofs_arguments} -b boot/grub/i386-pc/eltorito.img -no-emul-boot -boot-info-table \ - --embedded-boot ${embed_img}" + -grub-boot ${pc_dir}/boot.img" fi # build multiboot core.img @@ -308,7 +337,12 @@ fi # build iso image -"${xorriso}" -as mkisofs -graft-points ${grub_mkisofs_arguments} --protective-msdos-label -o ${output_image} -r ${iso9660_dir} --sort-weight 0 / --sort-weight 1 /boot ${source} +if [ x"$compress" != xno ]; then + sizeopts=-no-pad +else + sizeopts= +fi +"${xorriso}" -as mkisofs $sizeopts -graft-points ${grub_mkisofs_arguments} --protective-msdos-label -o "${output_image}" -r "${iso9660_dir}" --sort-weight 0 / --sort-weight 1 /boot ${source} rm -rf ${iso9660_dir} rm -f ${embed_img} --------------040504090704040606040302 Content-Type: text/x-diff; name="xor.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="xor.diff" diff -ur xorriso-1.2.1/libisofs/eltorito.c xorriso-1.2.1-mod/libisofs/eltorito.c --- xorriso-1.2.1/libisofs/eltorito.c 2012-01-27 17:02:41.000000000 +0100 +++ xorriso-1.2.1-mod/libisofs/eltorito.c 2012-01-30 02:34:48.266770529 +0100 @@ -1113,6 +1113,16 @@ ret = make_boot_info_table(buf, t->ms_block + (uint32_t) 16, t->bootsrc[idx]->sections[0].block, (uint32_t) imgsize); + if (1) { + uint64_t blk; + if (imgsize < 1024) + return iso_msg_submit(t->image->id, ISO_ISOLINUX_CANT_PATCH, 0, + "Isolinux image too small. We won't patch it."); + + blk = ((uint64_t) t->bootsrc[idx]->sections[0].block) * 4 + 2; + iso_lsb((buf + 1024 - 12), blk & 0xffffffff, 4); + iso_lsb((buf + 1024 - 8), blk >> 32, 4); + } return ret; } diff -ur xorriso-1.2.1/libisofs/system_area.c xorriso-1.2.1-mod/libisofs/system_area.c --- xorriso-1.2.1/libisofs/system_area.c 2012-01-27 17:02:41.000000000 +0100 +++ xorriso-1.2.1-mod/libisofs/system_area.c 2012-01-30 02:30:27.017381391 +0100 @@ -182,7 +182,7 @@ */ static int make_grub_msdos_label(uint32_t img_blocks, int sph, int hpc, - uint8_t *buf, int flag) + uint8_t *buf, int flag, uint64_t blk) { uint8_t *wpt; uint32_t end_lba, end_sec, end_head, end_cyl; @@ -191,6 +191,11 @@ iso_compute_cyl_head_sec(&img_blocks, hpc, sph, &end_lba, &end_sec, &end_head, &end_cyl, 0); + blk = blk * 4 + 1; + + wpt = buf + 0x5c; + for (i = 0; i < 8; i++) + wpt[i] = blk >> (i * 8); /* 1) Zero-fill 446-510 */ wpt = buf + 446; memset(wpt, 0, 64); @@ -739,7 +744,8 @@ if (sa_type == 0 && (t->system_area_options & 1)) { /* Write GRUB protective msdos label, i.e. a simple partition table */ ret = make_grub_msdos_label(img_blocks, t->partition_secs_per_head, - t->partition_heads_per_cyl, buf, 0); + t->partition_heads_per_cyl, buf, 0, + t->bootsrc[0]->sections[0].block); if (ret != ISO_SUCCESS) /* error should never happen */ return ISO_ASSERT_FAILURE; } else if(sa_type == 0 && (t->system_area_options & 2)) { @@ -766,7 +772,8 @@ } else if ((t->partition_offset > 0 || will_append) && sa_type == 0) { /* Write a simple partition table. */ ret = make_grub_msdos_label(img_blocks, t->partition_secs_per_head, - t->partition_heads_per_cyl, buf, 2); + t->partition_heads_per_cyl, buf, 2, + t->bootsrc[0]->sections[0].block); if (ret != ISO_SUCCESS) /* error should never happen */ return ISO_ASSERT_FAILURE; if (t->partition_offset == 0) { diff -ur xorriso-1.2.1/xorriso/emulators.c xorriso-1.2.1-mod/xorriso/emulators.c --- xorriso-1.2.1/xorriso/emulators.c 2012-01-27 17:02:41.000000000 +0100 +++ xorriso-1.2.1-mod/xorriso/emulators.c 2012-01-30 01:56:29.851351371 +0100 @@ -1895,6 +1895,14 @@ goto problem_handler_boot; if(strcmp(argpt, "-isohybrid-mbr")==0) xorriso->system_area_options= (xorriso->system_area_options & ~3) | 2; + } else if(strcmp(argpt, "--grub-boot")==0) { + if(i+1>=argc) + goto not_enough_args; + i++; + ret= Xorriso_set_system_area_path(xorriso, argv[i], 0); + if(ret <= 0) + goto problem_handler_boot; + xorriso->patch_isolinux_image= 1 | 4; } else if(strcmp(argpt, "--protective-msdos-label")==0) { xorriso->system_area_options= (xorriso->system_area_options & ~3) | 1; } else if(strcmp(argpt, "--boot-catalog-hide")==0) { --------------040504090704040606040302--