From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.90_1) id 1jjg12-0008GG-Vi for mharc-grub-devel@gnu.org; Fri, 12 Jun 2020 05:30:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40716) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjg11-0008G6-T9; Fri, 12 Jun 2020 05:30:23 -0400 Received: from mout.gmx.net ([212.227.17.21]:35265) by eggs.gnu.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjg0y-0004q3-IB; Fri, 12 Jun 2020 05:30:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1591954218; bh=IUBJ9sgQoVoZlZF9aiZpxIgQO82P6rXeRGBfnNXqExk=; h=X-UI-Sender-Class:Date:From:To:Subject:Cc:References:In-Reply-To; b=R7GpVyLwLmZdr0/OownjG0IlKdz/vqe1pApnjOdBgBhgCT3fxW53LBp4S6OgyMM/H 18XwDjiGBpSgpEyDmSFc7kSOkaHye5zCnlljeN4E4UurPm59fIzWwilqDK5zbTKpbr bWiCV+f1A40GUQ8g0m7HrpZEwGFQYZ74JTVStBbE= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from scdbackup.webframe.org ([84.179.247.214]) by mail.gmx.com (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MSc1B-1jL9wB0Kjp-00StZk; Fri, 12 Jun 2020 11:30:18 +0200 Date: Fri, 12 Jun 2020 11:30:56 +0200 From: "Thomas Schmitt" To: grub-devel@gnu.org Subject: Re: grub-mkrescue fails with HFS+ error possibly due to directory size Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Cc: othacehe@gnu.org,bug-xorriso@gnu.org References: <40657092990747773@scdbackup.webframe.org> In-Reply-To: <40657092990747773@scdbackup.webframe.org> Message-Id: <7705709255276733552@scdbackup.webframe.org> X-Provags-ID: V03:K1:hwqo51oeg0urvWP84Tm5RdIo5/zc5IYC82FdT6TNmVRkMsI3Ljg 2Stbol3Qp6LS80IeFT8xqUsa8jkPIybBbnEWp0AjBskielwxbBWcDltQ7ZN2EWtIPLUsK0E PhXnHM5t7sMMXEx89tDkO6DH90vObHUf0EVM1sHKsE9s28EbKzvyKMnqlKo6ot8IstO1ebQ vm35dqnxOpiwqX9/Zr6QA== X-UI-Out-Filterresults: notjunk:1;V03:K0:CHqa4bEY/Qo=:Yv91mjC9g/auCFWcW2QCIi XR9SYOJMG4FzZw4uGF0lEf3PNV/DoeguP2g5ds2OZ5fOeEraocZbGfLvd0aCOdk+Kcx+t7u57 Ab/Y2d/+AKVuZ1K5YaytUUa3kT3Fb/w1BvQX1Lmbh70TzSDINoF+SmJKjjhEc3q3r6BC5b6A4 CPCgCwCvamG/Ol7Ob/z73Rm1pBf5jZl7GscJa87m714jezOV7oussZfD+h+K+M/p18YTTNgUK 84dE9/sJO/JX2lwDVYq8ELciYqBnT6gO/CVHDAQTHC8ZJcvPWMdaviVCI4UkyDykQw0PfxhSn Ic9In/aAL5DbBe177+/90lvMZtGhJIxXGE3boldsZ1yvhIMiy1OvEtg6XEmxeYPrzEYBThKU0 F9j9gR9g42hSXi/X1/MUiu+awwYe+pJ9eVE4EBu4TssbKEWZFgSpvZSs8Xf93zMDp2CBaZNkI b6sd3HFpMJY4Tnrt3j0UsGF9j1y/USf0E2P/yiV7WpYnzuGG6a8J7AkJZCUgTPKx72WivFKZr bfm0MmhriRK2oHZlNmdnsuOg1K6ujYYSlIQ+sp5psbMJRafkt18AcxEpI+74MUgw5mD5Bc36A +aNRSg2WpES8Z0UavOqZOPmIN1ZtP/iGoVV1XCdQvH99nYI+RVAHZ5rUq+FnBk5JBNZGCrUIH 6j9qN6sTbNzUNJsTQGVQp/sefew5AOfEIR9cmCVLtfAqUrala4MF2CCAoWWLppeyptZnrSG0E 2ZAqFg6aEY/uI2g+/L7PiONbY3vQpXq958ih8DiJb3CIOAG3uXDwWTQZ+h0xQ5o3J6hBZsIYD gZhrDh78f0A07gK1Du6dZYJgrx/A7iP/v0Uz8SDlW3I1IyLkfYlQZKTbw0chkrcgVb98zHv8a 5F6znsMIGSme7OSW0DvoTWfvo6Of4yttg0U+9tVx/QpelTbg/3R6ri+qk3Jnwmz4nTITsx+cB 9mOdLWgxwWCxUCqK4E6f0dSpeComUe2NoEc+kEX5OTQX07Vt+Jo27WGocVRPbyRBzmMl+eBbP v4ULBvP06yli6eZRawNOUSt8jglEoeYudUC7+g9gUpTdJOAb2B/MoDHIbLF89R0mZovDU+kGX lmmHi+a09fxZxGkhCf6aQXN0SnT3uYn7fO1Rye7WI6jtegPyDijMror1IcHzsaJLmos2H+rlg XABwpiKvL8UgOzRkmIT+fRx8q186KgM8c+aBAfNaUwfqN1YLzDsAwnjGIHt6LylzNxEbbYG5Z Ho/HPUa1L4TlCSpCi Received-SPF: pass client-ip=212.227.17.21; envelope-from=scdbackup@gmx.net; helo=mout.gmx.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/12 05:30:18 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] X-Spam_score_int: -37 X-Spam_score: -3.8 X-Spam_bar: --- X-Spam_report: (-3.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Jun 2020 09:30:24 -0000 Hi, (cc-ing bug-xorriso@gnu.org and the reporter of the problem.) I now have the Guix ISO which fails when created by grub-mkrescue with HFS+ tree. To my newest theory it is not about the number of files in a directory but about the total number of files in the tree and their name lengths. I wonder how to describe this limit to the users of grub-mkrescue. Maybe: grub-mkrescue for platforms I386_EFI, X86_64_EFI, and POWERPC_IEEE1275 has a limit on the number of files multiplied by their average name length. Beginning with about 300,000 files of usual name length expect a xorriso error libisofs: FAILURE : HFS+ map nodes aren't implemented plus a rather misleading error message libisofs: FAILURE : Too much files to mangle, cannot guarantee unique file names The limit can only be avoided by suppressing xorrisofs option -hfsplus. This can be done by using xorriso script frontend/grub-mkrescue-sed.sh with MKRESCUE_SED_MODE=3D"mbr_only" or MKRESCUE_SED_MODE=3D"gpt_appended= ". Another method is to add as last arguments of grub-mkrescue these two -- -hfsplus off in order to leave xorriso's mkisofs emulation mode and to disable HFS+ production by a generic xorriso command. I still hope for a clarifying comment by Vladimir Serbinenko. =2D--------------------------------------------------------------------- Reasoning: Riddling what is overflowing in hfsplus.c i found some description of HFS+ in https://developer.apple.com/library/archive/technotes/tn/tn1150.html Since hfsplus.c reports to need "map nodes" i assume that it is the "header node" which contains a "map record". Map nodes would then be data structures which contain more map records. So for now i believe the overflow is in the "B-tree Map Record" of the header node. "It is a bitmap that indicates which nodes in the B-tree are used and which are free. The bits are interpreted in the same way as the bits in the allocation file." The number in target->hfsp_nnodes which causes the overflow is 35487. This is not the number of files 434,920. The number in target->hfsp_nleafs is 869842 which is (434920+1)*2. The loop which accumulates the number of target->hfsp_nnodes iterates over this number of 869842. I guess that the 869842 leafs are planned to get stored in allocation blocks of target->hfsp_cat_node_size which i now know is 4096. Each leaf occupies 50 to 200 bytes in the allocation block. This roughly matches the ratio of 35487 * 4096 / 869842 =3D 167. So it seems to be about the number of files and the sum of their name lengths. The size limit gets exceeded by about 19,525,632 bytes or (estimated by above ratio) 116,920 files. Have a nice day :) Thomas