From: Alper Nebi Yasak <alpernebiyasak@gmail.com>
To: u-boot@lists.denx.de
Subject: [PATCH 16/25] binman: Avoid reporting image-pos with compression
Date: Tue, 27 Oct 2020 01:25:17 +0300 [thread overview]
Message-ID: <ca80ba98-8658-2663-abcd-6f71f8b67b01@gmail.com> (raw)
In-Reply-To: <CAPnjgZ3WSsdpDjpYDYFUmzrpi67eDEhtG=-GFHstYo7ZqTBPwA@mail.gmail.com>
On 26/10/2020 22:22, Simon Glass wrote:
> Hi Alper,
>
> On Mon, 19 Oct 2020 at 15:29, Alper Nebi Yasak <alpernebiyasak@gmail.com> wrote:
>>
>> On 19/10/2020 05:41, Simon Glass wrote:
>>> When a section is compressed, all entries within it are grouped together
>>> into a compressed block of data. This obscures the start of each
>>> individual child entry.
>>>
>>> Avoid reporting bogus 'image-pos' properties in this case, since it is
>>> not possible to access the entry at the location provided. The entire
>>> section must be decompressed first.
>>>
>>> CBFS does not support compressing whole sections, only individual files,
>>> so needs no special handling here.
>>>
>>> Signed-off-by: Simon Glass <sjg@chromium.org>
>>> ---
>>
>> Maybe instead of this, 'image-pos' could be overloaded to a list of
>> recursive relative positions within compressed archives? Something like:
>>
>> section {
>> offset = <1000>;
>> compress = "lz4";
>> /* image-pos = <1000>; */
>>
>> blob {
>> filename = "foo";
>> offset = <100>;
>> /* image-pos = <1000>, <100>; */
>> };
>> };
>>
>> cbfs {
>> offset = <2000>;
>> /* image-pos = <2000>; */
>>
>> blob {
>> filename = "bar";
>> cbfs-compress = "lz4";
>> cbfs-offset = <200>;
>> /* image-pos = <2200>, <0>; */
>> };
>>
>> blob2 {
>> filename = "baz";
>> cbfs-offset = <800>;
>> /* image-pos = <2800>; */
>> };
>> };
>>
>
> What do these values actually mean though? What would we use them for?
What I meant is using pairs of <position-of-compressed-archive>,
<position-in-uncompressed-data> to avoid losing position information of
compressed entries, but honestly I'm not sure if any of this will be
necessary. I think the single use would be to avoid parsing uncompressed
data containing multiple entries to identify what is where.
To get to an entry in a compressed section we could get "size" bytes of
data starting from "image-pos[0]", decompress that to somewhere, then
get "uncomp-size" bytes of data starting from decompression address +
"image-pos[1]". I think it can even be extended to multiple layers of
compression (<pos-of-x>, <pos-of-y-in-d(x)>, <pos-of-z-in-d(y)>, ...).
> Note that CBFS compresses its data on a per-entry basis so the
> image-pos is actually supported.
On the compressed cbfs example, I assume the data we currently get at
image-pos = <2200> is compressed data and still needs to be decompressed
for it to be useable (technically bogus?). If so, I'd go a level deeper
by adding a <0> indicating the start of the uncompressed data is the
actual position of the entry contents.
next prev parent reply other threads:[~2020-10-26 22:25 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-19 2:41 [PATCH 00/25] binman: Support compression of sections Simon Glass
2020-10-19 2:41 ` [PATCH 01/25] binman: Give a sensible error if no command is given Simon Glass
2020-10-19 2:41 ` [PATCH 02/25] binman: Fix return from u-boot-ucode if there is no DT Simon Glass
2020-10-19 2:41 ` [PATCH 03/25] binman: Remove references to 'image' in entry_Section Simon Glass
2020-10-19 2:41 ` [PATCH 04/25] binman: Expand the error message for breaching a section Simon Glass
2020-10-19 2:41 ` [PATCH 05/25] binman: Move CompressData() into Entry base class Simon Glass
2020-10-19 2:41 ` [PATCH 06/25] binman: Use 'files-compress' to set compression for files Simon Glass
2020-10-19 19:01 ` Alper Nebi Yasak
2020-10-19 2:41 ` [PATCH 07/25] binman: Update testPackExtra with more checks Simon Glass
2020-10-19 2:41 ` [PATCH 08/25] binman: Expand docs and test for padding Simon Glass
2020-10-19 2:41 ` [PATCH 09/25] binman: Expand docs and test for alignment Simon Glass
2020-10-19 2:41 ` [PATCH 10/25] binman: Move section-building code into a function Simon Glass
2020-10-19 20:30 ` Alper Nebi Yasak
2020-10-26 19:22 ` Simon Glass
2020-10-26 23:17 ` Alper Nebi Yasak
2020-11-03 15:11 ` Simon Glass
2020-11-04 21:50 ` Alper Nebi Yasak
2021-01-23 16:15 ` Simon Glass
2020-10-19 2:41 ` [PATCH 11/25] binman: Refactor _BuildSectionData() Simon Glass
2020-10-19 2:41 ` [PATCH 12/25] binman: Move section padding to the parent Simon Glass
2020-10-19 2:41 ` [PATCH 13/25] binman: Make section padding consistent with other entries Simon Glass
2020-10-19 2:41 ` [PATCH 14/25] binman: Store the original data before compression Simon Glass
2020-10-19 2:41 ` [PATCH 15/25] binman: Set section contents in GetData() Simon Glass
2020-10-19 2:41 ` [PATCH 16/25] binman: Avoid reporting image-pos with compression Simon Glass
2020-10-19 21:15 ` Alper Nebi Yasak
2020-10-26 19:22 ` Simon Glass
2020-10-26 22:25 ` Alper Nebi Yasak [this message]
2020-10-30 18:15 ` Simon Glass
2020-11-04 19:45 ` Alper Nebi Yasak
2020-10-19 2:41 ` [PATCH 17/25] binman: Drop Entry.CheckOffset() Simon Glass
2020-10-19 2:41 ` [PATCH 18/25] binman: Move sort and expand to the main Pack() function Simon Glass
2020-10-19 2:41 ` [PATCH 19/25] binman: Drop the Entry.CheckSize() method Simon Glass
2020-10-19 2:41 ` [PATCH 20/25] binman: Call CheckSize() from the section's Pack() method Simon Glass
2020-10-19 2:41 ` [PATCH 21/25] binman: Drop CheckEntries() Simon Glass
2020-10-19 2:41 ` [PATCH 22/25] binman: Update CheckEntries() for compressed sections Simon Glass
2020-10-19 2:41 ` [PATCH 23/25] binman: Use the actual contents in CheckSize() Simon Glass
2020-10-19 2:41 ` [PATCH 24/25] binman: Support compression of sections Simon Glass
2020-10-19 2:41 ` [PATCH 25/25] binman: Avoid calculated section data repeatedly Simon Glass
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ca80ba98-8658-2663-abcd-6f71f8b67b01@gmail.com \
--to=alpernebiyasak@gmail.com \
--cc=u-boot@lists.denx.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox