[U-Boot] [PATCH] lib:crc32: Allow setting of the initial crc32 value

[U-Boot] [PATCH] lib:crc32: Allow setting of the initial crc32 value

[Lukasz Majewski]

The current approach set the initial value of crc32 calculation to zero,
which is correct for calculating checksum of the whole chunk of data.

It however, lacks the flexibility, when one wants to calculate CRC32 of
a file comprised of many smaller parts received separately.

In the proposed approach the output value is used as a starting condition
for the proper crc32 calculation at crc32_wd function. This behavior is
identical to the one provided by crc32() method implementation.

Additionally, comments were appropriately updated.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Cc: Marek Vasut <marex@denx.de>
Cc: Simon Glass <sjg@chromium.org>
---
 include/hash.h       |    2 +-
 include/u-boot/crc.h |    3 ++-
 lib/crc32.c          |    2 +-
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/include/hash.h b/include/hash.h
index dc21678..abf704d 100644
--- a/include/hash.h
+++ b/include/hash.h
@@ -101,7 +101,7 @@ int hash_command(const char *algo_name, int flags, cmd_tbl_t *cmdtp, int flag,
  * @algo_name:		Hash algorithm to use
  * @data:		Data to hash
  * @len:		Lengh of data to hash in bytes
- * @output:		Place to put hash value
+ * @output:		Place to put hash value - also the initial value (crc32)
  * @output_size:	On entry, pointer to the number of bytes available in
  *			output. On exit, pointer to the number of bytes used.
  *			If NULL, then it is assumed that the caller has
diff --git a/include/u-boot/crc.h b/include/u-boot/crc.h
index 754ac72..7a87911 100644
--- a/include/u-boot/crc.h
+++ b/include/u-boot/crc.h
@@ -19,7 +19,8 @@ uint32_t crc32_no_comp (uint32_t, const unsigned char *, uint);
  *
  * @input:	Input buffer
  * @ilen:	Input buffer length
- * @output:	Place to put checksum result (4 bytes)
+ * @output:	Place to provide initial CRC32 value and afterwards
+ *		put checksum result (4 bytes)
  * @chunk_sz:	Trigger watchdog after processing this many bytes
  */
 void crc32_wd_buf(const unsigned char *input, uint ilen,
diff --git a/lib/crc32.c b/lib/crc32.c
index 9759212..f6266c7 100644
--- a/lib/crc32.c
+++ b/lib/crc32.c
@@ -257,7 +257,7 @@ void crc32_wd_buf(const unsigned char *input, unsigned int ilen,
 {
 	uint32_t crc;
 
-	crc = crc32_wd(0, input, ilen, chunk_sz);
+	crc = crc32_wd(*(uint32_t *)output, input, ilen, chunk_sz);
 	crc = htonl(crc);
 	memcpy(output, &crc, sizeof(crc));
 }
-- 
1.7.10.4

[U-Boot] [PATCH] lib:crc32: Allow setting of the initial crc32 value

[Wolfgang Denk]

Dear Lukasz,

In message <1399295277-28334-1-git-send-email-l.majewski@samsung.com> you wrote:
> The current approach set the initial value of crc32 calculation to zero,
> which is correct for calculating checksum of the whole chunk of data.
> 
> It however, lacks the flexibility, when one wants to calculate CRC32 of
> a file comprised of many smaller parts received separately.
> 
> In the proposed approach the output value is used as a starting condition
> for the proper crc32 calculation at crc32_wd function. This behavior is
> identical to the one provided by crc32() method implementation.
> 
> Additionally, comments were appropriately updated.
> 
> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Simon Glass <sjg@chromium.org>
> ---
>  include/hash.h       |    2 +-
>  include/u-boot/crc.h |    3 ++-
>  lib/crc32.c          |    2 +-
>  3 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/include/hash.h b/include/hash.h
> index dc21678..abf704d 100644
> --- a/include/hash.h
> +++ b/include/hash.h
> @@ -101,7 +101,7 @@ int hash_command(const char *algo_name, int flags, cmd_tbl_t *cmdtp, int flag,
>   * @algo_name:		Hash algorithm to use
>   * @data:		Data to hash
>   * @len:		Lengh of data to hash in bytes
> - * @output:		Place to put hash value
> + * @output:		Place to put hash value - also the initial value (crc32)
>   * @output_size:	On entry, pointer to the number of bytes available in
>   *			output. On exit, pointer to the number of bytes used.
>   *			If NULL, then it is assumed that the caller has
> diff --git a/include/u-boot/crc.h b/include/u-boot/crc.h
> index 754ac72..7a87911 100644
> --- a/include/u-boot/crc.h
> +++ b/include/u-boot/crc.h
> @@ -19,7 +19,8 @@ uint32_t crc32_no_comp (uint32_t, const unsigned char *, uint);
>   *
>   * @input:	Input buffer
>   * @ilen:	Input buffer length
> - * @output:	Place to put checksum result (4 bytes)
> + * @output:	Place to provide initial CRC32 value and afterwards
> + *		put checksum result (4 bytes)
>   * @chunk_sz:	Trigger watchdog after processing this many bytes
>   */
>  void crc32_wd_buf(const unsigned char *input, uint ilen,
> diff --git a/lib/crc32.c b/lib/crc32.c
> index 9759212..f6266c7 100644
> --- a/lib/crc32.c
> +++ b/lib/crc32.c
> @@ -257,7 +257,7 @@ void crc32_wd_buf(const unsigned char *input, unsigned int ilen,
>  {
>  	uint32_t crc;
>  
> -	crc = crc32_wd(0, input, ilen, chunk_sz);
> +	crc = crc32_wd(*(uint32_t *)output, input, ilen, chunk_sz);
>  	crc = htonl(crc);
>  	memcpy(output, &crc, sizeof(crc));

This looks wrong to me, in a number of ways.

First, the *(uint32_t *)output cast, is likely to trigger unaligned
accesses with the resulting problems on some platforms.  Never, never
ever cast a character pointer into something that requires any
alignment!

Seconds, where exactly do you now initialize the CRC vlaue to start
with 0 ?


Finally we should keep in mind (this is nothing caused by your patch,
but when touching this area we really should consider it) that we have
a number of (slightly) different CRC implementations thare cry for
cleanup / unification: in addition to lib/crc32.c we have
disk/part_efi.c (which provides efi_crc32()), drivers/mtd/ubi/crc32.c
(which provides crc32_le() and crc32_be()), net/eth.c (which uses
ether_crc()), we have BZ2_crc32Table[256] in lib/bzlib_private.h /
lib/bzlib_crctable.c (which appears to be unsued), and we have
tools/pblimage.c (which provides pbl_crc32()).

What a mess :-(

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
What is tolerance? -- it is the consequence of humanity. We  are  all
formed  of frailty and error; let us pardon reciprocally each other's
folly -- that is the first law of nature.                  - Voltaire

[U-Boot] [PATCH] lib:crc32: Allow setting of the initial crc32 value

[Marek Vasut]

On Monday, May 05, 2014 at 07:47:30 PM, Wolfgang Denk wrote:
> Dear Lukasz,
> 
> In message <1399295277-28334-1-git-send-email-l.majewski@samsung.com> you 
wrote:
> > The current approach set the initial value of crc32 calculation to zero,
> > which is correct for calculating checksum of the whole chunk of data.
> > 
> > It however, lacks the flexibility, when one wants to calculate CRC32 of
> > a file comprised of many smaller parts received separately.
> > 
> > In the proposed approach the output value is used as a starting condition
> > for the proper crc32 calculation at crc32_wd function. This behavior is
> > identical to the one provided by crc32() method implementation.
> > 
> > Additionally, comments were appropriately updated.
> > 
> > Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> > Cc: Marek Vasut <marex@denx.de>
> > Cc: Simon Glass <sjg@chromium.org>
> > ---
> > 
> >  include/hash.h       |    2 +-
> >  include/u-boot/crc.h |    3 ++-
> >  lib/crc32.c          |    2 +-
> >  3 files changed, 4 insertions(+), 3 deletions(-)
> > 
> > diff --git a/include/hash.h b/include/hash.h
> > index dc21678..abf704d 100644
> > --- a/include/hash.h
> > +++ b/include/hash.h
> > @@ -101,7 +101,7 @@ int hash_command(const char *algo_name, int flags,
> > cmd_tbl_t *cmdtp, int flag,
> > 
> >   * @algo_name:		Hash algorithm to use
> >   * @data:		Data to hash
> >   * @len:		Lengh of data to hash in bytes
> > 
> > - * @output:		Place to put hash value
> > + * @output:		Place to put hash value - also the initial value 
(crc32)
> > 
> >   * @output_size:	On entry, pointer to the number of bytes available in
> >   *			output. On exit, pointer to the number of bytes used.
> >   *			If NULL, then it is assumed that the caller has
> > 
> > diff --git a/include/u-boot/crc.h b/include/u-boot/crc.h
> > index 754ac72..7a87911 100644
> > --- a/include/u-boot/crc.h
> > +++ b/include/u-boot/crc.h
> > @@ -19,7 +19,8 @@ uint32_t crc32_no_comp (uint32_t, const unsigned char
> > *, uint);
> > 
> >   *
> >   * @input:	Input buffer
> >   * @ilen:	Input buffer length
> > 
> > - * @output:	Place to put checksum result (4 bytes)
> > + * @output:	Place to provide initial CRC32 value and afterwards
> > + *		put checksum result (4 bytes)
> > 
> >   * @chunk_sz:	Trigger watchdog after processing this many bytes
> >   */
> >  
> >  void crc32_wd_buf(const unsigned char *input, uint ilen,
> > 
> > diff --git a/lib/crc32.c b/lib/crc32.c
> > index 9759212..f6266c7 100644
> > --- a/lib/crc32.c
> > +++ b/lib/crc32.c
> > @@ -257,7 +257,7 @@ void crc32_wd_buf(const unsigned char *input,
> > unsigned int ilen,
> > 
> >  {
> >  
> >  	uint32_t crc;
> > 
> > -	crc = crc32_wd(0, input, ilen, chunk_sz);
> > +	crc = crc32_wd(*(uint32_t *)output, input, ilen, chunk_sz);
> > 
> >  	crc = htonl(crc);
> >  	memcpy(output, &crc, sizeof(crc));
> 
> This looks wrong to me, in a number of ways.
> 
> First, the *(uint32_t *)output cast, is likely to trigger unaligned
> accesses with the resulting problems on some platforms.  Never, never
> ever cast a character pointer into something that requires any
> alignment!
> 
> Seconds, where exactly do you now initialize the CRC vlaue to start
> with 0 ?
> 
> 
> Finally we should keep in mind (this is nothing caused by your patch,
> but when touching this area we really should consider it) that we have
> a number of (slightly) different CRC implementations thare cry for
> cleanup / unification: in addition to lib/crc32.c we have
> disk/part_efi.c (which provides efi_crc32()), drivers/mtd/ubi/crc32.c
> (which provides crc32_le() and crc32_be()), net/eth.c (which uses
> ether_crc()), we have BZ2_crc32Table[256] in lib/bzlib_private.h /
> lib/bzlib_crctable.c (which appears to be unsued), and we have
> tools/pblimage.c (which provides pbl_crc32()).

Just an addition ... All those different implementations should use the hash_*() 
calls and this one central implementation.

Thanks for pointing this part out.

[U-Boot] [PATCH] lib:crc32: Allow setting of the initial crc32 value

[Lukasz Majewski]

Hi Wolfgang,

> Dear Lukasz,
> 
> In message <1399295277-28334-1-git-send-email-l.majewski@samsung.com>
> you wrote:
> > The current approach set the initial value of crc32 calculation to
> > zero, which is correct for calculating checksum of the whole chunk
> > of data.
> > 
> > It however, lacks the flexibility, when one wants to calculate
> > CRC32 of a file comprised of many smaller parts received separately.
> > 
> > In the proposed approach the output value is used as a starting
> > condition for the proper crc32 calculation at crc32_wd function.
> > This behavior is identical to the one provided by crc32() method
> > implementation.
> > 
> > Additionally, comments were appropriately updated.
> > 
> > Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> > Cc: Marek Vasut <marex@denx.de>
> > Cc: Simon Glass <sjg@chromium.org>
> > ---
> >  include/hash.h       |    2 +-
> >  include/u-boot/crc.h |    3 ++-
> >  lib/crc32.c          |    2 +-
> >  3 files changed, 4 insertions(+), 3 deletions(-)
> > 
> > diff --git a/include/hash.h b/include/hash.h
> > index dc21678..abf704d 100644
> > --- a/include/hash.h
> > +++ b/include/hash.h
> > @@ -101,7 +101,7 @@ int hash_command(const char *algo_name, int
> > flags, cmd_tbl_t *cmdtp, int flag,
> >   * @algo_name:		Hash algorithm to use
> >   * @data:		Data to hash
> >   * @len:		Lengh of data to hash in bytes
> > - * @output:		Place to put hash value
> > + * @output:		Place to put hash value - also the
> > initial value (crc32)
> >   * @output_size:	On entry, pointer to the number of bytes
> > available in
> >   *			output. On exit, pointer to the number
> > of bytes used.
> >   *			If NULL, then it is assumed that the
> > caller has diff --git a/include/u-boot/crc.h b/include/u-boot/crc.h
> > index 754ac72..7a87911 100644
> > --- a/include/u-boot/crc.h
> > +++ b/include/u-boot/crc.h
> > @@ -19,7 +19,8 @@ uint32_t crc32_no_comp (uint32_t, const unsigned
> > char *, uint); *
> >   * @input:	Input buffer
> >   * @ilen:	Input buffer length
> > - * @output:	Place to put checksum result (4 bytes)
> > + * @output:	Place to provide initial CRC32 value and
> > afterwards
> > + *		put checksum result (4 bytes)
> >   * @chunk_sz:	Trigger watchdog after processing this many
> > bytes */
> >  void crc32_wd_buf(const unsigned char *input, uint ilen,
> > diff --git a/lib/crc32.c b/lib/crc32.c
> > index 9759212..f6266c7 100644
> > --- a/lib/crc32.c
> > +++ b/lib/crc32.c
> > @@ -257,7 +257,7 @@ void crc32_wd_buf(const unsigned char *input,
> > unsigned int ilen, {
> >  	uint32_t crc;
> >  
> > -	crc = crc32_wd(0, input, ilen, chunk_sz);
> > +	crc = crc32_wd(*(uint32_t *)output, input, ilen, chunk_sz);
> >  	crc = htonl(crc);
> >  	memcpy(output, &crc, sizeof(crc));
> 
> This looks wrong to me, in a number of ways.
> 
> First, the *(uint32_t *)output cast, is likely to trigger unaligned
> accesses with the resulting problems on some platforms.  Never, never
> ever cast a character pointer into something that requires any
> alignment!

I admit that I was thinking about my (particular) platform.

Instead, I should memcpy the output to crc variable, which is defined as
uint32_t, and pass it to the crc32_wd.

> 
> Seconds, where exactly do you now initialize the CRC vlaue to start
> with 0 ?

The proposed approach (with setting initial value of CRC32) is working
fine with crc32() function at least in the DFU. Zeroing out of relevant
variable is performed there.

The venerable crc32() implementation allows passing initial value of
crc. As it is now, the hash_block() doesn't.

> 
> 
> Finally we should keep in mind (this is nothing caused by your patch,
> but when touching this area we really should consider it) that we have
> a number of (slightly) different CRC implementations thare cry for
> cleanup / unification: in addition to lib/crc32.c we have
> disk/part_efi.c (which provides efi_crc32()), drivers/mtd/ubi/crc32.c
> (which provides crc32_le() and crc32_be()), net/eth.c (which uses
> ether_crc()), we have BZ2_crc32Table[256] in lib/bzlib_private.h /
> lib/bzlib_crctable.c (which appears to be unsued), and we have
> tools/pblimage.c (which provides pbl_crc32()).

As Marek pointed out, we shall start using hash_block (from #include
<hash.h>).

However, as I pointed out in this patch hash_block (at least for crc32)
needs to be modified to preserve the functionality of bare metal
crc32() function.

To unify, we also may be forced to introduce some flags - like output
crc endianess (big, little). But this may wait for the moment.

> 
> What a mess :-(

+1

> 
> Best regards,
> 
> Wolfgang Denk
> 



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [PATCH] lib:crc32: Allow setting of the initial crc32 value

[Marek Vasut]

On Monday, May 05, 2014 at 03:07:57 PM, Lukasz Majewski wrote:
> The current approach set the initial value of crc32 calculation to zero,
> which is correct for calculating checksum of the whole chunk of data.
> 
> It however, lacks the flexibility, when one wants to calculate CRC32 of
> a file comprised of many smaller parts received separately.
> 
> In the proposed approach the output value is used as a starting condition
> for the proper crc32 calculation at crc32_wd function. This behavior is
> identical to the one provided by crc32() method implementation.
> 
> Additionally, comments were appropriately updated.
> 
> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
> Cc: Marek Vasut <marex@denx.de>
> Cc: Simon Glass <sjg@chromium.org>
> ---
>  include/hash.h       |    2 +-
>  include/u-boot/crc.h |    3 ++-
>  lib/crc32.c          |    2 +-
>  3 files changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/include/hash.h b/include/hash.h
> index dc21678..abf704d 100644
> --- a/include/hash.h
> +++ b/include/hash.h
> @@ -101,7 +101,7 @@ int hash_command(const char *algo_name, int flags,
> cmd_tbl_t *cmdtp, int flag, * @algo_name:		Hash algorithm to use
>   * @data:		Data to hash
>   * @len:		Lengh of data to hash in bytes
> - * @output:		Place to put hash value
> + * @output:		Place to put hash value - also the initial value (crc32)
>   * @output_size:	On entry, pointer to the number of bytes available in
>   *			output. On exit, pointer to the number of bytes used.
>   *			If NULL, then it is assumed that the caller has
> diff --git a/include/u-boot/crc.h b/include/u-boot/crc.h
> index 754ac72..7a87911 100644
> --- a/include/u-boot/crc.h
> +++ b/include/u-boot/crc.h
> @@ -19,7 +19,8 @@ uint32_t crc32_no_comp (uint32_t, const unsigned char *,
> uint); *
>   * @input:	Input buffer
>   * @ilen:	Input buffer length
> - * @output:	Place to put checksum result (4 bytes)
> + * @output:	Place to provide initial CRC32 value and afterwards
> + *		put checksum result (4 bytes)
>   * @chunk_sz:	Trigger watchdog after processing this many bytes
>   */
>  void crc32_wd_buf(const unsigned char *input, uint ilen,
> diff --git a/lib/crc32.c b/lib/crc32.c
> index 9759212..f6266c7 100644
> --- a/lib/crc32.c
> +++ b/lib/crc32.c
> @@ -257,7 +257,7 @@ void crc32_wd_buf(const unsigned char *input, unsigned
> int ilen, {
>  	uint32_t crc;
> 
> -	crc = crc32_wd(0, input, ilen, chunk_sz);
> +	crc = crc32_wd(*(uint32_t *)output, input, ilen, chunk_sz);

I sense alignment fault here.
[...]

Best regards,
Marek Vasut

[U-Boot] [PATCH v2] lib:crc32: Allow setting of the initial crc32 value

[Lukasz Majewski]

The current approach set the initial value of crc32 calculation to zero,
which is correct for calculating checksum of the whole chunk of data.

It however, lacks the flexibility, when one wants to calculate CRC32 of
a file comprised of many smaller parts received separately.

In the proposed approach the output value is used as a starting condition
for the proper crc32 calculation at crc32_wd function. This behavior is
identical to the one provided by crc32() method implementation.

Additionally comments were appropriately updated.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Cc: Marek Vasut <marex@denx.de>
---
Changes for v2:
- Replace casting from (u8*) to (u32*) with memcpy
---
 include/hash.h       |    2 +-
 include/u-boot/crc.h |    3 ++-
 lib/crc32.c          |    7 +++++--
 3 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/include/hash.h b/include/hash.h
index dc21678..abf704d 100644
--- a/include/hash.h
+++ b/include/hash.h
@@ -101,7 +101,7 @@ int hash_command(const char *algo_name, int flags, cmd_tbl_t *cmdtp, int flag,
  * @algo_name:		Hash algorithm to use
  * @data:		Data to hash
  * @len:		Lengh of data to hash in bytes
- * @output:		Place to put hash value
+ * @output:		Place to put hash value - also the initial value (crc32)
  * @output_size:	On entry, pointer to the number of bytes available in
  *			output. On exit, pointer to the number of bytes used.
  *			If NULL, then it is assumed that the caller has
diff --git a/include/u-boot/crc.h b/include/u-boot/crc.h
index 754ac72..7a87911 100644
--- a/include/u-boot/crc.h
+++ b/include/u-boot/crc.h
@@ -19,7 +19,8 @@ uint32_t crc32_no_comp (uint32_t, const unsigned char *, uint);
  *
  * @input:	Input buffer
  * @ilen:	Input buffer length
- * @output:	Place to put checksum result (4 bytes)
+ * @output:	Place to provide initial CRC32 value and afterwards
+ *		put checksum result (4 bytes)
  * @chunk_sz:	Trigger watchdog after processing this many bytes
  */
 void crc32_wd_buf(const unsigned char *input, uint ilen,
diff --git a/lib/crc32.c b/lib/crc32.c
index 9759212..80e078f 100644
--- a/lib/crc32.c
+++ b/lib/crc32.c
@@ -255,9 +255,12 @@ uint32_t ZEXPORT crc32_wd (uint32_t crc,
 void crc32_wd_buf(const unsigned char *input, unsigned int ilen,
 		unsigned char *output, unsigned int chunk_sz)
 {
-	uint32_t crc;
+	uint32_t crc = 0;
 
-	crc = crc32_wd(0, input, ilen, chunk_sz);
+	if (*output)
+		memcpy(&crc, output, sizeof(crc));
+
+	crc = crc32_wd(crc, input, ilen, chunk_sz);
 	crc = htonl(crc);
 	memcpy(output, &crc, sizeof(crc));
 }
-- 
1.7.10.4

[U-Boot] [PATCH v2] lib:crc32: Allow setting of the initial crc32 value

[Marek Vasut]

On Wednesday, May 07, 2014 at 08:10:20 AM, Lukasz Majewski wrote:

[...]

> --- a/lib/crc32.c
> +++ b/lib/crc32.c
> @@ -255,9 +255,12 @@ uint32_t ZEXPORT crc32_wd (uint32_t crc,
>  void crc32_wd_buf(const unsigned char *input, unsigned int ilen,
>  		unsigned char *output, unsigned int chunk_sz)
>  {
> -	uint32_t crc;
> +	uint32_t crc = 0;
> 
> -	crc = crc32_wd(0, input, ilen, chunk_sz);
> +	if (*output)
> +		memcpy(&crc, output, sizeof(crc));

Won't some sort of put_unaligned() work here ? The $crc is uint32_t afterall, so 
it might be a jiff faster. Please correct me if I'm wrong.

> +
> +	crc = crc32_wd(crc, input, ilen, chunk_sz);
>  	crc = htonl(crc);
>  	memcpy(output, &crc, sizeof(crc));
>  }

Best regards,
Marek Vasut

[U-Boot] [PATCH v2] lib:crc32: Allow setting of the initial crc32 value

[Lukasz Majewski]

Hi Marek,

> On Wednesday, May 07, 2014 at 08:10:20 AM, Lukasz Majewski wrote:
> 
> [...]
> 
> > --- a/lib/crc32.c
> > +++ b/lib/crc32.c
> > @@ -255,9 +255,12 @@ uint32_t ZEXPORT crc32_wd (uint32_t crc,
> >  void crc32_wd_buf(const unsigned char *input, unsigned int ilen,
> >  		unsigned char *output, unsigned int chunk_sz)
> >  {
> > -	uint32_t crc;
> > +	uint32_t crc = 0;
> > 
> > -	crc = crc32_wd(0, input, ilen, chunk_sz);
> > +	if (*output)
> > +		memcpy(&crc, output, sizeof(crc));
> 
> Won't some sort of put_unaligned() work here ? The $crc is uint32_t
> afterall, so it might be a jiff faster. 

We are concerned here with the use case of copying 4 bytes from
unaligned buffer defined on some architectures. 

I suppose, that the performance would be the same for both. 
However, since memcpy() is already used in this function, I would
prefer to use it here.

> Please correct me if I'm
> wrong.
> 
> > +
> > +	crc = crc32_wd(crc, input, ilen, chunk_sz);
> >  	crc = htonl(crc);
> >  	memcpy(output, &crc, sizeof(crc));
> >  }
> 
> Best regards,
> Marek Vasut


-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [PATCH v2] lib:crc32: Allow setting of the initial crc32 value

[Wolfgang Denk]

Dear Lukasz Majewski,

In message <1399443021-11748-1-git-send-email-l.majewski@samsung.com> you wrote:
> The current approach set the initial value of crc32 calculation to zero,
> which is correct for calculating checksum of the whole chunk of data.
... 
> +	if (*output)
> +		memcpy(&crc, output, sizeof(crc));
> +
> +	crc = crc32_wd(crc, input, ilen, chunk_sz);
>  	crc = htonl(crc);
>  	memcpy(output, &crc, sizeof(crc));

You can actually remove the "if (*output)" because output has always
to be a non-null pointer, as we're going to store the result there.

Which means that you cannot use this to implicitly initialize crc =0,
whichin turn means you MUST add porper initialization to all callers
of that function.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Die Freiheit des Menschen liegt nicht darin, dass er tun kann, was er
will, sondern darin, dass er nicht tun muss, was er nicht will.
                                             -- Jean-Jacques Rousseau

[U-Boot] [PATCH v2] lib:crc32: Allow setting of the initial crc32 value

[Lukasz Majewski]

Hi Wolfgang,

> Dear Lukasz Majewski,
> 
> In message <1399443021-11748-1-git-send-email-l.majewski@samsung.com>
> you wrote:
> > The current approach set the initial value of crc32 calculation to
> > zero, which is correct for calculating checksum of the whole chunk
> > of data.
> ... 
> > +	if (*output)
> > +		memcpy(&crc, output, sizeof(crc));
> > +
> > +	crc = crc32_wd(crc, input, ilen, chunk_sz);
> >  	crc = htonl(crc);
> >  	memcpy(output, &crc, sizeof(crc));
> 
> You can actually remove the "if (*output)" because output has always
> to be a non-null pointer, as we're going to store the result there.

I think, that the above statement would be correct if I had checked the
if (output).

The problem here is that *output refers to uint8 and only first/last
byte is checked. This is obviously wrong.

You are right that this check is not needed.

> 
> Which means that you cannot use this to implicitly initialize crc =0,
> whichin turn means you MUST add porper initialization to all callers
> of that function.

Ok.

> 
> Best regards,
> 
> Wolfgang Denk
> 



-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group

[U-Boot] [PATCH v3] lib:crc32:hash: Allow setting of the initial crc32 value

[Lukasz Majewski]

The current approach set the initial value of crc32 calculation to zero,
which is correct for calculating checksum of the whole chunk of data.

It however, lacks the flexibility, when one wants to calculate CRC32 of
a file comprised of many smaller parts received separately.

In the proposed approach the output value is used as a starting condition
for the proper crc32 calculation at crc32_wd function. This behavior is
identical to the one provided by crc32() method implementation.

Additionally comments were appropriately updated.

Signed-off-by: Lukasz Majewski <l.majewski@samsung.com>
Cc: Marek Vasut <marex@denx.de>
---
Changes for v2:
- Replace casting from (u8*) to (u32*) with memcpy
Changes for v3:
- Remove check on the output pointer
- As precaution, zero out the output buffer
---
 common/hash.c        |    6 ++++++
 include/hash.h       |    2 +-
 include/u-boot/crc.h |    3 ++-
 lib/crc32.c          |    3 ++-
 4 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/common/hash.c b/common/hash.c
index 7627b84..84c0a78 100644
--- a/common/hash.c
+++ b/common/hash.c
@@ -360,6 +360,12 @@ int hash_command(const char *algo_name, int flags, cmd_tbl_t *cmdtp, int flag,
 		u8 vsum[HASH_MAX_DIGEST_SIZE];
 		void *buf;
 
+		/*
+		 * This is a precaution for crc32, which allows output to be
+		 * overloaded to provide initial value of crc32.
+		 */
+		memset(output, 0, sizeof(output));
+
 		if (hash_lookup_algo(algo_name, &algo)) {
 			printf("Unknown hash algorithm '%s'\n", algo_name);
 			return CMD_RET_USAGE;
diff --git a/include/hash.h b/include/hash.h
index dc21678..abf704d 100644
--- a/include/hash.h
+++ b/include/hash.h
@@ -101,7 +101,7 @@ int hash_command(const char *algo_name, int flags, cmd_tbl_t *cmdtp, int flag,
  * @algo_name:		Hash algorithm to use
  * @data:		Data to hash
  * @len:		Lengh of data to hash in bytes
- * @output:		Place to put hash value
+ * @output:		Place to put hash value - also the initial value (crc32)
  * @output_size:	On entry, pointer to the number of bytes available in
  *			output. On exit, pointer to the number of bytes used.
  *			If NULL, then it is assumed that the caller has
diff --git a/include/u-boot/crc.h b/include/u-boot/crc.h
index 754ac72..7a87911 100644
--- a/include/u-boot/crc.h
+++ b/include/u-boot/crc.h
@@ -19,7 +19,8 @@ uint32_t crc32_no_comp (uint32_t, const unsigned char *, uint);
  *
  * @input:	Input buffer
  * @ilen:	Input buffer length
- * @output:	Place to put checksum result (4 bytes)
+ * @output:	Place to provide initial CRC32 value and afterwards
+ *		put checksum result (4 bytes)
  * @chunk_sz:	Trigger watchdog after processing this many bytes
  */
 void crc32_wd_buf(const unsigned char *input, uint ilen,
diff --git a/lib/crc32.c b/lib/crc32.c
index 9759212..f57eb87 100644
--- a/lib/crc32.c
+++ b/lib/crc32.c
@@ -257,7 +257,8 @@ void crc32_wd_buf(const unsigned char *input, unsigned int ilen,
 {
 	uint32_t crc;
 
-	crc = crc32_wd(0, input, ilen, chunk_sz);
+	memcpy(&crc, output, sizeof(crc));
+	crc = crc32_wd(crc, input, ilen, chunk_sz);
 	crc = htonl(crc);
 	memcpy(output, &crc, sizeof(crc));
 }
-- 
1.7.10.4

[U-Boot] [PATCH v3] lib:crc32:hash: Allow setting of the initial crc32 value

[Simon Glass]

Hi Lukasz,

On 7 May 2014 06:57, Lukasz Majewski <l.majewski@samsung.com> wrote:
>
> The current approach set the initial value of crc32 calculation to zero,
> which is correct for calculating checksum of the whole chunk of data.
>
> It however, lacks the flexibility, when one wants to calculate CRC32 of
> a file comprised of many smaller parts received separately.
>
> In the proposed approach the output value is used as a starting condition
> for the proper crc32 calculation at crc32_wd function. This behavior is
> identical to the one provided by crc32() method implementation.
>
> Additionally comments were appropriately updated.

Maybe I am missing something, but this doesn't seem necessary. In hash.h we have

hash_init()
hash_update()
hash_finish()

which permits you to pass more data through a hash function. Doesn't
this already do what you want?

What is missing is probably command-line access to this API. Something like:

hash init <envvar>, <algo>
hash update  <envvar>, <data>, <size>
hash finish <envvar>, [*]<result>

or similar.

Regards,
Simon

[U-Boot] [PATCH v3] lib:crc32:hash: Allow setting of the initial crc32 value

[Lukasz Majewski]

Hi Simon,

> Hi Lukasz,
> 
> On 7 May 2014 06:57, Lukasz Majewski <l.majewski@samsung.com> wrote:
> >
> > The current approach set the initial value of crc32 calculation to
> > zero, which is correct for calculating checksum of the whole chunk
> > of data.
> >
> > It however, lacks the flexibility, when one wants to calculate
> > CRC32 of a file comprised of many smaller parts received separately.
> >
> > In the proposed approach the output value is used as a starting
> > condition for the proper crc32 calculation at crc32_wd function.
> > This behavior is identical to the one provided by crc32() method
> > implementation.
> >
> > Additionally comments were appropriately updated.
> 
> Maybe I am missing something, but this doesn't seem necessary. In
> hash.h we have
> 
> hash_init()
> hash_update()
> hash_finish()
> 
> which permits you to pass more data through a hash function. Doesn't
> this already do what you want?

I thought, that I would get away with replacing crc32() function call
with similar one - hash_block(). As it was pointed out it doesn't
itself provide the same functionality.

However, I will try to implement the solution you suggested. Thanks for
tip.


> 
> What is missing is probably command-line access to this API.
> Something like:
> 
> hash init <envvar>, <algo>
> hash update  <envvar>, <data>, <size>
> hash finish <envvar>, [*]<result>
> 
> or similar.
> 
> Regards,
> Simon


-- 
Best regards,

Lukasz Majewski

Samsung R&D Institute Poland (SRPOL) | Linux Platform Group