From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5D650C433F5 for ; Mon, 10 Jan 2022 03:20:17 +0000 (UTC) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1C5E3833A2; Mon, 10 Jan 2022 04:16:45 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; unprotected) header.d=chromium.org header.i=@chromium.org header.b="e5sPjPh7"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id D0BDF830D0; Mon, 10 Jan 2022 04:15:55 +0100 (CET) Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id E3AAD830EA for ; Mon, 10 Jan 2022 04:15:27 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=sjg@chromium.org Received: by mail-io1-xd35.google.com with SMTP id y18so15770659iob.8 for ; Sun, 09 Jan 2022 19:15:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=NIVeSMi7YzgNT8858wlQNvSeGgYhA4eUfBJhynLbjPM=; b=e5sPjPh7ZboKMor0yRJ9S9fh1xG0mYjn3Q/LM9XiVLZbLhlT2WZi7thqOqsm02oHu6 oYfPDc2S9Ms8sXRMoyJ8F33OpY75Hy48Qm8kL3BN+bT84SyETFieg8iKS9CqF4aHyXFY qNjqIAeIGNRCO06f/tM6OG4J+CIKDGeDH6oJ8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=NIVeSMi7YzgNT8858wlQNvSeGgYhA4eUfBJhynLbjPM=; b=Ijl/CL2Xw/8muLqws03DB98ipoAbcai/Gwk06Rl4JIOgeGF1zE+NQo3UyO6NA+Xs34 bZdq2NIYpV3UwnMfzliC4o96fOPu31oS0hFj5PNoZmhMnXJZD2uDazdQV1qkbEtbQ7TZ UhzBia65gPHG/JiIvydpUjt6Qv/iM3yW5bPAMF8IHezqs+oY6JUMsNxpNCHxg3+ZmMWU /I9Xr9vMCiKX9fGTBhNEoEIgh3aCmR4nS1/4z9Gd9ZgDxosgswTgPzCzyGLEsINXa9A8 gMzjjfkf+KtF4f72ubgtXsnP7tAilD4SxaK8UBwQQ6pOu2tMb3YTarDi6CUvdT120UAe CmLw== X-Gm-Message-State: AOAM533ITwbNiE7INSDDdY0M8R52Ps3ebIPLip27ZKbNIbDHy1VMe1yQ OTMy5uYgtxVPY691M46DFt7+T3UrwQ/YPQ== X-Google-Smtp-Source: ABdhPJzwq9E0/BvuH8wpqha7JclhfzXgk6g0qHEhTETYo7i/r4Y7jZb2/8blGON8to9RWO/pS5aAdg== X-Received: by 2002:a05:6602:1410:: with SMTP id t16mr36109392iov.160.1641784526417; Sun, 09 Jan 2022 19:15:26 -0800 (PST) Received: from sjg1.roam.corp.google.com (c-67-190-101-114.hsd1.co.comcast.net. [67.190.101.114]) by smtp.gmail.com with ESMTPSA id d16sm3542182ioy.29.2022.01.09.19.15.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 09 Jan 2022 19:15:26 -0800 (PST) From: Simon Glass To: U-Boot Mailing List Cc: Simon Glass Subject: [PATCH v2 30/38] binman: Add a bintool implementation for lz4 Date: Sun, 9 Jan 2022 20:14:05 -0700 Message-Id: <20220110031413.1970836-31-sjg@chromium.org> X-Mailer: git-send-email 2.34.1.575.g55b058a8bb-goog In-Reply-To: <20220110031413.1970836-1-sjg@chromium.org> References: <20220110031413.1970836-1-sjg@chromium.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.39 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.103.2 at phobos.denx.de X-Virus-Status: Clean Add a Bintool for this, which is used to compress and decompress data. It supports the features needed by binman as well as installing via the lz4 package. Signed-off-by: Simon Glass --- (no changes since v1) tools/binman/btool/lz4.py | 140 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+) create mode 100644 tools/binman/btool/lz4.py diff --git a/tools/binman/btool/lz4.py b/tools/binman/btool/lz4.py new file mode 100644 index 00000000000..d165f52da92 --- /dev/null +++ b/tools/binman/btool/lz4.py @@ -0,0 +1,140 @@ +# SPDX-License-Identifier: GPL-2.0+ +# Copyright 2022 Google LLC +# +"""Bintool implementation for lz4 + +lz4 allows compression and decompression of files. + +Documentation is available via:: + + man lz4 + +Here is the help: + +*** LZ4 command line interface 64-bits v1.9.3, by Yann Collet *** +Usage : + lz4 [arg] [input] [output] + +input : a filename + with no FILE, or when FILE is - or stdin, read standard input +Arguments : + -1 : Fast compression (default) + -9 : High compression + -d : decompression (default for .lz4 extension) + -z : force compression + -D FILE: use FILE as dictionary + -f : overwrite output without prompting + -k : preserve source files(s) (default) +--rm : remove source file(s) after successful de/compression + -h/-H : display help/long help and exit + +Advanced arguments : + -V : display Version number and exit + -v : verbose mode + -q : suppress warnings; specify twice to suppress errors too + -c : force write to standard output, even if it is the console + -t : test compressed file integrity + -m : multiple input files (implies automatic output filenames) + -r : operate recursively on directories (sets also -m) + -l : compress using Legacy format (Linux kernel compression) + -B# : cut file into blocks of size # bytes [32+] + or predefined block size [4-7] (default: 7) + -BI : Block Independence (default) + -BD : Block dependency (improves compression ratio) + -BX : enable block checksum (default:disabled) +--no-frame-crc : disable stream checksum (default:enabled) +--content-size : compressed frame includes original size (default:not present) +--list FILE : lists information about .lz4 files (useful for files compressed + with --content-size flag) +--[no-]sparse : sparse mode (default:enabled on file, disabled on stdout) +--favor-decSpeed: compressed files decompress faster, but are less compressed +--fast[=#]: switch to ultra fast compression level (default: 1) +--best : same as -12 +Benchmark arguments : + -b# : benchmark file(s), using # compression level (default : 1) + -e# : test all compression levels from -bX to # (default : 1) + -i# : minimum evaluation time in seconds (default : 3s) +""" + +import re +import tempfile + +from binman import bintool +from patman import tools + +# pylint: disable=C0103 +class Bintoollz4(bintool.Bintool): + """Compression/decompression using the LZ4 algorithm + + This bintool supports running `lz4` to compress and decompress data, as + used by binman. + + It is also possible to fetch the tool, which uses `apt` to install it. + + Documentation is available via:: + + man lz4 + """ + def __init__(self, name): + super().__init__(name, 'lz4 compression') + + def compress(self, indata): + """Compress data with lz4 + + Args: + indata (bytes): Data to compress + + Returns: + bytes: Compressed data + """ + with tempfile.NamedTemporaryFile(prefix='comp.tmp', + dir=tools.GetOutputDir()) as tmp: + tools.WriteFile(tmp.name, indata) + args = ['--no-frame-crc', '-B4', '-5', '-c', tmp.name] + return self.run_cmd(*args, binary=True) + + def decompress(self, indata): + """Decompress data with lz4 + + Args: + indata (bytes): Data to decompress + + Returns: + bytes: Decompressed data + """ + with tempfile.NamedTemporaryFile(prefix='decomp.tmp', + dir=tools.GetOutputDir()) as inf: + tools.WriteFile(inf.name, indata) + args = ['-cd', inf.name] + return self.run_cmd(*args, binary=True) + + def fetch(self, method): + """Fetch handler for lz4 + + This installs the lz4 package using the apt utility. + + Args: + method (FETCH_...): Method to use + + Returns: + True if the file was fetched and now installed, None if a method + other than FETCH_BIN was requested + + Raises: + Valuerror: Fetching could not be completed + """ + if method != bintool.FETCH_BIN: + return None + return self.apt_install('lz4') + + def version(self): + """Version handler + + Returns: + str: Version number of lz4 + """ + out = self.run_cmd('-V').strip() + if not out: + return super().version() + m_version = re.match(r'.* (v[0-9.]*),.*', out) + return m_version.group(1) if m_version else out -- 2.34.1.575.g55b058a8bb-goog