git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "René Scharfe" <l.s.r@web.de>
To: Peter Krefting <peter@softwolves.pp.se>, Johannes Sixt <j6t@kdbg.org>
Cc: git@vger.kernel.org, Keith Goldfarb <keith@blackthorn-media.com>
Subject: [PATCH v3 1/5] archive-zip: add tests for big ZIP archives
Date: Mon, 24 Apr 2017 19:29:53 +0200	[thread overview]
Message-ID: <7361b84d-41c4-8224-ff83-36703837eb28@web.de> (raw)
In-Reply-To: <85f2b6d1-107b-0624-af82-92446f28269e@web.de>

Test the creation of ZIP archives bigger than 4GB and containing files
bigger than 4GB.  They are marked as EXPENSIVE because they take quite a
while and because the first one needs a bit more than 4GB of disk space
to store the resulting archive.

The big archive in the first test is made up of a tree containing
thousands of copies of a small file.  Yet the test has to write out the
full archive because unzip doesn't offer a way to read from stdin.

The big file in the second test is provided as a zipped pack file to
avoid writing another 4GB file to disk and then adding it.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
---
 t/t5004-archive-corner-cases.sh |  45 ++++++++++++++++++++++++++++++++++++++++
 t/t5004/big-pack.zip            | Bin 0 -> 7373 bytes
 2 files changed, 45 insertions(+)
 create mode 100644 t/t5004/big-pack.zip

diff --git a/t/t5004-archive-corner-cases.sh b/t/t5004-archive-corner-cases.sh
index cca23383c5..bc052c803a 100755
--- a/t/t5004-archive-corner-cases.sh
+++ b/t/t5004-archive-corner-cases.sh
@@ -155,4 +155,49 @@ test_expect_success ZIPINFO 'zip archive with many entries' '
 	test_cmp expect actual
 '
 
+test_expect_failure EXPENSIVE,UNZIP 'zip archive bigger than 4GB' '
+	# build string containing 65536 characters
+	s=0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef &&
+	s=$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s &&
+	s=$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s &&
+
+	# create blob with a length of 65536 + 1 bytes
+	blob=$(echo $s | git hash-object -w --stdin) &&
+
+	# create tree containing 65500 entries of that blob
+	for i in $(test_seq 1 65500)
+	do
+		echo "100644 blob $blob	$i"
+	done >tree &&
+	tree=$(git mktree <tree) &&
+
+	# zip it, creating an archive a bit bigger than 4GB
+	git archive -0 -o many-big.zip $tree &&
+
+	"$GIT_UNZIP" -t many-big.zip 9999 65500 &&
+	"$GIT_UNZIP" -t many-big.zip
+'
+
+test_expect_failure EXPENSIVE,UNZIP,ZIPINFO 'zip archive with files bigger than 4GB' '
+	# Pack created with:
+	#   dd if=/dev/zero of=file bs=1M count=4100 && git hash-object -w file
+	mkdir -p .git/objects/pack &&
+	(
+		cd .git/objects/pack &&
+		"$GIT_UNZIP" "$TEST_DIRECTORY"/t5004/big-pack.zip
+	) &&
+	blob=754a93d6fada4c6873360e6cb4b209132271ab0e &&
+	size=$(expr 4100 "*" 1024 "*" 1024) &&
+
+	# create a tree containing the file
+	tree=$(echo "100644 blob $blob	big-file" | git mktree) &&
+
+	# zip it, creating an archive with a file bigger than 4GB
+	git archive -o big.zip $tree &&
+
+	"$GIT_UNZIP" -t big.zip &&
+	"$ZIPINFO" big.zip >big.lst &&
+	grep $size big.lst
+'
+
 test_done
diff --git a/t/t5004/big-pack.zip b/t/t5004/big-pack.zip
new file mode 100644
index 0000000000000000000000000000000000000000..caaf614eeece6f818c525e433561e37560a75b05
GIT binary patch
literal 7373
zcmWIWW@Zs#U}E54xLY#A%SmVLl4u471|Jp%215oJhJwW8Y+ZvC^HlQ`3(KT5gS6zd
z#6$y=#3VyYlN6KGlw?bTG()50MB`LTb5p&{l#0+0P6p<OAOA*xaA^fM10%}|W(Ec@
z@jL#}{4)m*95`aY)ux;vb9C_xiI`WWKX0v%wv#-XR%01$R>-9-y6#!_6|EcGQyxED
zyZY^&^YwM2r?Wic_gf3wz0#?R{8hB^<Js4()@9Ms>$gt{iHiUEJN{iALjZ~|R(VzA
zOp{_@IDE*S!H8sEfc%Wl8*lHd?-DJPX@5BLD~n)>se}uQ{sHa{9CBhhi*hb3*-*jg
zc5o4&4%_XW4Ba<OG7P)VrhVaOTdmN<Ho4-C?R3NUpAYE!{5{5hc=zr*JFR#QUppur
z?|(T<>b?Enk9jiu`<7qbA8y|-Ck>2+hMBYN-pgCcFt>ev-5*|-&jb`JE>Hh`B*~cT
z&t2=?C44}E8GDaU*PD0ekK%{l7w@(f14RzJnfvea+fQlS75nRoU$s?(g=&R(fOLb%
zK_JPHAqeJ(fjJ$>oKcz4&>2l3kc=^!7e@2KXkHl23k(gTVK5p7z>;7z9gKznQp0()
zeK6WS7;PVn){Ud}!f4$%I)=h9I*!CJ8V10UU^E?!h5@KqG@1@Z!(cQWK$^#7<%OS{
z_HQoT!K!n;di%NDON?i(ygj-((dOTWgOj)4mXFhko42#<@9S6BTc6*5|Cc?$n~_P5
z8P`mn1SleafRRC^5k!+Qug40R*F&4rL$?-n>J8c2V<cM(nTW$>FDo0!BTPW}9!Q@6
I&6hC%0B76lcmMzZ

literal 0
HcmV?d00001

-- 
2.12.2


  reply	other threads:[~2017-04-24 17:30 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-04-21 21:08 Git archive doesn't fully support zip64 Keith Goldfarb
2017-04-22 19:22 ` [PATCH] archive-zip: Add zip64 headers when file size is too large for 32 bits Peter Krefting
2017-04-22 21:52   ` Johannes Sixt
2017-04-22 22:41     ` [PATCH v2] " Peter Krefting
2017-04-23  7:50       ` Johannes Sixt
2017-04-23 14:51         ` Peter Krefting
2017-04-23 19:49           ` Johannes Sixt
2017-04-24  8:04             ` Peter Krefting
2017-04-24 12:04               ` René Scharfe
2017-04-24 17:22                 ` [PATCH v3 0/5] archive-zip: support files and archives bigger than 4GB René Scharfe
2017-04-24 17:29                   ` René Scharfe [this message]
2017-04-24 17:30                   ` [PATCH v3 2/5] archive-zip: use strbuf for ZIP directory René Scharfe
2017-04-25  4:51                     ` Junio C Hamano
2017-04-25  5:28                       ` René Scharfe
2017-04-24 17:31                   ` [PATCH v3 3/5] archive-zip: write ZIP dir entry directly to strbuf René Scharfe
2017-04-24 17:32                   ` [PATCH v3 4/5] archive-zip: support archives bigger than 4GB René Scharfe
2017-04-24 18:24                     ` Peter Krefting
2017-04-24 20:06                       ` René Scharfe
2017-04-24 20:39                         ` René Scharfe
2017-04-24 21:02                         ` Johannes Sixt
2017-04-24 21:41                           ` René Scharfe
2017-04-25  7:55                         ` Peter Krefting
2017-04-25 16:24                           ` René Scharfe
2017-04-26 21:02                             ` Peter Krefting
2017-04-26 23:38                               ` René Scharfe
2017-04-27  4:57                                 ` Peter Krefting
2017-04-27 19:54                                   ` René Scharfe
2017-04-28  8:40                                     ` Peter Krefting
2017-04-24 17:33                   ` [PATCH v3 5/5] archive-zip: support files " René Scharfe
2017-04-24 21:11                     ` Keith Goldfarb
2017-04-25  4:46                     ` Junio C Hamano
2017-04-25  5:27                       ` René Scharfe
2017-04-29 21:00                   ` [PATCH v3 0/5] archive-zip: support files and archives " Torsten Bögershausen
2017-04-29 22:28                     ` René Scharfe
2017-04-30  5:31                       ` Torsten Bögershausen
2017-04-30  7:53                         ` René Scharfe
2017-04-30 13:06                           ` Torsten Bögershausen
2017-04-30 16:32                           ` Johannes Sixt
2017-04-30 16:40                             ` René Scharfe
2017-04-30 23:49                               ` Junio C Hamano
2017-05-01  8:30                                 ` René Scharfe
2017-04-23  0:16     ` [PATCH] archive-zip: Add zip64 headers when file size is too large for 32 bits René Scharfe
2017-04-23  6:42       ` Peter Krefting
2017-04-23  7:27         ` Johannes Sixt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7361b84d-41c4-8224-ff83-36703837eb28@web.de \
    --to=l.s.r@web.de \
    --cc=git@vger.kernel.org \
    --cc=j6t@kdbg.org \
    --cc=keith@blackthorn-media.com \
    --cc=peter@softwolves.pp.se \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).