From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS31976 209.132.180.0/23 X-Spam-Status: No, score=-4.1 required=3.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD shortcircuit=no autolearn=ham autolearn_force=no version=3.4.0 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by dcvr.yhbt.net (Postfix) with ESMTP id 6097A20899 for ; Fri, 11 Aug 2017 18:50:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753343AbdHKSuY (ORCPT ); Fri, 11 Aug 2017 14:50:24 -0400 Received: from pb-smtp1.pobox.com ([64.147.108.70]:61110 "EHLO sasl.smtp.pobox.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753158AbdHKSuX (ORCPT ); Fri, 11 Aug 2017 14:50:23 -0400 Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-smtp1.pobox.com (Postfix) with ESMTP id DDBA9AC975; Fri, 11 Aug 2017 14:50:15 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=JEdwME10ErvqoGfhD2EOEilp4fg=; b=k0Nymz vmAEhoTtzeanlKtqu3+/xHfgLJF1LKvmD+OZkXRSFIBhVyM9DP16QhkCozXhGEaA Hchqgv1xz2fupySRgekrE1U24t7jMgAz2NO8VLx/HOzB0qbXT1Uzudx42U3a+Dqe u+E4pVLrGnLYmjAkbizEYVS97aVwfxaZNpbdE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=yC5M4EjK588CiHXPcCCwgiSWco33sxmV PQn7zJtMAxzAzP2qMJu4kpzw8FB/bHYvAyLxc9YYsZmBw+wvogZE5XEfz2DNE8Kn vnrK46Giy52j7rs8sKWWGfix6EOu16jIipCEgsKCRpM/4wLsr9F0wLYLYoeYhzK4 RYBJVCxXcgU= Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1]) by pb-smtp1.pobox.com (Postfix) with ESMTP id D3272AC974; Fri, 11 Aug 2017 14:50:15 -0400 (EDT) Received: from pobox.com (unknown [104.132.0.95]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by pb-smtp1.pobox.com (Postfix) with ESMTPSA id 2A56CAC96F; Fri, 11 Aug 2017 14:50:15 -0400 (EDT) From: Junio C Hamano To: Martin Koegler Cc: git@vger.kernel.org, Johannes.Schindelin@gmx.de, Shawn Pearce Subject: Re: [PATCH 4/4] Fix delta offset overflow References: <1502388789-5775-1-git-send-email-martin@mail.zuhause> <1502388789-5775-2-git-send-email-martin@mail.zuhause> <20170811065732.GA15128@mail.zuhause> Date: Fri, 11 Aug 2017 11:50:13 -0700 In-Reply-To: <20170811065732.GA15128@mail.zuhause> (Martin Koegler's message of "Fri, 11 Aug 2017 08:57:32 +0200") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain X-Pobox-Relay-ID: E9A983FC-7EC5-11E7-85F2-FE4B1A68708C-77302942!pb-smtp1.pobox.com Sender: git-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: git@vger.kernel.org Martin Koegler writes: > On Thu, Aug 10, 2017 at 01:49:24PM -0700, Junio C Hamano wrote: >> The lower 4-byte of moff (before incrementing it with msize) were >> already encoded to the output stream before this hunk. Shouldn't >> we be checking if moff would fit in uint32_t _before_ that happens? > > moff is otherwise only decremented or assigned with an offset generated by > create_delta_index. These offsets are limited by 4GB. > > Any larger offets would be a programming bug - so qualify for just a "assert". OK, in that case, I agree that a check before encoding moff into (upto) 4 output bytes is unnecessary. Sorry, I didn't read the function that populates index->hash[] before responding, and I admit that I haven't read it for a while. >> Cutting it off at here by resetting msize to 0 might help the next >> iteration (I didn't check, but is the effect of it is to corrupt the >> "val" rolling checksum and make it unlikely that the hash >> computation would not find a correct match?) but it somehow feels >> like closing the barn door after the horse has already bolted... > > The current code produces incorrect deltas - its not just a checksum issue. Again, I mis-read what role msize was playing in the original (or in your update). I'd need to re-read that part of the code to make sure I get how your change will fix the issue. Thanks. > By the way: > > Somebody interested in JGIT should also look at these two bugs: > > https://github.com/eclipse/jgit/blob/005e5feb4ecd08c4e4d141a38b9e7942accb3212/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/pack/DeltaEncoder.java > copy would also encode beyond 4GB - producing truncated delta offset. > > https://github.com/eclipse/jgit/blob/005e5feb4ecd08c4e4d141a38b9e7942accb3212/org.eclipse.jgit/src/org/eclipse/jgit/internal/storage/pack/BinaryDelta.java > apply uses int for decoding length values. I'll cc: an obvious suspect; thanks for the note.