From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from list by lists.gnu.org with archive (Exim 4.71) id 1avzS9-0001OZ-Tx for mharc-grub-devel@gnu.org; Thu, 28 Apr 2016 23:50:53 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:60397) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1avzS7-0001MM-VS for grub-devel@gnu.org; Thu, 28 Apr 2016 23:50:52 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1avzS4-00012y-Q1 for grub-devel@gnu.org; Thu, 28 Apr 2016 23:50:51 -0400 Received: from mail-lf0-x235.google.com ([2a00:1450:4010:c07::235]:36850) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1avzS4-00012B-IQ for grub-devel@gnu.org; Thu, 28 Apr 2016 23:50:48 -0400 Received: by mail-lf0-x235.google.com with SMTP id u64so105106693lff.3 for ; Thu, 28 Apr 2016 20:50:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=2D1kMaxYXw51XE8kN0IH+OeKJVqHO4Nx0LAIaC+XgjI=; b=Y8+b7PUr0fOjfAScbfoLMiWcsEI/Km5z7pqN7SOyetFVYxZJKno7z1sMW0kNWy+NRK spQvJ0oBsKoUxGBPK89eOR+PB16DGzAbM0ag14yVUa97oKt9Mzpw/+YMZZQnBzhDeQFz funAz0ZdRQoPR1ZWKtX9urXxl/7R+D5JKrBvEh30Sul6iniseW5fTW+Jct7n20O7S9kO e1wcU+yaavk4fC6nA1AMmsVQKPdy72AqKKUuu9u9JStlJNr26dw7DdajfboucL8BBW8I m6fxnw0ssawkRWJDJat2b/vL0WUD7ZQTEccy+SBA/CPUW++k6VscuVg+XLTfIFF9S2ZH gOVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=2D1kMaxYXw51XE8kN0IH+OeKJVqHO4Nx0LAIaC+XgjI=; b=ZasC9PpE/I73tX0PcIZpBlpLhAxRWoBdGhECgVl4APrzhjIzTTfz2Yk4ue5T9BN54E dwI0NQY1RuELTO98uYRYXvT+J/7CL/pJE++yYe0iT/AttciH03P9oRqXqAGuXxNBGTy4 8wwjJ0k9Yd1gwdmsDLcLowjjwyOG+JgMSh615VvlY7A+II3cG732RTSqCDn5DU4JD31A X2H/WbpWGsaVPrBF+kCUku1xJg10Uacedl/mIEO2ku47YQpVgDYZbWfQfpp/OQeaCUKW JHOvO3dPKwkgkn8fpD1sDoIvTCZD9b4okNNjOPiISxbT8uVKlEBxJyWr0Gj2Ai35M5gk QJAg== X-Gm-Message-State: AOPr4FXrpqCtzu18iIuiCfB6thEe1lTq0EbHxRS48MiLVMTuA8P5zdEHwdXaLhXI9SHnag== X-Received: by 10.25.147.68 with SMTP id v65mr1714563lfd.9.1461901847701; Thu, 28 Apr 2016 20:50:47 -0700 (PDT) Received: from [192.168.1.42] (ppp109-252-90-74.pppoe.spdop.ru. [109.252.90.74]) by smtp.gmail.com with ESMTPSA id h8sm1988793lbb.27.2016.04.28.20.50.46 for (version=TLSv1/SSLv3 cipher=OTHER); Thu, 28 Apr 2016 20:50:46 -0700 (PDT) Subject: Re: [PATCH] Rewrite and fix grub_bufio_read() To: The development of GNU GRUB References: <57224D3F.1080704@gmail.com> From: Andrei Borzenkov Message-ID: <5722DA16.6040706@gmail.com> Date: Fri, 29 Apr 2016 06:50:46 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:4010:c07::235 X-BeenThere: grub-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: The development of GNU GRUB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2016 03:50:52 -0000 28.04.2016 22:52, Stefan Fritsch пишет: > On Thu, 28 Apr 2016, Andrei Borzenkov wrote: > >> 28.04.2016 14:11, Stefan Fritsch пишет: >>> We had problems downloading large (10-100 MB) gziped files via http >>> with grub. After some debugging, I think I have found the reason in >>> grub_bufio_read, which leads to wrong data being passed on. This patch >>> fixes the issues for me. I did my tests with a somewhat older >>> snapshot, but the bufio.c file is the same. >>> >> >> Please send minimal patch that fixes bug in bufio so we can actually see >> where the bug is and apply bug fix for 2.02. It is impossible to deduce >> from your complete rewrite and this patch is too intrusive for 2.02 >> irrespectively of considerations below. >> >>> Cheers, >>> Stefan >>> >>> The grub_bufio_read() had some issues: >>> >>> * in the calculation of next_buf, it assumed that bufio->block_size is a >>> power of 2, which is not always the case (see code in grub_bufio_open()). >>> >> >> All current callers set it to power of 2, although you are right that it >> should verify it. > > No: > > if ((size < 0) || ((unsigned) size > io->size)) > size = ((io->size > GRUB_BUFIO_MAX_SIZE) ? GRUB_BUFIO_MAX_SIZE : > io->size); > > ... > > bufio->block_size = size; > > io->size is the file size. So at least for files < 32K (which is the size > that bufio is called with by the net layer), block_size is not a power of > 2 but the size of the file. Then next_buf typically gets a value from 0 > to 8. > Ah, OK. Not sure why we need this check in the first place. Buffer size is unrelated to file size, so this looks more like micro-optimization of buffer size. Although this means that we always have the whole file in buffer anyway and so never hit power of 2 issue.