git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Liu Yubao <yubao.liu@gmail.com>
To: Junio C Hamano <gitster@pobox.com>
Cc: git list <git@vger.kernel.org>
Subject: [PATCH 0/5] support reading and writing uncompressed loose object
Date: Tue, 02 Dec 2008 09:48:29 +0800	[thread overview]
Message-ID: <493493ED.8090903@gmail.com> (raw)
In-Reply-To: <7voczws3np.fsf@gitster.siamese.dyndns.org>

Hi,

In original implementation, git stores loose object like this:
    loose object = deflate(typename + <space> + size + data)

The patches below add support to read and write uncompressed loose
object:
    loose object = typename + <space> + size + data

The cons and pros to use uncompressed loose object:

cons
    * old git can't read these uncompressed loose objects
      (I think it's not a big problem because old git can read
       pack files generated by new git)

    * uncompressed loose objects occupy more disk space
      (I also think it's not a big problem because loose objects
       aren't too many in general)

pros
    * avoid compressing and uncompressing loose objects that are likely
      frequently used when coding/merging with git add/diff/diff --cached/
      merge/rebase/log.

    * the code to read and write uncompressed loose objects is
      simpler, although there are now more code paths for compatibility.

    * better to share loose objects among multiple git processes because
      sha1 files can be used directly after mmapped. The original git
      uncompresses loose objects into heap memory area so that they
      can't be shared by other processes.
     (NOTICE: The patches below doesn't use mmapped sha1 files directly
      because I find parse_object() requires a buffer terminated with
      zero.)

    * easy to grep objects in .git/objects  (...stupid use case :-)


If these patches are worth being included into upstream branch,
I will add a new config variable core.uncompressedLooseObject.


Explanation to the patches:

1) avoid parse_sha1_header() accessing memory out of bound
  Just for more safety, no inflateInit() to detect errors for
  uncompressed loose objects.

2) don't die immediately when convert an invalid type name
  So we can fall back to compressed loose objects.

3) optimize parse_sha1_header() a little by detecting object type
  To quickly detect whether it seems an uncompressed loose object.

4) support reading uncompressed loose object
  The new feature.

5) support writing uncompressed loose object
  The new feature, need a git-config variable yet.


The patches are generated against git-1.6.1-rc, I have run the test cases
and it seems ok.


 object.c    |   14 +++++++++++++-
 object.h    |    1 +
 sha1_file.c |   58 +++++++++++++++++++++++++++++++++++++++++++++-------------
 3 files changed, 59 insertions(+), 14 deletions(-)

  parent reply	other threads:[~2008-12-02  1:49 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-12-01  8:00 two questions about the format of loose object Liu Yubao
2008-12-01  8:25 ` Junio C Hamano
2008-12-01  9:28   ` Liu Yubao
2008-12-01 11:32     ` Jakub Narebski
2008-12-02  2:19       ` Liu Yubao
2008-12-01 15:21     ` Shawn O. Pearce
2008-12-02  2:43       ` Liu Yubao
2008-12-02  1:48   ` Liu Yubao [this message]
2008-12-02  1:51   ` [PATCH 1/5] avoid parse_sha1_header() accessing memory out of bound Liu Yubao
2008-12-02 15:42     ` Shawn O. Pearce
2008-12-03  3:49       ` Liu Yubao
2008-12-02  1:53   ` [PATCH 2/5] don't die immediately when convert an invalid type name Liu Yubao
2008-12-02  1:55   ` [PATCH 3/5] optimize parse_sha1_header() a little by detecting object type Liu Yubao
2008-12-02 15:53     ` Shawn O. Pearce
2008-12-03  4:06       ` Liu Yubao
2008-12-02  1:56   ` [PATCH 4/5] support reading uncompressed loose object Liu Yubao
2008-12-02 15:58     ` Shawn O. Pearce
2008-12-03  4:09       ` Liu Yubao
2008-12-02  2:03   ` [PATCH 5/5] support writing " Liu Yubao
2008-12-02 16:07     ` Shawn O. Pearce
2008-12-03  4:22       ` Liu Yubao
2008-12-02  3:11   ` [PATCH 0/5] support reading and " Liu Yubao
2008-12-01 12:16 ` two questions about the format of " Nick Andrew
2008-12-02  2:26   ` Liu Yubao
2008-12-01 15:32 ` Shawn O. Pearce
2008-12-02  3:05   ` Liu Yubao
2008-12-04  0:54     ` Nicolas Pitre

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=493493ED.8090903@gmail.com \
    --to=yubao.liu@gmail.com \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).