* [PATCH v3 1/5] sha1_file: Added index_blob().
2010-05-10 9:51 [PATCH v3 0/5] Patches to avoid reporting conversion changes Henrik Grubbström (Grubba)
@ 2010-05-10 9:51 ` Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 2/5] strbuf: Added strbuf_add_uint32() Henrik Grubbström (Grubba)
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Henrik Grubbström (Grubba) @ 2010-05-10 9:51 UTC (permalink / raw)
To: git; +Cc: gitster, Henrik Grubbström
When conversion attributes have changed, it is useful
to be able to easily reconvert an existing blob.
Signed-off-by: Henrik Grubbström <grubba@grubba.org>
---
No changes since v1.
cache.h | 1 +
sha1_file.c | 19 +++++++++++++++++++
2 files changed, 20 insertions(+), 0 deletions(-)
diff --git a/cache.h b/cache.h
index 5eb0573..1fe2d7d 100644
--- a/cache.h
+++ b/cache.h
@@ -494,6 +494,7 @@ extern int ie_match_stat(const struct index_state *, struct cache_entry *, struc
extern int ie_modified(const struct index_state *, struct cache_entry *, struct stat *, unsigned int);
extern int ce_path_match(const struct cache_entry *ce, const char **pathspec);
+extern int index_blob(unsigned char *dst_sha1, const unsigned char *src_sha1, int write_object, const char *path);
extern int index_fd(unsigned char *sha1, int fd, struct stat *st, int write_object, enum object_type type, const char *path);
extern int index_path(unsigned char *sha1, const char *path, struct stat *st, int write_object);
extern void fill_stat_cache_info(struct cache_entry *ce, struct stat *st);
diff --git a/sha1_file.c b/sha1_file.c
index ff65328..c162321 100644
--- a/sha1_file.c
+++ b/sha1_file.c
@@ -2434,6 +2434,25 @@ static int index_mem(unsigned char *sha1, void *buf, size_t size,
#define SMALL_FILE_SIZE (32*1024)
+int index_blob(unsigned char *dst_sha1, const unsigned char *src_sha1,
+ int write_object, const char *path)
+{
+ void *buf;
+ unsigned long buflen = 0;
+ int ret;
+
+ memcpy(dst_sha1, src_sha1, 20);
+ buf = read_object_with_reference(src_sha1, typename(OBJ_BLOB),
+ &buflen, dst_sha1);
+ if (!buf)
+ return 0;
+
+ ret = index_mem(dst_sha1, buf, buflen, write_object, OBJ_BLOB, path);
+ free(buf);
+
+ return ret;
+}
+
int index_fd(unsigned char *sha1, int fd, struct stat *st, int write_object,
enum object_type type, const char *path)
{
--
1.7.0.4.369.g81e89
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH v3 2/5] strbuf: Added strbuf_add_uint32().
2010-05-10 9:51 [PATCH v3 0/5] Patches to avoid reporting conversion changes Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 1/5] sha1_file: Added index_blob() Henrik Grubbström (Grubba)
@ 2010-05-10 9:51 ` Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 3/5] cache: Keep track of conversion mode changes Henrik Grubbström (Grubba)
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Henrik Grubbström (Grubba) @ 2010-05-10 9:51 UTC (permalink / raw)
To: git; +Cc: gitster, Henrik Grubbström
Added convenience function for adding an unsigned 32bit
integer in network byte-order to a strbuf.
Signed-off-by: Henrik Grubbström <grubba@grubba.org>
---
New trivial function.
strbuf.h | 4 ++++
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/strbuf.h b/strbuf.h
index fac2dbc..52cd71e 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -107,6 +107,10 @@ static inline void strbuf_addbuf(struct strbuf *sb, const struct strbuf *sb2) {
strbuf_grow(sb, sb2->len);
strbuf_add(sb, sb2->buf, sb2->len);
}
+static inline void strbuf_add_uint32(struct strbuf *sb, uint32_t val) {
+ val = htonl(val);
+ strbuf_add(sb, &val, sizeof(val));
+}
extern void strbuf_adddup(struct strbuf *sb, size_t pos, size_t len);
typedef size_t (*expand_fn_t) (struct strbuf *sb, const char *placeholder, void *context);
--
1.7.0.4.369.g81e89
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH v3 3/5] cache: Keep track of conversion mode changes.
2010-05-10 9:51 [PATCH v3 0/5] Patches to avoid reporting conversion changes Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 1/5] sha1_file: Added index_blob() Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 2/5] strbuf: Added strbuf_add_uint32() Henrik Grubbström (Grubba)
@ 2010-05-10 9:51 ` Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 4/5] cache: Added index extension "CONV" Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 5/5] t/t0021: Test that conversion changes are detected Henrik Grubbström (Grubba)
4 siblings, 0 replies; 6+ messages in thread
From: Henrik Grubbström (Grubba) @ 2010-05-10 9:51 UTC (permalink / raw)
To: git; +Cc: gitster, Henrik Grubbström
The index now keeps track of the conversion mode that was active
when the entry was created. This can be used to detect the most
common cases of when the conversion mode has changed.
Signed-off-by: Henrik Grubbström <grubba@grubba.org>
---
No longer keeps track of the normalized sha1. Instead
CONV_NORM_NEEDED is set when the conversion mode affects
the normalized sha1, and the normalized sha1 (re-)generated
on demand.
The cache_entry is now smudged when CONV_NORM_NEEDED is set
or reset, since we don't know the size of the corresponding
working tree file in such cases.
Removed the prefix NORM_ from the conversion mode flags.
cache.h | 10 ++++++++++
convert.c | 31 +++++++++++++++++++++++++++++++
read-cache.c | 37 ++++++++++++++++++++++++++++++++++---
3 files changed, 75 insertions(+), 3 deletions(-)
diff --git a/cache.h b/cache.h
index 1fe2d7d..e5f54f2 100644
--- a/cache.h
+++ b/cache.h
@@ -151,10 +151,19 @@ struct cache_entry {
unsigned int ce_size;
unsigned int ce_flags;
unsigned char sha1[20];
+ unsigned int ce_conv_flags;
struct cache_entry *next;
char name[FLEX_ARRAY]; /* more */
};
+#define CONV_CRLF_GIT 0x0001
+#define CONV_CRLF_WT 0x0002
+#define CONV_CRLF_GUESS 0x0004
+#define CONV_IDENT 0x0008
+#define CONV_FILT 0x0010
+#define CONV_MASK 0x001f
+#define CONV_NORM_NEEDED 0x010000
+
#define CE_NAMEMASK (0x0fff)
#define CE_STAGEMASK (0x3000)
#define CE_EXTENDED (0x4000)
@@ -1014,6 +1023,7 @@ extern void trace_argv_printf(const char **argv, const char *format, ...);
/* convert.c */
/* returns 1 if *dst was used */
+extern unsigned int git_conv_flags(const char *path);
extern int convert_to_git(const char *path, const char *src, size_t len,
struct strbuf *dst, enum safe_crlf checksafe);
extern int convert_to_working_tree(const char *path, const char *src, size_t len, struct strbuf *dst);
diff --git a/convert.c b/convert.c
index 4f8fcb7..9c063c8 100644
--- a/convert.c
+++ b/convert.c
@@ -568,6 +568,37 @@ static int git_path_check_ident(const char *path, struct git_attr_check *check)
return !!ATTR_TRUE(value);
}
+unsigned int git_conv_flags(const char *path)
+{
+ struct git_attr_check check[3];
+ int crlf = CRLF_GUESS;
+ int ident = 0;
+ unsigned ret = 0;
+ struct convert_driver *drv = NULL;
+
+ setup_convert_check(check);
+ if (!git_checkattr(path, ARRAY_SIZE(check), check)) {
+ crlf = git_path_check_crlf(path, check + 0);
+ ident = git_path_check_ident(path, check + 1);
+ drv = git_path_check_convert(path, check + 2);
+ }
+
+ if (auto_crlf && (crlf != CRLF_BINARY)) {
+ ret |= CONV_CRLF_GIT;
+ if (crlf != CRLF_INPUT && auto_crlf > 0)
+ ret |= CONV_CRLF_WT;
+ if (crlf == CRLF_GUESS)
+ ret |= CONV_CRLF_GUESS;
+ }
+ if (ident) {
+ ret |= CONV_IDENT;
+ }
+ if (drv) {
+ ret |= CONV_FILT;
+ }
+ return ret;
+}
+
int convert_to_git(const char *path, const char *src, size_t len,
struct strbuf *dst, enum safe_crlf checksafe)
{
diff --git a/read-cache.c b/read-cache.c
index f1f789b..eeda928 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -88,12 +88,38 @@ void fill_stat_cache_info(struct cache_entry *ce, struct stat *st)
static int ce_compare_data(struct cache_entry *ce, struct stat *st)
{
int match = -1;
- int fd = open(ce->name, O_RDONLY);
-
+ int fd;
+ unsigned char norm_sha1[20];
+ unsigned int conv_flags = git_conv_flags(ce->name);
+ const unsigned char *cmp_sha1 = ce->sha1;
+
+ if ((conv_flags ^ ce->ce_conv_flags) & CONV_MASK) {
+ if (ce->ce_conv_flags & CONV_NORM_NEEDED) {
+ /* Smudge the entry since it was only correct
+ * for the old conversion mode. */
+ ce->ce_size = 0;
+ }
+ ce->ce_conv_flags = conv_flags;
+ } else
+ conv_flags = ce->ce_conv_flags & CONV_NORM_NEEDED;
+
+ if (conv_flags) {
+ index_blob(norm_sha1, ce->sha1, 0, ce->name);
+ if (!(conv_flags & CONV_NORM_NEEDED) &&
+ hashcmp(norm_sha1, ce->sha1)) {
+ ce->ce_conv_flags = conv_flags | CONV_NORM_NEEDED;
+ /* Smudge the entry since we don't know
+ * the correct value. */
+ ce->ce_size = 0;
+ }
+ cmp_sha1 = norm_sha1;
+ }
+
+ fd = open(ce->name, O_RDONLY);
if (fd >= 0) {
unsigned char sha1[20];
if (!index_fd(sha1, fd, st, 0, OBJ_BLOB, ce->name))
- match = hashcmp(sha1, ce->sha1);
+ match = hashcmp(sha1, cmp_sha1);
/* index_fd() closed the file descriptor already */
}
return match;
@@ -227,6 +253,11 @@ static int ce_match_stat_basic(struct cache_entry *ce, struct stat *st)
changed |= INODE_CHANGED;
#endif
+ /* ce_size can not be trusted if the conversion mode has changed. */
+ if ((ce->ce_mode & S_IFMT) == S_IFREG &&
+ ((ce->ce_conv_flags ^ git_conv_flags(ce->name)) & CONV_MASK))
+ return changed;
+
if (ce->ce_size != (unsigned int) st->st_size)
changed |= DATA_CHANGED;
--
1.7.0.4.369.g81e89
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH v3 4/5] cache: Added index extension "CONV".
2010-05-10 9:51 [PATCH v3 0/5] Patches to avoid reporting conversion changes Henrik Grubbström (Grubba)
` (2 preceding siblings ...)
2010-05-10 9:51 ` [PATCH v3 3/5] cache: Keep track of conversion mode changes Henrik Grubbström (Grubba)
@ 2010-05-10 9:51 ` Henrik Grubbström (Grubba)
2010-05-10 9:51 ` [PATCH v3 5/5] t/t0021: Test that conversion changes are detected Henrik Grubbström (Grubba)
4 siblings, 0 replies; 6+ messages in thread
From: Henrik Grubbström (Grubba) @ 2010-05-10 9:51 UTC (permalink / raw)
To: git; +Cc: gitster, Henrik Grubbström
The index can now store and retrieve the ce_conv_flags data.
Signed-off-by: Henrik Grubbström <grubba@grubba.org>
---
The extension name has been changed from "NORM" to "CONV",
since only conversion flags are stored.
read-cache.c | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++-----
1 files changed, 59 insertions(+), 6 deletions(-)
diff --git a/read-cache.c b/read-cache.c
index eeda928..630e001 100644
--- a/read-cache.c
+++ b/read-cache.c
@@ -27,6 +27,7 @@ static struct cache_entry *refresh_cache_entry(struct cache_entry *ce, int reall
#define CACHE_EXT(s) ( (s[0]<<24)|(s[1]<<16)|(s[2]<<8)|(s[3]) )
#define CACHE_EXT_TREE 0x54524545 /* "TREE" */
#define CACHE_EXT_RESOLVE_UNDO 0x52455543 /* "REUC" */
+#define CACHE_EXT_CONV 0x434f4e56 /* "CONV" */
struct index_state the_index;
@@ -1209,6 +1210,34 @@ static int verify_hdr(struct cache_header *hdr, unsigned long size)
return 0;
}
+/* The on disk format is the default conversion flags followed
+ * by alternating cache entry numbers and corresponding flags.
+ */
+static int conv_read(struct cache_entry **cache, unsigned int entries,
+ const unsigned int *data, unsigned long sz)
+{
+ unsigned int entry_no;
+ unsigned int default_conv_flags;
+ if (sz < sizeof(*data))
+ return 0;
+ default_conv_flags = ntohl(*data);
+ data++;
+ sz -= sizeof(*data);
+ if (default_conv_flags) {
+ for (entry_no = 0; entry_no < entries; entry_no++)
+ cache[entry_no]->ce_conv_flags = default_conv_flags;
+ }
+ while (sz >= 2*sizeof(*data)) {
+ entry_no = ntohl(*data);
+ data++;
+ if (entry_no >= entries) break;
+ cache[entry_no]->ce_conv_flags = ntohl(*data);
+ data++;
+ sz -= 2*sizeof(*data);
+ }
+ return 0;
+}
+
static int read_index_extension(struct index_state *istate,
const char *ext, void *data, unsigned long sz)
{
@@ -1219,6 +1248,9 @@ static int read_index_extension(struct index_state *istate,
case CACHE_EXT_RESOLVE_UNDO:
istate->resolve_undo = resolve_undo_read(data, sz);
break;
+ case CACHE_EXT_CONV:
+ return conv_read(istate->cache, istate->cache_nr, data, sz);
+ break;
default:
if (*ext < 'A' || 'Z' < *ext)
return error("index uses %.4s extension, which we do not understand",
@@ -1542,6 +1574,15 @@ static void ce_smudge_racily_clean_entry(struct cache_entry *ce)
}
}
+static void conv_write(struct strbuf *sb, const struct cache_entry *ce,
+ int entry_no)
+{
+ unsigned int entry[2];
+ entry[0] = htonl(entry_no);
+ entry[1] = htonl(ce->ce_conv_flags);
+ strbuf_add(sb, &entry, sizeof(entry));
+}
+
static int ce_write_entry(git_SHA_CTX *c, int fd, struct cache_entry *ce)
{
int size = ondisk_ce_size(ce);
@@ -1577,10 +1618,12 @@ int write_index(struct index_state *istate, int newfd)
{
git_SHA_CTX c;
struct cache_header hdr;
- int i, err, removed, extended;
+ int i, j, err, removed, extended;
struct cache_entry **cache = istate->cache;
int entries = istate->cache_nr;
struct stat st;
+ struct strbuf sb = STRBUF_INIT;
+ unsigned int default_conv_flags;
for (i = removed = extended = 0; i < entries; i++) {
if (cache[i]->ce_flags & CE_REMOVE)
@@ -1603,7 +1646,10 @@ int write_index(struct index_state *istate, int newfd)
if (ce_write(&c, newfd, &hdr, sizeof(hdr)) < 0)
return -1;
- for (i = 0; i < entries; i++) {
+ default_conv_flags = git_conv_flags("");
+ strbuf_add_uint32(&sb, default_conv_flags);
+
+ for (i = j = 0; i < entries; i++) {
struct cache_entry *ce = cache[i];
if (ce->ce_flags & CE_REMOVE)
continue;
@@ -1611,12 +1657,21 @@ int write_index(struct index_state *istate, int newfd)
ce_smudge_racily_clean_entry(ce);
if (ce_write_entry(&c, newfd, ce) < 0)
return -1;
+ if (ce->ce_conv_flags != default_conv_flags)
+ conv_write(&sb, ce, j);
+ j++;
}
/* Write extension data here */
+ if (default_conv_flags || sb.len > sizeof(default_conv_flags)) {
+ err = write_index_ext_header(&c, newfd, CACHE_EXT_CONV, sb.len) < 0
+ || ce_write(&c, newfd, sb.buf, sb.len) < 0;
+ strbuf_release(&sb);
+ if (err)
+ return -1;
+ } else
+ strbuf_release(&sb);
if (istate->cache_tree) {
- struct strbuf sb = STRBUF_INIT;
-
cache_tree_write(&sb, istate->cache_tree);
err = write_index_ext_header(&c, newfd, CACHE_EXT_TREE, sb.len) < 0
|| ce_write(&c, newfd, sb.buf, sb.len) < 0;
@@ -1625,8 +1680,6 @@ int write_index(struct index_state *istate, int newfd)
return -1;
}
if (istate->resolve_undo) {
- struct strbuf sb = STRBUF_INIT;
-
resolve_undo_write(&sb, istate->resolve_undo);
err = write_index_ext_header(&c, newfd, CACHE_EXT_RESOLVE_UNDO,
sb.len) < 0
--
1.7.0.4.369.g81e89
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH v3 5/5] t/t0021: Test that conversion changes are detected.
2010-05-10 9:51 [PATCH v3 0/5] Patches to avoid reporting conversion changes Henrik Grubbström (Grubba)
` (3 preceding siblings ...)
2010-05-10 9:51 ` [PATCH v3 4/5] cache: Added index extension "CONV" Henrik Grubbström (Grubba)
@ 2010-05-10 9:51 ` Henrik Grubbström (Grubba)
4 siblings, 0 replies; 6+ messages in thread
From: Henrik Grubbström (Grubba) @ 2010-05-10 9:51 UTC (permalink / raw)
To: git; +Cc: gitster, Henrik Grubbström
Signed-off-by Henrik Grubbström <grubba@grubba.org>
---
A subshell has been added to the CRLF conversion mode
tests to avoid the changed cwd leaking.
t/t0021-conversion.sh | 54 +++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 54 insertions(+), 0 deletions(-)
diff --git a/t/t0021-conversion.sh b/t/t0021-conversion.sh
index 6cb8d60..6d41ee7 100755
--- a/t/t0021-conversion.sh
+++ b/t/t0021-conversion.sh
@@ -89,4 +89,58 @@ test_expect_success expanded_in_repo '
cmp expanded-keywords expected-output
'
+# Check that files that have had their canonical representation
+# changed since being checked in aren't reported as modified
+# directly after being checked out.
+test_expect_success keywords_not_modified '
+ {
+ echo "File with foreign keywords"
+ echo "\$Id\$"
+ echo "\$Id: NoTerminatingSymbol"
+ echo "\$Id: Foreign Commit With Spaces \$"
+ echo "\$Id: GitCommitId \$"
+ echo "\$Id: NoTerminatingSymbolAtEOF"
+ } > expanded-keywords2 &&
+
+ git add expanded-keywords2 &&
+ git commit -m "File with keywords expanded" &&
+
+ echo "expanded-keywords2 ident" >> .gitattributes &&
+
+ rm -f expanded-keywords2 &&
+ git checkout -- expanded-keywords2 &&
+
+ test "x`git status --porcelain -- expanded-keywords2`" = x
+'
+
+# Test detection of CRLF conversion changes CRLF ==> LF.
+test_expect_success crlf_conversion_change_crlf_to_lf '
+(
+ # step 0. a blob with CRLF
+ git init one && cd one &&
+ echo -e "a quick brown fox\015" >kuzu &&
+ git add kuzu && git commit -m kuzu &&
+ # step 1. you want CRLF in work area, LF in repository
+ git config core.autocrlf true &&
+ # step 2. user edit and revert.
+ touch kuzu &&
+ git update-index --refresh
+)
+'
+
+# Test detection of CRLF conversion changes LF ==> CRLF.
+test_expect_success crlf_conversion_change_lf_to_crlf '
+(
+ # step 0 & 1. a project with LF ending
+ git init two && cd two &&
+ echo a quick brown fox >kuzu &&
+ git add kuzu && git commit -m kuzu &&
+ # step 2. you want CRLF in your work area
+ echo -e "a quick brown fox\015" >kuzu &&
+ git config core.autocrlf true &&
+ # step 3. oops, refresh
+ git update-index --refresh
+)
+'
+
test_done
--
1.7.0.4.369.g81e89
^ permalink raw reply related [flat|nested] 6+ messages in thread