* [PATCH 0/6] git-p4: view spec improvements
@ 2012-01-02 23:05 Pete Wyckoff
2012-01-02 23:05 ` [PATCH 1/6] git-p4: test client view handling Pete Wyckoff
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Pete Wyckoff @ 2012-01-02 23:05 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
Gary found and fixed a few bothersome problems with how
git-p4 parses the View lines in client specifications.
I got the itch to fix other issues, and ended up writing
a battery of test cases and redoing the view spec code
entirely.
Patch summary:
1. Add 22 test cases for client view spec handling,
most of which are expected to fail. Test review
welcome.
2-4. Fix issues with unsupported wildcards, incorrect
sorting, and add support for single-file maps.
5. Rewrite view spec code, fixing the rest of the tests.
This is new python that could use some review.
6. Update docs. This depends on pw/p4-docs-and-tests that
is in next.
Gary Gibbons (3):
git-p4: fix test for unsupported P4 Client Views
git-p4: sort client views by reverse View number
git-p4: support single file p4 client view maps
Pete Wyckoff (3):
git-p4: test client view handling
git-p4: rewrite view handling
git-p4: view spec documentation
Documentation/git-p4.txt | 40 ++++--
contrib/fast-import/git-p4 | 316 ++++++++++++++++++++++++++++++++---------
t/t9809-git-p4-client-view.sh | 290 +++++++++++++++++++++++++++++++++++++
3 files changed, 564 insertions(+), 82 deletions(-)
create mode 100755 t/t9809-git-p4-client-view.sh
--
1.7.8.1.407.gd70cb
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/6] git-p4: test client view handling
2012-01-02 23:05 [PATCH 0/6] git-p4: view spec improvements Pete Wyckoff
@ 2012-01-02 23:05 ` Pete Wyckoff
2012-01-02 23:05 ` [PATCH 2/6] git-p4: fix test for unsupported P4 Client Views Pete Wyckoff
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Pete Wyckoff @ 2012-01-02 23:05 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
Test many aspects of processing p4 client views with the
git-p4 option --use-client-spec. 16 out of 22 tests are
currently broken.
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
t/t9809-git-p4-client-view.sh | 290 +++++++++++++++++++++++++++++++++++++++++
1 files changed, 290 insertions(+), 0 deletions(-)
create mode 100755 t/t9809-git-p4-client-view.sh
diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
new file mode 100755
index 0000000..4259fb3
--- /dev/null
+++ b/t/t9809-git-p4-client-view.sh
@@ -0,0 +1,290 @@
+#!/bin/sh
+
+test_description='git-p4 client view'
+
+. ./lib-git-p4.sh
+
+test_expect_success 'start p4d' '
+ start_p4d
+'
+
+#
+# Construct a client with this list of View lines
+#
+client_view() {
+ (
+ cat <<-EOF &&
+ Client: client
+ Description: client
+ Root: $cli
+ View:
+ EOF
+ for arg ; do
+ printf "\t$arg\n"
+ done
+ ) | p4 client -i
+}
+
+#
+# Verify these files exist, exactly. Caller creates
+# a list of files in file "files".
+#
+check_files_exist() {
+ ok=0 &&
+ num=${#@} &&
+ for arg ; do
+ test_path_is_file "$arg" &&
+ ok=$(($ok + 1))
+ done &&
+ test $ok -eq $num &&
+ test_line_count = $num files
+}
+
+#
+# Sync up the p4 client, make sure the given files (and only
+# those) exist.
+#
+client_verify() {
+ (
+ cd "$cli" &&
+ p4 sync &&
+ find . -type f ! -name files >files &&
+ check_files_exist "$@"
+ )
+}
+
+#
+# Make sure the named files, exactly, exist.
+#
+git_verify() {
+ (
+ cd "$git" &&
+ git ls-files >files &&
+ check_files_exist "$@"
+ )
+}
+
+# //depot
+# - dir1
+# - file11
+# - file12
+# - dir2
+# - file21
+# - file22
+test_expect_success 'init depot' '
+ (
+ cd "$cli" &&
+ for d in 1 2 ; do
+ mkdir -p dir$d &&
+ for f in 1 2 ; do
+ echo dir$d/file$d$f >dir$d/file$d$f &&
+ p4 add dir$d/file$d$f &&
+ p4 submit -d "dir$d/file$d$f"
+ done
+ done &&
+ find . -type f ! -name files >files &&
+ check_files_exist dir1/file11 dir1/file12 \
+ dir2/file21 dir2/file22
+ )
+'
+
+# double % for printf
+test_expect_failure 'unsupported view wildcard %%n' '
+ client_view "//depot/%%%%1/sub/... //client/sub/%%%%1/..." &&
+ test_when_finished cleanup_git &&
+ test_must_fail "$GITP4" clone --use-client-spec --dest="$git" //depot
+'
+
+test_expect_failure 'unsupported view wildcard *' '
+ client_view "//depot/*/bar/... //client/*/bar/..." &&
+ test_when_finished cleanup_git &&
+ test_must_fail "$GITP4" clone --use-client-spec --dest="$git" //depot
+'
+
+test_expect_success 'wildcard ... only supported at end of spec' '
+ client_view "//depot/.../file11 //client/.../file11" &&
+ test_when_finished cleanup_git &&
+ test_must_fail "$GITP4" clone --use-client-spec --dest="$git" //depot
+'
+
+test_expect_failure 'basic map' '
+ client_view "//depot/dir1/... //client/cli1/..." &&
+ files="cli1/file11 cli1/file12" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'client view with no mappings' '
+ client_view &&
+ client_verify &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify
+'
+
+test_expect_failure 'single file map' '
+ client_view "//depot/dir1/file11 //client/file11" &&
+ files="file11" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'later mapping takes precedence (entire repo)' '
+ client_view "//depot/dir1/... //client/cli1/..." \
+ "//depot/... //client/cli2/..." &&
+ files="cli2/dir1/file11 cli2/dir1/file12
+ cli2/dir2/file21 cli2/dir2/file22" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'later mapping takes precedence (partial repo)' '
+ client_view "//depot/dir1/... //client/..." \
+ "//depot/dir2/... //client/..." &&
+ files="file21 file22" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+# Reading the view backwards,
+# dir2 goes to cli12
+# dir1 cannot go to cli12 since it was filled by dir2
+# dir1 also does not go to cli3, since the second rule
+# noticed that it matched, but was already filled
+test_expect_failure 'depot path matching rejected client path' '
+ client_view "//depot/dir1/... //client/cli3/..." \
+ "//depot/dir1/... //client/cli12/..." \
+ "//depot/dir2/... //client/cli12/..." &&
+ files="cli12/file21 cli12/file22" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+# since both have the same //client/..., the exclusion
+# rule keeps everything out
+test_expect_failure 'exclusion wildcard, client rhs same (odd)' '
+ client_view "//depot/... //client/..." \
+ "-//depot/dir2/... //client/..." &&
+ client_verify &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify
+'
+
+test_expect_success 'exclusion wildcard, client rhs different (normal)' '
+ client_view "//depot/... //client/..." \
+ "-//depot/dir2/... //client/dir2/..." &&
+ files="dir1/file11 dir1/file12" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'exclusion single file' '
+ client_view "//depot/... //client/..." \
+ "-//depot/dir2/file22 //client/file22" &&
+ files="dir1/file11 dir1/file12 dir2/file21" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'overlay wildcard' '
+ client_view "//depot/dir1/... //client/cli/..." \
+ "+//depot/dir2/... //client/cli/...\n" &&
+ files="cli/file11 cli/file12 cli/file21 cli/file22" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'overlay single file' '
+ client_view "//depot/dir1/... //client/cli/..." \
+ "+//depot/dir2/file21 //client/cli/file21" &&
+ files="cli/file11 cli/file12 cli/file21" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'exclusion with later inclusion' '
+ client_view "//depot/... //client/..." \
+ "-//depot/dir2/... //client/dir2/..." \
+ "//depot/dir2/... //client/dir2incl/..." &&
+ files="dir1/file11 dir1/file12 dir2incl/file21 dir2incl/file22" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify $files
+'
+
+test_expect_failure 'quotes on rhs only' '
+ client_view "//depot/dir1/... \"//client/cdir 1/...\"" &&
+ client_verify "cdir 1/file11" "cdir 1/file12" &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify "cdir 1/file11" "cdir 1/file12"
+'
+
+#
+# Rename directories to test quoting in depot-side mappings
+# //depot
+# - "dir 1"
+# - file11
+# - file12
+# - "dir 2"
+# - file21
+# - file22
+#
+test_expect_success 'rename files to introduce spaces' '
+ client_view "//depot/... //client/..." &&
+ client_verify dir1/file11 dir1/file12 \
+ dir2/file21 dir2/file22 &&
+ (
+ cd "$cli" &&
+ p4 open dir1/... &&
+ p4 move dir1/... "dir 1"/... &&
+ p4 open dir2/... &&
+ p4 move dir2/... "dir 2"/... &&
+ p4 submit -d "rename with spaces"
+ ) &&
+ client_verify "dir 1/file11" "dir 1/file12" \
+ "dir 2/file21" "dir 2/file22"
+'
+
+test_expect_failure 'quotes on lhs only' '
+ client_view "\"//depot/dir 1/...\" //client/cdir1/..." &&
+ files="cdir1/file11 cdir1/file12" &&
+ client_verify $files &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ client_verify $files
+'
+
+test_expect_failure 'quotes on both sides' '
+ client_view "\"//depot/dir 1/...\" \"//client/cdir 1/...\"" &&
+ client_verify "cdir 1/file11" "cdir 1/file12" &&
+ test_when_finished cleanup_git &&
+ "$GITP4" clone --use-client-spec --dest="$git" //depot &&
+ git_verify "cdir 1/file11" "cdir 1/file12"
+'
+
+test_expect_success 'kill p4d' '
+ kill_p4d
+'
+
+test_done
--
1.7.8.1.407.gd70cb
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 2/6] git-p4: fix test for unsupported P4 Client Views
2012-01-02 23:05 [PATCH 0/6] git-p4: view spec improvements Pete Wyckoff
2012-01-02 23:05 ` [PATCH 1/6] git-p4: test client view handling Pete Wyckoff
@ 2012-01-02 23:05 ` Pete Wyckoff
2012-01-02 23:05 ` [PATCH 3/6] git-p4: sort client views by reverse View number Pete Wyckoff
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Pete Wyckoff @ 2012-01-02 23:05 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
From: Gary Gibbons <ggibbons@perforce.com>
Change re method in test for unsupported Client View types
(containing %% or *) anywhere in the string rather than
at the begining.
[pw: two tests now succeed]
Signed-off-by: Gary Gibbons <ggibbons@perforce.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
contrib/fast-import/git-p4 | 5 ++++-
t/t9809-git-p4-client-view.sh | 4 ++--
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index d3c3ad8..c144c89 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -1889,9 +1889,12 @@ class P4Sync(Command, P4UserMap):
# p4 has these %%1 to %%9 arguments in specs to
# reorder paths; which we can't handle (yet :)
- if re.match('%%\d', v) != None:
+ if re.search('%%\d', v) != None:
print "Sorry, can't handle %%n arguments in client specs"
sys.exit(1)
+ if re.search('\*', v) != None:
+ print "Sorry, can't handle * mappings in client specs"
+ sys.exit(1)
if v.startswith('"'):
start = 1
diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
index 4259fb3..cbf2213 100755
--- a/t/t9809-git-p4-client-view.sh
+++ b/t/t9809-git-p4-client-view.sh
@@ -89,13 +89,13 @@ test_expect_success 'init depot' '
'
# double % for printf
-test_expect_failure 'unsupported view wildcard %%n' '
+test_expect_success 'unsupported view wildcard %%n' '
client_view "//depot/%%%%1/sub/... //client/sub/%%%%1/..." &&
test_when_finished cleanup_git &&
test_must_fail "$GITP4" clone --use-client-spec --dest="$git" //depot
'
-test_expect_failure 'unsupported view wildcard *' '
+test_expect_success 'unsupported view wildcard *' '
client_view "//depot/*/bar/... //client/*/bar/..." &&
test_when_finished cleanup_git &&
test_must_fail "$GITP4" clone --use-client-spec --dest="$git" //depot
--
1.7.8.1.407.gd70cb
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 3/6] git-p4: sort client views by reverse View number
2012-01-02 23:05 [PATCH 0/6] git-p4: view spec improvements Pete Wyckoff
2012-01-02 23:05 ` [PATCH 1/6] git-p4: test client view handling Pete Wyckoff
2012-01-02 23:05 ` [PATCH 2/6] git-p4: fix test for unsupported P4 Client Views Pete Wyckoff
@ 2012-01-02 23:05 ` Pete Wyckoff
2012-01-02 23:05 ` [PATCH 4/6] git-p4: support single file p4 client view maps Pete Wyckoff
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Pete Wyckoff @ 2012-01-02 23:05 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
From: Gary Gibbons <ggibbons@perforce.com>
Correct view sorting to support the Perforce order,
where client views are ordered and later views
override earlier view mappings.
[pw: one test now succeeds]
Signed-off-by: Gary Gibbons <ggibbons@perforce.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
contrib/fast-import/git-p4 | 11 +++++++++--
t/t9809-git-p4-client-view.sh | 2 +-
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index c144c89..2fd4d3b 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -1924,10 +1924,17 @@ class P4Sync(Command, P4UserMap):
else:
include = len(v)
- temp[v] = (include, cv)
+ # store the View #number for sorting
+ # and the View string itself (this last for documentation)
+ temp[v] = (include, cv, int(k[4:]),k)
self.clientSpecDirs = temp.items()
- self.clientSpecDirs.sort( lambda x, y: abs( y[1][0] ) - abs( x[1][0] ) )
+ # Perforce ViewNN with higher #numbers override those with lower
+ # reverse sort on the View #number
+ self.clientSpecDirs.sort( lambda x, y: y[1][2] - x[1][2] )
+ if self.verbose:
+ for val in self.clientSpecDirs:
+ print "clientSpecDirs: %s %s" % (val[0],val[1])
def run(self, args):
self.depotPaths = []
diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
index cbf2213..06652cb 100755
--- a/t/t9809-git-p4-client-view.sh
+++ b/t/t9809-git-p4-client-view.sh
@@ -133,7 +133,7 @@ test_expect_failure 'single file map' '
git_verify $files
'
-test_expect_failure 'later mapping takes precedence (entire repo)' '
+test_expect_success 'later mapping takes precedence (entire repo)' '
client_view "//depot/dir1/... //client/cli1/..." \
"//depot/... //client/cli2/..." &&
files="cli2/dir1/file11 cli2/dir1/file12
--
1.7.8.1.407.gd70cb
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/6] git-p4: support single file p4 client view maps
2012-01-02 23:05 [PATCH 0/6] git-p4: view spec improvements Pete Wyckoff
` (2 preceding siblings ...)
2012-01-02 23:05 ` [PATCH 3/6] git-p4: sort client views by reverse View number Pete Wyckoff
@ 2012-01-02 23:05 ` Pete Wyckoff
2012-01-02 23:05 ` [PATCH 5/6] git-p4: rewrite view handling Pete Wyckoff
2012-01-02 23:05 ` [PATCH 6/6] git-p4: view spec documentation Pete Wyckoff
5 siblings, 0 replies; 7+ messages in thread
From: Pete Wyckoff @ 2012-01-02 23:05 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
From: Gary Gibbons <ggibbons@perforce.com>
Perforce client views can map individual files,
mapping one //depot file path to one //client file path.
These mappings contain no meta/masking characters.
This patch add support for these file maps to
the currently supported '...' view mappings.
[pw: one test now suceeds]
Signed-off-by: Gary Gibbons <ggibbons@perforce.com>
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
contrib/fast-import/git-p4 | 29 ++++++++++++++++++++---------
t/t9809-git-p4-client-view.sh | 2 +-
2 files changed, 21 insertions(+), 10 deletions(-)
diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 2fd4d3b..8f28e0a 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -1217,6 +1217,7 @@ class P4Sync(Command, P4UserMap):
self.cloneExclude = []
self.useClientSpec = False
self.clientSpecDirs = []
+ self.haveSingleFileClientViews = False
if gitConfig("git-p4.syncFromOrigin") == "false":
self.syncWithOrigin = False
@@ -1274,6 +1275,16 @@ class P4Sync(Command, P4UserMap):
# will end up putting all foo/branch files into
# branch/foo/
for val in self.clientSpecDirs:
+ if self.haveSingleFileClientViews and len(path) == abs(val[1][0]) and path == val[0]:
+ # since 'path' is a depot file path, if it matches the LeftMap,
+ # then the View is a single file mapping, so use the entire rightMap
+ # first two tests above avoid the last == test for common cases
+ path = val[1][1]
+ # now strip out the client (//client/...)
+ path = re.sub("^(//[^/]+/)", '', path)
+ # the rest is local client path
+ return path
+
if path.startswith(val[0]):
# replace the depot path with the client path
path = path.replace(val[0], val[1][1])
@@ -1905,19 +1916,19 @@ class P4Sync(Command, P4UserMap):
# save the "client view"; i.e the RHS of the view
# line that tells the client where to put the
# files for this view.
- cv = v[index+3:].strip() # +3 to remove previous '...'
- # if the client view doesn't end with a
- # ... wildcard, then we're going to mess up the
- # output directory, so fail gracefully.
- if not cv.endswith('...'):
- print 'Sorry, client view in "%s" needs to end with wildcard' % (k)
- sys.exit(1)
- cv=cv[:-3]
+ # check for individual file mappings - those which have no '...'
+ if index < 0 :
+ v,cv = v.strip().split()
+ v = v[start:]
+ self.haveSingleFileClientViews = True
+ else:
+ cv = v[index+3:].strip() # +3 to remove previous '...'
+ cv=cv[:-3]
+ v = v[start:index]
# now save the view; +index means included, -index
# means it should be filtered out.
- v = v[start:index]
if v.startswith("-"):
v = v[1:]
include = -len(v)
diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
index 06652cb..1cc83c5 100755
--- a/t/t9809-git-p4-client-view.sh
+++ b/t/t9809-git-p4-client-view.sh
@@ -191,7 +191,7 @@ test_expect_success 'exclusion wildcard, client rhs different (normal)' '
git_verify $files
'
-test_expect_failure 'exclusion single file' '
+test_expect_success 'exclusion single file' '
client_view "//depot/... //client/..." \
"-//depot/dir2/file22 //client/file22" &&
files="dir1/file11 dir1/file12 dir2/file21" &&
--
1.7.8.1.407.gd70cb
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 5/6] git-p4: rewrite view handling
2012-01-02 23:05 [PATCH 0/6] git-p4: view spec improvements Pete Wyckoff
` (3 preceding siblings ...)
2012-01-02 23:05 ` [PATCH 4/6] git-p4: support single file p4 client view maps Pete Wyckoff
@ 2012-01-02 23:05 ` Pete Wyckoff
2012-01-02 23:05 ` [PATCH 6/6] git-p4: view spec documentation Pete Wyckoff
5 siblings, 0 replies; 7+ messages in thread
From: Pete Wyckoff @ 2012-01-02 23:05 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
The old code was not very complete or robust. Redo it.
This new code should be useful for a few possible additions
in the future:
- support for * and %%n wildcards
- allowing ... inside paths
- representing branch specs (not just client specs)
- tracking changes to views
Mark the remaining 12 tests in t9809 as fixed.
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
contrib/fast-import/git-p4 | 335 ++++++++++++++++++++++++++++++-----------
t/t9809-git-p4-client-view.sh | 24 ++--
2 files changed, 258 insertions(+), 101 deletions(-)
diff --git a/contrib/fast-import/git-p4 b/contrib/fast-import/git-p4
index 8f28e0a..3e1aa27 100755
--- a/contrib/fast-import/git-p4
+++ b/contrib/fast-import/git-p4
@@ -1169,6 +1169,218 @@ class P4Submit(Command, P4UserMap):
return True
+class View(object):
+ """Represent a p4 view ("p4 help views"), and map files in a
+ repo according to the view."""
+
+ class Path(object):
+ """A depot or client path, possibly containing wildcards.
+ The only one supported is ... at the end, currently.
+ Initialize with the full path, with //depot or //client."""
+
+ def __init__(self, path, is_depot):
+ self.path = path
+ self.is_depot = is_depot
+ self.find_wildcards()
+ # remember the prefix bit, useful for relative mappings
+ m = re.match("(//[^/]+/)", self.path)
+ if not m:
+ die("Path %s does not start with //prefix/" % self.path)
+ prefix = m.group(1)
+ if not self.is_depot:
+ # strip //client/ on client paths
+ self.path = self.path[len(prefix):]
+
+ def find_wildcards(self):
+ """Make sure wildcards are valid, and set up internal
+ variables."""
+
+ self.ends_triple_dot = False
+ # There are three wildcards allowed in p4 views
+ # (see "p4 help views"). This code knows how to
+ # handle "..." (only at the end), but cannot deal with
+ # "%%n" or "*". Only check the depot_side, as p4 should
+ # validate that the client_side matches too.
+ if re.search(r'%%[1-9]', self.path):
+ die("Can't handle %%n wildcards in view: %s" % self.path)
+ if self.path.find("*") >= 0:
+ die("Can't handle * wildcards in view: %s" % self.path)
+ triple_dot_index = self.path.find("...")
+ if triple_dot_index >= 0:
+ if not self.path.endswith("..."):
+ die("Can handle ... wildcard only at end of path: %s" %
+ self.path)
+ self.ends_triple_dot = True
+
+ def ensure_compatible(self, other_path):
+ """Make sure the wildcards agree."""
+ if self.ends_triple_dot != other_path.ends_triple_dot:
+ die("Both paths must end with ... if either does;\n" +
+ "paths: %s %s" % (self.path, other_path.path))
+
+ def match_wildcards(self, test_path):
+ """See if this test_path matches us, and fill in the value
+ of the wildcards if so. Returns a tuple of
+ (True|False, wildcards[]). For now, only the ... at end
+ is supported, so at most one wildcard."""
+ if self.ends_triple_dot:
+ dotless = self.path[:-3]
+ if test_path.startswith(dotless):
+ wildcard = test_path[len(dotless):]
+ return (True, [ wildcard ])
+ else:
+ if test_path == self.path:
+ return (True, [])
+ return (False, [])
+
+ def match(self, test_path):
+ """Just return if it matches; don't bother with the wildcards."""
+ b, _ = self.match_wildcards(test_path)
+ return b
+
+ def fill_in_wildcards(self, wildcards):
+ """Return the relative path, with the wildcards filled in
+ if there are any."""
+ if self.ends_triple_dot:
+ return self.path[:-3] + wildcards[0]
+ else:
+ return self.path
+
+ class Mapping(object):
+ def __init__(self, depot_side, client_side, overlay, exclude):
+ # depot_side is without the trailing /... if it had one
+ self.depot_side = View.Path(depot_side, is_depot=True)
+ self.client_side = View.Path(client_side, is_depot=False)
+ self.overlay = overlay # started with "+"
+ self.exclude = exclude # started with "-"
+ assert not (self.overlay and self.exclude)
+ self.depot_side.ensure_compatible(self.client_side)
+
+ def __str__(self):
+ c = " "
+ if self.overlay:
+ c = "+"
+ if self.exclude:
+ c = "-"
+ return "View.Mapping: %s%s -> %s" % \
+ (c, self.depot_side, self.client_side)
+
+ def map_depot_to_client(self, depot_path):
+ """Calculate the client path if using this mapping on the
+ given depot path; does not consider the effect of other
+ mappings in a view. Even excluded mappings are returned."""
+ matches, wildcards = self.depot_side.match_wildcards(depot_path)
+ if not matches:
+ return ""
+ client_path = self.client_side.fill_in_wildcards(wildcards)
+ return client_path
+
+ #
+ # View methods
+ #
+ def __init__(self):
+ self.mappings = []
+
+ def append(self, view_line):
+ """Parse a view line, splitting it into depot and client
+ sides. Append to self.mappings, preserving order."""
+
+ # Split the view line into exactly two words. P4 enforces
+ # structure on these lines that simplifies this quite a bit.
+ #
+ # Either or both words may be double-quoted.
+ # Single quotes do not matter.
+ # Double-quote marks cannot occur inside the words.
+ # A + or - prefix is also inside the quotes.
+ # There are no quotes unless they contain a space.
+ # The line is already white-space stripped.
+ # The two words are separated by a single space.
+ #
+ if view_line[0] == '"':
+ # First word is double quoted. Find its end.
+ close_quote_index = view_line.find('"', 1)
+ if close_quote_index <= 0:
+ die("No first-word closing quote found: %s" % view_line)
+ depot_side = view_line[1:close_quote_index]
+ # skip closing quote and space
+ rhs_index = close_quote_index + 1 + 1
+ else:
+ space_index = view_line.find(" ")
+ if space_index <= 0:
+ die("No word-splitting space found: %s" % view_line)
+ depot_side = view_line[0:space_index]
+ rhs_index = space_index + 1
+
+ if view_line[rhs_index] == '"':
+ # Second word is double quoted. Make sure there is a
+ # double quote at the end too.
+ if not view_line.endswith('"'):
+ die("View line with rhs quote should end with one: %s" %
+ view_line)
+ # skip the quotes
+ client_side = view_line[rhs_index+1:-1]
+ else:
+ client_side = view_line[rhs_index:]
+
+ # prefix + means overlay on previous mapping
+ overlay = False
+ if depot_side.startswith("+"):
+ overlay = True
+ depot_side = depot_side[1:]
+
+ # prefix - means exclude this path
+ exclude = False
+ if depot_side.startswith("-"):
+ exclude = True
+ depot_side = depot_side[1:]
+
+ m = View.Mapping(depot_side, client_side, overlay, exclude)
+ self.mappings.append(m)
+
+ def map_in_client(self, depot_path):
+ """Return the relative location in the client where this
+ depot file should live. Returns "" if the file should
+ not be mapped in the client."""
+
+ paths_filled = []
+ client_path = ""
+
+ # look at later entries first
+ for m in self.mappings[::-1]:
+
+ # see where will this path end up in the client
+ p = m.map_depot_to_client(depot_path)
+
+ if p == "":
+ # Depot path does not belong in client. Must remember
+ # this, as previous items should not cause files to
+ # exist in this path either. Remember that the list is
+ # being walked from the end, which has higher precedence.
+ # Overlap mappings do not exclude previous mappings.
+ if not m.overlay:
+ paths_filled.append(m.client_side)
+
+ else:
+ # This mapping matched; no need to search any further.
+ # But, the mapping could be rejected if the client path
+ # has already been claimed by an earlier mapping.
+ already_mapped_in_client = False
+ for f in paths_filled:
+ # this is View.Path.match
+ if f.match(p):
+ already_mapped_in_client = True
+ break
+ if not already_mapped_in_client:
+ # Include this file, unless it is from a line that
+ # explicitly said to exclude it.
+ if not m.exclude:
+ client_path = p
+
+ # a match, even if rejected, always stops the search
+ break
+
+ return client_path
+
class P4Sync(Command, P4UserMap):
delete_actions = ( "delete", "move/delete", "purge" )
@@ -1216,8 +1428,7 @@ class P4Sync(Command, P4UserMap):
self.p4BranchesInGit = []
self.cloneExclude = []
self.useClientSpec = False
- self.clientSpecDirs = []
- self.haveSingleFileClientViews = False
+ self.clientSpecDirs = None
if gitConfig("git-p4.syncFromOrigin") == "false":
self.syncWithOrigin = False
@@ -1268,30 +1479,7 @@ class P4Sync(Command, P4UserMap):
def stripRepoPath(self, path, prefixes):
if self.useClientSpec:
-
- # if using the client spec, we use the output directory
- # specified in the client. For example, a view
- # //depot/foo/branch/... //client/branch/foo/...
- # will end up putting all foo/branch files into
- # branch/foo/
- for val in self.clientSpecDirs:
- if self.haveSingleFileClientViews and len(path) == abs(val[1][0]) and path == val[0]:
- # since 'path' is a depot file path, if it matches the LeftMap,
- # then the View is a single file mapping, so use the entire rightMap
- # first two tests above avoid the last == test for common cases
- path = val[1][1]
- # now strip out the client (//client/...)
- path = re.sub("^(//[^/]+/)", '', path)
- # the rest is local client path
- return path
-
- if path.startswith(val[0]):
- # replace the depot path with the client path
- path = path.replace(val[0], val[1][1])
- # now strip out the client (//client/...)
- path = re.sub("^(//[^/]+/)", '', path)
- # the rest is all path
- return path
+ return self.clientSpecDirs.map_in_client(path)
if self.keepRepoPath:
prefixes = [re.sub("^(//[^/]+/).*", r'\1', prefixes[0])]
@@ -1441,19 +1629,17 @@ class P4Sync(Command, P4UserMap):
filesToDelete = []
for f in files:
- includeFile = True
- for val in self.clientSpecDirs:
- if f['path'].startswith(val[0]):
- if val[1][0] <= 0:
- includeFile = False
- break
+ # if using a client spec, only add the files that have
+ # a path in the client
+ if self.clientSpecDirs:
+ if self.clientSpecDirs.map_in_client(f['path']) == "":
+ continue
- if includeFile:
- filesForCommit.append(f)
- if f['action'] in self.delete_actions:
- filesToDelete.append(f)
- else:
- filesToRead.append(f)
+ filesForCommit.append(f)
+ if f['action'] in self.delete_actions:
+ filesToDelete.append(f)
+ else:
+ filesToRead.append(f)
# deleted files...
for f in filesToDelete:
@@ -1892,60 +2078,31 @@ class P4Sync(Command, P4UserMap):
def getClientSpec(self):
- specList = p4CmdList( "client -o" )
- temp = {}
- for entry in specList:
- for k,v in entry.iteritems():
- if k.startswith("View"):
-
- # p4 has these %%1 to %%9 arguments in specs to
- # reorder paths; which we can't handle (yet :)
- if re.search('%%\d', v) != None:
- print "Sorry, can't handle %%n arguments in client specs"
- sys.exit(1)
- if re.search('\*', v) != None:
- print "Sorry, can't handle * mappings in client specs"
- sys.exit(1)
-
- if v.startswith('"'):
- start = 1
- else:
- start = 0
- index = v.find("...")
-
- # save the "client view"; i.e the RHS of the view
- # line that tells the client where to put the
- # files for this view.
-
- # check for individual file mappings - those which have no '...'
- if index < 0 :
- v,cv = v.strip().split()
- v = v[start:]
- self.haveSingleFileClientViews = True
- else:
- cv = v[index+3:].strip() # +3 to remove previous '...'
- cv=cv[:-3]
- v = v[start:index]
-
- # now save the view; +index means included, -index
- # means it should be filtered out.
- if v.startswith("-"):
- v = v[1:]
- include = -len(v)
- else:
- include = len(v)
+ specList = p4CmdList("client -o")
+ if len(specList) != 1:
+ die('Output from "client -o" is %d lines, expecting 1' %
+ len(specList))
+
+ # dictionary of all client parameters
+ entry = specList[0]
+
+ # just the keys that start with "View"
+ view_keys = [ k for k in entry.keys() if k.startswith("View") ]
+
+ # hold this new View
+ view = View()
- # store the View #number for sorting
- # and the View string itself (this last for documentation)
- temp[v] = (include, cv, int(k[4:]),k)
+ # append the lines, in order, to the view
+ for view_num in range(len(view_keys)):
+ k = "View%d" % view_num
+ if k not in view_keys:
+ die("Expected view key %s missing" % k)
+ view.append(entry[k])
- self.clientSpecDirs = temp.items()
- # Perforce ViewNN with higher #numbers override those with lower
- # reverse sort on the View #number
- self.clientSpecDirs.sort( lambda x, y: y[1][2] - x[1][2] )
+ self.clientSpecDirs = view
if self.verbose:
- for val in self.clientSpecDirs:
- print "clientSpecDirs: %s %s" % (val[0],val[1])
+ for i, m in enumerate(self.clientSpecDirs.mappings):
+ print "clientSpecDirs %d: %s" % (i, str(m))
def run(self, args):
self.depotPaths = []
diff --git a/t/t9809-git-p4-client-view.sh b/t/t9809-git-p4-client-view.sh
index 1cc83c5..c9471d5 100755
--- a/t/t9809-git-p4-client-view.sh
+++ b/t/t9809-git-p4-client-view.sh
@@ -107,7 +107,7 @@ test_expect_success 'wildcard ... only supported at end of spec' '
test_must_fail "$GITP4" clone --use-client-spec --dest="$git" //depot
'
-test_expect_failure 'basic map' '
+test_expect_success 'basic map' '
client_view "//depot/dir1/... //client/cli1/..." &&
files="cli1/file11 cli1/file12" &&
client_verify $files &&
@@ -116,7 +116,7 @@ test_expect_failure 'basic map' '
git_verify $files
'
-test_expect_failure 'client view with no mappings' '
+test_expect_success 'client view with no mappings' '
client_view &&
client_verify &&
test_when_finished cleanup_git &&
@@ -124,7 +124,7 @@ test_expect_failure 'client view with no mappings' '
git_verify
'
-test_expect_failure 'single file map' '
+test_expect_success 'single file map' '
client_view "//depot/dir1/file11 //client/file11" &&
files="file11" &&
client_verify $files &&
@@ -144,7 +144,7 @@ test_expect_success 'later mapping takes precedence (entire repo)' '
git_verify $files
'
-test_expect_failure 'later mapping takes precedence (partial repo)' '
+test_expect_success 'later mapping takes precedence (partial repo)' '
client_view "//depot/dir1/... //client/..." \
"//depot/dir2/... //client/..." &&
files="file21 file22" &&
@@ -159,7 +159,7 @@ test_expect_failure 'later mapping takes precedence (partial repo)' '
# dir1 cannot go to cli12 since it was filled by dir2
# dir1 also does not go to cli3, since the second rule
# noticed that it matched, but was already filled
-test_expect_failure 'depot path matching rejected client path' '
+test_expect_success 'depot path matching rejected client path' '
client_view "//depot/dir1/... //client/cli3/..." \
"//depot/dir1/... //client/cli12/..." \
"//depot/dir2/... //client/cli12/..." &&
@@ -172,7 +172,7 @@ test_expect_failure 'depot path matching rejected client path' '
# since both have the same //client/..., the exclusion
# rule keeps everything out
-test_expect_failure 'exclusion wildcard, client rhs same (odd)' '
+test_expect_success 'exclusion wildcard, client rhs same (odd)' '
client_view "//depot/... //client/..." \
"-//depot/dir2/... //client/..." &&
client_verify &&
@@ -201,7 +201,7 @@ test_expect_success 'exclusion single file' '
git_verify $files
'
-test_expect_failure 'overlay wildcard' '
+test_expect_success 'overlay wildcard' '
client_view "//depot/dir1/... //client/cli/..." \
"+//depot/dir2/... //client/cli/...\n" &&
files="cli/file11 cli/file12 cli/file21 cli/file22" &&
@@ -211,7 +211,7 @@ test_expect_failure 'overlay wildcard' '
git_verify $files
'
-test_expect_failure 'overlay single file' '
+test_expect_success 'overlay single file' '
client_view "//depot/dir1/... //client/cli/..." \
"+//depot/dir2/file21 //client/cli/file21" &&
files="cli/file11 cli/file12 cli/file21" &&
@@ -221,7 +221,7 @@ test_expect_failure 'overlay single file' '
git_verify $files
'
-test_expect_failure 'exclusion with later inclusion' '
+test_expect_success 'exclusion with later inclusion' '
client_view "//depot/... //client/..." \
"-//depot/dir2/... //client/dir2/..." \
"//depot/dir2/... //client/dir2incl/..." &&
@@ -232,7 +232,7 @@ test_expect_failure 'exclusion with later inclusion' '
git_verify $files
'
-test_expect_failure 'quotes on rhs only' '
+test_expect_success 'quotes on rhs only' '
client_view "//depot/dir1/... \"//client/cdir 1/...\"" &&
client_verify "cdir 1/file11" "cdir 1/file12" &&
test_when_finished cleanup_git &&
@@ -266,7 +266,7 @@ test_expect_success 'rename files to introduce spaces' '
"dir 2/file21" "dir 2/file22"
'
-test_expect_failure 'quotes on lhs only' '
+test_expect_success 'quotes on lhs only' '
client_view "\"//depot/dir 1/...\" //client/cdir1/..." &&
files="cdir1/file11 cdir1/file12" &&
client_verify $files &&
@@ -275,7 +275,7 @@ test_expect_failure 'quotes on lhs only' '
client_verify $files
'
-test_expect_failure 'quotes on both sides' '
+test_expect_success 'quotes on both sides' '
client_view "\"//depot/dir 1/...\" \"//client/cdir 1/...\"" &&
client_verify "cdir 1/file11" "cdir 1/file12" &&
test_when_finished cleanup_git &&
--
1.7.8.1.407.gd70cb
^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 6/6] git-p4: view spec documentation
2012-01-02 23:05 [PATCH 0/6] git-p4: view spec improvements Pete Wyckoff
` (4 preceding siblings ...)
2012-01-02 23:05 ` [PATCH 5/6] git-p4: rewrite view handling Pete Wyckoff
@ 2012-01-02 23:05 ` Pete Wyckoff
5 siblings, 0 replies; 7+ messages in thread
From: Pete Wyckoff @ 2012-01-02 23:05 UTC (permalink / raw)
To: git; +Cc: Gary Gibbons
Signed-off-by: Pete Wyckoff <pw@padd.com>
---
Documentation/git-p4.txt | 40 +++++++++++++++++++++++++++-------------
1 files changed, 27 insertions(+), 13 deletions(-)
diff --git a/Documentation/git-p4.txt b/Documentation/git-p4.txt
index f97b1c5..84f5b91 100644
--- a/Documentation/git-p4.txt
+++ b/Documentation/git-p4.txt
@@ -230,12 +230,7 @@ git repository:
--use-client-spec::
Use a client spec to find the list of interesting files in p4.
- The client spec is discovered using 'p4 client -o' which checks
- the 'P4CLIENT' environment variable and returns a mapping of
- depot files to workspace files. Note that a depot path is
- still required, but files found in the path that match in
- the client spec view will be laid out according to the client
- spec.
+ See the "CLIENT SPEC" section below.
Clone options
~~~~~~~~~~~~~
@@ -304,6 +299,27 @@ p4 revision specifier on the end:
See 'p4 help revisions' for the full syntax of p4 revision specifiers.
+CLIENT SPEC
+-----------
+The p4 client specification is maintained with the 'p4 client' command
+and contains among other fields, a View that specifies how the depot
+is mapped into the client repository. Git-p4 can consult the client
+spec when given the '--use-client-spec' option or useClientSpec
+variable.
+
+The full syntax for a p4 view is documented in 'p4 help views'. Git-p4
+knows only a subset of the view syntax. It understands multi-line
+mappings, overlays with '+', exclusions with '-' and double-quotes
+around whitespace. Of the possible wildcards, git-p4 only handles
+'...', and only when it is at the end of the path. Git-p4 will complain
+if it encounters an unhandled wildcard.
+
+The name of the client can be given to git-p4 in multiple ways. The
+variable 'git-p4.client' takes precedence if it exists. Otherwise,
+normal p4 mechanisms of determining the client are used: environment
+variable P4CLIENT, a file referenced by P4CONFIG, or the local host name.
+
+
BRANCH DETECTION
----------------
P4 does not have the same concept of a branch as git. Instead,
@@ -387,9 +403,7 @@ git-p4.host::
git-p4.client::
Client specified as an option to all p4 commands, with
- '-c <client>'. This can also be used as a way to find
- the client spec for the 'useClientSpec' option.
- The environment variable 'P4CLIENT' can be used instead.
+ '-c <client>', including the client spec.
Clone and sync variables
~~~~~~~~~~~~~~~~~~~~~~~~
@@ -417,10 +431,10 @@ git config --add git-p4.branchList main:branchB
-------------
git-p4.useClientSpec::
- Specify that the p4 client spec to be used to identify p4 depot
- paths of interest. This is equivalent to specifying the option
- '--use-client-spec'. The variable 'git-p4.client' can be used
- to specify the name of the client.
+ Specify that the p4 client spec should be used to identify p4
+ depot paths of interest. This is equivalent to specifying the
+ option '--use-client-spec'. See the "CLIENT SPEC" section above.
+ This variable is a boolean, not the name of a p4 client.
Submit variables
~~~~~~~~~~~~~~~~
--
1.7.8.1.407.gd70cb
^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2012-01-02 23:08 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-01-02 23:05 [PATCH 0/6] git-p4: view spec improvements Pete Wyckoff
2012-01-02 23:05 ` [PATCH 1/6] git-p4: test client view handling Pete Wyckoff
2012-01-02 23:05 ` [PATCH 2/6] git-p4: fix test for unsupported P4 Client Views Pete Wyckoff
2012-01-02 23:05 ` [PATCH 3/6] git-p4: sort client views by reverse View number Pete Wyckoff
2012-01-02 23:05 ` [PATCH 4/6] git-p4: support single file p4 client view maps Pete Wyckoff
2012-01-02 23:05 ` [PATCH 5/6] git-p4: rewrite view handling Pete Wyckoff
2012-01-02 23:05 ` [PATCH 6/6] git-p4: view spec documentation Pete Wyckoff
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).