* [PATCH] Added sub get_owner_file which checks if there's a file with project owner name @ 2008-01-29 3:36 Bruno Ribas 2008-01-29 11:26 ` Jakub Narebski 0 siblings, 1 reply; 12+ messages in thread From: Bruno Ribas @ 2008-01-29 3:36 UTC (permalink / raw) To: git; +Cc: Bruno Ribas This file ($projectroot/$project/owner) is good to have when we don't want to maintain a project list AND when we share same SSH account for all projects, using ssh_acl for example. Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br> --- gitweb/gitweb.perl | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl index 6256641..fac5f78 100755 --- a/gitweb/gitweb.perl +++ b/gitweb/gitweb.perl @@ -1754,6 +1754,15 @@ sub git_get_project_list_from_file { } } +sub get_owner_file { + my $owner_file = shift; + + open my $fd, "$owner_file" or return undef; + my $owner = <$fd>; + close $fd; + return to_utf8($owner); +} + sub git_get_project_owner { my $project = shift; my $owner; @@ -1767,6 +1776,11 @@ sub git_get_project_owner { if (exists $gitweb_project_owner->{$project}) { $owner = $gitweb_project_owner->{$project}; } + + if ( -f "$projectroot/$project/owner" ) { + $owner = get_owner_file("$projectroot/$project/owner"); + } + if (!defined $owner) { $owner = get_file_owner("$projectroot/$project"); } -- 1.5.3.8 ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 3:36 [PATCH] Added sub get_owner_file which checks if there's a file with project owner name Bruno Ribas @ 2008-01-29 11:26 ` Jakub Narebski 2008-01-29 14:25 ` Bruno Cesar Ribas 0 siblings, 1 reply; 12+ messages in thread From: Jakub Narebski @ 2008-01-29 11:26 UTC (permalink / raw) To: Bruno Ribas; +Cc: git Bruno Ribas <ribas@c3sl.ufpr.br> writes: > This file ($projectroot/$project/owner) is good to have when we don't want to > maintain a project list AND when we share same SSH account for all projects, > using ssh_acl for example. > > Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br> This explanation is a bit too complicated; it explains farther reasons, instead of immediate ones: you don't want to maintain project list file, and all repository directories have to have the same owner (for example when the same SSH account is shared for all projects, using ssh_acl to control access instead). Besides with new faster config reader we probably would want to allow to use config file to set owner, instead of adding yet another file to the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3 "gitweb: Use config file for repository description and URLs" This would have the advantage that you could use system config (/etc/gitconfig) to set fallback owner instead of relying on filesystem. I'm not sure what should be the preference, though: gitweb.owner, then $GIT_DIR/owner, or vice versa? I guess that reading $GIT_DIR/owner should take preference, as it is needed also for projects list page, where ordinary we didn't read individual repositories configuration. I guess that it is meant to be post 1.5.4, isn't it? > --- > gitweb/gitweb.perl | 14 ++++++++++++++ > 1 files changed, 14 insertions(+), 0 deletions(-) > > diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl > index 6256641..fac5f78 100755 > --- a/gitweb/gitweb.perl > +++ b/gitweb/gitweb.perl > @@ -1754,6 +1754,15 @@ sub git_get_project_list_from_file { > } > } > > +sub get_owner_file { > + my $owner_file = shift; Here you use spaces instead of tabs in indent. > + > + open my $fd, "$owner_file" or return undef; open my $fd, $owner_file or return undef; would be simpler. > + my $owner = <$fd>; > + close $fd; > + return to_utf8($owner); > +} I wonder if we should just bite the bullet and replace all such by generic subroutine called e.g. read_singleline_file, or something like that. Or perhaps not, if we want to read alternatively from config, with different config variable key names and different preferences of file/config priority... > + > sub git_get_project_owner { > my $project = shift; > my $owner; > @@ -1767,6 +1776,11 @@ sub git_get_project_owner { > if (exists $gitweb_project_owner->{$project}) { > $owner = $gitweb_project_owner->{$project}; > } > + > + if ( -f "$projectroot/$project/owner" ) { > + $owner = get_owner_file("$projectroot/$project/owner"); > + } > + Here you use spaces. I think that you can lose spaces around condition in the above 'if'. > if (!defined $owner) { > $owner = get_file_owner("$projectroot/$project"); > } > -- > 1.5.3.8 I hope that doesn't mean that this patch is based on v1.5.3.8 gitweb... -- Jakub Narebski Poland ShadeHawk on #git ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 11:26 ` Jakub Narebski @ 2008-01-29 14:25 ` Bruno Cesar Ribas 2008-01-29 15:28 ` Jakub Narebski 0 siblings, 1 reply; 12+ messages in thread From: Bruno Cesar Ribas @ 2008-01-29 14:25 UTC (permalink / raw) To: Jakub Narebski; +Cc: git On Tue, Jan 29, 2008 at 03:26:31AM -0800, Jakub Narebski wrote: > Bruno Ribas <ribas@c3sl.ufpr.br> writes: > > > This file ($projectroot/$project/owner) is good to have when we don't want to > > maintain a project list AND when we share same SSH account for all projects, > > using ssh_acl for example. > > > > Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br> > > This explanation is a bit too complicated; it explains farther > reasons, instead of immediate ones: you don't want to maintain project > list file, and all repository directories have to have the same owner > (for example when the same SSH account is shared for all projects, > using ssh_acl to control access instead). I'm sorry about this complicated explanation. > > Besides with new faster config reader we probably would want to allow > to use config file to set owner, instead of adding yet another file to > the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3 > "gitweb: Use config file for repository description and URLs" > This would have the advantage that you could use system config > (/etc/gitconfig) to set fallback owner instead of relying on > filesystem. I'm not sure what should be the preference, though: > gitweb.owner, then $GIT_DIR/owner, or vice versa? I guess that > reading $GIT_DIR/owner should take preference, as it is needed also > for projects list page, where ordinary we didn't read individual > repositories configuration. Reading $GIT_DIR/owner would be the preference, Maybe it can generate project list page faster when machine have high IO waits (WA). Having gitweb.owner is good too, but as you said I don't need to read individual repositories configuration. Having another file at the repo area is not a problem (my say). Sometimes having files appears to be more organized than having everything in one file (my say again). I even made another patch about cloneURL, instead of looking for inside files and stuff, i made gitweb.conf a variable that says: - If i have a prefix path for HTTP,SSH,GIT[protocol] Then if this variable is set gitweb only mounts... like HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm" and gitweb sets it to $HTTPREFIX/$project I made this because I don't want to set each project it's clone URL, so this makes thing easier! What do you think? > > I guess that it is meant to be post 1.5.4, isn't it? > > > --- > > gitweb/gitweb.perl | 14 ++++++++++++++ > > 1 files changed, 14 insertions(+), 0 deletions(-) > > > > diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl > > index 6256641..fac5f78 100755 > > --- a/gitweb/gitweb.perl > > +++ b/gitweb/gitweb.perl > > @@ -1754,6 +1754,15 @@ sub git_get_project_list_from_file { > > } > > } > > > > +sub get_owner_file { > > + my $owner_file = shift; > > Here you use spaces instead of tabs in indent. > > > + > > + open my $fd, "$owner_file" or return undef; > > open my $fd, $owner_file or return undef; > > would be simpler. > > > + my $owner = <$fd>; > > + close $fd; > > + return to_utf8($owner); > > +} > > I wonder if we should just bite the bullet and replace all such by > generic subroutine called e.g. read_singleline_file, or something like > that. Or perhaps not, if we want to read alternatively from config, > with different config variable key names and different preferences of > file/config priority... > > > + > > sub git_get_project_owner { > > my $project = shift; > > my $owner; > > @@ -1767,6 +1776,11 @@ sub git_get_project_owner { > > if (exists $gitweb_project_owner->{$project}) { > > $owner = $gitweb_project_owner->{$project}; > > } > > + > > + if ( -f "$projectroot/$project/owner" ) { > > + $owner = get_owner_file("$projectroot/$project/owner"); > > + } > > + > > Here you use spaces. I think that you can lose spaces around condition > in the above 'if'. > > > if (!defined $owner) { > > $owner = get_file_owner("$projectroot/$project"); > > } > > -- > > 1.5.3.8 > > I hope that doesn't mean that this patch is based on v1.5.3.8 > gitweb... > > -- > Jakub Narebski > Poland > ShadeHawk on #git -- Bruno Ribas - ribas@c3sl.ufpr.br http://web.inf.ufpr.br/ribas C3SL: http://www.c3sl.ufpr.br ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 14:25 ` Bruno Cesar Ribas @ 2008-01-29 15:28 ` Jakub Narebski 2008-01-29 17:22 ` Bruno Cesar Ribas 2008-01-29 20:53 ` Nagy Balázs 0 siblings, 2 replies; 12+ messages in thread From: Jakub Narebski @ 2008-01-29 15:28 UTC (permalink / raw) To: Bruno Cesar Ribas; +Cc: git On Tue, 29 Jan 2008, Bruno Cesar Ribas wrote: > On Tue, Jan 29, 2008 at 03:26:31AM -0800, Jakub Narebski wrote: >> Bruno Ribas <ribas@c3sl.ufpr.br> writes: >> >>> This file ($projectroot/$project/owner) is good to have when we don't want to >>> maintain a project list AND when we share same SSH account for all projects, >>> using ssh_acl for example. >>> >>> Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br> >> >> This explanation is a bit too complicated; it explains farther >> reasons, instead of immediate ones: you don't want to maintain project >> list file, and all repository directories have to have the same owner >> (for example when the same SSH account is shared for all projects, >> using ssh_acl to control access instead). > > I'm sorry about this complicated explanation. It is not [that] bad description, but it could be better. Also, 80 columns word wrap is good, but 72-76 would be even better :-) >> Besides with new faster config reader we probably would want to allow >> to use config file to set owner, instead of adding yet another file to >> the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3 >> "gitweb: Use config file for repository description and URLs" >> This would have the advantage that you could use system config >> (/etc/gitconfig) to set fallback owner instead of relying on >> filesystem. I'm not sure what should be the preference, though: >> gitweb.owner, then $GIT_DIR/owner, or vice versa? I guess that >> reading $GIT_DIR/owner should take preference, as it is needed also >> for projects list page, where ordinary we didn't read individual >> repositories configuration. > > Reading $GIT_DIR/owner would be the preference, Maybe it can generate project > list page faster when machine have high IO waits (WA). Yes, I also think so. Two file reads (description + owner) should be still faster than one running git-config, and parsing its output. But I think if IO matters it is better to generate projects list; you can even use gitweb for that, or you can simply add a line with URL escaped project name (project path) relative to $projectroot, separated by space from the URL escaped (URI-encoded) project owner. See also "Gitweb repositories" section in gitweb/INSTALL. Adding projects is rare event. > Having gitweb.owner is good too, but as you said I don't need to read > individual repositories configuration. > > Having another file at the repo area is not a problem (my say). Sometimes > having files appears to be more organized than having everything in one file > (my say again). By the way, I have forgot to ask you to add description of new 'owner' file to "Per-repository gitweb configuration" section in gitweb/README > I even made another patch about cloneURL, instead of looking for inside files > and stuff, i made gitweb.conf a variable that says: > - If i have a prefix path for HTTP,SSH,GIT[protocol] > Then if this variable is set gitweb only mounts... like > HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm" > and gitweb sets it to $HTTPREFIX/$project > > I made this because I don't want to set each project it's clone URL, so this > makes thing easier! What do you think? I hope that this hack predates latest improvements to gitweb/README, as you have just reimplemented GITWEB_BASE_URL build configuration variable (only single base URL), and @git_base_url_list, which you can set in gitweb config file (by default gitweb_config.perl). If you have read current code carefully, you should notice that currently gitweb generates URLs for repository in the following way: 1. Per repository configuration: a. $projectroot/$project/cloneurl (one line perl URL) b. multivalued gitweb.url configuration variable in project config 2. Global gitweb configuration a. $prefix/$project for each $prefix element in @git_base_url_list, which is set in gitweb_config.perl 3. Build time defaults a. Single value in @git_base_url_list set using GITWEB_BASE_URL build configuration variable 4. Otherwise it is not set (it is empty). -- Jakub Narebski Poland ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 15:28 ` Jakub Narebski @ 2008-01-29 17:22 ` Bruno Cesar Ribas 2008-01-29 18:27 ` Jakub Narebski 2008-01-29 20:53 ` Nagy Balázs 1 sibling, 1 reply; 12+ messages in thread From: Bruno Cesar Ribas @ 2008-01-29 17:22 UTC (permalink / raw) To: Jakub Narebski; +Cc: git On Tue, Jan 29, 2008 at 04:28:19PM +0100, Jakub Narebski wrote: > On Tue, 29 Jan 2008, Bruno Cesar Ribas wrote: > > On Tue, Jan 29, 2008 at 03:26:31AM -0800, Jakub Narebski wrote: > >> Bruno Ribas <ribas@c3sl.ufpr.br> writes: > >> > >>> This file ($projectroot/$project/owner) is good to have when we don't want to > >>> maintain a project list AND when we share same SSH account for all projects, > >>> using ssh_acl for example. > >>> > >>> Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br> > >> > >> This explanation is a bit too complicated; it explains farther > >> reasons, instead of immediate ones: you don't want to maintain project > >> list file, and all repository directories have to have the same owner > >> (for example when the same SSH account is shared for all projects, > >> using ssh_acl to control access instead). > > > > I'm sorry about this complicated explanation. > > It is not [that] bad description, but it could be better. Also, 80 columns > word wrap is good, but 72-76 would be even better :-) > > >> Besides with new faster config reader we probably would want to allow > >> to use config file to set owner, instead of adding yet another file to > >> the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3 > >> "gitweb: Use config file for repository description and URLs" > >> This would have the advantage that you could use system config > >> (/etc/gitconfig) to set fallback owner instead of relying on > >> filesystem. I'm not sure what should be the preference, though: > >> gitweb.owner, then $GIT_DIR/owner, or vice versa? I guess that > >> reading $GIT_DIR/owner should take preference, as it is needed also > >> for projects list page, where ordinary we didn't read individual > >> repositories configuration. > > > > Reading $GIT_DIR/owner would be the preference, Maybe it can generate project > > list page faster when machine have high IO waits (WA). > > Yes, I also think so. Two file reads (description + owner) should be > still faster than one running git-config, and parsing its output. > > But I think if IO matters it is better to generate projects list; you > can even use gitweb for that, or you can simply add a line with URL > escaped project name (project path) relative to $projectroot, separated > by space from the URL escaped (URI-encoded) project owner. See also > "Gitweb repositories" section in gitweb/INSTALL. Adding projects is > rare event. > > > Having gitweb.owner is good too, but as you said I don't need to read > > individual repositories configuration. > > > > Having another file at the repo area is not a problem (my say). Sometimes > > having files appears to be more organized than having everything in one file > > (my say again). > > By the way, I have forgot to ask you to add description of new 'owner' > file to "Per-repository gitweb configuration" section in gitweb/README I'm on the way to add description of 'owner' file, before commint should I implement gitweb.owner too? then that README comes with two way of seting owner. Or let only owner file for now? > > > I even made another patch about cloneURL, instead of looking for inside files > > and stuff, i made gitweb.conf a variable that says: > > - If i have a prefix path for HTTP,SSH,GIT[protocol] > > Then if this variable is set gitweb only mounts... like > > HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm" > > and gitweb sets it to $HTTPREFIX/$project > > > > I made this because I don't want to set each project it's clone URL, so this > > makes thing easier! What do you think? > > I hope that this hack predates latest improvements to gitweb/README, > as you have just reimplemented GITWEB_BASE_URL build configuration > variable (only single base URL), and @git_base_url_list, which you > can set in gitweb config file (by default gitweb_config.perl). Ok, I'll check recent version of gitweb and i'll send this commit if relevant =) > > If you have read current code carefully, you should notice that > currently gitweb generates URLs for repository in the following way: > > 1. Per repository configuration: > a. $projectroot/$project/cloneurl (one line perl URL) > b. multivalued gitweb.url configuration variable in project config > 2. Global gitweb configuration > a. $prefix/$project for each $prefix element in @git_base_url_list, > which is set in gitweb_config.perl > 3. Build time defaults > a. Single value in @git_base_url_list set using GITWEB_BASE_URL > build configuration variable > 4. Otherwise it is not set (it is empty). > > -- > Jakub Narebski > Poland -- Bruno Ribas - ribas@c3sl.ufpr.br http://web.inf.ufpr.br/ribas C3SL: http://www.c3sl.ufpr.br ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 17:22 ` Bruno Cesar Ribas @ 2008-01-29 18:27 ` Jakub Narebski 0 siblings, 0 replies; 12+ messages in thread From: Jakub Narebski @ 2008-01-29 18:27 UTC (permalink / raw) To: Bruno Cesar Ribas; +Cc: git On Tue, 29 Jan 2008, Bruno Cesar Ribas wrote: > On Tue, Jan 29, 2008 at 04:28:19PM +0100, Jakub Narebski wrote: [cut] By the way, can I ask you to remove the parts of quoted email you do not reply to? It means leave only those parts you answer. It makes following the flow of discussion easier. TIA. >> By the way, I have forgot to ask you to add description of new 'owner' >> file to "Per-repository gitweb configuration" section in gitweb/README > > I'm on the way to add description of 'owner' file, before commint should I > implement gitweb.owner too? then that README comes with two way of seting > owner. Or let only owner file for now? If you add also implementation of gitweb.owner support (following for example git_get_project_description), then mention it in description of 'owner' file in gitweb/README. If you don't, just explain about 'owner'. Relevant fragment of gitweb/README you can follow: * description (or gitweb.description) Short (shortened by default to 25 characters in the projects list page) single line description of a project (of a repository). Plain text file; HTML will be escaped. By default set to Unnamed repository; edit this file to name it for gitweb. from the template during creating repository. You can use gitweb.description repo configuration variable, but the file takes precendence. Describe format, fallback to gitweb.owner, then fallback to directory owner (filesystem). >>> I even made another patch about cloneURL, instead of looking for inside files >>> and stuff, i made gitweb.conf a variable that says: >>> - If i have a prefix path for HTTP,SSH,GIT[protocol] >>> Then if this variable is set gitweb only mounts... like >>> HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm" >>> and gitweb sets it to $HTTPREFIX/$project >>> >>> I made this because I don't want to set each project it's clone URL, so this >>> makes thing easier! What do you think? >> >> I hope that this hack predates latest improvements to gitweb/README, >> as you have just reimplemented GITWEB_BASE_URL build configuration >> variable (only single base URL), and @git_base_url_list, which you >> can set in gitweb config file (by default gitweb_config.perl). > > Ok, I'll check recent version of gitweb and i'll send this commit > if relevant =) I think the change described above doesn't add any new functionality. [cut] -- Jakub Narebski Poland ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 15:28 ` Jakub Narebski 2008-01-29 17:22 ` Bruno Cesar Ribas @ 2008-01-29 20:53 ` Nagy Balázs 2008-01-29 21:36 ` Jakub Narebski 1 sibling, 1 reply; 12+ messages in thread From: Nagy Balázs @ 2008-01-29 20:53 UTC (permalink / raw) To: Jakub Narebski; +Cc: Bruno Cesar Ribas, git Hi, Jakub Narebski wrote: > But I think if IO matters it is better to generate projects list; you > can even use gitweb for that, or you can simply add a line with URL > escaped project name (project path) relative to $projectroot, separated > by space from the URL escaped (URI-encoded) project owner. See also > "Gitweb repositories" section in gitweb/INSTALL. Adding projects is > rare event. > Are you talking about I/O of an all-in CGI script? We can tune the performance of this script, but changing the GIT_DIR structure just because of a simple script is a bit overkill to me. What if this script creates the $projects_list file, for example when the $projectroot's mtime changes? We can even hold mtime info for every project's config file. Regards: -- Balazs Nagy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 20:53 ` Nagy Balázs @ 2008-01-29 21:36 ` Jakub Narebski 2008-01-30 15:59 ` Nagy Balázs 0 siblings, 1 reply; 12+ messages in thread From: Jakub Narebski @ 2008-01-29 21:36 UTC (permalink / raw) To: Nagy Balázs; +Cc: Bruno Cesar Ribas, git Nagy Balázs wrote: > Jakub Narebski wrote: >> But I think if IO matters it is better to generate projects list; you >> can even use gitweb for that, or you can simply add a line with URL >> escaped project name (project path) relative to $projectroot, separated >> by space from the URL escaped (URI-encoded) project owner. See also >> "Gitweb repositories" section in gitweb/INSTALL. Adding projects is >> rare event. >> > Are you talking about I/O of an all-in CGI script? I am talking there between I/O difference between situation (configuration) when $projects_list is a directory (default), or is a file. If $projects_list is a directory, gitweb scans directory structure to find git repositories, which for large number of repositories might take time, even with filesystem cache, and with depth of searching bound by $project_maxdepth. Add to that finding symbolic name of the owner of repository directory, or (with the patch) reading a file per repo with repo owner. Reading and pasing single text file avoids this; it is faster. But for small repos it is easier to scan directory, and difference in performance is not much. > We can tune the > performance of this script, but changing the GIT_DIR structure just > because of a simple script is a bit overkill to me. > > What if this script creates the $projects_list file, for example when > the $projectroot's mtime changes? We can even hold mtime info for every > project's config file. I don't understand what you wanted to say here. $projects_list file lists only project path (project name) and project owner. -- Jakub Narebski Poland ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-29 21:36 ` Jakub Narebski @ 2008-01-30 15:59 ` Nagy Balázs 2008-02-01 13:18 ` Jakub Narebski 0 siblings, 1 reply; 12+ messages in thread From: Nagy Balázs @ 2008-01-30 15:59 UTC (permalink / raw) To: Jakub Narebski; +Cc: Bruno Cesar Ribas, git Jakub Narebski wrote: > Nagy Balázs wrote: > >> Are you talking about I/O of an all-in CGI script? >> > > I am talking there between I/O difference between situation > (configuration) when $projects_list is a directory (default), > or is a file. If $projects_list is a directory, gitweb scans > directory structure to find git repositories, which for large > number of repositories might take time, even with filesystem > cache, and with depth of searching bound by $project_maxdepth. > Add to that finding symbolic name of the owner of repository > directory, or (with the patch) reading a file per repo with repo > owner. > We have two configurable options here: $projectroot and $projects_list. If $projects_list is a directory, we'll end up using a directory to get project list info, and using another one to actually handle the projects. In small repo area it's safe to have $projects_list empty. This is why I reference $projects_list as a file. If $projects_list is a file, we'll rely on a file which was generated some time ago and can't reflect the latest changes of $projectroot (but see below). >> We can tune the >> performance of this script, but changing the GIT_DIR structure just >> because of a simple script is a bit overkill to me. >> >> What if this script creates the $projects_list file, for example when >> the $projectroot's mtime changes? We can even hold mtime info for every >> project's config file. >> > > I don't understand what you wanted to say here. $projects_list file > lists only project path (project name) and project owner. > I mean it would be better to add this kind of metadata like description and owner's shoesize to config instead of a raw file. I understand row files are easier to read but reading a single cache file adn doing some stat()s are much easier. I can think of $project_lists as a cache file name, which can be maintained by gitweb.cgi, and these mtime values could be saved to $project_list to verify entries' validity. All we have to do is to maintain $project_list to be up to date. The best would be to have a separate projectlist maintainer script which handles two scenarios: 1| repo addition/deletion 2| repo config changes I don't have solution for the first scenario which would be a speed improvement in gitweb.cgi, this is why I suggest to put $project_list updater to a separate script. The second scenario could be handled by gitweb.cgi though, but it would be mere code duplication. Regards: -- Balazs Nagy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-01-30 15:59 ` Nagy Balázs @ 2008-02-01 13:18 ` Jakub Narebski 2008-02-01 16:11 ` Nagy Balázs 0 siblings, 1 reply; 12+ messages in thread From: Jakub Narebski @ 2008-02-01 13:18 UTC (permalink / raw) To: Nagy Balázs; +Cc: Bruno Cesar Ribas, git On Wed, 30 Jan 2008, Nagy Balázs wrote: > Jakub Narebski wrote: >> Nagy Balázs wrote: >> >>> Are you talking about I/O of an all-in CGI script? >>> >> >> I am talking there between I/O difference between situation >> (configuration) when $projects_list is a directory (default), >> or is a file. If $projects_list is a directory, gitweb scans >> directory structure to find git repositories, which for large >> number of repositories might take time, even with filesystem >> cache, and with depth of searching bound by $project_maxdepth. >> Add to that finding symbolic name of the owner of repository >> directory, or (with the patch) reading a file per repo with repo >> owner. >> > We have two configurable options here: $projectroot and $projects_list. > If $projects_list is a directory, we'll end up using a directory to get > project list info, and using another one to actually handle the > projects. In small repo area it's safe to have $projects_list empty. > This is why I reference $projects_list as a file. Besides the fact that using $projects_list file can speed up generating 'projects_list' page, it can also be used to either just restrict visibility of certain projects (some projects will be not visible in the projects list page, but will be still available when provided with project name), or restrict/refuse access (if GITWEB_STRICT_EXPORT aka $strict_export is true, only files shown in projects list page would be available to browse; it can be further restricted using "export-ok" mechanism). You can use $projects_list pointing to directory with symlinks to selected repositories residing under $projectroot for that. So it is not only $projects_list a file, or $projects_list undef (and fallback to $projectroot and $projects_list as directory). $projects_list as a directory different from $projectroot has sense in some cases too. > If $projects_list is a file, we'll rely on a file which was generated > some time ago and can't reflect the latest changes of $projectroot (but > see below). Creating projects is a rare event. You cannot do this remotely with git tools only. So I think it would be not very difficult and not very suprising to use some script to add new project, script which would ensure proper project configuration, perhaps setup proper SSH keys, and regenerate $projects_list file if it is what gitweb is using. [...] >>> What if this script creates the $projects_list file, for example when >>> the $projectroot's mtime changes? We can even hold mtime info for every >>> project's config file. >>> >> >> I don't understand what you wanted to say here. $projects_list file >> lists only project path (project name) and project owner. >> > I mean it would be better to add this kind of metadata like description > and owner's shoesize to config instead of a raw file. I understand row > files are easier to read but reading a single cache file adn doing some > stat()s are much easier. I can think of $project_lists as a cache file > name, which can be maintained by gitweb.cgi, and these mtime values > could be saved to $project_list to verify entries' validity. Err... I think that having some kind of cache for 'projects_list' page is a separate issue than using $projects_list file for a list (and owners) of projects. Besides I'd rather opt for the other side of spectrum: instead of gitweb checking for freshness of a 'cache', regenerate the cache or just delete it when you know that contents change: from a script adding a repository, from a script renaming or changing description or an owner of repository, from a script deleting repository or removing it from a list, from a post-update / post-receive hook if the cached info includes last change, etc. > All we have to do is to maintain $project_list to be up to date. The > best would be to have a separate projectlist maintainer script which > handles two scenarios: > > 1| repo addition/deletion > 2| repo config changes > > I don't have solution for the first scenario which would be a speed > improvement in gitweb.cgi, this is why I suggest to put $project_list > updater to a separate script. The second scenario could be handled by > gitweb.cgi though, but it would be mere code duplication. I was thinking about gitconfig file, but with limited syntax to be easily parseable from Perl, like git-cvsserver does, put in $projectroot, e.g. $projectroot/gitconfig, which would contain parts of repo config relevant to 'projects_list' page. It would use gitweb.<repo>.<key> syntax, where <key> is one of owner, description, and perhaps url. Or we could put it in gitweb_config.perl file, in the form of parsed config hash... well, it should be fairly easy to combine those two approaches with current code: use %config hash, and fill it from $projectroot/gitconfig if not set. Of course you would have the usual danger when dealing with data duplication, naley that they would get out of sync. And usual danger dealing with caches, that the validating needed and other system caches would make it perform *worse* than without cache. -- Jakub Narebski Poland ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-02-01 13:18 ` Jakub Narebski @ 2008-02-01 16:11 ` Nagy Balázs 2008-02-01 19:10 ` Robin Rosenberg 0 siblings, 1 reply; 12+ messages in thread From: Nagy Balázs @ 2008-02-01 16:11 UTC (permalink / raw) To: Jakub Narebski; +Cc: Bruno Cesar Ribas, git Jakob Narebski wrote: > Besides I'd rather opt for the other side of spectrum: instead of > gitweb checking for freshness of a 'cache', regenerate the cache > or just delete it when you know that contents change: from a script > adding a repository, from a script renaming or changing description > or an owner of repository, from a script deleting repository or > removing it from a list, from a post-update / post-receive hook if > the cached info includes last change, etc. > [...] > I was thinking about gitconfig file, but with limited syntax to be > easily parseable from Perl, like git-cvsserver does, put in $projectroot, > e.g. $projectroot/gitconfig, which would contain parts of repo config > relevant to 'projects_list' page. It would use gitweb.<repo>.<key> > syntax, where <key> is one of owner, description, and perhaps url. > > Or we could put it in gitweb_config.perl file, in the form of parsed > config hash... well, it should be fairly easy to combine those two > approaches with current code: use %config hash, and fill it from > $projectroot/gitconfig if not set. > > Of course you would have the usual danger when dealing with data > duplication, naley that they would get out of sync. And usual danger > dealing with caches, that the validating needed and other system > caches would make it perform *worse* than without cache. Well, I came from ClearCase world, were registry service holds all vob/view (repo) stuff. What if we implement more or less the same here? I mean if you have a lot of public git repos, or at least common for a couple of people, we could store all the fundamental data (local storage path, share URL) of a group of repos in a single database. We could call them depots. Moreover, we could use this as the source of git clone, God forbid, even remotely (some kind of git config depot.url=https://user:pass@git.example.org/depot.cgi/depotname). I could continue but I'm afraid I'll run out of oxygen :) I'm thinking of something similar: $ git depot # not the actual usage info, but to show all the parameters Usage: git depot create [-d | --description=<descr>] <depot> <path> git depot remove [-f | --force] <depot> git depot modify [-d | --description=<descr> ] [-p | --path=<base path>] [-n | --name=<depot>] <depot> git depot list git depot add [-a | --access=public|private] [-o | --owner=<owner>] <depot>[:<name>] <path> git depot update [-a | --access=public|private] [-o | --owner=<owner>] [-p | --path=<path>] <depot>:<name> git depot delete <depot>:<name> git depot show [-a | --all] [-l | --long] <depot> $ git depot list public /pub/scm "Public repository" $ git depot show public public:git /pub/scm/git.git (public, owner: "Junio C Hamano \ <gitster@pobox.com>", description: "The core git plumbing") public:linux/kernel/torvalds/linux-2.6 \ /pub/scm/linux/kernel/torvalds/linux-2.6.git (owner: \ "Linus Torvalds <torvalds@linux-foundation.org>, description: \ "Linus' Kernel Tree") $ git depot show --long public:git Repository: public:git Path: /pub/scm/git.git Clone URL: git://git.kernel.org/pub/scm/git.git Clone URL: http://www.kernel.org/pub/scm/git/git.git Access: public Owner: Junio C Hamano <gitster@pobox.com> Description: The core git plumbing -- # Access: private means only git depot show -a shows it $ mkdir ~/git; cd ~/git $ git clone public:git ... $ git clone . public:git/mygit ... I'd add some kind of cloneURL setter too, but the basic idea is this. Regards: -- Balazs Nagy ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name 2008-02-01 16:11 ` Nagy Balázs @ 2008-02-01 19:10 ` Robin Rosenberg 0 siblings, 0 replies; 12+ messages in thread From: Robin Rosenberg @ 2008-02-01 19:10 UTC (permalink / raw) To: Nagy Balázs; +Cc: Jakub Narebski, Bruno Cesar Ribas, git fredagen den 1 februari 2008 skrev Nagy Balázs: > Well, I came from ClearCase world, were registry service holds all > vob/view (repo) stuff. What if we implement more or less the same > here? I mean if you have a lot of public git repos, or at least common > for a couple of people, we could store all the fundamental data (local > storage path, share URL) of a group of repos in a single database. We > could call them depots. Moreover, we could use this as the source of > git clone, God forbid, even remotely (some kind of git config > depot.url=https://user:pass@git.example.org/depot.cgi/depotname). I > could continue but I'm afraid I'll run out of oxygen :) I'll fill in: LDAP. -- robin ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2008-02-01 20:01 UTC | newest] Thread overview: 12+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-01-29 3:36 [PATCH] Added sub get_owner_file which checks if there's a file with project owner name Bruno Ribas 2008-01-29 11:26 ` Jakub Narebski 2008-01-29 14:25 ` Bruno Cesar Ribas 2008-01-29 15:28 ` Jakub Narebski 2008-01-29 17:22 ` Bruno Cesar Ribas 2008-01-29 18:27 ` Jakub Narebski 2008-01-29 20:53 ` Nagy Balázs 2008-01-29 21:36 ` Jakub Narebski 2008-01-30 15:59 ` Nagy Balázs 2008-02-01 13:18 ` Jakub Narebski 2008-02-01 16:11 ` Nagy Balázs 2008-02-01 19:10 ` Robin Rosenberg
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).