[PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Bruno Ribas]

This file ($projectroot/$project/owner) is good to have when we don't want to
maintain a project  list AND when we share same SSH account for all projects,
using ssh_acl for example.

Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br>
---
 gitweb/gitweb.perl |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 6256641..fac5f78 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1754,6 +1754,15 @@ sub git_get_project_list_from_file {
 	}
 }
 
+sub get_owner_file {
+    my $owner_file = shift;
+
+	open my $fd, "$owner_file" or return undef;
+	my $owner = <$fd>;
+	close $fd;
+	return to_utf8($owner);
+}
+
 sub git_get_project_owner {
 	my $project = shift;
 	my $owner;
@@ -1767,6 +1776,11 @@ sub git_get_project_owner {
 	if (exists $gitweb_project_owner->{$project}) {
 		$owner = $gitweb_project_owner->{$project};
 	}
+
+    if ( -f "$projectroot/$project/owner" ) {
+        $owner = get_owner_file("$projectroot/$project/owner");
+    }
+
 	if (!defined $owner) {
 		$owner = get_file_owner("$projectroot/$project");
 	}
-- 
1.5.3.8

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Jakub Narebski]

Bruno Ribas <ribas@c3sl.ufpr.br> writes:

> This file ($projectroot/$project/owner) is good to have when we don't want to
> maintain a project  list AND when we share same SSH account for all projects,
> using ssh_acl for example.
>
> Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br>

This explanation is a bit too complicated; it explains farther
reasons, instead of immediate ones: you don't want to maintain project
list file, and all repository directories have to have the same owner
(for example when the same SSH account is shared for all projects,
using ssh_acl to control access instead).

Besides with new faster config reader we probably would want to allow
to use config file to set owner, instead of adding yet another file to
the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3
  "gitweb: Use config file for repository description and URLs"
This would have the advantage that you could use system config
(/etc/gitconfig) to set fallback owner instead of relying on
filesystem.  I'm not sure what should be the preference, though:
gitweb.owner, then $GIT_DIR/owner, or vice versa?  I guess that
reading $GIT_DIR/owner should take preference, as it is needed also
for projects list page, where ordinary we didn't read individual
repositories configuration.

I guess that it is meant to be post 1.5.4, isn't it?

> ---
>  gitweb/gitweb.perl |   14 ++++++++++++++
>  1 files changed, 14 insertions(+), 0 deletions(-)
> 
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 6256641..fac5f78 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -1754,6 +1754,15 @@ sub git_get_project_list_from_file {
>  	}
>  }
>  
> +sub get_owner_file {
> +    my $owner_file = shift;

Here you use spaces instead of tabs in indent.

> +
> +	open my $fd, "$owner_file" or return undef;

	open my $fd, $owner_file or return undef;

would be simpler.

> +	my $owner = <$fd>;
> +	close $fd;
> +	return to_utf8($owner);
> +}

I wonder if we should just bite the bullet and replace all such by
generic subroutine called e.g. read_singleline_file, or something like
that.  Or perhaps not, if we want to read alternatively from config,
with different config variable key names and different preferences of
file/config priority...

> +
>  sub git_get_project_owner {
>  	my $project = shift;
>  	my $owner;
> @@ -1767,6 +1776,11 @@ sub git_get_project_owner {
>  	if (exists $gitweb_project_owner->{$project}) {
>  		$owner = $gitweb_project_owner->{$project};
>  	}
> +
> +    if ( -f "$projectroot/$project/owner" ) {
> +        $owner = get_owner_file("$projectroot/$project/owner");
> +    }
> +

Here you use spaces. I think that you can lose spaces around condition
in the above 'if'.

>  	if (!defined $owner) {
>  		$owner = get_file_owner("$projectroot/$project");
>  	}
> -- 
> 1.5.3.8

I hope that doesn't mean that this patch is based on v1.5.3.8
gitweb...

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Bruno Cesar Ribas]

On Tue, Jan 29, 2008 at 03:26:31AM -0800, Jakub Narebski wrote:
> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> 
> > This file ($projectroot/$project/owner) is good to have when we don't want to
> > maintain a project  list AND when we share same SSH account for all projects,
> > using ssh_acl for example.
> >
> > Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br>
> 
> This explanation is a bit too complicated; it explains farther
> reasons, instead of immediate ones: you don't want to maintain project
> list file, and all repository directories have to have the same owner
> (for example when the same SSH account is shared for all projects,
> using ssh_acl to control access instead).

I'm sorry about this complicated explanation.

> 
> Besides with new faster config reader we probably would want to allow
> to use config file to set owner, instead of adding yet another file to
> the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3
>   "gitweb: Use config file for repository description and URLs"
> This would have the advantage that you could use system config
> (/etc/gitconfig) to set fallback owner instead of relying on
> filesystem.  I'm not sure what should be the preference, though:
> gitweb.owner, then $GIT_DIR/owner, or vice versa?  I guess that
> reading $GIT_DIR/owner should take preference, as it is needed also
> for projects list page, where ordinary we didn't read individual
> repositories configuration.

Reading $GIT_DIR/owner would be the preference, Maybe it can generate project
list page faster when machine have high IO waits (WA).

Having gitweb.owner is good too, but as you said I don't need to read
individual repositories configuration.

Having another file at the repo area is not a problem (my say). Sometimes
having files appears to be more organized than having everything in one file
(my say again).

I even made another patch about cloneURL, instead of looking for inside files
and stuff, i made gitweb.conf a variable that says:
- If i have a prefix path for HTTP,SSH,GIT[protocol]
Then if this variable is set gitweb only mounts... like
HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm"
and gitweb sets it to $HTTPREFIX/$project

I made this because I don't want to set each project it's clone URL, so this 
makes thing easier! What do you think?



> 
> I guess that it is meant to be post 1.5.4, isn't it?
> 
> > ---
> >  gitweb/gitweb.perl |   14 ++++++++++++++
> >  1 files changed, 14 insertions(+), 0 deletions(-)
> > 
> > diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> > index 6256641..fac5f78 100755
> > --- a/gitweb/gitweb.perl
> > +++ b/gitweb/gitweb.perl
> > @@ -1754,6 +1754,15 @@ sub git_get_project_list_from_file {
> >  	}
> >  }
> >  
> > +sub get_owner_file {
> > +    my $owner_file = shift;
> 
> Here you use spaces instead of tabs in indent.
> 
> > +
> > +	open my $fd, "$owner_file" or return undef;
> 
> 	open my $fd, $owner_file or return undef;
> 
> would be simpler.
> 
> > +	my $owner = <$fd>;
> > +	close $fd;
> > +	return to_utf8($owner);
> > +}
> 
> I wonder if we should just bite the bullet and replace all such by
> generic subroutine called e.g. read_singleline_file, or something like
> that.  Or perhaps not, if we want to read alternatively from config,
> with different config variable key names and different preferences of
> file/config priority...
> 
> > +
> >  sub git_get_project_owner {
> >  	my $project = shift;
> >  	my $owner;
> > @@ -1767,6 +1776,11 @@ sub git_get_project_owner {
> >  	if (exists $gitweb_project_owner->{$project}) {
> >  		$owner = $gitweb_project_owner->{$project};
> >  	}
> > +
> > +    if ( -f "$projectroot/$project/owner" ) {
> > +        $owner = get_owner_file("$projectroot/$project/owner");
> > +    }
> > +
> 
> Here you use spaces. I think that you can lose spaces around condition
> in the above 'if'.
> 
> >  	if (!defined $owner) {
> >  		$owner = get_file_owner("$projectroot/$project");
> >  	}
> > -- 
> > 1.5.3.8
> 
> I hope that doesn't mean that this patch is based on v1.5.3.8
> gitweb...
> 
> -- 
> Jakub Narebski
> Poland
> ShadeHawk on #git

-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br 

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Jakub Narebski]

On Tue, 29 Jan 2008, Bruno Cesar Ribas wrote:
> On Tue, Jan 29, 2008 at 03:26:31AM -0800, Jakub Narebski wrote:
>> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
>> 
>>> This file ($projectroot/$project/owner) is good to have when we don't want to
>>> maintain a project  list AND when we share same SSH account for all projects,
>>> using ssh_acl for example.
>>>
>>> Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br>
>> 
>> This explanation is a bit too complicated; it explains farther
>> reasons, instead of immediate ones: you don't want to maintain project
>> list file, and all repository directories have to have the same owner
>> (for example when the same SSH account is shared for all projects,
>> using ssh_acl to control access instead).
> 
> I'm sorry about this complicated explanation.

It is not [that] bad description, but it could be better. Also, 80 columns
word wrap is good, but 72-76 would be even better :-)

>> Besides with new faster config reader we probably would want to allow
>> to use config file to set owner, instead of adding yet another file to
>> the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3
>>   "gitweb: Use config file for repository description and URLs"
>> This would have the advantage that you could use system config
>> (/etc/gitconfig) to set fallback owner instead of relying on
>> filesystem.  I'm not sure what should be the preference, though:
>> gitweb.owner, then $GIT_DIR/owner, or vice versa?  I guess that
>> reading $GIT_DIR/owner should take preference, as it is needed also
>> for projects list page, where ordinary we didn't read individual
>> repositories configuration.
> 
> Reading $GIT_DIR/owner would be the preference, Maybe it can generate project
> list page faster when machine have high IO waits (WA).

Yes, I also think so. Two file reads (description + owner) should be
still faster than one running git-config, and parsing its output.

But I think if IO matters it is better to generate projects list; you
can even use gitweb for that, or you can simply add a line with URL
escaped project name (project path) relative to $projectroot, separated
by space from the URL escaped (URI-encoded) project owner.  See also
"Gitweb repositories" section in gitweb/INSTALL.  Adding projects is
rare event.
 
> Having gitweb.owner is good too, but as you said I don't need to read
> individual repositories configuration.
> 
> Having another file at the repo area is not a problem (my say). Sometimes
> having files appears to be more organized than having everything in one file
> (my say again).

By the way, I have forgot to ask you to add description of new 'owner'
file to "Per-repository gitweb configuration" section in gitweb/README

> I even made another patch about cloneURL, instead of looking for inside files
> and stuff, i made gitweb.conf a variable that says:
> - If i have a prefix path for HTTP,SSH,GIT[protocol]
> Then if this variable is set gitweb only mounts... like
> HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm"
> and gitweb sets it to $HTTPREFIX/$project
> 
> I made this because I don't want to set each project it's clone URL, so this 
> makes thing easier! What do you think?

I hope that this hack predates latest improvements to gitweb/README,
as you have just reimplemented GITWEB_BASE_URL build configuration
variable (only single base URL), and @git_base_url_list, which you
can set in gitweb config file (by default gitweb_config.perl).
 
If you have read current code carefully, you should notice that
currently gitweb generates URLs for repository in the following way:

 1. Per repository configuration:
    a. $projectroot/$project/cloneurl (one line perl URL)
    b. multivalued gitweb.url configuration variable in project config
 2. Global gitweb configuration
    a. $prefix/$project for each $prefix element in @git_base_url_list,
       which is set in gitweb_config.perl
 3. Build time defaults
    a. Single value in @git_base_url_list set using GITWEB_BASE_URL
       build configuration variable
 4. Otherwise it is not set (it is empty).

-- 
Jakub Narebski
Poland

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Bruno Cesar Ribas]

On Tue, Jan 29, 2008 at 04:28:19PM +0100, Jakub Narebski wrote:
> On Tue, 29 Jan 2008, Bruno Cesar Ribas wrote:
> > On Tue, Jan 29, 2008 at 03:26:31AM -0800, Jakub Narebski wrote:
> >> Bruno Ribas <ribas@c3sl.ufpr.br> writes:
> >> 
> >>> This file ($projectroot/$project/owner) is good to have when we don't want to
> >>> maintain a project  list AND when we share same SSH account for all projects,
> >>> using ssh_acl for example.
> >>>
> >>> Signed-off-by: Bruno Ribas <ribas@c3sl.ufpr.br>
> >> 
> >> This explanation is a bit too complicated; it explains farther
> >> reasons, instead of immediate ones: you don't want to maintain project
> >> list file, and all repository directories have to have the same owner
> >> (for example when the same SSH account is shared for all projects,
> >> using ssh_acl to control access instead).
> > 
> > I'm sorry about this complicated explanation.
> 
> It is not [that] bad description, but it could be better. Also, 80 columns
> word wrap is good, but 72-76 would be even better :-)
> 
> >> Besides with new faster config reader we probably would want to allow
> >> to use config file to set owner, instead of adding yet another file to
> >> the repo area; see commit 0e121a2cd42d28bc4034feedf8a13c5a91f85bd3
> >>   "gitweb: Use config file for repository description and URLs"
> >> This would have the advantage that you could use system config
> >> (/etc/gitconfig) to set fallback owner instead of relying on
> >> filesystem.  I'm not sure what should be the preference, though:
> >> gitweb.owner, then $GIT_DIR/owner, or vice versa?  I guess that
> >> reading $GIT_DIR/owner should take preference, as it is needed also
> >> for projects list page, where ordinary we didn't read individual
> >> repositories configuration.
> > 
> > Reading $GIT_DIR/owner would be the preference, Maybe it can generate project
> > list page faster when machine have high IO waits (WA).
> 
> Yes, I also think so. Two file reads (description + owner) should be
> still faster than one running git-config, and parsing its output.
> 
> But I think if IO matters it is better to generate projects list; you
> can even use gitweb for that, or you can simply add a line with URL
> escaped project name (project path) relative to $projectroot, separated
> by space from the URL escaped (URI-encoded) project owner.  See also
> "Gitweb repositories" section in gitweb/INSTALL.  Adding projects is
> rare event.
>  
> > Having gitweb.owner is good too, but as you said I don't need to read
> > individual repositories configuration.
> > 
> > Having another file at the repo area is not a problem (my say). Sometimes
> > having files appears to be more organized than having everything in one file
> > (my say again).
> 
> By the way, I have forgot to ask you to add description of new 'owner'
> file to "Per-repository gitweb configuration" section in gitweb/README

I'm on the way to add description of 'owner' file, before commint should I
implement gitweb.owner too? then that README comes with two way of seting
owner. Or let only owner file for now?

> 
> > I even made another patch about cloneURL, instead of looking for inside files
> > and stuff, i made gitweb.conf a variable that says:
> > - If i have a prefix path for HTTP,SSH,GIT[protocol]
> > Then if this variable is set gitweb only mounts... like
> > HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm"
> > and gitweb sets it to $HTTPREFIX/$project
> > 
> > I made this because I don't want to set each project it's clone URL, so this 
> > makes thing easier! What do you think?
> 
> I hope that this hack predates latest improvements to gitweb/README,
> as you have just reimplemented GITWEB_BASE_URL build configuration
> variable (only single base URL), and @git_base_url_list, which you
> can set in gitweb config file (by default gitweb_config.perl).

Ok, I'll check recent version of gitweb and i'll send this commit if relevant
=)

>  
> If you have read current code carefully, you should notice that
> currently gitweb generates URLs for repository in the following way:
> 
>  1. Per repository configuration:
>     a. $projectroot/$project/cloneurl (one line perl URL)
>     b. multivalued gitweb.url configuration variable in project config
>  2. Global gitweb configuration
>     a. $prefix/$project for each $prefix element in @git_base_url_list,
>        which is set in gitweb_config.perl
>  3. Build time defaults
>     a. Single value in @git_base_url_list set using GITWEB_BASE_URL
>        build configuration variable
>  4. Otherwise it is not set (it is empty).
> 
> -- 
> Jakub Narebski
> Poland

-- 
Bruno Ribas - ribas@c3sl.ufpr.br
http://web.inf.ufpr.br/ribas
C3SL: http://www.c3sl.ufpr.br 

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Jakub Narebski]

On Tue, 29 Jan 2008, Bruno Cesar Ribas wrote:
> On Tue, Jan 29, 2008 at 04:28:19PM +0100, Jakub Narebski wrote:

[cut]

By the way, can I ask you to remove the parts of quoted email you
do not reply to? It means leave only those parts you answer. It makes
following the flow of discussion easier. TIA.
 
>> By the way, I have forgot to ask you to add description of new 'owner'
>> file to "Per-repository gitweb configuration" section in gitweb/README
> 
> I'm on the way to add description of 'owner' file, before commint should I
> implement gitweb.owner too? then that README comes with two way of seting
> owner. Or let only owner file for now?

If you add also implementation of gitweb.owner support (following
for example git_get_project_description), then mention it in
description of 'owner' file in gitweb/README. If you don't, just
explain about 'owner'.

Relevant fragment of gitweb/README you can follow:

 * description (or gitweb.description)
   Short (shortened by default to 25 characters in the projects list page)
   single line description of a project (of a repository). Plain text file;
   HTML will be escaped. By default set to
     Unnamed repository; edit this file to name it for gitweb.
   from the template during creating repository. You can use
   gitweb.description repo configuration variable, but the file takes
   precendence.

Describe format, fallback to gitweb.owner, then fallback to directory
owner (filesystem).

>>> I even made another patch about cloneURL, instead of looking for inside files
>>> and stuff, i made gitweb.conf a variable that says:
>>> - If i have a prefix path for HTTP,SSH,GIT[protocol]
>>> Then if this variable is set gitweb only mounts... like
>>> HTTPPREFIX="http://git.c3sl.ufpr.br/pub/scm"
>>> and gitweb sets it to $HTTPREFIX/$project
>>> 
>>> I made this because I don't want to set each project it's clone URL, so this 
>>> makes thing easier! What do you think?
>> 
>> I hope that this hack predates latest improvements to gitweb/README,
>> as you have just reimplemented GITWEB_BASE_URL build configuration
>> variable (only single base URL), and @git_base_url_list, which you
>> can set in gitweb config file (by default gitweb_config.perl).
> 
> Ok, I'll check recent version of gitweb and i'll send this commit
> if relevant =)

I think the change described above doesn't add any new functionality.

[cut]
-- 
Jakub Narebski
Poland

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Nagy Balázs]

Hi,

Jakub Narebski wrote:
> But I think if IO matters it is better to generate projects list; you
> can even use gitweb for that, or you can simply add a line with URL
> escaped project name (project path) relative to $projectroot, separated
> by space from the URL escaped (URI-encoded) project owner.  See also
> "Gitweb repositories" section in gitweb/INSTALL.  Adding projects is
> rare event.
>   
Are you talking about I/O of an all-in CGI script?  We can tune the 
performance of this script, but changing the GIT_DIR structure just 
because of a simple script is a bit overkill to me.

What if this script creates the $projects_list file, for example when 
the $projectroot's mtime changes?  We can even hold mtime info for every 
project's config file.

Regards:
-- 
Balazs Nagy

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Jakub Narebski]

Nagy Balázs wrote:
> Jakub Narebski wrote:

>> But I think if IO matters it is better to generate projects list; you
>> can even use gitweb for that, or you can simply add a line with URL
>> escaped project name (project path) relative to $projectroot, separated
>> by space from the URL escaped (URI-encoded) project owner.  See also
>> "Gitweb repositories" section in gitweb/INSTALL.  Adding projects is
>> rare event.
>>   
> Are you talking about I/O of an all-in CGI script?  

I am talking there between I/O difference between situation
(configuration) when $projects_list is a directory (default),
or is a file. If $projects_list is a directory, gitweb scans
directory structure to find git repositories, which for large
number of repositories might take time, even with filesystem
cache, and with depth of searching bound by $project_maxdepth.
Add to that finding symbolic name of the owner of repository
directory, or (with the patch) reading a file per repo with repo
owner.

Reading and pasing single text file avoids this; it is faster.
But for small repos it is easier to scan directory, and difference
in performance is not much.

> We can tune the  
> performance of this script, but changing the GIT_DIR structure just 
> because of a simple script is a bit overkill to me.
> 
> What if this script creates the $projects_list file, for example when 
> the $projectroot's mtime changes?  We can even hold mtime info for every 
> project's config file.

I don't understand what you wanted to say here. $projects_list file
lists only project path (project name) and project owner.

-- 
Jakub Narebski
Poland

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Nagy Balázs]

Jakub Narebski wrote:
> Nagy Balázs wrote:
>   
>> Are you talking about I/O of an all-in CGI script?  
>>     
>
> I am talking there between I/O difference between situation
> (configuration) when $projects_list is a directory (default),
> or is a file. If $projects_list is a directory, gitweb scans
> directory structure to find git repositories, which for large
> number of repositories might take time, even with filesystem
> cache, and with depth of searching bound by $project_maxdepth.
> Add to that finding symbolic name of the owner of repository
> directory, or (with the patch) reading a file per repo with repo
> owner.
>   
We have two configurable options here: $projectroot and $projects_list.  
If $projects_list is a directory, we'll end up using a directory to get 
project list info, and using another one to actually handle the 
projects.  In small repo area it's safe to have $projects_list empty.  
This is why I reference $projects_list as a file.

If $projects_list is a file, we'll rely on a file which was generated 
some time ago and can't reflect the latest changes of $projectroot (but 
see below).
>> We can tune the  
>> performance of this script, but changing the GIT_DIR structure just 
>> because of a simple script is a bit overkill to me.
>>
>> What if this script creates the $projects_list file, for example when 
>> the $projectroot's mtime changes?  We can even hold mtime info for every 
>> project's config file.
>>     
>
> I don't understand what you wanted to say here. $projects_list file
> lists only project path (project name) and project owner.
>   
I mean it would be better to add this kind of metadata like description 
and owner's shoesize to config instead of a raw file.  I understand row 
files are easier to read but reading a single cache file adn doing some 
stat()s are much easier.  I can think of $project_lists as a cache file 
name, which can be maintained by gitweb.cgi, and these mtime values 
could be saved to $project_list to verify entries' validity.

All we have to do is to maintain $project_list to be up to date.  The 
best would be to have a separate projectlist maintainer script which 
handles two scenarios:

1| repo addition/deletion
2| repo config changes

I don't have solution for the first scenario which would be a speed 
improvement in gitweb.cgi, this is why I suggest to put $project_list 
updater to a separate script.  The second scenario could be handled by 
gitweb.cgi though, but it would be mere code duplication.

Regards:
-- 
Balazs Nagy

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Jakub Narebski]

On Wed, 30 Jan 2008, Nagy Balázs wrote:
> Jakub Narebski wrote:
>> Nagy Balázs wrote:
>>   
>>> Are you talking about I/O of an all-in CGI script?  
>>>     
>>
>> I am talking there between I/O difference between situation
>> (configuration) when $projects_list is a directory (default),
>> or is a file. If $projects_list is a directory, gitweb scans
>> directory structure to find git repositories, which for large
>> number of repositories might take time, even with filesystem
>> cache, and with depth of searching bound by $project_maxdepth.
>> Add to that finding symbolic name of the owner of repository
>> directory, or (with the patch) reading a file per repo with repo
>> owner.
>>   
> We have two configurable options here: $projectroot and $projects_list.  
> If $projects_list is a directory, we'll end up using a directory to get 
> project list info, and using another one to actually handle the 
> projects.  In small repo area it's safe to have $projects_list empty.  
> This is why I reference $projects_list as a file.

Besides the fact that using $projects_list file can speed up generating
'projects_list' page, it can also be used to either just restrict
visibility of certain projects (some projects will be not visible
in the projects list page, but will be still available when provided
with project name), or restrict/refuse access (if GITWEB_STRICT_EXPORT
aka $strict_export is true, only files shown in projects list page
would be available to browse; it can be further restricted using
"export-ok" mechanism).

You can use $projects_list pointing to directory with symlinks to
selected repositories residing under $projectroot for that. So it
is not only $projects_list a file, or $projects_list undef (and fallback
to $projectroot and $projects_list as directory). $projects_list as
a directory different from $projectroot has sense in some cases too.

> If $projects_list is a file, we'll rely on a file which was generated 
> some time ago and can't reflect the latest changes of $projectroot (but 
> see below).

Creating projects is a rare event. You cannot do this remotely with git
tools only. So I think it would be not very difficult and not very
suprising to use some script to add new project, script which would
ensure proper project configuration, perhaps setup proper SSH keys, and
regenerate $projects_list file if it is what gitweb is using.

[...]
>>> What if this script creates the $projects_list file, for example when 
>>> the $projectroot's mtime changes?  We can even hold mtime info for every 
>>> project's config file.
>>>     
>>
>> I don't understand what you wanted to say here. $projects_list file
>> lists only project path (project name) and project owner.
>>   
> I mean it would be better to add this kind of metadata like description 
> and owner's shoesize to config instead of a raw file.  I understand row 
> files are easier to read but reading a single cache file adn doing some 
> stat()s are much easier.  I can think of $project_lists as a cache file 
> name, which can be maintained by gitweb.cgi, and these mtime values 
> could be saved to $project_list to verify entries' validity.

Err... I think that having some kind of cache for 'projects_list' page
is a separate issue than using $projects_list file for a list
(and owners) of projects.

Besides I'd rather opt for the other side of spectrum: instead of
gitweb checking for freshness of a 'cache', regenerate the cache
or just delete it when you know that contents change: from a script
adding a repository, from a script renaming or changing description
or an owner of repository, from a script deleting repository or
removing it from a list, from a post-update / post-receive hook if
the cached info includes last change, etc.
 
> All we have to do is to maintain $project_list to be up to date.  The 
> best would be to have a separate projectlist maintainer script which 
> handles two scenarios:
> 
> 1| repo addition/deletion
> 2| repo config changes
> 
> I don't have solution for the first scenario which would be a speed 
> improvement in gitweb.cgi, this is why I suggest to put $project_list 
> updater to a separate script.  The second scenario could be handled by 
> gitweb.cgi though, but it would be mere code duplication.

I was thinking about gitconfig file, but with limited syntax to be
easily parseable from Perl, like git-cvsserver does, put in $projectroot,
e.g. $projectroot/gitconfig, which would contain parts of repo config
relevant to 'projects_list' page.  It would use gitweb.<repo>.<key>
syntax, where <key> is one of owner, description, and perhaps url.

Or we could put it in gitweb_config.perl file, in the form of parsed
config hash... well, it should be fairly easy to combine those two
approaches with current code: use %config hash, and fill it from
$projectroot/gitconfig if not set.

Of course you would have the usual danger when dealing with data
duplication, naley that they would get out of sync. And usual danger
dealing with caches, that the validating needed and other system
caches would make it perform *worse* than without cache.

-- 
Jakub Narebski
Poland

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Nagy Balázs]

Jakob Narebski wrote:
> Besides I'd rather opt for the other side of spectrum: instead of
> gitweb checking for freshness of a 'cache', regenerate the cache
> or just delete it when you know that contents change: from a script
> adding a repository, from a script renaming or changing description
> or an owner of repository, from a script deleting repository or
> removing it from a list, from a post-update / post-receive hook if
> the cached info includes last change, etc.
>   
[...]

> I was thinking about gitconfig file, but with limited syntax to be
> easily parseable from Perl, like git-cvsserver does, put in $projectroot,
> e.g. $projectroot/gitconfig, which would contain parts of repo config
> relevant to 'projects_list' page.  It would use gitweb.<repo>.<key>
> syntax, where <key> is one of owner, description, and perhaps url.
>
> Or we could put it in gitweb_config.perl file, in the form of parsed
> config hash... well, it should be fairly easy to combine those two
> approaches with current code: use %config hash, and fill it from
> $projectroot/gitconfig if not set.
>
> Of course you would have the usual danger when dealing with data
> duplication, naley that they would get out of sync. And usual danger
> dealing with caches, that the validating needed and other system
> caches would make it perform *worse* than without cache.
Well, I came from ClearCase world, were registry service holds all 
vob/view (repo) stuff.  What if we implement more or less the same 
here?  I mean if you have a lot of public git repos, or at least common 
for a couple of people, we could store all the fundamental data (local 
storage path, share URL) of a group of repos in a single database.  We 
could call them depots.  Moreover, we could use this as the source of 
git clone, God forbid, even remotely (some kind of git config 
depot.url=https://user:pass@git.example.org/depot.cgi/depotname).  I 
could continue but I'm afraid I'll run out of oxygen :)

I'm thinking of something similar:

$ git depot # not the actual usage info, but to show all the parameters
Usage: git depot create [-d | --description=<descr>] <depot> <path>
       git depot remove [-f | --force] <depot>
       git depot modify [-d | --description=<descr> ]
         [-p | --path=<base path>] [-n | --name=<depot>] <depot>
       git depot list
       git depot add [-a | --access=public|private] [-o | --owner=<owner>]
         <depot>[:<name>] <path>
       git depot update [-a | --access=public|private]
         [-o | --owner=<owner>] [-p | --path=<path>] <depot>:<name>
       git depot delete <depot>:<name>
       git depot show [-a | --all] [-l | --long] <depot>
$ git depot list
public /pub/scm "Public repository"
$ git depot show public
public:git /pub/scm/git.git (public, owner: "Junio C Hamano \
<gitster@pobox.com>", description: "The core git plumbing")
public:linux/kernel/torvalds/linux-2.6 \
/pub/scm/linux/kernel/torvalds/linux-2.6.git (owner: \
"Linus Torvalds <torvalds@linux-foundation.org>, description: \
"Linus' Kernel Tree")
$ git depot show --long public:git
Repository: public:git
  Path: /pub/scm/git.git
  Clone URL: git://git.kernel.org/pub/scm/git.git
  Clone URL: http://www.kernel.org/pub/scm/git/git.git
  Access: public
  Owner: Junio C Hamano <gitster@pobox.com>
  Description: The core git plumbing
--
# Access: private means only git depot show -a shows it
$ mkdir ~/git; cd ~/git
$ git clone public:git
...
$ git clone . public:git/mygit
...

I'd add some kind of cloneURL setter too, but the basic idea is this.

Regards:
-- 
Balazs Nagy

Re: [PATCH] Added sub get_owner_file which checks if there's a file with project owner name

[Robin Rosenberg]

fredagen den 1 februari 2008 skrev Nagy Balázs:
> Well, I came from ClearCase world, were registry service holds all 
> vob/view (repo) stuff.  What if we implement more or less the same 
> here?  I mean if you have a lot of public git repos, or at least common 
> for a couple of people, we could store all the fundamental data (local 
> storage path, share URL) of a group of repos in a single database.  We 
> could call them depots.  Moreover, we could use this as the source of 
> git clone, God forbid, even remotely (some kind of git config 
> depot.url=https://user:pass@git.example.org/depot.cgi/depotname).  I 
> could continue but I'm afraid I'll run out of oxygen :)

I'll fill in: LDAP.

-- robin