[PATCH 2/2] gitweb: support hiding projects from user-visible lists

[PATCH 2/2] gitweb: support hiding projects from user-visible lists

[Matt McCutchen]

Signed-off-by: Matt McCutchen <matt@mattmccutchen.net>
---

My Web site has a single gitweb installation in which some of the
repositories are protected by a basic authentication login.  By virtue
of my aforementioned setup with gitweb and pulling at the same URL, the
login applies uniformly to both.  I had to include these repositories in
the projects_list because I use strict_export, but I want to hide them
when the user views the project list.  This patch implements that
feature, and the previous one fixes a bug I noticed along the way.

Matt

 gitweb/gitweb.perl |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)

diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
index 5357bcc..085cc60 100755
--- a/gitweb/gitweb.perl
+++ b/gitweb/gitweb.perl
@@ -1144,7 +1144,7 @@ sub untabify {
 
 sub project_in_list {
 	my $project = shift;
-	# Tell git_get_projects_list to include forks.
+	# Tell git_get_projects_list to include forks and hidden repositories.
 	my @list = git_get_projects_list(undef, 1);
 	return @list && scalar(grep { $_->{'path'} eq $project } @list);
 }
@@ -2174,15 +2174,18 @@ sub git_get_projects_list {
 		# 'git%2Fgit.git Linus+Torvalds'
 		# 'libs%2Fklibc%2Fklibc.git H.+Peter+Anvin'
 		# 'linux%2Fhotplug%2Fudev.git Greg+Kroah-Hartman'
+		#
+		# 1 in the third field hides the project from user-visible lists, e.g.:
+		# 'linux%2Fembargoed-security-fixes.git John+Doe 1'
 		my %paths;
 		open my ($fd), $projects_list or return;
 	PROJECT:
 		while (my $line = <$fd>) {
 			chomp $line;
-			my ($path, $owner) = split ' ', $line;
+			my ($path, $owner, $hidden) = split ' ', $line;
 			$path = unescape($path);
 			$owner = unescape($owner);
-			if (!defined $path) {
+			if (!defined $path || ($hidden && !$for_strict_export)) {
 				next;
 			}
 			if ($filter ne '') {
@@ -2227,6 +2230,8 @@ sub git_get_projects_list {
 	return @list;
 }
 
+# This is used to look up the owner of a project the user is already allowed to
+# see, so we shouldn't omit hidden repositories.
 our $gitweb_project_owner = undef;
 sub git_get_project_list_from_file {
 
@@ -2241,7 +2246,7 @@ sub git_get_project_list_from_file {
 		open (my $fd , $projects_list);
 		while (my $line = <$fd>) {
 			chomp $line;
-			my ($pr, $ow) = split ' ', $line;
+			my ($pr, $ow, $hidden) = split ' ', $line;
 			$pr = unescape($pr);
 			$ow = unescape($ow);
 			$gitweb_project_owner->{$pr} = to_utf8($ow);
-- 
1.6.1.rc2.27.gc7114

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

[Jakub Narebski]

Matt McCutchen <matt@mattmccutchen.net> writes:

Commit message, please?

> Signed-off-by: Matt McCutchen <matt@mattmccutchen.net>
> ---
> 
> My Web site has a single gitweb installation in which some of the
> repositories are protected by a basic authentication login.  By virtue
> of my aforementioned setup with gitweb and pulling at the same URL, the
> login applies uniformly to both.  I had to include these repositories in
> the projects_list because I use strict_export, but I want to hide them
> when the user views the project list.  This patch implements that
> feature, and the previous one fixes a bug I noticed along the way.
> 
> Matt

Cannot you do this with new $export_auth_hook gitweb configuration
variable, added by Alexander Gavrilov in 
   dd7f5f1 (gitweb: Add a per-repository authorization hook.)
It is used in check_export_ok subroutine, and is is checked also when
getting list of project from file

>From gitweb/INSTALL

  - Finally, it is possible to specify an arbitrary perl subroutine that
    will be called for each project to determine if it can be exported.
    The subroutine receives an absolute path to the project as its only
    parameter.

    For example, if you use mod_perl to run the script, and have dumb
    http protocol authentication configured for your repositories, you
    can use the following hook to allow access only if the user is
    authorized to read the files:

      $export_auth_hook = sub {
          use Apache2::SubRequest ();
          use Apache2::Const -compile => qw(HTTP_OK);
          my $path = "$_[0]/HEAD";
          my $r    = Apache2::RequestUtil->request;
          my $sub  = $r->lookup_file($path);
          return $sub->filename eq $path
              && $sub->status == Apache2::Const::HTTP_OK;
      };


>  gitweb/gitweb.perl |   13 +++++++++----
>  1 files changed, 9 insertions(+), 4 deletions(-)

No documentation, in gitweb/README or gitweb/INSTALL

> 
> diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl
> index 5357bcc..085cc60 100755
> --- a/gitweb/gitweb.perl
> +++ b/gitweb/gitweb.perl
> @@ -1144,7 +1144,7 @@ sub untabify {
>  
>  sub project_in_list {
>  	my $project = shift;
> -	# Tell git_get_projects_list to include forks.
> +	# Tell git_get_projects_list to include forks and hidden repositories.
>  	my @list = git_get_projects_list(undef, 1);
>  	return @list && scalar(grep { $_->{'path'} eq $project } @list);
>  }
> @@ -2174,15 +2174,18 @@ sub git_get_projects_list {
>  		# 'git%2Fgit.git Linus+Torvalds'
>  		# 'libs%2Fklibc%2Fklibc.git H.+Peter+Anvin'
>  		# 'linux%2Fhotplug%2Fudev.git Greg+Kroah-Hartman'
> +		#
> +		# 1 in the third field hides the project from user-visible lists, e.g.:
> +		# 'linux%2Fembargoed-security-fixes.git John+Doe 1'

I guess I'd rather use _last_ field, in the event we add project
description to project list file format.

>  		my %paths;
>  		open my ($fd), $projects_list or return;
>  	PROJECT:
>  		while (my $line = <$fd>) {
>  			chomp $line;
> -			my ($path, $owner) = split ' ', $line;
> +			my ($path, $owner, $hidden) = split ' ', $line;
>  			$path = unescape($path);
>  			$owner = unescape($owner);
> -			if (!defined $path) {
> +			if (!defined $path || ($hidden && !$for_strict_export)) {
>  				next;
>  			}
>  			if ($filter ne '') {
> @@ -2227,6 +2230,8 @@ sub git_get_projects_list {
>  	return @list;
>  }
>  
> +# This is used to look up the owner of a project the user is already allowed to
> +# see, so we shouldn't omit hidden repositories.
>  our $gitweb_project_owner = undef;
>  sub git_get_project_list_from_file {
>  
> @@ -2241,7 +2246,7 @@ sub git_get_project_list_from_file {
>  		open (my $fd , $projects_list);
>  		while (my $line = <$fd>) {
>  			chomp $line;
> -			my ($pr, $ow) = split ' ', $line;
> +			my ($pr, $ow, $hidden) = split ' ', $line;
>  			$pr = unescape($pr);
>  			$ow = unescape($ow);
>  			$gitweb_project_owner->{$pr} = to_utf8($ow);
> -- 
> 1.6.1.rc2.27.gc7114
> 

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

[Jakub Narebski]

By the way, your message [PATCH 2/2] should be threaded, i.e. be
response to [PATCH 1/2] (or to cover letter [PATCH 0/2]), to not
mistake it with other [PATCH 2/2] patches.

-- 
Jakub Narebski
Poland
ShadeHawk on #git

Sending a threaded patch series with Evolution

[Matt McCutchen]

On Sat, 2008-12-13 at 14:05 -0800, Jakub Narebski wrote: 
> By the way, your message [PATCH 2/2] should be threaded, i.e. be
> response to [PATCH 1/2] (or to cover letter [PATCH 0/2]), to not
> mistake it with other [PATCH 2/2] patches.

I'm using Evolution 2.24 as my mail client, and I have a wrapper script
"git draft-patch" that runs "git format-patch" and loads the patches
into my Evolution Drafts folder so I can look them over in the composer
and add non-commit-message text below the "---" if I wish before
sending.  I did pass --thread, but the composer changed the Message-Id,
breaking the threading.

I'll try to get this right in the future.  I can see two approaches to
doing so:

1. Check over / edit the patches in a text editor before loading them
into Drafts, and then send them by moving them directly to Outbox
(without using the composer) so that the Message-Id won't change.

2. Send the first patch, then manually edit the Message-Id Evolution
assigned it into the second patch, etc.

Neither of these approaches is particularly nice.  Does anyone have a
better idea?

-- 
Matt

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

[Matt McCutchen]

On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:
> Cannot you do this with new $export_auth_hook gitweb configuration
> variable, added by Alexander Gavrilov in 
>    dd7f5f1 (gitweb: Add a per-repository authorization hook.)
> It is used in check_export_ok subroutine, and is is checked also when
> getting list of project from file
> 
> >From gitweb/INSTALL
> 
>   - Finally, it is possible to specify an arbitrary perl subroutine that
>     will be called for each project to determine if it can be exported.
>     The subroutine receives an absolute path to the project as its only
>     parameter.
> 
>     For example, if you use mod_perl to run the script, and have dumb
>     http protocol authentication configured for your repositories, you
>     can use the following hook to allow access only if the user is
>     authorized to read the files:
> 
>       $export_auth_hook = sub {
>           use Apache2::SubRequest ();
>           use Apache2::Const -compile => qw(HTTP_OK);
>           my $path = "$_[0]/HEAD";
>           my $r    = Apache2::RequestUtil->request;
>           my $sub  = $r->lookup_file($path);
>           return $sub->filename eq $path
>               && $sub->status == Apache2::Const::HTTP_OK;
>       };

$export_auth_hook would work, and it would have the nice (but not
essential) feature of including private projects in the list shown to
suitably authenticated users.  The only problem is that my Web host
doesn't support mod_perl.  Is there a practical way to accomplish the
same thing as the above example in a CGI script?  I would like to avoid
reimplementing Apache authentication-checking functionality if at all
possible.

-- 
Matt

Re: [PATCH 2/2] gitweb: support hiding projects from user-visible lists

[Jakub Narebski]

On Wed, 2008-12-24, Matt McCutchen wrote:
> On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:
> >
> > Cannot you do this with new $export_auth_hook gitweb configuration
> > variable, added by Alexander Gavrilov in 
> >    dd7f5f1 (gitweb: Add a per-repository authorization hook.)
> > It is used in check_export_ok subroutine, and is is checked also when
> > getting list of project from file
> > 
> > From gitweb/INSTALL
[...]
> >     For example, if you use mod_perl to run the script, and have dumb
> >     http protocol authentication configured for your repositories, you
> >     can use the following hook to allow access only if the user is
> >     authorized to read the files:
[...]
 
> $export_auth_hook would work, and it would have the nice (but not
> essential) feature of including private projects in the list shown to
> suitably authenticated users.  The only problem is that my Web host
> doesn't support mod_perl.  Is there a practical way to accomplish the
> same thing as the above example in a CGI script?  I would like to avoid
> reimplementing Apache authentication-checking functionality if at all
> possible.

I know it is written that the example code is for mod_perl, but I
don't think it is mod_perl specific; have you checked if it works
for you? I assume that you use Apache, and have Apache Perl bindings
installed...

-- 
Jakub Narebski
Poland

gitweb config with some public, some basic-authenticated repos

[Matt McCutchen]

This thread's topic has moved from a proposed patch to how I should
configure my gitweb, so I'm updating the subject.  As a review: I have
several public repos and several basic-authentication realms, each of
which requires a single user and contains a single repo (some realms
might contain multiple repos in the future).  Each request has its
authorization checked by the Web server before it reaches gitweb, so my
main concern here is to avoid publicly disclosing the private repos'
paths, authors, and descriptions in the main project list.

On Fri, 2009-01-02 at 20:33 +0100, Jakub Narebski wrote: 
> On Wed, 2008-12-24, Matt McCutchen wrote:
> > On Sat, 2008-12-13 at 14:02 -0800, Jakub Narebski wrote:
> > >
> > > Cannot you do this with new $export_auth_hook gitweb configuration
> > > variable, added by Alexander Gavrilov in 
> > >    dd7f5f1 (gitweb: Add a per-repository authorization hook.)
> > > It is used in check_export_ok subroutine, and is is checked also when
> > > getting list of project from file
> > > 
> > > From gitweb/INSTALL
> [...]
> > >     For example, if you use mod_perl to run the script, and have dumb
> > >     http protocol authentication configured for your repositories, you
> > >     can use the following hook to allow access only if the user is
> > >     authorized to read the files:
> [...]
>  
> > $export_auth_hook would work, and it would have the nice (but not
> > essential) feature of including private projects in the list shown to
> > suitably authenticated users.  The only problem is that my Web host
> > doesn't support mod_perl.  Is there a practical way to accomplish the
> > same thing as the above example in a CGI script?  I would like to avoid
> > reimplementing Apache authentication-checking functionality if at all
> > possible.
> 
> I know it is written that the example code is for mod_perl, but I
> don't think it is mod_perl specific; have you checked if it works
> for you? I assume that you use Apache, and have Apache Perl bindings
> installed...

I'm quite sure that the code is mod_perl specific.  CGI scripts do get
some information from Apache via the environment, but interaction as
rich as executing Apache subrequests is only possible when the code is
running inside Apache via mod_perl.  In fact, the Apache2::SubRequest
and Apache2::RequestUtil modules are part of mod_perl.  To make sure I'm
not missing something, I tested the code on an Apache with mod_perl
enabled but gitweb executing as a CGI, and gitweb failed with the
following message:

        Can't locate object method "request" via package
        "Apache2::RequestUtil" at gitweb_config.perl line 60.

So this approach won't work for me.

But even ignoring this problem, I'm now thinking that trying to show
repos from *multiple authentication realms* in the main list according
to the user's credentials was a foolish idea.  I don't want to ask
anonymous visitors to my main list for multiple logins they probably
don't have, yet I think it would be poor practice from a predictability
standpoint for the list to behave differently if the user volunteers
login information that hasn't been requested.

Instead, I will use a separate project list file for public repositories
and for each realm, and no export_auth_hook.  This is simple and
requires no change to gitweb; my rewrite rule just has to tell my
gitweb_config via an environment variable which list to use.  Comments
on this solution?

(Note: I'm no longer advocating the hidden-repos feature at this time,
but I think I will still advocate the forks-and-strict-export bug fix
now that I have it written.)

-- 
Matt