Cannot access git-scm.com with Tor

Cannot access git-scm.com with Tor

[opal hart]

Hi,

I assume this community is at least slightly familiar with Tor and its
usefulness, so I'll cut to the chase. The Git website git-scm.com is
behind Cloudflare, which is currently configured to block suspicious
requests which unfortunately include Tor users. It would be greatly
appreciated to lower Cloudflare security settings to allow Tor users.
If the additional traffic really becomes an issue (I see the website
uses RoR backend [1]) then consider caching pages if that isn't already
done.

Ideally the use of Cloudflare would be avoided entirely, but
I will not get into the politics of that here. Being able to access the
static portions of the site behind Tor is enough for me currently.

[1]<https://github.com/git/git-scm.com>

Thanks,
-- 
wowaname <https://wowana.me/pgp>

Re: Cannot access git-scm.com with Tor

[Jeff King]

On Tue, May 07, 2019 at 07:00:28AM +0000, opal hart wrote:

> I assume this community is at least slightly familiar with Tor and its
> usefulness, so I'll cut to the chase. The Git website git-scm.com is
> behind Cloudflare, which is currently configured to block suspicious
> requests which unfortunately include Tor users. It would be greatly
> appreciated to lower Cloudflare security settings to allow Tor users.
> If the additional traffic really becomes an issue (I see the website
> uses RoR backend [1]) then consider caching pages if that isn't already
> done.

The "security settings" toggle for the site on Cloudflare is already set
to "low", and there are no other restrictions. Is it actually blocking?
Or is it asking for a CAPTCHA?

I can drop it to "essentially off". I don't think the site is very high
risk.  The site is _basically_[1] serving up static content (which isn't
to say there couldn't be some attack against the rails infrastructure,
but it's not like people are logging in, or that there's any non-public
data on the site).

Using Cloudflare or some other caching layer is a necessity. We serve
about 50-60GB of traffic daily, over 98% of which is cached. The load
was quite high when the caching was all done at the Rails layer (though
I wouldn't rule out that there are ways to configure Rails to make this
more optimal; none of the people active in maintaining the site is
particularly knowledgeable there).

-Peff

[1] It actually could be a static site entirely; there was some push in
    that direction a while ago, but there's a lot of tedious work that
    never got finished. Of course, we'd still need a caching layer.

Re: Cannot access git-scm.com with Tor

[opal hart]

On Tue, 7 May 2019 03:33:22 -0400
Jeff King <peff@peff.net> wrote:

> The "security settings" toggle for the site on Cloudflare is already set
> to "low", and there are no other restrictions. Is it actually blocking?
> Or is it asking for a CAPTCHA?

Sorry, it is indeed asking for a CAPTCHA. Google reCAPTCHA does not do
a great job at accomodating Tor users; it effectively blocks us with an
endless barrage of puzzles unless we log in to a Google account.

> I can drop it to "essentially off". I don't think the site is very high
> risk.  The site is _basically_[1] serving up static content (which isn't
> to say there couldn't be some attack against the rails infrastructure,
> but it's not like people are logging in, or that there's any non-public
> data on the site).

> Using Cloudflare or some other caching layer is a necessity. We serve
> about 50-60GB of traffic daily, over 98% of which is cached.

That's fine to use it as a CDN/cache; setting the site protection to
"essentially off" should indeed help. For whatever reason, Cloudflare
still isn't smart enough to allow greylisted IP addresses during times
of low/no abuse -- either that or a lot of people use CF with the
misconception that it's for anti-spam on blogs and forums, so it's
easier to just block Tor. CF representatives have been involved in
public discussions elsewhere, claiming they are working to improve the
experience for Tor Browser users, but these discussions have been going
on for years now, so I wouldn't hold my breath. That's why I have to
make webmasters aware of this, because not everyone realises how
inaccessible the web becomes for people who use Tor. Personally I use
it although I'm in a relatively free country, but there are others who
use it to escape censorship. And when services like Cloudflare
inadvertently act as censors, it defeats the purpose of Tor for this
group of people.

> The load
> was quite high when the caching was all done at the Rails layer (though
> I wouldn't rule out that there are ways to configure Rails to make this
> more optimal; none of the people active in maintaining the site is
> particularly knowledgeable there).

Yeah, if caching *is* handled directly on git-scm's side, I would
recommend something more suited for caching such as Varnish or
memcached (depending on the type of caching you're doing). Personally I
use memcached on a Tor hidden service I host and it seems to cut down
on server pressure from bot requests. This is why I even suggest to
webmasters to reconsider their use of CF; I'm not naïve and I do have
some experience with handling webserver abuse. But again, I won't argue
if the site genuinely benefits from using CF as a CDN.

> [1] It actually could be a static site entirely; there was some push in
>     that direction a while ago, but there's a lot of tedious work that
>     never got finished. Of course, we'd still need a caching layer.

It is indeed a lot of effort to change a site's inner workings. I'm
glad that this possibility is at least considered and I hope it's an
accomplishable goal in the future. It'd be nice to see the web revert
to its more static, sustainable days.

-- 
wowaname <https://wowana.me/pgp>

Re: Cannot access git-scm.com with Tor

[Jeff King]

On Wed, May 08, 2019 at 08:39:17AM +0000, opal hart wrote:

> On Tue, 7 May 2019 03:33:22 -0400
> Jeff King <peff@peff.net> wrote:
> 
> > The "security settings" toggle for the site on Cloudflare is already set
> > to "low", and there are no other restrictions. Is it actually blocking?
> > Or is it asking for a CAPTCHA?
> 
> Sorry, it is indeed asking for a CAPTCHA. Google reCAPTCHA does not do
> a great job at accomodating Tor users; it effectively blocks us with an
> endless barrage of puzzles unless we log in to a Google account.

Yeah, even though it at least provides _a_ way to get to the content,
I'd just as soon not inconvenience people for no reason (and in fact,
avoiding CAPTCHAs is why I had initially lowered it from the default).

I dropped the setting in Cloudflare to "essentially off". Using
torbrowser I didn't get a CAPTCHA either before or after the setting, so
I have no idea if it actually changed anything. But hopefully it is
better for you now.

I also checked the settings for git-scm.org (same content, different
domain), which had accidentally been left at "medium". Those are now at
off, too.

-Peff