Proxy

Proxy

[maillists0]

With NFS4's support for referrals and Kerberos, it seems like the
original reasons to prevent re-exporting of an NFS share might no
longer exist. With fs-proxy making its way into the mainline kernel
and things like cachefilesd, there are also very good reasons to allow
it. A proxy server with a persistent cache could give the ability to
robustly use shares across a WAN or do failover pairs with no need for
more complex replication. Speaking as an end-user, this would be very
desirable.

I see that others have implemented proxies with user-space NFS, which
seems reasonable but not optimal. What is the obstacle to allowing
re-exports with the standard nfs implentation? Is it possible at the
moment to patch a kernel to make this work? Anyone have experience
with it? Any input is appreciated.

Re: Proxy

[J. Bruce Fields]

On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
> With NFS4's support for referrals and Kerberos, it seems like the
> original reasons to prevent re-exporting of an NFS share might no
> longer exist. With fs-proxy making its way into the mainline kernel
> and things like cachefilesd, there are also very good reasons to allow
> it. A proxy server with a persistent cache could give the ability to
> robustly use shares across a WAN or do failover pairs with no need for
> more complex replication. Speaking as an end-user, this would be very
> desirable.
> 
> I see that others have implemented proxies with user-space NFS, which
> seems reasonable but not optimal. What is the obstacle to allowing
> re-exports with the standard nfs implentation? Is it possible at the
> moment to patch a kernel to make this work? Anyone have experience
> with it? Any input is appreciated.

It's probably possible, but some kernel hacking would be required.

Off the top of my head:

	- filehandles: you probably can't pass your server's filehandles
	  unchanged back to your client.  At a minimum you'd want to add
	  a header allowing you to distinguish filehandles for the
	  different filesystems you export.  What if you get a
	  filehandle from the server that's already at the protocol's
	  maximum size?  Are you going to try to maintain your own
	  persistent mapping of filehandles, and if so, is it possible
	  to do that with reasonable performance?
	- what do you do if your server takes a really long time to
	  answer a request?  Or stops responding completely?

--b.

Re: Proxy

[Trond Myklebust]

On Mon, 2010-05-03 at 14:56 -0400, J. Bruce Fields wrote: 
> On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
> > With NFS4's support for referrals and Kerberos, it seems like the
> > original reasons to prevent re-exporting of an NFS share might no
> > longer exist. With fs-proxy making its way into the mainline kernel
> > and things like cachefilesd, there are also very good reasons to allow
> > it. A proxy server with a persistent cache could give the ability to
> > robustly use shares across a WAN or do failover pairs with no need for
> > more complex replication. Speaking as an end-user, this would be very
> > desirable.
> > 
> > I see that others have implemented proxies with user-space NFS, which
> > seems reasonable but not optimal. What is the obstacle to allowing
> > re-exports with the standard nfs implentation? Is it possible at the
> > moment to patch a kernel to make this work? Anyone have experience
> > with it? Any input is appreciated.
> 
> It's probably possible, but some kernel hacking would be required.
> 
> Off the top of my head:
> 
> 	- filehandles: you probably can't pass your server's filehandles
> 	  unchanged back to your client.  At a minimum you'd want to add
> 	  a header allowing you to distinguish filehandles for the
> 	  different filesystems you export.  What if you get a
> 	  filehandle from the server that's already at the protocol's
> 	  maximum size?  Are you going to try to maintain your own
> 	  persistent mapping of filehandles, and if so, is it possible
> 	  to do that with reasonable performance?
> 	- what do you do if your server takes a really long time to
> 	  answer a request?  Or stops responding completely?

      * If you want to use Kerberos, then how do you proxy an RPCSEC_GSS
        session? 
      * How does the proxy server figure out the real server's export
        rules so that it can re-export them?

Re: Proxy

[maillists0]

On Mon, May 3, 2010 at 3:25 PM, Trond Myklebust
<trond.myklebust@fys.uio.no> wrote:
> On Mon, 2010-05-03 at 14:56 -0400, J. Bruce Fields wrote:
>> On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
>> > With NFS4's support for referrals and Kerberos, it seems like the
>> > original reasons to prevent re-exporting of an NFS share might no
>> > longer exist. With fs-proxy making its way into the mainline kernel
>> > and things like cachefilesd, there are also very good reasons to allow
>> > it. A proxy server with a persistent cache could give the ability to
>> > robustly use shares across a WAN or do failover pairs with no need for
>> > more complex replication. Speaking as an end-user, this would be very
>> > desirable.
>> >
>> > I see that others have implemented proxies with user-space NFS, which
>> > seems reasonable but not optimal. What is the obstacle to allowing
>> > re-exports with the standard nfs implentation? Is it possible at the
>> > moment to patch a kernel to make this work? Anyone have experience
>> > with it? Any input is appreciated.
>>
>> It's probably possible, but some kernel hacking would be required.
>>

Have a look at this old thing from 2006:
http://www.usenix.org/event/fast07/tech/full_papers/gulati/gulati_html/nache.html
. They claim to have implemented a proxy with only the tools I
mentioned above, along with their own modified version of nfs to allow
multi-hops.

I have a workload of lots of reads/almost no writes, and their
approach makes sense. It would be a great feature. Is something
missing from that paper that makes it unrealistic?

Re: Proxy

[Trond Myklebust]

On Mon, 2010-05-03 at 17:14 -0400, maillists0@gmail.com wrote: 
> On Mon, May 3, 2010 at 3:25 PM, Trond Myklebust
> <trond.myklebust@fys.uio.no> wrote:
> > On Mon, 2010-05-03 at 14:56 -0400, J. Bruce Fields wrote:
> >> On Mon, May 03, 2010 at 12:53:15PM -0400, maillists0@gmail.com wrote:
> >> > With NFS4's support for referrals and Kerberos, it seems like the
> >> > original reasons to prevent re-exporting of an NFS share might no
> >> > longer exist. With fs-proxy making its way into the mainline kernel
> >> > and things like cachefilesd, there are also very good reasons to allow
> >> > it. A proxy server with a persistent cache could give the ability to
> >> > robustly use shares across a WAN or do failover pairs with no need for
> >> > more complex replication. Speaking as an end-user, this would be very
> >> > desirable.
> >> >
> >> > I see that others have implemented proxies with user-space NFS, which
> >> > seems reasonable but not optimal. What is the obstacle to allowing
> >> > re-exports with the standard nfs implentation? Is it possible at the
> >> > moment to patch a kernel to make this work? Anyone have experience
> >> > with it? Any input is appreciated.
> >>
> >> It's probably possible, but some kernel hacking would be required.
> >>
> 
> Have a look at this old thing from 2006:
> http://www.usenix.org/event/fast07/tech/full_papers/gulati/gulati_html/nache.html
> . They claim to have implemented a proxy with only the tools I
> mentioned above, along with their own modified version of nfs to allow
> multi-hops.
> 
> I have a workload of lots of reads/almost no writes, and their
> approach makes sense. It would be a great feature. Is something
> missing from that paper that makes it unrealistic?

Possibly not for your workload, but none of the issues Bruce and I
raised appear to be addressed in that paper.

Furthermore, we do know several of the authors, and none of them have
ever approached us with a proposal to merge their implementation. I
therefore assume that it was written more as a proof of concept in
support of the paper, rather than something IBM is actually planning to
market.

Cheers
  Trond

RE: Proxy

[Craig McDonald]

Try this site. It has got some good examples.
http://www.linuxguruz.org/iptables/

Craig

-----Original Message-----
From: Ray Olszewski [mailto:ray@comarre.com]
Sent: 21 September 2002 03:07
To: Le, Paul [Contractor]; linux-newbie@vger.kernel.org
Subject: RE: Proxy


At 08:38 PM 9/20/02 -0500, Le, Paul [Contractor] wrote:
>May I ask what is iptables, and where can I get a how to for it.
[old stuff deleted]

Oopf! A trickier question than it looks like, I'm sad to say.

iptables is the application used to configure some of the advanced routing 
functions (including firewalling, NAT, and port forwarding) in Linux 2.4.x 
kernels. If you have the application installed, "man iptables" will tell 
you the basic commands. But this listing is pretty barebones ... useful if 
you need to check a detail in your iptables ruleset, but not much help 
getting started.

To my surprise, there does not seem to be an "official" Ipchains HowTo at 
linuxdoc, but several routing HowTos have been updated recently enough to 
give at least a bit of an introduction. Find them via this URL

         http://www.tldp.org/HOWTO/HOWTO-INDEX/networking.html#NETROUTING

(www.tldp.org is the site formerly called linuxdoc, and it should always be 
the first place to check for HowTos or other Linux documentation).

There is an actual Iptables HowTo, which you can find many links for by 
searching on "linux iptables" at Google (the second place to check for 
HowTos and other Linux documentation). I don't know why it is not on the 
list at tldp (maybe times have changed and my "first" and "second" ratings 
need to be reversed?).

Another way to get a handle on Ipfilters is to look at an application that 
uses it. One example is Shorewall, a drop-in firewalling package. Find it 
the usual ways (some distros include it in their package-management
systems).

Finally, I came across this link:

         http://www.linuxnewbie.org/nhf/Security/IPtables_Basics.html

Personally, I didn't find the content all that helpful, but with so little 
for the beginner to choose from on this fundamentally difficult topice, I 
thought it worth mentioning anyway.


--
-------------------------------------------"Never tell me the odds!"--------
Ray Olszewski					-- Han Solo
Palo Alto, California, USA			  ray@comarre.com
----------------------------------------------------------------------------
---

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Proxy

[Ray Olszewski]

At 08:38 PM 9/20/02 -0500, Le, Paul [Contractor] wrote:
>May I ask what is iptables, and where can I get a how to for it.
[old stuff deleted]

Oopf! A trickier question than it looks like, I'm sad to say.

iptables is the application used to configure some of the advanced routing 
functions (including firewalling, NAT, and port forwarding) in Linux 2.4.x 
kernels. If you have the application installed, "man iptables" will tell 
you the basic commands. But this listing is pretty barebones ... useful if 
you need to check a detail in your iptables ruleset, but not much help 
getting started.

To my surprise, there does not seem to be an "official" Ipchains HowTo at 
linuxdoc, but several routing HowTos have been updated recently enough to 
give at least a bit of an introduction. Find them via this URL

         http://www.tldp.org/HOWTO/HOWTO-INDEX/networking.html#NETROUTING

(www.tldp.org is the site formerly called linuxdoc, and it should always be 
the first place to check for HowTos or other Linux documentation).

There is an actual Iptables HowTo, which you can find many links for by 
searching on "linux iptables" at Google (the second place to check for 
HowTos and other Linux documentation). I don't know why it is not on the 
list at tldp (maybe times have changed and my "first" and "second" ratings 
need to be reversed?).

Another way to get a handle on Ipfilters is to look at an application that 
uses it. One example is Shorewall, a drop-in firewalling package. Find it 
the usual ways (some distros include it in their package-management systems).

Finally, I came across this link:

         http://www.linuxnewbie.org/nhf/Security/IPtables_Basics.html

Personally, I didn't find the content all that helpful, but with so little 
for the beginner to choose from on this fundamentally difficult topice, I 
thought it worth mentioning anyway.


--
-------------------------------------------"Never tell me the odds!"--------
Ray Olszewski					-- Han Solo
Palo Alto, California, USA			  ray@comarre.com
-------------------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Proxy

[Bob Batson]

At 7:06 PM -0700 9/20/02, Ray Olszewski wrote:

>At 08:38 PM 9/20/02 -0500, Le, Paul [Contractor] wrote:
>>May I ask what is iptables, and where can I get a how to for it.
>[old stuff deleted]
>
>Oopf! A trickier question than it looks like, I'm sad to say.

A lot of info on iptables is at http://www.netfilter.org/

-- 
Bob Batson
rcb@kc.rr.com

Chocolate is the true opiate of the masses!
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Proxy

[Le, Paul [Contractor]]

May I ask what is iptables, and where can I get a how to for it.

-----Original Message-----
From: 'Alan Womack' [mailto:arwbackup@worldnet.att.net]
Sent: Friday, September 20, 2002 8:19 PM
To: Majordomo leben.com
Subject: re: Proxy


As Ray said, it can be a piece.

I have in my Linux server, which started as a Redhat machine and become a
LinuxFromScratch machine because I was frustrated not knowing enough about
the internals of linux.

I have two network cards in my machine:

eth0 is connected to my DSL modem
eth1 is connected to my hub

The linux machine runs iptables in a simple firewall from the howto's to
allow my local machines to use eth1 as a gateway to the internet.

Net cost was a pair of laying around network cards and a few days learning.

Alan

 >>  I am sorry kind of jump in the middle here. I am looking for a software
so
 >>  more than one computers in my LAN can share a same internet connection.
Is
 >>  Squid doing this???

 >>  Thanks,
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

re: Proxy

['Alan Womack']

As Ray said, it can be a piece.

I have in my Linux server, which started as a Redhat machine and become a LinuxFromScratch machine because I was frustrated not knowing enough about the internals of linux.

I have two network cards in my machine:

eth0 is connected to my DSL modem
eth1 is connected to my hub

The linux machine runs iptables in a simple firewall from the howto's to allow my local machines to use eth1 as a gateway to the internet.

Net cost was a pair of laying around network cards and a few days learning.

Alan

 >>  I am sorry kind of jump in the middle here. I am looking for a software so
 >>  more than one computers in my LAN can share a same internet connection. Is
 >>  Squid doing this???

 >>  Thanks,
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Proxy

[Ray Olszewski]

At 03:41 PM 9/20/02 -0500, Le, Paul [Contractor] wrote:
>I am sorry kind of jump in the middle here. I am looking for a software so
>more than one computers in my LAN can share a same internet connection. Is
>Squid doing this???

No. squid can be *part* of the solution to this problem, but it is not the 
core part.

The details of how to do this with Linux depend a bit on what kind of 
Internet connection you are sharing. But the basic idea is that the Linux 
host functions as a router. It has two network connections -- one to the 
LAN, the other to the Internet. It runs a kernel configured to support 
routing, and uses either the ipchains (kernel 2.2.x) or iptables (2.4.x) 
facility to set up a firewall. Among other things, the firewall does NAT 
(Network Address Translation), a facility by which the kernel lets LAN 
hosts that use non-routable IP addresses "share" the one routable address 
the Linux host itself has on its external (Internet) interface.

Actually, Squid can substitute for the NAT part, as long as you only want 
to "share" access to the services that it can proxy (http, https, probably 
ftp). The Linux host still needs to connect to both the LAN and the 
Internet, and it needs a suitable routing table to access both networks.

I've only skimmed the surface here; look through routing and firewalling 
HowTos to get a more complete overview. Please feel free to ask additiona, 
more specific question here, once you've gotten this overview.

I may have made this sound hard, but it really is not. It's a very common 
arrangement, either using a general-purpose Linux host as the router or a 
specialized one running on old equipment (for more on this, look at 
leaf.sourceforge.net, among other places).


--
-------------------------------------------"Never tell me the odds!"--------
Ray Olszewski					-- Han Solo
Palo Alto, California, USA			  ray@comarre.com
-------------------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Proxy

[Le, Paul [Contractor]]

I am sorry kind of jump in the middle here. I am looking for a software so
more than one computers in my LAN can share a same internet connection. Is
Squid doing this???

Thanks,

-----Original Message-----
From: Alan Womack [mailto:arwbackup@worldnet.att.net]
Sent: Friday, September 20, 2002 3:39 PM
To: linux-newbie@vger.kernel.org
Subject: re: Proxy


I am using squid for this now, I also understand Squidguard goes further and
has abilities to download maintained lists of approved and disapproved sites
from a central server on the web.  e.g. other people say what is
approved/disapproved..

> I want to be able to have all web traffic filter through my Linux
server. I need to be able to log all activity. Who made the request and
what pages they went to. I also need to be to use a white list. Only
allowing access to certain sites rather then a blacklist which just
refuses access to certain sites. I have been looking at the squid
package but it seems to be more of a web cache then a proxy. At least
all the documentation I have read on has be cache specific. Any
suggestions on what software I should run and maybe a link to the docs.
Thanks! <
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

re: Proxy

[Alan Womack]

I am using squid for this now, I also understand Squidguard goes further and has abilities to download maintained lists of approved and disapproved sites from a central server on the web.  e.g. other people say what is approved/disapproved..

> I want to be able to have all web traffic filter through my Linux
server. I need to be able to log all activity. Who made the request and
what pages they went to. I also need to be to use a white list. Only
allowing access to certain sites rather then a blacklist which just
refuses access to certain sites. I have been looking at the squid
package but it seems to be more of a web cache then a proxy. At least
all the documentation I have read on has be cache specific. Any
suggestions on what software I should run and maybe a link to the docs.
Thanks! <
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

RE: Proxy

[Craig McDonald]

You can create acl's with Squid to deny/permit certain sites, 
so you could create an acl with a list of site that you users are permitted
to view.
You can run a perl program called Calamaris to create reports

Squid documentation
http://squid-docs.sourceforge.net/latest/html/book1.htm

Calamaris
http://cord.de/tools/squid/calamaris/

Hope thats what you are looking for.

Craig


-----Original Message-----
From: Paul Kraus [mailto:pkraus@pelsupply.com]
Sent: 20 September 2002 18:33
To: linux-newbie@vger.kernel.org
Subject: Proxy


I want to be able to have all web traffic filter through my Linux
server. I need to be able to log all activity. Who made the request and
what pages they went to. I also need to be to use a white list. Only
allowing access to certain sites rather then a blacklist which just
refuses access to certain sites. I have been looking at the squid
package but it seems to be more of a web cache then a proxy. At least
all the documentation I have read on has be cache specific. Any
suggestions on what software I should run and maybe a link to the docs.
Thanks!

Paul Kraus
Network Administrator
PEL Supply Company
216.267.5775 Voice
216-267-6176 Fax
www.pelsupply.com

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs
-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Proxy

[Paul Kraus]

I want to be able to have all web traffic filter through my Linux
server. I need to be able to log all activity. Who made the request and
what pages they went to. I also need to be to use a white list. Only
allowing access to certain sites rather then a blacklist which just
refuses access to certain sites. I have been looking at the squid
package but it seems to be more of a web cache then a proxy. At least
all the documentation I have read on has be cache specific. Any
suggestions on what software I should run and maybe a link to the docs.
Thanks!

Paul Kraus
Network Administrator
PEL Supply Company
216.267.5775 Voice
216-267-6176 Fax
www.pelsupply.com

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs

Re: Proxy

[Ray Olszewski]

At 01:32 PM 9/20/02 -0400, Paul Kraus wrote:
>I want to be able to have all web traffic filter through my Linux
>server. I need to be able to log all activity. Who made the request and
>what pages they went to. I also need to be to use a white list. Only
>allowing access to certain sites rather then a blacklist which just
>refuses access to certain sites. I have been looking at the squid
>package but it seems to be more of a web cache then a proxy. At least
>all the documentation I have read on has be cache specific. Any
>suggestions on what software I should run and maybe a link to the docs.
>Thanks!

squid is a caching proxy server. Whether it will do what you want 
specifically, that I do not know, especially the whitelist part. But it is 
a proxy server.

The other well-known proxy server for Linux is junkbuster (I think 
www.junkbusters.com, but try Google to be sure). A recent(?) successor to 
it is something called "privoxy", but I'm not familiar with the details 
(may just be a Debian repackaging to deal with trademark issues). Its focus 
is a bit different from yours, but it might do what you want.

A few years ago, there was a project to develop a proxy server for schools, 
designed to prevent kids from accessing "inappropriate" sites. I think it 
died out for lack of developer interest, but I'm not certain of that ... 
you might search around on K-12 lists for links to any version of it that 
survives.


--
-------------------------------------------"Never tell me the odds!"--------
Ray Olszewski					-- Han Solo
Palo Alto, California, USA			  ray@comarre.com
-------------------------------------------------------------------------------

-
To unsubscribe from this list: send the line "unsubscribe linux-newbie" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.linux-learn.org/faqs