Login load balancing

Login load balancing

[Drew Leske]

Hi all,

I'm looking for a solution (and I'm not afraid of devving one if necessary)
to load-balance SSH logins over several mostly identical systems.  So far
the closest I have come is a solution using iptables, but I'm not sure it
will work, and I may well be overlooking some other solution.  Any ideas
would be appreciated.  My research has so far turned up little.

We have several systems that are, from a user's perspective, identical.
Their home directories are network mounted, libraries are synchronised, and
so on, so they don't really care which system they log in to.  Their work on
these systems can be quite intensive and may consume quite a few resources,
but must remain interactive (so a batch system running on a cluster won't do
it).

For the users it's a guessing game as to which of the machines they should
log in to at any point.  They may log in to the first and find it's heavily
loaded, and so log in to another, until they find the best.  A second
difficulty with this is the users have be aware of which machines are
available--and they are named, due to historical reasons, using a
non-contiguous numbering scheme.

So instead of the users logging in to bob3, bob6 or bob8, I'd like for them
to be able to simply log in to "bob" and be directed to the least-loaded
machine.

Round-robining on the switch won't do it, because if one of the systems is
absolutely pinned, every Nth login will still wind up there.

Determining which machines are least loaded will not be a problem.  The
metrics may be gathered using SNMP or some other means from the
participating hosts.  The problem is entirely in the redirection from 'bob'
to 'bob3', 'bob6', 'bob8'.

Logins are exclusively through SSH.  There is no need, and I don't
anticipate one (which means there will be some fantastic new request coming
in tomorrow) to support other protocols in this manner.

The only half-solution I have come up with so far is to define a 'director'
box with the 'bob' alias, and then periodically grab load metrics from the
participating hosts, determine of the 'bob's which is the least loaded, and
then *cough* update a DNAT rule to redirect requests coming in for 'bob' to
the least-loaded 'bobX'.

The last part feels horky, and I'm not even sure it will work, since later
packets coming in may be DNAT'ed to a different machine.  Also, the director
then routes all the packets for logins to all the boxes.  I can't see any
way to redirect the initial connection that won't cause all sorts of
problems with the client's firewalls.

Any ideas?

Thanks,
Drew.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[Mailings'AT'netzwerk.cc]

Drew Leske wrote:
> Hi all,
> 
> I'm looking for a solution (and I'm not afraid of devving one if necessary)
> to load-balance SSH logins over several mostly identical systems.  So far
> the closest I have come is a solution using iptables, but I'm not sure it
> will work, and I may well be overlooking some other solution.  Any ideas
> would be appreciated.  My research has so far turned up little.
> 
> We have several systems that are, from a user's perspective, identical.
> Their home directories are network mounted, libraries are synchronised, and
> so on, so they don't really care which system they log in to.  Their work on
> these systems can be quite intensive and may consume quite a few resources,
> but must remain interactive (so a batch system running on a cluster won't do
> it).
> 
> For the users it's a guessing game as to which of the machines they should
> log in to at any point.  They may log in to the first and find it's heavily
> loaded, and so log in to another, until they find the best.  A second
> difficulty with this is the users have be aware of which machines are
> available--and they are named, due to historical reasons, using a
> non-contiguous numbering scheme.
> 
> So instead of the users logging in to bob3, bob6 or bob8, I'd like for them
> to be able to simply log in to "bob" and be directed to the least-loaded
> machine.
> 
> Round-robining on the switch won't do it, because if one of the systems is
> absolutely pinned, every Nth login will still wind up there.
> 
> Determining which machines are least loaded will not be a problem.  The
> metrics may be gathered using SNMP or some other means from the
> participating hosts.  The problem is entirely in the redirection from 'bob'
> to 'bob3', 'bob6', 'bob8'.
> 
> Logins are exclusively through SSH.  There is no need, and I don't
> anticipate one (which means there will be some fantastic new request coming
> in tomorrow) to support other protocols in this manner.
> 
> The only half-solution I have come up with so far is to define a 'director'
> box with the 'bob' alias, and then periodically grab load metrics from the
> participating hosts, determine of the 'bob's which is the least loaded, and
> then *cough* update a DNAT rule to redirect requests coming in for 'bob' to
> the least-loaded 'bobX'.
> 
> The last part feels horky, and I'm not even sure it will work, since later
> packets coming in may be DNAT'ed to a different machine.  Also, the director
> then routes all the packets for logins to all the boxes.  I can't see any
> way to redirect the initial connection that won't cause all sorts of
> problems with the client's firewalls.
> 
> Any ideas?
> 
> Thanks,
> Drew.
> 
Hi Drew,

maybe you should take a look on "iptables random" - target.

Something like

iptables -t nat -A PREROUTING -p tcp --dport 22 -i $whatever \
	 -m random --average $[100/$howmuchserveryouvegot] \
	 -j DNAT --to $server1

iptables -t nat -A PREROUTING -p tcp --dport 22 -i $whatever \
	 -m random --average $[100/$howmuchserveryouvegot] \
	 -j DNAT --to $server2

...

Only one idea, but remember "the last rule should realy match" ;-)

Hope this is the right syntax.

Best

Sven

Re: Login load balancing

[Daniel Ivanov]

The last one is not the best solution, because of the fact that you rely 
on randomness. I would suggest you take a more comprehensive approach. 
As the machines are snmp enabled, you just have to write a custom 
daemon, receiving on port 22 (ssh) as a front-end and check which 
machine is most idle and dnat the user there, for the DNAT to be able to 
work, you would have to send an RST packet back to the ssh client and 
wait for it to reconnect to the already DNAT-ted machine. That would be 
a working solution. As long as you don't wanna have millions of rules on 
the redirecting machine, you just have to "count" the active logins(use 
pam_script for example) and remove the rules as long as the last shell 
quits. You would like to have all simultaneous logins on the same 
machine, so you'll have to check on a new login if the user is still 
there and put it on the same machine. Just think about the RST packet, 
cause i think it's not the most elegant solution as long as the user 
will get a "Connection closed by remote site" msg.

Mailings'AT'netzwerk.cc wrote:

> Drew Leske wrote:
>
>> Hi all,
>>
>> I'm looking for a solution (and I'm not afraid of devving one if 
>> necessary)
>> to load-balance SSH logins over several mostly identical systems.  So 
>> far
>> the closest I have come is a solution using iptables, but I'm not 
>> sure it
>> will work, and I may well be overlooking some other solution.  Any ideas
>> would be appreciated.  My research has so far turned up little.
>>
>> We have several systems that are, from a user's perspective, identical.
>> Their home directories are network mounted, libraries are 
>> synchronised, and
>> so on, so they don't really care which system they log in to.  Their 
>> work on
>> these systems can be quite intensive and may consume quite a few 
>> resources,
>> but must remain interactive (so a batch system running on a cluster 
>> won't do
>> it).
>>
>> For the users it's a guessing game as to which of the machines they 
>> should
>> log in to at any point.  They may log in to the first and find it's 
>> heavily
>> loaded, and so log in to another, until they find the best.  A second
>> difficulty with this is the users have be aware of which machines are
>> available--and they are named, due to historical reasons, using a
>> non-contiguous numbering scheme.
>>
>> So instead of the users logging in to bob3, bob6 or bob8, I'd like 
>> for them
>> to be able to simply log in to "bob" and be directed to the least-loaded
>> machine.
>>
>> Round-robining on the switch won't do it, because if one of the 
>> systems is
>> absolutely pinned, every Nth login will still wind up there.
>>
>> Determining which machines are least loaded will not be a problem.  The
>> metrics may be gathered using SNMP or some other means from the
>> participating hosts.  The problem is entirely in the redirection from 
>> 'bob'
>> to 'bob3', 'bob6', 'bob8'.
>>
>> Logins are exclusively through SSH.  There is no need, and I don't
>> anticipate one (which means there will be some fantastic new request 
>> coming
>> in tomorrow) to support other protocols in this manner.
>>
>> The only half-solution I have come up with so far is to define a 
>> 'director'
>> box with the 'bob' alias, and then periodically grab load metrics 
>> from the
>> participating hosts, determine of the 'bob's which is the least 
>> loaded, and
>> then *cough* update a DNAT rule to redirect requests coming in for 
>> 'bob' to
>> the least-loaded 'bobX'.
>>
>> The last part feels horky, and I'm not even sure it will work, since 
>> later
>> packets coming in may be DNAT'ed to a different machine.  Also, the 
>> director
>> then routes all the packets for logins to all the boxes.  I can't see 
>> any
>> way to redirect the initial connection that won't cause all sorts of
>> problems with the client's firewalls.
>>
>> Any ideas?
>>
>> Thanks,
>> Drew.
>>
> Hi Drew,
>
> maybe you should take a look on "iptables random" - target.
>
> Something like
>
> iptables -t nat -A PREROUTING -p tcp --dport 22 -i $whatever \
>      -m random --average $[100/$howmuchserveryouvegot] \
>      -j DNAT --to $server1
>
> iptables -t nat -A PREROUTING -p tcp --dport 22 -i $whatever \
>      -m random --average $[100/$howmuchserveryouvegot] \
>      -j DNAT --to $server2
>
> ...
>
> Only one idea, but remember "the last rule should realy match" ;-)
>
> Hope this is the right syntax.
>
> Best
>
> Sven
>

Re: Login load balancing

[Drew Leske]

Hi Daniel,

> The last one is not the best solution, because of the fact that you rely
> on randomness. I would suggest you take a more comprehensive approach.

Agreed.  If I had enough users and enough nodes, randomness would approach
other methods for effectiveness, but that's not the case here.

> As the machines are snmp enabled, you just have to write a custom
> daemon, receiving on port 22 (ssh) as a front-end and check which
> machine is most idle and dnat the user there, for the DNAT to be able to

Slight aside: I don't want to check the load at time of login, because that
would significantly slow down the login process.  The load checking would be
done periodically--say every 5 or 15 minutes or so--and the results would
force a change to the DNAT rule.

> work, you would have to send an RST packet back to the ssh client and
> wait for it to reconnect to the already DNAT-ted machine. That would be
> a working solution. As long as you don't wanna have millions of rules on
> the redirecting machine, you just have to "count" the active logins(use
> pam_script for example) and remove the rules as long as the last shell
> quits. You would like to have all simultaneous logins on the same
> machine, so you'll have to check on a new login if the user is still
> there and put it on the same machine. Just think about the RST packet,
> cause i think it's not the most elegant solution as long as the user
> will get a "Connection closed by remote site" msg.

I've considered that (keeping all logins together on the same head node).
My feelings on that are:

(0) The users should not actually need to have multiple logins on the same
real host.  If their environment is not consistent across the hosts, there
is another problem.

(1) Once the user logs in and is redirected, they have the option to
'manually' log in to that node for subsequent sessions.

(2) Tracking user logins so I can make this automatic for them is desirable,
however, it would be non-trivial to implement robustly.

(3) So, that would be 'version 2'! :)

>> Hi Drew,
>>
>> maybe you should take a look on "iptables random" - target.
>>
>> Something like
>>
>> iptables -t nat -A PREROUTING -p tcp --dport 22 -i $whatever \
>>      -m random --average $[100/$howmuchserveryouvegot] \
>>      -j DNAT --to $server1
>>
>> iptables -t nat -A PREROUTING -p tcp --dport 22 -i $whatever \
>>      -m random --average $[100/$howmuchserveryouvegot] \
>>      -j DNAT --to $server2
>>
>> ...
>>
>> Only one idea, but remember "the last rule should realy match" ;-)
>>
>> Hope this is the right syntax.
>>
>> Best
>>
>> Sven

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

RE: Login load balancing

[Pablo Sanchez]

 

> -----Original Message-----
> From: netfilter-bounces@lists.netfilter.org 
> [mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Drew Leske
> Sent: Wednesday, April 26, 2006 1:34 PM
> To: netfilter@lists.netfilter.org
> Subject: Login load balancing
> 
> The only half-solution I have come up with so far is to define a
'director'
> box with the 'bob' alias, and then periodically grab load metrics from the
> participating hosts, determine of the 'bob's which is the least loaded,
and
> then *cough* update a DNAT rule to redirect requests coming in for 'bob'
to
> the least-loaded 'bobX'.

Hi Drew,

I believe the above is what you'll want to implement.  As your research has
probably already shown, the load balancers in the market are for HTTP.  A
good load balancer will need to communicate with the backend clients so it
has data on load and other metrics necessary for it to make a decision on
which server to serve.

You could use wget to fetch metrics from all the servers (include a
timestamp so you know when your data is stale) and have the director
consider this information when it punches down new IPTABLEs rules.

Cheers,
-pablo

Re: Login load balancing

[Drew Leske]

Hi Pablo,

Pablo Sanchez wrote:
>> The only half-solution I have come up with so far is to define a 'director'
>> box with the 'bob' alias, and then periodically grab load metrics from the
>> participating hosts, determine of the 'bob's which is the least loaded, and
>> then *cough* update a DNAT rule to redirect requests coming in for 'bob' to
>> the least-loaded 'bobX'.
> 
> I believe the above is what you'll want to implement.  As your research has
> probably already shown, the load balancers in the market are for HTTP.  A
> good load balancer will need to communicate with the backend clients so it
> has data on load and other metrics necessary for it to make a decision on
> which server to serve.

You're right about load-balancing HTTP.  Everybody and their dog wants to
load-balance HTTP for some reason. ;)  But my dog insists on load-balancing SSH.

I have also found something called LVS, but as I've mentioned in another
post this is unsuitable for us.  Grabbing the load data as I've said is no
problem--the default SNMP daemon provides CPU load I believe by default, and
it's no problem at all to provide for additional information.  This part is
trivial since I've already implemented SNMP elsewhere.

> You could use wget to fetch metrics from all the servers (include a
> timestamp so you know when your data is stale) and have the director
> consider this information when it punches down new IPTABLEs rules.

SNMP would be faster and more lightweight I believe; wget implies I'd have
either an HTTP or FTP service running on each of those machines.  Plus,
these connections would be subject to TCP timeouts, so if one of the
machines is down, my metric-gathering script would take forever timing out
on it.  SNMP fails a lot faster.  Also, there'd be quite a bit less parsing
to do of the results.

Thanks for your response!
Drew.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[Drew Leske]

>> I'm looking for a solution (and I'm not afraid of devving one if necessary)
>> to load-balance SSH logins over several mostly identical systems.
> 
> This sounds like a job for LVS. Have a look at
> http://www.linuxvirtualserver.org/

Thanks Sebastian.  I should have mentioned however that I have looked at
this and I'd like to avoid it.  I'm not afraid of compiling my own kernel or
software, but here at work we avoid using anything but our distribution's
standard kernel package.  If it comes down to the choice between using LVS
and not providing the load-balancing service at all, I will probably have to
choose the latter.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[Arnt Karlsen]

On Wed, 26 Apr 2006 11:27:51 -0700, Drew wrote in message 
<444FBBA7.2020501@uvic.ca>:

> >> I'm looking for a solution (and I'm not afraid of devving one if
> >necessary) > to load-balance SSH logins over several mostly identical
> >systems.
> > 
> > This sounds like a job for LVS. Have a look at
> > http://www.linuxvirtualserver.org/
> 
> Thanks Sebastian.  I should have mentioned however that I have looked
> at this and I'd like to avoid it.  I'm not afraid of compiling my own
> kernel or software, but here at work we avoid using anything but our
> distribution's standard kernel package.  If it comes down to the
> choice between using LVS and not providing the load-balancing service
> at all, I will probably have to choose the latter.
 
..check out sdm and sdm-terminal, if you wanna provide X logins over
ssh, the user sees a menu to choose from and you should be able to 
"pile up the good boxes" on top of that menu listing:
arnt@a45:~ $ apt-cache search sdm
bsdmainutils - collection of more utilities from FreeBSD
bsdutils - Basic utilities from 4.4BSD-Lite
cfv - versatile file checksum creator and verifier
sdm - Secure Display Manager - secure remote access to X11
sdm-terminal - Secure Display Manager - terminal files
turqstat - Fidonet and Usenet statistics program
xturqstat - Fidonet and Usenet statistics program for X
arnt@a45:~ $ apt-cache show sdm sdm-terminal
Package: sdm
Priority: optional
Section: x11
Installed-Size: 124
Maintainer: Jonas Smedegaard <dr@jones.dk>
Architecture: all
Version: 0.4.0b-3
Depends: openssh-server | ssh | ssh-server, dash, xbase-clients,
x11-common | xfree86-common Recommends: xdialog
Suggests: wmanager, selectwm, icewm | x-window-manager, xterm |
x-terminal-emulator Filename: pool/main/s/sdm/sdm_0.4.0b-3_all.deb
Size: 14108
MD5sum: 0fdab9298ea0f4e42426d67ccb31b9c4
Description: Secure Display Manager - secure remote access to X11
 sdm is an X11 display manager similar to xdm, gdm and kdm, but unlike
 those it wraps the X11 traffic within an ssh tunnel to provide a secure
 login mechanism for remote X sessions.
 sdm provides access only through SSH, not locally. It is technically
 possible to access an sdm server from same host, but probably a waste
of CPU power.
 .
 This package should be installed on any server acting as SDM server.
 .
  Homepage: http://www.lessdisks.net/
Tag: interface::daemon, interface::x11, role::sw:server, use::login,
x11::display-manager

Package: sdm-terminal
Priority: optional
Section: x11
Installed-Size: 108
Maintainer: Jonas Smedegaard <dr@jones.dk>
Architecture: all
Source: sdm
Version: 0.4.0b-3
Depends: openssh-client | ssh | ssh-client, dash, xserver-xorg |
xserver-xfree86 | xserver, xbase-clients Recommends: xdialog
Filename: pool/main/s/sdm/sdm-terminal_0.4.0b-3_all.deb
Size: 13692
MD5sum: 93ad42913ddf30c04da3a2a4c239c49d
Description: Secure Display Manager - terminal files
 sdm is an X11 display manager similar to xdm, gdm and kdm, but unlike
 those it wraps the X11 traffic within an ssh tunnel to provide a secure
 login mechanism for remote X sessions.
 sdm provides access only through SSH, not locally. It is technically
 possible to access an sdm server from same host, but probably a waste
of CPU power.
 .
 This package contains helper files for a terminal to connect to an sdm
 server, and should be installed on any computer accessing an sdm
server. .
  Homepage: http://www.lessdisks.net/
Tag: admin::login, role::content:data, security::authentication,
use::login, x11::display-manager

arnt@a45:~ $                                                            
                   


-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.

Re: Login load balancing

[Drew Leske]

> ..check out sdm and sdm-terminal, if you wanna provide X logins over
> ssh, the user sees a menu to choose from and you should be able to 
> "pile up the good boxes" on top of that menu listing:

Interesting, and I might find this useful elsewhere, but for this issue I
need to support console logins as well.

Thanks though!
Drew.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[Arnt Karlsen]

On Thu, 27 Apr 2006 10:34:52 -0700, Drew wrote in message 
<445100BC.1080907@uvic.ca>:

> > ..check out sdm and sdm-terminal, if you wanna provide X logins over
> > ssh, the user sees a menu to choose from and you should be able to 
> > "pile up the good boxes" on top of that menu listing:
> 
> Interesting, and I might find this useful elsewhere, but for this
> issue I need to support console logins as well.

..and this can't?  At the very least you should be able to offer console
logins from the sdm-terminal X menu, and then there's offering a console
menu from /etc/inittab instead of /bin/bash or whatever you guys use.

-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.

Re: Login load balancing

[Drew Leske]

Arnt Karlsen wrote:
> On Thu, 27 Apr 2006 10:34:52 -0700, Drew wrote in message 
> <445100BC.1080907@uvic.ca>:
> 
>>> ..check out sdm and sdm-terminal, if you wanna provide X logins over
>>> ssh, the user sees a menu to choose from and you should be able to 
>>> "pile up the good boxes" on top of that menu listing:
>> Interesting, and I might find this useful elsewhere, but for this
>> issue I need to support console logins as well.
> 
> ..and this can't?  At the very least you should be able to offer console
> logins from the sdm-terminal X menu, and then there's offering a console
> menu from /etc/inittab instead of /bin/bash or whatever you guys use.

I'm not sure I understand, but you seem to be suggesting a way by which I
could use a console window in X.  As a base case, I have to support somebody
connecting with a vt100 and a 9600 baud modem.  This solution needs to be
completely independent of X.

Thanks,
Drew.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[Arnt Karlsen]

On Fri, 28 Apr 2006 09:37:49 -0700, Drew wrote in message 
<445244DD.8040100@uvic.ca>:

> Arnt Karlsen wrote:
> > On Thu, 27 Apr 2006 10:34:52 -0700, Drew wrote in message 
> > <445100BC.1080907@uvic.ca>:
> > 
> >>> ..check out sdm and sdm-terminal, if you wanna provide X logins
> >over >> ssh, the user sees a menu to choose from and you should be
> >able to  >> "pile up the good boxes" on top of that menu listing:
> >> Interesting, and I might find this useful elsewhere, but for this
> >> issue I need to support console logins as well.
> > 
> > ..and this can't?  At the very least you should be able to offer
> > console logins from the sdm-terminal X menu, and then there's
> > offering a console menu from /etc/inittab instead of /bin/bash or
> > whatever you guys use.
> 
> I'm not sure I understand, but you seem to be suggesting a way by
> which I could use a console window in X.  As a base case, I have to
> support somebody connecting with a vt100 and a 9600 baud modem.  This
> solution needs to be completely independent of X.

..yup, and I was thinking of the various ttys, on which again you offer
a shell menu to choose from, instead of the usual shell prompt, any tty
(except mingetty or fgetty) should do this for you, a few quick ideas:
arnt@a45:~ $ apt-cache search tty |grep tty
a2ps - GNU a2ps - 'Anything to PostScript' converter and pretty-printer
bibclean - pretty-printer for BibTeX databases
boxshade - [Biology] Pretty-printing of multiple sequence alignments
brltty - Access software for a blind person using a soft braille
terminal brltty-flite - Access software for a blind person using a soft
braille terminal brltty-x11 - Access software for a blind person using a
soft braille terminal detachtty - Attach/detach from interactive
processes across the network discus - Pretty version of df(1) command.
dvi2tty - Previewing dvi-files on text-only devices
enscript - Converts ASCII text to Postscript, HTML, RTF or Pretty-Print
eskuel - A pretty PHP administration tool for MySQL databases
fbgetty - A console getty with and without frame buffer capability
fgetty - very small, efficient, console-only getty and login
fillets-ng - puzzle game about witty fish saving the world sokoban-style
fvwm-crystal - Pretty Desktop Environment based on fvwm
hztty - Translates GB, Big5, zW/HZ Chinese encodings in a tty session
kitty - a Qt/KDE based RSS podcast and video aggregator
libemail-mime-contenttype-perl - Parse a MIME Content-Type Header
libio-pty-perl - Perl module for pseudo tty IO
libio-stty-perl - Interface to secure pseudo ttys
libmlrisctools-smlnj - Library for parsing and pretty printing SML code
libmodem-vgetty-perl - Perl module for interfacing with vgetty
(Modem::Vgetty) libpty-ruby - pseudo tty interface for Ruby
libpty-ruby1.6 - pseudo tty interface for Ruby 1.6.x
libterm-query-perl - Subroutines that handle simple tty-based UI
libxml-filter-reindent-perl - Perl module for reformatting whitespace
for pretty printing XML linuxvnc - VNC server to monitor a tty
lyskom-tty-client - TTY client for LysKOM
mgetty - Smart Modem getty replacement
mgetty-docs - Documentation Package for mgetty
mgetty-fax - Faxing tools for mgetty
mgetty-pvftools - Programs for listening and manipulating pvf and rmd
files mgetty-viewfax - Program for displaying Group-3 Fax files under X
mgetty-voice - Voicemail handler for mgetty
mingetty - Console-only getty
mp - pretty-printer for email messages and other text files
muttprint - Pretty printing of mails
owl - A curses-based tty Zephyr client.
pretzel - Prettyprinter generator for noweb
putty - Telnet/SSH client for X
putty-tools - command-line tools for SSH, SCP, and SFTP
rungetty - minimal console getty that can run any process
trueprint - pretty printing of source code
ttv - tty TV application
ttyd - Remote Modem Utility for Unix
ttylog - serial port logger
ttyrec - A tty recorder
ttysnoop - TTY Snoop - allows you to spy on telnet+serial connections
zope-atcontenttypes - archetypes-based replacement for Plone/CMF types
arnt@a45:~ $                                                            
                          

-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.

Re: Login load balancing

[Drew Leske]

Arnt Karlsen wrote:
>> I'm not sure I understand, but you seem to be suggesting a way by
>> which I could use a console window in X.  As a base case, I have to
>> support somebody connecting with a vt100 and a 9600 baud modem.  This
>> solution needs to be completely independent of X.
> 
> ..yup, and I was thinking of the various ttys, on which again you offer
> a shell menu to choose from, instead of the usual shell prompt, any tty
> (except mingetty or fgetty) should do this for you, a few quick ideas:
> arnt@a45:~ $ apt-cache search tty |grep tty
> [...]

Okay, so what you're talking about now has nothing to do with sdm-terminal,
and is just a script run when users log in to the director, which will give
the user a menu and then shunt them off through ssh or some other means to
one of the participating hosts.

I wouldn't bother with the menu, though, because that defeats the
'load-balacing' part of it (unless I put the latest load figures in the menu
and let the user choose).

This solution requires login access to the director host.  Not necessarily a
show-stopper, but it's a drawback.

Thanks for your input,
Drew.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[Arnt Karlsen]

On Fri, 28 Apr 2006 11:36:47 -0700, Drew wrote in message 
<445260BF.5010406@uvic.ca>:

> Arnt Karlsen wrote:
> >> I'm not sure I understand, but you seem to be suggesting a way by
> >> which I could use a console window in X.  As a base case, I have to
> >> support somebody connecting with a vt100 and a 9600 baud modem. 
> >This > solution needs to be completely independent of X.
> > 
> > ..yup, and I was thinking of the various ttys, on which again you
> > offer a shell menu to choose from, instead of the usual shell
> > prompt, any tty (except mingetty or fgetty) should do this for you,
> > a few quick ideas: arnt@a45:~ $ apt-cache search tty |grep tty
> > [...]
> 
> Okay, so what you're talking about now has nothing to do with
> sdm-terminal,

..yes and no, it can remain as an alternative on the console login menu
and vice versa.

> and is just a script run when users log in 

..yup.

> to the director

..no need, just have each box fetch the menu text du jour from it.

>, which will
> give the user a menu and then shunt them off through ssh or some other
> means to one of the participating hosts.
> 
> I wouldn't bother with the menu, though, because that defeats the
> 'load-balacing' part of it (unless I put the latest load figures in
> the menu and let the user choose).
> 
> This solution requires login access to the director host.  Not
> necessarily a show-stopper, but it's a drawback.


-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.

Re: Login load balancing

[Carl-Daniel Hailfinger]

Hi Drew,

what about using a DNS CNAME for bob to bob[368]? If you set the
TTL low enough and update your DNS server with the latest data
from your SNMP agents continuously, you will achieve exactly what
you want without any iptables trickery.
Such a solution is running here and it works fine.

Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

Re: Login load balancing

[Drew Leske]

Hi Carl,

Carl-Daniel Hailfinger wrote:
> what about using a DNS CNAME for bob to bob[368]? If you set the
> TTL low enough and update your DNS server with the latest data
> from your SNMP agents continuously, you will achieve exactly what
> you want without any iptables trickery.
> Such a solution is running here and it works fine.

That's an interesting solution.  I like it.

Unfortunately we don't control the DNS--another group here has
responsibility for that.  I'll chat with them and see if they have provision
for remote updates.

Thanks,
Drew.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[michael]

On Wed, 26 Apr 2006, Drew Leske wrote:

> Unfortunately we don't control the DNS--another group here has
> responsibility for that.  I'll chat with them and see if they have provision
> for remote updates.

There is another possibility you could consider. Instead of having them 
enable remote updates, get them to delegate a new zone for you.

All you would need to do then is setup some bind servers. You would then 
have direct access to update the A records.

--
Michael
michael@grife.net

Re: Login load balancing

[Drew Leske]

michael@grife.net wrote:
> On Wed, 26 Apr 2006, Drew Leske wrote:
> 
>> Unfortunately we don't control the DNS--another group here has
>> responsibility for that.  I'll chat with them and see if they have
>> provision
>> for remote updates.
> 
> There is another possibility you could consider. Instead of having them
> enable remote updates, get them to delegate a new zone for you.
> 
> All you would need to do then is setup some bind servers. You would then
> have direct access to update the A records.

I considered that, but then I'm implementing a service provided by experts
in another group.  (I provide a BIND server internally for our cluster, but
not one with public access.)  This is, shall we say, "discouraged" (for good
reason--it's one more thing to be an expert in and we've already got experts
elsewhere).

Thanks!
Drew.

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)

Re: Login load balancing

[Drew Leske]

A big thanks to everybody who took the time to consider my problem and
especially to those who responded.  I got a lot of great suggestions.  I'll
see what I come up with and if I find one that works using iptables I'll let
you all know--it just might be useful to somebody else as well.

Cheers,
Drew.

Drew Leske wrote:
> Hi all,
> 
> I'm looking for a solution (and I'm not afraid of devving one if necessary)
> to load-balance SSH logins over several mostly identical systems.  So far
> the closest I have come is a solution using iptables, but I'm not sure it
> will work, and I may well be overlooking some other solution.  Any ideas
> would be appreciated.  My research has so far turned up little.
> 
> We have several systems that are, from a user's perspective, identical.
> Their home directories are network mounted, libraries are synchronised, and
> so on, so they don't really care which system they log in to.  Their work on
> these systems can be quite intensive and may consume quite a few resources,
> but must remain interactive (so a batch system running on a cluster won't do
> it).
> 
> For the users it's a guessing game as to which of the machines they should
> log in to at any point.  They may log in to the first and find it's heavily
> loaded, and so log in to another, until they find the best.  A second
> difficulty with this is the users have be aware of which machines are
> available--and they are named, due to historical reasons, using a
> non-contiguous numbering scheme.
> 
> So instead of the users logging in to bob3, bob6 or bob8, I'd like for them
> to be able to simply log in to "bob" and be directed to the least-loaded
> machine.
> 
> Round-robining on the switch won't do it, because if one of the systems is
> absolutely pinned, every Nth login will still wind up there.
> 
> Determining which machines are least loaded will not be a problem.  The
> metrics may be gathered using SNMP or some other means from the
> participating hosts.  The problem is entirely in the redirection from 'bob'
> to 'bob3', 'bob6', 'bob8'.
> 
> Logins are exclusively through SSH.  There is no need, and I don't
> anticipate one (which means there will be some fantastic new request coming
> in tomorrow) to support other protocols in this manner.
> 
> The only half-solution I have come up with so far is to define a 'director'
> box with the 'bob' alias, and then periodically grab load metrics from the
> participating hosts, determine of the 'bob's which is the least loaded, and
> then *cough* update a DNAT rule to redirect requests coming in for 'bob' to
> the least-loaded 'bobX'.
> 
> The last part feels horky, and I'm not even sure it will work, since later
> packets coming in may be DNAT'ed to a different machine.  Also, the director
> then routes all the packets for logins to all the boxes.  I can't see any
> way to redirect the initial connection that won't cause all sorts of
> problems with the client's firewalls.
> 
> Any ideas?
> 
> Thanks,
> Drew.
> 

-- 
Drew Leske :: Systems Group/Unix, Computing Services, University of Victoria
  dleske@uvic.ca / +1250 472 5055 (office) / +1250 588 4311 (cel)