Re: [SAGE] distributed ssh

[Chuck Yerkes <chuck+sage@snew.com>]

Quoting Philip Brown (phil@bolthole.com):
> On Fri, Apr 11, 2003 at 05:37:41PM -0500, Quentin Fennessy wrote:
> >  > #3.
> >  >     for host in `cat hostlist` ; do
> >  >       ssh $host ksh -s <shscript >$host.output 2>&1 &
> >  >     done
> > That is an unthrottled invocation -- in my environment that might spawn 2000
> > ssh sessions--which could overload the invoking machine or fileservers or
> > nameservers or... . 
> 
> if that overloaded the machine, it would be a sad, sad machine badly in
> need of upgrading.

Actually, the startup costs of 2000 concurrent SSH sessions are
non trivial.  Perhaps it might be 2k IPSec connections.  Perhaps
it might be going through some firewall that's busy.  Whatever.
Uncontrolled background jobs can be like campfires in the Bay Area.
Pleasant at first but with serious repurcussions when things go
awry.

Or perhaps it involves 20,000 machines (real L.i.s.a sites).

Either way, a manager on this which says "allow N concurrent
connections, kill them after M minutes, and do it to X machines,
here's a file with a list of machines and the script is HERE"
is good.  Some of these tools look interesting.

I've done this on production environments; sometimes expect
is needed; sometime you need to kill the job after a while
(ultrix's rsh and rshd would never ever ever time out;
nothing like finding your push to 4000 machines hung because
at 10 concurrent, you ended up hung on 10 ultrix boxes).
Perl and spawning fixed that, at the time.