Re: [SAGE] Email passwords are.. special?

[Brad Knowles <brad@shub-internet.org>]

At 11:27 PM -0800 2/15/07, Michael T. Halligan wrote:

>  How did you handle this at AOL?

In two different ways.  Most of the Operations guys actually had Unix 
boxes of one sort or another on their desks, and @aol.net addresses 
to go along with them.  You accessed aol.net e-mail via POP3, whether 
remote or local -- no, not POP3S, or anything else that was 
encrypted.  On the aol.net mail system, SMTP and POP3 were the only 
services running.  I didn't actually run those boxes myself, but I 
did work very closely with the guys who did run them, and I made sure 
to keep them fully up-to-date with all the relevant stuff we were 
doing in production at the time.

All system access to the actual boxes themselves was via telnet 
internally, and the admin accounts were usually created on the boxes 
as part of our custom install process.  Any admin accounts that had 
to be created afterwards would be the responsibility of the 
application admin team in question, and it was up to them to decide 
how they wanted to distribute that data -- I think most of us just 
did the simple thing and kept a single /etc/password file for our 
group that we shared out to all the various machines in our area. 
And the internal support guys did the same for the desktop machines. 
We had started with AFS on the internal operations network for doing 
any shared data, but after we grew too large that got canned and we 
went with simple NFS.

The security guys kept pushing for Kerberos, but they were never able 
to deliver it during the time I was there.  I was one of the guys who 
pushed back hard enough that we managed to get them to allow us to 
install ssh and use that until the Kerberos stuff was ready.  I left 
before any of the Kerberos stuff was rolled out, if it was ever 
rolled out.

There were some Bastion machines you could ssh into from the outside 
world, and from there you could access any of the other internal 
systems.  These boxes were locked up as tight as physically possible, 
with access controlled by SecurID plus passwords that had to meet 
some very stringent requirements.  Of course, they couldn't 
physically prevent you from writing down your password, but if the 
security guys ever found out then you probably would have been fired 
on the spot.

None of the system access passwords had anything to do with your 
e-mail passwords -- those were totally separate, and while you had 
some control over what password you used in both of those places, you 
were forced to use their modified password generation tools on the 
Bastion hosts.  So, you had control over two legs of that triangle, 
but not the third.


Then there was the service side.  The AOL client has (had?) a 
proprietary Stratus-based communications protocol that they used to 
download e-mail, and while definitely obfuscated, it wasn't wasn't 
actually encrypted -- at least, not initially.  You could get various 
versions of clients for various OSes, although Unix was never really 
one of them -- at least, not until AOL shipped a MacOS X client, and 
even that is probably still a client written for MacOS 9 with Carbon 
Libraries, which is running under Carbon in MacOS X.

Later, the upper management guys got really ticked off that all the 
operations guys couldn't stand to use the AOL client, so they pretty 
much killed all the old aol.net systems, and forced you to use the 
AOL client.  They hacked in some Kerberos-style "Realms" stuff, so 
that you could ensure that all communications of certain sorts was 
not allowed to be sent to anyone who didn't have the "AOL Employee" 
bit set in their Realm.

Needless to say, there were a number of people who weren't willing to 
stick around to see how the dogfood experiment worked out.


Anyway, loss of an e-mail password was just that -- loss of an e-mail 
password.  That didn't get you into any of the Bastion hosts, that 
didn't get you into any of the production boxes, etc....  Of course, 
someone could have been stupid and shared their e-mail password with 
their admin account on their operational boxes, but most people were 
smart enough not to do that.  Moreover, that wouldn't help you get 
onto any of the production boxes, unless you had some way to get onto 
the Bastion hosts.


Personally, I think the use of a centralized Kerberos scheme is a 
good idea, but you do need to do the work to tie that into all the 
other systems.  And you want to think about what kind of backup 
you're going to have if Kerberos goes down or if the box gets knocked 
off the 'net and you're forced to go in through the console terminal 
server -- you don't want root being unable to log in, because the 
network is down.

Of course, you also don't want the single root account being used for 
all remote admin access just because it's simpler that way.  Real 
root logins should be restricted to the console only, and everyone 
else should be required to go through their personal account, and 
sudo (or sudosh, or whatever) only as necessary.

And I do kinda like the idea of separate passwords for e-mail versus 
actual remote terminal access, in part because most people aren't 
going to need remote terminal access to most boxes -- even if I am 
the mail admin, the web server boxes should not even know about me, 
unless there's some reason I should be accessing the web server boxes 
to set up their mail system for reporting of errors, etc....  Even 
then, my access should be restricted to the functions I'd be expected 
to perform.

I'm also a fan of functional decomposition -- the mail boxes should 
be doing pretty much just mail and nothing else, so even if they get 
compromised that doesn't necessarily help anyone break into any other 
part of the system.  Well, at least it doesn't help them as much.

Oh, and I'm a big fan of encrypting just about everything you can. 
Remote system access, e-mail access, log data transmissions, etc.... 
And two-factor authentication for those mission-critical systems.

-- 
Brad Knowles <brad@shub-internet.org>, Consultant & Author
LinkedIn Profile: <http://tinyurl.com/y8kpxu>
Slides from Invited Talks: <http://tinyurl.com/tj6q4>