Re: [SAGE] Change control and patching for Linux

[Chris Henderson <sage@xxxxxxxxxxxxxxxxxx>]

Since you are using Red Hat, you might want to look into their RHN
Satellite Server product. You can clone a channel such that it will
remain static, then subscribe the servers to that channel. You can also
take a snapshot of the profile of a server before you up2date it, which
makes rollback easier.

--Chris Henderson

On 01/02/2008 02:36 PM, Jim Ankenbrandt wrote:
> True.  That is great input on scheduling.. The problem with our
> process is that a patch has to be placed in certification, wait, then
> test, wait, then there has to be a change control meeting for DR, 1
> day, patch, then 2 days, then we can do production as long as it is
> not in the monthend freeze,  is a Saturday and the DBA's agree ( :(
> ).  This process can be streamlined but, patching something Oracle
> touches ( God forbid the kernel, etc ) slows the process down.   If it
> is critical, there is a process to shortcut this, but there has to be
> crash or failure of some kind.
> The technology question is how to apply the same patch bundle to five
> servers in the window provided ( typically 2 am to 6 am Saturday night
> )  and make sure it is the same bundle applied to the prior 3
> clusters, and is consistent.
> Scripting would be nice.
>
> Carlson, Scott wrote:
>> Jim,
>>
>> The important thing to consider with your response to your security team
>> is that you need to define a "process", not come to a technology
>> decision.
>>
>> I work for a large financial and we have not defined technology
>> solutions as part of our process, we have defined criticality and
>> timelines.
>>
>> For instance
>>
>>
>> Criticality 1 Patches - Must be applied within 7 days
>> Criticality 2 patches - Must be applied within 30 days
>> Criticality 3 patches - Must be applied quarterly, and are included in
>> patch "clusters" that are certified by our engineering team (Yum based)
>>
>> We then have a quarterly patch process that hits every device
>> (exceptions can apply) applies the quarterly patch cluster to every
>> device in production & development, following a very similar methodology
>> to what you've outlined below.
>>
>> Related to criticality 1 patches (like the patch Tuesday critical
>> stuff), every time that we have a device that is important enough to
>> apply "right now", it expected that the patch will just get applied and
>> that technology owners are responsible for making sure an environment
>> won't break - and they have 7 days.
>>
>> It's all about management will, imho, it's not about technology
>> solutions....  Actual patching is the easy part.
>>
>> Scott
>>
>> -----Original Message-----
>> From: owner-sage-members@xxxxxxxxxx
>> [mailto:owner-sage-members@xxxxxxxxxx] On Behalf Of Jim Ankenbrandt
>> Sent: Wednesday, January 02, 2008 12:29 PM
>> To: SAGE mailing list
>> Subject: [SAGE] Change control and patching for Linux
>>
>> I work for a bank in the American mid-west.  Our data security group has
>> handed down a ruling that we need to begin a routine patching
>> process. Joy.  By routine they mean a log of patches considered,
>> reasons they
>> where rejected or passed, and a defined process to follow when
>> installing the patches.  This will be subject to audit by the Fed's and
>> our internal  groups.
>>
>> Our environment is that we are running Red Hat Linux supporting Oracle
>> 10g  RAC clusters.  The problem is that by the time we start a migration
>> from the certification cluster, test cluster, disaster recovery cluster,
>> to production cluster patches may have been outdated. What with change
>> windows, meetings with DBA's and sign offs it may take close to a month
>> to complete this path. Up2date is too blunt a tool, we cannot have
>> different levels on different clusters.  We are considering installing
>> yum and defining our own repositories, but any experience or group
>> insight would be welcome
>>
>> Jim Ankenbrandt
>>   
>