Re: [SAGE] Change control and patching for Linux

[Narayan Desai <desai@xxxxxxxxxxx>]

On Wed, 02 Jan 2008 15:36:17 -0500 Jim Ankenbrandt wrote:

  Jim> True.  That is great input on scheduling.. The problem with our
  Jim> process is that a patch has to be placed in certification, wait, then
  Jim> test, wait, then there has to be a change control meeting for DR, 1
  Jim> day, patch, then 2 days, then we can do production as long as it is
  Jim> not in the monthend freeze,  is a Saturday and the DBA's agree ( :( ).
  Jim> This process can be streamlined but, patching something Oracle touches
  Jim> ( God forbid the kernel, etc ) slows the process down.   If it is
  Jim> critical, there is a process to shortcut this, but there has to be
  Jim> crash or failure of some kind. 

  Jim> The technology question is how to apply the same patch bundle to five
  Jim> servers in the window provided ( typically 2 am to 6 am Saturday night
  Jim> )  and make sure it is the same bundle applied to the prior 3
  Jim> clusters, and is consistent. 

  Jim> Scripting would be nice.

We have a similar issue on one of our production clusters that we use
bcfg2 on. (In the interests of full disclosure, bcfg2 is one of my pet
projects) In our case, changes can't be made while user jobs are
running. Basically we batch them up, and then deploy them out during
the maintenance window. This handily either applying the changes to
the production repository at any point and running clients in dry-run
mode outside of the maintenance window, or using svn commits to batch
the changes and applying them to the repository during the maintenance
window.

Bcfg2 should support what you are describing using multiple servers,
each serving a repository corresponding to the three states you've
described (dev/test/prod). Feel free to contact me off list and I can
walk you through how it would be setup.
 -nld