[official] letter of clarification re certification [long]

[Barb Dijker <barb@usenix.org>]

This letter is from the SAGE certification subcommittee:
Kim Trudel, Tim Gassaway, and Barb Dijker.  We hope to respond
to the concerns addressed on sage-members regarding the recent
certification announcement "heads-up" posted 2/17.  In addition,
we would like to provide general information regarding direction
and framework.

The SAGE certification subcommittee met on 3/7 and 3/8.  Not the
least of our agenda was summarizing, considering, and addressing the
feedback recently received via sage-members mailing list and others
send directly to the board.  The information here is organized by
specific concerns that members have raised as we understand them.
The discussion on the list has been extremely constructive and
focused on some key issues. Thank you!


1) The Plan

The "Plan" as referenced in the announcement (2/17/98) was merely:
appoint a subcommittee.  That "Plan" included a meeting of the
subcommittee and results of that meeting for consideration at a
future SAGE board meeting before we continue the process. The
initial outcome of the subcommittee will look more like a real
plan, some preliminary concepts of which are incorporated in the
comments below.


2) Motivation

The certification effort is not motivated by the
survey. The survey merely confirmed what SAGE was hearing from the
membership. As many have noted, certification has been discussed at SAGE
board meetings almost every time since SAGE's founding 6 years ago.
There have been some relatively recent developments that have pushed
this effort to the point where it is now, more than ever, appropriate,
necessary, and feasible. SAGE board members have done considerable
research in the area of certification.  Whether the members are aware
of it or not, there are bodies certifying system administrators in a
non-vendor specific way.  It is happening. We can either let others
do it for us, or we can do it right... by setting skills requirements
and influencing education. 

The primary motivation for SAGE embarking on certification at this
time is the unprecedented demand in the market for system administration
professionals.  Certification is by far not our only means of
addressing this problem.  But it can serve to set goals of education
and standards of skill requirements for at least entry level positions.

To borrow from Elizabeth's posting dated 2/18:

        1) It provides employers, who are not often knowledgeable about
system administration, with an objective standard of evaluation.
        2) It provides system administrators with a way of objectively
evaluating their own skills.
        3) It provides a basis for educational programs.
        4) It's a whole lot cheaper than a college degree, and provides 
some of the same advantages to its holders. 

It is important to note here, and expanded later below, that in no way
is the core certification being contemplated by SAGE intended for
existing practicing system administrators with more than 3 years of
experience.  SAGE is addressing core/entry skill sets.

Many professions, from nursing to day care to jewelers, engage in
certification efforts.  Certification, specified by the profession
itself, serves to set a standard of excellence at the designated
certification level.  There is no evidence we have found to support
reduced salaries or a standard of mediocrity.  Generally the opposite
tends to be the case.

Finally, the argument that certifying system administrators is too hard
simply doesn't fly anymore.  System Administration is the not the first
nor only profession that relies heavily on "soft skills" such as
communication, problem solving, and the like.  There is an entire
industry of professionals who devise methods of assessing skills.  It
may take more than a multiple choice test, but it is neither hard nor
impossible to assess such skills.  Methods of skills assessment have
improved considerably over the years.


3) Relative Prioritization

Certification is not being pursued by SAGE at the expense of any other
efforts.  Most specifically, efforts in the area of education are at the
top of our list, and certification goes hand-in-hand with that effort.
We expect that given SAGE certification, many educational bodies will
then have the detailed skill requirement goals they need to help them to
develop appropriate curricula to satisfy those requirements.  Part of the
effort of certification will include the logistics to make that available
and marketing, for lack of a better word, to ensure the information
reaches educational bodies.

For perspective, in 1997 SAGE spent about $85K in publications including
the booklet series and ;login:.  That is not going to diminish.  In fact
the 1998 budget for such "educational" publications is over $110K.  We
anticipate three booklets published this year: hiring, education, and
site audits. SAGE had already planned to do a direct mailing to
educational institutions, for the purposes already discussed on this
list, as part of the education and certification goals. In addition,
some ambitious members are prototyping a mentoring program that the
board is involved with and hopes to adopt next year. Other things on
our short term goals list:

	- sponsor another K12 event like the Maryland Virtual High School
	  event last year
	- locals focus (clarify/expand policy/benefits, more groups)
	- establish "speaker's bureau" to facilitate technical talks
	  for locals and other groups
	- expand relationships with outside groups (like we did with UGU)

Given that "educational" publications accounts for about one third of
SAGE's budget, it is not unreasonable to spend a few $K, if any, to
round out education goals with a certification effort.  If more dollars
are required, they will not come at the expense of other programs,
existing or future.


4) Authority, Mandate, etc.

The Board sees itself as a steward of the organization,
responsible for general oversight and long-term planning and
strategizing.  In doing so we respond to industry and community
demands as well as the desires of existing members.

The goal of certification is to provide skills requirements and
assessment for budding system administrators for their benefit and
the benefit of the rest of the community and the profession as a whole.
We are pursuing certification a result of member interest and
community/industry demand.

SAGE would not be doing this if we thought it was opposed by a majority.
The survey was in part to confirm our understanding of our implicit
authority.  We can poll, survey, or vote the issue to death.  However,
the SAGE board's duty is to interpret the needs of and act on behalf
of the membership and the broader community it is formed to serve.

SAGE is uniquely positioned to specify certification
for our profession.  Certification is most successful and beneficial
for the community when specified by the profession.  SAGE (US) is the
largest and most globally represented group of system administration
professionals.  SAGE is the only such organization of which we are
aware in the US.  While SAGE does not represent the entire system
administration population, it represents a significant population.
Most professional organizations without mandatory licensure or
certification do not represent 100%, or any where near, of their
practicing population.  One might argue that it is incumbent upon SAGE
in this unique position to specify certification if it is to be done.

There are certainly other SAGE and SAGE-like organizations around the
world, most notably SAGE-AU.  While we can't pretend to represent those
organizations, it would seem a bit wasteful and redundant to reinvent
the wheel N times, one for each regional SAGE.  We hope to gain
participation globally to prevent confusing or conflicting certification
of system administrators by "the profession" itself.


5) Communication

It should be obvious at this point that we haven't actually _done_
anything yet.  No one has been left out of the process.  Every single
one of the messages posted to sage-members were read by everyone on the
certification subcommittee (through at least 3/1).  The certification
subcommittee has developed a communication strategy for the upcoming
process to ensure that the system administration community has
appropriate opportunity for information and feedback.  SAGE certification
will not be successful unless it truly serves the broader sysadmin
community and the employers they serve, not just SAGE members.

The communication strategy is as follows:

  a) Appoint an Advisory Council - < 20 people representative of the
  community for tight and detailed feedback loops, care will be taken to
  ensure representation is both diverse and global.  Details on the
  Advisory Council will be emailed separately.

  b) Develop and post (via web) a FAQ on the project.

  c) Provide regular informational dissemination at project milestones
     in ;login:, sage-announce, and comp.org.usenix.

  d) Schedule certification BoFs at upcoming conferences.

  e) Do at least one directed mailing to all members to ensure
     membership awareness.

The topic of certification will of course be included in SAGE efforts
in general to reach out to the larger system administration community
beyond its membership: vendors, other groups, educators, etc.


6) Framework and Basic Logistics

The initial announcement did not include any conceptual details.  The
following should correct some common misconceptions about what SAGE
means when it says "certification."

  - Certification assesses primarily core skills.  We'd like
    to structure the program in such a way that concentrations in
    specific areas, such as security, webmaster, postmaster, newsmaster,
    or performance tuner, may also be incorporated into the program. The
    specifics concerning skill set, and how this may relate to the
    existing SAGE Job Descriptions, will be defined over the coming 
    months. Again, the certification program is entirely optional and
    not intended to target seasoned sysadmins.

  - Certification will be de-coupled from education, i.e. it is a
    distinctly separate event.  Therefore, there is no requirement
    for expensive books or courses.  This also allows wide implementation
    of satisfying educational curricula, from home study to university and
    commercial programs.  This also means that certification is not just
    attending a specific set of courses but rather requires passing
    a skills assessment.

  - Certification is not required in any way, shape or form.


7) Testing Pitfalls

Unfortunately poor examples of "certification" abound in our industry.
We recognize that and plan to work hard to ensure that any skills
assessment in this certification program will:

  - be of merit, i.e. have meaning, not be guessable, etc.
  - explore assessment methods beyond multiple choice tests
  - assess concepts and comprehension thereof, not trivia
  - not be vendor specific

MSCE, Sun, and Cisco certification are an examples of certification that
are specifically designed to certify product knowledge.  They are specific
to a topic, not a profession.  Thus they primarily test knowledge, not
necessarily skills.  Skills are the application of knowledge and what
is important in system administration.


8) Hiring Impact

Unfortunately SAGE has little if any control on how certification might
be used by employers.  Certification of system administrators, no matter
who implements it, is bound to be misunderstood or misused by some.
This will happen even if (especially if?) SAGE does not produce a
certification program.  What we can do is encourage, through things such
as the upcoming Hiring & Interviewing booklet, appropriate and complete
means of assessing job candidates which may include certification, but
is not certification alone.  SAGE can educate and encourage sound and
reasonable hiring practices, but can not enforce them.


9) Legal Liability

Certainly a potential for legal liability exists.  Legal counsel will
advise SAGE so we can make an informed decision whether the liability risk
is acceptably minimized.

Many states have laws which prevent employers from requiring testing which
has not been developed by certified test writers.  For this reason, and
because professional test developers will undoubtedly do a much better
job than we can, SAGE will need to engage outside professionals to
develop certification skills assessment mechanisms.  Such organizations
provide liability safeguards through validation using proven mechanisms.
Most offer services which are touted as "legally defensible."  Examples
are http://www.proexam.org/, http://www.humrro.org/, and
http://www.chauncey.com/.


Other sites that might be of interest:
http://www.iccp.org/
http://www.ibm.com/certify/
http://www.learningtree.com/us/certific/735.htm
http://suned.sun.com/suned/index.html


Thank you for your time and valued feedback.  We appreciate the comments
we have received.  We expect to continue this dialog on sage-members,
particurlarly as the certification framework is more fully defined and
reaches specific milestones.


Kim Trudel <kim@usenix.org>
Tim Gassaway <gassaway@usenix.org>
Barb Dijker <barb@usenix.org>