Staffing the Data Center

By M. E. Kabay, PhD, TEMPAssociate Professor, Information Assurance
Dept. of Computer Information Systems
Norwich University, Northfield, VT 05663-1035 USA

1 INTRODUCTION

How can we best use our investment in information systems (IS)? As we all know after countless articles and seminars, we must

· Identify how information helps us attain our strategic objectives (e.g., for a car fleet servicing operation, this might be to provide the most timely and precise preventative maintenance schedules to individual corporate clients);

· Define our information processing methods accurately (e.g., requiring our computer system to bring up a customer's full dossier for a particular car within 1 second of entering its license plate number);

· Choose our computers and peripherals wisely (e.g., selecting a computer with the highest ratio of throughput to cost of ownership calculated over three years).

Although these are surely sound principles, they all assume that the people who operate our data centers will be competent and motivated to make the best use possible of our valuable systems. Good staff can make a success of IS operations even when budgets are less than optimal by courteous, sensitive, imaginative and competent professionalism. For instance,

· An intelligent operator can notice and question anomalies such as jobs which read scratch tapes before writing to them (a security violation known as 'scavenging');

· A good quality control supervisor can catch and correct reports with illegible printing, find the corresponding spoolfile, and reprint the pages where the printer ribbon failed;

· A creative system manager can find workarounds to keep a mainframe running even if a bug in the operating system threatens to cause system failures.

On the other hand, people can make mistakes with disastrous consequences; for example,

· A bored, tired, drugged, or stupid operator can destroy magnetic tapes by simply mounting the wrong unlabelled tape on a tape unit--overwriting information quite possibly worth more than a decade of his or her salary;

· A malicious system manager can make life hell for every programmer in the shop by insisting on following rules in emergencies even though the consequences of normal procedures are worse than the consequences of breaking the rules;

· A neurotic quality-control supervisor can offend everyone in the company--including the VP of finance--by snarling defensively at them on the phone whenever the system is slower than usual.

This article summarizes the issues to consider when deciding whom to hire for IS operations, what these people should do, and how to train them.

2 OPERATIONS MANDATE

Operations, sometimes called technical support, has the mandate of providing support to software designers, programmers and users. Support includes

· Equipment acquisition and maintenance (buying, selling, insuring and providing repair contracts),

· Problem solving (running a help desk, problem diagnosis, keeping track of previous cases, interacting with manufacturers' support services),

· Training users on central and microcomputer system operating systems, utilities and communications.

3 STAFF SELECTION

One might think that data center operations must involve primarily relationships with machines. On the contrary, all of my experiences as a data center operations manager and information systems operations consultant convince me that personality is the key to long-term success in computer operations. Personality implies, in the words of a dictionary, "the personal or individual qualities that make one person be different or act differently from another;....character, individuality." Operations staff are mediators between large computer systems and their users; therefore, a large part of their work must involve contact with human beings. Personality informs not only how we approach all aspects of our work with computers, it is the fundamental basis of all human interaction as well. It is a commonplace that differences of upbringing and socio-economic status can cause misunderstandings; what is business-like in one milieu is considered brusque in another, and conversely what is friendly in the second is considered inefficient in the first.

Social differences are superimposed on a base of personality and individual development; it should not surprise us, then, that personality is a major factor in how well we run our data centers operations.

So what kinds of personality traits can be particularly helpful in the business of running an IS operation?

· Focus on the overall goals of their organization rather than looking only to the short-term, here-and-now demands of filling the requirements of a job description. For example, if a system manager often sees accounting employees laboriously re-entering data from a printed report into a microcomputer spreadsheet, she should suggest to upper management that they either authorize a change in the printed report to meet the obvious needs of the accounting department or provide support for evaluating, buying and installing a package to select and download data from mainframe databases, files or reports to microcomputer packages.

· Can suspend rules when faced with unusual situations. For example, if a user is desperately waiting for a major report, there's a bug in the program, and the programmer wants to run a compilation in high priority, it would be inappropriate for a system manager to refuse to even consider the request because of rules forcing compiles into the batch processing queues.

· Want to improve their methods. For example, an employee might see an opportunity to create an online system documentation system rather than continuing to use outdated paper copies.

· Help other people accomplish their goals by sharing knowledge. Such a shift supervisor might, for example, respond to a junior operator's interest in how a tape drive works by showing her the appropriate section in a reference manual and encouraging further questions.

The above traits are not guarantees of excellence; they are merely my impression of what seems to help people work effectively in operations.

Avoid hiring people with counter-productive personality traits, many of which were identified as part of the "authoritarian personality" in the late 1940s and early 1950s by social psychologists such as Gordon W. Allport. Beware of those who are

· Process-oriented rather than goal-oriented. Such people often fear the unknown so much they'd rather follow accepted procedures even when conditions are inappropriate than risk the consequences of breaking the rules or parting with precedent.

· Intolerant of ambiguity. That is, people who can't say or accept that someone else could reasonably say, "I don't know" are likely to cause problems in data center operations. They will waste time and effort either denying the possibility of alternatives or criticizing colleagues for asking for more information.

· Judgemental (quick to attribute blame or to devalue a person after a disagreement). This trait leads to grudges and resentments which can poison not only the individual's relations with others but which may also contaminate the atmosphere for the whole work group.

· Excessively sensitive to structural power (e.g., those who flatter their "superiors" and harass their "inferiors" in a hierarchy). A supervisor who glories in finding fault with his employees causes poor performance and higher turnover. One who humiliates her employees in public is asking for sabotage. By the same token, employees who dare not question their supervisor's decisions when they disagree with them are partly responsible for "group-think". Organizations in which dissent is punished are missing the full value of the salaries they pay--the insights available from the range of perspectives and knowledge of their employees.

· Apt to form excessively strong in-group affiliations (e.g., those who tend to make us/them distinctions readily and refer to out-groups in derogatory terms such as "those dumb users"). A sense of teamwork and pride in excellence is laudable; but beware the potential employee who speaks disparagingly about the worthlessness of other groups. Such people may respond automatically to suggestions with "NIH"--"not invented here" (meaning that if the idea were any good, someone in their group would have thought of it by now).

· Likely to follow charismatic leaders and apt to confuse personality, reputation or authority with infallibility (e.g., members of doctrinaire cults and racial-supremacy movements).

· Prejudiced (form judgements about individuals based on membership in identifiable groups or on external appearance instead of on the basis of behavior or performance).

Many fine operations staff members show some of the traits above, and no one should be rejected simply because they show a few of the problems listed above. In any case, people may show this undesirable behavior because of troubles they have experienced in their childhood and youth. Psychologists tell us that rigid, authoritarian behavior is often a response to insecurity. At the root of this insecurity is a feeling of inadequacy--the fear of falling short, of displeasing an all-powerful father figure. Many people have been traumatized by their experiences in schools, where some teachers unfortunately humiliate some children for making mistakes; sadly, these children come to believe that authority figures hold all the answers, and that there's something wrong in not knowing those answers. Such people can be helped by various kinds of therapy to become aware of their fears, face them, and rebuild their self-esteem and confidence.

4 ESSENTIAL SKILLS

What are the essential skills we should look for in our potential operations employees? Some of the most critical are the ability to

· To notice anomalies (e.g., "Why is this job taking 20 times longer than ever before?");

· Question outliers (unusual or extreme cases; e.g., "Why is this program taking 20 times more CPU than the next most intensive program?");

· Keep detailed records (e.g., "Just before the system failure, I typed a file equation and misspelled the device class name.")

· Break down problems into operational questions (e.g., "The bizarre characters on screen could be due to the terminal, the cable between the terminal and the modem, the modem, the phone lines, the ... [etc. through the data communications chain] ... or finally, the software").

· Formulate models (hypotheses) that can be falsified (e.g., "If the problem resides in the terminal, then switching the terminal to another data communications channel will leave the problem on the terminal").

· Remain free of ego-involvement in problem solution, allowing the person to discard an hypothesis without resistance once it's disproved (e.g., "Oh, it's not that, then. So much for that theory. Now, the next idea is....").

· Adapt to different operating environments (e.g., mainframe vs mini vs micro).

· Capable of switching quickly among different tools and appreciating their different strengths and weaknesses without complaint or resistance; e.g., command sets, fourth-generation languages vs third-generation languages, and different editors or debuggers.

5 ACADEMIC BACKGROUND

Computer skills seem almost independent of academic training. Many expert TS and SM staff have little or no formal training in computer science. One should evaluate the candidate's current skills rather than paying overmuch attention to theoretical implications of academic background. However, other factors being equal, one might favor different academic disciplines in order of preference as

It would be a mistake to refuse a good candidate merely on the grounds that he or she had completed a university degree in a "soft" subject (e.g., psychology, anthropology, literature, or art). Clear thinking and self-discipline are not the exclusive properties of people from the “hard” sciences.

6 JOB EXPERIENCE

Ideally, one should look for experienced operations staff to fill senior positions. In contrast, lower levels such as operators can be hired with less experience because their required skill set is less complex and the learning curve is much shorter than for management skills.

In particular, the operations manager should have a direct knowledge of technical support or operations, not just programming management. Managing operations is distinctly different from managing programming:

· Operations planning timescales are longer than programming timescales. Programmers have control over their implementation schedules (to the extent that development is on time) but operations have to take into account long-term capacity planning (usually measured in years) and hardware- and software-acquisition lead times (usually measured in months).

· Operations problem-solving timescales are shorter than programming timescales. Problems arise hour by hour; interruptions arise minute by minute.

· Operations staff must handle multiple problems at the same time because there are usually tests to be run or actions to be taken in sequence during problem solution. Programmers usually work on one program or system at a time because it doesn't make sense to interrupt programming to do a bit on another program.

· Operations must adopt a global view of the entire system or network; they cannot benefit only one user group at the expense of another. Programmers must make their own client community's needs paramount.

7 ADMINISTRATION AND PRODUCTION

Another distinction to consider when evaluating a candidate's job experience is the contrast between the administrative manager and the line manager (someone responsible for some kind of production). Administrators who have not experienced the joys and terrors of having 80 users waiting for the system to come back up after a failure cannot reasonably be expected to empathize with operations staff's typical emphasis on getting the job done instead of worrying about what the procedures should be. The temperamental differences between career administrators and career problem solvers only complicate the task of managing operations. An administratively-oriented manager may conflict with technically-oriented staff:

· Administrative managers may emphasize building an administrative staff within operations because paper-pushing is inherently more comfortable for them to manage than technical demands; technical managers will push for adequate staffing of problem-solvers and problem-preventers.

· Administrative managers may de-emphasize contacts with technical personnel because of insecurity about the technical concepts and vocabulary used in discussing their operations staff's needs; technical managers will relish the chance for nuts-and-bolts discussions of options and solutions.

8 SUMMARY

Staff selection makes a big difference to the success of a data center. Operations staff have direct responsibility for the availability of computer services; nonetheless, they also have direct contact with users and decision makers. Personality plays a major role in how well our staff interact with our users. Candidates should focus on overall goals, be flexible about applying rules, and enjoy sharing knowledge. Avoid those who are process-oriented rather than goal-oriented, intolerant of ambiguity, judgemental, excessively sensitive to structural power, apt to form excessively strong in-group affiliations, prejudiced, and likely to follow charismatic leaders.

Some of the essential skills for operations staff include an ability to assimilate information quickly, meticulous attention to detail, penchant for analysis, and flexibility.

Experience in operations, as distinct from programming, is highly desirable because of the differences in planning and problem-solving timescales, and the global view required for operations. Another useful background is line management, which can help operations managers handle the technical demands of support and planning.

[*] This article was first published in ACM Ubiquity online magazine in 2003
< http://campus.acm.org/forums/ubiquity/messageview.cfm?catid=4&threadid=251 >

STAFFING THE DATA CENTER [*]