The Problem with Certification Programs

Beware of superficial qualifications

Professional computer certification programs have been around for a long time, but none has been as successful as Novell's Certified NetWare Engineer (CNE) program or the Microsoft Certified Systems Engineer (MCSE) program. Novell and Microsoft created these programs to provide an additional revenue stream for themselves and for third-party companies offering training aids and services. This motivation doesn't mean these programs aren't good, and it doesn't mean the companies' original intent--­to provide a certification process that gives computer professionals recognition for their skills--­was a poor one.

I first ran into a problem with certification programs about 7 years ago when I was hiring the staff for a computer-testing lab. At that time, Microsoftcertification programs didn't exist, and Novell's CNE program had been aroundfor only a couple years. The lab's classified advertisements for these staffpositions asked for candidates with a 4-year degree or related computerexperience. Many applicants for these networking team positions were CNEs.

After interviewing a half dozen promising candidates, I discovered thatthey all had attended Novell training and had worked on classroom networks, buthad no real-world experience. Not that this lack of experience was necessarilybad. All the candidates qualified for entry-level tech positions. However, noneof the candidates had applied for entry-level positions, nor did they have whatI call the inference engine. That is, not one of the candidates coulddraw on the information they had learned in the classroom and apply it in anunfamiliar situation. The candidates were able to answer the questions we askedthem or perform the tasks we gave them on our test network--­as long as thecandidates had covered the questions or tasks in their class work.

I don't mean that every CNE on the planet fell into this category; I knewquite a few who didn't. But I had the misfortune of getting lots of applicantsfrom local computer training schools that promised students they could "bea CNE in 8 weeks; no experience necessary." And I'm pretty sure theseapplicants weren't the cream of the crop.

I talked to people from Novell about the quality of the CNEs thesecertification mills were turning out. They listened with great concern, butdidn't think they could prevent the existence of such programs. And becausebusinesses were beginning to require CNE training as the price of admission tojob interviews, I doubt Novell felt compelled to make any serious changes.

When Microsoft introduced its certification programs, the company reallyraised the bar compared to the CNE training. Microsoft's program required ahigher level of knowledge, offered more intense training, and gave moredifficult tests. But the same thing that happened to the CNE program is startingto happen to the MCSE program. As businesses have begun to require MCSE training(in some cases, certification is the only requirement), certification traininghas become an industry that has spawned dozens of third-party products designedto teach the novice the information necessary to pass the tests. Given the depthof knowledge necessary to pass the tests, the availability of training productsseems like a good thing.

The problem is that a noticeable percentage of the MCSE certificationtest questions have three answers: the right answer, the wrong answer, and theMicrosoft answer. An experienced computer professional usually knows the right answer, but also understands the testing process wellenough to know the Microsoft answer. The result is that people who have receivedMCSE training, but don't have experience to back it up, get shortchanged. Thesepeople learn how to answer the test questions but don't learn a nonbiased answeror develop the skills to apply the knowledge they've been force-fed. They neverrealize that MCSE training prepares them for a life in a Microsoft productenvironment.

I'm starting to see more and more people with the letters MCSE after theirname who are less than qualified to be certified systems engineers. I subscribeto seven active NT mailing lists that cover topics ranging from general NTinformation to hardcore NT development. Over the past 6 months, I've noticed adisturbing trend on these lists: People with the letters MCSE in their .sigfiles have been asking very basic computer questions. And the trend is notlimited to two or three people--­at least 10 to 20 people on almost everylist ask questions that anyone with a year or two of on-the-job experience cananswer.

If you're reading this magazine, you're probably not one of those people.However, you might be interviewing one of them in the future. Don't let yourhard work earning MCSE credentials blind you to the shortcomings of the nextMCSE whose résumé lands on your desk. I also worry aboutsmall-business owners who think they're hiring computer experts when they hirecandidates solely on the basis of their MCSE credentials.

Now that thousands of MCSE people are ready to excoriate me, I'm going to make another group really angry--­ HTML developers. The ability towrite HTML code doesn't make you a programmer. I don't mean that an HTMLdeveloper can't be a good programmer, but spending time developing HTML skills doesn't make someone a programmer. Anyone who has ever coded for a living knows what I mean. We've all seen really cool Web pages that result from dozens of lines of spaghetti code. This code would appall anyone with a strong programming background. You're probably asking yourself, "What's the difference? The pages work, and who are the HTML developers harming?"

The problem comes to light when you realize that these HTML developers have started writing Java code, and Microsoft is pushing Visual J++ (VJ++) as the next great development tool. To make matters worse, VJ++ has wizards to help build component object model (COM) objects. Do you really want your Web-site designer building distributable COM objects for your business? Although I applaud Microsoft's goal of simplifying the development process, I'm concerned about the quality of code that this wizard-based development encourages--­especially when the folks doing the development might never have learned good coding techniques or the basics of writing in a structured language (much less an object-oriented language; inheritance and polymorphism are concepts you rarelyencounter in HTML code).

But my concern is not only about VJ++ (though I have to admit to reallyliking VJ++ 6.0). In a meeting with Microsoft's Visual C++ (VC++) team, Ibrought up a problem that I have, as a casual programmer, with some wizardtools. When I mess up the code that I'm writing to complete the wizard's task, the only reliable way for me to fix the problem is to go through the entire routine and re-create what the wizard has done from scratch. The VC++ team responded with the tale of a VC++ developer who had asked job applicants what they would do if the development wizards were broken and they had to code from scratch. Apparently, the developer wasn't amused when job applicants offered to bring in their functioning VC++ wizards from home.

The only justification the VC++ team offered me for the Let the wizard do it attitude was the example of college students who were good C++ programmers but had never been exposed to the Win32 development environment, having worked solely on UNIX. Such students, they said, could now become more productive Win32 developers in a short time by taking advantage of all the available wizard-based tools. If these kids are good programmers, I'd rather they learn the Win32 API and its behaviors as quickly as possible instead of trying to figure out what went wrong as they combine their scratch code with the code the wizard created. After all, what would they do if the wizard broke?

Such developments as MCSE certification without practical experience and the availability of wizards are creating a job pool of people who aresuperficially qualified at best. If you are looking for qualified professionals, you need to make sure your candidates can actually do the work you hire them to do.