Performance-based questions (PBQs) are increasingly prevalent in IT certification exams.Certification is an assertion of authenticity, competency and feasibility. In a world of constantly upgrading technology, certification acts as a gatekeeper to ensure that individuals are knowledgeable, skilled, and able to work with hardware devices and software programs. Each certification benefits and reflects upon three essential stakeholders — the holder of the credential, the employer for whom the certified individual works, and the organization supporting the certification.

As training moves from theoretical to application-oriented curriculum, assessment of users is increasingly being done on the basis of practical knowledge and ability. No longer are job candidates only asked what they know, but now they are required to show that they know it through hands-on performance verified via official certification exams.

Traditional multiple-choice certification exams

Traditional certification exams relied heavily on a basic format of presenting a question followed by a set of multiple-choice answers. Preparation typically involves a student reading books, attending lectures and completing a number of practice exams containing similar questions. This exam format is a good way for a candidate to show their “book-knowledge” on a particular subject.

The weakness is that candidates can memorize questions and the various formats in which they are presented and then select the proffered answer that best fits question. There is no real demonstration of reasoning skill or ability on the part of the test taker.

Comprehensive skills rarely find their expression during such exams and a candidate’s theoretical knowledge about the subject matter is often not an adequate enough indicator of their ability to face the job situation and deal with real-world operational issues. Experts have often opined that multiple-choice tests cannot be the sole determinate of an individual’s skills and abilities, especially in IT-related programs.

Performance-based exams

Performance-based exams require candidates to show their real skills. These are known as fidelity tests, where the examiners are able to observe and evaluate a candidate’s actual abilities to work with the specified technologies in a hands-on environment. These types of exams are effective in ensuring that certification holders are actually capable.

These exams contain performance-based questions (PBQs) that test a candidate’s ability to solve problems in an environment that simulates an actual real-world situation. The popularity of such exams has increased due to innovative developments in technology, particularly in regards to lab simulations.

The advantage of PBQs is that the candidate must go beyond “book knowledge” and actually demonstrate his or her ability to solve problems in a hands-on, simulation. It also allows the candidate to demonstrate competency with the correct technology. Because PBQs require procedural knowledge and skills, they tend to reduce the likelihood of cheating on the exam.

Initially, there was concern that performance-based exams were too costly and cumbersome to offer to large numbers of individuals. With the recent advances in cloud technology and simulation, however, such exams can now be offered on a large scale and in a secure manner.

Performance-based testing is a great way to facilitate an organization’s efforts to successfully screen job applicants, assess learning, and determine expertise for job assignment and acknowledgment. Almost all IT certification entities now include a performance-based segment on their exams, including Red Hat (Red Hat’s exams are actually 100 percent performance-based), Cisco with its Cisco Certified Internetwork Expert (CCIE), SAS Global certification program and Novell’s Practicum.

Disadvantages?

Despite the many clear advantages offered by PBQ items, there are some less obvious challenges that need to be met when conducting performance-based certification testing.

One challenge is in designing the exam. A thorough and correct approach to a comprehensive exam is difficult and requires great attention to detail and verifying that you are actually testing for the correct skills and knowledge. This can be time-consuming and expensive. Such exams can also be costly for the candidates and the credentialing organization.

For the credentialing organization, an adequate testing environment must be established, either physically or virtually. On top of that, they must also recruit qualified individuals to administer and score the exams.

Practicing can be expensive for candidates as they will need to set up a virtual environment or subscribe to a virtual service provider. Candidates must also take into account the location of the performance-based exam — it costs money to travel for an exam as well as a loss of time away from the job.

As with any exams, there are also security issues that can harm the authenticity of the exam. However, these are the same concerns that one would have for MCQ exams. For example: brain dumps or a test taker snapping photographs of questions and sharing them online.

Fortunately, most of these concerns are being addressed via modern technology which is increasingly enabling companies to integrate performance-based testing in a scalable and cost effective manner.

Also, there is no industry-wide standard for such testing. As the market evolves, however, the testing organizations and processes are transforming and a consistent standard seems to be developing. The standard for the tests are being designed as per industry standards and different tests are designed to meet different sets of certification requirements.

Performance-based test prep

Performance-based questions (PBQs) are increasingly prevalent in IT certification exams.A PBQ exam is different than a multiple-choice question (MCQ) exam. Therefore a different study regimen must be followed: one that is hands-on. Some basic suggestions include:

  • Learn all you can about the exam. There are numerus forums and discussion boards for every certification exam. Join in and ask for advice and tips about the exam. Don’t use brain-dumps because that is unethical, but ask others about how they prepared, what worked and what they might have done differently.
  • Especially check out the official site of the certifying organization. Their information will usually always be up-to-date and contain data about developments with the certification. If you’re going to take a Microsoft exam, it is a good idea to read their site about what to expect when you take a Microsoft exam.
  • Practice, practice and whenever you have an opportunity, practice some more. Play around with software and hardware that matches the exam questions and especially practice solving scenarios that may be presented on the exam. Your goal should be to practice several times every possible scenario that could arise on the exam. So that when it does, you’ll be familiar and comfortable working through the situation.
  • Get ahold of some actual practice exams and complete them under actual exam conditions. The reason is that many PBQ exams typically run for several hours — some for several days. Prepare your body and mind to meet the rigors of the specific exam.
  • Home and virtual labs are great for practicing scenarios and trouble-shooting issues. If you can, set up a home lab by purchasing inexpensive hardware and software, that you will continuously disassemble and reconfigure, do so. Look for online deals and auctions for second hand equipment. There are also some excellent providers of virtual labs. See which ones fit our budget and practice needs.

Are performance-based tests more effective at verifying IT skills?

Almost every IT certification exam now includes a performance-based component, so it’s a fair bet to say that these tests are good at verifying IT skills. It’s been said that if you want to see if someone can ride a bike, you don’t give them a multiple-choice test — you hand them a bike. Performance-based exams are the bike.