Certification exams

Yesterday I sat the beta exam 071-540 Windows Mobile Application Development. I provided feedback on a number of questions but I think that the whole exam process is fundamentally flawed. Most of the questions were of the format “select one of the following code snippets that addresses this scenario…”. Some of these were quite well put together, for example performance around creating webservice instances. A lot of the questons were almost “which spelling is correct”. Hello, most developers dont worry about the spelling any more as we have intellisence. In the worst case where intellisense isn’t available we have well documented help system. Questions that test developers recall of a specific API or method overload are just a waste. They achieve nothing as the only people who will get them are geeks who have nothing better to do than study for an exam.

At TechEd Australia I was chatting with Rob Farley who is a big advocate for exams and certification. He is encouraging as many people as possible to take the beta exams (which are typically free) without studying. This is an interesting test as exams should distinguish the good from the bad, those that can from those that can, but most importantly those who understand from those that don’t. For example, someone who has been building mobile apps for 3 years, understand the limitations and unique design criteria, should be able to pass the mobile exam without studying. However the exam still needs to filter out those people who have just RTFM and expect to be able to pass the exam.  This is an interesting dilema that needs to be addressed as certification as it stands is far from a good indicator of ability!