Before you make your action plan, this article hopes to convince you of two things. First, the CFA exam passing rate is not determined cavalierly. There is no hype in all that talk about the CFA standards being global, and passing each exam is indeed worth the cost of failing it at least once. Second, no matter how much your pride tells you otherwise, requesting a recheck will likely be just a waste of your money. I know of only one time when the CFA Institute sent out erroneous results. That was when they first offered the December Level 1 exam in 2003. The affected candidates then took the Level 2 exam for free.
The CFA Board of Governors set the minimum passing score after each exam. They consult psychometricians and review the results of a widely-used standard-setting procedure called the Angoff Process. The Angoff procedure requires a panel of competent practitioners to act as “judges” for the questions in the exam. These judges estimate the proportion of borderline candidates who would answer each test question correctly. This probability level could never be lower than 25% in an exam with four choices, because a probability lower than 25% would mean that the borderline candidate will do worse than chance, and that does not make sense at all. The cut-off score is the average of the sums of the probability estimates.
|
|
|
|
|
Judge 1 |
Judge 2 |
Judge 3 |
|
Item |
|
|
|
|
1 |
.33 |
.50 |
.40 |
|
2 |
.80 |
.90 |
1.00 |
|
3 |
.30 |
.33 |
.33 |
|
4 |
.30 |
.90 |
.33 |
|
5 |
.50 |
.75 |
.50 |
|
Totals |
2.23 |
3.38 |
2.56 |
|
Ave of 3 Judges |
|
|
2.72 |
qualified candidate should be able to answer all questions correctly, all the entries would be 1.00, and the average of the 3 judges would be 5. If the judges think the questions are very tough and assign the minimum probability of 0.25 to each question, the average would be 1.25, which says that the just-qualified candidate who studied will do no better than one who randomly selected his answers.
I don’t think the Angoff Process is also used for the essay part of the Level 3 exam, but the essay grading process is no less rigorous. Your fate is not determined by just one grader. There will be as many graders as number of questions. After the first set of graders finish, senior graders review all borderline papers plus a certain percentage of the other papers.
One can argue that the process is still highly subjective. The borderline candidate, the just- qualified candidate, is determined by each individual judge. Who do they have in mind? Their average student at MIT? The typical analyst at Goldman Sachs? If so, doesn’t the process make passing more difficult for most Asians and most developing countries?
It is probable that there is a Western bias, but that can’t be avoided since most financial innovations still emanate from the West. However, the CFA Institute makes a heroic effort at trying to remove whatever bias the exams may have. They try to involve many Asians as graders for the essay part of the exams, aiming to minimize the cultural and communication biases by the multiple grader process. In our own society, at least four have been graders. And this September, at least one member of our society will go to Bangkok to review and comment on the December exam.
Some rumors about the exam are worth confirming. First, since the beginning of the CFA exam, anybody who got 70% of the points always passed the exam. Second, it is true that a borderline case with a good score in the ethics portion will pass the exam. And third, the CFA Level 3 exam will never be completely itemized, and for security reasons, they are not considering the use of computers. Graders claim that they do not give up on handwriting. Given the context of the problem, and after seeing a number of papers, it is possible to discern what a candidate is trying to say, notwithstanding the handwriting. Those of us who have some teaching experience could validate this – after a while, you just learn to read bad handwriting.
When I received my charter the newspaper ad announced us as “those who have met a higher standard.” None of my previous achievements made me feel as much a part of a global learning community as the CFA Charter. Whatever August will bring, remember that when climbing Mt. Everest, even before you reach base camp, you are already above more than 80% of the planet. Enjoy your July.