The attached letter to the editor is of my own making, and some of it is based on a relatively long connection to the NCEES exams. I did not consult with NCEES about my comments (and I’m guessing that Joel did not consult with them either), and thus any errors or misstatements are mine alone. I have copied a few people that have also had some exam experience and understand the importance of exam content validity.
February 9, 2009
Mr. Marc Cheves, L.S., Editor
The American Surveyor
905 W. 7th Street #331
Frederick, MD. 21701
Re: Point to Point – January 2009
Dear Mr. Cheves,
The January issue of The American Surveyor has an interesting commentary by Joel Leininger, L.S. regarding “Minimum Level of Competence” (Point to Point, page 6). The term has generated tomes of discourse and discussion, especially with regard to those professionals licensed by government. On the surface one can agree with Mr. Leininger’s annoyance with the phase, and his discussion about the term being indefinable and lacking exactness has some merit. However, the premise that there is some required clarity should be from a public protection vantage, not with regard to one’s livelihood.
Regardless, the issue (and part of the term) is well founded in professional regulation, and that concept needs to be embraced as the standard for professional practice. Typically surveying licensure has three requirements; education, experience, and examination(s). Unfortunately, these requirements do vary from state to state, but there is a general solidarity that all three are required. Educational and experience requirements are nearly never defined in terms of competence, minimal or otherwise. Educational fulfillment is most often validated by a degree (hopefully accredited) and/or a transcript of approved coursework. There is a proposed trend for some engineering professionals to start defining educational competence in terms of levels of achievement in accordance with Bloom’s taxonomy. Experience is vitally important, but some state statues lack the structure and exactitude necessary to validate that requirement. Consequently experience is most often NOT measurable in terms of what is required from a minimally competent candidate.
This brings us to the third licensure requirement, that of examinations. Typically for surveyors there is a national fundamental exam, and professional practice exam and a state specific exam. Here is where Mr. Leininger should have done a little more research before coming to the conclusion that the exam procedures (adhered to by NCEES) are “a wild stab at the minimum competence issue”. First of all NCEES does not determine the minimum competence of the candidates, state licensing boards have that function.
Exam development procedures at NCEES adhere to the Standards for Educational and Psychological Testing (most current edition), and by professional consensus, these standards define the necessary components of quality testing. The standards emphasize the concept of content validity, and a job analysis (PAKS – professional activities and knowledge study) is needed to obtain specific task information and the knowledge required to competently perform those tasks. The claim that … “the idea is to poll newly licensed surveyors about the tasks they are facing and then structure the NCEES test…..” is quite misleading if not untrue.
The Surveying PAKS that I am familiar with were completed by NCEES in 1979 (Boyles), 1984 (Boyles and Palmer), 1991 (Warner and Warner), 1998 (Warner and Warner) and 2003 (Chauncey Group). In 1991 nearly 75 % of the respondent population had been practicing for 6 or less years (newly licensed). However, in 1998, 87% of the respondents had more the 5 years experience (12% 1-4 years), and in 2003 over 85% had more than 5 years of experience (10% 1-4 years). Clearly the last two PAKS show very experienced respondent surveyors, and predominately these licensed surveyors were 40-60 years old and in private practice. Consequently, the question of “circular logic” is not logical. The PAKS committee is comprised of highly experienced and very diverse surveyors and subject matter experts, and the respondent population (detailed above) is asked how important a task and knowledge area is to protect the public (i.e. , licensure). Thus only the most important task and knowledge statements are included in the exam specification, which is certainly not a wild stab at the minimum competence issue. The NCEES exams do, in my opinion, satisfy the concept of content validity.
Most importantly and the purpose of this response, the annoying “level of minimum competence” phrase. I too am annoyed, but just with the word minimum, whether dealing with competency or standards, as it connotes a lesser amount. Why should we all just be minimum? The NCEES Model Law has no mention of minimum competence, only minimum requirements for licensure. In 2006 a NCEES task force recommended that (relative to their exam cut scores) the definition of minimum competence must be standardized, and that the standard should emphasize the critical knowledge necessary to protect the public ( See Action Items and Conference Reports page 140 NCEES 2006). After much study and discussion the committee proposed a generic definition, and this was accepted by the Council and incorporated into their Exam Development Procedures Manual. In part this definition says, “The standard of competence is the minimum level of knowledge and skills an individual must demonstrate in order to practice (land surveying) and be in responsible charge in a manner that will protect the general health….”. The exam specifications are a part of this definition and consequently the definition encompasses all of the tasks and knowledge from the PAKS. There is no “minimum competence”. There is also a different standard for the fundamental exam, as it is strictly for entry into the licensure process.
Hopefully a licensed surveyor will continue to grow, become more knowledgeable, and increase their level of expertise. Part of being competent is knowing what you don’t know, and that standard dictates that a professional stay within their level of expertise.
Very Truly Yours
Robert C. Krebs, P.E., L.L.S.
134 East Shore Road
South Hero, Vermont 05486
Tel. 802-372-4567