In the autumn of 2015, KITLV conducted a large-scale opinion survey on the six Caribbean islands of the Kingdom of the Netherlands. The Antillean press has already published various reports about the results of the survey. This interest in the media is not surprising. The results reveal penetrating and often surprising insights into the opinions of the population of the islands about politics and governance on their own island and about relations with the Netherlands. The results of the survey are freely accessible on http://www.kitlv.nl/research-projects-confronting-caribbean-challenges-opinion-survey/.
The press has also reported that the survey was a failure on St. Maarten. Apparently, there has been some misunderstanding about the causes of this failure. With this press release, we would like to set the record straight, in particular regarding the role of the Department of Statistics (STAT), the governmental statistics agency on St. Maarten. In a formal sense, STAT was only affiliated with our survey by supplying a random sample of addresses to be visited. Logistically, however, there was an additional link with STAT: the department assigned its fieldwork coordinator to our survey, and experienced STAT interviewers were recruited for the fieldwork. As a STAT employee, our fieldwork coordinator had been involved with the design and execution of previous STAT surveys, our interviewers form part of STAT’s pool of interviewers, and we trained the interviewers at the STAT office. We had no reason whatsoever to doubt the professionalism and integrity of these persons recommended to us by or via STAT.
We are aware that both in St. Maarten and elsewhere, questions are now being raised about the professionalism and integrity of STAT. It is not for us to judge this broader question. It should, however, be clear that, for our survey, cooperation with STAT resulted in a painful failure.
How the survey was organized: processes and procedures:
On all six islands, we first established contact with the local bureau of statistics. We asked these bureaus for their assistance in compiling a random sample of addresses to be visited, in recruiting interviewers, and in providing us with advice regarding the organization and set-up of the survey. On each island we collaborated with a local fieldwork coordinator, who, in most cases, was selected based on consultation with the local bureau of statistics. Wouter Veenendaal, the principal researcher for the survey, conducted trainings for the interviewers on each of the islands. These interviewers, in some cases, also worked for the local statistics office. On five of the six islands, this process went ahead without any significant problems, and eventually resulted in reliable survey data.
On St. Maarten, we first established contact with STAT. It was agreed that STAT would provide a sample of addresses, would provide us with a list of experienced interviewers, and would offer advice regarding the organization of the survey on St. Maarten. It was also agreed that STAT’s fieldwork coordinator would coordinate and supervise the interview process on St. Maarten. She would be doing this in her spare time, and would be directly reimbursed for her work by KITLV, just like the interviewers that she recruited.
On the 7th and 8th of September, 2015, Wouter Veenendaal trained the interviewers at the STAT office in Philipsburg. On the 14th of September, the survey on St. Maarten began. Just like on the other islands, it was agreed that STAT’s fieldwork coordinator would organize and supervise the survey, and would make sure that the established deadlines were met. She would also conduct some of the interviews herself.
We were notified that the survey on St. Maarten was completed on time, and the boxes with completed questionnaires were returned to the Netherlands. The reported figures soon revealed that the survey response on St. Maarten was extremely high: it was alleged that a successful interview had been conducted at 92% of addresses on the island. On the other islands the response rates were between 50 and 75%, a methodologically much more plausible figure. In addition, during the data-entry process, various inconsistencies and contradictions in the completed questionnaires were discovered. Subsequently, we learned that the fieldwork coordinator had not used the sample that was provided to her by her own employer STAT but, instead, on her own initiative had drawn up and used an alternate address sample.
The implausibly high response rates, when coupled with the use of an unsanctioned address sample, led to a strong suspicion of interviewer fraud during the course of conducting the survey on St. Maarten. An impartial check commissioned by the KITLV and conducted in January and February 2016 confirmed our suspicions and irrefutably proved this fraud. Our conclusion is that only a few interviewers have conducted their work appropriately and professionally, while the majority of them – including the fieldwork coordinator – have committed fraud.
Prof. dr. Gert Oostindie, KITLV Director, Leiden
Dr. Wouter Veenendaal, researcher and principal survey investigator, KITLV, Leiden