Abraham, Manja D., Hendrien L. Kaal, & Peter D.A. Cohen (2002), Licit and illicit drug use in the Netherlands 2001. Amsterdam: CEDRO/Mets en Schilt. Pp. 81-97.
© Copyright 2002 CEDRO Centrum voor Drugsonderzoek.

[Press release] [Index] [Previous Chapter] [Next Chapter]

Licit and illicit drug use in the Netherlands 2001

Chapter 3: Non-response survey

Manja D. Abraham, Hendrien L. Kaal, & Peter D.A. Cohen

Tables

3.1 Introduction

This chapter describes the non-response study for NPO 2001. Of all persons approached during the main survey with the request to participate, 47.1 per cent eventually did (table 3.1). This percentage was higher in the municipalities with low address density, and somewhat lower in the higher address density municipalities. Overall, this means that 52.9 per cent, equivalent to 19,852 persons, did not respond. The previous chapters emphasized the importance of representativity of the survey results. The sampling and weighting methods used ensured that the response data represent the research population on a range of demographic variables such as age, gender, marital status and address density. However, it is still possible that the final response group is different from the research population on one or more of the research variables. For example, people with a relatively low societal position were found to be underrepresented. In population surveys in the Netherlands by CBS this led to biased outcomes even after weighting (Riele 2002). A non-response survey was set up to verify to what extent this is the case.

The non-response survey described here focuses on two issues. First, it sets out to verify whether, after correction for demographic variables, the response group of the main survey represents the research population on target variables such as alcohol and cannabis use. Secondly, it aimed to determine how differences in non-response between the two survey methods used (CAPI or MM, see Chapter 2) influence the research findings. To this end, the non-response survey was set up so that separate samples were drawn from the non-respondents of each survey method.

The first part of this chapter focuses on the extent to which the response group of the non-response survey is representative for the non-respondents of the main survey. The following paragraph discusses the reasons respondents had for not participating in the main survey. Finally, participants in the non-response survey are compared with the respondents of the main survey on variables such as alcohol and cannabis use.

3.2 Sample and response of the non-response survey

Types of non-response

In 1997, two types of non-respondents were differentiated: those who were not at home and those who refused to participate (Abraham et al. 1999). The CAPI methodology allowed this straightforward distinction to be made: those who were not at home had no possibility to participate in the main survey; those who refused had the opportunity, but chose not to. The results of the 1997 non-response survey showed that, in comparison to national estimates, those who were not at home had higher rates for cannabis lifetime use and alcohol recent use whereas those who refused to participate in the main survey showed lower rates for cannabis lifetime use. In 2001, when the bulk of the respondents was interviewed with MM, the clear-cut differentiation between those not at home and those who refused to take part was no longer possible.

The best part of the NPO 2001 respondents was approached following the MM protocol. As was explained in chapter 2, the many ways in which respondents could participate in the survey leads to an increase in non-response categories. On the basis of prior experience non-response was subdivided into the following five categories: refusal; no contact achieved; contact achieved (but no successful interview); language problems; and non-valid cases.

The difficulty in comparing CAPI and MM non-response categories lies in the ambiguousness of the response status, partially caused by the absence of an interviewer. 'No contact achieved' in MM includes no reaction to the invitation to participate, no phone answered or no questionnaire returned in reminder phase. Thus, it could be a 'frame error' if the person selected did not actually live on the given address, but could also be a 'refusal' if the person deliberately chose not to respond. The category 'contact achieved' - where the contact did not result in any interview - is also regarded as a 'soft refusal'. It could be that an appointment was not followed up by a successful interview, but also that a person who responded by indicating the desired mode of interview never actually completed the interview. The uncertainty about the reasons for non-response with MM makes it impossible to compare non-response categories between CAPI and MM, and thus between 1997 and 2001. Non-response due to language problems and non-valid cases is not further looked into in the non-response analysis.

Non-response sample

The non-response sample was drawn from the non-respondents of the main survey as described above. The sample was stratified to mode, stratum and to refusal/no contact achieved. As a consequence it is not possible to look at unweighted results. Response was not equally spread over demographic characteristics like age and address density. Thus, the non-response sample frame is unevenly distributed in the opposite direction: because younger people were relatively more likely to participate in the main-survey, there were relatively more older people amongst the non-response group. Therefore, weights were also calculated and applied for mode, age, gender, marital status, stratum and non-response type (no contact achieved, (soft-)refusal).

Fieldwork

The persons in the non-response sample were asked by telephone to participate in a very short interview; if their telephone number was not listed they were sent a two-page questionnaire. The interview contained questions about life-style (number of evenings spent at home) and the use of cannabis and alcohol (lifetime, last year and last month use). To encourage response of this hard to reach group, again an incentive of ƒ25,- (€11.34) was offered for participation.

Table 3.2 shows the size of the samples and the response of the non-response survey for CAPI and MM. Not surprisingly, response percentages of the non-response survey are even lower than those of the main survey. The average response rate in the non-response survey is 27.2 per cent, versus a 47.1 per cent response rate in the main survey. It is interesting to see that for Amsterdam the response rates in the non-response survey are slightly higher than they are nationally, whereas the Amsterdam response rates for the main survey were lower than the national average. Response rates are higher amongst those who were initially approached by CAPI (35.1%) than amongst those initially approached by MM (26.2%). Table 3.3 presents the same information as 3.2, but now per response type. In the group 'refusals' response rates are higher than in the group 'no contact'.

3.3 Reasons for non-response in the main survey

All respondents in the non-response survey were asked about their reasons to refuse or decline participation in the main survey. The results can be found in table 3.4a. Almost one third of respondents indicated that they had not participated in the main survey because it had been inconvenient or because of lack of time. The rest of the respondents gave a wide range of reasons why they had not taken part in the survey, none of which stands out in particular. What is important for the data quality of the main survey, is that only a small proportion of the non-response respondents indicated that they had not participated because of the quality or the subject of the study; three per cent of respondents in the non-response survey indicated they had not participated because they never used any of the substances mentioned in the main survey, while two per cent did not agree with the aims of the survey or failed to see its relevance. An even smaller proportion indicated they did not participate because they thought that this was a bad survey or that it was a waste of money.

Table 3.4b shows the reasons for non-response by survey method and non-response type. No reasons for CAPI no-contact are given - these people never heard about the survey, and thus did not make a choice regarding participation. Reasons are given for MM no-contact: many of these people received correspondence and decided not reply. Although there are some slight differences in the reasons for non-response given by each subgroup of non-respondents, the similarities in their answer patterns are more striking.

3.4 Some characteristics of the non-response sample

Table 3.5 compares the response group of the non-response survey with the response group of the main survey on a number of characteristics. These characteristics were chosen to estimate the reliability of the figures produced by the main survey. Alcohol use amongst the non-response sample is slightly lower than what was found for the main sample, and also lower than what was found for the main sample in 1997. Lifetime cannabis use is lower in the non-response group as well, but no differences between the respondents of the main sample and the non-response sample are found for last year and last month cannabis use. At the same time, the respondents of the non-response survey are going out more frequently than those of the main survey. This may partly explain their non-response, but seems at odds with their lower substance use. On the basis of these findings, one can hesitantly conclude that the prevalence figures of cannabis use based on the main survey are unlikely to underestimate cannabis use because of the non-response. However, due to the low response in the non-response survey there is no guarantee for the soundness of this conlusion.

Table 3.6 shows that there is a difference between those who were never reached by the interviewers, and those who had refused to co-operate. This comparison was only made for MM - the number of respondents in the CAPI non-response survey was too small to do the weighting needed for both subgroups. Those who never had contact with the researchers were going out more and used more cannabis than those who refused to participate. However, they used less alcohol than those who had refused to participate. Both types of non-response thus have a different influence on the findings of the main survey: those who had refused to take part in the survey did not influence the alcohol figures, but potentially inflated the cannabis figures and caused overestimating pleasure seeking behaviour. At the same time, those with whom no contact had been established caused a slight overestimation of alcohol use and underestimation of outgoing behaviour.

In a similar way one can compare the non-response of the CAPI sample to the non-response of the MM sample (table 3.7). This shows that in the CAPI study non-response led to a slight underestimation of alcohol use and last year and last month cannabis use, whereas in the MM study non-response led to a relatively strong overestimation of alcohol use and of lifetime cannabis use. Thus, drug and alcohol users seem to be less likely to participate in a CAPI survey, and more likely to participate when they are allowed to choose their own means of completing the questionnaire (MM). Both methods of research thus influence the findings in a different way.

On the basis of the prevalence rates found in the non-response survey and the response rates in the main survey, corrected alcohol and cannabis prevalence rates were calculated. This was first done for the CAPI and the MM samples separately. Within the MM sample the distinction between the categories 'no contact' and 'refused' was also included in the calculation, proportionally to the number of people who fell in each category. As was explained earlier, due to the small number of respondents, this distinction could not be made for the CAPI sample. Subsequently, corrected prevalence rates were calculated for the population, based on the corrected prevalence figures for each sample and the relative size of the CAPI and MM samples. The results of these calculations can be found in table 3.8. It shows the influences of the (different types of) non-response on alcohol and cannabis prevalence figures as described in the previous paragraphs. The alcohol prevalence rates found with the main survey might be 1.9 to 2.5 per cent point too high, whereas the lifetime prevalence rate found for cannabis might be 1.4 per cent point to high. For last year and last month use of cannabis, hardly any adjustments seem necessary. As the MM sample was much larger than the CAPI sample, the figures for the total sample closely resemble those for MM; for the same reason the adjustments as a result of the non-response analysis are in the same direction for the complete sample as they are for the MM sample.

A note of caution is in place. The above presented figures are only an indication of what might be the result of non-response on prevalence rates. It would be easy to take the 'corrected' figures as the 'correct' figures. However, as the response rate of the non-response survey was very low, this would assume a level of accurateness that cannot be achieved. Around three-quarters of the sampled non-respondents were not reached with the non-response survey, and there is hardly any knowledge available about this selective group. There is information on their gender, age, and marital status, and the response group has been weighted for any bias with regard to these variables. Yet, it is not unthinkable that there are other biases that are not strongly correlated to the known variables. The calculations show that, when reading the figures presented in the remainder of this report, one should keep in mind that the actual drug prevalence rates might be slightly lower. However, whether this is really the case, and if so, the extent to which this is so, cannot be established here.

3.5 Conclusions

The aim of conducting a non-response survey is to establish the extent to which the research sample is representative for the population. In this study the non-response survey served a second goal as well, namely to determine the impact of the survey method used on the type of non-response, and thus on outcomes. The benefit of non-response surveys is limited as those who did not participate in the first round, are unlikely to be either willing or reached a second time around. Although the non-response survey gives us some insight into validity of the main survey, at the same time one has to be very careful drawing conclusions.

It was found that only very few people decided against participation because of the nature or the quality of the survey. Some prevalence figures were found to be slightly lower for the response group of the main survey than for the response group of the non-response survey. As a consequence, if anything, prevalence figures based on the main survey might overestimate actual drug use. This finding only holds for MM. For CAPI it is found that non-response might lead to an underestimate of some prevalence figures. This is in line with the findings of the CAPI non-response survey in 1997. However, as the MM sample is much larger, it determines the directions of the overall influence of non-response found in this study to a much higher degree than the CAPI sample.