2. Methodology
Table of contents
2.1 Sampling and weighting
The 2008 Census Test was carried out in all provinces, with oversampling in Quebec. The sample was restricted to dwellings in mail-out areas that responded to the 2B questionnaire during the 2006 Census. The test sample excluded the territories, Indian reserves, military bases, collective dwellings, and citizens abroad or on ships. Dwellings that were unoccupied during the 2006 Census were excluded, as were dwellings solely occupied by temporary and foreign residents.
The content test examined variants of the 2A and 2B census questionnaires, also known as the short and long forms respectively, using a split-panel methodology consisting of six panels.
The six panels consisted of four panels for the general population and two panels for the Aboriginal population. The control questionnaire had the same1 content that was used in the 2006 Census and the alternate questionnaire contained the modified content.
The split-panel design permitted the analysis of the control and alternate panels, the short and long forms, and the content changes relating to Aboriginal identity.
The total sample size was 26,024 dwellings across the 10 provinces. The sample size for the general population was set at about 4,750 dwellings for each of the control and alternate questionnaire types, or around 19,000 dwellings in total. The sample size for the Aboriginal population was about 3,500 dwellings for each questionnaire type, or about 7,000 in total. Table 2.1 shows the distribution of the sample by panel for the population groups and questionnaire types.
Table 2.1
2008 Census Test sample
Description
Table 2.1 shows the distribution of the sample by panel for the population groups and questionnaire types.| Population group | Form type | Control panels | Alternate panels | Total |
|---|---|---|---|---|
| number | ||||
| General population | 2A | 4,767 | 4,739 | 9,506 |
| General population | 2B | 4,746 | 4,759 | 9,505 |
| Aboriginal population | 2B | 3,479 | 3,534 | 7,013 |
| Total sample | 12,992 | 13,032 | 26,024 | |
The samples for the panels were drawn from occupied dwellings in mail-out areas that responded to the 2B questionnaire in the 2006 Census. Only dwellings that included at least one household member that self-identified him or herself as a Métis or North American Indian in the Aboriginal questions were considered for the Aboriginal panels. A systematic stratified sampling method was used. The sampling frames were stratified by province and language of questionnaire returned in 2006.
Each response household and person was weighted2 to represent its targeted population group (i.e., the number of households and people in the general population or Aboriginal population). For each panel, the sum of the person weights equalled the total number of people in the targeted population group. Likewise, the sum of the household weight equals the number of dwellings in the targeted population group. The sample for the panels represented the populations and dwellings shown in Table 2.2.
Table 2.2
Number of all persons, persons 15 years and older, and households for each targeted population group
Description
The sample for the panels represented the populations and dwellings shown in Table 2.2.| Population group | Target population | |
|---|---|---|
| General population | Aboriginal population | |
| number | ||
| All persons | 21,328,245 | 678,940 |
| Persons aged 15 and over | 17,586,626 | 514,772 |
| Households | 8,592,707 | 242,584 |
Response to the census test was voluntary. Respondents had the option of completing the paper questionnaire that was mailed to them or completing the questionnaire online by entering the secure access code assigned to the household. A help line was made available for respondents to call for information throughout the collection period.
The control and alternate versions of the questionnaire were tested in both English and French.
2.2 Total response rates
The response rates are calculated based on the initial sample sizes shown in Table 2.1. For the 2A and 2B panels, a response household is a household whose returned questionnaire contained at least one response person. A response person for the 2A questionnaire (2A panels) has a valid age and sex, at minimum. A response person for the 2B questionnaire (2B and 3B panels) has a valid age and sex, plus at least two other answers among questions 4 to 10. For the 3B panels, a response household is a household whose returned questionnaire contained at least one response person that could be linked to the 2006 Census. It is defined this way to reflect the 2006 Aboriginal population in the targeted areas. The 3B panels are calculated from the final 3B files and are used as the real response rate.
Given the census test was voluntary, the non-response follow-up procedures were different from those used in the mandatory census. Since the response rates are relatively low, the results should be interpreted with caution.
Table 2.3 shows the number and rate in percentages of questionnaires completed by households for each panel. The response rates were highest for the 2A panels and lowest for the 3B panels. Table 2.4 shows the number and the rate of responses using the initial design weights of the sample.
Table 2.3
Response rate by panel (unweighted)
Description
Table 2.3 shows the number and rate in percentages of questionnaires completed by households for each panel.| Panel | Control panels | Alternate panels | ||
|---|---|---|---|---|
| number | percentage | number | percentage | |
| 2A | 2,478 | 52.0 | 2,416 | 51.0 |
| 2B | 2,181 | 46.0 | 2,194 | 46.1 |
| 3B | 1,166 | 33.5 | 1,184 | 33.5 |
Source: Statistics Canada, 2008 Census Test results.
Table 2.4
Response rate by panel (weighted)
Description
Table 2.4 shows the number and the rate of responses using the initial design weights of the sample.| Panel | Control panels | Alternate panels | ||
|---|---|---|---|---|
| number | percentage | number | percentage | |
| 2A | 4,451,526 | 51.6 | 4,304,792 | 50.6 |
| 2B | 3,863,709 | 45.1 | 3,890,374 | 44.8 |
| 3B | 78,865 | 32.5 | 80,731 | 33.1 |
Source: Statistics Canada, 2008 Census Test results.
Table 2.5 shows the responses by mode of response. The Internet response is slightly higher for the 2B than the 2A. The 3B have the lowest Internet response rate. The weighted responses in Table 2.6 were calculated using the post-stratification weights obtained after re-weighting only for response households.
Table 2.5
Response rate by mode of response (unweighted)
Description
Table 2.5 shows the responses by mode of response.| Panel | Response mode | Control panels | Alternate panels | ||
|---|---|---|---|---|---|
| number | percentage | number | percentage | ||
| 2A | Internet | 379 | 15.3 | 363 | 15.0 |
| Paper | 2,099 | 84.7 | 2,053 | 85.0 | |
| 2B | Internet | 384 | 17.6 | 390 | 17.8 |
| Paper | 1,797 | 82.4 | 1,804 | 82.2 | |
| 3B | Internet | 136 | 11.7 | 165 | 13.9 |
| Paper | 1,030 | 88.3 | 1,019 | 86.1 | |
Source: Statistics Canada, 2008 Census Test results.
Table 2.6
Response rate by mode of response (weighted)
Description
The weighted responses in Table 2.6 were calculated using the post-stratification weights obtained after re-weighting only for response households.| Panel | Response mode | Control panels | Alternate panels | ||
|---|---|---|---|---|---|
| number | percentage | number | percentage | ||
| 2A | Internet | 1,428,300 | 16.6 | 1,497,496 | 17.4 |
| Paper | 7,164,407 | 83.4 | 7,095,211 | 82.6 | |
| 2B | Internet | 1,825,078 | 21.2 | 1,721,956 | 20.0 |
| Paper | 6,767,630 | 78.8 | 6,870,751 | 80.0 | |
| 3B | Internet | 30,458 | 12.6 | 35,318 | 14.6 |
| Paper | 212,126 | 87.4 | 207,267 | 85.4 | |
Source: Statistics Canada, 2008 Census Test results.
In order to provide enough responses to evaluate the results of content changes to the French questionnaires, households that completed a French questionnaire in the 2006 Census were sampled at a rate of 35.0% instead of 22.3% for the general population; and at a rate 15.0% instead of the 8.1% for the Aboriginal population, as shown in Tables 2.7 and 2.8, respectively.
Table 2.7
Sample for 2A and 2B panels by language for general population
Description
In order to provide enough responses to evaluate the results of content changes to the French questionnaires, households that completed a French questionnaire in the 2006 Census were sampled at a rate of 35.0% instead of 22.3% for the general population.| General Population | Unweighted sample | Population | |
|---|---|---|---|
| number | percentage | percentage | |
| English | 12,355 | 65.0 | 77.7 |
| French | 6,656 | 35.0 | 22.3 |
| Total | 19,011 | 100.0 | 100.0 |
Table 2.8
Sample for 3B panels by language for Aboriginal population
Description
In order to provide enough responses to evaluate the results of content changes to the French questionnaires, households that completed a French questionnaire in the 2006 Census were sampled at a rate of 15.0% instead of the 8.1% for the Aboriginal population| Aboriginal population | Unweighted sample | Population | |
|---|---|---|---|
| number | percentage | percentage | |
| English | 5,961 | 85.0 | 91.9 |
| French | 1,052 | 15.0 | 8.1 |
| Total | 7,013 | 100.0 | 100.0 |
Unoccupied and out-of-scope dwellings accounted for about 0.5% of the dwellings in each panel. Out-of scope refers to dwellings that were solely occupied by persons that have a usual place of residence elsewhere or are not part of the target population (e.g., members of the Canadian Forces, foreign diplomats). These dwellings are excluded in the calculation of response rates.
2.3 Analysis strategy
The results from each of the 'control' and 'alternate' panels—for each pairing of form type or population group—were analysed to evaluate the effects of the proposed content changes. The analysis mainly focused on the impact of a change on the non-response rate and historical comparability. In addition, for respondents who lived in the same dwelling during the 2006 Census, the answers given in the 2008 Census Test were compared to the answers given in 2006 to understand changes in the data.
For new questions, the results from the alternate panel were compared to the results of the control panel to ensure questions located near them or other related questions were not affected. The test results were also compared to the results of other surveys that collected similar data, including the differences in the question or methodology to explain any inconsistencies.
For changes to existing content, the results from the control and alternate panels were compared, and the differences were assessed. For accuracy and consistency, the results were also compared to the results for same question in the 2006 Census.
The questions that did not change were also analyzed by comparing the results of the control and alternate panels, the results of the 2006 Census for accuracy and consistency and, if necessary, the data reported by individuals in the 2006 Census to explain change.
In some cases, conclusions could not be drawn from the 2008 Census Test and, as a result, further testing was required. Where appropriate, the analysis presented in Chapter 3 describes the results of subsequent qualitative testing and/or the results of the 2009 Census Test that informed the content recommendations for the 2011 Census.
- The 2B control questionnaire included some exceptions. These are noted in the subject matter analysis, where applicable.
- The weights presented in Table 2.2 are not the initial design weights of the sample. The rates were adjusted for non-response.
- Date modified:
