View the most recent version.

11. Historical estimates of population coverage error

11.1 Estimates

11.2 Coverage studies design changes

11.1 Estimates

This section presents historical estimates of population coverage error. Chart 11.1 presents the estimated population undercoverage rate  for the 1971 Census to the 2006 Census, and the estimated population overcoverage rate  and the estimated population net undercoverage rate  for the 1991 Census to the 2006 Census. The series for overcoverage and net undercoverage begin in 1991 because the overcoverage was first estimated for the 1991 Census following an experimental study done for the 1986 Census.

Population coverage error is a growing data quality concern; undercoverage has doubled since 1981 and overcoverage has doubled since 1996. Changes in net undercoverage from one census to the next reflect changes in undercoverage and/or overcoverage which, in turn reflect changes in population demographics, changes in the living arrangements of Canadians, changes in census methodology, and changes in the methodology of the coverage studies. The last is discussed in Section 11.2.

As seen in Chart 11.1, there is an increasing trend in both the population undercoverage rate and the overcoverage rate. First measured at 1.93% for the 1971 Census, the rates of undercoverage were similar for 1976 and 1981 at 2.04% and 2.01% respectively. Undercoverage jumped to 3.21% for the 1986 Census, increased to 3.43% for the 1991 Census, and then decreased by about the same amount to 3.18% for the 1996 Census. The rate of undercoverage increased notably to 3.95% for 2001 and then increased again to 4.26% for the 2006 Census.

The overcoverage rate increased from 0.74% for the 1996 Census to 0.96% for the 2001 Census. The increase from 1991 to 1996 is due to a change in the methodology of the coverage studies. The increase in overcoverage from 0.96% for 2001 to 1.59% for the 2006 Census is the largest increase in the series. From 2001 to 2006, the most significant increases were for the province of Newfoundland and Labrador and for the Northwest Territories (i.e., 1.00% and 0.98% respectively). We also noted very significant increases for the Yukon Territory and Nunavut (i.e., 0.76% and 0.85% respectively).

Although net undercoverage diminished slightly from 2001 to 2006, both undercoverage and overcoverage increased. Coverage error reflects error on the part of the respondent such as when the rules on whom to include are applied incorrectly, and on the part of census operations such as when new dwellings recently under construction are erroneously excluded. As for many surveys, reduced respondent participation continues to be an issue for the census. This is evidenced by both increased non-response and increased undercoverage.

It should also be noted that the methodology of the 2006 Census included a number of changes and, therefore, there is the potential for changes in undercoverage and overcoverage. Even though there are high quality standards governing all census operations, these changes may have resulted in changes in population coverage error. The 2006 Census of Population moved from a decentralized, manual operation to a more centralized and automated one:

• Questionnaires were mailed by Canada Post Corporation in a majority of urban areas.
• The Address Register, which is updated by listing operations, provided the mailing addresses.
• Follow-up became centralized.

Further, in some regions, it was difficult to recruit enough staff.

Looking back at undercoverage since the 1981 Census, the increase in undercoverage observed in the 1986 Census led to introducing the Address Register (AR) for the 1991 Census. The AR provided a separate list of those urban dwellings which should have been enumerated. For the 1996 Census, the introduction of enumeration by an enumerator (EN) rather than self‑enumeration in some large city inner-city enumeration areas (EAs) reduced undercoverage. Also, moving Census Day from early June to mid-May helped to control undercoverage because people were more likely to be at home and less likely to be moving.

Table 11.1 and Table 11.2 present estimates of undercoverage. Note that 1971 is not included in Table 11.2 because estimates were produced for different age groups for those above age 24.

These tables reveal that:

Undercoverage is usually higher in British Columbia. Among the provinces, British Columbia had the highest rate of undercoverage in every census from 1971 to 2001 except for 1991 and 2006. Ontario had the highest rate in 1991 at 4.23% and in 2006 at 5.18%. Undercoverage rates for Quebec, the Atlantic provinces and the Prairie provinces tend to be lower than the national rate.

Undercoverage is higher for young adults and higher for males. There are two persistent demographic trends. First, undercoverage for males is higher than undercoverage for females. Second, undercoverage is highest for young adults regardless of sex. As seen in Chart 11.2, undercoverage for males is higher than undercoverage for females for every census year since 1971, increasing from 2.27% to 5.51% for males and from 1.59% to 3.04% for females. Chart 11.2 also shows that undercoverage for young males aged 20 to 24 is higher than undercoverage for all males. This is also the case for females, but the young adult female rates are lower than the young adult male rates. The 2006 Census marked the highest rates of undercoverage for both young adult males and young adult females, 12.21% for males 20 to 24 and 8.71% for females 20 to 24. Higher undercoverage for young adults is due in part to less stable living arrangements. Young adults are more likely than older adults or children to change their living arrangements because they are, for example, moving away from home to attend a post-secondary institution or moving in with friends or spouses.

Table 11.3 and Table 11.4 present estimates of overcoverage.

These tables reveal that: 

Overcoverage is consistently higher for British Columbia than for the other provinces. Among the provinces, the rate of population overcoverage is highest for British Columbia. This has also been the case for the past three censuses.

Overcoverage is more common for school-aged children and young adults. There is a trend of higher overcoverage for children aged 5 to 17 and for young adults aged 18 to 24. For school‑aged children, it is largely due to children whose parents do not live in the same household who are often enumerated with each parent. Overcoverage for young adults likely reflects the same less stable living arrangements that can also lead to undercoverage.

Note that in Table 11.4 the age group 15 to 17 contains data for those 15 to 19 for the 1996 Census and the 2001 Census. The 2006 estimates revealed that persons aged 18 and 19 behaved more like young adults than like children in their propensity for undercoverage and overcoverage.

Provincially and nationally, differences in the total overcoverage estimate for 2001 and 2006 were all statistically significant at the 95% level, except for Saskatchewan. In comparing 2006 provincial and territorial estimates of overcoverage, there are some small methodological differences to be aware of. We should remember that overcoverage for Saskatchewan was high in 2001, due to the important impact of a Reverse Record Check (RRC) observation with a higher weight. We should also note that for Nunavut in 2001, only the Automated Match Study (AMS) and the Collective Dwelling Study (CDS) contributed to the overcoverage. We therefore expected to see an increase in the number of overcounted persons in Nunavut. In addition, the AMS covered four distinct regions in 2001: Eastern Canada (Newfoundland and Labrador, Prince Edward Island, Nova Scotia and New Brunswick); Quebec; Ontario; and Western and Northern Canada (Manitoba, Saskatchewan, Alberta, British Columbia, the Yukon Territory, the Northwest Territories and Nunavut). Accordingly, no overcoverage was measured between Nunavut (or any other territory) and the Eastern provinces, Quebec or Ontario. This may also explain the significant increases in Newfoundland and Labrador, the Northwest Territories and the Yukon Territory.

11.2 Coverage studies design changes

Differences in the design of the coverage studies over time mean that the rates in Table 11.1, Table 11.2, Table 11.3, and Table 11.4 are not strictly comparable. A list of methodological changes since the 1976 coverage studies is given below. It is remarkable that the fundamentals of the Reverse Record Check (RRC) approach for measuring undercoverage has not changed in any substantive manner since it was first carried out for the 1966 Census of Population. A sample is taken from a frame representing the census target population that is independent of the census. Census records are then checked ('Reverse Record Check') to determine if the sampled persons were indeed enumerated. There have been more changes in the measurement of overcoverage. Multiple studies were carried out for 1991, 1996 and 2001. In 1996, the RRC was expanded to include the measurement of overcoverage. For 2006 there was a new study to measure all overcoverage that exploited exact and probabilistic matching involving names, age and sex.

2006:

Both the RRC and the Census Overcoverage Study (COS) made optimal use of the name field added to the 2006 Census Response Database (RDB) in their matching and searching operations. Further:

• The measurement of overcoverage was restricted to the COS. The methodology of the RRC was subsequently changed so that not all cases were sent for field collection. New for the 2006 RRC, a processing step was carried out prior to collection in order to determine whether or not collection was required. A search of the RRC version of the 2006 Census response database (RRC RDB) for the persons selected in the sample using data from the sampling frame and the various update sources such as tax data was done. If the search resulted in locating the sampled person on the RRC RDB, collection was not required. The exception was a sample of those that had been found in order to collect data required for the non‑response adjustment.


• The three studies used for the 2001 coverage studies to measure overcoverage were replaced by the 2006 COS. The study used a methodology that was different from any previous overcoverage study. Essentially, the COS exploited the use of name matching to identify overcoverage.

As for 1996 and 2001, the 2006 RRC did not estimate the number of persons missed in incompletely enumerated Indian reserves and Indian settlements. You can find more information on this topic in Section 12.2.

2001:

• The institutional component of the Collective Dwelling Study (CDS) was dropped and overcoverage estimates in this population were produced by the RRC.


• The Dwelling Classification Study (DCS) replaced the Vacancy Check (VC) which was used in previous censuses to re-examine dwellings classified as unoccupied by the enumerator. The DCS is an extension of the VC in order to estimate the number of persons living in non‑response dwellings.

1996:

• The 1996 RRC did not estimate the persons missed on incompletely enumerated Indian reserves.


• The Temporary Residents Study was cancelled because of concerns about the quality of the data, and because it was recognized that the RRC would measure most of this type of undercoverage with sufficient quality.


• Compared to 1991, a more comprehensive measure of overcoverage was produced due to integrating the Private Dwelling Study into the RRC so that each sampled person could be identified as having been enumerated more than once. This approach resulted in an increase of addresses to be processed where overcoverage could have occurred. Second, the Automated Match Study (AMS) was substantially expanded from the 1991 approach of measuring overcoverage within an enumeration area (EA) to measuring overcoverage within a large region (Atlantic, Quebec, Ontario, rest of Canada).

1991:

• Non-permanent residents were included in the target population for the first time.


• Following experimental studies in 1986, the measurement of population overcoverage commenced with the 1991 coverage studies. The results of three studies were combined to form a comprehensive estimate: the Private Dwelling Study (PDS), the Collective Dwelling Study (CDS), and the Automated Match Study (AMS).

1986:  The rates in Table 11.1 for the 1986 Census differ from the results published in the User's Guide to the Quality of 1986 Census Data: Coverage as they include revisions made after the 1986 publication when incompletely enumerated Indian reserves were included as missed. In the original 1986 publication, they were included as 'enumerated' since published provincial census counts included an estimate of persons missed on such reserves.

1976:  Census counts did not include estimates from the vacancy check (VC) of persons missed in dwellings incorrectly classified as unoccupied. The 1976 population undercoverage rate would have been 1.78% had it included the results of the 1976 VC. There was no VC in the 1971 Census.

   Previous page | Table of contents | Next page