View the most recent version.

3. Data Quality Measurement

3.1 General

Throughout the census-taking process, every effort was made to ensure high-quality results. Rigorous quality standards were set for data collection and processing, and the Public Communications Program assisted in minimizing non-response. A Data Quality Measurement Program was established to provide users with information on the quality and limitations of census data.

Although considerable effort is made throughout the entire process to ensure high standards of data quality, the resulting data are subject to a certain degree of inaccuracy. To assess the usefulness of census data for their purposes and to understand the risk involved in drawing conclusions or making decisions on the basis of these data, users should be aware of their inaccuracies and appreciate their origin and composition.

Within the 2001 Census Technical Reports Series, users will find detailed 2001 Census information on Coverage and Sampling and Weighting. These two reports are scheduled to be released in November and December 2004 respectively.

3.2 Dwelling, Household and Shelter Cost Data

In general, the evaluation of dwelling, household and shelter cost data consisted of the following:

The results of data evaluation are summarized below.

Tenure – The total imputation rate was 3.4% (2.2% in 1996). At the provincial/territorial level, the non-response rate was very high in Yukon (11.9% in 2001, compared with 8.9% in 1996) and the Northwest Territories (9.8% in 2001, 2.8% in 1996), followed by Nunavut (4.5%). The reason for the high imputation rate in Yukon and the Northwest Territories in 2001 was due primarily to non-response. The causes for non-responses were not known. The rates for Quebec, Ontario and British Columbia ranged from 3.5% to 3.8%.

There were 9,442 records with invalid (i.e. multiple) responses. This translates into 0.4% of the total number of records processed.

A total of 86,182 records (non-responses + invalid responses) required imputation, yielding a total imputation rate of 3.8%. This rate was not considered excessively high, and the impact of imputation was not expected to affect the final data in a significant manner.

By and large, the 2001 data for tenure were comparable to the 1996 data. The increase in owned dwellings reflected the trend in residential housing market towards home ownership, particularly among persons living alone. This change was a continuation of a secular trend, already observed in 1996. The increase in Band housing may be attributed in part to the improved coverage of households on reserves, where the population has also increased.

The 2001 Census counts were compared with the counts from the 2000 Survey of Household Spending (SHS). There were differences between the 2001 Census data and the SHS estimates. In absolute terms, the census count of owned dwellings fell within two coefficients of variation (CVs) of the SHS estimate at the Canada level. However, the SHS reported more rented dwellings than the census. In terms of percentage distribution of dwellings by tenure, the two sources reported similar proportions.

At the provincial level, census data were comparable with SHS estimates for most provinces except Newfoundland and Labrador, Quebec and Ontario, where the census counts of rented dwellings were slightly lower than the SHS counts.

Period of Construction – The total imputation rate for this variable was 5.2%. Since knowledge of the period of construction among respondents was not expected to be universal, particularly among renter households, this non-response rate was not considered excessively high. In all, 118,936 records (unweighted count) required imputation. Edit and imputation did not alter the data distribution in any significant manner.

At the Canada and provincial levels, the 2001 data were generally comparable with the 1996 data. Some minor numerical differences could be observed for dwellings built between 1981 and 1985, but the differences could have resulted from demolitions, or a certain degree of response error (either in the 1991 Census, or in the 1996 Census, or in both censuses), or sampling variance. All in all, the magnitude of the differences did not indicate that there was a serious problem with data quality. Comparison with SHS data showed some differences between the census and SHS estimates for "1971-1980". It was not possible to say which survey provided a more accurate estimate, as both were subject to the same response error from the responding households.

Rooms and Bedrooms – The total imputation rate for Rooms at the Canada level was 5.8%. About 131,866 records out of 2.29 million required imputation. The imputation rate was high in Ontario and Alberta (both at about 6.2%), British Columbia (8.9%), Yukon (16.3%) and Northwest Territories (12%). The total imputation rate was a combination of non-responses and invalid data resulting from inter-variable (Rooms and Bedrooms) edits. Non-responses for Rooms as well as for Bedrooms could not be attributed to respondents' lack of knowledge, but may have resulted from an unwillingness to provide the information.

Data processing did not result in any distortion of data for rooms. Historical comparison with data from the two previous censuses indicated that the counts of one- and two-room dwellings in 2001 may be slightly underestimated. The census estimates were also slightly lower than the SHS estimates for dwellings with one to four rooms, but higher for dwellings with seven or more rooms.

The total imputation rate for Bedroom data was 5.7% for Canada. A total of 130,198 records required imputation. Provincially, Ontario, British Columbia, the Yukon and Northwest Territories had higher imputation rates (6.2%, 8%, 16.3% and 11.4% respectively) than elsewhere in Canada.

The most noticeable change from the 1996 Census was the decrease in dwellings with no bedrooms. Given the overestimation of these dwellings in 1996, and viewed in the longer-term context, the 2001 count of zero-bedroom dwellings was reasonable. There were some absolute and proportionate differences between the census and SHS estimates of dwellings with 0-1 bedrooms and 3+ bedrooms.

Condition of Dwelling – No data processing errors were detected, although 2.2% of the records (a total of 51,432 records) required imputation.

The 2001 and 1996 percentage distributions of dwellings according to the condition of dwelling were practically identical at the Canada level, as well as at the provincial/territorial levels of geography.

As in previous censuses, the data distribution in 2001 differed somewhat from the distribution from the 2000 Survey of Household Spending. Census reported fewer, absolutely and proportionately, dwellings requiring regular maintenance than SHS. The reverse was true of dwellings requiring minor repairs. A high degree of comparability could be seen for the major repairs response category.

Similar results were obtained in the 1991 and 1996 Censuses; viz., that SHS reported a lower count of minor repairs than census, and a higher estimate for regular maintenance. This may be due to the reverse order of listing for the three response categories in the census and SHS.

Structural Type of Dwelling – In pre-census consultations, some users indicated the need for new categories of structural type of dwelling. In response, Statistics Canada evaluated the feasibility of coding two new types: "Apartment without direct ground access in a building that has fewer than five storeys" and "Apartment with direct ground access in a building that has fewer than five storeys". Based upon the results of the pre-census evaluation, Statistics Canada decided to code these two types in the 2001 Census. These two new categories were in fact the subsets of the category "Apartment in a building that has fewer than five storeys" in previous censuses.

Data for this variable were coded by trained census representatives in the field. One point seven percent (1.7%) of the records were non-responses and required imputation. The comparison of edited and unedited data indicated that imputation did not alter the overall percentage distribution of structural type.

At the Canada level, three types of dwellings decreased between 2001 and 1996: other movable dwelling (-891 or -13.5%), apartment/flat in a detached duplex (-31,370 or -7.03%) and mobile homes (-1,919 or -1.25%). All other types showed an increase. With the possible exception of apartment/flat in a detached duplex, the absolute changes in the other categories seem reasonable.

However, a significant difference was found when the 2001 Census counts for "multiple dwellings" (comprising row houses, semi-attached, apartment/flat in a detached duplex, apartment in a building that has fewer than five storeys, apartments in a building that has five or more storeys) were compared with estimates based on the addition of flow data (i.e. housing starts and completions less demolitions) to the 1996 Census data. The reasons for this discrepancy remain under investigation.

The comparison with data from the Survey of Household Spending (SHS) indicated that the data from the two sources were by and large comparable. The only exception was the "Other" category (SHS classification, referring mainly to mobile homes and other movable dwellings), where the SHS estimate was considerably higher than the census count. Note that, for this category, SHS estimates were not available in some provinces and the territories because of small cell size. Among the provinces for which SHS estimates were shown, Alberta was the only area where the census count fell outside the lower limit of the SHS estimates.

Evaluation of the new type "Apartment with direct ground access" consisted of field observation of selected areas in some major urban centres. This turned out to be the only means available for evaluation. No other survey in Statistics Canada had the same dwelling type, and only very limited administrative data existed. The results of the field evaluation showed a significant degree of misclassification for "Apartment with ground access in a building that has fewer than five storeys".

The data anomaly with the two new categories is described below (see Subsection 3.3.2). However, if the counts of these two new categories are aggregated, then the data are comparable with the 1996 count for "Apartment in a building with fewer than five storeys". As a result, only the aggregated counts for "Apartment – less than five storeys" are released in 2001.

Household Maintainer – At the Canada level, only 36,982 out of 2.29 million private households (both unweighted counts) did not respond to Question H1. This translates into a non-response rate of 1.6% (compared with 1.3% in 1996).

As a result of subsequent edits (i.e. re-ordering persons in the household as part of relationship edit), a total of 38,170 records (1.7% of all households processed) required imputation. All except 11 records were imputed from perfectly matched donor households. The 11 records with no perfectly matched donors resulted in default imputation.

Data evaluation also involved the examination of the primary household maintainer by age group and sex for both 1996 and 2001. The results of the 2001-1996 comparison indicated that:

In terms of the percentage distribution of private households by household size showing the number of household maintainers, it was found that the proportion of households in Canada with only one maintainer decreased very slightly to 64.7% in 2001 from 65.8% in 1996, while the share of two-maintainer households increased slightly.

Among larger households (four persons or more), the proportion with only one maintainer decreased, while the proportion with two or three + maintainers increased.

However, the overall percentage distribution of households by the number of household maintainers in 2001 was still comparable with that of 1996.

With respect to the number of household maintainers, the slight shift in the proportion of one-maintainer households vis-à-vis two-maintainer households was quite evident. This was true of all types of households, except one-person households. It is to be emphasized that, in spite of the slight shift, the proportionate distribution of households by number of maintainers in 2001 was still comparable to the 1996 distribution for each type of household.

As regards the percentage distribution of household maintainers by relationship to Person 1, the 2001 distribution was almost identical to the percentage distribution in 1996 and 1991. The only difference at the Canada level was the very slight increase (0.13% points) in son/daughter of Person 1 as maintainers. This difference was not significant, and may be attributed to the inclusion of same-sex partner's son/daughter in 2001. These were minor changes that did not detract from the general conclusion that the 2001 percentage distribution of maintainers by relationship to Person 1 was virtually identical to the 1996 distribution.

These evaluation results indicated that, overall, the 2001 data are comparable to the 1996 data in terms of age group, sex and relationship to Person 1, as well as the number of household maintainers by household size and household type.

The total imputation rates for different shelter cost variables by tenure were as follows. The results of data evaluation showed that the data are of acceptable quality.

The comparisons of unedited with edited data for the above variables indicated that, by and large, imputation of non-responses and invalid responses conformed to the overall distribution of data, and did not result in any distortion of the distribution.

Of particular concern was the imputation of high values for each of the components. The evaluation showed that some imputations were in the order of $2,000 and over. However, as aforementioned, there was no evidence of a bias or over-representation of imputation of such values.

For each component of shelter cost, as well as for the derived variables Owner's Major Payments and Gross Rent, the 2001 data were compared with the 1996 data. In general, the data distributions from the two censuses were quite similar for all of the components of shelter cost, although more households reported slightly higher payments. The comparison for each component is summarized as follows:

Electricity – The 2001 data distribution was very similar to the 1996 distribution. The proportion of owners spending $1,000+ increased in Saskatchewan (from 35% to 44%) and Alberta (31% to 49%). Among renters, there has been very little change, except in Alberta, where the percentage of households spending $1,000+ increased to 18% from 11% in 1996.

For 84% of owners, the cost of electricity represented less than 30% of the owner's major payments in 2001. Among renters, about 92% of the households spent less than 30% of their gross rent on electricity.

Oil, Gas, Coal, Wood and Other Fuels – There has been a significant increase in owner households spending $1,000+ for fuel between 1996 and 2001, and a general decrease in the lower cost categories. About 45% of owners in Canada reported spending $1,000 (annual payment) in fuel, compared with 24% of renters.

Among renters, the increase in the $1,000+ category was far less dramatic (from 192,300 or 5% of all renters in 1996 to 347,110 or 9% of renters in 2001). The number and proportion of renters reporting fuel expenditures (responses for the category "Included in rent or other payments"), or who marked the answer circle "None", remained roughly the same as in 1996.

For close to 90% of owners and 99% of the renters, the cost of fuel as a percentage of total shelter cost in 2001 was less than 30%.

The increase in the number of owners reporting $1,000+ fuel cost may reflect the rising cost of energy and possibly increased consumption.

Water and Other Municipal Services – Among renters, the number and proportion of households reporting none or included in other payments for this shelter cost component in 2001 were almost the same as in 1996. Only 9.3% of renters reported some cost for this component. For those reporting renters, the modal annual payment was $200-$399.

Among owners, 59% reported annual payment for this item. The distribution of households by the amount of payment in 2001 was very similar to that in 1996. The majority of households with payment spent between $200 and $599.

Cash Rent – The 2001 and 1996 distributions were similar. There were more households paying rent between $800 and $1,999 in 2001, and fewer households paying less than $500 per month. The most significant increases in higher rents occurred in Ontario, where a distinct upward shift in rents occurred. The majority of tenants (about 70%) paid between $400 and $1,000 in rent. This modal amount of rent fell within expectations.

Mortgage – The 1996 and 2001 distributions of owners by monthly mortgage payments were similar, with the exception of the $1,000+ category. The absolute increase in this category was close to 300,000 households. As a share of all owners, those spending $1,000+ increased from 15% in 1996 to 18% in 2001.

The increase in owners spending $1,000+ was most noticeable in Ontario. The increase of about 150,000 Ontarian households in this expenditure category accounted for half of the total increase in Canada. Alberta and British Columbia also witnessed increases of about 50,000 households each.

Property Taxes – Compared with the 1996 data, absolute and proportionate increases in the $1,000+ category have been reported in all provinces and territories. The increases were more pronounced in Ontario (253,060 or 13.6%), Quebec (148,190 or 13.6%) and Alberta (121,305 or 31.5%).

The increase in property taxes in many provinces may be related, at least in part, to the general price increase in the value of housing.

Condominium Fees – The historical comparison revealed that condominium fees for the 668,815 owners in 2001 were comparable to the fees reported for 1996. As in 1996, the vast majority of owners paid less than $400 for condominium fees.

Owner's Major Payments (derived) – No data problems were detected.

The overall distribution of owners by owner's major payments for 2001 was similar to that for 1996, with the exception of the $1,000 and over category. While the number of owner households increased by 11.1% between 1996 and 2001, those spending $1,000 and over monthly increased by 31.8%. However, the average payment among these high-cost owners was $1,524, an amount that is comparable to the 1996 average of $1,494. This average amount was not considered excessive relative to 2001 market prices.

In all provinces and territories, there were fewer households reporting lower costs (up to $299 per month), and more households reporting higher costs. Between 1996 and 2001, the biggest absolute increase in households spending $1,000 or more was in Ontario (about 280,000 households). For these Ontarian households, the average payment in 2001 was $1,566, compared with the $1,539 reported for 1996.

Overall, about 67% of owners in Canada spent less than $1,000 per month on shelter, while some 33% spent $1,000 and over.

Gross Rent (derived) – No data problems were detected. The intercensal comparison of the distributions of household by gross rent showed the same results as for cash rent. This is not surprising, as cash rent constitutes the major component of shelter cost for the vast majority of tenants.

The average shelter costs from the census were comparable to the averages obtained from the 2001 SHS.

Value of Dwelling – The total imputation rate for this variable was 12.8%. A total of 187,276 records required imputation. Comparison of edited and unedited data showed that the overall data distribution was not changed by the imputation for non-responses and invalid responses. At the high end of the spectrum ($300,000+), imputed data represented 10% of the final unweighted data. In other words, 90% of the high values on the final database were respondent-provided data. The highest concentration of these imputed records was in Ontario and British Columbia.

The comparison of 1996 and 2001 data for value of dwelling corroborated the general upward trend in housing prices. However, the intercensal changes varied from province to province and from census metropolitan area (CMA) to CMA.

In terms of intercensal percentage change, increases in the average and median values of dwelling were more pronounced in Alberta and Saskatchewan. At the provincial level, British Columbia was the only province where respondents reported slightly lower expectations with respect to the sale value of their homes. It should be mentioned that, according to Statistics Canada's New Housing Price Index, there has been a decrease of close to 13% for British Columbia between May 1996 and May 2001.

Among CMAs, Saskatoon and Calgary had the highest percentage increases (about 33% in both cases) in the median value of dwelling, followed by Toronto and Regina (about 22% in both cases). The comparison of New Housing Price Index for these CMAs shows similar increases, except in Saskatoon where the price index showed only a 12% increase between May 1996 and May 2001.

Overall, there have been increases in most urban areas, as reported in the media. Statistics Canada's New Housing Price Index for the period between the 1996 and 2001 Censuses also showed general increases, although the magnitude of change may differ somewhat from the changes in census in different CMAs.

Average house prices for multiple-listing sales from the Canadian Real Estate Association (CREA) was another source of data used for data evaluation. A comparison of CREA and census data should take into account some differences inherent in the two data sets.

It should be noted that the CREA averages were based on transaction values in the real estate market. The transaction values pertained almost entirely to the resale prices of homes and rarely included the price of new homes that were typically sold directly by the developers to the consumers.

The census data, in contrast, were based on the expected sale value of the dwelling for all owner-occupied dwellings, both old and new stock, and irrespective of any actual market transaction. Even among the old stock, not every dwelling was in the market, and the values of these dwellings were not reflected in the CREA data.

Then, too, there may be differences between expected value and actual transaction value under normal market conditions. In exceptional circumstances (for example, an "over-heated" real estate market where demand far exceeds supply), however, the difference between expected (roughly the asking price) and the transaction price may be minimal.

The CREA averages may also be affected by the relative weight of the sales. If, for any particular urban centre, more high-priced dwellings were resold than medium-priced or lower-priced dwellings, the average CREA price would be higher than the average of all dwellings. Of course, the converse would also be true. The census averages, on the other hand, were based on the full range of the price spectrum.

Finally, the CREA geography referred to their sales regions, and not the census geographic delineation.

Notwithstanding the above-noted differences, the 2001 Census data compared very well with the CREA sales averages in 2001. There were differences in some CMAs between the CREA and census averages, but the differences were by no means excessive. Even in CMAs where the averages differed, the CREA averages were still very comparable to the census median values. It is safe to conclude that, given the above-mentioned factors that may affect the comparability of data, the 2001 Census average and median values of dwellings were still representative of the market prices in most major urban centres.

3.3 Sources of Errors and Evaluation Studies

For 2001, the count of total dwellings in some areas is substantially higher than reported for the 1996 Census of Canada. The increase in the total number of dwellings between 1996 and 2001 is directly linked to our efforts to improve the coverage of seasonal dwellings. Based on our consultation process for 2001 and the requirements to simplify collection procedures and improve overall coverage of dwellings, the 2001 Census private dwelling definition was modified slightly from previous censuses to eliminate one criterion – access to a source of drinking water throughout the year. The result was that more private dwellings were counted in the 2001 Census – specifically, more seasonal dwellings (secondary residences such as cottages, cabins and/or chalets) that now meet the private dwelling definition. Care should be exercised in comparing the 2001 counts of total dwellings (including both occupied and unoccupied) with dwelling counts from the 1996 and earlier population and dwelling count release.

3.3.2 Structural Type of Dwelling

In response to user demand for more detailed classifications for structural type of dwelling, the 2001 Census collected data for two new categories for structural type of dwelling:

Apartment with direct ground access in a building that has fewer than five storeys

and

Apartment without direct ground access in a building that has fewer than five storeys.

Postcensal data evaluation has revealed a serious misclassification problem with these dwellings. As a result, the data will not be released.

The problem seems to be the census representatives' interpretation of "with/without ground access" when they coded the dwellings. By and large, these misclassified dwellings were units in apartments in buildings with fewer than five storeys. Data for "Apartment in a building that has fewer than five storeys" have been released in 2001 products. This category is an aggregate of the two new previously mentioned categories, and is directly comparable with the same category from previous censuses. It presents no data problems.