Education Reference Guide, National Household Survey, 2011

Definitions and concepts

The 2011 National Household Survey (NHS) collects information on the Canadian population in private households. It collects information on the education characteristics of the population aged 15 years and over.

Education information is collected in the NHS because education plays a major role in Canadian society. Education has an impact on the quality of life of Canadians (in areas such as income, work, health and participation in the community) and the knowledge and skills of the Canadian labour force have an impact on Canada's overall economic performance.

The 2011 NHS measured four main concepts through its education questions:

  • completed education credentials: certificates, diplomas and degrees (Questions 27 to 30)
  • major field of study (Question 31)
  • location of study (Question 32)
  • attendance at school (Question 33).

Within these concepts, data for several analytical variables have been released:

For definitions of these variables, see the National Household Survey Dictionary, Catalogue no. 99‑000‑X.

Respondents were asked to respond to the education questions according to their status on or up to May 10, 2011, even if they responded to the questionnaire after this date. This is the same reference date used for the 2011 Census.

Early enumeration – collection of the 2011 Census and NHS questionnaires prior to May 10, 2011 – was conducted in remote, isolated parts of the provinces and territories in February, March and April 2011. When enumeration took place before May 10, 2011, the reference date used was the date on which the household was enumerated.

For general information on the overall content, collection, design, processing and data quality for the NHS, please refer to the National Household Survey User Guide, Catalogue no. 99‑001‑X.

Questions

The 2011 National Household Survey Form N1 questionnaire contains seven questions related to education that were asked only for the population aged 15 years and over.

Questions 27 to 30 collected information on all certificates, diplomas and/or degrees that the respondent had completed from secondary (high) school (Question 27); apprenticeship and other trades programs (Question 28); college, CEGEP, or other non-university institutions (Question 29); or university (Question 30).

Question 31 collected information on the major field of study of the highest certificate, diploma or degree above the secondary (high) school level.

Question 32 asked about the province, territory or country where the highest certificate, diploma or degree was completed (location of study) for persons who had completed a certificate, diploma or degree above the high school level.

Question 33 collected information on whether the person attended school in the previous nine months; that is at any time between September 2010 and May 10, 2011. It also collected information on the type of school that was attended (attendance at school).

The 2011 National Household Survey Form N2 questionnaire was used to collect information from persons living in private households on Indian reserves, Indian settlements and in remote areas. The N2 Education questions were the same; however, some of the examples were adapted to reflect education programs that were more common in those areas.

For more information on all the questions included in the 2011 NHS questionnaire, please refer to Reasons why the NHS questions are asked.

For more information on the help information for the NHS questionnaire, please see National Household Survey (NHS) guide: short version and National Household Survey (NHS) guide: long version.

Classifications

Classification of major field of study write-in responses

In the 2011 NHS, the major field of study of the highest postsecondary certificate, diploma or degree of the respondent was classified using the newly updated Classification of Instructional Programs (CIP) Canada 2011. Write-in responses for major field of study were assigned a six-digit code at the most detailed level. Persons without a certificate, diploma or degree above the secondary (high) school level are included in the category 'no postsecondary certificate, diploma or degree.'

Programs of study at the six-digit level can be grouped into sub-series (four-digit codes) and series (two-digit codes). Series 21, 32 to 37 and 53 and their sub-components are not used in the NHS. There is also a variant of CIP Canada 2011 that consists of primary groupings.

The NHS major field of study responses were also classified using the older CIP Canada 2000 classification to allow comparisons with other sources of data that use that classification.

Data users are recommended not to make comparisons between individual categories of the two classification systems on the basis of their labels. Even though many entries in the two classifications are similar, direct comparison could be inappropriate, given the numerous changes made at the detailed level to update the classification.

For more information on the CIP classifications, see the Classification of Instructional Programs (CIP) Canada 2011 (PDF, 3,897 kb), Catalogue no. 12‑590‑X and the Classification of Instructional Programs, Canada 2000 (PDF, 2,457 kb), Catalogue no. 12‑590‑XPE available from: www.statcan.gc.ca/concepts/classification-eng.htm.

Coding of the written field of study responses is done using a combination of automated and manual coding. Both processes use a reference file that contains the most common responses from the 2006 and past censuses and the associated CIP Canada 2011 and CIP Canada 2000 6‑digit codes. When the reference file was updated to add the CIP Canada 2011 codes, improvements in interpretation were made, while the CIP Canada 2000 code on the file remained the same in order to respect the decisions taken for the 2006 Census as closely as possible.

There are a number of cases in the reference file in which a single write-in has CIP Canada 2011 and CIP Canada 2000 codes that are conceptually different, resulting in dissimilar CIP 2011 and CIP 2000 variables for a small number of records. The main reasons for the differences are: (a) differing interpretations under the CIP Canada 2011 and CIP Canada 2000 coding processes of insufficiently specific write-ins or write-ins involving multiple fields of study, (b) the use of some data processing routines for the CIP 2011 and CIP 2000 field of study variables that operate independently, and (c) an acceptable level of coding error.

Two significant examples of differences in coding practices are: (a) written responses including text such as 'doctor of philosophy' or 'doctorate in philosophy' with no other field of study information were coded to '38.0101 – Philosophy' for the CIP 2000 field of study variable, but for the CIP 2011 variable they were processed as if the written response was missing; (b) written responses including both 'education' and 'psychology' in the text were coded to '42.1801 – Educational Psychology' for the CIP 2000 variable, but coded to '13.1335 – Psychology Teacher Education' under the CIP 2011 variable rules.

Note that major field of study responses were classified at the 6-digit level only to the degree that the respondents provided precision in their responses. For example, it is likely that some people who studied civil engineering provided a response of only 'engineering' to the major field of study question. This means that frequencies in very specific categories are likely to be underestimated as some of the responses are likely found in the frequently-reported 'general' categories (e.g., '14.0101 Engineering, general'). Some general categories also include those people who took general programs without further specialization.

Classification of location of study responses

The location of study of the institution from which the person received his/her highest certificate, diploma or degree above the secondary (high) school level was reported as a province or a territory (inside Canada) or a country (outside Canada). Countries outside Canada were classified according to the Standard Classification of Countries and Areas of Interest (SCCAI) 2010. A variant of this classification allows the countries to be aggregated into geographical sub-regions and geographical macro-regions. For countries outside Canada which had a very small number of responses, data was aggregated at the macro-region level into a 'not included elsewhere (n.i.e.)' category. For more information on the SCCAI 2010 refer to: www.statcan.gc.ca/subjects-sujets/standard-norme/sccai-ctpzi/2010/sccai-ctpzi-eng.htm.

Persons without a certificate, diploma or degree above the secondary (high) school level were not asked to provide a location of study.

Data and other products

Data for the 2011 National Household Survey education variables were released on June 26, 2013 as part of an integrated release with Labour, Place of work, Commuting to work, Mobility and migration and Language of work variables.

The products published using 2011 NHS education data include:

For more information on and access to 2011 NHS data, please refer to the Census Program website.

Data quality

The National Household Survey (NHS) underwent a thorough data quality assessment similar to what was done for the 2011 Census of Population and past censuses. It consisted of an assessment of various data quality indicators (such as response rate), and an evaluation of the overall results, in comparison with other data sources such as Census of Population data.

Based on the results of this exercise, the NHS estimates for the education variables are generally consistent with, or similar to, estimates and trends from other data sources used for comparison at the national, provincial and territorial levels.

Survey process

Quality indicators were calculated and assessed at each of the key steps of the survey. During the collection and processing of the data, the quality and consistency of the responses provided were assessed as were the non-response rates. The quality of the imputed responses was assessed after the completion of the control and imputation steps.

Certification of final estimates

Once data processing and imputation were completed, the data were weighted to represent the total Canadian population. These weighted data (the final estimates) were then certified to determine if they were coherent and reliable in comparison to other independent data sources. This is the final stage of data validation. The main highlights of this assessment are presented below.

Non-response bias

Non-response bias is a potential source of error for all surveys, including the NHS. This issue arises when the characteristics of those who choose to participate in a survey are different than those who refuse. Statistics Canada adapted its collection and estimation procedures in order to mitigate, to the extent possible, the effect of non-response bias. (For more details please refer to the National Household Survey User Guide, Catalogue no. 99‑001‑X.)

It is impossible to definitively determine how much the NHS may be affected by non-response bias. However, based on information from other data sources, evidence of non-response bias does exist for certain populations and for certain geographic areas.

Generally, the risk of error increases for lower levels of geography and for smaller populations. At the same time, the data sources used to evaluate these results are also less reliable making it difficult to certify these smaller estimates.

For more information on NHS non-response bias and mitigation strategies employed by Statistics Canada, please refer to the National Household Survey User Guide, Catalogue no. 99‑001‑X.

Data quality indicators

Of all the quality indicators used for the evaluation, two are presented: the global non-response rate and the imputation rate by question.

  • The global non-response rate combines the non-response at the household level and the non-response at the question level. It is provided for geographic areas. The global non-response rate is the key criterion that determines whether or not the NHS results will be released for a given geographic area. Information on the global non-response rate is available in the National Household Survey User Guide, Catalogue no. 99‑001‑X.
  • The imputation rate is the proportion of respondents who did not answer a given question or whose response is deemed invalid and for which a value was imputed. Imputation improves data quality by reducing the gaps caused by non-response.
Table 1
Imputation rates in the National Household Survey, Canada, provinces and territories

Table summary
This table displays the results of Imputation rates in the National Household Survey, Canada, provinces and territories, calculated using Question, Q. 27 Secondary (high) school diploma or equivalent, Q. 28 Registered Apprenticeship or other trades certificate or diploma, and Q. 29 College, CEGEP or other non-university certificate or diploma (appearing as column headers) Percentage units of measure.
Question Q. 27 Secondary (high) school diploma or equivalent Q. 28 Registered Apprenticeship or other trades certificate or diploma Q. 29 College, CEGEP or other non-university certificate or diploma
percentage
Canada 4.6 5.5 5.5
Newfoundland and Labrador 4.1 4.9 4.9
Prince Edward Island 4.3 5.0 5.1
Nova Scotia 3.7 4.8 4.7
New Brunswick 3.7 4.5 4.3
Quebec 4.0 4.9 4.7
Ontario 5.0 5.9 6.0
Manitoba 4.2 4.9 4.9
Saskatchewan 4.2 4.8 4.9
Alberta 4.8 5.5 5.7
British Columbia 4.9 5.7 5.9
Yukon 5.0 5.4 5.7
Northwest Territories 1.2 2.1 1.8
Nunavut 2.1 2.5 2.4
Table 1
Imputation rates in the National Household Survey, Canada, provinces and territories (Continued)

Table summary
This table displays the results of Imputation rates in the National Household Survey, Canada, provinces and territories, calculated using Question, Q. 30 University certificate, diploma or degree, Q. 31 Major field of study, Q. 32 Location of study and Q. 33 Attendance at school (appearing as column headers) Percentage units of measure.
Question Q. 30 University certificate, diploma or degree Q. 31 Major field of study Q. 32 Location of study Q. 33 Attendance at school
percentage
Canada 4.7 14.2 12.1 6.1
Newfoundland and Labrador 4.1 11.3 10.4 6.1
Prince Edward Island 4.1 12.5 10.6 5.7
Nova Scotia 3.8 11.5 10.3 5.4
New Brunswick 3.8 11.6 10.0 5.0
Quebec 4.1 15.0 12.2 5.3
Ontario 5.2 14.3 12.3 6.7
Manitoba 4.2 12.6 11.1 5.5
Saskatchewan 4.1 11.6 10.6 5.4
Alberta 4.9 13.4 11.6 6.2
British Columbia 5.1 15.2 13.3 6.4
Yukon 4.9 13.6 12.0 5.7
Northwest Territories 1.1 4.7 5.0 2.1
Nunavut 1.3 9.1 7.9 2.9

Questions 31 (Major field of study) and 32 (Location of study) have higher rates of imputation than the other questions. They were open-ended, written response questions which generally have higher rates of non-response than multiple-choice questions. For questions 31 and 32, imputation was also used in certain cases when written responses were not detailed enough to assign a valid code within the classification system. There was also an instruction in Question 31 for those without a certificate, diploma or degree above the high school level, to skip Question 32 and go directly to Question 33. Some respondents may have skipped Question 32, even when they should have answered it.

Lower rates of imputation for the Northwest Territories and Nunavut reflect lower item non-response due to the greater use of interviewer-led collection.

The imputation rates for the 2011 NHS education questions are around 3 percentage points higher than the corresponding imputation rates for the 2006 Census at the national level.

Coding processes for major field of study and location of study

The written responses for major field of study were coded with the new Classification of Instructional Programs (CIP), Canada 2011. Most of the write-in responses (80.9%) were coded automatically from a reference file that contained the most common responses in 2006 and past censuses and the associated CIP Canada 2011 codes. The remaining write-ins (19.1%) were processed by subject-matter coders who assigned a CIP Canada 2011 code by associating each write-in with its nearest match from the reference file. A study was carried out to measure coding quality. A sample of some 3,000 entries was recoded using the CIP Canada 2011, and the overall accuracy rate was found to be 98.5%.

For the question on location of study, 97.2% of the write-in responses were coded automatically from a reference file. The remaining write-ins (2.8%) were processed by subject-matter coders.

Certification

Several data sources were used to evaluate the NHS estimates for education variables such as: the 2006 and 2001 censuses, the Labour Force Survey and population projections based on microsimulation.

Table 2 shows a sample comparison between data from the May 2006 Labour Force Survey (LFS), the 2006 Census, the May 2011 LFS and the 2011 NHS for the variable 'highest certificate, diploma or degree.' These kinds of comparisons are useful for assessing data quality by comparing distributions and trends.

Table 2
Comparing 'Highest certificate, diploma or degree' with other data sources, Canada excluding territories, 2006 and 2011

Table summary
This table displays the results of Comparing 'Highest certificate, diploma or degree' with other data sources, Canada excluding territories, 2006 and 2011, calculated using Variable and distribution by category, 2006 LFS, 2006 Census1, 2011 LFS, 2011 NHS, 2006 Census1 − 2006 LFS and 2011 NHS − 2011 LFS (appearing as column headers) Percentage distribution and Percentage point difference units of measure.

Variable and distribution by category
2006 LFS 2006 CensusFootnote 1 2011 LFS 2011 NHS 2006 CensusFootnote 1 − 2006 LFS 2011 NHS − 2011 LFS
Percentage distribution Percentage point difference
Highest certificate, diploma or degree 100.0 100.0 100.0 100.0 0.0 0.0
No certificate, diploma or degree 25.0 23.3 20.8 19.8 -1.7 -1.0
High school diploma or equivalent 26.7 25.7 26.4 25.6 -1.0 -0.8
Apprenticeship or trades certificate or diploma 10.3 10.9 10.5 10.8 0.6 0.3
College, CEGEP or other non‑university certificate or diploma 16.7 17.4 18.2 18.3 0.7 0.1
University certificate or diploma below bachelor level 2.7 4.4 2.5 4.4 1.7 1.9
Bachelor's degree 12.9 11.7 14.7 13.4 -1.2 -1.3
University certificate, diploma or degree above bachelor level 5.7 6.6 6.8 7.6 0.9 0.8

Table 2 shows that the differences between the 2006 Census and the 2006 LFS percentage estimates are generally maintained between the 2011 NHS and the 2011 LFS estimates for this variable.

The category 'university certificate or diploma below bachelor level,' shows a percentage point difference between the 2011 LFS and the 2011 NHS that is fairly large compared to the overall size of the category. See the following section 'Respondent error' for more information. See the Education Reference Guide, 2006 Census, Catalogue no. 97-560-GWE2006003 for information on the difference between the 2006 LFS, the 2006 Census and the 2001 Census for this category.

Based on the estimates and trends from other data sources, evidence suggests that the NHS estimate for the population born in the Philippines is overestimated at the national level. Data users should take this into account when using data on 'location of study' for the Philippines, as those estimates may also be affected. For more information on place of birth, please refer to the Place of Birth, Generation Status, Citizenship and Immigration Reference Guide, National Household Survey, Catalogue no. 99‑010‑X2011008.

Response error

Response error occurs when a respondent misunderstands a question and reports an incorrect response.

Comparison of the data between the individual questions and write-in responses of the 2011 NHS, and with other data sources (2011 and historical data from the Labour Force Survey and 2006 and historical data from the census) indicates that there may be some response error in certain education question categories. Changes were made to the 2011 NHS questions to reduce such response error. These changes appear to have improved the situation, but have not removed the response error completely.

Apprenticeship and other trades versus college certificates and diplomas
Analysis of the categories for the apprenticeship, other trades, and college certificates and diplomas suggests that some respondents reported their apprenticeship or other trade certificates as college certificates, or reported them in both the trades and college questions.
University certificate or diploma below the bachelor's level
Comparisons with other data sources suggest that the category 'university certificate or diploma below bachelors level' was over-reported in the NHS. This category likely includes some responses that are actually college certificates or diplomas, bachelor's degrees or other types of education (e.g., university transfer programs, bachelor's programs completed in other countries, incomplete bachelor's programs, non-university professional designations). Data users are advised to interpret the results for the 'university certificate or diploma below the bachelor's level' category with caution.
Location of study

Location of study is defined as the province or territory (in Canada) or country (outside Canada) of the institution from which the highest postsecondary certificate, diploma or degree was obtained. Data users should be aware that some respondents may have reported the physical location of study rather than the location of the certificate, diploma or degree-granting institution. The question did not specify that the location of the granting institution was to be used, although the online help text did. In most cases, the physical location of study and the location of the institution granting the qualification are the same.

Inconsistency with the definition may occur in the responses of individuals who obtained a certificate, diploma or degree through a joint program or by distance learning with credentials granted by an institution in another province or country. In particular, a number of persons reported a location of study for a university credential in one of the territories (Yukon, Northwest Territories, Nunavut), even though there were no educational institutions in the territories with the authority to grant university degrees. These locations of study were generally kept as reported, because there are joint programs at the university level offered at colleges in the territories and the question was not specific.

This practice contrasts with what was done for the 2006 Census where a location of study in a territory reported for anyone with a university certificate, diploma or degree was replaced with a location of study in a province.

Attendance at school
Data users are advised that certification analysis of results from the 2011 NHS 'attendance at school' question showed variations with the Labour Force Survey. It is believed that a small proportion of respondents interpreted the NHS question as 'Has this person ever attended a school, college, CEGEP or university?' rather than the actual question 'At any time since September 2010, has this person attended a school, college, CEGEP or university?' As a result, the 2011 NHS could have overestimated categories with small proportions of school attendance, such as older age groups and/or where the respondent selected attendance at more than one type of institution. Since these categories already have small estimates, the impact of any reporting error would be proportionally more important. Data users should interpret the data on school attendance with caution.
Definition of 'highest' certificate, diploma or degree
The questions for major field of study and for location of study asked respondents to report for their 'highest certificate, diploma or degree' above the high school level. It is possible that respondents did not define 'highest' according to the same criteria used to derive the variable 'highest certificate, diploma or degree' and reported for a different credential. Some of these response inconsistencies for major field of study were addressed with edits or imputation, but they could not always be identified.

Cross-classification of education variables

Education variables are often crossed with other variables in a table to analyse a subject in more depth. Data users should be aware when examining small populations, either by selecting small geographical areas or by crossing multiple variables, that the estimates will tend to have greater variability due to sampling error.

Further references related to data quality

For general information on the overall content, collection, design, processing and data quality for the NHS, please refer to the National Household Survey User Guide, Catalogue no. 99‑001‑X.

Comparability with other data sources

Statistics Canada produces a range of education data from various sources for different uses. Comparability of education data with other data sources is affected by differences in survey target populations or administrative sources, survey sampling and collection methodologies; survey objectives, question wording, format, examples and instructions; approaches to data processing; the social and political climate at the time of data collection; and other factors.

As with every survey, the quality of the 2011 NHS education information released was evaluated internally prior to publication. The data were compared, as much as possible, with alternative data sources. The two main sources of comparison were the 2006 Census of Population and the Labour Force Survey (LFS).

Census of Population

Education data were collected in the census prior to 2011, reflecting a long-standing, continuing and widespread demand for information about education characteristics of the Canadian population. Over time, there have been differences in the question wording, format, examples and instructions of the education questions used. The historical comparability of education data has thus been affected by these factors, as well as by changes in data processing and in education systems over time. The 2006 Census included many changes to the education questions compared with the previous censuses. For more information, see the Education Reference Guide, 2006 Census, Catalogue no. 97-560-GWE2006003. Users should be prudent as variables may not have retained comparability over time.

Questionnaire changes

Although the 2011 NHS education questions were essentially the same as the 2006 Census education questions, there were some minor differences. Most of these differences took the form of additional instructions, different examples of programs or changes to formatting.

Question 28 on apprenticeship and other trades certificates and diplomas: In the 2011 NHS, the phrase '(including Certificate of Qualification, Journeyperson's designation)' was added to the 'Registered Apprenticeship certificate' response category to improve reporting. This appears to have resulted in a small increase in the 'Registered Apprenticeship certificate' category and a small decrease in the 'other trades certificate or diploma' category compared with the 2006 Census.

For additional information on the general comparison of NHS data with other sources including census data, please see the National Household Survey User Guide, Catalogue no. 99‑001‑X.

Labour Force Survey

The Labour Force Survey (LFS) includes questions on educational attainment, location of study (for immigrants born outside Canada who have postsecondary credentials) and attendance at school.

At the same time, differences in question wording, question order, methodology and coverage between the two surveys, mean that results are not expected to be perfectly identical. The LFS interviewers also reduce response error.

The LFS question on school attendance refers to attendance at school in the week prior to the LFS, while the NHS question on attendance refers to attendance at any time during the entire period from September 2010 to the NHS reference date (May 10, 2011). These periods are not comparable. As a result, the LFS school attendance estimates can be expected to be lower than those from the NHS for this period of time.

For more information on the general comparability of the NHS and Labour Force Survey data refer to the National Household Survey Dictionary, Catalogue no. 99‑000‑X, Appendix 2.1.

Date modified: