5

Related to this question: Raw data from US Census and BLS what we want is occupational data for natural language processing research (i.e,. the text that people fill in, not the bubbles) probably from the census long form, but other data sources would suffice.

Looking through the different census sites, I didn't see anything that had that type of data. Does anyway have any other sources, or is there a way to get the raw (transcribed) long-form text?

Thanks.

Community
  • 1
ndunn
  • 153
  • 2
  • 2
    There is a lot to breakdown from this post. First off, the Census Bureau no longer has a long form survey; it is now known as the American Community Survey. Second, the occupational data is derived from a write-in question. Would you be interested in the version that allows you to see a respondent's NAICS or SOC code? – Kotebiya Jun 22 '14 at 04:17
  • You might want to check out this webse which tries to make some of the Census/ACS data more accessitible; here is a direct link to their page about employment: http://censusreporter.org/topics/employment/. I'm not 100% sure they include the free-text data though. – Mark Silverberg Jun 22 '14 at 14:55
  • Perhaps try http://censusreporter.org/data/table/?table=B24124&geo_ids=01000US&primary_geo_id=01000US - is that specific enough? – Mark Silverberg Jun 22 '14 at 15:42
  • To answer the first question from the first post, I was aware that the last year the long-form was available was 2000. For the second part, I would be VERY INTERESTEd in any transcribed free-text survey information. – ndunn Jun 22 '14 at 22:38
  • Mark, I'm not seeing specific free-text. – ndunn Jun 22 '14 at 22:42
  • Well, before I provide an answer to your question, let me just comment that the Census Bureau has an entire team of transcribers devoted to interpreting people's responses to the write-in question. Their specialty is to fit the responses to a Classification System for North American Industry. It would be a shame to not appreciate their work when it is specifically designed for your convenience. – Kotebiya Jun 22 '14 at 22:51

1 Answers1

2

The American Community Survey (ACS) Public Use Microdata Sample (PUMS) is likely what you are looking for. The ACS includes occupational codes for respondents based on questions that people responded to in this questionnaire (page 11). In the PUMS dataset, there is the INDP variable which houses 2012 NAICS occupation codes which you look at in a data dictionary (e.g. The 2012 1-Year ACS PUMS (page 60)). The raw files are present on the Census Bureau's FTP server here.

However, if you are looking for the most approachable method to diving into the ACS PUMS, I recommend doing it through the University of Minnesota's iPUMS project. The data is open access, registration is free (although there is wait time for registration approval wait period that might take about a day). It allows you to download data that you can import into Stata and SAS.

Kotebiya
  • 2,999
  • 1
  • 15
  • 22
  • [http://www.census.gov/acs/www/Downloads/questionnaires/2012/Quest12.pdf] (page 11) – ndunn Jun 23 '14 at 02:39
  • (sorry, took me too long to write the previous comment) The link on page 11 is the type of raw data that I want. Specifically the raw (pre-coded) answers to 42, 43, 45, and 47. The raw data also seems to be encoded, as well. We are working on algorithms to encode the raw data, so having the raw pre-coded data would be the most helpful. – ndunn Jun 23 '14 at 02:44
  • I'm willing to guess that the raw answers given constitutes protected information. The only way to access this type of information is to apply to do research with the information. And even if you do manage to get approval, you'd have to do the research in-house at one of the Census Research Data Centers (RDC's). How is your SAS btw; because that is what you'll have to work in order to use this data. – Kotebiya Jun 23 '14 at 02:59
  • As far as developing an algorithm goes, I personally don't see how that is possible. There are a lot of reasons that the National Processing Center needs a team of people to decipher the responses. Firstly, occupational identification is a constantly evolving process on a macro-scale where jobs come-and-go in a decadal span. An algorithm wouldn't be able to pick up on changing trends of human work over time like a human could. OCR technologies even now still need to be refined to recognize people's written responses. – Kotebiya Jun 23 '14 at 03:08
  • Both of these answers are very helpful. Part of our research is to see how close we could get to human answers as we want to observe these over decades. Thanks! – ndunn Jun 23 '14 at 03:55