Downloading Census Data
This page explains a workflow for downloading US Census data for the purposes of mapping. The procedure for getting census data into shape for mapping, involves downloading GIS files representing census geography and joining this to a table of demographic attributes corresponding to those geometries.
The Workflow Map
The Decennial Census is primarily a set of tables. It happens that every row in a census table relates to some specific geographical area. To begin exploring census data one must think up a question, find pertinent census products and tables. To make maps, you will need a geographic shape file -- which can also be downloaded. Then you need to join the tabular information with the shape files.
Understanding and Downloading Demographic Data
Before digging in and exploring census data, you might want to get familiar with the basics of the census products and geography by consulting this page About Census Data. For this tutorial we will be using The comemrcial site, Social Explorer. Harvard users can log in with their ID and PIn to have full access to this site. For users outside the Harvard community, a similar procedure may be followed using the Census Bureau's American FactFinder Web Site. This procedure is coverd on a very thorough tutorial by Barbara Parmenter Department of Urban and Environmental Policy & Planning Tufts University.
For the purposes of this tutorial we will explore the American Community Survey (ACS) 5 year data covering 2006-2010. This product provides estimates for a very diverse set of demographic and housing characteristics and are available at the blockgroup level of aggregation.
Just to get an idea of the sorts of data columns that are available at the block group level from the ACS, we could use the mapping tools of social explorer and play with the legend settings to browse and choose data fields for mapping. With the mapping tools you can zoom in to a block group and click on it to check the numerical attributes associated with that block group. You can do a lot of exploring this way, and this is particularly informative regarding which attributes of the ACS are available at the block group level of aggregation.
Once you have acquainted yourself with the data, you may want to download some of it as a table that you can use in GIS.
Generate a Report for Exporting from Social Explorer
The process of downloading data begins with the generation of a report. This process begins with choosing the census product, and then requesting a specific type of geometry to export over a specific geographic extent. For my example, I click on the Reports tab at the top of the Social Explorer page and then choose ACS 2006-2010 from the left side of the page.
Obtain Geography Data
The web page About Census Data explains the different geographies of the census and how the various census products are aggregated to different geographies. For the 2000 census, the Summary File 1 data are available at the Block level, The Summary File 3 has more interesting data items, but is much less fine-grained geographically -- aggregated to the Blockgroup level.
Downloading your Geography Files
If you are working in Massachusetts, you can simply use the selected TIGER boundary files included with the sample dataset. If you are working in another state -- or across state, then you can easily download your own Block and Blockgroup files following the instructions below. In either case, you may want to explore what is available from the bureau of Census Web site.
- Go to the TIGER Line Download Page
- In our example case, we want blockgroups, so we choose this from the pulldown menu and choose Submit.
- On the next page we choose Massachusetts in the top blank for 2010 census geography.
- On the next page we can choose a county or choose the first option to download all blockgroups in the state. This is what we want.
- WHat you get is a bunch of shape files that are zip-compressed and all within another zip archive. You can right-click to expand it.
- Your shape files should now open in ArcMap!
Join your Census Data Table with the BlockGroups Feature Class
As you see, the census data is in a comma delimited text file and the blockgroups are in a shape file. Before we can start mapping these, we need to join them together. This procedure uses the FIPS code (federal information processing standard) as the linking file to uniquely identify the blockgroups in each table. IN the Social Explorer table the FIPS code is named GEOID10.
Don't Open the CSV File in Excel!!
You should open your CSV file directly in arcmap to join it to the TIGER shape file. Resist the urge to double-click on it to poen it in Excel. If you do do this, DO NOT SAVE THE FILE!!.
The CSV suffix on your file denotes a file that represents a table as a plain text values separated by commas. By convention a value enclosed in double-quotes is supposed to be treated as a plain text string. To make the join between the CSV table and excel, it is necessary that both the Census FIPS and the TIGER GEOID10 are both treated as text values. Unfortunately Microsoft Excel does not honor this convention. So if you open your csv file in excel, it will see that the FIPS column is made up of numerals and it will try to be helpful and treat this is a numeric type field. This is silly because FIPS codes are not numbers!!! If you must open the file in ecxel then you should do so by opening a blank workbook, going to the Data Tab, and choosing From Text this will allow you to control the data format for each column in the CSV file. Why does excel make it difficult to interroperate with other programs through the simple open exchange convention of CSV? Could it be that this is an insidious conspiracy to promote their own, more complicated .xlsx exchange format? Naaah!
- Use the ArcMap Add Data button Open the csv file from the social explorer folder and the shape file from the tiger folder.
- Open the tables associated with each of these datasets. Open the data dictionary from the social_exploer file for reference.
- Right-Click your TIGER shape file and choose Joins and Relates < Join
- Fill in the blanks in the Join dialog to choose your social exploerer table as the table to join to. Choose FIPS and GEOID10 as the linking fields, respectively.
- Once the join has been done, inspect the attribute table for your tiger shape layer.
- Zoom To layer on your new blockgroups Layer.
Reduce the Size of your Dataset
If you have downloaded all of the blockgroups for an entire State as descrived in this tutorial, you may want to reduce the size of the dataset. I recommend that you zoom in to an area that is slightly larger than your region of interest, and export the blockgroups within the current view. This will serve to eliminate a substantial amount of useless data from your census datset, and will also fuse the census fielsd with the blockgroup geometry which wil lmake mapping operations run much faster.
- Zoom to an area that includes the region of interest. Be sure to allow plenty of slack.
- Right-click the layer containing joined blockgroups and census data
- Choose Data > Export Data
- In the top pulldown, choose Export Features within Current Extent
- Export the features as a new shape file dataset.
Now for some mapping, proceed to the tutorial on Mapping Census Data Tutorial.