Metropolitan Scale 3d Models
3D Models of Manhattan and New York City From GIS
This page discusses how students can extract three detailed 3d GIS datasets and CAD files from the Harvard Graduate School of Design's detailed three dimensional GIS reposotries for the Boston New York City metropolitan areas. Each of these models include: a high-resolution aerial photo, terrain model, edge of pavement, property parcels, open space, water-covered areas and a massing model for each building in the city. This project is a product of ongoing research concering strategies for developing models of places that interoperate among a variety of tools. For a deeper understanding of these strategies, see Workflows for Site Modeling in Context.
The slideshow below provides an overview of the elements and the organization of the metropolitan models )both of which are nearly identical in their structure. The tutorial covers the procedures for extracting detailed models that can be used as a referencing frame for other georeferenced information and as a point of departure for design studies. The products of these extractions can then be handed off to various tools that may be hooked together in various useful workflows, including: Adobe Illustrator, AutoCAD, Google Sketchup, Google Earth, RhinoTerrain, and ArcGlobe.
Accessing the Models
The full datasets for Boston and NYC are several gigabytes apiece. The data reside on enterprise-scale geodata servers maintained by the Harvard Geospatial Library. The zip arcives linked below include map documents that link to the net-based models. The connection to the data servers work only on computers that are connected via ethernet to a registered Harvard subnet. Howevwer, each zip archive includes demonstration models that contain a representative sample of the data schema in GIS and CAD and sketchup formats. Also within the You will find tools for doing interesting things with these data.
The GIS resources and tools described in this document can be used without much knowledge of GIS. If you are interested in how these resources and procedures were put together, you should take a look at some of the GIS user manuals:
- Using ArcMap for background on GIS data layers and attributes, and how to navigate in the ArcMap interface.
- Geoprocessing in ArcGIS Especially for background on setting the geoprocessing environment for Output Extent.
- Using ArcMap 3d Analyst for information about Creating Surface Models
Boston: Much of the information that enables our compilation of the 3d Boston Model originated from a LIDAR survey of the Boston Area that was flown by the Massachusetts GIS in 2001. This survey amounted to the 3d scan of the city that collected the heights of everything on the ground at a resolution of 1 meter. A portion of this survey was turned into vector-based building roofprints each having attributes for their roof height and footprint elevation. MassGIS has also made available color orthophotography for the region at a pixel-resolution of one-half meter. In addition to this, we also use the MassGIS 1:5000 digital elevation model, in which the height of the terrain is given for each 5 meter cell.
The New York City model: is made from several data layers obtained from The NYC Department of Information Technology and Telecommunications and NYC Planning Bytes of the Big Apple web site. The elevation model and aerial photographs was obtained from the The United States Geological Survey. Each building has been associated with the attributes of the property parcel that it falls within. To view metadata for these attributes, Click Here.
Scenario Selection and Rendering
Having All of these models in a large, scaleable database schema allow us to use database logic to retrieve logically consistent views of the city based on past, present and future conditions. Any numer of different thematic views may be drawn upon -- including commercial or emergency management themes.
This is a very large amount of data, which would be very awkward to deal with as files. We are assisted by the GSD's enterprise-scale GIS database -- Spatial Data Engine -- which uses an Oracle database server to store and retrieve data requested by ArcMap and ArcGlobe. The New York model is hosted by the Harvard University Library's Office of Information Technology. We have developed an ArcMap project that links to all of these datasets, and has some tools that will extract all of the layers for a user-defined area.
Download and explore the Metropolitan Model Archive ToolsYou may access this entire metropolitan dataset (if you are on the GSD's network) by excecuting the following steps:
- New York City Model
- Boston Model
- Open one of these zip archive and extract its contents to your c:\temp directory.
- Use windows explorer to check out the contents of your new metromodel folder. If you are interested, take a look at the metadata.txt file which documents the contents of this archive. Note the contents of the gis folder, its tools folder, and the clip_schema folder. These are organized according to the principles described in Framework for Organizing Place-Based Data
- Double-click on the ArcMap document, map_docs/compilation.mxd to start arcmap with this prepared document that links layers the GSD's Spatial Data Engine server.
- Explore these layers, by zooming in and out and panning. Notice that the building roofprints don't draw unless you are really zoomed in. There are multipatch objects around the area of the big dig, gund hall, and fenway park.
- Choose View->Bookmarks->Gund to zoom to the neighborhood of the GSD Notice how there are some roofprints and some multipatch objects all overlapping with eachother. In New York is is good tozoom into the area in Queens denoted by the red box.
- Take a look at the attribute tables for the various layers.
- Poke at each layer with the information tool to dynamically see the attributes of specific vector objects or raster cells.
Extracting Local Clip from the Boston ModelThe data layers connected to our Boston_Metro_2d data repository represent a 3d model of the city, but this model is rather awkward to visualize in a three dimensions all at once -- it can be done, but but it is not recommnded for casual introductory exercises such as this. So our first goal will be to extract a local clip of the data to our local hard drive. This will allow us to explore the model with more flexibility, and to serve as a base for our model improvement efforts.
The clip procedure amounts to choosing an area by zooming into it, and then cutting out each raster layer based on this area, selecting all of the roofprint and multipatch objects within this clip area. All of this is written into the clip folder, which can then be a free-standing copy of the model database that you can copy and take with you! All of these procedures have been encapsulated in a geoprocessing model that will be found in the the toolbox, Clip_Context.
Create a Clipping Frame
It is very useful to have a nice rectangle that you can use as a referencing frame for positioning various things that must be transferred between programs, some of which do not deal with actual georeferencing very easily. Vor example, we may be making a broad-scale model to use in Google Earth, and a tighter scale model to serve as the setting for some alternatives we will dream up using SKetchup or using the RhinoTerrain plugin for Rhinoceros. But we may wish to crewate groundplan diagrams using Illustrator and Photoshop or (my favorite) colored pencils and tracing paper. All of the three dimensional tools mentioned abover can easily drape our groundplan ideas onto their own terrain, and it is very easy to excahange these groundplan images between all of the programs -- especially if they all honor the same extent. With a geographically congruent registration frame in each of these programs, checking dragtss of the groundplan in and out is a snap. Click here for instructions on making a clipping frame.
Exporting a Clipped Image
It is useful to have a georeferenced ground-plan image draped on terrain in your 3d model. You can use this technique to create groundplan images that include the aerial photo or more simplified diagrams that include the water and road outlines (now available in the NYC model.) It is particularly useful to use the layering in GIS to create combinations of these layers that colorize the aerial to make it less distracting by overlyaing the land/water layers with soem transparency over the image.
- Zoom in to the area that you want to export. Consider that for a sketchup export you don't want more than, say 200 buildings, but for Google Earth, you can take a much broader area.
- Use the measure tool to measure the width of your window in meters.
- Turn on or Off any layers you want or don't want in your image.
- Choose File->Export Map
- Set your Files of Type to export a Jpeg image and set your options -- especially the Save World File option as shown in this screenshot.
- Adjust the DPI option to make the width of the image to be one pixel per meter. This is a good starting place, you canmake yours more or less once you have worked through this and determine what works best for you.
- Save the image to your clip_schema/gis folder, as groundplan.jpg.
Exporting 3D Buildings to a local GIS File
The idea here is that we want to extract buildings to a local GIS layer that we can work with without being connected to the server. Later we will export these buildings to the Google Earth exchange format, KML,AutoCAD's DXF format, Sketchup or Adobe illustrator.
- With your window zoomed into more or less the same frame that covers your image, right-click your buildings layer in the table of contents, and choose. Data > Export Data.
- It is crucial to choose the option in the first pulldown menu to All Features in View Extent. If you forget to do this, your export may never finish!
- Save your buildings as a File or Personal Geodatabase Feature Class in the geodatabase that has been prepared in your clip folder.
Exporting Buildings to Google Earth
Google Earth is a great tool for visualizing lots and lots of 3d data. If you look at the attributes table of your buildings layer, you will see that we have height information for each of the building polygons this can be used to extrude the buildings. We are fortunate to live on the same planet with Kevin Martin of the City of Portland Oregon. Kevin has produced a wonderful plugin for ArcMap that lets us very easily export data from ArcMap to KML, which is the way the Google Earth likes to get data. You can read about this plugin and download the latest version at the ArcMap plugins page. I have included version 2.5.3 of Kevin's script in the gis\tools folder of the Boston and NYC datasets. The following steps will take you through installing the plugin and exportng your buildings.
Installing the Export to KML Plugin
- In ArcMap, choose Tools->Customize.
- Click the Add From File button
- Browse to the GIS/Tools/Export_to_KML folder.
- Choose the file exporttokml.dll.
- Export to KML will now appear in your list of tool options in the main window of the Customize Tools dialog. Check its box
- See the toolbar?
Exporting the Buildings to KML
This is pretty straightforward. The one thing to figure out before exporting the buildings is which attribute holds the height information. Its a good idea to open the attribute table of the buildings layer to see what the names of the fields are and if the values make sense. In the boston model the field you want to look at is named Height in the nyc model the attribute is Hght_abv.
- Right-Click on your Buildings layer and choose Open Attribute Table
- Examine the attribute fields. IN the NYC model the hght_abv column is near the right hand side of the table, you need to scroll over to see it.
- Fine. Now click the Export to KML button
- For now, just fill in the blanks for the layer and the attribute to use for height. I recommend unchecking the box to group the buildings until you have been through the simple way first.
- Choose to put the output data in your clip folder with everyting else.
- Click OK and soon you will have your buildings in Google Earth!
A great thing about having a massing model exported from GIS this way is that it is easy to control the visibility of each building in a highly granular and also persistent way. This is something that currently can't be done with Google's own wholesale models (at version 5.). To do this, open the Features folder for your new layer. Then tilt your view in to a near vertical perspective and click on a building. You will see its entry light up in the Places window and you can turn it off.
Exporting your Model to Sketchup
This procedure is now covered in the document, Beginning a Sketchup Model from GIS Data
Make A Clipped GIS and CAD Dataset(advanced)
- Use Tools->Extensions to enable the Spatial Analyst and 3d Analyst extensions.
- Make sure you are zoomed into a relatively small area. For this demo, let me suggest the area covered by the Gund Hall bookmark.
- Find the toolbox, Clip Context and double-click on its model, Clip_Data.
- You will be offered a dialog to set the model's parameters.
- Make sure to set the >Output Extent option to "Same as Display" or else you will be attempting to download the entire metropolitan dataset to your hard drive, which will lock up your computer.
- Click OK, then click ok to delete the Clip folder (twice).
- Wait for the procedure to finish, then click close
The preceding sequence of steps has produced a TIN model aand made clipped copies of your LIDAR dataset, the building footprints and the raster terrain model. It would be good to have an image to drape onto our terrain when we make 3d models, so the next step will allow you to compose a map image and export it as a georeferenced JPEG image. We do this step outside of the ClipData model because it is much faster, and it will allow you to compose the image with whatever GIS datalayers and resolution you wish.
Your New Database
The previous process started by wiping out the existing clip folder, replacing it with a copy of the empty clip_template, which contains an empty personal geodatabase and ArcMap and an ArcGlobe documents ready to recieve the clipped data layers. Now if you take a look in the clip folder you will see that it has several new files in it. We will make our own copy of this new database, before exploring what it contains. Your copy of the clip folder is a self-contained GIS database that you can move anywhere you want. The Data layers and the ArcMap and ArcGlobe documents and the toolbox file in here are all related to eachother through relative pathnames so you can put this onto your own drive and use without being connected to the data repository. The arcmap document is useful for editing datalayers when necessary, the ArcGlobe dosument is useful for visualizing the data in 3d. The Scenarios.tbx toolbox contains database queries for sorting out which elements from the database should be selected for viewing different historic or future scenarios.
Taking Posession of your New Data and exploring it in 2d
First we will make our own copy of the clip folder, and then we will explore our new data in 2d with arcmap.
- Use windows explorer to look inside your clip folder.
- Make a copy of this folder and name it something else (e.g. "Gund") make sure that you leave the existing clip folder, or replace it with a copy of the clip_template, otherwise the clipdata model won't run again.
- Open the Arcmap document, clipped_2d.mxd Take a look at all of the layers. These are all basically the same as the layers that you examined from the repository, but they have been clipped.
- There is one additoional layer, the TIN (Triangulated Irregular Network) layer, which is a triangulated representation of the terrain useful for exporting to sketchup (later). See picture.
- Examine the attributes of the various overlapping multipatches and roofprints just east of Gund Hall. See how the multipatches are tagged with different min_dates and max_dates?
- Switch your table of contents to "Source" and open your copy of the substitutions table. Take a look at the pairs of references to multipatches and roofprints here. Try to visualize how this table stores a substitution relationship between multipatches and roofprints.
Do it Again?
So, you see that the Clip_Context model creates a new GIS database in the Clip folder. If you want to create a new clip dataset, you should make a copy of your clip folder and create a new empty clip folder by making a copy of schema_Template and renaming it clip_schema. Note that you will have to quit all ESRI applications and close the clip folder in all of your explorer windows or else Windows won't let you do this.
Exploring your Data in ArcGlobe
In the next step we will look at the clipped data in 3d using the 3d GIS Viewer, ArcGlobe.
Creating Scenario Views in ArcGlobe
- Double-Click the ArcGlobe document Clipped3D.3dd to start arcglobe.
- The layers in this document reference your clipped dataset, but they need to be refreshed to clear any data that acglobe may have cached. So right-click on each layer and choose Refresh.
- Zoom to layer on your othophoto.
- For more references and tips for using ArcGlobe, see Developng Rough Massing Models in ArcGlobe