Currently, the Arches project team is concentrating on finalization of v3.0 (see Roadmap). In the meantime, here’s a sneak peek of the documentation for the new shapefile upload functionality. It’s written by Cyrus Hiatt, one of our team members at Farallon Geographics, and outlines how incredibly easy the process will be!
Uploading shapefiles into the Arches Version 3.0, Heritage Inventory Package (HIP) application:
Shapefile upload is one of the nice, new features of Arches 3. Though a little configuration is needed to load your shapefiles into Arches, the process is very simple. This post will describe the steps needed to prepare your shapefile and load it into your Arches application.There are just a couple of limitations to loading data using shapefiles.
- Your shapefile can contain only one resource type per shapefile. For example, you would need one shapefile to load Heritage Resources and a second shapefile for Activities.
- Your shapefile projection must be defined as WGS84 Lat/Lon (EPSG:4326). Otherwise, loading data into Arches from a shapefile is a simple 3 step process.
Step 1. Create your configuration file.
You’ll need to create configuration file to tell Arches how the fields in your shapefile correspond to nodes in a resource graph.The configuration file should have the same base name as your shapefile with the extension ‘.config’. For example, if your shapefile is called ‘heritage_resources.shp’, you will will need a ‘heritage_resources.config’ file along with your ‘heritage_resources.dbf’, etc.
The .config file is formatted in JSON as you can see in this example:
“RESOURCE_TYPE” is the resource type of your shapefile.
“GEOM_TYPE” is simply the name of the entitytype that Arches HIP uses to manage geometry.
“FIELD_MAP” is where you list your shapefile fields and their corresponding entitytypes. Each field must be enclosed in brackets, with the shapefile column name in the first position and the entitytypeid (e.g.: node name) in the second.
Note also that the curly braces at the beginning and end of the file are important!
Step 2. Confirm Your Shapefile Data
There are just a couple of details to keep in mind when creating your shapefile.
- Remember that the projection of your data must be EPSG:4326 (WGS84 Lat/Lon).
- The data type of your shapefile columns need to match the datatype of the nodes (entitytypes) to which they are mapped. For example, if you want to load a “name” column in your shapefile to “NAME.E41” (which the HIP defines as a “string” data type), you must ensure that your shapefile treats “name” as a string.
- If you want to load a field in your shapefile to an Arches controlled vocabulary, you’ll need to make sure that the values in the shapefile match a preferred label in Arches’ Reference Data Manager (RDM). For example, if you want to load values from your shapefile into the “HERITAGE_RESOURCE_TYPE.E55” node, the values in your shapefile column must match a preferred label in the RDM scheme (by default Arches uses the “HERITAGE_RESOURCE_TYPE_AUTHORITY_DOCUMENT.csv” scheme)
Step 3. Load the Shapefile into Arches.
Version 3 of Arches comes with a shapefile load command. To use it, open a terminal window on the Arches server and type:
So, if the path to your shapefile is: /home/me/data/shps/heritage_resources.shp you would simply place the full path between the single quotation marks:
Press the return key and Arches will start loading your data!
What if I Screw Up?
If you managed to load data that you did not intend to load, or if for some reason you changed your mind about loading your data, don’t worry. You can easily unload the shapefile data from Arches.
To do this you will need the Load ID that was created when you loaded your data. Arches writes the Load ID to the terminal you used to start the import, and the Load ID is also located in the notes column of your data.edit_log table.
The Load ID will look something like this: “LOADID:2014-12-2-7-17-23884” (e.g.: the timestamp by year-month-day-hour-minute-microsecond) of the start of your resource load, so that may help you find the correct Load ID if you are unsure.
Once you have the correct Load ID, you can delete all of the resources created during a load with the following command:
Troubleshooting Data Loads
If your data did not load successfully, it is most likely because you are trying to load a value from your shapefile into a controlled vocabulary that does not include that value as a preferred label.
In these cases, Arches will stop trying to load your shapefile and won’t write anything into the database. You can refer to the “shapefile_loading_errors.txt” file in HIP log directory for more details.
If you are having another problem, feel free to post your question on the Arches forum https://groups.google.com/forum/#!forum/archesproject
