Social Inequality in Colombia

The purpose of this tutorial is to examine several indicators of social inequality collected by the Colombian Census by municipality in order to calculate levels of vulnerability within the country’s population.

 Social Inequality  refers to the unequal distribution or access to goods, services, and opportunities across a population and can be quantified in a number of different ways. There are numerous indicators for social inequality, ranging from employment, food access, education level, health status, income, adequate housing, sanitation, and many more. These indicators can be combined to calculate a measure of social inequality or vulnerability within a population.

The key question we want to answer with this tutorial is:

Which municipalities in Colombia exhibit the highest vulnerability, as determined by three distinct indicators of social inequality?

The Social Indicators downloaded from Colombia’s  National Administrative Department of Statistics (DANE)  used to determine vulnerability for this particular analysis are:

• Percentage of Households with water sanitation services per municipality, 2011
• Percentage of Households living in conditions of poverty or misery according to unsatisfied basic needs per municipality, 2010 (Unmet Basic Needs Index – NBI) Gross
• Education Index Per Municipality, 2013

The learning objectives of this tutorial are to:

• Explore the basic attribute table tools
• Perform operations and analysis using selection tools and calculate
• Create a map that communicates the results of the level of vulnerability model

Some of the functions we will cover are:

• Joins
• Add fields
• Field calculator
• Statistics
• Summary
• Classification methods

If you have not already done so, download the Data_Vulnerability_Colombia.zip file from Box. Otherwise the data is also in the canvas site. Extract the data in your Box account (GIS work space).

Create a project for this exercise in ArcGIS Pro. You can name it Social Vulnerability in Colombia.

• In the Catalog panel, right-click Folders and select Add Folder Connection.
• Navigate to the place where you download and saved the folder Data_Vulnerability_Colombia.zip (where you extracted the data). Note: don’t double-click on it, simply select and then click OK to add.
• Look for the geodatabase called Municipality_CensusData.gdb. Double click again to expand the geodatabase to see the feature class data sets.
• Drag the Municipalities_WaterSanitation_2011 feature class data set into the ArcGIS Pro Session. The data for this tutorial is all at the Municipality level, which is their 2nd level administrative boundary in Colombia.

To begin, let's gain a comprehensive understanding of the dataset at hand. We can accomplish this by exploring the metadata associated with the feature class. To do so, follow these steps:

• Open Catalog and locate the Municipalities_WaterSanitation_2011 feature class.
• Right-click on the feature class and choose the option View Metadata.

Catalog View will open and you will see some basic metadata, including a description of the dataset so you can understand what the data is measuring.

If you just see a basic description of the metadata:

• Click on menu Project
• Click on Options (a window will open)
• Scroll down -- > click on Metadata
• Select a different format (e.g., FGDC CSDGM Metadata)

In this particular case, the dataset Municipalities_WaterSanitation_2011 provides information regarding the percentage of households equipped with sanitation services (such as toilets) in the year 2011. owever, at present, the dataset only displays the boundaries of the municipalities as the symbology has not been configured yet. To identify the field that contains the desired information on the percentage of households with sanitation services, please follow these steps:

• Right-click on Municipalities_WaterSanitation_2011 > Open Attribute Table
• Examine the contents of the Attribute Table. Make sure to remember the field-heading name or even better, change the Alias using Fields View.

Please take note of the unique identifier assigned to each municipality, which is labeled as SDE_dv_m_1. This identifier holds significant importance for our future progress in this tutorial. As we are calculating a vulnerability index, it is crucial to analyze multiple indicators of vulnerability. While the "percent water sanitation coverage" indicator is readily available as a feature class, the remaining two indicators, namely the Unmet Basic Needs Index (NBI) and Gross Education Index, were obtained as Excel tables from the Colombian Census. Consequently, we need to perform a Join operation to combine these indicators with our existing municipality data, which already includes information on water sanitation.

In Catalog, expand the excel workbook by clicking the plus sign to show the sheets.

• Select and drag in both sheets into the data frame as you would a layer (shapefile or feature layer from a geodatabase).
• In the Table of Contents, right click on NBI$ and Open. Here, we can examine the data that is in this excel table.

We see three columns:

• The Municipality Code (SDE_dv_m_1)
• The NBI Index
• The year the data was compiled (Date)

It’s important that we know what the NBI is measuring. Let’s refer to the metadata. In Windows Folders, navigate to where you have downloaded the data and open the metadata word document for NBI.

Q.1. What does the NBI measure? Is a high or low number more or less vulnerable?

To simplify the process, we will join the NBI stand-alone table with the existing Municipalities_WaterSanitation_2011 feature class. Since both datasets utilize the same unique identifier for municipalities (which is of great significance), we can effortlessly combine the NBI data with the feature class. By doing so, all the relevant data will be consolidated into a single attribute table, eliminating the need for subsequent data merging. This approach saves time and ensures that all the required information is readily accessible in one unified location.

• Close the table and then right click on Municipalities_WaterSanitation_2011 > Joins and Relates > Add Join
• Make sure the NBI$ is the selected table in step 2 of the Join window. For step 1, select SDE_dv_m_1, which was the Unique Identifier in the Municipalities layer. Now, it automatically recognizes the matching field in the excel table, SDE_dv_m_12. It is important to make sure that the matching identifiers are chosen in step 1 and 3. Otherwise, the join will not work. In any case, you can also click on Validate Join.

Ensure that the tool is filled out like the Add Join screen-shot and press OK.

• Open the attribute table for Municipalities_WaterSanitation_2011 and scroll to the right to ensure the data joined properly.
• You should see the 3 fields from the excel table now at the end of the attribute table.

Now, let’s join the GrossEducationCoverage$ excel file to the same Municipalities_WaterSanitation_2011 layer as well. It will be helpful to have all this census data in the same feature class' attribute table.

• Open the GrossEducationCoverage$ and make sure to check on the metadata. You can find the metadata word document in the same folder. Read through the description of the data.
• Right-click on Municipalities_WaterSanitation_2011 > Joins and Relates > Join…
• Fill out the join table as we did for NBI, but make sure the GrossEducationCoverage$ is selected for step 2. Or else, we’re just rejoining the NBI data.
• Press ok.
• Once the join is performed, open the attribute table for Municipalities_WaterSanitation_2011 and scroll to the right to ensure the data joined properly. You should see the 6 fields from the 2 excel tables now at the end.

It’s good practice to Export a Join into a new shapefile or feature class under the default geodatabase. This is because joins are not permanent and if you were to run some analysis on Municipalities_WaterSanitation_2011, it would drop the two tables you just joined. But if you export the data, it saves the join as the attribute table and will be permanent.

• Right-click on Municipalities_WaterSanitation_2011 > Data > Export Features Name the file Municipality_Vulnerability and press Ok.

It’s important to choose a name that tells what the feature class is mapping (municipalities) and by adding to the name the word Vulnerability, we know this is the feature class that has all of our indicator data in the attribute table.

All vulnerability indicators will be ranked using two different approaches. The first approach involves the use of symbology to rank the indicators and the field calculator to assign these ranks to the respective municipalities. The second approach utilizes a tool called Reclassify Field, which efficiently applies a classification method to rank the indicators and assign corresponding values to the municipalities.

In our pursuit to create a Vulnerability Model for Social Inequality, we will eventually combine three indicators. However, a simple addition of these indicators is not feasible due to their differing relative scales. To address this, we propose ranking each indicator on a vulnerability scale ranging from low (1) to high (4). By assigning ranks to each field, we can then aggregate them to calculate a comprehensive social inequality score, which will range from 1 to 12. To determine the rankings for the vulnerability data, it is advisable to thoroughly examine the symbology and classification methods employed.

Note: In your future analyses, you have the flexibility to determine the number of classes you wish to employ for ranking your data. In this particular case, we have chosen to utilize four classes (1, 2, 3, 4). However, it is important to emphasize that the selection of class intervals is not limited to four. As the researcher, it is your responsibility to evaluate and decide what classification scheme is most suitable for your analysis.

• Open the symbology properties for our new Municipality_Vulnerability layer (right-click and then Symbology).

The objective is to visually represent the percentage of households with sanitation services in each municipality, which is a quantitative measure. To achieve this, please choose the Graduate color scheme from the drop-down menu.

• Next, set the field value to dbo_V_Da_1 or Water Sanitation (% of households with sanitation services). The map has now used graduated colors (low to high) to show the data.
• Next, click on Method. Currently, the data is broken up using the  Natural Breaks method . Natural breaks classes are based on natural groupings inherent in the data. Class breaks are identified that best group similar values and that maximize the differences between classes. In this exercise you will use the method Natural Breaks to determine your ranking classification from 1-4.

Now we can see the distribution of the values in this field by examining the histogram.

• Click on Histogram tab next to Classes. This data is already broken up into 5 classes. Each blue line represents a class break. For example, 0 – 12.5 is the 1st class, then 12.5 to 31.1 is a 2nd, and so forth and so on.
• Next, set the classification method back to Natural Breaks and change the number of classes to 4. You will use these natural breaks to rank your data from low (1) to high (4) Vulnerability.

• Open the attribute table for Municipality_Vulnerability. First, we need to add a field to hold our vulnerability rankings for the values of Sanitation Coverage.
• Click on Add at the top of the attribute table.
• Name the new field Sanitation_Rank and keep it as a Short Integer. Remember, no spaces or characters other than an underscore (_). Also, short integer means that the field will be a numeric field, but only allow for integers (aka no decimals).

• Once you add the field, go to the top under Feature Layer and click Save.
• Go back to the attribute table and scroll to the right. You will see that a new field has been added with our new name. However, all the values are because we have not calculated them yet.

Now, we need to decide how we are going to Rank Percent of households with sanitation services. We decided when exploring the classification methods in symbology that the natural breaks method was a good way of dividing the data. Let’s use those break values for the 4 classes we have already set. Something to remember is that we are measuring Inequality. So LOW percent of sanitation coverage is HIGH inequality/vulnerability. This is very easy to get backwards, so be careful with EACH indicator.

Using the natural breaks classification from Symbology, the data on percent sanitation coverage will be ranked as follows:

• 0.000000 - 20.800000 % = Maximum Vulnerability (Value 4)
• 20.800001 - 44.800000% = High Vulnerability (3)
• 44.800001 - 70.500000% = Medium Vulnerability (2)
• 70.500001 - 98.300000% = Low Vulnerability (1)

Note: If your break classification is slightly different, you can continue with your values in the next steps.

• Use Select by Attributes to Query the data and select all rows < 20.80
• Change the input Rows to Municipality_Vulnerability. Click on Add Clause and fill out the expression boxes.
• Click on dbo_V_Da_1 (or % of households with sanitation services) to add it to the query window.
• Click on expression “is less than”, then select 20.80 in the next box. This should produce an expression that looks like: “dbo_V_Da_1” “is less than” “20.80”.
• Click on Add.

The results show that 293 out of 1118 rows are selected and have values ranging from 0 – 20.8.

• Now, with these values selected, right click on the Sanitation_Rank field, and select Calculate Field.
• Next, assign the Ranking/Score value to our selected municipalities. These are the municipalities with the lowest percentages with sanitation services, there for they are the most vulnerable and will be given a score of 4.
• Enter 3 in the field calculator. The calculator already knows the expression is Sanitation_Rank = because we clicked on this field when opening the field calculator.
• Press ok.

Now all the selected municipalities have been given a score of 4 while the unselected rows remained. As a reminder, when a tool or operation is run with selected values, it ONLY runs the tool or applies the operation on the selected rows. This is true for ALL tools, not just field calculator.

Next, we will assign the next values a score of 3 for medium-high vulnerability, which we have determined will be from the range of values 20.8 to 44.8. Repeat the same steps, this time selecting the rows from 20.8 down to 44.8.

• Click on Select by Attributes.
• Double click on dbo_V_Da_1 (or % of households with sanitation services) to add it to the query window.
• Click on the "greater than or equal".
• Scroll down to 20.80 and double click on it.
• Next, click on the Add Clause and use the scroll-down to get to dbo_V_Da_1 again, and then "less than". Scroll down to 44.79 and select this value - you can also type it but there's always a risk of a typo.
• You should now have 361 out of 1118 rows selected.
• Again, right click on the field Sanitation_Rank and select Calculate Field
• Assign the value 3 and press ok.

We are now assigning a score of 3 to the medium-high vulnerability municipalities.

• Press Run and the selected cells will populate with 3.
• Repeat the same steps, now using a query to select the rows between 44.79 and 70.5. You should have 276 rows selected and your expression should match the image below.

Repeat the operations to calculate the ranking values 2 and 1.

• Right click on Sanitation_Rank and select Calculate Field.
• Change the value from 3 to 2 and press ok. We are now assigning a score of 2 to the medium-low vulnerability municipalities. Press Ok and the selected cells will populate with value 2.
• Clear the last selection.

Next, we will assign the municipalities with high percentages of sanitation (70.5 – 100%) selected a score of 1 - they are least vulnerable.

• Once the selection is done, return to the field calculator and give these values a score of 1.
• Press ok. After it populates, clear selection.

Currently, the entire Sanitation_Rank Attribute Field should be populated with vulnerability scores from 1 - 4. Make sure you did not miss any and there are no values by sorting the field descending and ascending.

The  Reclassify Field tool , reclassifies values in a numerical or text field into classes based on bounds defined manually or using a reclassification method. Similarly to the previous process, this tool uses the reclassification method and assign the values to each rank. The next two variables are NBI and and Education. The steps are next:

• Open the Tools under the Analysis menu.
• Under the Find Tools, type Reclassify Field and click on top of it. A window will open.
• The input table is the Municipality_Vulnerability layer.
• Now, as we did with water sanitation and symbology, we want to reclassify NBI, so look for the field and select it. This time, the classification will be run in the back end - we won't see it - instead of using a classification method through the symbology.

For the sake of this exercise, we will continue to use Natural breaks (Jenks).

• First, always read the messages embedded on the tools (see message on blue).
• Select under Reclassification Methods, and pick Natural breaks (Jenks).
• Next, type the number of classes. We are using 4.
• Right below the Number of Classes, we have a check box that says Reverse Values (Descending). We don't need to use it given that the higher the NBI, the higher the vulnerability (check on metadata to understand what NBI is, if you have questions).
• Next, under the Output Field Name, we want to change the name to NBI_Rank.
• Click Run.

Once the tool has run, open the attribute table and scroll to the right to see the new two fields.

Now, let's apply the same process to the Education Coverage variable. When examining the metadata for this variable, we find that it represents the ratio of students per educational level to the population in the corresponding age range per municipality for the year 2013. Therefore, a higher value indicates greater coverage. It is crucial to consider the metadata carefully and determine whether the "Reverse Values (Descending)" option should be selected, as explained in the previous steps for clarity. This option is used when you want a low value to correspond to a high number.

Note: there will be some warnings due to some Null values. No further action is needed.

Now that we have completed ranking all 3 indicators from lowest vulnerable municipalities (1) to highest vulnerable populations (4), we can combine them using field calculator get an overall vulnerability score from 3 – 12.

• First step is Add a Field to this attribute table to hold our vulnerability scores.
• Open the Attribute Table for Municipality_Vulnerability.
• In the Table Options, select Add a Field… this is a different way to add a field.
• Name this new field Vulnerability and leave it as a Long Integer.
• Scroll all to the right of the Municipality_Vulnerability attribute table and you will see your new attribute field.
• Right-click on the heading Vulnerability > Calculate Field.

We simply need to add together the 3 fields that hold are ranking scores. This is why it was so important to give these fields identifiable names.

• Double click on Sanitation_Rank, then click the + sign, then double click on NBI_Rank then click the + sign, then double click on Education_Rank. Make sure your calculation looks like the screen shot.
• Press OK.

Now the Vulnerability Field calculates with a score from 1 – 12 with 1 being lowest possible vulnerability (all 3 indicators equal 1) and 12 being highest possible vulnerability (all 3 indicators scored 4).

• Double-click on Vulnerability field heading to sort it from low to high. Scroll down to get an idea of how the municipalities scored. You’ll notice the field that had Null in the rankings remained Null in the final score.
• Right-click on Vulnerability > Statistics… Here, we can see the distribution of scores by looking at the histogram. We can see the min and max score and the mean vulnerability score.

Please consider incorporating the graph into the StoryMap.

We need to ask the following question first:

Are the scores considered Categories or Quantities?

Your job will be to use the best type and method for symbology and prepare the layer so we can export it into ArcGIS online. All you will need is to rename the layer appropriately, change the symbology and make sure that you only have in your map the colombia_vulnerability layer and an ESRI baseline.

Once your map is ready, we will export it into ArcGIS Online. The process is relatively easy, all we need to do is to follow these steps:

• On the ribbon, click the Share tab. In the Share As group, click Web Map. The Share As Web Map pane appears.
• Make sure that the name provides basic information for the web map. The default name is the name of your Map; avoid leaving it as "Map" if that's the default name.
• Optionally, complete the Summary and Tags fields. You can enter a maximum of 128 tags.
• Select a configuration for your web map. Exploratory works for this exercise.
• On Share with, select Tufts University.
• Click Analyze.

Depending on your active portal, different configurations are available. However, you need to sign-in in order for you to share the map.

When you click on Analyze, you will potentially have two types of errors. Check on the following video to see how to solve them.

Pop-ups contain information about features and images in map layers. Pop-ups can include attachments, images, charts, and text, and they can link to external web pages. It is important to note that the default pop-up appearance for a feature layer—if the layer owner has not configured it—is a table of attributes and values. So if you look at the attribute table of each of your layers, and then you click on top of any feature display on the map, it will show jus that.

As a layer owner, you have the ability to save a new configuration, allowing the pop-ups on a map to display relevant information effortlessly, without requiring any additional steps from the map author. This feature enables you to consider the needs of decision-makers or the audience accessing this information and determine the key information that will greatly facilitate the decision-making process. By carefully selecting and presenting crucial information, you can enhance the efficiency and effectiveness of decision-making.

1. On the Contents (dark) toolbar, click Layers to open the Layers pane, and select the layer that contains the feature data you want to show in a pop-up - in this case, we just have the layer on vulnerability.
2. On the Settings (light) toolbar, click Pop-ups Pop-ups.
3. First, I will select a Title and make sure that {SDE_dv_m_5} is the field used for the title.
4. Click on Field List and turn off (by clicking on the cross-check) all fields that are not needed. All except Educational Coverage, NBI, Water Sanitation, all ranks, and the vulnerability score.
5. Next, I will go to Fields option and when you click on each one of the fields, you can change the number of decimals for any numeric fields.
6. Save the project.

Watch the following video to see how it is done.

 StoryMaps from Esri  are a platform and a format for creating and sharing interactive and engaging digital stories that combine maps, text, images, and multimedia content. StoryMaps allow users to tell a narrative or convey information using a combination of maps and other visual elements. With StoryMaps, users can create dynamic presentations that guide viewers through a series of interactive maps, images, videos, and text, enhancing the understanding and engagement of the audience.

The StoryMaps platform provides a range of templates and tools that simplify the process of creating these digital stories. Users can choose from various templates that offer different layouts and designs, allowing them to customize the appearance of their story to suit their needs. The platform supports a variety of map styles, including 2D and 3D maps, satellite imagery, and custom basemaps. The interactive nature of StoryMaps enables users to incorporate multimedia content, such as photos, videos, and audio, to enhance their narratives. They are particularly useful for visualizing spatial data, highlighting location-specific information, and presenting geographic stories in a compelling and accessible manner.

Once created, StoryMaps can be easily shared and embedded on websites, social media platforms, or viewed directly on the Esri StoryMaps website. Let's try making one!

• Go to  StoryMaps  and sign-in.
• Click on New Story.
• Click on Start from scratch.

Click on Title your story to type in your title. Click below the title to add a subtitle (it's optional), and, below that to adjust your byline.

You'll notice in the upper right that there's an option to Add cover image or video. We'll deal with that in the Adjust the design section below. ArcGIS StoryMaps also includes two additional cover designs; we'll cover those later as well.

• Start by adding a title and introductory text to set the context for your story.
• Follow this step by adding a subtitle.
• Next, start by telling the story. You can add first a section to introduce the concept of vulnerability and talk about the three different variables we used to create an index (see my example).
• Once you select a title for the section, assign the category of Heading 1.
• Next, you can introduce the concept of vulnerability. Just write a brief two paragraphs.
• You can enhance your StoryMap with multimedia content such as photos, videos, and audio. You can upload your own media or use external sources.
• Add descriptive captions, annotations, and text blocks to provide additional context and information for your maps and media elements.
• Add the map you have created.

• Click on the 'Design' option at the top of the page.
• Select the 'Cover'.
• Under 'Optional story sections', turn on the option for 'Navigation'.
• Select the 'Theme' you prefer.
• Upload a 'Logo'
• Finally, you can get out from the Design Menu and access the 'Preview' option. It is important to ensure everything is displayed correctly and the interactive elements work as intended.
• Make any necessary revisions and edits to improve the overall quality and coherence of your StoryMap.
• Once you are satisfied with your StoryMap, click the 'Publish' button to make it accessible to others (make it only available to Tufts as this is only an exercise).
• Share your StoryMap with me by copying the link and sharing it on Canvas.

Feel free to play around. Try different blocks—it's easy to delete items that you add. For text, just highlight and delete, like you would in any text document. To delete graphics, click three dots at the end of the toolbar that appears when hovering over the item, and select Delete.

You can use the link  Getting started with ArcGIS StoryMaps  for more useful information about creating a StoryMap.