{"id":209,"date":"2013-04-22T00:32:15","date_gmt":"2013-04-22T00:32:15","guid":{"rendered":"http:\/\/blogs.oregonstate.edu\/geo599spatialstatistics\/?p=209"},"modified":"2013-04-22T00:32:15","modified_gmt":"2013-04-22T00:32:15","slug":"on-hot-spots-and-zero-inflation","status":"publish","type":"post","link":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/2013\/04\/22\/on-hot-spots-and-zero-inflation\/","title":{"rendered":"On Hot Spots and Zero Inflation"},"content":{"rendered":"

EDITOR\u2019S NOTE: The data used for this analysis contained spatially duplicated values. Meaning there are many instances of two separate measurements taken from the same site. These values might be temporally different (same site measured more than once on different dates), distinct but adjacent (different samples from same field, close enough to have the same lat\/long within rounding error), or true duplicates. Data was not cleaned for these error prior to analysis <\/em><\/p>\n

Introduction<\/strong><\/p>\n

This week I fiddled around with the two Mapping Clusters tools in the Spatial Statistics toolbox. Here\u2019s a rundown of each tool:<\/p>\n

Cluster and Outlier Analysis<\/em><\/span><\/p>\n

Given a set of weighted features, identifies statistically significant hot spots, cold spots, and spatial outliers using the Anselin Local Moran’s I statistic.<\/em><\/p>\n

Hot Spot Analysis<\/em><\/span><\/p>\n

Given a set of weighted features, identifies statistically significant hot spots and cold spots using the Getis-Ord Gi* statistic.<\/em><\/p>\n

Methodology<\/strong><\/p>\n

I wanted to challenge each of these tools to see if they could cope with a common natural resources data phenomenon: zero-value inflation. The problem of a large proportion of zero values is common with data obtained from ecological studies involving counts of abundance, presence\u2013absence, or occupancy rates. The zero values are real and cannot simply be tossed out, but because 0.0 is a singular value, it results in a major spike in the data distribution. This can adversely affect many common modeling techniques. The graphs below are from my data set of organic matter in Oregon soil pedons and have this zero-value complication:<\/p>\n

\"\"\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/p>\n

At left, the full data set with an enormous spike in the first bar (caused by all the zero values). At right, the same data set with the almost 400 zero-value data points removed. It\u2019s still right skewed, but looks like a more reasonable distribution spread.<\/p>\n

I ran each data set through both cluster analysis tools and visually compared results. There were noticeable differences between all four maps. In this \u201cspot the difference\u201d exercise, some major changes included: Hot spot in the northeast in full data only. Willamette Valley everywhere from cold to hot. Bigger east-west trend in the no-zero data sets.<\/p>\n

\"\"<\/p>\n

Discussion<\/strong><\/p>\n

Clearly, having a large number of zero-values in one\u2019s dataset will skew cluster analysis results. This was to be expected. In my data, the zero values were enough to overwhelm the generally \u201cwarm\u201d signature in the Willamette Valley. Also, the values in the north-east were \u201csignificantly large\u201d when balanced against a large zero data set, but \u201cnot significant\u201d in the more balanced no-zero data set.<\/p>\n

Regarding the Hot Spot Analysis versus the Cluster and Outlier Analysis, for this data set the Cluster & Outlier tool resulted in more stringent (smaller) hot and cold spots. Also, outlier values of cold in hot spots and hot in cold spots can been seen in the Cluster and Outlier Data Set, but vanish and it fact take the same label as the surrounding data in the Hot Spot Analysis (see below).<\/p>\n

\"\"\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0 \u00a0\u00a0\"\"<\/p>\n

At left, a close-up of soutwest Oregon in the Hot Spot analysis of the no-zero dataset. At right, the Cluster and Outlier analysis of the same area. Note the two dots in the middle that are classified as \u201chot\u201d in the left image but \u201clow, surrounded by high\u201d in the right image.<\/p>\n

Conclusion<\/strong><\/p>\n

I would be very wary of using these tools with zero-inflated and skewed data. This study was crude in that I completely removed zero values rather than re-casting them as low (but not zero) values. Nonetheless, there were drastic differences between all four maps in\u00a0 large sample (more than 1000 point) dataset.<\/p>\n

Between Hot Spot or Cluster and Outlier, for this data I would prefer to use the Cluster and Outlier data set. The same general hot spot pattern is maintained between the two. It seems there is a trade-off between smoothness and information regarding outliers within hot and cold spots. For scientific mapping purposes it would seem very important to be aware of those outliers.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

EDITOR\u2019S NOTE: The data used for this analysis contained spatially duplicated values. Meaning there are many instances of two separate measurements taken from the same site. These values might be temporally different (same site measured more than once on different dates), distinct but adjacent (different samples from same field, close enough to have the same… Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":5012,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-209","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/posts\/209","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/users\/5012"}],"replies":[{"embeddable":true,"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/comments?post=209"}],"version-history":[{"count":10,"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/posts\/209\/revisions"}],"predecessor-version":[{"id":221,"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/posts\/209\/revisions\/221"}],"wp:attachment":[{"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/media?parent=209"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/categories?post=209"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dev.blogs.oregonstate.edu\/geo599spatialstatistics\/wp-json\/wp\/v2\/tags?post=209"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}