Hierarchical and K-Means Clustering in R on Hofstede Cultural Patterns.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clustering: Hierarchical and K-Means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Purpose"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What follow is an exploration of clustering, via hierarchies and K-Means, using the Hofstede patterns data, available from my Public folder, https://drive.google.com/folderview?id=0B5Mi6_CV6da5aFc2Q0g3cHowVms&usp=sharing.\n",
    "\n",
    "For a deeper understanding of clustering and the various related techniques I suggest the following:\n",
    "\n",
    " - Cluster analysis (Wikipedia): https://en.wikipedia.org/wiki/Cluster_analysis\n",
    " - An Introduction to Clustering: https://www.analyticsvidhya.com/blog/2016/11/an-introduction-to-clustering-and-different-methods-of-clustering/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# load data\n",
    "Hofstede.df.preclean <- read.csv(\"HofstedePatterns.csv\", na.strings = c(\"\", \"NA\"))\n",
    "#nrow(Hofstede.df.preclean)\n",
    "\n",
    "# remove NULLs\n",
    "Hofstede.df.preclean <- na.omit(Hofstede.df.preclean)\n",
    "#nrow(Hofstede.df.preclean)\n",
    "\n",
    "Hofstede.df <- Hofstede.df.preclean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hierachical Clustering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Run hclust, Generate Dendogram"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first attempt is the simplest analysis using the dist() and hclust() functions to generate a hierarchy of grouped data. The cluster size is derived from a reading of the dendrogram, although there are automated ways of selecting the cluster number, shown further along in this demo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'Hofstede' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[1], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[43mHofstede\u001b[49m\u001b[38;5;241m.\u001b[39mdist \u001b[38;5;241m<\u001b[39m\u001b[38;5;241m-\u001b[39m dist(Hofstede\u001b[38;5;241m.\u001b[39mdf, method \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124meuclidean\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m      3\u001b[0m Hofstede\u001b[38;5;241m.\u001b[39mhClustModel \u001b[38;5;241m<\u001b[39m\u001b[38;5;241m-\u001b[39m hclust(Hofstede\u001b[38;5;241m.\u001b[39mdist, method \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mward.D2\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m      4\u001b[0m hClust\u001b[38;5;241m.\u001b[39mplot \u001b[38;5;241m<\u001b[39m\u001b[38;5;241m-\u001b[39m plot(Hofstede\u001b[38;5;241m.\u001b[39mhClustModel)\n",
      "\u001b[1;31mNameError\u001b[0m: name 'Hofstede' is not defined"
     ]
    }
   ],
   "source": [
    "Hofstede.dist <- dist(Hofstede.df, method = \"euclidean\")\n",
    "\n",
    "Hofstede.hClustModel <- hclust(Hofstede.dist, method = \"ward.D2\")\n",
    "hClust.plot <- plot(Hofstede.hClustModel)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select the Number of Cuts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can asses the diagram visually, noting the heights and deciding the clustering. In this case I've chosent 10 clusters, which the code overlays on the dendrogram."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Hofstede.dist <- dist(Hofstede.df, method = \"euclidean\")\n",
    "Hofstede.hClustModel <- hclust(Hofstede.dist, method = \"ward.D2\")\n",
    "hClust.plot <- plot(Hofstede.hClustModel)\n",
    "\n",
    "kClusters <- 10\n",
    "Hofstede.hClustModel.groups <- cutree(Hofstede.hClustModel, k = kClusters)\n",
    "rect.hclust(Hofstede.hClustModel, k = kClusters, border = \"red\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Join"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After clustering, the source data is joined with the grouping from the cluster model, so it can then be analyzed.\n",
    "\n",
    "A side note, the data can also be exported to CSV for analysis in other tools. In this case, I conducted some rudimentary analysis in Excel, forming pivot tables off the cluster groups to assess the validity of the grouping."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# join data against grouping\n",
    "Hofstede.hclust.combined <- as.data.frame(cbind(Hofstede.hClustModel.groups, Hofstede.df))\n",
    "#head(Hofstede.hclust.combined)\n",
    "\n",
    "# review data in Excel\n",
    "# export data\n",
    "#install.packages(\"rio\", dependencies = TRUE)\n",
    "#library(rio)\n",
    "#export(Hofstede.hclust.combined, \"Hofstede.hclust.combined.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The statements below output the groupings, and immediately one can see the similarities within the groups. They are grouped numerically, but obvious relationships show between countries, their regions and their cultures, even though the data does not contain specific categorization for either region or culture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# review data, natively\n",
    "#(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 1,])\n",
    "(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 2,])\n",
    "#(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 3,])\n",
    "#(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 4,])\n",
    "(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 5,])\n",
    "(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 6,])\n",
    "(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 7,])\n",
    "#(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 8,])\n",
    "#(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 9,])\n",
    "#(Hofstede.hclust.combined[Hofstede.hclust.combined$Hofstede.hClustModel.groups == 10,])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## K-Means"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are different ways of calculating K-Means, of which I work through two examples, the second having more refined graphs, as well as a vectorized equation for finding the elbow.\n",
    "\n",
    "In the resulting output, the first shows the clusters' averages, and the second shows the count of countries per cluster. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Calculate K-Means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set.seed(20)\n",
    "Hofstede.kMean <- kmeans(Hofstede.df[,2:5], 6, nstart = 20)\n",
    "Hofstede.kMean$centers\n",
    "\n",
    "plot(Hofsted.plot <- as.factor(Hofstede.kMean$cluster))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Plot Partial Two-dimensional result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are interesting plot types for displaying multi-dimensional results, which I hope to explore in later analyses, but for this review I am limiting myself to just a two-dimensional plot, seen below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Hofsted.plot <- as.factor(Hofstede.kMean$cluster)\n",
    "library(ggplot2)\n",
    "ggplot(Hofstede.df, aes(Individuality, Masculinity, color = Hofsted.plot)) + geom_point()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Determine Clusters - Find the 'elbow'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rather than visually estimating the number of clusters, there are programmatic techniques, two of which I explore below. Essentially, look for where the curve flattens, past which further clustering is assumed to be less informative. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Finding the Elbow - Version 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Hofstede.scaled <- scale(Hofstede.df[, 2:5])\n",
    "wss <- (nrow(Hofstede.scaled) - 1) * sum(apply(Hofstede.scaled, 2, var))\n",
    "for (i in 2:20)\n",
    "    wss[i] <- sum(kmeans(Hofstede.scaled, centers = i)$withinss)\n",
    "    plot(1:20, wss, type = \"b\", xlab = \"Number of Clusters\", ylab = \"Within groups sum of squares\")\n",
    "wss <- NA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Finding the Elbow - Version 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following code, the equation for calculatng clusters is both vectorized and uses piping, similar to lambda notation.\n",
    "\n",
    "Note, the code below is disabled, since it will not run on this platform, but works correctly in R itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#install.packages(c('broom','magrittr'), dependencies = TRUE)\n",
    "#library(broom)\n",
    "#library(magrittr)\n",
    "\n",
    "#kclusts <- NA\n",
    "#temp.df <- Hofstede.df[, 2:5]\n",
    "#kclusts <- data.frame(k = 1:20) %>% group_by(k) %>% do(kclust = kmeans(temp.df, .$k))\n",
    "\n",
    "#clusterings <- kclusts %>% group_by(k) %>% do(glance(.$kclust[[1]]))\n",
    "\n",
    "#library(ggplot2)\n",
    "#ggplot(clusterings, aes(k, tot.withinss)) + geom_line()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Set Clusters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Depending on the elbow - the graphs seem to change on each run - one can choose how many clusters would be optimal. in this case, I chose 6. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set clusters\n",
    "kClusters = 6\n",
    "\n",
    "# K-Means Cluster Analysis\n",
    "Hofstede.scaled.fit <- kmeans(Hofstede.scaled, kClusters)\n",
    "\n",
    "# get cluster means \n",
    "aggregate(Hofstede.scaled, by = list(Hofstede.scaled.fit$cluster), FUN = mean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Graph K-Means"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# append cluster assignment\n",
    "Hofstede.scaled.combined <- data.frame(Hofstede.scaled.fit$cluster, Hofstede.scaled)\n",
    "\n",
    "# store centers\n",
    "Hofstede.scaled.fit.centers <- as.data.frame(Hofstede.scaled.fit$centers)\n",
    "\n",
    "# plot\n",
    "library(ggplot2)\n",
    "centerColor <- 'Blue'\n",
    "plot <- ggplot(Hofstede.scaled.combined, aes(x = Individuality, y = Masculinity, color = Hofstede.scaled.fit$cluster)) + geom_point()\n",
    "plotCenter1 <- plot + geom_point(data = Hofstede.scaled.fit.centers, aes(x = Individuality, y = Masculinity), color = centerColor)\n",
    "plotCenter1 + geom_point(data = Hofstede.scaled.fit.centers, aes(x = Individuality, y = Masculinity), color = centerColor, size = 60, alpha = .3)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}