keyword_spotting_dataset_curation.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "keyword-spotting-dataset-curation.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Keyword Spotting Dataset Curation\n",
        "\n",
        "[![Open In Colab <](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/ShawnHymel/precise-hey-jorvon/blob/main/keyword_spotting_dataset_curation.ipynb)\n",
        "\n",
        "Use this tool to download the Google Speech Commands Dataset, combine it with your own keywords, and mix in some background noise.\n",
        "\n",
        " 1. Upload samples of your own keyword (optional)\n",
        " 2. Adjust parameters in the Settings cell\n",
        " 3. Run the rest of the cells! ('shift' + 'enter' on each cell)\n",
        "\n",
        "*Author:* Shawn Hymel\n",
        "\n",
        "*Date:* March 11, 2022\n",
        "\n",
        "*License:* [0BSD](https://opensource.org/licenses/0BSD)\n",
        "\n",
        "Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted.\n",
        "\n",
        "THE SOFTWARE IS PROVIDED \"AS IS\" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE."
      ],
      "metadata": {
        "id": "0uwNlV7aDRie"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Upload your own keyword samples\n",
        "You are welcome to use my [custom keyword dataset](https://github.com/ShawnHymel/custom-speech-commands-dataset), but note that it's limited and that I can't promise it will work well. If you want to use it, uncomment the `###Download custom dataset` cell below. You may also add your own recorded keywords to the extracted folder (`/content/custom_keywords`) to augment what's already there.\n",
        "\n",
        "If you'd rather upload your own custom keyword dataset, follow these instructions:\n",
        "\n",
        "On the left pane, in the file browser, create a directory structure containing space for your keyword audio samples. All samples for each keyword should be in a directory with that keyword's name.\n",
        "\n",
        "The audio samples should be `.wav` format, mono, and 1 second long. Bitrate and bitdepth should not matter. Samples shorter than 1 second will be padded with 0s, and samples longer than 1 second will be truncated to 1 second. The exact name of each `.wav` file does not matter, as they will be read, mixed with background noise, and saved to a separate file with an auto-generated name. Directory name does matter (it is used to determine the name of the class during neural network training).\n",
        "\n",
        "Right-click on each keyword directory and upload all of your samples. Your directory structor should look like the following:\n",
        "\n",
        "```\n",
        "/\n",
        "|- content\n",
        "|--- custom_keywords\n",
        "|----- keyword_1\n",
        "|------- 000.wav\n",
        "|------- 001.wav\n",
        "|------- ...\n",
        "|----- keyword_2\n",
        "|------- 000.wav\n",
        "|------- 001.wav\n",
        "|------- ...\n",
        "|----- ...\n",
        "```"
      ],
      "metadata": {
        "id": "xhIIxnciDkBW"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "ZA67ORiqDL3-"
      },
      "outputs": [],
      "source": [
        "### Settings (You probably do not need to change these)\n",
        "BASE_DIR = \"/content\"\n",
        "TEMP_DIR = \"temp_dir\"\n",
        "OUT_DIR = \"keywords_curated\"\n",
        "GOOGLE_DATASET_FILENAME = \"speech_commands_v0.02.tar.gz\"\n",
        "GOOGLE_DATASET_URL = \"http://download.tensorflow.org/data/\" + GOOGLE_DATASET_FILENAME\n",
        "GOOGLE_DATASET_DIR = \"google_speech_commands\"\n",
        "CUSTOM_KEYWORDS_FILENAME = \"main.zip\"\n",
        "CUSTOM_KEYWORDS_URL = \"https://github.com/ShawnHymel/custom-speech-commands-dataset/archive/\" + CUSTOM_KEYWORDS_FILENAME\n",
        "CUSTOM_KEYWORDS_DIR = \"custom_keywords\"\n",
        "CUSTOM_KEYWORDS_REPO_NAME = \"custom-speech-commands-dataset-main\"\n",
        "CURATION_SCRIPT = \"dataset-curation.py\"\n",
        "CURATION_SCRIPT_URL = \"https://raw.githubusercontent.com/ShawnHymel/ei-keyword-spotting/master/\" + CURATION_SCRIPT\n",
        "UTILS_SCRIPT_URL = \"https://raw.githubusercontent.com/ShawnHymel/ei-keyword-spotting/master/utils.py\"\n",
        "NUM_SAMPLES = 1500    # Target number of output samples per class\n",
        "WORD_VOL = 1.0        # Relative volume of word in output sample\n",
        "BG_VOL = 0.1          # Relative volume of noise in output sample\n",
        "SAMPLE_TIME = 1.5     # Time (seconds) of output sample\n",
        "SAMPLE_RATE = 16000   # Sample rate (Hz) of output sample\n",
        "BIT_DEPTH = \"PCM_16\"  # Options: [PCM_16, PCM_24, PCM_32, PCM_U8, FLOAT, DOUBLE]\n",
        "BG_DIR = \"_background_noise_\"\n",
        "TEST_RATIO = 0.2      # 20% reserved for test set, rest is for training\n",
        "EI_INGEST_TEST_URL = \"https://ingestion.edgeimpulse.com/api/test/data\"\n",
        "EI_INGEST_TRAIN_URL = \"https://ingestion.edgeimpulse.com/api/training/data\""
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "### Download Google Speech Commands Dataset\n",
        "!cd {BASE_DIR}\n",
        "!wget {GOOGLE_DATASET_URL}\n",
        "!mkdir {GOOGLE_DATASET_DIR}\n",
        "!echo \"Extracting...\"\n",
        "!tar xfz {GOOGLE_DATASET_FILENAME} -C {GOOGLE_DATASET_DIR}"
      ],
      "metadata": {
        "id": "pm-IhXsUD-2W",
        "outputId": "03cf98df-bcca-4458-ecf0-2afd5350fc0e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2025-01-29 01:03:29--  http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz\n",
            "Resolving download.tensorflow.org (download.tensorflow.org)... 173.194.194.207, 142.250.152.207, 172.217.214.207, ...\n",
            "Connecting to download.tensorflow.org (download.tensorflow.org)|173.194.194.207|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 2428923189 (2.3G) [application/gzip]\n",
            "Saving to: ‘speech_commands_v0.02.tar.gz’\n",
            "\n",
            "speech_commands_v0. 100%[===================>]   2.26G  44.3MB/s    in 22s     \n",
            "\n",
            "2025-01-29 01:03:51 (107 MB/s) - ‘speech_commands_v0.02.tar.gz’ saved [2428923189/2428923189]\n",
            "\n",
            "Extracting...\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "### Pull out background noise directory\n",
        "!cd {BASE_DIR}\n",
        "!mv \"{GOOGLE_DATASET_DIR}/{BG_DIR}\" \"{BG_DIR}\""
      ],
      "metadata": {
        "id": "h0Daat4qEAK0",
        "outputId": "3c4c7c06-c32c-4da2-e4e0-d275643fbe58",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "mv: cannot stat 'google_speech_commands/_background_noise_': No such file or directory\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "### (Optional) Download custom dataset--uncomment the code in this cell if you want to use my custom datase\n",
        "\n",
        "## Download, extract, and move dataset to separate directory\n",
        "# !cd {BASE_DIR}\n",
        "# !wget {CUSTOM_KEYWORDS_URL}\n",
        "# !echo \"Extracting...\"\n",
        "# !unzip -q {CUSTOM_KEYWORDS_FILENAME}\n",
        "# !mv \"{CUSTOM_KEYWORDS_REPO_NAME}/{CUSTOM_KEYWORDS_DIR}\" \"{CUSTOM_KEYWORDS_DIR}\""
      ],
      "metadata": {
        "id": "wDg23w7uGEa9"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "### User Settings (do change these)\n",
        "\n",
        "# Location of your custom keyword samples (e.g. \"/content/custom_keywords\")\n",
        "# Leave blank (\"\") for no custom keywords. set to the CUSTOM_KEYWORDS_DIR\n",
        "# variable to use samples from my custom-speech-commands-dataset repo.\n",
        "CUSTOM_DATASET_PATH = \"/content/custom_keywords\"\n",
        "\n",
        "# Comma separated words. Must match directory names (that contain samples).\n",
        "TARGETS = \"enola, not_enola\""
      ],
      "metadata": {
        "id": "rBjkSGZTGIT-"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "### Download curation and utils scripts\n",
        "!wget {CURATION_SCRIPT_URL}\n",
        "!wget {UTILS_SCRIPT_URL}"
      ],
      "metadata": {
        "id": "s3YkfwPYji8w",
        "outputId": "f2c3e857-f791-47ca-cd2b-817a4d5fb0d3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2025-01-29 01:06:39--  https://raw.githubusercontent.com/ShawnHymel/ei-keyword-spotting/master/dataset-curation.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 17427 (17K) [text/plain]\n",
            "Saving to: ‘dataset-curation.py’\n",
            "\n",
            "dataset-curation.py 100%[===================>]  17.02K  --.-KB/s    in 0.002s  \n",
            "\n",
            "2025-01-29 01:06:39 (8.97 MB/s) - ‘dataset-curation.py’ saved [17427/17427]\n",
            "\n",
            "--2025-01-29 01:06:39--  https://raw.githubusercontent.com/ShawnHymel/ei-keyword-spotting/master/utils.py\n",
            "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ...\n",
            "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 3966 (3.9K) [text/plain]\n",
            "Saving to: ‘utils.py’\n",
            "\n",
            "utils.py            100%[===================>]   3.87K  --.-KB/s    in 0s      \n",
            "\n",
            "2025-01-29 01:06:39 (49.9 MB/s) - ‘utils.py’ saved [3966/3966]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "### Perform curation and mixing of samples with background noise\n",
        "!cd {BASE_DIR}\n",
        "!python {CURATION_SCRIPT} \\\n",
        "  -t \"{TARGETS}\" \\\n",
        "  -n {NUM_SAMPLES} \\\n",
        "  -w {WORD_VOL} \\\n",
        "  -g {BG_VOL} \\\n",
        "  -s {SAMPLE_TIME} \\\n",
        "  -r {SAMPLE_RATE} \\\n",
        "  -e {BIT_DEPTH} \\\n",
        "  -b \"{BG_DIR}\" \\\n",
        "  -o \"{TEMP_DIR}\" \\\n",
        "  \"{GOOGLE_DATASET_DIR}\" \\\n",
        "  \"{CUSTOM_DATASET_PATH}\""
      ],
      "metadata": {
        "id": "Nm6HVnPIjmGw",
        "outputId": "0ac6ffbb-8803-4c47-baed-154ab285f35d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "-----------------------------------------------------------------------\n",
            "Keyword Dataset Curation Tool\n",
            "v0.1\n",
            "-----------------------------------------------------------------------\n",
            "ERROR: Target word 'enola' not found as subdirectory in input directories. Exiting.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "### Split and move samples to train and test folders\n",
        "!cd {BASE_DIR}\n",
        "\n",
        "# Imports\n",
        "import os\n",
        "import random\n",
        "import shutil\n",
        "\n",
        "# Seed with system time\n",
        "random.seed()\n",
        "\n",
        "# Remove output directory (start from scratch)\n",
        "if os.path.exists(OUT_DIR) and os.path.isdir(OUT_DIR):\n",
        "  shutil.rmtree(OUT_DIR)\n",
        "\n",
        "# Go through each category in our curated dataset\n",
        "for dir in os.listdir(TEMP_DIR):\n",
        "\n",
        "  # Ignore notebook checkpoint\n",
        "  if dir == \".ipynb_checkpoints\":\n",
        "    continue\n",
        "\n",
        "  # Create output directories\n",
        "  os.makedirs(os.path.join(OUT_DIR, \"train\", dir))\n",
        "  os.makedirs(os.path.join(OUT_DIR, \"test\", dir))\n",
        "\n",
        "  # Create list of files for one category\n",
        "  paths = []\n",
        "  for filename in os.listdir(os.path.join(TEMP_DIR, dir)):\n",
        "    paths.append(os.path.join(TEMP_DIR, dir, filename))\n",
        "\n",
        "  # Shuffle and divide into test and training sets\n",
        "  random.shuffle(paths)\n",
        "  num_test_samples = int(TEST_RATIO * len(paths))\n",
        "  test_paths = paths[:num_test_samples]\n",
        "  train_paths = paths[num_test_samples:]\n",
        "\n",
        "  # Copy files\n",
        "  for file in train_paths:\n",
        "    out_path = os.path.join(OUT_DIR, \"train\", dir, os.path.basename(file))\n",
        "    shutil.copy(file, out_path)\n",
        "  for file in test_paths:\n",
        "    out_path = os.path.join(OUT_DIR, \"test\", dir, os.path.basename(file))\n",
        "    shutil.copy(file, out_path)"
      ],
      "metadata": {
        "id": "96tiG5ynv6t5"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "### Zip dataset for easy download\n",
        "!cd {BASE_DIR}\n",
        "!zip -r -q \"{OUT_DIR}.zip\" \"{OUT_DIR}\""
      ],
      "metadata": {
        "id": "ZbKgs5LRjoW1",
        "outputId": "5081a05a-c5a6-4029-9edf-c5dc0a3eb12a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "zip error: Nothing to do! (try: zip -r -q keywords_curated.zip . -i keywords_curated)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "7B3qQxNCyF1D"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}