huggingtweets-demo.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.2"
    },
    "colab": {
      "name": "huggingtweets-demo.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/borisdayma/huggingtweets/blob/master/huggingtweets-demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tvHAWMW78AG9"
      },
      "source": [
        "# HuggingTweets - Train a model to generate tweets\n",
        "\n",
        "Choose your favorite Twitter account and train a language model to write new tweets based on their unique voice in just 5 minutes.\n",
        "\n",
        "Here is an example where I fine-tuned a neural network to predict Elon Musk's next breakthrough 😉\n",
        "\n",
        "![huggingtweets illustration](https://raw.githubusercontent.com/borisdayma/huggingtweets/master/img/example.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qwj6GL6LJvDI"
      },
      "source": [
        "##👉 To start the demo, click on menu on top, \"Runtime\" → \"Run all\"\n",
        "\n",
        "###⚠️ On mobile you also need to set \"Desktop View\"\n",
        "\n",
        "###🙏 Please let me know if the demo breaks and requires an upgrade → [@borisdayma](https://twitter.com/borisdayma)\n",
        "\n",
        "###⚠️ UPDATE: Unfortunately Twitter API is not free anymore and it would now cost $5,000 per month to keep this app working 😢"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "jupyter": {
          "source_hidden": true
        },
        "id": "ZSCf6QyF8AG-"
      },
      "source": [
        "#@title ⠀ {display-mode: \"form\"}\n",
        "\n",
        "def stylize():\n",
        "    \"Handle dark mode\"\n",
        "    display(HTML('''\n",
        "    <style>\n",
        "    :root {\n",
        "        --table_bg: #EBF8FF;\n",
        "    }\n",
        "    html[theme=dark] {\n",
        "        --colab-primary-text-color: #d5d5d5;\n",
        "        --table_bg: #2A4365;\n",
        "    }\n",
        "    .jupyter-widgets {\n",
        "        color: var(--colab-primary-text-color);\n",
        "    }\n",
        "    table {\n",
        "        border-collapse: collapse !important;\n",
        "    }\n",
        "    td {\n",
        "        text-align:left !important;\n",
        "        border: solid var(--table_bg) !important;\n",
        "        border-width: 1px 0 !important;\n",
        "        padding: 6px !important;\n",
        "    }\n",
        "    tr:nth-child(even) {\n",
        "        background-color: var(--table_bg) !important;\n",
        "    }\n",
        "    .table_odd {\n",
        "        background-color: var(--table_bg) !important;\n",
        "        margin: 0 !important;\n",
        "    }\n",
        "    .table_even {\n",
        "        border: solid var(--table_bg) !important;\n",
        "        border-width: 1px 0 !important;\n",
        "        margin: 0 !important;\n",
        "    }\n",
        "    .jupyter-widgets {\n",
        "        margin: 6px;\n",
        "    }\n",
        "    .widget-html-content {\n",
        "        font-size: var(--colab-chrome-font-size) !important;\n",
        "        line-height: 1.24 !important;\n",
        "    }\n",
        "    </style>'''))\n",
        "\n",
        "def print_html(x):\n",
        "    \"Better printing\"\n",
        "    x = x.replace('\\n', '<br>')\n",
        "    display(HTML(x))\n",
        "\n",
        "# Check we use GPU\n",
        "import torch\n",
        "from IPython.display import display, HTML, Javascript, clear_output\n",
        "if not torch.cuda.is_available():\n",
        "    print_html('Error: GPU was not found\\n1/ click on the \"Runtime\" menu and \"Change runtime type\"\\n'\\\n",
        "          '2/ set \"Hardware accelerator\" to \"GPU\" and click \"save\"\\n3/ click on the \"Runtime\" menu, then \"Run all\" (below error should disappear)')\n",
        "    raise ValueError('No GPU available')\n",
        "else:\n",
        "    # colab requires special handling\n",
        "    try:\n",
        "        import google.colab\n",
        "        IN_COLAB = True\n",
        "    except:\n",
        "        IN_COLAB = False\n",
        "\n",
        "    # Install dependencies (mainly for colab)\n",
        "    if IN_COLAB:\n",
        "        !pip install torch transformers wandb -qqq\n",
        "        !curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash\n",
        "        !sudo apt-get install git-lfs\n",
        "\n",
        "    import ipywidgets as widgets\n",
        "    from IPython import get_ipython\n",
        "    import json\n",
        "    import urllib3\n",
        "    import pathlib\n",
        "    import shutil\n",
        "    import requests\n",
        "    import os\n",
        "    import re\n",
        "    import random\n",
        "    import wandb\n",
        "    from urllib.parse import urlencode\n",
        "    from PIL import Image\n",
        "    from io import BytesIO\n",
        "\n",
        "    stylize()\n",
        "\n",
        "    log_debug = widgets.Output()\n",
        "\n",
        "    # Have global access\n",
        "    trainer = None\n",
        "    artifact_dataset = None\n",
        "    metadata = {}\n",
        "    card_val = {}\n",
        "    model_preview = None\n",
        "    hfapi, token, namespace = None, None, None\n",
        "    handles_processed = []\n",
        "    model_url = ''\n",
        "    bot = 'bot'\n",
        "\n",
        "    # W&B variables\n",
        "    WANDB_PROJECT = 'huggingtweets'\n",
        "    WANDB_NOTES = \"Github repo: https://github.com/borisdayma/huggingtweets\"\n",
        "    WANDB_ENTITY = 'wandb'\n",
        "    HW_VERSION = 0.6\n",
        "    os.environ['WANDB_NOTEBOOK_NAME'] = 'huggingtweets-demo.ipynb'  # used in wandb cli\n",
        "\n",
        "    # HYPER-PARAMETERS\n",
        "    ALLOW_NEW_LINES = False     # seems to work better\n",
        "    LEARNING_RATE = 1.372e-4\n",
        "    EPOCHS = 4\n",
        "\n",
        "    def fix_text(text):\n",
        "        text = text.replace('&amp;', '&')\n",
        "        text = text.replace('&lt;', '<')\n",
        "        text = text.replace('&gt;', '>')\n",
        "        return text\n",
        "\n",
        "    def html_table(data, title=None):\n",
        "        'Create a html table'\n",
        "        width_twitter = '75px'\n",
        "        def html_cell(i, twitter_button=False):\n",
        "            nl = \"\\n\"\n",
        "            return f'<td style=\"width:{width_twitter}\">{i}</td>' if twitter_button else f'<td>{i.replace(nl, \"<br>\")}</td>'\n",
        "        def html_row(row):\n",
        "            return f'<tr>{\"\".join(html_cell(r, not i if len(row)>1 else False) for i,r in enumerate(row))}</tr>'\n",
        "        body = f'<table style=\"width:100%\">{\"\".join(html_row(r) for r in data)}</table>'\n",
        "        title_html = f'<h3>{title}</h3>' if title else ''\n",
        "        html = '<html><body>' + title_html + body + '</body></html>'\n",
        "        return(html)\n",
        "\n",
        "    def clean_tweet(tweet, allow_new_lines = ALLOW_NEW_LINES):\n",
        "        bad_start = ['http:', 'https:']\n",
        "        for w in bad_start:\n",
        "            tweet = re.sub(f\" {w}\\\\S+\", \"\", tweet)      # removes white space before url\n",
        "            tweet = re.sub(f\"{w}\\\\S+ \", \"\", tweet)      # in case a tweet starts with a url\n",
        "            tweet = re.sub(f\"\\n{w}\\\\S+ \", \"\", tweet)    # in case the url is on a new line\n",
        "            tweet = re.sub(f\"\\n{w}\\\\S+\", \"\", tweet)     # in case the url is alone on a new line\n",
        "            tweet = re.sub(f\"{w}\\\\S+\", \"\", tweet)       # any other case?\n",
        "        tweet = re.sub(' +', ' ', tweet)                # replace multiple spaces with one space\n",
        "        if not allow_new_lines:                         # TODO: predictions seem better without new lines\n",
        "            tweet = ' '.join(tweet.split())\n",
        "        return tweet.strip()\n",
        "\n",
        "    def boring_tweet(tweet):\n",
        "        \"Check if this is a boring tweet\"\n",
        "        boring_stuff = ['http', '@', '#']\n",
        "        not_boring_words = len([None for w in tweet.split() if all(bs not in w.lower() for bs in boring_stuff)])\n",
        "        return not_boring_words < 3\n",
        "\n",
        "    def create_model_card(card_val, output_dir):\n",
        "        model_card_url = 'https://github.com/borisdayma/huggingtweets/raw/master/model_card/README.md'\n",
        "        model_card = requests.get(model_card_url).content.decode('utf-8')\n",
        "        for k, v in card_val.items():\n",
        "            model_card = model_card.replace(k, v)\n",
        "        with open(f'{output_dir}/README.md', 'w') as f:\n",
        "            f.write(model_card)\n",
        "\n",
        "    def on_preview_clicked(b):\n",
        "        global model_preview\n",
        "        global hfapi, token, namespace\n",
        "        model_preview = f\"http://www.huggingtweets.com/{'-'.join(sorted(handles_processed))}/{b.url_id}/predictions.png\"\n",
        "        card_val['SOCIAL_LINK'] = model_preview\n",
        "        create_model_card(card_val, '-'.join(sorted(handles_processed)))\n",
        "        commit_files('-'.join(sorted(handles_processed)), f'Update model preview')\n",
        "\n",
        "        # Reset view\n",
        "        log_model.clear_output(wait=True)\n",
        "        with log_model:\n",
        "            print_html(\"<h2>Model Preview (select a tweet to update)</h2>\")\n",
        "            show_image_preview(model_preview)\n",
        "\n",
        "    def show_image_preview(url):\n",
        "        response = requests.get(url)\n",
        "        img = Image.open(BytesIO(response.content))\n",
        "        display(img.resize((560,293)))\n",
        "\n",
        "    def commit_files(model_name, message):\n",
        "        with log_debug:\n",
        "            %cd $model_name\n",
        "            !git add .\n",
        "            !git commit -m \"{message}\"\n",
        "            !git push\n",
        "            %cd ..\n",
        "\n",
        "    def create_button(url_id):\n",
        "        layout = widgets.Layout(width='70px', min_width='70px') #set width and height\n",
        "        button = widgets.Button(\n",
        "            description='Preview',\n",
        "            button_style='info',\n",
        "            layout = layout,\n",
        "            tooltip = 'Set as model preview'\n",
        "        )\n",
        "        button.url_id = url_id\n",
        "        button.on_click(on_preview_clicked)\n",
        "        return button\n",
        "\n",
        "    def ensure_widgets_updated(n_iter=5):\n",
        "        '''ensure we get correct inputs and states are updated'''\n",
        "        pass\n",
        "        # used to be necessary in colab ; seems not needed anymore and create issues like in Jupyter\n",
        "        #if IN_COLAB:  # not a problem with jupyter + create print output issues\n",
        "        #    for _ in range(n_iter):\n",
        "        #        get_ipython().kernel.do_one_iteration()\n",
        "\n",
        "\n",
        "    def dl_tweets():\n",
        "        for handle_widget in handle_widgets:\n",
        "            handle_widget.disabled = True\n",
        "        run_dl_tweets.disabled = True\n",
        "        run_dl_tweets.button_style = 'primary'\n",
        "        ensure_widgets_updated()\n",
        "        global handles_processed\n",
        "        handles = []\n",
        "        for handle_widget in handle_widgets:\n",
        "            handle = handle_widget.value.strip()\n",
        "            if not handle: continue\n",
        "            handle = handle[1:] if handle and handle[0] == '@' else handle\n",
        "            handles.append(handle.lower().strip())\n",
        "\n",
        "        log_dl_tweets.clear_output(wait=True)\n",
        "\n",
        "        success_try = False\n",
        "\n",
        "        with log_dl_tweets:\n",
        "            try:\n",
        "                cool_tweets = []\n",
        "                handles_processed = []\n",
        "                raw_tweets = []\n",
        "                user_names = []\n",
        "                n_tweets_dl = []\n",
        "                n_retweets = []\n",
        "                n_short_tweets = []\n",
        "                n_tweets_kept = []\n",
        "                i = 0\n",
        "                global card_val\n",
        "                card_val = {'USER_PROFILE_1': '', 'DISPLAY_1': 'none',\n",
        "                            'USER_PROFILE_2': '', 'DISPLAY_2': 'none',\n",
        "                            'USER_PROFILE_3': '', 'DISPLAY_3': 'none'}\n",
        "                for handle in handles:\n",
        "                    if handle in handles_processed: continue\n",
        "                    i += 1\n",
        "                    handles_processed.append(handle)\n",
        "                    print_html(f'\\nDownloading @{handle} tweets... This should take no more than a minute!')\n",
        "                    http = urllib3.PoolManager(retries=urllib3.Retry(3))\n",
        "                    res = http.request(\"GET\", f\"http://us-central1-huggingtweets.cloudfunctions.net/get_tweets?handle={handle}&force=1\")\n",
        "                    res = json.loads(res.data.decode('utf-8'))\n",
        "                    user_names.append(res['user_name'])\n",
        "                    card_val[f'USER_PROFILE_{i}'] = res['user_profile'].replace('_normal.', '_400x400.')\n",
        "                    card_val[f'DISPLAY_{i}'] = 'inherit'\n",
        "\n",
        "                    all_tweets = res['tweets']\n",
        "                    raw_tweets.append(all_tweets)\n",
        "                    curated_tweets = [fix_text(tweet) for tweet in all_tweets]\n",
        "                    #log_dl_tweets.clear_output(wait=True)\n",
        "                    print_html(f\"\\n{res['n_tweets']} tweets from @{handle} downloaded!\\n\\n\")\n",
        "\n",
        "                    # create dataset\n",
        "                    clean_tweets = [clean_tweet(tweet) for tweet in curated_tweets]\n",
        "                    cool_tweets.append([tweet for tweet in clean_tweets if not boring_tweet(tweet)])\n",
        "\n",
        "                    # save count\n",
        "                    n_tweets_dl.append(str(res['n_tweets']))\n",
        "                    n_retweets.append(str(res['n_RT']))\n",
        "                    n_short_tweets.append(str(len(all_tweets) - len(cool_tweets[-1])))\n",
        "                    n_tweets_kept.append(str(len(cool_tweets[-1])))\n",
        "\n",
        "                    # display a few tweets\n",
        "                    display(HTML(html_table([[t] for t in curated_tweets[:8]])))\n",
        "\n",
        "                    if len('<|endoftext|>'.join(cool_tweets[-1])) < 6000:\n",
        "                        # need about 4000 chars for one data sample (but depends on spaces, etc)\n",
        "                        raise ValueError(f\"Error: this user does not have enough tweets to train a Neural Network\\n{res['n_tweets']} tweets downloaded, including {res['n_RT']} RT's and {len(all_tweets) - len(cool_tweets)} boring tweets... only {len(cool_tweets)} tweets kept!\")\n",
        "                    if len('<|endoftext|>'.join(cool_tweets[-1])) < 40000:\n",
        "                        print_html('\\n<b>Warning: this user does not have many tweets which may impact the results of the Neural Network</b>\\n')\n",
        "\n",
        "                    print_html(f\"\\n{n_tweets_dl[-1]} tweets downloaded, including {n_retweets[-1]} RT's and {n_short_tweets[-1]} short tweets... keeping {n_tweets_kept[-1]} tweets\\n\\n\\n\")\n",
        "                    ensure_widgets_updated()  # for auto-scroll\n",
        "\n",
        "                global bot\n",
        "                bot = 'bot' if len(handles_processed) == 1 else 'cyborg'\n",
        "\n",
        "                # save user info\n",
        "                card_val['USER_HANDLE'] = '-'.join(sorted(handles_processed))\n",
        "                card_val['USER_NAME'] = ' & '.join(user_names)\n",
        "                card_val['BOT'] = bot.upper()\n",
        "                card_val['SOCIAL_LINK'] = res['social_link']\n",
        "                card_val['TABLE_USER'] = ' | '.join(user_names)\n",
        "                card_val['TABLE_SPLIT'] = ' | '.join(['---'] * len(user_names))\n",
        "\n",
        "                # Save data info\n",
        "                card_val['TWEETS_DL'] = ' | '.join(n_tweets_dl)\n",
        "                card_val['RETWEETS'] = ' | '.join(n_retweets)\n",
        "                card_val['SHORT_TWEETS'] = ' | '.join(n_short_tweets)\n",
        "                card_val['TWEETS_KEPT'] = ' | '.join(n_tweets_kept)\n",
        "\n",
        "                # create a file based on multiple epochs with tweets mixed up\n",
        "                seed_data = random.randint(0,2**32-1)\n",
        "                dataRandom = random.Random(seed_data)\n",
        "                total_text = '<|endoftext|>'\n",
        "                all_handle_tweets = []\n",
        "                epoch_len = max(len(''.join(cool_tweet)) for cool_tweet in cool_tweets)\n",
        "                for _ in range(EPOCHS):\n",
        "                    for cool_tweet in cool_tweets:\n",
        "                        dataRandom.shuffle(cool_tweet)\n",
        "                        current_tweet = cool_tweet\n",
        "                        current_len = len(''.join(current_tweet))\n",
        "                        while current_len < epoch_len:\n",
        "                            for t in cool_tweet:\n",
        "                                current_tweet.append(t)\n",
        "                                current_len += len(t)\n",
        "                                if current_len >= epoch_len: break\n",
        "                        dataRandom.shuffle(current_tweet)\n",
        "                        all_handle_tweets.extend(current_tweet)\n",
        "                total_text += '<|endoftext|>'.join(all_handle_tweets) + '<|endoftext|>'\n",
        "\n",
        "                print_html('\\nCreating dataset...')\n",
        "                ensure_widgets_updated() # for auto-scroll\n",
        "\n",
        "                # log dataset\n",
        "                with log_debug:\n",
        "                    wandb.login(key=res['wandb'])\n",
        "\n",
        "                    with wandb.init(name=f\"@{'-'.join(handles_processed)}-preprocess\",\n",
        "                                    job_type='preprocess',\n",
        "                                    config={'huggingtweets version':HW_VERSION,\n",
        "                                            'handle':', '.join(handles_processed),\n",
        "                                            'seed data':seed_data},\n",
        "                                    project = WANDB_PROJECT,\n",
        "                                    entity = WANDB_ENTITY,\n",
        "                                    notes = WANDB_NOTES,\n",
        "                                    reinit=True) as run:\n",
        "                        # log raw tweets as input\n",
        "                        global metadata\n",
        "                        metadata={'handle':', '.join(handles_processed),\n",
        "                                  'huggingtweets version': HW_VERSION}\n",
        "                        artifact_input = wandb.Artifact(\n",
        "                            f\"tweets-{'-'.join(sorted(handles_processed))}\",\n",
        "                            type='raw-dataset',\n",
        "                            description=f\"Raw tweets from {', '.join(handles_processed)} downloaded with Tweepy\",\n",
        "                            metadata=metadata)\n",
        "                        with artifact_input.new_file('tweets.txt') as f:\n",
        "                            json.dump(raw_tweets, f, indent=0)\n",
        "                        run.use_artifact(artifact_input)\n",
        "\n",
        "                        # log dataset as output\n",
        "                        metadata={'handle':handle,\n",
        "                                  'seed data': seed_data,\n",
        "                                  'epochs': EPOCHS,\n",
        "                                  'huggingtweets version': HW_VERSION}\n",
        "                        global artifact_dataset\n",
        "                        artifact_dataset = wandb.Artifact(\n",
        "                            f\"dataset-{'-'.join(sorted(handles_processed))}\",\n",
        "                            type='train-dataset',\n",
        "                            description=f\"Dataset created from tweets of {', '.join(handles_processed)}\",\n",
        "                            metadata=metadata)\n",
        "                        with open(f\"data_{'-'.join(sorted(handles_processed))}_train.txt\", 'w') as f:\n",
        "                            f.write(total_text)\n",
        "                        artifact_dataset.add_file(f\"data_{'-'.join(sorted(handles_processed))}_train.txt\")\n",
        "                        run.log_artifact(artifact_dataset)\n",
        "\n",
        "                        # keep track of url\n",
        "                        wandb_url = wandb.run.get_url()\n",
        "                        card_val['WANDB_PREPROCESS'] = wandb_url\n",
        "\n",
        "                success_try = True\n",
        "\n",
        "            except Exception as e:\n",
        "                print('\\nAn error occured...\\n')\n",
        "                print(e)\n",
        "                run_dl_tweets.button_style = 'danger'\n",
        "\n",
        "        if success_try:\n",
        "            run_dl_tweets.button_style = 'success'\n",
        "            log_finetune.clear_output(wait=True)\n",
        "            with log_finetune:\n",
        "                print_html('\\nFine-tune your model by clicking on \"Train Neural Network\"')\n",
        "            run_finetune.disabled = False\n",
        "            with log_dl_tweets:\n",
        "                print_html(f\"\\n🎉 Dataset created\")\n",
        "\n",
        "        else:\n",
        "            display(log_debug)\n",
        "\n",
        "        for handle_widget in handle_widgets:\n",
        "            handle_widget.disabled = False\n",
        "        run_dl_tweets.disabled = False\n",
        "\n",
        "    handle_widgets = [widgets.Text(value='@elonmusk',\n",
        "                                   placeholder='Enter twitter handle'),\n",
        "                      widgets.Text(placeholder='Optional: 2nd handle for humanoids'),\n",
        "                      widgets.Text(placeholder='Optional: 3rd handle for humanoids')]\n",
        "\n",
        "    run_dl_tweets = widgets.Button(\n",
        "        description='Download tweets',\n",
        "        button_style='primary')\n",
        "    def on_run_dl_tweets_clicked(b):\n",
        "        dl_tweets()\n",
        "    run_dl_tweets.on_click(on_run_dl_tweets_clicked)\n",
        "\n",
        "    log_restart = widgets.Output()\n",
        "    log_dl_tweets = widgets.Output()\n",
        "\n",
        "    def finetune():\n",
        "        # transformers imports later as wandb needs to have logged in\n",
        "        import transformers\n",
        "        from transformers import (\n",
        "            AutoTokenizer, AutoModelForCausalLM,\n",
        "            TextDataset, DataCollatorForLanguageModeling,\n",
        "            Trainer, TrainingArguments,\n",
        "            get_cosine_schedule_with_warmup)\n",
        "        from huggingface_hub.hf_api import HfApi\n",
        "\n",
        "        if run_finetune.button_style == 'success':\n",
        "            # user double clicked before start of function\n",
        "            return\n",
        "\n",
        "        for handle_widget in handle_widgets:\n",
        "            handle_widget.disabled = True\n",
        "        run_dl_tweets.disabled = True\n",
        "        run_finetune.disabled = True\n",
        "        run_finetune.button_style = 'primary'\n",
        "\n",
        "        global handles_processed\n",
        "        global model_url\n",
        "        model_url = f\"http://huggingface.co/huggingtweets/{'-'.join(sorted(handles_processed))}\"\n",
        "        log_finetune.clear_output(wait=True)\n",
        "        clear_output(wait=True)\n",
        "\n",
        "        success_try = False\n",
        "\n",
        "        with log_finetune:\n",
        "            print_html(f\"\\nTraining Neural Network on @{' & @'.join(handles_processed)} tweets... This could take up to 3-5 minutes!\\n\")\n",
        "            progress = widgets.FloatProgress(value=0.1, min=0.0, max=1.0, bar_style = 'info')\n",
        "            label_progress = widgets.Label('Downloading pre-trained neural network...')\n",
        "            display(widgets.HBox([progress, label_progress]))\n",
        "\n",
        "        with log_debug:\n",
        "            try:\n",
        "                # Setting up pre-trained neural network\n",
        "                global trainer\n",
        "                tokenizer = AutoTokenizer.from_pretrained('gpt2')\n",
        "                model = AutoModelForCausalLM.from_pretrained('gpt2', cache_dir=pathlib.Path('cache').resolve())\n",
        "                block_size = tokenizer.model_max_length\n",
        "                train_dataset = TextDataset(tokenizer=tokenizer, file_path=f\"data_{'-'.join(sorted(handles_processed))}_train.txt\", block_size=block_size, overwrite_cache=True)\n",
        "                data_collator = DataCollatorForLanguageModeling(tokenizer=tokenizer, mlm=False)\n",
        "                seed = random.randint(0,2**32-1)\n",
        "                training_args = TrainingArguments(\n",
        "                    output_dir=f\"output/{'-'.join(sorted(handles_processed))}\",\n",
        "                    overwrite_output_dir=True,\n",
        "                    do_train=True,\n",
        "                    num_train_epochs=1,\n",
        "                    per_device_train_batch_size=1,\n",
        "                    prediction_loss_only=True,\n",
        "                    logging_steps=5,\n",
        "                    save_steps=0,\n",
        "                    seed=seed,\n",
        "                    learning_rate = LEARNING_RATE)\n",
        "\n",
        "                # create wandb run (before it's done automatically by Trainer)\n",
        "                combined_dict = {**model.config.to_dict(), **training_args.to_sanitized_dict()}\n",
        "                run = wandb.init(name=f\"@{'-'.join(handles_processed)}-train\",\n",
        "                                 job_type='train',\n",
        "                                 config={'huggingtweets version':HW_VERSION,\n",
        "                                         'pytorch version': torch.__version__,\n",
        "                                         'transformers version': transformers.__version__,\n",
        "                                         'handle':', '.join(handles_processed),\n",
        "                                         **combined_dict},\n",
        "                                 project = WANDB_PROJECT,\n",
        "                                 entity = WANDB_ENTITY,\n",
        "                                 notes = WANDB_NOTES,\n",
        "                                 reinit=True)\n",
        "\n",
        "                # keep track of url\n",
        "                wandb_url = wandb.run.get_url()\n",
        "                card_val['WANDB_TRAIN'] = wandb_url\n",
        "\n",
        "                # Set-up Trainer\n",
        "                os.environ['WANDB_WATCH'] = 'false'  # used in Trainer\n",
        "                trainer = Trainer(\n",
        "                    model=model,\n",
        "                    tokenizer=tokenizer,\n",
        "                    args=training_args,\n",
        "                    data_collator=data_collator,\n",
        "                    train_dataset=train_dataset)\n",
        "\n",
        "                # Update lr scheduler\n",
        "                train_dataloader = trainer.get_train_dataloader()\n",
        "                num_train_steps = len(train_dataloader)\n",
        "                trainer.create_optimizer_and_scheduler(num_train_steps)\n",
        "                trainer.lr_scheduler = get_cosine_schedule_with_warmup(\n",
        "                    trainer.optimizer,\n",
        "                    num_warmup_steps=0,\n",
        "                    num_training_steps=num_train_steps)\n",
        "\n",
        "                progress.value = 0.3\n",
        "                label_progress.value = 'Logging input artifacts to W&B...'\n",
        "\n",
        "                # log dataset and pretrained model\n",
        "                artifact_dataset.wait()\n",
        "                run.use_artifact(artifact_dataset)\n",
        "                artifact_gpt2 = wandb.Artifact(\n",
        "                    f'gpt2',\n",
        "                    type='pretrained-model',\n",
        "                    description=f'Pretrained model from OpenAI downloaded from 🤗 Transformers: https://huggingface.co/gpt2',\n",
        "                    metadata={'huggingtweets version': HW_VERSION})\n",
        "                artifact_gpt2.add_dir('cache', name='gpt2')\n",
        "                run.use_artifact(artifact_gpt2)\n",
        "                progress.value = 0.4\n",
        "                label_progress.value = 'Training neural network...'\n",
        "\n",
        "                p_start, p_end = 0.4, 0.9\n",
        "                def progressify(f):\n",
        "                    \"Control progress bar when calling f\"\n",
        "                    def inner(*args, **kwargs):\n",
        "                        if trainer.state.epoch is not None:\n",
        "                            # we only have one epoch, EPOCHS is built into dataset\n",
        "                            progress.value = p_start + trainer.state.epoch * (p_end - p_start)\n",
        "                        return f(*args, **kwargs)\n",
        "                    return inner\n",
        "\n",
        "                trainer.training_step = progressify(trainer.training_step)\n",
        "\n",
        "                # Training neural network\n",
        "                with log_finetune:\n",
        "                    display(wandb.run)\n",
        "                    print_html('\\n')\n",
        "                    display(widgets.HBox([progress, label_progress]))\n",
        "                trainer.train()\n",
        "\n",
        "                # set model config parameters\n",
        "                trainer.model.config.task_specific_params['text-generation'] = {\n",
        "                    'do_sample': True,\n",
        "                    'min_length': 10,\n",
        "                    'max_length': 160,\n",
        "                    'temperature': 1.,\n",
        "                    'top_p': 0.95,\n",
        "                    'prefix': '<|endoftext|>'}\n",
        "\n",
        "                # create model repo\n",
        "                label_progress.value = 'Setting up Hugging Face model repo'\n",
        "                model_name = '-'.join(sorted(handles_processed))\n",
        "                shutil.rmtree(model_name, ignore_errors=True)\n",
        "                model_path = pathlib.Path(model_name)\n",
        "                try:\n",
        "                    hfapi = HfApi()\n",
        "                    user, namespace = 'huggingtweets-app', 'hf_huggingtweets'\n",
        "                    assert hfapi.whoami(namespace)['name'] == user, \"Could not log into Hugging Face\"\n",
        "                    url = hfapi.create_repo(token=namespace, repo_id=f\"huggingtweets/{model_name}\", exist_ok=True)\n",
        "                    !GIT_LFS_SKIP_SMUDGE=1 git clone https://$user:$namespace@huggingface.co/huggingtweets/$model_name\n",
        "\n",
        "                except Exception as e:\n",
        "                    with log_finetune:\n",
        "                        print_html(f'\\n<b>Could not create a model repo</b>\\n{e}')\n",
        "                # remove non-git files\n",
        "                for f in pathlib.Path(model_name).glob('*'):\n",
        "                    if f.suffix:\n",
        "                        f.unlink()\n",
        "\n",
        "                # save new model files\n",
        "                trainer.save_model(model_name)\n",
        "\n",
        "                # log model to huggingface\n",
        "                label_progress.value = 'Committing model to Hugging Face (up to 1mn)'\n",
        "                hf_urls = []\n",
        "                try:\n",
        "                    create_model_card(card_val, model_name)\n",
        "\n",
        "                    # upload files\n",
        "                    !git config --global user.email \"boris.dayma@gmail.com\"\n",
        "                    !git config --global user.name \"huggingtweets\"\n",
        "                    commit_files(model_name, f'New model from {wandb_url}')\n",
        "\n",
        "                    # get files url\n",
        "                    assert model_path.is_dir(), f\"Expected {model_path} to be a directory\"\n",
        "                    hf_urls = [f'https://huggingface.co/huggingtweets/{model_name}/resolve/main/{f.name}' for f in model_path.glob('*') if f.suffix]\n",
        "\n",
        "                except Exception as e:\n",
        "                    with log_finetune:\n",
        "                        print_html(f'\\n<b>Could not upload the model to Hugging Face</b>\\n{e}')\n",
        "\n",
        "                # log model to W&B\n",
        "                label_progress.value = 'Logging model to W&B...'\n",
        "                global metadata\n",
        "                metadata={'model url':model_url,\n",
        "                          'seed trainer':seed,\n",
        "                          **metadata}\n",
        "                artifact_trained = wandb.Artifact(\n",
        "                    model_name,\n",
        "                    type='finetuned-model',\n",
        "                    description=f\"Model fine-tuned on tweets from @{' & @'.join(handles_processed)}\",\n",
        "                    metadata=metadata)\n",
        "                for hf_url in hf_urls:\n",
        "                    artifact_trained.add_reference(hf_url, checksum = False)\n",
        "                run.log_artifact(artifact_trained)\n",
        "                progress.value = 0.98\n",
        "\n",
        "                run_finetune.button_style = 'success'\n",
        "                run_predictions.disabled = False\n",
        "\n",
        "                progress.value = 1.0\n",
        "                progress.bar_style = 'success'\n",
        "                success_try = True\n",
        "\n",
        "                label_progress.value = '🎉 Neural network trained successfully!'\n",
        "                log_predictions.clear_output(wait=True)\n",
        "                with log_predictions:\n",
        "                    print_html('\\nEnter the start of a sentence and click \"Run predictions\"')\n",
        "                with log_restart:\n",
        "                    print_html('\\n<b>To change user, refresh the page</b>\\n')\n",
        "\n",
        "            except Exception as e:\n",
        "                print('\\nAn error occured...\\n')\n",
        "                print(e)\n",
        "                run_finetune.button_style = 'danger'\n",
        "                run_finetune.disabled = False\n",
        "\n",
        "        if not success_try:\n",
        "            display(log_debug)\n",
        "            progress.bar_style = 'danger'\n",
        "\n",
        "    run_finetune = widgets.Button(\n",
        "        description='Train Neural Network',\n",
        "        button_style='primary',\n",
        "        disabled=True)\n",
        "    def on_run_finetune_clicked(b):\n",
        "        finetune()\n",
        "    run_finetune.on_click(on_run_finetune_clicked)\n",
        "\n",
        "    log_finetune = widgets.Output()\n",
        "    with log_finetune:\n",
        "        print_html('\\nWaiting for Step 1 to complete...')\n",
        "\n",
        "    predictions = []\n",
        "\n",
        "    def shorten_text(text, max_char):\n",
        "        while len(text) > max_char:\n",
        "            text = ' '.join(text.split()[:-1]) + '…'\n",
        "        return text\n",
        "\n",
        "    def predict():\n",
        "        run_predictions.disabled = True\n",
        "        start_widget.disabled = True\n",
        "        run_predictions.button_style = 'primary'\n",
        "        global handles_processed\n",
        "        global model_url\n",
        "        log_predictions.clear_output(wait=True)\n",
        "\n",
        "        # tweet buttons don't appear well in colab if within log_predictions widget\n",
        "        # we reset the entire cell\n",
        "        clear_output(wait=True)\n",
        "        display(widgets.VBox([start_widget, run_predictions, log_model, log_predictions]))\n",
        "        stylize()\n",
        "\n",
        "        def tweet_html(tweet_text, tweet_url):\n",
        "            tweet_text = shorten_text(tweet_text, 250)\n",
        "            params = urlencode({'text': tweet_text, 'url': tweet_url, 'related': 'borisdayma'})\n",
        "            url=f'https://twitter.com/intent/tweet?{params}'\n",
        "            return f'''\n",
        "            <div style=\"width: 76px;\">\n",
        "                <a target=\"_blank\" href=\"{url}\" style='background-color:rgb(27, 149, 224);border-bottom-left-radius:4px;border-bottom-right-radius:4px;border-top-left-radius:4px;border-top-right-radius:4px;box-sizing:border-box;color:rgb(255, 255, 255);cursor:pointer;display:inline-block;font-family:\"Helvetica Neue\", Arial, sans-serif;font-size:13px;font-stretch:100%;font-style:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-weight:500;height:28px;line-height:26px;outline-color:rgb(255, 255, 255);outline-style:none;outline-width:0px;padding-bottom:1px;padding-left:9px;padding-right:10px;padding-top:1px;position:relative;text-align:left;text-decoration-color:rgb(255, 255, 255);text-decoration-line:none;text-decoration-style:solid;text-decoration-thickness:auto;user-select:none;vertical-align:top;white-space:nowrap;zoom:1;'>\n",
        "                <i style='background-attachment:scroll;background-clip:border-box;background-color:rgba(0,0,0,0);background-image:url(data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2072%2072%22%3E%3Cpath%20fill%3D%22none%22%20d%3D%22M0%200h72v72H0z%22%2F%3E%3Cpath%20class%3D%22icon%22%20fill%3D%22%23fff%22%20d%3D%22M68.812%2015.14c-2.348%201.04-4.87%201.744-7.52%202.06%202.704-1.62%204.78-4.186%205.757-7.243-2.53%201.5-5.33%202.592-8.314%203.176C56.35%2010.59%2052.948%209%2049.182%209c-7.23%200-13.092%205.86-13.092%2013.093%200%201.026.118%202.02.338%202.98C25.543%2024.527%2015.9%2019.318%209.44%2011.396c-1.125%201.936-1.77%204.184-1.77%206.58%200%204.543%202.312%208.552%205.824%2010.9-2.146-.07-4.165-.658-5.93-1.64-.002.056-.002.11-.002.163%200%206.345%204.513%2011.638%2010.504%2012.84-1.1.298-2.256.457-3.45.457-.845%200-1.666-.078-2.464-.23%201.667%205.2%206.5%208.985%2012.23%209.09-4.482%203.51-10.13%205.605-16.26%205.605-1.055%200-2.096-.06-3.122-.184%205.794%203.717%2012.676%205.882%2020.067%205.882%2024.083%200%2037.25-19.95%2037.25-37.25%200-.565-.013-1.133-.038-1.693%202.558-1.847%204.778-4.15%206.532-6.774z%22%2F%3E%3C%2Fsvg%3E);background-origin:padding-box;background-position-x:0px;background-position-y:0px;background-repeat-x;background-repeat-y;background-size:auto;color:rgb(255,255,255);cursor:pointer;display:inline-block;font-family:\"Helvetica Neue\",Arial,sans-serif;font-size:13px;font-stretch:100%;font-style:italic;font-variant-caps:normal;font-variant-east-asian:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-weight:500;height:18px;line-height:26px;position:relative;text-align:left;text-decoration-thickness:auto;top:4px;user-select:none;white-space:nowrap;width:18px;'></i>\n",
        "                <span style='color:rgb(255,255,255);cursor:pointer;display:inline-block;font-family:\"Helvetica Neue\",Arial,sans-serif;font-size:13px;font-stretch:100%;font-style:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-weight:500;line-height:26px;margin-left:4px;text-align:left;text-decoration-thickness:auto;user-select:none;vertical-align:top;white-space:nowrap;zoom:1;'>Tweet</span>\n",
        "            </a>\n",
        "            </div>\n",
        "            '''\n",
        "\n",
        "        success_try = False\n",
        "\n",
        "        # get start sentence\n",
        "        ensure_widgets_updated()\n",
        "        start = start_widget.value.strip()\n",
        "\n",
        "        with log_predictions:\n",
        "            print_html(f'\\nPerforming predictions of @{\" & @\".join(handles_processed)} starting with \"{start}\"...\\nThis should take no more than 10 seconds!')\n",
        "\n",
        "        with log_debug:\n",
        "            try:\n",
        "                # start a wandb run (should never happen)\n",
        "                if wandb.run is None:\n",
        "                    print('Unexpected missing W&B run process')\n",
        "                    wandb.init()\n",
        "\n",
        "                # prepare input\n",
        "                start_with_bos = '<|endoftext|>' + start\n",
        "                encoded_prompt = trainer.tokenizer(start_with_bos, add_special_tokens=False, return_tensors=\"pt\").input_ids\n",
        "                encoded_prompt = encoded_prompt.to(trainer.model.device)\n",
        "\n",
        "                # prediction\n",
        "                output_sequences = trainer.model.generate(\n",
        "                    input_ids=encoded_prompt,\n",
        "                    max_length=160,\n",
        "                    min_length=10,\n",
        "                    temperature=1.,\n",
        "                    top_p=0.95,\n",
        "                    do_sample=True,\n",
        "                    num_return_sequences=10\n",
        "                    )\n",
        "                generated_sequences = []\n",
        "\n",
        "                # decode prediction\n",
        "                for generated_sequence_idx, generated_sequence in enumerate(output_sequences):\n",
        "                    generated_sequence = generated_sequence.tolist()\n",
        "                    text = trainer.tokenizer.decode(generated_sequence, clean_up_tokenization_spaces=True, skip_special_tokens=True)\n",
        "                    if not ALLOW_NEW_LINES:\n",
        "                        limit = text.find('\\n')\n",
        "                        text = text[: limit if limit != -1 else None]\n",
        "                    generated_sequences.append(text.strip())\n",
        "\n",
        "                for i, g in enumerate(generated_sequences):\n",
        "                    predictions.append([start, g])\n",
        "\n",
        "                # create previews\n",
        "                r = requests.post('https://us-central1-huggingtweets.cloudfunctions.net/get_screenshot',\n",
        "                                  data = {\"NAME\": card_val['USER_NAME'],\n",
        "                                          \"HANDLE\": card_val['USER_HANDLE'],\n",
        "                                          \"URL1\": card_val['USER_PROFILE_1'],\n",
        "                                          \"URL2\": card_val['USER_PROFILE_2'],\n",
        "                                          \"URL3\": card_val['USER_PROFILE_3'],\n",
        "                                          \"DISPLAY1\": card_val['DISPLAY_1'],\n",
        "                                          \"DISPLAY2\": card_val['DISPLAY_2'],\n",
        "                                          \"DISPLAY3\": card_val['DISPLAY_3'],\n",
        "                                          \"BOT\": card_val['BOT'],\n",
        "                                          \"INPUT\": start,\n",
        "                                          \"OUTPUTS\": generated_sequences})\n",
        "                ids = r.json()\n",
        "                global model_preview\n",
        "                global hfapi, token, namespace\n",
        "                if model_preview is None:\n",
        "                    model_preview = f\"http://www.huggingtweets.com/{'-'.join(sorted(handles_processed))}/{ids[0]}/predictions.png\"\n",
        "                    card_val['SOCIAL_LINK'] = model_preview\n",
        "                    create_model_card(card_val, '-'.join(sorted(handles_processed)))\n",
        "                    commit_files('-'.join(sorted(handles_processed)), f'Update model preview')\n",
        "                    with log_model:\n",
        "                        print_html(\"<h2>Model Preview (select a tweet to update)</h2>\")\n",
        "                        show_image_preview(model_preview)\n",
        "\n",
        "                # log predictions\n",
        "                wandb.log({'examples': wandb.Table(data=predictions, columns=['Input', 'Prediction'])})\n",
        "\n",
        "                # display tweets\n",
        "                widgets_tweet = []\n",
        "                center = widgets.Layout(align_items='center', display='flex')\n",
        "                layout_twitter = widgets.Layout(width = '76px')\n",
        "                global bot\n",
        "                for i, (g, id) in enumerate(zip(generated_sequences, ids)):\n",
        "                    preview_button = create_button(id)\n",
        "                    tweet_pred = start + ' → ' + g[len(start):].strip()\n",
        "                    tweet_button = tweet_html(f\"I love this tweet from my AI {bot} of @{' & @'.join(handles_processed)} with #huggingtweets:\\n{tweet_pred}\",\n",
        "                                              f\"http://www.huggingtweets.com/{'-'.join(sorted(handles_processed))}/{id}/predictions.html\")\n",
        "                    w = widgets.HBox([preview_button,\n",
        "                                      widgets.HTML(tweet_button, layout=layout_twitter),\n",
        "                                      widgets.HTML(g)],\n",
        "                                     layout=center)\n",
        "                    w.add_class(\"table_odd\" if i%2 else \"table_even\")\n",
        "                    widgets_tweet.append(w)\n",
        "\n",
        "                # make model share table\n",
        "                tweet_share = f\"I created an AI {bot} of @{' & @'.join(handles_processed)} with #huggingtweets!\\nPlay with my model or create your own!\"\n",
        "                link_model = f'<a href=\"{model_url}\" rel=\"noopener\" target=\"_blank\">{model_url}</a>'\n",
        "                share_data = [[tweet_html(tweet_share, model_url),\n",
        "                               f\"🎉 Share @{' & @'.join(handles_processed)} model: {link_model} <i>(may take 30 seconds to become active)</i>\"]]\n",
        "                share_table = HTML(html_table(share_data))\n",
        "\n",
        "                run_predictions.button_style = 'success'\n",
        "                success_try = True\n",
        "\n",
        "            except Exception as e:\n",
        "                print('\\nAn error occured...\\n')\n",
        "                print(e)\n",
        "                run_predictions.button_style = 'danger'\n",
        "\n",
        "        if success_try:\n",
        "            log_predictions.clear_output()\n",
        "            with log_predictions:\n",
        "                # twitter button does not update within widget in colab\n",
        "                if not IN_COLAB:\n",
        "                    print_html('\\n')\n",
        "                    display(share_table)\n",
        "                    print_html('\\n<b>Share your model and favorite tweets or try new predictions!\\nTwitter will display the image (reload the tweet to preview)!</b>\\n\\n')\n",
        "                    for w in widgets_tweet:\n",
        "                        display(w)\n",
        "                    print_html('\\n<b>Share your model and favorite tweets or try new predictions!\\nTwitter will display the image (reload the tweet to preview)!</b>\\n\\n')\n",
        "\n",
        "            if IN_COLAB:\n",
        "                print_html('\\n')\n",
        "                display(share_table)\n",
        "                print_html('\\n<b>Share your model and favorite tweets or try new predictions!\\nTwitter will display the image (reload the tweet to preview)!</b>\\n\\n')\n",
        "                display(widgets.VBox([*widgets_tweet]))\n",
        "                print_html('\\n<b>Share your model and favorite tweets or try new predictions!\\nTwitter will display the image (reload the tweet to preview)!</b>\\n\\n')\n",
        "        else:\n",
        "            display(log_debug)\n",
        "\n",
        "        run_predictions.disabled = False\n",
        "        start_widget.disabled = False\n",
        "\n",
        "    start_widget = widgets.Text(value='My dream is',\n",
        "                                placeholder='Start a sentence')\n",
        "\n",
        "    run_predictions = widgets.Button(\n",
        "        description='Run predictions',\n",
        "        button_style='primary',\n",
        "        disabled=True)\n",
        "    def on_run_predictions_clicked(b):\n",
        "        predict()\n",
        "    run_predictions.on_click(on_run_predictions_clicked)\n",
        "\n",
        "    log_predictions = widgets.Output()\n",
        "    with log_predictions:\n",
        "        print_html('\\nWaiting for Step 2 to complete...')\n",
        "    log_model = widgets.Output()\n",
        "\n",
        "    clear_output(wait=True)\n",
        "    print_html(\"🎉 Environment set-up correctly! You're ready to move to Step 1!\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OMpSPr0T8AHD"
      },
      "source": [
        "## Step 1 - Enter a Twitter handle\n",
        "\n",
        "Enter a Twitter handle and click Download tweets. This gives the model a dataset of examples to train on."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "jupyter": {
          "source_hidden": true
        },
        "id": "6O-8Kr_m8AHE"
      },
      "source": [
        "#@title ⠀ {display-mode: \"form\"}\n",
        "stylize()\n",
        "if IN_COLAB:\n",
        "    display(Javascript('''google.colab.output.setIframeHeight(0, true, {maxHeight: 2000})'''))\n",
        "display(widgets.VBox([*handle_widgets, run_dl_tweets, log_restart, log_dl_tweets]))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vc_ArgCZ8AHH"
      },
      "source": [
        "## Step 2 - Train your Neural Network\n",
        "\n",
        "Fine-tune a language model on your unique set of tweets to generate predictions.\n",
        "\n",
        "The model is downloaded from [HuggingFace transformers](https://huggingface.co/), an awesome open source library for Natural Language Processing and training is logged through [Weights & Biases](http://docs.wandb.com/)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "jupyter": {
          "source_hidden": true
        },
        "id": "CpxBQYF88AHJ"
      },
      "source": [
        "#@title ⠀ {display-mode: \"form\"}\n",
        "stylize()\n",
        "if IN_COLAB:\n",
        "    display(Javascript('''google.colab.output.setIframeHeight(0, true, {maxHeight: 2000})'''))\n",
        "display(widgets.VBox([run_finetune, log_finetune]))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GfRPh4V18AHM"
      },
      "source": [
        "## Step 3: Generate tweets\n",
        "\n",
        "Type the beginning of a tweet, press Run predictions, and the model will try to come up with a realistic ending to your tweet."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "jupyter": {
          "source_hidden": true
        },
        "id": "FlMACi0-8AHN"
      },
      "source": [
        "#@title ⠀ {display-mode: \"form\"}\n",
        "stylize()\n",
        "if IN_COLAB:\n",
        "    display(Javascript('''google.colab.output.setIframeHeight(0, true, {maxHeight: 2000})'''))\n",
        "display(widgets.VBox([start_widget, run_predictions, log_model, log_predictions]))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rlRI2hsBKtz6"
      },
      "source": [
        "Huggingtweets is still in its infancy and will get better over time!\n",
        "\n",
        "In the future, it will train continuously to become a Twitter expert!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "b4iawnVxrItM"
      },
      "source": [
        "## About\n",
        "\n",
        "*Built by Boris Dayma*\n",
        "\n",
        "[![Follow](https://img.shields.io/twitter/follow/borisdayma?style=social)](https://twitter.com/intent/follow?screen_name=borisdayma)\n",
        "\n",
        "My main goals with this project are:\n",
        "* to experiment with how to train, deploy and maintain neural networks in production ;\n",
        "* to make AI accessible to everyone ;\n",
        "* to have fun!\n",
        "\n",
        "For more details, visit the project repository.\n",
        "\n",
        "[![GitHub stars](https://img.shields.io/github/stars/borisdayma/huggingtweets?style=social)](https://github.com/borisdayma/huggingtweets)\n",
        "\n",
        "**Disclaimer: this project is not to be used to publish any false generated information but to perform research on Natural Language Generation.**\n",
        "\n",
        "## Resources\n",
        "\n",
        "* [Explore the W&B report](https://wandb.ai/wandb/huggingtweets/reports/HuggingTweets-Train-a-Model-to-Generate-Tweets--VmlldzoxMTY5MjI) to understand how the model works\n",
        "* [HuggingFace and W&B integration documentation](https://docs.wandb.com/library/integrations/huggingface)\n",
        "\n",
        "## Got questions about W&B?\n",
        "\n",
        "If you have any questions about using W&B to track your model performance and predictions, please reach out to the [slack community](http://bit.ly/wandb-forum)."
      ]
    }
  ]
}