二、函数.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 14. 尽量使用异常来表示特殊情况，而不是None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 用None来表示特殊含义，这比较容易出错，因为None和0、还有一些空值都会在条件表达式中被评估为False。\n",
    "* 函数应该在遇到特殊情况时抛出异常，这时调用者看到异常后，就会编写`try/except`语句来处理，更加有效。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 15. 如何在闭包里使用外围作用域中的变量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "假设我们现在一个任务是对数字排序，但有特殊的要求，即：要求某个集合中的元素要排在前面。我们可以这样做："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这是我们输入的数据\n",
    "numbers = [8, 3, 1, 2, 5, 4, 7, 6]\n",
    "group = {2, 3, 5, 7}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 写一个函数, inplace\n",
    "def sort_priority(values, group):\n",
    "    def helper(x):\n",
    "        if x in group:\n",
    "            return (0, x)\n",
    "        return (1, x)\n",
    "    values.sort(key = helper)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[2, 3, 5, 7, 1, 4, 6, 8]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 运行\n",
    "sort_priority(numbers, group)\n",
    "numbers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个函数的成功离不开以下几点：\n",
    "\n",
    "* Python支持闭包（closure），即一种定义在某个作用域中的函数，这个函数可以引用那个作用域中的变量，比如上面的`helper`函数，其可以利用`group`。\n",
    "* Python的函数是first-class object，一级对象，所以其可以像其他对象一样进行赋值、比较或者作为参数使用（`key`接受一个函数，这个函数接受元素，然后返回一个值，实际上使用这个值来进行排序）。\n",
    "* Python使用特殊的规则来比较两个tuple（这样的规则也适用于两个list），先比较tuple中索引为0的元素，如果相等再比较索引为1的元素，依次类推。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们希望进行一些改进，比如numbers中出现了group的元素，就返回True，否则返回False。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sort_priority2(values, group):\n",
    "    found = False\n",
    "    def helper(x):\n",
    "        if x in group:\n",
    "            found = True\n",
    "            return (0, x)\n",
    "        return (1, x)\n",
    "    values.sort(key = helper)\n",
    "    return found"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found: False\n",
      "[2, 3, 5, 7, 1, 4, 6, 8]\n"
     ]
    }
   ],
   "source": [
    "found = sort_priority2(numbers, group)\n",
    "print(\"Found:\", found)\n",
    "print(numbers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**为什么是False？**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们需要了解python作用域的概念：\n",
    "\n",
    "当某个表达式中使用到一个变量的时候，Python会安装以下的顺序来查找这个变量：\n",
    "1. 当前函数的作用域；\n",
    "2. 任何其他外围作用域（比如包含当前函数的其他函数）；\n",
    "3. 包含当前代码的那个模块（也叫做全局作用域，global scope）；\n",
    "4. 内置作用域（包含`len`和`str`等函数的作用域）。\n",
    "\n",
    "如果以上都没有找到名称相符的变量，就抛出`NameError`异常。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当我们给变量赋值的时候，规则略有不同：\n",
    "1. 如果当前作用域中已经有这个变量，则该变量使用新值。\n",
    "2. 如果当前作用域中没有这个变量，则本次赋值被看做是当前变量的定义（包含在本作用域中，当本作用域关闭的时候，变量删除）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这就容易解释以上`sort_priority2`函数返回值错误的原因了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def sort_priority2(numbers, group):\n",
    "    found = False  # Scope: sort_priority2\n",
    "    def helper(x):\n",
    "        if x in group:  # 因为是使用，所以会往外作用域进行一层一层的搜索\n",
    "            found = True # 在Scope：helper中并没有找到found，赋值操作不会再上一层scope去搜索了\n",
    "            return (0, x)\n",
    "        return (1, x)\n",
    "    numbers.sort(key=helper)\n",
    "    return found"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这称之为**作用域bug（scoping bug）**。这样是为了防止函数中的局部变量污染外面的那个模块。解决的方法有如下几种："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第一种，使用`nonlocal`语句。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* `nonlocal`语句告诉我们，如果要对这个变量进行赋值，要去外一层作用域中寻找这个变量，但这不会影响到global scope。\n",
    "* `global`语句则用来补充`nonlocal`，其定义的变量再进行赋值的时候直接到global scope中去寻找这个变量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found: True\n",
      "[2, 3, 5, 7, 1, 4, 6, 8]\n"
     ]
    }
   ],
   "source": [
    "def sort_priority3(numbers, group):\n",
    "    found = False\n",
    "    def helper(x):\n",
    "        nonlocal found\n",
    "        if x in group:\n",
    "            found = True\n",
    "            return (0, x)\n",
    "        return (1, x)\n",
    "    numbers.sort(key = helper)\n",
    "    return found\n",
    "\n",
    "found = sort_priority3(numbers, group)\n",
    "print(\"Found:\", found)\n",
    "print(numbers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "但就像`global`语句一样，`nonlocal`容易被滥用，而且很难去追踪。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二种，这种状态封装到辅助类中，这个类主要用来作为函数使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "[2, 3, 5, 7, 1, 4, 6, 8]\n"
     ]
    }
   ],
   "source": [
    "class Sorter:\n",
    "    def __init__(self, group):\n",
    "        self.group = group\n",
    "        self.found = found\n",
    "    \n",
    "    def __call__(self, x):\n",
    "        if x in self.group:\n",
    "            self.found = True\n",
    "            return (0, x)\n",
    "        return (1, x)\n",
    "\n",
    "sorter = Sorter(group)\n",
    "numbers.sort(key = sorter)\n",
    "print(sorter.found)\n",
    "print(numbers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第三种，在python2中也经常使用的方法（python2中没有`nonlocal`），利用list的可变属性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found: True\n",
      "[2, 3, 5, 7, 1, 4, 6, 8]\n"
     ]
    }
   ],
   "source": [
    "def sort_priority4(numbers, group):\n",
    "    found = [False]\n",
    "    def helper(x):\n",
    "        if x in group:\n",
    "            found[0] = True\n",
    "            return (0, x)\n",
    "        return (1, x)\n",
    "    numbers.sort(key=helper)\n",
    "    return found[0]\n",
    "\n",
    "found = sort_priority4(numbers, group)\n",
    "print(\"Found:\", found)\n",
    "print(numbers)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 16. 使用生成器来改写直接返回列表的函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def index_words(text):\n",
    "    result = []\n",
    "    if text:\n",
    "        result.append(0)\n",
    "    for index, letter in enumerate(text):\n",
    "        if letter == \" \":\n",
    "            result.append(index + 1)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这是一个返回一句话里面每个单词开头索引的函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 5, 11, 15, 21, 27]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "address = \"Four score and seven years ago\"\n",
    "result = index_words(address)\n",
    "result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实以上的工作更加适合使用生成器函数来完成"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def index_words_iter(text):\n",
    "    # 每次调用next，则就会把生成器推进到下一个yield表达式那里\n",
    "    if text:\n",
    "        yield 0\n",
    "    for index, letter in enumerate(text):\n",
    "        if letter == \" \":\n",
    "            yield index + 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 5, 11, 15, 21, 27]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(index_words_iter(address))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 17. 在参数上面迭代时要多加小心"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们需要完成这样的一个工作：读取一份文件中每个城市储存的游客人数，然后计算每个城市游客人数所占的比例。这个文件可能很大，所以我们使用上面提到的生成器的方式来完成。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_visits(data_path):\n",
    "    with open(data_path, \"r\") as f:\n",
    "        for line in f:\n",
    "            yield int(line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize(numbers):\n",
    "    # 输入一个可迭代对象，输出列表\n",
    "    total = sum(numbers)  # sum可以作用与可迭代对象\n",
    "    result = []\n",
    "    for value in numbers:\n",
    "        percent = 100 * value / total\n",
    "        result.append(percent)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "visits = read_visits(\"./tmp/my_numbers.txt\")\n",
    "normalize(visits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们发现并没有返回我们期望的值。**这是因为迭代器或生成器只能做一轮循环，当一轮循环将其中的内容耗尽后，再次调用`next`方法并不会重新迭代其值**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[15, 25, 35, 80]\n",
      "[]\n"
     ]
    }
   ],
   "source": [
    "visits = read_visits(\"./tmp/my_numbers.txt\")\n",
    "print(list(visits))\n",
    "print(list(visits))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的函数中，`sum`已经将其内容耗尽，所有后面的`for`循环并不会再进行迭代了。（注意，使用`for`、`list`或其他Python标准库的许多函数比如`sum`来对迭代器或生成器进行迭代的时候，其不会输出`StopIteration`异常，只有调用`__next__`或`next()`的时候才会出现）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "ename": "StopIteration",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-21-1bf184739c37>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mvisits\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m__next__\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m: "
     ]
    }
   ],
   "source": [
    "visits.__next__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "ename": "StopIteration",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-22-7e6019bed7fa>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mvisits\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m: "
     ]
    }
   ],
   "source": [
    "next(visits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第一种解决思路是：在函数中将其变成列表复制一份。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize_copy(numbers):\n",
    "    numbers = list(numbers)\n",
    "    total = sum(numbers)\n",
    "    results = []\n",
    "    for value in numbers:\n",
    "        percent = 100 * value / total\n",
    "        results.append(percent)\n",
    "    return results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9.67741935483871, 16.129032258064516, 22.580645161290324, 51.61290322580645]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "visits = read_visits(\"./tmp/my_numbers.txt\")\n",
    "normalize_copy(visits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这样的问题在于，每次将其变成一个列表的时候可能会造成耗尽内存而崩溃。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二种思路是：让`normalize`函数接受的是一个返回迭代器本身的函数，则每次使用迭代迭代器的时候都重新生成一个。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize_func(get_iter):\n",
    "    total = sum(get_iter())\n",
    "    result = []\n",
    "    for value in get_iter():\n",
    "        percent = 100 * value / total\n",
    "        result.append(percent)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9.67741935483871, 16.129032258064516, 22.580645161290324, 51.61290322580645]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "normalize_func(lambda: read_visits(\"./tmp/my_numbers.txt\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "但这样传递函数，总让人感觉比较怪，不容易理解。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第三种思路是去实现一个实现了**迭代器协议（iterator protocol）的容器类（container+iterable，但不是一个iterator）**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在执行`for`循环的时候，python实际上做的工作是如下几个步骤：\n",
    "1. 首先调用`iter(foo)`函数；\n",
    "2. 这实际上调用了`foo.__iter__()`方法，此方法返回的是迭代器本身；\n",
    "3. 迭代器本身实现了`__next__()`方法，所以`for`只需要不断的使用内置的`next`函数调用这个方法，知道出现`StopIteration`异常。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这听上去很复杂，但实际上只要实现一个`__iter__`方法，其返回的是一个迭代器即可。即将`__iter__`方法实现为生成器。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 和之前的工作进行对比后，我们可以发现`iter`函数的一个秘密，即其如果接受的是迭代器对象，则其直接返回这个迭代器对象（所以这导致了迭代器耗尽后并不会重新开始），如果接受的是一个容器类型的对象，则其每次都返回新的迭代器对象（比如`list`或刚才我们编写的东西）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里我们来统一界定一下关于generator、iterator、iterable、container的定义，可以见下图。\n",
    "\n",
    "1. container（容器），实际上比如list、set、dict都是一种容器，**如果一个对象可以被询问某个元素是否在其中时（`in`、`not in`），此对象就是一个容器**。绝大多数容器都可以有某种方式来依次得到其中的每个元素（有些容器不行，比如bloom filter），但实际上这个功能是此对象作为**可迭代对象**提供的。\n",
    "2. iterable（可迭代对象），**`foo.__iter__()`或`iter(foo)`后返回一个迭代器，则`foo`被称为一个可迭代对象**，大部分容器是一个iterable，当然，不是容器的也有很多可以看做是一个iterable。**其基础就是需要实现一个`__iter__()`方法来返回一个迭代器（不一定要实现`__next__()`，只要返回一个迭代器就行，即构建一个迭代器不一定用类实现）**，有意思的是，**迭代器本身也是一个可迭代对象，其`iter`后返回的是自身**。\n",
    "3. iterator（迭代器），**其代码基础是需要实现`__iter__()`和`__next__()`**，`__iter__()`需要返回其自身`self`，`__next__() return`一个值作为每一次循环出现的值。由此我们可以解释为什么看似相似的东西在`for`循环中有不同的行为：**看`iter()`返回的是不是自身。作为一个不是迭代器的可迭代对象，其`__iter__()`返回的并不是自身，其就像一个函数一样，每次调用返回的是不同的东西。而如果是迭代器，则其每次返回的都是其自身，而其自身并没有改变，所以当耗尽后就没有了**。\n",
    "4. generator（生成器），**这是一种特殊的迭代器，其特点是不需要实现`__iter__`和`__next__`方法，只需要使用`yield`关键字像编写函数一样即可**。\n",
    "\n",
    "以上4中概念的关系可以简略表示为：**generator是iterator，iterator是iterable，iterable和container有交集，反之不成立**，这个特殊的函数的返回值就是一个迭代器。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./imgs/iter.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ReadVisits:\n",
    "    def __init__(self, data_path):\n",
    "        self.data_path = data_path\n",
    "        \n",
    "    def __iter__(self):\n",
    "        with open(self.data_path, \"r\") as f:\n",
    "            for line in f:\n",
    "                yield int(line)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9.67741935483871, 16.129032258064516, 22.580645161290324, 51.61290322580645]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "visits = ReadVisits(\"./tmp/my_numbers.txt\")\n",
    "normalize(visits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "则我们可以来编写一个函数，其可以判断我们需要的是不是一个迭代器，而是一个非iterator的iterable："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def normalize_defensive(numbers):\n",
    "    if iter(numbers) is iter(numbers):\n",
    "        raise TypeError(\"Must supply an iterable but not an iterator\")\n",
    "    total = sum(numbers)\n",
    "    result = []\n",
    "    for value in numbers:\n",
    "        percent = 100 * value / total\n",
    "        result.append(percent)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[9.67741935483871, 16.129032258064516, 22.580645161290324, 51.61290322580645]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "visits = ReadVisits(\"./tmp/my_numbers.txt\")\n",
    "normalize_defensive(visits)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "Must supply an iterable but not an iterator",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-31-d74e4b5a1180>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[0mvisits\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mread_visits\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"./tmp/my_numbers.txt\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 2\u001b[1;33m \u001b[0mnormalize_defensive\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mvisits\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;32m<ipython-input-29-5b107b13a63e>\u001b[0m in \u001b[0;36mnormalize_defensive\u001b[1;34m(numbers)\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[1;32mdef\u001b[0m \u001b[0mnormalize_defensive\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnumbers\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      2\u001b[0m     \u001b[1;32mif\u001b[0m \u001b[0miter\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnumbers\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;32mis\u001b[0m \u001b[0miter\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnumbers\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m         \u001b[1;32mraise\u001b[0m \u001b[0mTypeError\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"Must supply an iterable but not an iterator\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      4\u001b[0m     \u001b[0mtotal\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0msum\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnumbers\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      5\u001b[0m     \u001b[0mresult\u001b[0m \u001b[1;33m=\u001b[0m \u001b[1;33m[\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mTypeError\u001b[0m: Must supply an iterable but not an iterator"
     ]
    }
   ],
   "source": [
    "visits = read_visits(\"./tmp/my_numbers.txt\")\n",
    "normalize_defensive(visits)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 18. 用数量可变的未知参数减少视觉杂讯（visual noise）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "就是合理使用`*args`。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "但这个的使用涉及到两个问题："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第一，**`args`实际上可以是iterable，因为每次这些参数传入函数中的时候，都会自动`args = tuple(args)`一下**。这时候就要注意，如果iterable很大，可能会造成内存崩溃。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二，当我们对函数进行改动的时候，可能会在前面增加`*args`前增加了一个变量，则我们使用修改后的函数去运行之前的代码，可能会出错，**`args`中的元素可能会被认成前面新增的变量**，这种错误难以追踪，所以最好使用keyword-only argument来实现。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 19. 用关键字参数来表达可选的行为"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "没啥可说的"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 20. 使用`None`和文档字符串来描述具有动态默认值的参数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "考虑一个需要：我们需要一个函数来打印日志，其中需要打印时间，为了能够更加灵活，我们希望把时间作为一个参数，其默认值是调用函数时的时间，我们可以如下编写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "def log(message, when=datetime.now()):\n",
    "    print(\"%s: %s\" % (when, message))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-01-19 13:54:01.491519: Hi there!\n",
      "2020-01-19 13:54:01.491519: Hi again!\n"
     ]
    }
   ],
   "source": [
    "import time\n",
    "\n",
    "log(\"Hi there!\")\n",
    "time.sleep(0.1)\n",
    "log(\"Hi again!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们发现时间是一样的！\n",
    "\n",
    "这是因为**函数的默认值是在函数加载的时候被求解的**，这里就是我们就函数的定义运行的时候。当调用的时候，程序不会再次求解默认值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所以处理这种的一般方式是使用`None`作为其默认参数并在文档中进行说明"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "def log(message, when=None):\n",
    "    \"\"\"Log a message with a timestamp.\n",
    "    \n",
    "    Args:\n",
    "        message: Message to print.\n",
    "        when: datetime of when the message occurred.\n",
    "            Defaults to the present time.\n",
    "    \"\"\"\n",
    "    when = datetime.now() if when is None else when\n",
    "    print(\"%s: %s\" % (when, message))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-01-19 14:06:54.519400: Hi there!\n",
      "2020-01-19 14:06:54.620128: Hi again!\n"
     ]
    }
   ],
   "source": [
    "log(\"Hi there!\")\n",
    "time.sleep(0.1)\n",
    "log(\"Hi again!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "还有，**如果其默认值是可变类型（mutable）,一定记得使用`None`作为形式上的默认值**。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "例：一个函数，其解析json格式的data，如果解析失败，则返回`default`。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "def decode(data, default={}):\n",
    "    try:\n",
    "        return json.loads(data)\n",
    "    except ValueError:\n",
    "        return default\n",
    "# 载入这个函数的时候，就已经得到了那个字典。又因为字典是可变的，所以之后的使用默认值eval的操作，都\n",
    "# 将共享同一个字典"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Foo: {'stuff': 5, 'meep': 1}\n",
      "Bar: {'stuff': 5, 'meep': 1}\n"
     ]
    }
   ],
   "source": [
    "foo = decode(\"bad data\")\n",
    "foo[\"stuff\"] = 5\n",
    "bar = decode(\"also bad\")\n",
    "bar[\"meep\"] = 1\n",
    "print(\"Foo:\", foo)\n",
    "print(\"Bar:\", bar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "assert foo is bar"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "解决方案和上面相同"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "def decode(data, default=None):\n",
    "    \"\"\"\n",
    "    略\n",
    "    \"\"\"\n",
    "    if default is None:\n",
    "        default = {}\n",
    "    try:\n",
    "        return json.loads(data)\n",
    "    except ValueError:\n",
    "        return default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Foo: {'stuff': 5}\n",
      "Bar: {'meep': 1}\n"
     ]
    }
   ],
   "source": [
    "foo = decode(\"bad data\")\n",
    "foo[\"stuff\"] = 5\n",
    "bar = decode(\"also bad\")\n",
    "bar[\"meep\"] = 1\n",
    "print(\"Foo:\", foo)\n",
    "print(\"Bar:\", bar)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 21. 用只能以关键字形式指定的参数来确保代码明晰"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先介绍一个进行除法的函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def safe_division(number, divisor, ignore_overflow, ignore_zero_division):\n",
    "    try:\n",
    "        return number / divisor\n",
    "    except OverflowError:\n",
    "        if ignore_overflow:\n",
    "            return 0\n",
    "        else:\n",
    "            raise\n",
    "    except ZeroDivisionError:\n",
    "        if ignore_zero_division:\n",
    "            return float(\"inf\")\n",
    "        else:\n",
    "            raise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "inf"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "safe_division(1, 0, False, True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上的这种调用方式实在是太难看懂了，**我们是不是有什么办法强制使用者使用后面的两个参数的时候必须以关键字参数的形式来使用呢**？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "def safe_division(number, divisor, *, ignore_overflow, ignore_zero_division):  # *标志着未知参数的终结\n",
    "    try:\n",
    "        return number / divisor\n",
    "    except OverflowError:\n",
    "        if ignore_overflow:\n",
    "            return 0\n",
    "        else:\n",
    "            raise\n",
    "    except ZeroDivisionError:\n",
    "        if ignore_zero_division:\n",
    "            return float(\"inf\")\n",
    "        else:\n",
    "            raise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "safe_division() takes 2 positional arguments but 4 were given",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-48-3b4f4f8c2f29>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0msafe_division\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;36m0\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;32mFalse\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;32mTrue\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m: safe_division() takes 2 positional arguments but 4 were given"
     ]
    }
   ],
   "source": [
    "safe_division(1, 0, False, True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "inf"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "safe_division(1, 0, ignore_overflow=False, ignore_zero_division=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为python2中没有这样的机制，所以需要利用`**kwargs`来实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}