diff --git a/Online/README.md b/Online/README.md index 14cf136..368f55c 100644 --- a/Online/README.md +++ b/Online/README.md @@ -36,7 +36,7 @@ | [ShuffleNet](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/05-ShuffleNet) | 8.0.RC3.alpha002 | 2.4.10 | 8T16G | | [SSD](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/06-SSD) | 8.1.RC1 | 2.6.0 | 8T8G | | [RNN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/07-RNN) | 8.1.RC1 | 2.6.0 | 8T8G | -| [LSTM+CRF](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/08-LSTM%2BCRF) | 8.0.RC3.alpha002 | 2.4.10 | 8T16G | +| [LSTM+CRF](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/08-LSTM%2BCRF) | 8.1.RC1 | 2.6.0 | 8T16G | | [GAN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/09-GAN) | 8.0.RC3.alpha002 | 2.4.10 | 8T16G | | [DCGAN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/10-DCGAN) | 8.1.RC1 | 2.6.0 | 8T8G | | [Pix2Pix](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/11-Pix2Pix) | 8.0.RC3.alpha002 | 2.4.10 | 8T16G | @@ -76,7 +76,7 @@ | [VideoClassification](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/VideoClassification) | 8.0.0.beta1 | 2.6.0 |8T16G | | [MaskGeneration](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/MaskGeneration) | 8.1.RC1 | 2.6.0 | 8T16G | | [DocumentQuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DocumentQuestionAnswering) | 8.0.0.beta1 | 2.6.0 | 20T24G | - +| [DepthEstimation](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DepthEstimation) | 8.1.RC1 | 2.6.0 | 8T16G | ## 学习资源 diff --git a/Online/community/README.md b/Online/community/README.md index 1230890..7666f27 100644 --- a/Online/community/README.md +++ b/Online/community/README.md @@ -17,4 +17,4 @@ | [VideoClassification](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/VideoClassification) | 8.0.0.beta1 | 2.6.0 |8T16G | | [MaskGeneration](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/MaskGeneration) | 8.1.RC1 | 2.6.0 | 8T16G | | [DocumentQuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DocumentQuestionAnswering) | 8.0.0.beta1 | 2.6.0 | 20T24G | - +| [DepthEstimation](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DepthEstimation) | 8.1.RC1 | 2.6.0 | 8T16G | diff --git a/Online/inference/08-LSTM+CRF/mindspore_sequence_labeling.ipynb b/Online/inference/08-LSTM+CRF/mindspore_sequence_labeling.ipynb index 1691a6e..0eb2c6d 100644 --- a/Online/inference/08-LSTM+CRF/mindspore_sequence_labeling.ipynb +++ b/Online/inference/08-LSTM+CRF/mindspore_sequence_labeling.ipynb @@ -31,19 +31,26 @@ "id": "3b5517e9", "metadata": {}, "source": [ - "## 设置运行环境\n", + "## 环境准备\n", "\n", - "由于资源限制，需开启性能优化模式，具体设置如下参数：\n", + "开发者拿到香橙派开发板后，首先需要进行硬件资源确认，镜像烧录及CANN和MindSpore版本的升级，才可运行该案例，具体如下：\n", "\n", - " max_device_memory=\"2GB\" : 设置设备可用的最大内存为2GB。\n", + "- 硬件： 香橙派AIpro 8G 8T开发板\n", + "- 镜像： 香橙派官网ubuntu镜像\n", + "- CANN：8.1.RC1\n", + "- MindSpore： 2.6.0\n", "\n", - " mode=mindspore.GRAPH_MODE : 表示在GRAPH_MODE模式中运行。\n", + "### 镜像烧录\n", "\n", - " device_target=\"Ascend\" : 表示待运行的目标设备为Ascend。\n", + "运行该案例需要烧录香橙派官网ubuntu镜像，烧录流程参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--镜像烧录](https://www.mindspore.cn/tutorials/zh-CN/r2.6.0/orange_pi/environment_setup.html#1-%E9%95%9C%E5%83%8F%E7%83%A7%E5%BD%95%E4%BB%A5windows%E7%B3%BB%E7%BB%9F%E4%B8%BA%E4%BE%8B)章节。\n", "\n", - " jit_config={\"jit_level\":\"O2\"} : 编译优化级别开启极致性能优化，使用下沉的执行方式。\n", + "### CANN升级\n", "\n", - " ascend_config={\"precision_mode\":\"allow_mix_precision\"} : 自动混合精度，自动将部分算子的精度降低到float16或bfloat16。" + "CANN升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--CANN升级](https://www.mindspore.cn/tutorials/zh-CN/r2.6.0/orange_pi/environment_setup.html#3-cann%E5%8D%87%E7%BA%A7)章节。\n", + "\n", + "### MindSpore升级\n", + "\n", + "MindSpore升级参考[昇思MindSpore官网--香橙派开发专区--环境搭建指南--MindSpore升级](https://www.mindspore.cn/tutorials/zh-CN/r2.6.0/orange_pi/environment_setup.html#4-mindspore%E5%8D%87%E7%BA%A7)章节。" ] }, { @@ -51,10 +58,43 @@ "execution_count": null, "id": "f01741d9", "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "/usr/local/miniconda3/lib/python3.9/site-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/usr/local/miniconda3/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n", + "/usr/local/miniconda3/lib/python3.9/site-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for type is zero.\n", + " setattr(self, word, getattr(machar, word).flat[0])\n", + "/usr/local/miniconda3/lib/python3.9/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for type is zero.\n", + " return self._float_to_str(self.smallest_subnormal)\n", + "[WARNING] ME(15713:255085896294432,MainProcess):2026-01-21-14:55:23.419.460 [mindspore/context.py:1402] For 'context.set_context', the parameter 'device_target' will be deprecated and removed in a future version. Please use the api mindspore.set_device() instead.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "MindSpore version: 2.6.0\n" + ] + } + ], "source": [ - "import mindspore\n", - "mindspore.set_context(max_device_memory=\"2GB\", mode=mindspore.GRAPH_MODE, device_target=\"Ascend\", jit_config={\"jit_level\":\"O2\"}, ascend_config={\"precision_mode\":\"allow_mix_precision\"})" + "import os\n", + "import requests\n", + "import numpy as np\n", + "\n", + "import mindspore as ms\n", + "import mindspore.nn as nn\n", + "import mindspore.mint as mint\n", + "from mindspore import load_checkpoint, load_param_into_net\n", + "from mindspore.common.initializer import initializer, Uniform\n", + "\n", + "ms.set_context(mode=ms.PYNATIVE_MODE, device_target=\"Ascend\")\n", + "print(\"MindSpore version:\", ms.__version__)\n" ] }, { @@ -124,7 +164,7 @@ }, { "cell_type": "code", - "execution_count": 1, + "execution_count": 2, "id": "d09936ad-6b20-4423-9f57-8e14917e61d4", "metadata": {}, "outputs": [], @@ -142,7 +182,7 @@ " score = start_trans[tags[0]]\n", " # score += 第一次发射概率\n", " # shape: (batch_size,)\n", - " score += emissions[0, mnp.arange(batch_size), tags[0]]\n", + " score += emissions[0, mint.arange(batch_size), tags[0]]\n", "\n", " for i in range(1, seq_length):\n", " # 标签由i-1转移至i的转移概率（当mask == 1时有效）\n", @@ -151,11 +191,11 @@ "\n", " # 预测tags[i]的发射概率（当mask == 1时有效）\n", " # shape: (batch_size,)\n", - " score += emissions[i, mnp.arange(batch_size), tags[i]] * mask[i]\n", + " score += emissions[i, mint.arange(batch_size), tags[i]] * mask[i]\n", "\n", " # 结束转移\n", " # shape: (batch_size,)\n", - " last_tags = tags[seq_ends, mnp.arange(batch_size)]\n", + " last_tags = tags[seq_ends, mint.arange(batch_size)]\n", " # score += 结束转移概率\n", " # shape: (batch_size,)\n", " score += end_trans[last_tags]\n", @@ -185,7 +225,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 3, "id": "d9a0ef6a-1c3a-400e-9053-e0659e8f9e7e", "metadata": {}, "outputs": [], @@ -217,18 +257,18 @@ "\n", " # 对score_i做log_sum_exp运算，用于下一个Token的score计算\n", " # shape: (batch_size, num_tags)\n", - " next_score = ops.logsumexp(next_score, axis=1)\n", + " next_score = mint.logsumexp(next_score, dim=1)\n", "\n", " # 当mask == 1时，score才会变化\n", " # shape: (batch_size, num_tags)\n", - " score = mnp.where(mask[i].expand_dims(1), next_score, score)\n", + " score = mint.where(mask[i].expand_dims(1), next_score, score)\n", "\n", " # 最后加结束转移概率\n", " # shape: (batch_size, num_tags)\n", " score += end_trans\n", " # 对所有可能的路径得分求log_sum_exp\n", " # shape: (batch_size,)\n", - " return ops.logsumexp(score, axis=1)" + " return mint.logsumexp(score, dim=1)" ] }, { @@ -251,7 +291,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": null, "id": "7c286935-74eb-413a-8e47-b8fb5264e6b6", "metadata": {}, "outputs": [], @@ -262,20 +302,22 @@ "\n", " seq_length = mask.shape[0]\n", "\n", - " score = start_trans + emissions[0]\n", + " score = (start_trans + emissions[0]).astype(ms.float32)\n", " history = ()\n", - "\n", + " mask_bool = mask.astype(ms.uint8)\n", " for i in range(1, seq_length):\n", " broadcast_score = score.expand_dims(2)\n", " broadcast_emission = emissions[i].expand_dims(1)\n", " next_score = broadcast_score + trans + broadcast_emission\n", "\n", " # 求当前Token对应score取值最大的标签，并保存\n", + " #indices = next_score.argmax(axis=1)\n", " indices = next_score.argmax(axis=1)\n", " history += (indices,)\n", "\n", " next_score = next_score.max(axis=1)\n", - " score = mnp.where(mask[i].expand_dims(1), next_score, score)\n", + " score = mint.where(mask_bool[i].expand_dims(1), next_score, score)\n", + "\n", "\n", " score += end_trans\n", "\n", @@ -304,7 +346,44 @@ " best_tags.reverse()\n", " best_tags_list.append(best_tags)\n", "\n", - " return best_tags_list" + " return best_tags_list\n", + "\n", + "def viterbi_decode_numpy(emissions, seq_length, trans, start_trans, end_trans):\n", + " \"\"\"\n", + " 参数：\n", + " emissions: (B, T, C) float numpy\n", + " seq_length: (B,) int numpy\n", + " trans: (C, C) from_tag -> to_tag\n", + " start_trans/end_trans: (C,)\n", + " 返回：\n", + " paths: List[List[int]]，每条路径长度 = 对应 seq_length\n", + " \"\"\"\n", + " B, T, C = emissions.shape\n", + " paths = []\n", + "\n", + " for b in range(B):\n", + " L = int(seq_length[b])\n", + " emit = emissions[b, :L, :] # (L, C)\n", + "\n", + " dp = start_trans + emit[0] # (C,)\n", + " backp = np.zeros((L, C), dtype=np.int32)\n", + "\n", + " for t in range(1, L):\n", + " scores = dp[:, None] + trans # (C, C)\n", + " backp[t] = scores.argmax(axis=0) # best prev tag for each current tag\n", + " dp = scores.max(axis=0) + emit[t] # (C,)\n", + "\n", + " dp = dp + end_trans\n", + " last = int(dp.argmax())\n", + "\n", + " best = [last]\n", + " for t in range(L - 1, 0, -1):\n", + " last = int(backp[t, last])\n", + " best.append(last)\n", + " best.reverse()\n", + " paths.append(best)\n", + "\n", + " return paths\n" ] }, { @@ -321,37 +400,15 @@ }, { "cell_type": "code", - "execution_count": 4, + "execution_count": 5, "id": "120a39c7-c89d-4cd3-8a75-da27d2853f0e", "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "[WARNING] ME(97316:281472950050832,MainProcess):2024-09-06-10:50:02.452.307 [mindspore/run_check/_check_version.py:357] MindSpore version 2.2.14 and Ascend AI software package (Ascend Data Center Solution)version 7.0 does not match, the version of software package expect one of ['7.1']. Please refer to the match info on: https://www.mindspore.cn/install\n", - "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", - " setattr(self, word, getattr(machar, word).flat[0])\n", - "/home/ma-user/anaconda3/envs/MindSpore/lib/python3.9/site-packages/numpy/core/getlimits.py:549: UserWarning: The value of the smallest subnormal for type is zero.\n", - " setattr(self, word, getattr(machar, word).flat[0])\n", - "[WARNING] ME(97316:281472950050832,MainProcess):2024-09-06-10:50:04.855.681 [mindspore/run_check/_check_version.py:375] MindSpore version 2.2.14 and \"te\" wheel package version 7.0 does not match. For details, refer to the installation guidelines: https://www.mindspore.cn/install\n", - "[WARNING] ME(97316:281472950050832,MainProcess):2024-09-06-10:50:04.861.443 [mindspore/run_check/_check_version.py:382] MindSpore version 2.2.14 and \"hccl\" wheel package version 7.0 does not match. For details, refer to the installation guidelines: https://www.mindspore.cn/install\n", - "[WARNING] ME(97316:281472950050832,MainProcess):2024-09-06-10:50:04.862.319 [mindspore/run_check/_check_version.py:396] Please pay attention to the above warning, countdown: 3\n", - "[WARNING] ME(97316:281472950050832,MainProcess):2024-09-06-10:50:05.864.215 [mindspore/run_check/_check_version.py:396] Please pay attention to the above warning, countdown: 2\n", - "[WARNING] ME(97316:281472950050832,MainProcess):2024-09-06-10:50:06.866.175 [mindspore/run_check/_check_version.py:396] Please pay attention to the above warning, countdown: 1\n" - ] - } - ], + "outputs": [], "source": [ - "import mindspore as ms\n", - "import mindspore.nn as nn\n", - "import mindspore.ops as ops\n", - "import mindspore.numpy as mnp\n", - "from mindspore.common.initializer import initializer, Uniform\n", "\n", "def sequence_mask(seq_length, max_length, batch_first=False):\n", " \"\"\"根据序列实际长度和最大长度生成mask矩阵\"\"\"\n", - " range_vector = mnp.arange(0, max_length, 1, seq_length.dtype)\n", + " range_vector = mint.arange(0, max_length, 1, dtype=seq_length.dtype)\n", " result = range_vector < seq_length.view(seq_length.shape + (1,))\n", " if batch_first:\n", " return result.astype(ms.int64)\n", @@ -385,7 +442,7 @@ " max_length, batch_size = tags.shape\n", "\n", " if seq_length is None:\n", - " seq_length = mnp.full((batch_size,), max_length, ms.int64)\n", + " seq_length = mint.full((batch_size,), max_length, ms.int64)\n", "\n", " mask = sequence_mask(seq_length, max_length)\n", "\n", @@ -412,7 +469,7 @@ " batch_size, max_length = emissions.shape[:2]\n", "\n", " if seq_length is None:\n", - " seq_length = mnp.full((batch_size,), max_length, ms.int64)\n", + " seq_length = mint.full((batch_size,), max_length, ms.int64)\n", "\n", " mask = sequence_mask(seq_length, max_length)\n", "\n", @@ -437,26 +494,158 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": null, "id": "c07555e3-f2a2-4c25-beff-5a78491ab2d1", "metadata": {}, "outputs": [], "source": [ + "class ManualLinear(nn.Cell):\n", + " \"\"\"\n", + " 注意：\n", + " - 参数本体用 float32 存储\n", + " - 计算时强制转为 float16 做 matmul\n", + " - bias 加法用显式 matmul 生成\n", + " \"\"\"\n", + " def __init__(self, in_features: int, out_features: int):\n", + " super().__init__()\n", + " self.weight = ms.Parameter(\n", + " ms.Tensor(np.zeros((out_features, in_features), np.float32)),\n", + " name=\"weight\"\n", + " )\n", + " self.bias = ms.Parameter(\n", + " ms.Tensor(np.zeros((out_features,), np.float32)),\n", + " name=\"bias\"\n", + " )\n", + "\n", + " def _bias_batch(self, b, batch_size, dtype):\n", + " ones = mint.ones((batch_size, 1), dtype=dtype)\n", + " b_row = b.view((1, -1)).astype(dtype)\n", + " return mint.matmul(ones, b_row) # (B, out)\n", + "\n", + " def construct(self, x):\n", + " # x: (B, T, in)\n", + " B, T, _ = x.shape\n", + "\n", + " x16 = x.astype(ms.float16)\n", + " w16 = self.weight.astype(ms.float16) # (out, in)\n", + " b16 = self.bias.astype(ms.float16)\n", + "\n", + " w_t = w16.swapaxes(0, 1) # (in, out)\n", + " y = mint.matmul(x16, w_t) # (B, T, out) fp16\n", + "\n", + " bB = self._bias_batch(b16, B, ms.float16).view((B, 1, -1))\n", + " return y + bB\n", + "\n", + "\n", + "class ManualBiLSTM(nn.Cell):\n", + " \"\"\"\n", + " ckpt keys：\n", + " lstm.weight_ih_l0, lstm.weight_hh_l0, lstm.bias_ih_l0, lstm.bias_hh_l0\n", + " lstm.weight_ih_l0_reverse, lstm.weight_hh_l0_reverse, lstm.bias_ih_l0_reverse, lstm.bias_hh_l0_reverse\n", + "\n", + " 实现要点：\n", + " - 内部统一用 fp16 做 matmul + 激活\n", + " - bias 显式展开到 (B,4H)\n", + " - stack/concat 使用 mint 的 dim 参数\n", + " \"\"\"\n", + " def __init__(self, input_size: int, hidden_size: int):\n", + " super().__init__()\n", + " self.input_size = input_size\n", + " self.hidden_size = hidden_size\n", + "\n", + " H, D = hidden_size, input_size\n", + "\n", + " # forward\n", + " self.weight_ih_l0 = ms.Parameter(ms.Tensor(np.zeros((4 * H, D), np.float32)), name=\"weight_ih_l0\")\n", + " self.weight_hh_l0 = ms.Parameter(ms.Tensor(np.zeros((4 * H, H), np.float32)), name=\"weight_hh_l0\")\n", + " self.bias_ih_l0 = ms.Parameter(ms.Tensor(np.zeros((4 * H,), np.float32)), name=\"bias_ih_l0\")\n", + " self.bias_hh_l0 = ms.Parameter(ms.Tensor(np.zeros((4 * H,), np.float32)), name=\"bias_hh_l0\")\n", + "\n", + " # backward\n", + " self.weight_ih_l0_reverse = ms.Parameter(ms.Tensor(np.zeros((4 * H, D), np.float32)), name=\"weight_ih_l0_reverse\")\n", + " self.weight_hh_l0_reverse = ms.Parameter(ms.Tensor(np.zeros((4 * H, H), np.float32)), name=\"weight_hh_l0_reverse\")\n", + " self.bias_ih_l0_reverse = ms.Parameter(ms.Tensor(np.zeros((4 * H,), np.float32)), name=\"bias_ih_l0_reverse\")\n", + " self.bias_hh_l0_reverse = ms.Parameter(ms.Tensor(np.zeros((4 * H,), np.float32)), name=\"bias_hh_l0_reverse\")\n", + "\n", + " def _bias_batch(self, b_4h, batch_size, dtype):\n", + " ones = mint.ones((batch_size, 1), dtype=dtype)\n", + " b_row = b_4h.view((1, -1)).astype(dtype) # (1, 4H)\n", + " return mint.matmul(ones, b_row) # (B, 4H)\n", + "\n", + " def _run_dir(self, x, reverse: bool):\n", + " # x: (B, T, D) -> return: (B, T, H)\n", + " B, T, _ = x.shape\n", + " H = self.hidden_size\n", + "\n", + " x = x.astype(ms.float16)\n", + "\n", + " if not reverse:\n", + " W_ih, W_hh = self.weight_ih_l0, self.weight_hh_l0\n", + " b_ih, b_hh = self.bias_ih_l0, self.bias_hh_l0\n", + " t_iter = range(T)\n", + " else:\n", + " W_ih, W_hh = self.weight_ih_l0_reverse, self.weight_hh_l0_reverse\n", + " b_ih, b_hh = self.bias_ih_l0_reverse, self.bias_hh_l0_reverse\n", + " t_iter = range(T - 1, -1, -1)\n", + "\n", + " W_ih_t = W_ih.astype(ms.float16).swapaxes(0, 1) # (D, 4H)\n", + " W_hh_t = W_hh.astype(ms.float16).swapaxes(0, 1) # (H, 4H)\n", + "\n", + " b_ih_B = self._bias_batch(b_ih.astype(ms.float16), B, ms.float16)\n", + " b_hh_B = self._bias_batch(b_hh.astype(ms.float16), B, ms.float16)\n", + "\n", + " h = mint.zeros((B, H), dtype=ms.float16)\n", + " c = mint.zeros((B, H), dtype=ms.float16)\n", + "\n", + " h_list = [None] * T\n", + "\n", + " for t in t_iter:\n", + " x_t = x[:, t, :] # (B, D)\n", + " gates = (\n", + " mint.matmul(x_t, W_ih_t)\n", + " + mint.matmul(h, W_hh_t)\n", + " + b_ih_B\n", + " + b_hh_B\n", + " ) # (B, 4H)\n", + "\n", + " i = mint.sigmoid(gates[:, 0:H])\n", + " f = mint.sigmoid(gates[:, H:2 * H])\n", + " g = mint.tanh(gates[:, 2 * H:3 * H])\n", + " o = mint.sigmoid(gates[:, 3 * H:4 * H])\n", + "\n", + " c = f * c + i * g\n", + " h = o * mint.tanh(c)\n", + "\n", + " h_list[t] = h\n", + "\n", + " return mint.stack(h_list, dim=1) # (B, T, H)\n", + "\n", + " def construct(self, x):\n", + " h_f = self._run_dir(x, reverse=False) # (B, T, H)\n", + " h_b = self._run_dir(x, reverse=True) # (B, T, H)\n", + " return mint.concat((h_f, h_b), dim=2) # (B, T, 2H)\n", + "\n", + "\n", "class BiLSTM_CRF(nn.Cell):\n", + " \"\"\"\n", + " - 参数 key :\n", + " embedding.embedding_table\n", + " lstm.*\n", + " hidden2tag.weight / hidden2tag.bias\n", + " crf.start_transitions / crf.end_transitions / crf.transitions\n", + " \"\"\"\n", " def __init__(self, vocab_size, embedding_dim, hidden_dim, num_tags, padding_idx=0):\n", " super().__init__()\n", " self.embedding = nn.Embedding(vocab_size, embedding_dim, padding_idx=padding_idx)\n", - " self.lstm = nn.LSTM(embedding_dim, hidden_dim // 2, bidirectional=True, batch_first=True)\n", - " self.hidden2tag = nn.Dense(hidden_dim, num_tags, 'he_uniform')\n", + " self.lstm = ManualBiLSTM(embedding_dim, hidden_dim // 2) # 双向 -> hidden_dim\n", + " self.hidden2tag = ManualLinear(hidden_dim, num_tags) # 输出 emissions\n", " self.crf = CRF(num_tags, batch_first=True)\n", "\n", - " def construct(self, inputs, seq_length, tags=None):\n", - " embeds = self.embedding(inputs)\n", - " outputs, _ = self.lstm(embeds, seq_length=seq_length)\n", - " feats = self.hidden2tag(outputs)\n", - "\n", - " crf_outs = self.crf(feats, tags, seq_length)\n", - " return crf_outs" + " def construct(self, inputs, seq_length=None, tags=None):\n", + " x = self.embedding(inputs) # fp32\n", + " h = self.lstm(x) # fp16\n", + " emissions = self.hidden2tag(h) # fp16\n", + " return emissions\n" ] }, { @@ -469,7 +658,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 7, "id": "fa53f535-34bc-49a3-b769-e85d3a184cc0", "metadata": {}, "outputs": [], @@ -497,7 +686,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": 8, "id": "5c255358-938b-4e88-a810-2fe4e616a044", "metadata": {}, "outputs": [ @@ -507,7 +696,7 @@ "21" ] }, - "execution_count": 7, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -526,7 +715,7 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 9, "id": "e6d3f2bb-0ea7-457c-8a74-a8e0ea78a109", "metadata": {}, "outputs": [], @@ -544,7 +733,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 10, "id": "7899b6e6-95a4-4ffe-ba49-b27cebe8b306", "metadata": {}, "outputs": [], @@ -569,7 +758,7 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 11, "id": "0e984dcb-6f89-4520-940f-620cdb997529", "metadata": {}, "outputs": [ @@ -579,7 +768,7 @@ "((2, 11), (2, 11), (2,))" ] }, - "execution_count": 10, + "execution_count": 11, "metadata": {}, "output_type": "execute_result" } @@ -602,20 +791,42 @@ "execution_count": null, "id": "65304086-de03-4edc-b3a2-3ca5e43c4795", "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "([], [])" + ] + }, + "execution_count": 12, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "from mindspore import load_param_into_net, load_checkpoint\n", - "from download import download\n", - "\n", - "# download ckpt\n", - "lstm_crf_url = \"https://modelers.cn/coderepo/web/v1/file/MindSpore-Lab/cluoud_obs/main/media/examples/mindspore-courses/orange-pi-online-infer/08-LSTM%2BCRF/lstm-crf.ckpt\"\n", - "path = \"./lstm-crf.ckpt\"\n", - "ckpt_path = download(lstm_crf_url, path, replace=True)\n", - "\n", - "load_checkpoint=load_checkpoint(ckpt_path)\n", - "load_param_into_net(model, load_checkpoint)\n", - "score, history = model(data, seq_length)\n", - "score" + "def download_file(url: str, path: str, replace: bool = True) -> str:\n", + " \"\"\"从 URL 下载文件到本地\"\"\"\n", + " if (not replace) and os.path.exists(path):\n", + " return path\n", + " os.makedirs(os.path.dirname(os.path.abspath(path)), exist_ok=True)\n", + " r = requests.get(url, stream=True, timeout=60)\n", + " r.raise_for_status()\n", + " with open(path, \"wb\") as f:\n", + " for chunk in r.iter_content(chunk_size=1024 * 1024):\n", + " if chunk:\n", + " f.write(chunk)\n", + " return path\n", + "\n", + "\n", + "# 下载/加载 ckpt（参数名与课程 ckpt 完全一致）\n", + "lstm_crf_url = (\n", + " \"https://modelers.cn/coderepo/web/v1/file/MindSpore-Lab/cluoud_obs/main/media/examples/\"\n", + " \"mindspore-courses/orange-pi-online-infer/08-LSTM%2BCRF/lstm-crf.ckpt\"\n", + ")\n", + "ckpt_path = download_file(lstm_crf_url, \"./lstm-crf.ckpt\", replace=False)\n", + "\n", + "param_dict = load_checkpoint(ckpt_path)\n", + "load_param_into_net(model, param_dict)\n" ] }, { @@ -628,24 +839,32 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 13, "id": "ed5ff78a-099c-41f8-ac1e-8f9fbfab7659", "metadata": {}, "outputs": [ { - "data": { - "text/plain": [ - "[[0, 1, 1, 1, 2, 2, 2, 2, 2, 0, 1], [0, 1, 2, 2, 2, 2, 2, 2, 2]]" - ] - }, - "execution_count": 20, - "metadata": {}, - "output_type": "execute_result" + "name": "stdout", + "output_type": "stream", + "text": [ + "predict idx: [[0, 1, 1, 1, 2, 2, 2, 2, 2, 0, 1], [0, 1, 2, 2, 2, 2, 2, 2, 2]]\n" + ] } ], "source": [ - "predict = post_decode(score, history, seq_length)\n", - "predict" + "# 1) 前向：得到 emissions (B, T, C)\n", + "emissions = model(data, seq_length) # fp16\n", + "emissions_np = emissions.astype(ms.float32).asnumpy()\n", + "\n", + "# 2) 取出 CRF 参数（仍来自课程 ckpt）\n", + "trans = model.crf.transitions.asnumpy()\n", + "start = model.crf.start_transitions.asnumpy()\n", + "end = model.crf.end_transitions.asnumpy()\n", + "\n", + "# 3) numpy viterbi 解码（避免 Ascend 上某些算子缺失/编译失败）\n", + "predict = viterbi_decode_numpy(emissions_np, seq_length.asnumpy(), trans, start, end)\n", + "\n", + "print(\"predict idx:\", predict)\n" ] }, { @@ -658,7 +877,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 14, "id": "df505983-9c1f-4e09-8a67-1cd78ce69fd0", "metadata": {}, "outputs": [], @@ -674,9 +893,11 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 15, "id": "e30d43c7-7c09-445e-94a4-33fccfaeaf11", - "metadata": {}, + "metadata": { + "tags": [] + }, "outputs": [ { "data": { @@ -685,7 +906,7 @@ " ['B', 'I', 'O', 'O', 'O', 'O', 'O', 'O', 'O']]" ] }, - "execution_count": 22, + "execution_count": 15, "metadata": {}, "output_type": "execute_result" } @@ -697,9 +918,9 @@ ], "metadata": { "kernelspec": { - "display_name": "MindSpore", + "display_name": "base", "language": "python", - "name": "mindspore" + "name": "base" }, "language_info": { "codemirror_mode": { @@ -711,7 +932,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.18" + "version": "3.9.2" }, "vscode": { "interpreter": { diff --git a/Online/inference/README.md b/Online/inference/README.md index 12f52d2..e503c44 100644 --- a/Online/inference/README.md +++ b/Online/inference/README.md @@ -17,7 +17,7 @@ |[ShuffleNet](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/05-ShuffleNet)| 8.0.RC3.alpha002 | 2.4.10| 8T16G | |[SSD](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/06-SSD)|8.1.RC1 | 2.6.0| 8T8G | |[RNN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/07-RNN)|8.1.RC1 | 2.6.0| 8T8G | -|[LSTM+CRF](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/08-LSTM%2BCRF)|8.0.RC3.alpha002 | 2.4.10| 8T16G | +|[LSTM+CRF](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/08-LSTM%2BCRF)|8.1.RC1 | 2.6.0| 8T16G | |[GAN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/09-GAN)|8.0.RC3.alpha002 | 2.4.10| 8T16G | |[DCGAN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/10-DCGAN)|8.1.RC1 | 2.6.0| 8T8G | |[Pix2Pix](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/11-Pix2Pix)|8.0.RC3.alpha002 | 2.4.10| 8T16G | diff --git a/README.md b/README.md index 7dc424b..f275eab 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ |[ShuffleNet](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/05-ShuffleNet)| 推理 | 8.0.RC3.alpha002 | 2.4.10| 8T16G | |[SSD](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/06-SSD)| 推理 | 8.1.RC1 | 2.6.0| 8T8G | |[RNN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/07-RNN)| 推理 | 8.0.RC3.alpha002 | 2.4.10| 8T16G | -|[LSTM+CRF](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/08-LSTM%2BCRF)| 推理 | 8.0.RC3.alpha002 | 2.4.10| 8T16G | +|[LSTM+CRF](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/08-LSTM%2BCRF)| 推理 | 8.1.RC1 | 2.6.0| 8T16G | |[GAN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/09-GAN)| 推理 | 8.0.RC3.alpha002 | 2.4.10| 8T16G | |[DCGAN](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/10-DCGAN)| 推理 | 8.1.RC1 | 2.6.0| 8T8G | |[Pix2Pix](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/inference/11-Pix2Pix)| 推理 | 8.0.RC3.alpha002 | 2.4.10| 8T16G | @@ -83,6 +83,7 @@ | [VideoClassification](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/VideoClassification) | 推理 | 8.0.0.beta1 | 2.6.0 |8T16G | | [MaskGeneration](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/MaskGeneration) | 推理 | 8.1.RC1 | 2.6.0 | 8T16G | | [DocumentQuestionAnswering](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DocumentQuestionAnswering) | 推理 | 8.0.0.beta1 | 2.6.0 | 20T24G | +| [DepthEstimation](https://github.com/mindspore-courses/orange-pi-mindspore/tree/master/Online/community/DepthEstimation) | 推理 | 8.1.RC1 | 2.6.0 | 8T16G | > 注：在线案例指导请参考Online文件夹中的README文档