Skip to content

Commit 01f3119

Browse files
author
madi
committed
Instructions how to train a neural network based on a pre-existing model, using TF object detection API
1 parent 71504cf commit 01f3119

File tree

1 file changed

+194
-0
lines changed

1 file changed

+194
-0
lines changed

‎docs/train.md

+194
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,194 @@
1+
2+
# Procedure for installing TensorFlow object detection API
3+
4+
```sh
5+
$ # First install TensorFlow and its dependencies
6+
$ sudo pip install pillow
7+
$ sudo pip install lxml
8+
$ sudo pip install jupyter
9+
$ sudo pip install matplotlib
10+
$ # For CPU
11+
$ sudo pip install tensorflow
12+
$ # Install object detection API
13+
$ mkdir tensorflow
14+
$ cd tensorflow/
15+
$ git clone https://github.com/madi/models.git
16+
$ cd models/research/
17+
$ protoc object_detection/protos/*.proto --python_out=.
18+
$ export PYTHONPATH=$PYTHONPATH:$HOME/tensorflow/models/research:$HOME/tensorflow/models/research/slim
19+
$ sudo python setup.py install
20+
$ # Test the installation
21+
$ python object_detection/builders/model_builder_test.py
22+
```
23+
24+
25+
# Procedure for training a new neural network with TensorFlow
26+
27+
### STEP 1: Labelling the images using labelImg
28+
29+
Label the target classes in the images using
30+
[labelImg](https://github.com/tzutalin/labelImg).
31+
32+
```sh
33+
$ sudo pip install labelImg
34+
```
35+
36+
### STEP 2: Split images in 2 folders, train (90%) and test (10%)
37+
$HOME/TensorFlow_utils/trees_recognition/images/train
38+
$HOME/TensorFlow_utils/trees_recognition/images/test
39+
40+
### STEP 3: Convert xml to csv using the utility xml_to_csv.py
41+
Original source: [xml_to_csv.py](https://github.com/datitran/raccoon_dataset/blob/master/xml_to_csv.py).
42+
Run the modified version in:
43+
44+
```sh
45+
$ cd $HOME/TensorFlow_utils/trees_recognition/
46+
$ python xml_to_csv.py
47+
```
48+
49+
### STEP 4: Create TFRecord using the utility generate_tfrecord.py
50+
Original source:: [generate_tfrecord.py](https://github.com/datitran/raccoon_dataset/blob/master/generate_tfrecord.py)
51+
Run the modified version in:
52+
53+
```sh
54+
$ cd $HOME/TensorFlow_utils/trees_recognition/
55+
$ # Create train data:
56+
$ python generate_tfrecord.py \
57+
--csv_input=$HOME/TensorFlow_utils/trees_recognition/data/train_labels.csv \
58+
--output_path=$HOME/TensorFlow_utils/trees_recognition/data/train.record \
59+
--images_path=$HOME/TensorFlow_utils/trees_recognition/images/train
60+
$
61+
$ # Create test data:
62+
$ python generate_tfrecord.py --csv_input=$HOME/TensorFlow_utils/trees_recognition/data/test_labels.csv \
63+
--output_path=$HOME/TensorFlow_utils/trees_recognition/data/test.record \
64+
--images_path=$HOME/TensorFlow_utils/trees_recognition/images/test
65+
```
66+
67+
### STEP 5: Create configuration file and choose the model
68+
69+
If you want, you can create your own model and you will need to create
70+
your config file following the instructions in [Configuring the Object
71+
Detection Training Pipeline](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/configuring_jobs.md).
72+
Alternatively, if you want to use a pre-existing model, you can pick an
73+
existing config file from [samples configs](https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs).
74+
For a comparison among available models, see [Tensorflow detection model
75+
zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md).
76+
77+
We choose
78+
[faster_rcnn_resnet101_coco](http://download.tensorflow.org/models/object_detection/faster_rcnn_resnet101_coco_2017_11_08.tar.gz).
79+
80+
In the config file faster_rcnn_resnet101_coco.config, we need to change:
81+
82+
* Number of classes
83+
* batch_size: if you get memory error you have to lower this number (not recommended!)
84+
* All paths indicated by "PATH_TO_BE_CONFIGURED"
85+
* Name of the .record file
86+
* label_map_path = $HOME/TensorFlow_utils/trees_recognition/training/trees_detection.pbtxt
87+
88+
Create $HOME/TensorFlow_utils/trees_recognition/training/trees_detection.pbtxt
89+
indicating the classes:
90+
91+
```sh
92+
item {
93+
id: 1
94+
name: 'sick'
95+
}
96+
item {
97+
id: 2
98+
name: 'dead'
99+
}
100+
```
101+
102+
### STEP 6: Create training folder
103+
104+
Create the following folder:
105+
106+
$HOME/TensorFlow_utils/trees_recognition/training
107+
108+
and put trees_detection.pbtxt and config file faster_rcnn_resnet101_coco.config
109+
inside it.
110+
111+
112+
### STEP 7: Launch the training
113+
114+
```sh
115+
$ cd $HOME/tensorflow/models/research/object_detection
116+
$ python train.py --logtostderr \
117+
--train_dir=$HOME/TensorFlow_utils/trees_recognition/training \
118+
--pipeline_config_path=$HOME/TensorFlow_utils/trees_recognition/training/faster_rcnn_resnet101_coco.config
119+
```
120+
121+
### STEP 8: Monitoring the training
122+
123+
Launch TensorBoard
124+
125+
```sh
126+
$ cd $HOME/tensorflow/models/research/
127+
$ tensorboard --logdir=$HOME/TensorFlow_utils/trees_recognition/training/
128+
129+
```
130+
131+
This command will create a file event* in the training folder, that is
132+
used by TensorBoard.
133+
Open TensorBoard in the browser.
134+
135+
### STEP 9: Stop the training
136+
137+
Watch the TotalLoss function in TensorBoard and stop the training when it
138+
converges towards 0.
139+
140+
### STEP 10: Export inference graph and use it for prediction
141+
142+
Use the utility
143+
[export_inference_graph.py](https://github.com/tensorflow/models/blob/master/research/object_detection/export_inference_graph.py)
144+
in the object_detection folder.
145+
146+
```sh
147+
$ cd $HOME/tensorflow/models/research/object_detection
148+
$ python export_inference_graph.py \
149+
--input_type image_tensor \
150+
--pipeline_config_path $HOME/TensorFlow_utils/trees_recognition/training/faster_rcnn_resnet101_coco.config \
151+
--trained_checkpoint_prefix $HOME/TensorFlow_utils/trees_recognition/training/model.ckpt-42845 \
152+
--output_directory $HOME/TensorFlow_utils/trees_recognition/training/tree_detection_graph
153+
```
154+
155+
where you should put the actual last recorded step in place of the number.
156+
157+
### STEP 11
158+
159+
Run prediction
160+
161+
```sh
162+
$ cd $HOME/TensorFlow_utils/
163+
$ python run_pred_bulk.py \
164+
--imagesPath=$HOME/TensorFlow_utils/trees_recognition/images/pred
165+
```
166+
167+
### STEP 12
168+
169+
If prediction is not satisfactory, we can resume the training. In config
170+
file, change:
171+
172+
```sh
173+
fine_tune_checkpoint: "$HOME/TensorFlow_utils/trees_recognition/training/model.ckpt-9261"
174+
```
175+
where 9261 is the last checkpoint.
176+
177+
Then launch the training again:
178+
179+
```sh
180+
$ cd $HOME/tensorflow/models/research/object_detection
181+
$ python train.py --logtostderr \
182+
--train_dir=$HOME/TensorFlow_utils/trees_recognition/training \
183+
--pipeline_config_path=$HOME/TensorFlow_utils/trees_recognition/training/faster_rcnn_resnet101_coco.config
184+
```
185+
186+
### STEP 13
187+
188+
Convert boxes into shapefiles using
189+
[convert_coords_boxes.py](https://github.com/madi/geo_tensorflow/blob/master/convert_coords_boxes.py)
190+
191+
```sh
192+
$ cd $HOME/TensorFlow_utils/
193+
$ python convert_coords_boxes.py
194+
```

0 commit comments

Comments
 (0)