1
- #Python Diffbot API Client
1
+ # Python Diffbot API Client
2
2
3
3
4
- ##Preface
4
+ ## Preface
5
5
Identify and extract the important parts of any web page in Python! This client currently supports calls to Diffbot's Automatic APIs and Crawlbot.
6
6
7
7
@@ -10,7 +10,7 @@ To install activate a new virtual environment and run the following command:
10
10
11
11
$ pip install -r requirements.txt
12
12
13
- ##Configuration
13
+ ## Configuration
14
14
15
15
To run the example, you must first configure a working API token in config.py:
16
16
@@ -20,9 +20,9 @@ Then replace the string "SOME_TOKEN" with your API token. Finally, to run the e
20
20
21
21
$ python example.py
22
22
23
- ##Usage
23
+ ## Usage
24
24
25
- ###Article API
25
+ ### Article API
26
26
An example call to the Article API:
27
27
28
28
```
@@ -34,7 +34,7 @@ api = "article"
34
34
response = diffbot.request(url, token, api, version=2)
35
35
```
36
36
37
- ###Product API
37
+ ### Product API
38
38
An example call to the Product API:
39
39
40
40
```
@@ -46,7 +46,7 @@ api = "product"
46
46
response = diffbot.request(url, token, api, version=version)
47
47
```
48
48
49
- ###Image API
49
+ ### Image API
50
50
An example call to the Image API:
51
51
52
52
```
@@ -58,7 +58,7 @@ api = "image"
58
58
response = diffbot.request(url, token, api, version=version)
59
59
```
60
60
61
- ###Analyze API
61
+ ### Analyze API
62
62
An example call to the Analyze API:
63
63
64
64
```
@@ -70,7 +70,7 @@ api = "analyze"
70
70
response = diffbot.request(url, token, api, version=version)
71
71
```
72
72
73
- ###Crawlbot API
73
+ ### Crawlbot API
74
74
To start a new crawl, specify a crawl name, seed URLs, and the API via which URLs should be processed. An example call to the Crawlbot API:
75
75
76
76
```
@@ -117,12 +117,12 @@ To pass additional arguments to a crawl:
117
117
sampleCrawl = DiffbotCrawl(token,name,seeds,apiUrl,maxToCrawl=100,maxToProcess=50,notifyEmail="[email protected] ")
118
118
```
119
119
120
- ##Testing
120
+ ## Testing
121
121
122
122
First install the test requirements with the following command:
123
123
124
124
$ pip install -r test_requirements.txt
125
125
126
126
Currently there are some simple unit tests that mock the API calls and return data from fixtures in the filesystem. From the project directory, simply run:
127
127
128
- $ nosetests
128
+ $ nosetests
0 commit comments