1
1
# proxy_py
2
2
3
- proxy_py is a program which collects proxies, saves them in database and makes periodically checks. It has server for getting proxies with nice API(see below).
3
+ proxy_py is a program which collects proxies, saves them in a database
4
+ and makes periodically checks.
5
+ It has a server for getting proxies with nice API(see below).
4
6
5
7
## How to build?
6
8
@@ -19,7 +21,7 @@ pip3 install -r requirements.txt
19
21
20
22
` cp config_examples/settings.py proxy_py/settings.py `
21
23
22
- 4 (Optional) Change database in settings.py file
24
+ 4 Install postgresql and change database configuration in settings.py file
23
25
24
26
5 (Optional) Configure alembic
25
27
@@ -31,13 +33,18 @@ pip3 install -r requirements.txt
31
33
32
34
## I'm too lazy. Can I just use it?
33
35
34
- Yes, you can download virtualbox image [ here] ( https://drive.google.com/file/d/1oPf6xwOADRH95oZW0vkPr1Uu_iLDe9jc/view?usp=sharing ) . After downloading check that port forwarding is still working, you need forwarding of 55555 host port to 55555 guest.
36
+ Yes, you can download virtualbox image
37
+ [ here] ( https://drive.google.com/file/d/1oPf6xwOADRH95oZW0vkPr1Uu_iLDe9jc/view?usp=sharing ) .
38
+ After downloading check that port forwarding is still working,
39
+ you need forwarding of 55555 host port to 55555 guest.
35
40
36
41
## How to get proxies?
37
42
38
- proxy_py has server based on aiohttp which is listening 127.0.0.1:55555
39
- (you can change it in settings file) and provides proxies.
40
- To get proxies you should send following json request:
43
+ proxy_py has a server, based on aiohttp, which is listening 127.0.0.1:55555
44
+ (you can change it in the settings file) and provides proxies.
45
+ To get proxies you should send the following json request
46
+ on address ` http://127.0.0.1:55555/api/v1/ `
47
+ (or other domain if behind reverse proxy):
41
48
42
49
``` json
43
50
{
@@ -47,10 +54,11 @@ To get proxies you should send following json request:
47
54
}
48
55
```
49
56
50
- Note: order_by makes result sorting by one or more fields separated by comma.
57
+ Note: order_by makes the result sorted
58
+ by one or more fields(separated by comma).
51
59
You can skip it. The required fields are ` model ` and ` method ` .
52
60
53
- It will return json response like this:
61
+ It's gonna return you the json response like this:
54
62
55
63
``` json
56
64
{
@@ -75,7 +83,7 @@ It will return json response like this:
75
83
```
76
84
77
85
Note: All fields except * protocol* , * domain* , * port* , * auth_data* ,
78
- * checking_period* and * address* can be null
86
+ * checking_period* and * address* CAN be null
79
87
80
88
Or error if something went wrong:
81
89
@@ -91,13 +99,13 @@ Note: status_code is also duplicated in HTTP status code
91
99
92
100
Example using curl:
93
101
94
- ` curl -X POST http://example.com :55555 -H "Content-Type: application/json" --data '{"model": "proxy", "method": "get"}' `
102
+ ` curl -X POST http://127.0.0.1 :55555/api/v1/ -H "Content-Type: application/json" --data '{"model": "proxy", "method": "get"}' `
95
103
96
104
Example using httpie:
97
105
98
- ` http POST http://example.com :55555 model=proxy method=get `
106
+ ` http POST http://127.0.0.1 :55555/api/v1/ model=proxy method=get `
99
107
100
- Example using python requests library:
108
+ Example using python's ` requests ` library:
101
109
102
110
``` python
103
111
import requests
@@ -111,7 +119,7 @@ def get_proxies():
111
119
" method" : " get" ,
112
120
}
113
121
114
- response = requests.post(' http://example.com :55555' , json = json_data)
122
+ response = requests.post(' http://127.0.0.1 :55555/api/v1/ ' , json = json_data)
115
123
if response.status_code == 200 :
116
124
response = json.loads(response.text)
117
125
for proxy in response[' data' ]:
@@ -136,7 +144,7 @@ async def get_proxies():
136
144
}
137
145
138
146
async with aiohttp.ClientSession() as session:
139
- async with session.post(' http://example.com :55555' , json = json_data) as response:
147
+ async with session.post(' http://127.0.0.1 :55555/api/v1/ ' , json = json_data) as response:
140
148
if response.status == 200 :
141
149
response = json.loads(await response.text())
142
150
for proxy in response[' data' ]:
@@ -152,9 +160,13 @@ async def get_proxies():
152
160
153
161
Read more about API [ here] ( https://github.com/DevAlone/proxy_py/tree/master/docs/API.md )
154
162
163
+ ## How to contribute?
164
+
165
+ ` TODO: write guide about it `
166
+
155
167
## How to test it?
156
168
157
- If you made changes to code and want to check that you didn't break
169
+ If you made the changes to code and want to check that you didn't break
158
170
anything, go [ here] ( https://github.com/DevAlone/proxy_py/tree/master/docs/tests.md )
159
171
160
172
## How to deploy on production using supervisor, nginx and postgresql in 8 steps?
@@ -165,20 +177,25 @@ anything, go [here](https://github.com/DevAlone/proxy_py/tree/master/docs/tests.
165
177
166
178
2 Create virtual environment and install requirements on it
167
179
168
- 3 Install psycopg2
180
+ 3 Copy settings.py example:
169
181
170
- ` ( proxy_py) proxy_py @server:~/proxy_py$ pip3 install psycopg2 `
182
+ ` proxy_py@server:~/proxy_py$ cp config_examples/settings.py proxy_py/ `
171
183
172
- 4 create unprivileged user in postgresql database and add database authentication data to settings.py
184
+ 4 create unprivileged user in postgresql database
185
+ and change database authentication data in settings.py
173
186
174
187
``` bash
175
188
proxy_py@server:~ /proxy_py$ vim proxy_py/settings.py
176
189
```
177
190
178
191
``` bash
179
- DATABASE_CONNECTION_ARGS = (
180
- ' postgresql://USERNAME:PASSWORD@localhost/DB_NAME' ,
181
- )
192
+ DATABASE_CONNECTION_KWARGS = {
193
+ ' database' : ' YOUR_POSTGRES_DATABASE' ,
194
+ ' user' : ' YOUR_POSTGRES_USER' ,
195
+ ' password' : ' YOUR_POSTGRES_PASSWORD' ,
196
+ # number of simultaneous connections
197
+ # 'max_connections': 20,
198
+ }
182
199
```
183
200
184
201
5 Copy supervisor config example and change it for your case
@@ -196,7 +213,7 @@ root@server:~$ ln -s /etc/nginx/sites-available/proxy_py /etc/nginx/sites-enable
196
213
root@server:~ $ vim /etc/nginx/sites-available/proxy_py
197
214
```
198
215
199
- 7 Restart supervisor and nginx
216
+ 7 Restart supervisor and Nginx
200
217
201
218
``` bash
202
219
root@server:~ $ supervisorctl reread
@@ -209,4 +226,4 @@ root@server:~$ /etc/init.d/nginx restart
209
226
210
227
## What is it depend on?
211
228
212
- See requirements.txt
229
+ See ` requirements.txt `
0 commit comments