Skip to content
This repository was archived by the owner on Mar 8, 2020. It is now read-only.

Commit 70a1076

Browse files
juanjuxjuanjux
committed
Finished goget. Changed json to dont need cmdline parameter. Work on the normalizer
Added more mappings Bump up pydetector dependency version
1 parent d8879c5 commit 70a1076

36 files changed

+1357
-115
lines changed

‎.travis.yml

+18-13
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,18 @@
1-
language: python
2-
python:
3-
- "3.6"
4-
before_install:
5-
- sudo apt-get -qq update
6-
- sudo apt-get install python2.7
7-
install:
8-
- pip install -r requirements.txt
9-
- pip2 install pydetector
10-
- pip install -U .
11-
script:
12-
- python3.6 -m unittest discover test/
13-
- bash test/integration_test.sh
1+
language: go
2+
3+
go:
4+
- 1.8
5+
6+
services:
7+
- docker
8+
9+
before_script:
10+
- go get -v github.com/bblfsh/sdk/...
11+
- bblfsh-sdk prepare-build .
12+
- go get -v -t ./...
13+
14+
script:
15+
- make test
16+
17+
after_success:
18+
- make push

‎Dockerfile

+10-9
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,12 @@
1-
FROM python:3.6-alpine
2-
MAINTAINER source{d}
1+
# Dockerfile represents the container being use to run the driver, should be
2+
# small as possible containing strictly only the tools required to run the
3+
# driver.
34

4-
RUN apk add --update python py-pip git && rm -rf /var/cache/apk/*
5-
RUN pip3 install msgpack-python six \
6-
git+https://github.com/juanjux/python-pydetector.git
7-
RUN pip2 install msgpack-python six \
8-
git+https://github.com/juanjux/python-pydetector.git
9-
ADD bin /bin
5+
# The prefered base image is the lastest stable Alpine image, if alpine doesn't
6+
# meet the requirements you can switch the from to the latest stable slim
7+
# version of Debian (eg.: `debian:jessie-slim`). If the excution environment
8+
# is equals to the build environment the build image can be use as FROM:
9+
# bblfsh/<language>-driver-build
10+
FROM alpine:3.5
1011

11-
CMD ["python3", "bin/python_driver.py"]
12+
CMD /opt/driver/bin/driver

‎Dockerfile.build

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Dockerfile.build represents the build environment of the driver, used during
2+
# the development phase to test and in CI to build and test.
3+
4+
# The prefered base image is the lastest stable Alpine image, if alpine doesn't
5+
# meet the requirements you can switch the from to the latest stable slim
6+
# version of Debian (eg.: `debian:jessie-slim`).
7+
FROM alpine:3.5
8+
9+
# To avoid files written in the volume by root or foreign users, we create a
10+
# container local user with the same UID of the user executing the build.
11+
# The following commands are defined to use in busybox based distributions,
12+
# if you are using a standard distributions, replace the `adduser` command with:
13+
# `useradd --uid ${BUILD_UID} --home /opt/driver ${BUILD_USER}`
14+
RUN mkdir -p /opt/driver/src && \
15+
adduser ${BUILD_USER} -u ${BUILD_UID} -D -h /opt/driver/src
16+
17+
18+
# As minimal build tools you need: make, curl and git, install using the same
19+
# command the specific tools required to build the driver.
20+
RUN apk add --no-cache make git curl ca-certificates
21+
22+
23+
# The volume with the full source code is mounted at `/opt/driver/src` so, we
24+
# set the workdir to this path.
25+
WORKDIR /opt/driver/src

‎LICENSE

+674
Large diffs are not rendered by default.

‎Makefile

+11
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
include .sdk/Makefile
2+
3+
test-native:
4+
cd native; \
5+
echo "not implemented"
6+
7+
build-native:
8+
cd native; \
9+
echo "not implemented"
10+
echo -e "#!/bin/bash\necho 'not implemented'" > $(BUILD_PATH)/native
11+
chmod +x $(BUILD_PATH)/native

‎README.md

+13-76
Original file line numberDiff line numberDiff line change
@@ -1,88 +1,25 @@
1-
# Python driver [![Build Status](https://travis-ci.org/bblfsh/python-driver.svg?branch=master)] ![Python version](https://img.shields.io/badge/python%20version-3.6-0.svg)
1+
# python-driver ![Driver Status](https://img.shields.io/badge/status-beta-dbd25c.svg) [![Build Status](https://travis-ci.org/bblfsh/python-driver.svg?branch=master)](https://travis-ci.org/bblfsh/python-driver) ![Native Version](https://img.shields.io/badge/python%20version-1:2:3-aa93ea.svg) ![Go Version](https://img.shields.io/badge/go%20version-1.8-63afbf.svg)
22

33

4-
## Description
54

6-
This is the Python driver for bblfsh.
5+
Development Environment
6+
-----------------------
77

8-
## Status
8+
Requirements:
9+
- `docker`
10+
- [`bblfsh-sdk`](https://github.com/bblfsh/sdk) _(go get -u github.com/bblfsh/sdk/...)_
911

10-
### Implemented:
12+
To initialize the build system execute: `bblfsh-sdk prepare-build`, at the root of the project. This will install the SDK at `.sdk` for this driver.
1113

12-
- Python version detection
14+
To execute the tests just execute `make test`, this will execute the test over the native and the go components of the driver. Use `make test-native` to run the test only over the native component or `make test-driver` to run the test just over the go component.
1315

14-
- Decoupled design independent of protocol handlers or input/output buffers
16+
The build is done executing `make build`. To evaluate the result, a docker container, execute:
17+
`docker run -it bblfsh/java-driver:dev-<commit[:6]>`
1518

16-
- Improvements over the Python ast module: includes comments and whitespace.
17-
18-
- PEP426 package with dependency information instalable with PIP from github but
19-
not from Pypi; it will probably never be uploaded there since it doesn't
20-
make much sense to use this outside of bblfsh. The Python detection and rich
21-
AST extraction features are modularized in another package called "pydetector"
22-
that is already on Pypi.
23-
24-
- Fully typed used Python 3.5+3.6 type annotations.
25-
26-
- Unittest + integration tests + statical typecheck (using mypy). All are set as
27-
mandatore in the Travis config.
28-
29-
- Source code fully documented using docstrings.
30-
31-
32-
### TO-DO:
33-
34-
- Adapt to bblfsh SDK and package conventions.
35-
36-
- Byte perfect bidirectional conversion. The generated AST and the whitespace
37-
and comments added group more than one space as one and changes semicolons for
38-
newlines.
39-
40-
- Optimization. There is a lot of room for optimization in the current driver
41-
implementation (the typical Python tricks, pypy-friendliness, etc.)
42-
43-
- Move the rich-AST extractor to its own package (outside of Pydetector) and
44-
upload to pypi.
45-
46-
- Full unittest coverage of all modules (the NoopExtractor specifically
47-
lacks many tests).
48-
49-
50-
Installation
51-
------------
52-
53-
Using virtualenv is nice. Using [pyenv](https://github.com/yyuu/pyenv) +
54-
[pyenv virtualenv](https://github.com/yyuu/pyenv-virtualenv) plugin to avoid
55-
the mess of Linux distributions Python's deployments is god-like.
56-
57-
```bash
58-
# [Install Python 2.7.13 and 3.6.0 in whatever way its needed here]
59-
# (Or change master for the specific tag)
60-
python3.6 -m pip install -U https://github.com/bblfsh/python-driver/archive/master.zip
61-
# Python2.7 is needed to evaluate AST of Python 2.7
62-
python2.7 -m pip install pydetector
63-
```
64-
65-
If you want to develop and/or run the unit/typing/integration tests do this instead:
66-
67-
```bash
68-
# Or whatever you have to do to install these versions on your environment:
69-
pyenv install 3.6.0
70-
pyenv install 2.7.13 # (or whatever you have to do to install these python
71-
versions)
72-
73-
# Enable Python 2.7 to install pydetector
74-
pyenv global 2.7.13
75-
pip install pydetector
76-
77-
# Download the driver
78-
git clone https://github.com/bblfsh/python-driver python_driver
79-
cd python_driver
80-
pyenv global 3.6.0
81-
pip install -U .
82-
pip install -r requirements.txt
83-
cd test && python3.6 -m unittest discover && sh integration_test.sh
84-
```
8519

8620
License
8721
-------
22+
8823
GPLv3, see [LICENSE](LICENSE)
24+
25+

‎README_native.md

+88
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Python driver [![Build Status](https://travis-ci.org/bblfsh/python-driver.svg?branch=master)] ![Python version](https://img.shields.io/badge/python%20version-3.6-0.svg)
2+
3+
4+
## Description
5+
6+
This is the Python driver for bblfsh.
7+
8+
## Status
9+
10+
### Implemented:
11+
12+
- Python version detection
13+
14+
- Decoupled design independent of protocol handlers or input/output buffers
15+
16+
- Improvements over the Python ast module: includes comments and whitespace.
17+
18+
- PEP426 package with dependency information instalable with PIP from github but
19+
not from Pypi; it will probably never be uploaded there since it doesn't
20+
make much sense to use this outside of bblfsh. The Python detection and rich
21+
AST extraction features are modularized in another package called "pydetector"
22+
that is already on Pypi.
23+
24+
- Fully typed used Python 3.5+3.6 type annotations.
25+
26+
- Unittest + integration tests + statical typecheck (using mypy). All are set as
27+
mandatore in the Travis config.
28+
29+
- Source code fully documented using docstrings.
30+
31+
32+
### TO-DO:
33+
34+
- Adapt to bblfsh SDK and package conventions.
35+
36+
- Byte perfect bidirectional conversion. The generated AST and the whitespace
37+
and comments added group more than one space as one and changes semicolons for
38+
newlines.
39+
40+
- Optimization. There is a lot of room for optimization in the current driver
41+
implementation (the typical Python tricks, pypy-friendliness, etc.)
42+
43+
- Move the rich-AST extractor to its own package (outside of Pydetector) and
44+
upload to pypi.
45+
46+
- Full unittest coverage of all modules (the NoopExtractor specifically
47+
lacks many tests).
48+
49+
50+
Installation
51+
------------
52+
53+
Using virtualenv is nice. Using [pyenv](https://github.com/yyuu/pyenv) +
54+
[pyenv virtualenv](https://github.com/yyuu/pyenv-virtualenv) plugin to avoid
55+
the mess of Linux distributions Python's deployments is god-like.
56+
57+
```bash
58+
# [Install Python 2.7.13 and 3.6.0 in whatever way its needed here]
59+
# (Or change master for the specific tag)
60+
python3.6 -m pip install -U https://github.com/bblfsh/python-driver/archive/master.zip
61+
# Python2.7 is needed to evaluate AST of Python 2.7
62+
python2.7 -m pip install pydetector
63+
```
64+
65+
If you want to develop and/or run the unit/typing/integration tests do this instead:
66+
67+
```bash
68+
# Or whatever you have to do to install these versions on your environment:
69+
pyenv install 3.6.0
70+
pyenv install 2.7.13 # (or whatever you have to do to install these python
71+
versions)
72+
73+
# Enable Python 2.7 to install pydetector
74+
pyenv global 2.7.13
75+
pip install pydetector
76+
77+
# Download the driver
78+
git clone https://github.com/bblfsh/python-driver python_driver
79+
cd python_driver
80+
pyenv global 3.6.0
81+
pip install -U .
82+
pip install -r requirements.txt
83+
cd test && python3.6 -m unittest discover && sh integration_test.sh
84+
```
85+
86+
License
87+
-------
88+
GPLv3, see [LICENSE](LICENSE)

‎driver/main.go

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
package main
2+
3+
import (
4+
"fmt"
5+
"os"
6+
7+
"github.com/bblfsh/sdk"
8+
9+
_ "github.com/bblfsh/python-driver/driver/normalizer"
10+
)
11+
12+
var version string
13+
var build string
14+
15+
func main() {
16+
fmt.Printf("version: %s\nbuild: %s\n", version, build)
17+
18+
_, err := os.Stat(sdk.NativeBin)
19+
if err == nil {
20+
fmt.Println("native: ok")
21+
return
22+
}
23+
24+
fmt.Printf("native: %s\n", err)
25+
}

0 commit comments

Comments
 (0)