Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .env_example
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,7 @@ SENTRY_DSN=
PAPERSWITHCODE_USER=
PAPERSWITHCODE_PASS=
DB_URI=
DB_URI=
TWITTER_FREQ_HOURS=
PAPERS_WITH_CODE_FREQ_HOURS=
ARXIV_FREQ_HOURS=
FRONTEND_URL=
FRONTEND_URL=
80 changes: 70 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,80 @@
Read, collaborate and talk about scientific papers.

## Getting started
1. Install Postgres
2. Install `pandoc` to support references extraction from here: https://pandoc.org/installing.html
3. Install `pdftotext` to support acronyms extraction: `sudo apt-get install poppler-utils`
4. Create your Python 3.7 virtual env
5. Create `.env` file based on the `.env_example` and fill out the values
6. Run `flask fetch-papers` to grab some papers (you can stop the function after fetching ~200 papers)
7. Run `flask run`
8. See `SciHive.postman_collection.json` for some examples of queries

### 1) Install Postgres

```
podman run --name postgresql -p 5432:5432 -d postgres:latest
```

### Install pgadmin (optional)

Change the default settings.

```
podman run --name pgadmin -p 5050:80 -e "PGADMIN_DEFAULT_EMAIL=pgadmin4@pgadmin.org" -e "PGADMIN_DEFAULT_PASSWORD=admin" -d dpage/pgadmin4
```

Comment on lines +13 to +20
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that this is necessary here

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is optional. At least, people will know it is an option they can use and explore. It is a simple podman command to get started. Ideally, you would have a Dockerfile with a docker-compose file in this project. The setup and configuration time would have been saved for anyone who wants to contribute to this project.

For your information, Podman 2.0 was released less than one month ago.

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree in regards to Docker :)
#9

### 2) Install Pandoc

Pandoc is needed to support references extraction. Installation can be found here: https://pandoc.org/installing.html.

#### Archlinux
```
sudo pacman -S pandoc
```

### 3) Install pdftotext

`pdftotext` is needed to support acronyms extraction.

#### Archlinux
```
sudo pacman -S poppler
```

#### Ubuntu
```
sudo apt-get install poppler-utils
```

### 4) Configure a virtualenv

```
mkvirtualenv scihive
workon scihive

pip install -r requirements.txt
```

### 5) Set the configuration variables in the .env

Create `.env` file based on the `.env_example` and fill out the values


### 6) Prepare the backend

Run `flask fetch-arxiv` to grab some papers (you can stop the function after fetching ~200 papers)

### 7) Run the backend

To start the backend
```
flask run
```

To visualize the routes
```
flask routes
```

### 8) See `SciHive.postman_collection.json` for some examples of queries

## Production
- Repeat steps 2-5
- Run `sh restart_server.sh`

###Changelog
### Changelog

- May 31, 2019 - Acronym extraction and enrichment (from other papers)
- May 31, 2019 - Acronym extraction and enrichment (from other papers)
4 changes: 3 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@ flask-caching

schedule

bson

boto3
Flask-RESTful
Flask-JWT-Extended
Expand Down Expand Up @@ -44,4 +46,4 @@ pep8
pylint-flask
pylint-flask-sqlalchemy
sqlalchemy-easy-profile
itsdangerous==1.1.0
itsdangerous==1.1.0
3 changes: 2 additions & 1 deletion src/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from flask_caching import Cache
from .routes.paper import app as paper_routes
from .routes.comments import app as comments_routes
from .routes.replies import app as replies_routes
from .routes.paper_list import app as paper_list_routes
from .routes.user import app as user_routes
from .routes.groups import app as groups_routes
Expand Down Expand Up @@ -77,6 +78,7 @@ def before_send(event, hint):

app.register_blueprint(paper_list_routes, url_prefix='/papers')
app.register_blueprint(paper_routes, url_prefix='/paper')
app.register_blueprint(replies_routes, url_prefix='/reply')
app.register_blueprint(comments_routes, url_prefix='/paper')
app.register_blueprint(user_routes, url_prefix='/user')
app.register_blueprint(groups_routes, url_prefix='/groups')
Expand Down Expand Up @@ -108,7 +110,6 @@ def background_tasks():
def hello_world():
return 'Hello, World!'


@app.cli.command("migrate-db")
@click.option('--path', help='folder path')
def migrate_db(path):
Expand Down
55 changes: 55 additions & 0 deletions src/routes/replies.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import logging
from datetime import datetime

from flask import Blueprint
from flask_jwt_extended import get_jwt_identity, jwt_required
from flask_restful import (Api, Resource, abort, fields, inputs, marshal_with, reqparse)
from sqlalchemy import or_
from typing import Optional
from src.new_backend.models import Collection, Comment, Paper, Reply, db, User
from .user_utils import get_user

app = Blueprint('replies', __name__)
api = Api(app)
logger = logging.getLogger(__name__)

replies_fields = {
'id': fields.String,
'user': fields.String(attribute='user.username'),
'text': fields.String,
'createdAt': fields.DateTime(dt_format='rfc822', attribute='creation_date'),
}

EMPTY_FIELD_MSG = 'This field cannot be blank'

class ReplyResource(Resource):
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be confusing to have 2 ReplyResource in the code base. I suggest consolidating the two resources and replacing the post method below with patch

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll also ensure that we won't have two copies of replies_fields

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you. I would suggest the owner of the code to clean and remove unused code because it can be confusing. For example, MongoDB seems to be replaced with PostgreSQL. Also, new_backend, main, app, src in the same project makes it a little bit confusing. I think one common convention is app/main.py.

Copy link
Copy Markdown
Owner

@ranihorev ranihorev Jul 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely right (and I apologize about that), the code is still patchy but we're working on fixing that.
Would you like to update your code and we'll take care of re-organizing the rest?

method_decorators = [jwt_required]

def _get_reply(self, reply_id):

reply = Reply.query.get_or_404(reply_id)

user = get_user()

if reply.user != user:
abort(403, message='unauthorized to modify the reply')

return reply


@marshal_with(replies_fields, envelope='reply')
def post(self, reply_id):

reply = self._get_reply(reply_id)

edit_reply_parser = reqparse.RequestParser()
edit_reply_parser.add_argument('text', help=EMPTY_FIELD_MSG, type=str, location='json', required=False)
data = edit_reply_parser.parse_args()
reply.text = data['text']

db.session.commit()

return reply


api.add_resource(ReplyResource, "/<reply_id>")