Skip to content

Latest commit

 

History

History
237 lines (159 loc) · 4.82 KB

setup.md

File metadata and controls

237 lines (159 loc) · 4.82 KB

Setup and install

You can install Mage using Docker, pip, or conda.


Using Docker

  1. Create new project and launch tool

    docker run -it -p 6789:6789 -v $(pwd):/home/src mageai/mageai \
  mage start [project_name]

    Windows

    If you are running Mage in Docker on Windows OS, $(pwd) won’t work. Instead, use the following command:

    docker run -it -p 6789:6789 -v "C:\Some Path\To Your Current Directory":/home/src mageai/mageai \
  mage start [project_name]

  2. Open http://localhost:6789 in your browser and build a pipeline.

  3. Run pipeline after building it in the tool

    docker run -it -p 6789:6789 -v $(pwd):/home/src mageai/mageai \
  mage run [project_name] [pipeline]

Initialize new project

If you want to create a different project with a different name, run the following:

docker run -it -p 6789:6789 -v $(pwd):/home/src mageai/mageai \
  mage init [project_name]

Using PostgreSQL as database

Mage uses SQLite as the default database engine. To use PostgreSQL as the database engine, you'll need to run your Docker container slightly differently:

  1. Create Docker network

    docker network create mage-app

  2. Start PostgreSQL Docker container in a separate window

    docker run --network mage-app --network-alias postgres_db \
   -it -p 5432:5432 -v pgdata:/var/lib/postgresql/data \
   -e POSTGRES_USER=<username> -e POSTGRES_PASSWORD=<password> \
   -e POSTGRES_DB=<database> -e PG_DATA=/var/lib/postgresql/data/pgdata \
   postgres:13-alpine3.17 postgres

  3. Launch Mage with MAGE_DATABASE_CONNECTION_URL environment variable

    docker run --network mage-app -it -p 6789:6789 -v $(pwd):/home/src \
   -e MAGE_DATABASE_CONNECTION_URL=postgresql+psycopg2://<username>:<password>@postgres_db:5432/<database> \
   mageai/mageai \
   mage start another_repo

Using pip or conda

  1. Install Mage

    pip install mage-ai

    or

    conda install -c conda-forge mage-ai

    If you run into errors, see the errors section below.

  2. Create new project and launch tool

    mage start [project_name]

  3. Open http://localhost:6789 in your browser and build a pipeline.

  4. Run pipeline after building it in the tool

    mage run [project_name] [pipeline]

Initialize new project

If you want to create a different project with a different name, run the following:

mage init [project_name]

Download new version of Mage

Using Docker

docker pull mageai/mageai:latest

Using pip

pip install -U mage-ai

Environment variables

If you’re running Mage using pip or conda, your local machine’s environment variables are accessible within the running Mage app.

If you’re running Mage using Docker, you must add the following command line flags:

-e SOME_VARIABLE_NAME_1=secret_value_1 -e SOME_VARIABLE_NAME_2=secret_value_2

The command to run Mage using Docker could look like this:

docker run -it -p 6789:6789 -v $(pwd):/home/src mageai/mageai \
    -e SOME_VARIABLE_NAME_1=secret_value_1 -e SOME_VARIABLE_NAME_2=secret_value_2 \
    mage start demo_project

Install

Installing extra packages

Mage also has the following extras:

  • spark: to use Spark in your Mage pipeline
  • bigquery: to connect to BigQuery for data import or export
  • hdf5: to process HDF5 file format
  • postgres: to connect to PostgreSQL for data import or export
  • redshift: to connect to Redshift for data import or export
  • s3: to connect to S3 for data import or export
  • snowflake: to connect to Snowflake for data import or export
  • all: to install all of the above to use all functionalities

Example:

pip install "mage-ai[spark]"

Errors

You may need to install development libraries for MIT Kerberos to use some Mage features.

On Ubuntu, this can be installed as:

apt install libkrb5-dev

Development environment

To setup a development environment for editing source code, please check out this document.


Building your own Docker image

docker build -t mage/dangerous .

Testing from branch

Install mage-ai library from your branch:

$ pip3 install git+https://github.com/mage-ai/mage-ai.git@branch_name#egg=mage-ai

Build the front-end code:

$ cd mage_ai/frontend
$ yarn install
$ yarn export_prod

Testing

Backend

$ ./scripts/server/test.sh

Front-end

To check proper types in TypeScript, run:

$ ./scripts/test.sh