This project aims to convert GUI photos or screenshots to Domain-Specific Language (DSL) code. The generated DSL code can be compiled and converted to various target languages such as HTML/CSS, Python's Tkinter, etc... . The goal is to provide an automated process for translating graphical user interfaces into code that can be used for development in different programming languages.
The repository contains the following components:
The core model that processes GUI images or screenshots and converts them into DSL code. This model is inspired by the Pix2Code paper, which demonstrates how deep learning can be used for translating UI designs to code.
A compiler that converts the generated DSL code into various target languages. Currently, it supports:
- HTML/CSS
The backend of the project, which includes the necessary endpoints to interact with the model and compile DSL code. This section contains all the logic to handle the conversion and compilation process.
The frontend for the project can be found in a separate repository: Frontend Repository
To set up this project locally:
-
Clone the repository:
git clone https://github.com/MohabASHRAF-byte/cody-generator
-
Install the necessary dependencies for the backend and the model:
Backend:
pip install -r requirements.txt
Frontend:
Follow the instructions in the frontend repository to set up the frontend.
To use the project:
-
Start the backend server:
uvicorn app.main:app --reload
-
Access the frontend and upload a GUI image or screenshot to be converted into DSL code.
-
Once the code is generated, it can be compiled into HTML, CSS, or Python Tkinter code.
The concept of converting GUI screenshots into code is inspired by the Pix2Code paper:
- Pix2Code: https://arxiv.org/pdf/1705.07962