Academic Research Explorer

Purpose:

The purpose of this application is to provide a platform for students to find potential advisors and for faculty members to explore research trends.

It targets students and faculty at universities, aiming to help students identify the most suitable professors based on research interests and publications. Additionally, faculty can track research trends and collaboration opportunities.

The objective is to provide an easy-to-use interface for querying academic data, viewing research trends, and managing favorite professors.

Demo: https://www.youtube.com/watch?v=9M2eCvHyTOY

Installation:

1. Clone the repository from https://github.com/NICKLIN13/Academic-Research-Explorer.git
2. Create the favorite_faculty table in MySQL
   • After cloning, in your local MySQL database, add a new table named favorite_faculty, which is used by widget 5 and widget 6.
   • This table is not included in the original dataset. Run the following SQL query to create it:

     CREATE TABLE favorite_faculty (
         faculty_name VARCHAR(255) NOT NULL,
         university_name VARCHAR(255),
         related_keyword TEXT,
         PRIMARY KEY (faculty_name, university_name)
     );

3. Create the dataset in MongoDB
   • For the mongodb, we didn't have publication_keyword or keywords in academicworld, so we took the existing excel sheets we had and wrote scripts to convert csv into json, then loaded the json into academicworld database.
4. Create a virtual environment
5. Install dependencies
   • Run pip install -r requirements.txt to install the necessary libraries listed in the requirements.txt file.
6. Run the application
   • On macOS, run python3 backend.py first, then run python3 frontend.py.
   • On Windows, run python backend.py first, then run python frontend.py.

Usage:

1. Users can start by viewing the "Top 10 U.S. universities with the most research publications" widget, shown in a bar chart to highlight research-oriented institutions.
2. Then, in the "Top 10 Publications" widget, by searching a university, users can see its 10 most cited publications, showing which works are most valued in academic community.
3. In the "Popular Research Keywords" widget, users can enter a year range to see the most cited keywords in that peroid of time, helping identify trending research areas.
4. In the "Professors and Publications" widget, users select a keyword to see related professors and their publications, revealing connections in that field.
5. In the "Search Faculty" widget, users can search for a professor by name and view their university and top 3 most related research keywords. Clicking "Add to Favorites" adds the professor to the "favorite_faculty" table in MySQL database.
6. The "Favorite Faculty" widget shows a list of favorite professors. Users can click "Clear" to remove all entries from the database. The data will remain until the "Clear" button is pressed, even if the page is accidentally refreshed.

Design:

Architecture:

• The application consists of a Dash frontend and three separate backend databases: MySQL, MongoDB, and Neo4j. These databases are accessed through REST APIs.
• User interacts with a widget → Dash frontend sends a request → Backend queries the appropriate database → Response is returned → Data is displayed on the frontend.
• Each widget is connected to a specific backend depending on the type of query or update it performs.

Components:

• Six different interactive widgets for users to query and update the database.
• Present the data clearly using bar charts, search bars, dropdown menus, buttons, and tables.
• The overall color scheme uses UIUC's ILI-orange and ILI-blue for a clear and distinct presentation.

Implementation:

We implemented the app using Dash and Flask. The frontend was built with Dash components such as dcc, html, dash_table, and visualized bar chart using Plotly Express.

For HTTP communication, we used Flask along with Flask-CORS to handle cross-origin requests, and requests for calling REST APIs. We used Pandas to process tabular data and integrate results from different databases. The backend connects to MySQL, MongoDB, and Neo4j using RESTful APIs.

Database Techniques:

• MySQL: Use Constraints such as PRIMARY KEY to manage data integrity. For example, in favorite_faculty, a composite primary key (faculty_name, university_name) ensures no duplicate favorites.
• MongoDB: Use Aggregation Pipeline to process data in stages. For example, in get_top_keywords, we use stages like match‘,‘match, match‘,‘unwind, group‘,‘group, group‘,‘sort, and $limit to filter by year, expand arrays, group by keyword, calculate metrics, and return the most cited keywords.
• Neo4j: Use Indexes to improve query performance on frequently searched properties. For example, CREATE INDEX IF NOT EXISTS FOR (k:KEYWORD) ON (k.name) speeds up keyword-based searches across the graph database's nodes and relationships.

The app uses REST ful endpoints to interact with all three databases in a unified way.

Contributions:

Joy:

• Tasks done:
  • Implemented widget 3 and completed MongoDB related features. Wrote Python scripts to convert publication_keyword.csv and keywords.csv into json files, then loaded json files into the academicworld database.
  • Implemented widget 4 and completed Neo4j related features.
  • Recorded the video demo and worked on README.
• Time spent: 20 hours

Nick:

• Tasks done:
  • Designed the overall app architecture and defined widget functionalities.
  • Implemented widgets 1, 2, 5, and 6, and completed MySQL related features.
  • Integrated project code and worked on README.
• Time spent: 20 hours