Integration of InternAloha into RadGrad

[philipmjohnson]

One goal for Summer 2021 is to integrate the InternAloha system into RadGrad. InternAloha was a project we began last summer, initially as a standalone system to help Hawaii CS students find internships. Our goal was always for it to be integrated into RadGrad, because the information in RadGrad about each student's skills and interests can form useful information to help identify appropriate internships.

Ideally, InternAloha can not only be given "inputs" from RadGrad to support its task, but it can also provide "inputs" back into RadGrad. For example, if students indicate which Internships they are interested in, RadGrad could potentially recommend courses and opportunities that would help the student be a more attractive candidate for the Internship.

In this discussion, I want to solicit ideas on how to best integrate InternAloha. Currently, InternAloha works as follows:

1. A developer runs a set of "scrapers", which produce a set of JSON files with Internship information.
2. The developer deploys the InternAloha UI, which reads in those JSON files and presents potentially relevant internship opportunities to students based upon their selections for interests, location, and so forth.

Integration issues exist on both the back-end and the front-end:

Back-End Issues

It is unlikely that we will discover a way to completely automate the scraping process. Therefore, an integration with RadGrad will still require a developer to run scrapers regularly (perhaps twice a month?) to obtain the latest listings and put them into JSON files. Those files will be committed to the InternAloha repository where they are publicly accessable.

The interesting question is what happens next. Does RadGrad simply read the JSON files on demand? This is simple, but not having a database makes certain operations hard: for example, if a student wants to "ignore" a certain internship, how do we represent that data.

On the other hand, if we do read the data into a MongoDB collection, how do we handle "updates"? What happens when a listing is not found upon a future scraping? How do we determine that when we upload a new scraping snapshot, some are "the same" as they were before?

Front-End issues

What does the UI look like, now that we already know the student's interests, career goals, and (to a certain extent) their "skillset" (based on their completed course and opportunities)

How can we go beyond what InternAloha currently does? For example, how do we provide notifications when relevant Internships that the student has not already seen become available?

Cross-cutting issues

How do we allow people to manually add Internships? Is there a separate Collection type for Internships?

How do we integrate Internships as a "plugin" to RadGrad, so that departments who do not want this feature can disable it?

Development process issues

Do we use the RadGrad Project Board or the InternAloha project board?

[cammoore]

You talk about a student's skills. RadGrad doesn't have the concept of skills. Are we just talking about interests? Or do we need to add another collection and add skills to courses and opportunities?

For the updating from a JSON file, we could retire the internship that wasn't in the scraped data. This doesn't remove it from the system, just marks it as not being available.

[philipmjohnson]

Correct, "skills" would need to be added as a representation of what students learned from a course or opportunity. The goal is to then match this with the "prerequisites" for an internship so that the system could figure out what Internships the student is qualified for, and/or what courses/opps to take to become qualified.

Unfortunately, I don't think the current scrapers are good enough to extract "prerequisites" from the descriptions, and if so, then there's no point is representing skills.

The viability of skills will be something that the InternAloha team will need to figure out this summer.