How Should We Organize the Wiki?

[liampond]

Following up on #302

I might be getting ahead of myself at this stage, but I'd like to think about how we should organize the Wiki. Ideally, I think I'd like to have a directory-like structure where the top-level directories are things like Getting Data Dumps, Reconciliation, and Virtuoso (#303). Sub-directories for Virtuoso in particular could include much of what we already have in the root directory like Virtuoso Setup Guide and Working with Virtuoso. However, it doesn't seem that there is an easy and efficient way to do this in GitHub (see this thread.

Here are some ideas:

---

Custom Sidebar

We could use a custom sidebar with indentation to give the sidebar the visual appearance of having a directory-like structure. However, I don't believe this would update automatically, so every time a new page is created, the author would have to remember to place it in the appropriate directory of the custom sidebar. If this system is not well-maintained, this could be more trouble than it's worth and we might end up with more confusing documentation.

Everything in Root

We could continue with the system we're using now where everything's in root. However, it already feels somewhat cluttered and this problem will probably only get worse.

Custom Sub-sidebar

The main sidebar can be done by GitHub automatically, but individual pages can have their own custom sidebars. These sub-sidebars (and even sub-sub-sidebars) can link to other pages and display a directory-like structure. However, I don't think this is ideal because even if we make a Virtuoso page and give it a sub-sidebar that has pages like Setup Guide and Data Import, these subpages will still appear in the main sidebar done by GitHub.

---

I'm surprised that there isn't a more organized built-in feature, because this seems like a fairly common problem. I haven't spent a ton of time researching so if anyone comes across a nice solution, or if anything I've said is incorrect, please let me know.

[liampond]

I think what we should do is have everything in root and have GitHub automatically create the sidebar. We can significantly condense and combine the pages we have to reduce clutter. Below is a draft schema that ChatGPT made. Obviously, for now, we will omit or combine many of the later pages it suggested because we're not at that point yet.

---

LinkedMusic Documentation Wiki

Below is a high-level structure for our revamped GitHub/GitLab Wiki. Each top-level page corresponds to a major step or theme in the LinkedMusic workflow, with subpages diving into specifics.

---

1. Home / Overview

• Purpose & Scope: What is LinkedMusic? Why this project?
• Architecture at a Glance: Diagram + narrative of how data flows through the system.
• Implemented Databases: List of the 14 current sources (with links).
• Planned/Future Databases: Roadmap for upcoming integrations.
• Glossary & Acronyms: High‑level definitions of key terms.

---

2. Data Ingestion

2.1 Process Overview

• Goals & design decisions
• Folder structure, naming conventions

2.2 Source–Specific Guides

• Source A (e.g., MusicBrainz)

  • Download approach (API vs dump)
  • File formats, schema notes

• Source B (e.g., Discogs)

• …

2.3 Automation & Scheduling

• Scripts, cron jobs, error handling, logging

---

3. Data Reconciliation

3.1 Why Reconcile?

• Challenges of heterogeneous metadata
• Matching & merging strategy

3.2 OpenRefine Workflow

• Clustering methods (key collision, Levenshtein, etc.)
• Reconciliation services & reconciliation schema

3.3 Per-Database Reconciliation Notes

• Custom rules for each source

---

4. Exporting & Preparing Data

• OpenRefine Exports (RDF, TSV, JSON-LD)
• Mapping configuration files
• Validation steps

---

5. Data Import into Virtuoso

• RDF bulk loader usage
• Graph naming conventions
• Namespace & prefix setup

---

6. Schema & Ontology

• Our core ontology design principles
• Mapping source fields → ontology classes/properties
• SPARQL inferencing rules

---

7. SPARQL & NLQ-to-SPARQL

7.1 Introduction to SPARQL

• Query patterns, examples

7.2 NLQ2SPARQL Pipeline

• Architecture (NLQ service → parser → query executor)
• Models & templates used

7.3 Developer Guide

• How to extend grammar, add new templates

---

8. Metasearch with SESEMMI

• API endpoints
• Query expansion & federated search logic

---

9. Client & Frontend

• Example queries UI
• Embeddable widgets
• Authentication & permissions

---

10. Internationalization & Accessibility

• Language support
• RTL, fonts, i18n framework

---

11. CI/CD & Deployment

• Automating ingestion & reconciliation on commit
• Virtuoso VM/container setup
• Monitoring & alerts

---

12. Contributing & Governance

• How to propose a new source
• Branching & PR guidelines

---

13. Appendix

• Troubleshooting common errors
• FAQ
• References & further reading

[liampond]

Here's how we might structure our documentation:

Documentation Audience Separation

Wiki (general audience):

• High-level project vision, roadmap, “what and why,” glossaries, contribution guide.
• Anyone (domain experts, new hires, musicologists) can wander in and quickly understand what LinkedMusic is and how to get involved.

In-repo /docs (developer audience):

• Step-by-step tutorials: running the pipeline, configuring reconciliations, SPARQL extensions, CI/CD hookups.
• API specs, config reference, code-adjacent deep dives that must stay in sync with the codebase.

README.md (everyone):

• One-page elevator pitch + “clone & run” quick start + pointers to both the Wiki and /docs.