Merge pull request #237 from nuest/main

Author: nuest

.github/workflows/ci.yml

@@ -52,21 +52,29 @@ jobs:
         python -m pip install -r requirements-dev.txt
   
     - name: Run Django migrations
+      env:
+        DATABASE_URL: postgis://optimap:optimap@localhost:5432/optimap
       run: |
         python manage.py migrate
 
     - name: Load all testdata to see if it is up to date with the Django migrations
+      env:
+        DATABASE_URL: postgis://optimap:optimap@localhost:5432/optimap
       run: |
         python manage.py loaddata fixtures/test_data_optimap.json
         python manage.py loaddata fixtures/test_data_partners.json
 
     - name: Run deploy checks
+      env:
+        DATABASE_URL: postgis://optimap:optimap@localhost:5432/optimap
       run: |
         python -Wa manage.py check --deploy
 
     - name: Run Tests
+      env:
+        DATABASE_URL: postgis://optimap:optimap@localhost:5432/optimap
       run: |
-        coverage run --source='publications' --omit='*/migrations/**' manage.py test tests
+        coverage run --source='works' --omit='*/migrations/**' manage.py test tests
 
     - name: Check coverage and save it to files
       run: |

.gitignore

@@ -151,3 +151,7 @@ publications/management/commands/marine_regions_iho.geojson
 publications/management/commands/world_continents.geojson
 
 .claude/temp.md
+
+works/management/commands/marine_regions_iho.geojson
+
+works/management/commands/world_continents.geojson

CHANGELOG.md

@@ -1,9 +1,47 @@
 # Changelog
 
+All notable changes to OPTIMAP are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
 ## [Unreleased]
 
 ### Added
 
+- **Geoextent API** - REST API exposing the [geoextent library](https://github.com/nuest/geoextent) for extracting geospatial and temporal extents from various file formats and remote repositories. Features include:
+  - `/api/v1/geoextent/extract/` - Extract from uploaded files (GeoJSON, GeoTIFF, Shapefile, GeoPackage, KML, CSV, etc.)
+  - `/api/v1/geoextent/extract-remote/` - Extract from remote repositories (Zenodo, PANGAEA, OSF, Figshare, Dryad, GFZ Data Services, Dataverse)
+  - `/api/v1/geoextent/extract-batch/` - Batch processing of multiple files with combined extent
+  - Multiple response formats: GeoJSON (default), WKT, WKB
+  - Support for bbox, convex hull, temporal extent, and placename geocoding
+  - Interactive web UI at `/geoextent/` with file upload, remote extraction, and map preview
+  - Comprehensive documentation and integration tests
+- **Geoextent web interface** - Interactive tool at `/geoextent/` for extracting spatial/temporal extents from data files:
+  - File upload with drag-and-drop support and size validation
+  - Remote resource extraction via DOI/URL (comma-separated identifiers)
+  - Interactive Leaflet map preview with clickable features showing properties
+  - Parameter customization (bbox, tbox, convex hull, placename, gazetteer selection)
+  - Response format selection (GeoJSON, WKT, WKB)
+  - Download results in selected format
+  - Documentation section with supported formats and providers
+  - Added to main menu and sitemaps
+- **Feeds sitemap** - Dynamic `/sitemap-feeds.xml` listing all regional feeds (continents and oceans) for search engine discovery
+- **Wikidata export** - Export publication metadata to Wikibase/Wikidata instances:
+  - Export works with spatial metadata to Wikidata
+  - Support for complex geometries (points, lines, polygons, multigeometry)
+  - Export extreme points (northernmost, southernmost, easternmost, westernmost) and geometric center
+  - Configurable via `WIKIBASE_*` environment variables
+- **Geocoding/gazetteer search** - Map search functionality allowing users to search for locations by name:
+  - Nominatim geocoder integration (default)
+  - Optional GeoNames support (requires username configuration)
+  - Search results displayed on map with zoom to location
+  - Accessible via search box in map interface
+- **Works list with pagination** - Browse all works page at `/works/list/` with:
+  - Configurable pagination (default 50 items per page)
+  - User-selectable page size with min/max limits
+  - Cached publication statistics (total works, published works, metadata completeness)
+  - Direct links to work landing pages
 - **Regional subscription system** - Users can subscribe to receive notifications for new publications from specific continents and oceans. Features include:
   - Checkbox-based UI with 8 continents and 7 oceans
   - "All Regions" checkbox to select/deselect all at once
@@ -26,6 +64,29 @@
 
 ### Changed
 
+- **Contribution page pagination** - Added full pagination support to the contribution page (`/contribute/`) with:
+  - Configurable page size (default 50, min 10, max 200 works per page)
+  - User-selectable page size dropdown with automatic form submission
+  - Full pagination controls at top and bottom (First, Previous, page numbers, Next, Last)
+  - Shows current range (e.g., "Showing 1 to 50 of 150 works")
+  - Fixed variable name bugs (`publication` → `work` throughout template)
+  - Reuses the same pagination layout as works listing page for consistency
+- **Model terminology alignment** - Renamed base entity from "publications" to "works" throughout the codebase to align with [OpenAlex terminology](https://docs.openalex.org/api-entities/works):
+  - Django app renamed from `publications/` to `works/`
+  - `Publication` model renamed to `Work`
+  - API endpoint changed from `/api/v1/publications/` to `/api/v1/works/`
+  - Sitemap updated from `/sitemap-publications.xml` to `/sitemap-works.xml`
+  - URL patterns updated from `/publication/<id>/` to `/work/<id>/`
+  - All import statements, templates, and configuration files updated
+  - Fresh migrations created from scratch
+  - All test fixtures updated
+- **Work type taxonomy** - Added comprehensive `type` field to works using Crossref/OpenAlex controlled vocabulary:
+  - 39 work types supported (article, book, book-chapter, dataset, preprint, dissertation, etc.)
+  - Type set from source's `default_work_type` during harvesting
+  - Overridden by OpenAlex API type when available
+  - Indexed and filterable in admin interface
+- **Removed external CDN dependencies** - All JavaScript and CSS libraries now served locally for improved privacy, security, and offline functionality
+- **Improved map accessibility** - Enhanced keyboard navigation and screen reader support for map interactions
 - **Regional subscription email notifications** - Notification emails now group publications by region with dedicated sections for each subscribed continent or ocean. Each region section includes:
   - Region name and type (Continent/Ocean)
   - Count of new publications in that region

CLAUDE.md

@@ -16,18 +16,18 @@ Part of the KOMET project (<https://projects.tib.eu/komet>), continuing from OPT
   - `settings.py` - All configuration via environment variables prefixed with `OPTIMAP_`
   - `.env` file for local config (see `.env.example` for all available parameters)
 
-- **publications/** - Main application containing all models, views, and business logic
-  - **Models** ([models.py](publications/models.py)):
-    - `Publication` - Core model with spatial (`GeometryCollectionField`) and temporal metadata
+- **works/** - Main application containing all models, views, and business logic
+  - **Models** ([models.py](works/models.py)):
+    - `Work` - Core model with spatial (`GeometryCollectionField`) and temporal metadata
     - `Source` - OAI-PMH harvesting sources
     - `HarvestingEvent` - Tracks harvesting jobs
     - `Subscription` - User subscriptions with spatial/temporal filters
     - `CustomUser` - Extended Django user model
     - `BlockedEmail`/`BlockedDomain` - Anti-spam mechanisms
-  - **Views** ([views.py](publications/views.py)) - Handles passwordless login, subscriptions, data downloads
-  - **Tasks** ([tasks.py](publications/tasks.py)) - Django-Q async tasks for harvesting and data export
-  - **API** ([api.py](publications/api.py), [viewsets.py](publications/viewsets.py), [serializers.py](publications/serializers.py)) - DRF REST API at `/api/v1/`
-  - **Feeds** ([feeds.py](publications/feeds.py), [feeds_geometry.py](publications/feeds_geometry.py)) - GeoRSS/GeoAtom feed generation
+  - **Views** ([views.py](works/views.py)) - Handles passwordless login, subscriptions, data downloads
+  - **Tasks** ([tasks.py](works/tasks.py)) - Django-Q async tasks for harvesting and data export
+  - **API** ([api.py](works/api.py), [viewsets.py](works/viewsets.py), [serializers.py](works/serializers.py)) - DRF REST API at `/api/v1/`
+  - **Feeds** ([feeds.py](works/feeds.py), [feeds_geometry.py](works/feeds_geometry.py)) - GeoRSS/GeoAtom feed generation
 
 ### Key Technologies
 
@@ -38,8 +38,8 @@ Part of the KOMET project (<https://projects.tib.eu/komet>), continuing from OPT
 
 ### Data Flow
 
-1. **Harvesting**: OAI-PMH sources → `HarvestingEvent` → parse XML → create `Publication` records with spatial/temporal metadata
-2. **API**: Publications exposed via REST API at `/api/v1/publications/` with spatial filtering
+1. **Harvesting**: OAI-PMH sources → `HarvestingEvent` → parse XML → create `Work` records with spatial/temporal metadata
+2. **API**: Publications exposed via REST API at `/api/v1/works/` with spatial filtering
 3. **Feeds**: Dynamic GeoRSS/GeoAtom feeds filtered by region or global
 4. **Data Export**: Scheduled tasks generate cached GeoJSON/GeoPackage dumps in `/tmp/optimap_cache/`
 
@@ -147,7 +147,7 @@ python manage.py flush                       # Clear all data from database (car
 
 # Shell access
 python manage.py shell                       # Django shell with models loaded
-python manage.py shell -c "from publications.tasks import regenerate_geojson_cache; regenerate_geojson_cache()"
+python manage.py shell -c "from works.tasks import regenerate_geojson_cache; regenerate_geojson_cache()"
 python manage.py dbshell                     # Direct PostgreSQL shell
 
 # Development server
@@ -166,7 +166,7 @@ python -Wa manage.py test                    # Show deprecation warnings
 
 #### Custom OPTIMAP Commands
 
-Located in [publications/management/commands/](publications/management/commands/)
+Located in [works/management/commands/](works/management/commands/)
 
 ```bash
 # Global regions setup
@@ -222,7 +222,7 @@ python manage.py loaddata fixtures/test_data_partners.json
 python manage.py loaddata fixtures/test_data_global_feeds.json
 
 # Manually regenerate GeoJSON/GeoPackage cache (without Django-Q)
-python manage.py shell -c "from publications.tasks import regenerate_geojson_cache; regenerate_geojson_cache()"
+python manage.py shell -c "from works.tasks import regenerate_geojson_cache; regenerate_geojson_cache()"
 ```
 
 ## Important Patterns
@@ -241,7 +241,7 @@ All deployment-specific config uses `OPTIMAP_*` environment variables loaded fro
 
 1. Create/configure `Source` in admin with OAI-PMH URL
 2. Django-Q task creates `HarvestingEvent`
-3. Fetch XML → parse → extract DOI, spatial, temporal metadata → save `Publication` records
+3. Fetch XML → parse → extract DOI, spatial, temporal metadata → save `Work` records
 4. Track status in `HarvestingEvent.status` (pending/in_progress/completed/failed)
 
 ### Authentication
@@ -271,7 +271,7 @@ All deployment-specific config uses `OPTIMAP_*` environment variables loaded fro
 ```
 optimap/
 ├── optimap/          # Django project settings
-├── publications/     # Main app (models, views, tasks, API)
+├── works/     # Main app (models, views, tasks, API)
 │   ├── management/commands/  # Custom Django commands
 │   ├── static/       # Frontend assets, logos
 │   └── templates/    # Django templates
@@ -366,7 +366,7 @@ GeoJSON, GeoTIFF, Shapefile, GeoPackage, KML, GML, GPX, FlatGeobuf, CSV (with la
 
 ### Geoextent Web UI
 
-Interactive web interface at [/geoextent](publications/templates/geoextent.html) for extracting geospatial/temporal extents from data files.
+Interactive web interface at [/geoextent](works/templates/geoextent.html) for extracting geospatial/temporal extents from data files.
 
 **Features:**
 
@@ -383,10 +383,10 @@ Interactive web interface at [/geoextent](publications/templates/geoextent.html)
 
 **Implementation:**
 
-- View: [publications/views.py](publications/views.py) - `geoextent()` function
+- View: [works/views.py](works/views.py) - `geoextent()` function
   - Uses `geoextent.lib.features.get_supported_features()` to dynamically load supported formats and providers
   - No hardcoded format lists - always reflects current geoextent capabilities
-- Template: [publications/templates/geoextent.html](publications/templates/geoextent.html)
+- Template: [works/templates/geoextent.html](works/templates/geoextent.html)
   - Uses Fetch API for AJAX requests (jQuery slim doesn't include $.ajax)
   - Interactive file management with add/remove functionality
   - Multiple file selection from different locations
@@ -406,8 +406,8 @@ Size limits passed from Django settings:
 
 **Navigation:**
 
-- Footer link added to [publications/templates/footer.html](publications/templates/footer.html)
-- URL route: `path("geoextent/", views.geoextent, name="geoextent")` in [publications/urls.py](publications/urls.py)
+- Footer link added to [works/templates/footer.html](works/templates/footer.html)
+- URL route: `path("geoextent/", views.geoextent, name="geoextent")` in [works/urls.py](works/urls.py)
 
 ## Version Management
 

fixtures/create_global_feeds_fixture.py

@@ -180,7 +180,7 @@
 def create_source(pk, name, issn_l=None, is_oa=True):
     """Create a source object."""
     return {
-        "model": "publications.source",
+        "model": "works.source",
         "pk": pk,
         "fields": {
             "name": name,
@@ -194,6 +194,7 @@ def create_source(pk, name, issn_l=None, is_oa=True):
             "is_oa": is_oa,
             "cited_by_count": random.randint(500, 50000),
             "is_preprint": random.choice([True, False]),
+            "default_work_type": "article",
         }
     }
 
@@ -247,7 +248,7 @@ def create_publication(pk, source_pk, title, abstract, geometry_wkt, region_desc
     )
 
     return {
-        "model": "publications.publication",
+        "model": "works.work",
         "pk": pk,
         "fields": {
             "status": "p",  # all published for UI testing
@@ -272,6 +273,7 @@ def create_publication(pk, source_pk, title, abstract, geometry_wkt, region_desc
             "openalex_is_retracted": is_retracted,
             "openalex_ids": openalex_ids,
             "openalex_open_access_status": openalex_open_access_status,
+            "type": "article",
         }
     }
 
@@ -564,7 +566,7 @@ def main():
         json.dump(fixture_data, f, indent=2)
 
     # Calculate statistics
-    publications = [item for item in fixture_data if item["model"] == "publications.publication"]
+    publications = [item for item in fixture_data if item["model"] == "works.work"]
 
     with_authors = sum(1 for p in publications if p["fields"]["authors"])
     with_keywords = sum(1 for p in publications if p["fields"]["keywords"])