Merge branch 'develop' into 11159-anon-access-doc

doc/release-notes/11129-send-feedback-to-contacts.md

@@ -0,0 +1,14 @@
+This feature adds a new API to send feedback to the Collection, Dataset, or DataFile's contacts.
+Similar to the "admin/feedback" API the "sendfeedback" API sends an email to all the contacts listed for the Dataset. The main differences for this feature are:
+1. This API is not limited to Admins
+2. This API does not return the email addresses in the "toEmail" and "ccEmail" elements for privacy reasons
+3. This API can be rate limited to avoid spamming
+4. The body size limit can be configured
+5. The body will be stripped of any html code to prevent malicious scripts or links
+6. The fromEmail will be validated for correct format
+
+To set the Rate Limiting for guest users (See Rate Limiting Configuration for more details. This example allows 1 send per hour for any guest)
+``curl http://localhost:8080/api/admin/settings/:RateLimitingCapacityByTierAndAction -X PUT -d '[{\"tier\": 0, \"limitPerHour\": 1, \"actions\": [\"CheckRateLimitForDatasetFeedbackCommand\"]}]'``
+
+To set the message size limit (example limit of 1080 chars):
+``curl -X PUT -d 1080 http://localhost:8080/api/admin/settings/:ContactFeedbackMessageSizeLimit``

doc/sphinx-guides/source/api/native-api.rst

@@ -2680,6 +2680,7 @@ The fully expanded example above (without environment variables) looks like this
 The review process can sometimes resemble a tennis match, with the authors submitting and resubmitting the dataset over and over until the curators are satisfied. Each time the curators send a "reason for return" via API, that reason is sent by email and is persisted into the database, stored at the dataset version level.
 Note the reason is required, unless the `disable-return-to-author-reason` feature flag has been set (see :ref:`feature-flags`). Reason is a free text field and could be as simple as "The author would like to modify his dataset", "Files are missing", "Nothing to report" or "A curation report with comments and suggestions/instructions will follow in another email" that suits your situation.
 
+The :ref:`send-feedback-admin` Admin only API call may be useful as a way to move the conversation to email. However, note that these emails go to contacts (versus authors) and there is no database record of the email contents. (:ref:`dataverse.mail.cc-support-on-contact-email` will send a copy of these emails to the support email address which would provide a record.)
 The :ref:`send-feedback` API call may be useful as a way to move the conversation to email. However, note that these emails go to contacts (versus authors) and there is no database record of the email contents. (:ref:`dataverse.mail.cc-support-on-contact-email` will send a copy of these emails to the support email address which would provide a record.)
 
 Link a Dataset
@@ -6832,10 +6833,10 @@ A curl example using allowing access to a dataset's metadata
 Please see :ref:`dataverse.api.signature-secret` for the configuration option to add a shared secret, enabling extra
 security.
 
-.. _send-feedback:
+.. _send-feedback-admin:
 
-Send Feedback To Contact(s)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Send Feedback To Contact(s) Admin API
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 This API call allows sending an email to the contacts for a collection, dataset, or datafile or to the support email address when no object is specified.
 The call is protected by the normal /admin API protections (limited to localhost or requiring a separate key), but does not otherwise limit the sending of emails.
@@ -6858,6 +6859,44 @@ A curl example using an ``ID``
 
 Note that this call could be useful in coordinating with dataset authors (assuming they are also contacts) as an alternative/addition to the functionality provided by :ref:`return-a-dataset`.
 
+.. _send-feedback:
+
+Send Feedback To Contact(s)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This API call allows sending an email to the contacts for a collection, dataset, or datafile or to the support email address when no object is specified.
+The call is protected from embedded html in the body as well as the ability to configure body size limits and rate limiting to avoid the potential for spam.
+
+The call is a POST with a JSON object as input with four keys:
+- "targetId" - the id of the collection, dataset, or datafile. Persistent ids and collection aliases are not supported. (Optional)
+- "identifier" - the alias of a collection or the persistence id of a dataset or datafile. (Optional)
+- "subject" - the email subject line. (Required)
+- "body" - the email body to send (Required)
+- "fromEmail" - the email to list in the reply-to field. (Dataverse always sends mail from the system email, but does it "on behalf of" and with a reply-to for the specified user. Authenticated users will have the 'fromEmail' filled in from their profile if this field is not specified)
+
+A curl example using an ``ID``
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export JSON='{"targetId":24, "subject":"Data Question", "body":"Please help me understand your data. Thank you!"}'
+
+  curl -X POST -H "X-Dataverse-key:$API_KEY" -H 'Content-Type:application/json' -d "$JSON" "$SERVER_URL/api/sendfeedback"
+
+
+A curl example using a ``Dataverse Alias or Dataset/DataFile PersistentId``
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export JSON='{"identifier":"root", "subject":"Data Question", "body":"Please help me understand your data. Thank you!"}'
+
+  curl -X POST -H "X-Dataverse-key:$API_KEY" -H 'Content-Type:application/json' -d "$JSON" "$SERVER_URL/api/sendfeedback"
+
+Note that this call could be useful in coordinating with dataset authors (assuming they are also contacts) as an alternative/addition to the functionality provided by :ref:`return-a-dataset`.
+
 .. _thumbnail_reset:
 
 Reset Thumbnail Failure Flags

doc/sphinx-guides/source/installation/config.rst

@@ -4419,7 +4419,12 @@ This is enabled via the new setting `:MDCStartDate` that specifies the cut-over
 
 ``curl -X PUT -d '2019-10-01' http://localhost:8080/api/admin/settings/:MDCStartDate``
 
+:ContactFeedbackMessageSizeLimit
+++++++++++++++++++++++++++++++++
+
+Maximum length of the text body that can be sent to the contacts of a Collection, Dataset, or DataFile. Setting this limit to Zero will denote unlimited length.
 
+``curl -X PUT -d 1080 http://localhost:8080/api/admin/settings/:ContactFeedbackMessageSizeLimit``
 
 .. _:Languages:
 

src/main/java/edu/harvard/iq/dataverse/api/SendFeedbackAPI.java

@@ -0,0 +1,126 @@
+package edu.harvard.iq.dataverse.api;
+
+import edu.harvard.iq.dataverse.*;
+import edu.harvard.iq.dataverse.api.auth.AuthRequired;
+import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser;
+import edu.harvard.iq.dataverse.authorization.users.User;
+import edu.harvard.iq.dataverse.branding.BrandingUtil;
+import edu.harvard.iq.dataverse.engine.command.impl.CheckRateLimitForDatasetFeedbackCommand;
+import edu.harvard.iq.dataverse.feedback.Feedback;
+import edu.harvard.iq.dataverse.feedback.FeedbackUtil;
+import edu.harvard.iq.dataverse.util.BundleUtil;
+import edu.harvard.iq.dataverse.util.cache.CacheFactoryBean;
+import edu.harvard.iq.dataverse.util.json.JsonUtil;
+import edu.harvard.iq.dataverse.validation.EMailValidator;
+import jakarta.ejb.EJB;
+import jakarta.json.*;
+import jakarta.mail.internet.InternetAddress;
+import jakarta.ws.rs.Consumes;
+import jakarta.ws.rs.POST;
+import jakarta.ws.rs.Path;
+import jakarta.ws.rs.container.ContainerRequestContext;
+import jakarta.ws.rs.core.Context;
+import jakarta.ws.rs.core.Response;
+
+import java.text.MessageFormat;
+import java.util.logging.Logger;
+
+@Path("sendfeedback")
+public class SendFeedbackAPI extends AbstractApiBean {
+    private static final Logger logger = Logger.getLogger(SendFeedbackAPI.class.getCanonicalName());
+    @EJB
+    MailServiceBean mailService;
+    @EJB
+    CacheFactoryBean cacheFactory;
+    /**
+     * This method mimics the contact form and sends an email to the contacts of the
+     * specified Collection/Dataset/DataFile, optionally ccing the support email
+     * address, or to the support email address when there is no target object.
+     **/
+    @POST
+    @AuthRequired
+    public Response submitFeedback(@Context ContainerRequestContext crc, String jsonString) {
+        try {
+            JsonObject jsonObject = JsonUtil.getJsonObject(jsonString);
+            if (!jsonObject.containsKey("subject") || !jsonObject.containsKey("body")) {
+                return badRequest(BundleUtil.getStringFromBundle("sendfeedback.body.error.missingRequiredFields"));
+            }
+
+            JsonNumber jsonNumber = jsonObject.containsKey("targetId") ? jsonObject.getJsonNumber("targetId") : null;
+            // idtf will hold the "targetId" or the "identifier". If neither is set then this is a general feedback to support
+            String idtf = jsonNumber != null ? jsonNumber.toString() : jsonObject.containsKey("identifier") ? jsonObject.getString("identifier") : null;
+            DvObject feedbackTarget = null;
+
+            if (jsonNumber != null) {
+                feedbackTarget = dvObjSvc.findDvObject(jsonNumber.longValue());
+            } else if (idtf != null) {
+                if (feedbackTarget == null) {
+                    feedbackTarget = dataverseSvc.findByAlias(idtf);
+                }
+                if (feedbackTarget == null) {
+                    feedbackTarget = dvObjSvc.findByGlobalId(idtf, DvObject.DType.Dataset);
+                }
+                if (feedbackTarget == null) {
+                    feedbackTarget = dvObjSvc.findByGlobalId(idtf, DvObject.DType.DataFile);
+                }
+            }
+
+            // feedbackTarget and idtf are both null this is a support feedback and is ok
+            if (feedbackTarget == null && idtf != null) {
+                return error(Response.Status.BAD_REQUEST, BundleUtil.getStringFromBundle("sendfeedback.request.error.targetNotFound"));
+            }
+            // Check for rate limit exceeded.
+            if (!cacheFactory.checkRate(getRequestUser(crc), new CheckRateLimitForDatasetFeedbackCommand(null, feedbackTarget))) {
+                return error(Response.Status.TOO_MANY_REQUESTS, BundleUtil.getStringFromBundle("sendfeedback.request.rateLimited"));
+            }
+
+            DataverseSession dataverseSession = null;
+            String userMessage = sanitizeBody(jsonObject.getString("body"));
+            InternetAddress systemAddress = mailService.getSupportAddress().orElse(null);
+            String userEmail = getEmail(jsonObject, crc);
+            String messageSubject = jsonObject.getString("subject");
+            String baseUrl = systemConfig.getDataverseSiteUrl();
+            String installationBrandName = BrandingUtil.getInstallationBrandName();
+            String supportTeamName = BrandingUtil.getSupportTeamName(systemAddress);
+            JsonArrayBuilder jab = Json.createArrayBuilder();
+            Feedback feedback = FeedbackUtil.gatherFeedback(feedbackTarget, dataverseSession, messageSubject, userMessage, systemAddress, userEmail, baseUrl, installationBrandName, supportTeamName, SendFeedbackDialog.ccSupport(feedbackTarget));
+            jab.add(feedback.toLimitedJsonObjectBuilder());
+            mailService.sendMail(feedback.getFromEmail(), feedback.getToEmail(), feedback.getCcEmail(), feedback.getSubject(), feedback.getBody());
+            return ok(jab);
+        } catch (WrappedResponse resp) {
+            return resp.getResponse();
+        } catch (JsonException je) {
+            return error(Response.Status.BAD_REQUEST, "Invalid JSON; error message: " + je.getMessage());
+        }
+    }
+
+    private String getEmail(JsonObject jsonObject, ContainerRequestContext crc) throws WrappedResponse {
+        String fromEmail = jsonObject.containsKey("fromEmail") ? jsonObject.getString("fromEmail") : "";
+        if (fromEmail.isBlank() && crc != null) {
+            User user = getRequestUser(crc);
+            if (user instanceof AuthenticatedUser) {
+                fromEmail = ((AuthenticatedUser) user).getEmail();
+            }
+        }
+        if (fromEmail == null || fromEmail.isBlank()) {
+            throw new WrappedResponse(badRequest(BundleUtil.getStringFromBundle("sendfeedback.fromEmail.error.missing")));
+        }
+        if (!EMailValidator.isEmailValid(fromEmail)) {
+            throw new WrappedResponse(badRequest(MessageFormat.format(BundleUtil.getStringFromBundle("sendfeedback.fromEmail.error.invalid"), fromEmail)));
+        }
+        return fromEmail;
+    }
+    private String sanitizeBody (String body) throws WrappedResponse {
+        // remove malicious html
+        String sanitizedBody = body == null ? "" : body.replaceAll("\\<.*?>", "");
+
+        long limit = systemConfig.getContactFeedbackMessageSizeLimit();
+        if (limit > 0 && sanitizedBody.length() > limit) {
+            throw new WrappedResponse(badRequest(MessageFormat.format(BundleUtil.getStringFromBundle("sendfeedback.body.error.exceedsLength"), sanitizedBody.length(), limit)));
+        } else if (sanitizedBody.length() == 0) {
+            throw new WrappedResponse(badRequest(BundleUtil.getStringFromBundle("sendfeedback.body.error.isEmpty")));
+        }
+
+        return sanitizedBody;
+    }
+}

src/main/java/edu/harvard/iq/dataverse/engine/command/impl/CheckRateLimitForDatasetFeedbackCommand.java

@@ -0,0 +1,18 @@
+package edu.harvard.iq.dataverse.engine.command.impl;
+
+import edu.harvard.iq.dataverse.DvObject;
+import edu.harvard.iq.dataverse.engine.command.AbstractVoidCommand;
+import edu.harvard.iq.dataverse.engine.command.CommandContext;
+import edu.harvard.iq.dataverse.engine.command.DataverseRequest;
+import edu.harvard.iq.dataverse.engine.command.exception.CommandException;
+
+public class CheckRateLimitForDatasetFeedbackCommand extends AbstractVoidCommand {
+
+    public CheckRateLimitForDatasetFeedbackCommand(DataverseRequest aRequest, DvObject dvObject) {
+        super(aRequest, dvObject);
+    }
+
+    @Override
+    protected void executeImpl(CommandContext ctxt) throws CommandException { }
+}
+

src/main/java/edu/harvard/iq/dataverse/feedback/Feedback.java

@@ -55,4 +55,10 @@ public JsonObjectBuilder toJsonObjectBuilder() {
                 .add("body", body);
     }
 
+    public JsonObjectBuilder toLimitedJsonObjectBuilder() {
+        return new NullSafeJsonBuilder()
+                .add("fromEmail", fromEmail)
+                .add("subject", subject)
+                .add("body", body);
+    }
 }

src/main/java/edu/harvard/iq/dataverse/settings/SettingsServiceBean.java

@@ -684,7 +684,9 @@ Whether Harvesting (OAI) service is enabled
          * When ingesting tabular data files, store the generated tab-delimited 
          * files *with* the variable names line up top. 
          */
-        StoreIngestedTabularFilesWithVarHeaders
+        StoreIngestedTabularFilesWithVarHeaders,
+
+        ContactFeedbackMessageSizeLimit
         ;
 
         @Override
@@ -749,6 +751,23 @@ public Long getValueForKeyAsLong(Key key){
             return null;
         }
 
+    }
+
+    /**
+     * Attempt to convert the value to an integer
+     *  - Applicable for keys such as MaxFileUploadSizeInBytes
+     *
+     * On failure (key not found or string not convertible to a long), returns defaultValue
+     * @param key
+     * @param defaultValue
+     * @return
+     */
+    public Long getValueForKeyAsLong(Key key, Long defaultValue) {
+           Long val = getValueForKeyAsLong(key);
+           if (val == null) {
+               return defaultValue;
+           }
+           return val;
     }
 
        /**

src/main/java/edu/harvard/iq/dataverse/util/SystemConfig.java

@@ -1173,4 +1173,8 @@ public String getRateLimitsJson() {
     public String getRateLimitingDefaultCapacityTiers() {
         return settingsService.getValueForKey(SettingsServiceBean.Key.RateLimitingDefaultCapacityTiers, "");
     }
+
+    public long getContactFeedbackMessageSizeLimit() {
+        return settingsService.getValueForKeyAsLong(SettingsServiceBean.Key.ContactFeedbackMessageSizeLimit, 0L);
+    }
 }

src/main/java/edu/harvard/iq/dataverse/util/cache/CacheFactoryBean.java

@@ -52,6 +52,9 @@ public boolean checkRate(User user, Command command) {
         int capacity = RateLimitUtil.getCapacity(systemConfig, user, action);
         if (capacity == RateLimitUtil.NO_LIMIT) {
             return true;
+        } else if (capacity == RateLimitUtil.RESET_CACHE) {
+            rateLimitCache.clear();
+            return true;
         } else {
             String cacheKey = RateLimitUtil.generateCacheKey(user, action);
             return (!RateLimitUtil.rateLimited(rateLimitCache, cacheKey, capacity));

src/main/java/edu/harvard/iq/dataverse/util/cache/RateLimitUtil.java

@@ -25,6 +25,8 @@ public class RateLimitUtil {
     static final List<RateLimitSetting> rateLimits = new CopyOnWriteArrayList<>();
     static final Map<String, Integer> rateLimitMap = new ConcurrentHashMap<>();
     public static final int NO_LIMIT = -1;
+    public static final int RESET_CACHE = -2;
+    static String settingRateLimitsJson = "";
 
     static String generateCacheKey(final User user, final String action) {
         return (user != null ? user.getIdentifier() : GuestUser.get().getIdentifier()) +
@@ -34,6 +36,15 @@ static int getCapacity(SystemConfig systemConfig, User user, String action) {
         if (user != null && user.isSuperuser()) {
             return NO_LIMIT;
         }
+
+        // If the setting changes then reset the cache
+        if (!settingRateLimitsJson.equals(systemConfig.getRateLimitsJson())) {
+            settingRateLimitsJson = systemConfig.getRateLimitsJson();
+            logger.fine("Setting RateLimitingCapacityByTierAndAction changed (" + settingRateLimitsJson + ").  Resetting cache");
+            rateLimits.clear();
+            return RESET_CACHE;
+        }
+
         // get the capacity, i.e. calls per hour, from config
         return (user instanceof AuthenticatedUser authUser) ?
                 getCapacityByTierAndAction(systemConfig, authUser.getRateLimitTier(), action) :

src/main/java/propertyFiles/Bundle.properties

@@ -3144,6 +3144,16 @@ authenticationServiceBean.errors.unauthorizedBearerToken=Unauthorized bearer tok
 authenticationServiceBean.errors.invalidBearerToken=Could not parse bearer token.
 authenticationServiceBean.errors.bearerTokenDetectedNoOIDCProviderConfigured=Bearer token detected, no OIDC provider configured.
 
+#SendFeedbackAPI.java
+sendfeedback.request.error.targetNotFound=Feedback target object not found.
+sendfeedback.request.rateLimited=Too many requests to send feedback.
+sendfeedback.body.error.exceedsLength=Body exceeds feedback length: {0} > {1}}.
+sendfeedback.body.error.isEmpty=Body can not be empty.
+sendfeedback.body.error.missingRequiredFields=Body missing required fields.
+sendfeedback.fromEmail.error.missing=Missing fromEmail
+sendfeedback.fromEmail.error.invalid=Invalid fromEmail: {0}
+
 #DataverseFeaturedItems.java
 dataverseFeaturedItems.errors.notFound=Could not find dataverse featured item with identifier {0}
 dataverseFeaturedItems.delete.successful=Successfully deleted dataverse featured item with identifier {0}
+