SOLR-14142 Enable jetty's request log by default (#536)

(cherry picked from commit 1d761d2)

Author: janhoy

Diff for: solr/CHANGES.txt

@@ -457,6 +457,9 @@ Other Changes
 * SOLR-14858: Add the server WEB-INF/lib directory to the classpath for the solr-exporter script. Will allow the script
   to work when the dist/solrj-lib jars are missing in the Docker image. (Houston Putman)
 
+* SOLR-14142: Jetty's RequestLog is enabled by default. If you don't want these logs, you can disable
+  via SOLR_REQUESTLOG_ENABLED=false.  (rmuir, janhoy)
+
 * SOLR-15924: Remove lucene-libs from contrib module packaging. The lucene libraries will be included in lib/ with all other dependencies. (Houston Putman)
 
 Bug Fixes

Diff for: solr/bin/solr

@@ -270,7 +270,7 @@ else
 fi
 
 # Requestlog options
-if [ "$SOLR_REQUESTLOG_ENABLED" == "true" ]; then
+if [ "${SOLR_REQUESTLOG_ENABLED:-true}" == "true" ]; then
   SOLR_JETTY_CONFIG+=("--module=requestlog")
 fi
 

Diff for: solr/bin/solr.cmd

@@ -150,6 +150,9 @@ IF "%SOLR_SSL_ENABLED%"=="true" (
 )
 
 REM Requestlog options
+IF NOT DEFINED SOLR_REQUESTLOG_ENABLED (
+  set SOLR_REQUESTLOG_ENABLED=true
+)
 IF "%SOLR_REQUESTLOG_ENABLED%"=="true" (
   set "SOLR_JETTY_CONFIG=!SOLR_JETTY_CONFIG! --module=requestlog"
 )

Diff for: solr/bin/solr.in.cmd

@@ -102,7 +102,7 @@ REM Location where Solr should write logs to. Absolute or relative to solr start
 REM set SOLR_LOGS_DIR=logs
 
 REM Enables jetty request log for all requests
-REM set SOLR_REQUESTLOG_ENABLED=false
+REM set SOLR_REQUESTLOG_ENABLED=true
 
 REM Sets the port Solr binds to, default is 8983
 REM set SOLR_PORT=8983

Diff for: solr/bin/solr.in.sh

@@ -126,7 +126,7 @@
 #SOLR_LOGS_DIR=logs
 
 # Enables jetty request log for all requests
-#SOLR_REQUESTLOG_ENABLED=false
+#SOLR_REQUESTLOG_ENABLED=true
 
 # Sets the port Solr binds to, default is 8983
 #SOLR_PORT=8983

Diff for: solr/server/etc/jetty-requestlog.xml

@@ -30,7 +30,7 @@
         <New class="org.eclipse.jetty.server.AsyncRequestLogWriter">
           <Arg><Property name="solr.log.dir" default="logs"/>/yyyy_mm_dd.request.log</Arg>
           <Set name="filenameDateFormat">yyyy_MM_dd</Set>
-          <Set name="retainDays">90</Set>
+          <Set name="retainDays"><Property name="solr.log.requestlog.retaindays" default="3"/></Set>
           <Set name="append">true</Set>
           <Set name="timeZone">UTC</Set>
         </New>

Diff for: solr/solr-ref-guide/src/configuring-logging.adoc

@@ -150,3 +150,10 @@ The log file under which you can find all these queries is called `solr_slow_req
 
 In addition to the logging options described above, it's possible to log only a selected list of request parameters (such as those sent with queries) with an additional request parameter called `logParamsList`.
 See the section on <<common-query-parameters.adoc#logparamslist-parameter,logParamsList Parameter>> for more information.
+
+== Request Logging
+
+Every incoming HTTP(s) request is by default logged in the standard https://en.wikipedia.org/wiki/Common_Log_Format[`NCSA format`]
+in files with name `$SOLR_LOG_DIR/<yyyy_mm_dd>.request.log`, rolled over daily. By default, 3 days worth of request logs are retained.
+You can disable request logging by setting `SOLR_REQUESTLOG_ENABLED=false` via environment variable or in `solr.in.sh`/`solr.in.cmd`.
+You can change the number of days to retain by system property `-Dsolr.log.requestlog.retaindays`.

Diff for: solr/solr-ref-guide/src/major-changes-in-solr-9.adoc

@@ -85,6 +85,8 @@ All the usages are replaced by BaseHttpSolrClient.RemoteSolrException and BaseHt
 
 * SOLR-11623: Every request handler in Solr now implements PermissionNameProvider. Any custom or 3rd party request handler must also do this
 
+* SOLR-14142: Jetty low level request-logging in NCSA format is now enabled by default, with a retention of 3 days worth of logs.
+  This may require some more disk space for logs than was the case in 8.x. See Reference Guide chapter "Configuring Logging" for how to change this.
 
 == New Features & Enhancements
 

Diff for: solr/solr-ref-guide/src/securing-solr.adoc

@@ -80,11 +80,6 @@ The authorization plugins that ship with Solr are:
 Audit logging will record an audit trail of incoming reqests to your cluster, such as users being denied access to admin APIs.
 Learn more about audit logging and how to implement an audit logger plugin in the section <<audit-logging.adoc#,Audit Logging>>.
 
-== Request Logging
-
-Solr can optionally log every incoming HTTP(s) request in the standard https://en.wikipedia.org/wiki/Common_Log_Format[`NCSA format`].
-You can enable request logging by setting `SOLR_REQUESTLOG_ENABLED=true` via environment variable or in `solr.in.sh`/`solr.in.cmd`.
-
 == IP Access Control
 
 Restrict network access to specific hosts, by setting `SOLR_IP_ALLOWLIST`/`SOLR_IP_DENYLIST` via environment variables or in `solr.in.sh`/`solr.in.cmd`.