Implement locking of indexed files. (#690)

* add: processUuid and processId

* add: create a new table `file_lock`

* refactor: suppress PMD warnings

* fix: sql statement creating a new table `file_lock`

* wip: wip

* fix: a PMD warning

* fix: a SpotBug warning

* refactor: remove printStackTrace()

* fix: check formats of indexed files to open

* test: remove unnecessary checks

* fix: change constants to avoid overlaps

* fix: add a file existing check

* test: fix the test of opening an invalid indexed file

* test: fix load.at not to call default error handlers

* chore: end a transaction

* refactor: CobolIndexedFile.java

* test: prevent infinite loops

* refactor: CobolIndexedFile.java

* wip: wip

* fix: improve open_ in CobolIndexedFile.java

* change: set transaction settings

* set the transaction mode to EXCLUSIVE
* set busy_timeout to 5000

* feat: implement file locks of indexed files

* fix: change the condition of creating file_lock

* change: set the transaction isolation to SERIALIZABLE

* test: add tests of file locking of indexed files

* change: the schema of the record key table

* fix: an INSERT statement

* fix: an insert statement for primary record tables

* feat: add a new option `-default_select_lock_mode`

* channge: insert information when calling read_()

* fix: the method read_

* Implement file locking of indexed files (#678)

* add: processUuid and processId

* add: create a new table `file_lock`

* refactor: suppress PMD warnings

* fix: sql statement creating a new table `file_lock`

* wip: wip

* fix: a PMD warning

* fix: a SpotBug warning

* refactor: remove printStackTrace()

* fix: check formats of indexed files to open

* test: remove unnecessary checks

* fix: change constants to avoid overlaps

* fix: add a file existing check

* test: fix the test of opening an invalid indexed file

* test: fix load.at not to call default error handlers

* chore: end a transaction

* refactor: CobolIndexedFile.java

* test: prevent infinite loops

* refactor: CobolIndexedFile.java

* wip: wip

* fix: improve open_ in CobolIndexedFile.java

* change: set transaction settings

* set the transaction mode to EXCLUSIVE
* set busy_timeout to 5000

* feat: implement file locks of indexed files

* fix: change the condition of creating file_lock

* change: set the transaction isolation to SERIALIZABLE

* test: add tests of file locking of indexed files

* Update libcobj/app/src/main/java/jp/osscons/opensourcecobol/libcobj/file/CobolIndexedFile.java

Co-authored-by: Copilot <[email protected]>

* Update libcobj/app/src/main/java/jp/osscons/opensourcecobol/libcobj/file/CobolIndexedFile.java

Co-authored-by: Copilot <[email protected]>

* refactor: remove unnecessary comments

* refactor: fix typo

* refactor: fix typo

---------

Co-authored-by: Copilot <[email protected]>

* refactor: CobolIndexedFile.java

* add: lock a record when READ NEXT is executed

* add: unlock records when closing indexed files

* fix: the process of unlocking a file lock

* fix: controling transactions when rewriting records

* fix: commit timing when writing records

* fix: commits and rollbacks in WRITE, REWRITE and DELETE

* change: -default_lock_mode to -lock-mode-automatic

* refactor: code for commiting data

* add: unlock the previous record after writing

* add: unlock previous records after rewriting

* add: impelment record lockings of DELETE

* fix: constant numbers

change the value of COB_LOCK_AUTOMATIC and COB_LOCK_MULTIPLE
in CobolFile.java

* chore: remove duplicated code

* fix: close ResultSet not to lock database implicitly

* fix: close ResultSet in CobolIndexedFile.java

* add: NewIndexedCursor.java

* wip: read forward

* wip: read prev

* fix: fetchRecord

* fix: READ NEXT/PREVIOUS

* fix: READ

* fix: fetchRecord

* fix: fetchRecord

* fix: START

* fix: START ~ KEY IS GREATER THAN

* fix: START ~ KEY IS <=

* fix: START and READ NEXT RECORD

* refactor: resolve PMD warnings

* chore: migrate NewIndexedCursor to IndexedCursor

* fix: allow READ WITH LOCK/WITH NO LOCK if LOCK MODE IS AUTOMATIC

* fix: call commit() in the end of READ and READ NEXT/PREV

* fix: initialize previousLockedReclordKey in close_()

* test: add some tests for record locking

* fix: unlock Previous records at the end of READ

* test: accelerate file locking tests

* test: wip

* test: add tests for `READ WITH LOCK`

* test: check the final state of an indexed file

* fix: do not read data when the file status is not 00

* test: move common java code for record lock tests

* test: add tests for different records

* test: add tests for input mode

* test: add tests for the same process

* fix: commit at the end of start_

* test: guarantee that OPEN,START,WRITE,REWRITE cannot lock indexed file records

* test: fix tests for OPEN,START,WRITE and REWRITE

* fix: processes of unlocking records

* tests: add tests for unlocking records

* tests: add tests of OPEN INPUT

* test: add tests for LOCK MODE

* test: add tests of -lock-mode-automatic

* tests: move tests for indexed files lock to tests/indexed-lock

* ci: run tests `indexed-lock`

* chore: fix typo

* fix: the syntax error of an sql statement in IndexedCursor.java

Co-authored-by: Copilot <[email protected]>

---------

Co-authored-by: Copilot <[email protected]>

Author: yutaro-sakamoto

.github/workflows/coverage.yml

@@ -41,7 +41,7 @@ jobs:
       - name: Run other tests
         working-directory: tests
         run: |
-          tests=("command-line-options" "data-rep" "i18n_sjis" "jp-compat" "run" "syntax" "cobj-idx" "misc")
+          tests=("command-line-options" "data-rep" "i18n_sjis" "jp-compat" "run" "syntax" "cobj-idx" "indexed-lock" "misc")
           for test in "${tests[@]}"; do
             ./"$test" || true
           done

.github/workflows/pull-request.yml

@@ -54,6 +54,7 @@ jobs:
           - "syntax"
           - "cobj-idx"
           - "misc"
+          - "indexed-lock"
         os: ["ubuntu:24.04", "almalinux:9", "amazonlinux:2023"]
     uses: ./.github/workflows/test-other.yml
     with:
@@ -74,6 +75,7 @@ jobs:
           - "run"
           - "syntax"
           - "cobj-idx"
+          - "indexed-lock"
           #- "misc"
         os: ["ubuntu:24.04", "almalinux:9", "amazonlinux:2023"]
     uses: ./.github/workflows/test-other.yml

.github/workflows/push.yml

@@ -58,6 +58,7 @@ jobs:
           - "run"
           - "syntax"
           - "cobj-idx"
+          - "indexed-lock"
           - "misc"
         os: ["ubuntu:24.04"]
     uses: ./.github/workflows/test-other.yml
@@ -79,6 +80,7 @@ jobs:
           - "run"
           - "syntax"
           - "cobj-idx"
+          - "indexed-lock"
           #- "misc"
         os: ["ubuntu:24.04"]
     uses: ./.github/workflows/test-other.yml

.gitignore

@@ -54,6 +54,7 @@ tests/jp-compat
 tests/cobj-idx
 tests/misc
 tests/run
+tests/indexed-lock
 tests/*.log
 tests/syntax
 tests/cobol85/*/*.class

cobj/cobj.c

@@ -145,10 +145,14 @@ int cb_flag_main = 0;
 
 int cb_default_byte_specified = 0;
 unsigned char cb_default_byte = 0;
+
+int cb_default_select_lock_mode = COB_LOCK_MANUAL;
+
 #define OPTION_ID_DEFAULT_BYTE (1024)
 #define OPTION_ID_SINGLE_JAR (1025)
 #define OPTION_ID_JAR (1026)
 #define OPTION_ID_INFO_JSON (1027)
+#define OPTION_ID_LOCK_MODE_AUTOMATIC (1028)
 
 int external_flg = 0;
 int errorcount = 0;
@@ -311,6 +315,7 @@ static const struct option long_options[] = {
     {"reference_check", no_argument, NULL, 'K'},
     {"constant", optional_argument, NULL, '3'},
     {"fdefaultbyte", required_argument, NULL, OPTION_ID_DEFAULT_BYTE},
+    {"lock-mode-automatic", no_argument, NULL, OPTION_ID_LOCK_MODE_AUTOMATIC},
 #undef CB_FLAG
 #define CB_FLAG(var, name, doc)                                                \
   {"f" name, no_argument, &var, 1}, {"fno-" name, no_argument, &var, 0},
@@ -901,6 +906,8 @@ static void cobc_print_usage(void) {
          "representing a character"));
   puts(_("                                    * octodecimal 00..0377 "
          "representing a character"));
+  puts(_("  -lock-mode-automatic              Set the default lock mode of "
+         "select clauses to AUTOMATIC"));
   puts(_("  -info-json-dir=<dir>              Specify the directory path of "
          "JSON files that hold information of COBOL programs"));
   puts(_("  -java-package(=<package name>)    Specify the package name of the "
@@ -1156,6 +1163,10 @@ static int process_command_line(const int argc, char *argv[]) {
       fflush(stderr);
       break;
 
+    case OPTION_ID_LOCK_MODE_AUTOMATIC:
+      cb_default_select_lock_mode = COB_LOCK_AUTOMATIC;
+      break;
+
     case '3': /* --constant */
       if (optarg) {
         cb_constant_list_add(optarg);

cobj/cobj.h

@@ -383,4 +383,5 @@ extern int cb_verify(const enum cb_support tag, const char *feature);
 extern int cb_flag_info_json;
 extern char *cb_info_json_dir;
 
+extern int cb_default_select_lock_mode;
 #endif /* CB_COBC_H */

cobj/parser.c

@@ -1,8 +1,8 @@
-/* A Bison parser, made by GNU Bison 3.8.2.  */
+/* A Bison parser, made by GNU Bison 3.7.4.  */
 
 /* Bison interface for Yacc-like parsers in C
 
-   Copyright (C) 1984, 1989-1990, 2000-2015, 2018-2021 Free Software Foundation,
+   Copyright (C) 1984, 1989-1990, 2000-2015, 2018-2020 Free Software Foundation,
    Inc.
 
    This program is free software: you can redistribute it and/or modify
@@ -16,7 +16,7 @@
    GNU General Public License for more details.
 
    You should have received a copy of the GNU General Public License
-   along with this program.  If not, see <https://www.gnu.org/licenses/>.  */
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.  */
 
 /* As a special exception, you may create a larger work that contains
    part or all of the Bison parser skeleton and distribute that work
@@ -515,8 +515,6 @@ typedef int YYSTYPE;
 
 extern YYSTYPE yylval;
 
-
 int yyparse (void);
 
-
 #endif /* !YY_YY_PARSER_H_INCLUDED  */

cobj/parser.h

@@ -5382,9 +5382,7 @@ read_statement:
 		     CB_FILE(cb_ref ($3))->organization != COB_ORG_INDEXED)) {
 			current_statement->handler_id = COB_EC_I_O_PERMANENT_ERROR;
 		}
-		if ($7 && (CB_FILE(cb_ref ($3))->lock_mode & COB_LOCK_AUTOMATIC)) {
-			cb_error (_("LOCK clause invalid with file LOCK AUTOMATIC"));
-		} else if ($8 &&
+		if ($8 &&
 		      (CB_FILE(cb_ref ($3))->organization != COB_ORG_RELATIVE &&
 		       CB_FILE(cb_ref ($3))->organization != COB_ORG_INDEXED)) {
 			cb_error (_("KEY clause invalid with this file type"));

cobj/parser.y

@@ -1725,6 +1725,7 @@ struct cb_file *build_file(cb_tree name) {
   p->access_mode = COB_ACCESS_SEQUENTIAL;
   p->handler = CB_LABEL(cb_standard_error_handler);
   p->handler_prog = current_program;
+  p->lock_mode = cb_default_select_lock_mode;
   if (external_flg == 1) {
     p->external_assign = 1;
   }