Initial commit

Author: thedatabaseme

Changelog.txt

@@ -0,0 +1,112 @@
+Changelog of Pythia:
+
+Version 2.0.0
+  - Pythia is now OpenSource. All former Changes are no longer part of this repository. Changelog history is only for a better overview still included
+
+Version 1.5.2
+  - Fixed (reopened) Issue #97: The psmisc Package is now also a required Package for SLES OS
+  - Fixed Issue #142: acl Package is now a prerequisite Package for all Linux OS
+  - Fixed Issue #141: Maximum Datafiles are now set to 500 in the DBCA Templates
+  - Fixed Issue #124: You can now specify an additional directory in rdbms_dict, that will be created / permission set when installing an RDBMS
+  - Fixed Issue #116: For SLES Systems and the NMDA Module RPM Installation, the Zypper Module will be used without making a gpg check
+
+Version 1.5.1
+  - Fixed Issue #138: Fixed Archivelog Backup Template during duplicate
+
+Version 1.5.0
+  - Fixed Issue #125: Backup of Archivelogs will now be altered before running a duplicate. So Archivelogs will be preserved during duplicate and be restored after a finished Duplicate
+  - Fixed Issue #132: Listener Names will now be checked before Duplicate.
+  - Fixed Issue #131: RMAN Duplicate will now run in "no_log" Mode. So no passwords will be visible
+  - Fixed Issue #133: Duplicate now runs inside a block and copies back the backed up rman_arc.par file always
+  - Fixed Issue #134: RMAN Duplicate will now run in async and poll mode (background) to prevent SSH Timeouts
+  - New Feature #123: You can now execute a SQL Script as a Duplicate Post-processing task by specifiying the duplicate_run_post_script Variable
+  - New Feature #135: A List of User Passwords can now be preserved during duplicate and will be set after duplication
+
+Version 1.4.3
+  - Fixed Issue #129: Patch Archive and Gather Databases running are now running as oracle_user.
+
+Version 1.4.2
+  - New Feature: Duplicate will now handle more complex situations when the target database already exists
+  - New Feature: Duplicate now uses "using backupset" for RMAN duplicate which will enable "Null Block Compression"
+  - New Feature: Duplicate now supports preserving existing SPFILE / Database on Target System
+  - Fixed Issue #121: == true and == false usages have been replaced
+  - Fixed Issue #122: Names are now specified for all set_facts and debug tasks
+  - Fixed Issue #95: It is now checked if a Target Database already exists. If you don't specify the duplicate_remove_target=true Variable, Pythia will fail
+
+Version 1.4.1
+  - Fixed Issue #113: sqlscriptoutput TAG has been removed from README.md
+
+Version 1.4
+  - New Feature: Pythia can now Upgrade Oracle Databases using the Autoupgrade Mechanism
+  - New Feature: Added Support for RHEL8, OEL8 and CentOS8 (beware, CentOS is not officially certified for Oracle Database)
+  - Changed #98: The Tag sqlscriptoutput has been removed. Use the Variable sqlscriptoutput True / False instead
+  - Fixed Issue #111: Wrong Package Name for SLES12 fixed
+
+Version 1.3.4
+  - Fixed Issue #99: UID and GID of Oracle OS User and DBA Group are now 1500 instead of 500
+
+Version 1.3.3
+  - Fixed Issue #93: When duplicating, the Target Listener will be configured and started if not already configured or started
+
+Version 1.3.2
+  - Fixed Issue #87: .bash_profile is now created if not existant
+  - Fixed Issue #88: Grub Configuration is now divided into SLES and RHEL specific Task
+
+Version 1.3.1
+  - Fixed Issue #82: Diagnostic Destination is now created before Database Creation
+
+Version 1.3
+  - New Feature: Oracle Client Software can now be installed on Linux Systems with using the "client" tag
+  - New Feature: Duplicate from Active Database is now possible with using the "duplicate" tag
+  - Fixed Issue #78: sysctl_stable is not respected correctly in prerequisite.yml
+
+Version 1.2.3
+  - Fixed Issue #73: ora_set.sh now displays Oracle Version and Release correctly for Oracle 18 onwards
+
+Version 1.2.2
+  - Fixed Issue #69: Listener and Tnsnames Configuration will now be appended when there is already an existing listener on the Oracle Home
+  - New Feature: Listener Logging Configuration is now dynamic. You can now Specify Variable listener_logging to either ON of OFF (Default OFF)
+
+Version 1.2.1
+  - Fixed Issue #59: You can now specify autoextend=true/false (Default false) when calling a playbook. This controls if datafiles will be autoextensible or not
+  - Fixed Issue #57: Default sysctl.conf Settings will now be deleted
+  - New Feature: A converttohugepage Tag has been implemented to convert a non Hugepage configured System into a Hugepage configured System
+  - New Feature: A prepare Tag has been implemented to prepare a target System for an Oracle Installation later on. (User / Group, Kernel Parameters, Hugepages, Shell Scripts)
+
+Version 1.2
+  - New Feature: SGA and PGA can now be dynamically set for DB Creation. sga_max_size and pga_aggregate_target can be set as extra Variable when calling the Playbook
+  - New Feature: Hugepages can now be configured by Pythia using the hugepage tag.
+  - New Feature: DBCA Logfile location is now shown before running DBCA
+  - Fixed Issue #54: Done a complete Code Review. Added dozens of Comments, Restructure Prerequisite and Precheck Tasks
+
+Version 1.1.1
+  - Fixed Issue #44: DBCA does not honor the Controlfile Initialization Parameter unless you call dbca with -honorControlFileInitParam
+  
+Version 1.1
+  - New Feature: SQL Scripts can be executed on a remote Oracle Database. Files to be executed can be located under local_sql_stage_directory path
+  - New Feature: Pythia supports Oracle SE2 (Standard Edition 2) now. There are Responsefiles for SE2 deployed
+  - New Feature: Characterset / NLS Configuration is now dynamic. You can handover nls_length_semantics, character_set and national_character_set as extra Variables when calling the Playbook.
+  - New Feature: Support Installation of additional Software Packages (RPM) by specifying the "rpm" tag
+  - New Feature: Make EM Express installation / configuration dynamic when creating a Database. Variable empress has been implemented
+  - New Feature: Dynamic Management Pack Access. You can Specify the Variable on Playbook Call management_pack_access. This controls the Oracle Parameter control_management_pack_access.
+  - New Feature: You can setup Autostart for a Database you specify under the Linux System.
+
+Version 1.0.2
+  - Fixed Issue #5: The RDBMS Software and Patch Archives will now be copied to the Remote Staging Area before uncompressing. Thos solves, that user temp directories or /var/tmp will become full.
+  - Fixed Issue #12: Overloaded the Variable space_needed_gb from rdbms_dict.yml into main.yml. When calling the Playbook with the variable set, this value will be checked for free space available rather than the value in rdbms_dict.yml
+  - Fixed Issue #30: Remote Stage Directory will now be created also when only tagging Patch or Patchonly
+  - Fixed Issue #48: Wrong Package Names for SLES12 and SLES15 Systems
+
+Version 1.0.1
+  - Fixed Issue #3, #4: Tags in prereq_packages included. Include Filename Changed
+  - Fixed Issue #6: Check if Oracle OS User and Group exist
+  - Fixed Issue #9: Value of needed Free Space when installing a RDBMS is now configurable in the rdbms_dict.yml
+  - Fixed Issue #11: If the files 97* under sysctl.d and limits.d exist, they will now not be overwritten. Also Owner and Group are now setup for security.d template file
+  - Fixed Issue #10 #15: Split RDBMS Installation and Prechecks into separate Task Files to clean up the main.yml Files
+
+Version 1.0
+  - Initial Version
+  - Features: 
+    - Install Oracle RDBMS on a target system
+    - Create a Database on top of a new or an existing RDBMS
+    - Install Patches on top of a new and existing RDBMS and or Database

INSTALL.md

@@ -0,0 +1,106 @@
+# Installation Instructions
+For first time installation, please read this Instructions carefully. Else, Pythia will not work as expected!
+
+General
+===============
+
+You will need to provide several Files to ensure, that Pythia will work correctly. All Files you need to create are listed in this Instruction. These files can and should be changed by you, to fulfill your needs for installing and maintaining an Oracle Database. You may want to adapt all Variables or Templates to meet your or your companies requirements. Hence you want to change the Folder Structure to fulfill your needs.
+For every File that needs to exist, there is an *EXAMPLE* file in the respective directory. If you want to, you can copy the *EXAMPLE* file (and remove the EXAMPLE from the filename) and Pythia will work. This is not the recommended way however.
+
+Variables
+===============
+
+Variables are defined under the role/pythia/vars/ directory.
+
+client_dict.yml
+---------------
+The client_dict.yml contains Variables / Dictionaries regarding the Oracle Client Installation. There is an *client_dict_EXAMPLE.yml* file.
+There is a dictionary structure for every Client Version you may want to install (e.G. 19CLNT for all Variables, when you want to install a 19c Client).
+
+You may want to adapt the following Variables in the file to suffice your needs:
+
+  - client_dir: Path where the Client Installation Archives are located on the Stage Directory (Ansible Host), relative to the local_stage_directory Variable
+  - oracle_base: Oracle Base Directory for the specified Client
+  - oracle_home: Oracle Home Directory for the specified Client
+  - oracle_inventory: Path to the Oracle Inventory Location
+
+patch_dict.yml
+---------------
+The patch_dict.yml contains Variables / Dictionaries regarding the Oracle Database Patches (RUs). There is an *patch_dict_EXAMPLE.yml* file.
+There is a dictionary structure for every Patchset you may want to install (e.G. 30125133 for all Variables, when you want to install the 19.5 Release Update).
+
+You may want to adapt the following Variables in the file to suffice your needs:
+
+  - patch_dir: Path where the patch is located, relative to the local_stage_directory variable. Must reside under local_stage_directory.
+  - opatch_dir: Path where the needed OPATCH for patch installation is located. Relative to the local_stage_directory variable. Must reside under local_stage_directory.
+
+When you want to create a new entry for a new Release Update (RU) you want to rollout using Pythia, you need to provide a new dictionary entry with the correct Patch ID, also insert the corresponding Opatch Archive on your local_stage_directory and specify the Archive filename in the opatch_file Variable.
+The same applies for the Release Update Archive itself. You need to specify the Archive filename under the Variable patch_file.
+
+rdbms_dict.yml
+---------------
+The rdbms_dict.yml contains Variables / Dictionaries regarding the Oracle Database RDBMS. There is an *rdbms_dict_EXAMPLE.yml* file.
+There is a dictionary structure for every Database Release / Edition you may want to install (e.G. 19EE for all Variables, when you want to install the 19c Enterprise Edition).
+
+You may want to adapt the following Variables in the file to suffice your needs:
+
+  - rdbms_dir: Path where the RDBMS is located on the Stage Directory, relative to the local_stage_directory Variable
+  - oracle_base: Oracle Base Directory for the specified Client
+  - oracle_home: Oracle Home Directory for the specified Client
+  - oracle_inventory: Path to the Oracle Inventory Location
+  - additional_dir: You may specify an additional directory that should be created / permission set when install RDBMS
+
+When you want to create a new entry for a new Database Release (e.G. 21c) you want to rollout using Pythia, you need to provide a new dictionary entry with any ID that you want to reference during the Installation with Pythia.
+You want to specify RDBMS Archive itself. You need to specify the Archive filename under the Variable rdbms_file.
+
+rpm_dict.yml
+---------------
+The RPM Dictionary is a bit of a special case in Pythia. Pythia is able to install user specific RPMs (e.G. Backup Agents). There is an *rpm_dict_EXAMPLE.yml* file.
+Please be aware, that to Install a RPM, you also need to provide the Playbook under roles/pythia/tasks/ directory (like the rpm_DEMO.yml) which handles the RPM Installation and pre- / post-tasks.
+
+The rpm_dict.yml contains Variables / Dictionaries regarding user specific RPMs you want to install. 
+
+  - rpm_dir: Path where the patch is located, relative to the local_stage_directory variable. Must reside under local_stage_directory.
+  - rpm_file: Filename of the RPM / Archive.
+  - archive: Controls if the RPM is contained within a TAR or ZIP Archive
+  - unarchive_file: Specifies the filename when the RPM is unarchived
+
+sid_directories.yml
+---------------
+The Variables in the sid_directories.yml specify your Installation / Folder Structure for the Database itself. All Folders specified here will be created during the Installation Process and are relevant in the Database Creation response files (see below *Database.rsp.j2). For instance, you can specify a user defined location for your Online Redologs or Datafiles. When you want to mirror your Redologs to an additional folder / Volume, you also need to specify this in the DBCA Responsefiles (see below).
+
+  - sid_directory_list: A list of Directories that are created when installing a Database. All Directories that are configured in the Response Files for Database Creation (can be found under the template folder) are created before installing the database. If you want to change your Datafile, Redo Log Location, you may also have to change the list of Directories that are created here.
+
+All other Variables in here are referenced in the respective Responsefile for the DBCA. 
+
+  - diag_dest: Specifies the Diagnostic Dest. This Variable is referenced in the DBCA Templates and the Listener configuration Templates
+  - fast_recovery_area: Specifies the Fast Recovery Area location of your Database (db_recovery_file_dest). This Variable is referenced in the DBCA Templates (*_Database.rsp.j2)
+  - admin_dest: Specifies the Admin Destination Folder for your Database Installation. Here one of the controlfile will be places. This Variable is referenced in the DBCA Templates (*_Database.rsp.j2)
+  - data_dest: Specifies the Location of the Database Datafiles. This Variable is referenced in the DBCA Templates (*_Database.rsp.j2)
+  - temp_dest: Specifies the Location of the Database Tempfiles. This Variable is referenced in the DBCA Templates (*_Database.rsp.j2)
+  - redo_dest1, redo_dest2: Specifies the Location of the Online Redo Logs. This Variable is referenced in the DBCA Templates (*_Database.rsp.j2)
+  - archive_dest: Specifies the Location of the Archived Redo Logs. This Variable is referenced in the DBCA Templates (*_Database.rsp.j2)
+  - audit_dest: Specifies the Location of the Audit Files. This Variable is referenced in the DBCA Templates (*_Database.rsp.j2)
+
+Templates
+===============
+
+Templates can be found under the role/pythia/templates/ directory.
+
+*_Client.rsp.j2
+---------------
+There is a Template for every Oracle Client Version you want to deploy with Pythia. There needs to be an according entry in the client_dict.yml for the Version. For instance 19CLNT_Client.rsp.j2 for Version 19c which is defined as 19CLNT in the client_dict.yml. The Client.rsp.j2 Templates are used, to provide a Responsefile for the Oracle Installer when installing an Oracle Client with Pythia.
+
+*_listener.ora
+---------------
+There is a listener.ora Template for every Database Version and Edition you want to deploy with Pythia. There needs to be an according entry in the rdbms_dict.yml for the Version. For instance 19EE_listener.ora for Version 19c Enterprise Edition which is defined as 19EE in the rdbms_dict.yml. The listener.ora Templates are used, to create a valid Listener when you deploy a database with Pythia.
+
+*_SoftwareOnly.rsp.j2
+---------------
+There is a Template for every Database Version you want to deploy with Pythia. There needs to be an according entry in the rdbms_dict.yml for the Version. For instance 19EE_SoftwareOnly.rsp.j2 for Version 19c Enterprise Edition which is defined as 19EE in the rdbms_dict.yml. The SoftwareOnly.rsp.j2 Templates are used, to provide a Responsefile for the Oracle Installer when installing the Oracle Database Software (and Software Only, no Database) with Pythia.
+
+*_Database.rsp.j2
+---------------
+There is a Template for every Database Version you want to deploy with Pythia. There needs to be an according entry in the rdbms_dict.yml for the Version. For instance 19EE_Database.rsp.j2 for Version 19c Enterprise Edition which is defined as 19EE in the rdbms_dict.yml. The Database.rsp.j2 Templates are used, to provide a Responsefile for the Oracle Installer when installing the Oracle Database Software and Database with Pythia. There is an *Database_EXAMPLE.rsp.j2* file for every Database Version and Edition.
+
+As mentioned above (see sid_directories.yml), the sid_directories.yml and the Database.rsp.j2 Template are directly related. When you add a Directory in the sid_directories.yml, you may also need to change the Database Template as well. (e.G. adding another Archive Destination or another Mirrorlog). Be aware, that the Responsefiles are XML formatted, so any Syntax Error might lead to an Error during the Database Creation with Pythia. But don't blame the Hammer if you want to bang a Screw in the wall with it.