Conversation
|
@baacha123 is aws-s3.sh ready? |
Yes Jong its is ready and in PR |
|
Please for the aw-s3 shell script. i have leave instructions to update file path in script |
| @@ -1,103 +1,140 @@ | |||
| pipeline { | |||
| agent { | |||
There was a problem hiding this comment.
Can you undo all of the whitespace changes? It's hard to see what's changed. I can help you on this.
| stage('RUN DATA SCRIPT') { | ||
| steps { | ||
| echo "Pulling input-data-20220414, Creating Docker image & Pushing it to DockerHub" | ||
| sh 'PASSWORD= ${DockerHub_PASSWORD} /path/to/your/aws-s3.sh' |
There was a problem hiding this comment.
You don't need the PASSWORD= ${DockerHub_PASSWORD} part, you'll set an environment variable and import the credential from the Jenkins credential store. Check out the Handling credentials section in https://www.jenkins.io/doc/book/pipeline/jenkinsfile/. I can help you with this too.
Also, use "${WORKSPACE}/aws-s3.sh" for the path. Although, I'm not sure if the script should be checked in at the root of the repo, maybe in a scripts folder? @jkbk2004
| declare -a subfolders=("CICE_FIX/100" "CICE_IC/100" "CPL_FIX/aC96o100" "FV3_fix/fix_co2_proj" "FV3_fix_tiled/C96" "FV3_input_data/INPUT" "FV3_input_data/INPUT_L127" "FV3_input_data/INPUT_L127_mx100" "FV3_input_data/ORO_FLAKE" "FV3_input_data/RESTART" "FV3_input_data_gsd/FV3_input_data_C96_with_aerosols" "FV3_input_data_gsd/drag_suite" "FV3_input_data_INCCN_aeroclim/mg2_IN_CCN" "FV3_input_data_INCCN_aeroclim/MERRA2" "FV3_input_data_INCCN_aeroclim/aer_data" "FV3_input_data_INCCN_aeroclim/aer_data/LUTS" "FV3_input_frac/C96_L64.mx100_frac" "FV3_input_frac/C96_L127.mx100_frac" "MOM6_IC/100" "MOM6_IC/100/2011100100" "MOM6_FIX/100" "fv3_regional_control/INPUT" "fv3_regional_control/RESTART" "GOCART/p7/ExtData/dust" "GOCART/p7/ExtData/MERRA2/sfc" "GOCART/p7/ExtData/QFED/2013/04" "GOCART/p7/ExtData/QFED/2013/save03" "GOCART/p7/ExtData/CEDS/v2019/2013" "GOCART/p7/ExtData/monochromatic" "GOCART/p7/ExtData/optics" "GOCART/p7/ExtData/MEGAN_OFFLINE_BVOC/v2019-10/2013" "GOCART/p7/ExtData/PIESA/L127" "GOCART/p7/ExtData/PIESA/sfc/HTAP/v2.2" "GOCART/p7/ExtData/volcanic" "GOCART/p7/rc" "GOCART/p8/ExtData/dust" "GOCART/p8/ExtData/MERRA2/sfc" "GOCART/p8/ExtData/QFED/2021/03" "GOCART/p8/ExtData/QFED/2013/04" "GOCART/p8/ExtData/CEDS/v2019/2021" "GOCART/p8/ExtData/CEDS/v2019/2013" "GOCART/p8/ExtData/monochromatic" "GOCART/p8/ExtData/optics" "GOCART/p8/ExtData/MEGAN_OFFLINE_BVOC/v2019-10/2021" "GOCART/p8/ExtData/MEGAN_OFFLINE_BVOC/v2019-10/2013" "GOCART/p8/ExtData/PIESA/L127" "GOCART/p8/ExtData/PIESA/sfc/HTAP/v2.2" "GOCART/p8/ExtData/volcanic" "GOCART/p8/rc" "GOCART/ExtData/dust" "GOCART/ExtData/monochromatic" "GOCART/ExtData/optics" "GOCART/ExtData/PIESA/sfc/HTAP/v2.2" "GOCART/ExtData/PIESA/sfc/QFED/NRT/v2.5" "GOCART/ExtData/volcanic" "GOCART/rc") | ||
| PASSWORD=${PASSWORD:-'abc'} | ||
|
|
||
| #update line number 6, 13, and 15 with path where script will be running |
There was a problem hiding this comment.
You can use the variable "${WORKSPACE}" for where the repo will be cloned.
3 participants
This is the new jenkins file that will cp files from s3, create and image and push to dockerhub and use that image to run pipeline.
NOTE: The script path has to be updated once confirmed. Also i did not want to hard code dockerhub password in code since it is a bad practice, I have added a variable $PASSWORD which will reference the password from jenkins credentials.