Skip to content

MansurAshraf/gradle-oozie-plugin

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Introduction

gradle-oozie-plugin provides a simple Groovy DSL for Apache oozie workflow because creating flows in XML causes serious brian damage!

Installation

maven repo: http://repository-uncommon-configuration.forge.cloudbees.com/release/
groupId: org.github.mansur.oozie
artifactId: gradle-oozie-plugin
version: 0.1

Supported workflow actions

Following workflow actions are supported

  1. java
  2. mapreduce
  3. pig
  4. ssh
  5. fs
  6. shell

In addition following decision nodes are also supported

  1. start
  2. end
  3. kill
  4. decision
  5. frok and join

Usage


buildscript {
    repositories {
        maven {
            url "http://repository-uncommon-configuration.forge.cloudbees.com/release/"
        }
        mavenCentral()
    }

    dependencies {
        classpath 'org.github.mansur.oozie:gradle-oozie-plugin:0.1'

    }

}
apply plugin: 'oozie'

This is how a mapreduce flow looks like in XML

<action name='first_map_reduce'>
    <map-reduce>
      <job-tracker>http://jobtracker</job-tracker>
      <name-node>http://namenode</name-node>
      <prepare>
        <delete path='http://jobtracker/pattern' />
      </prepare>
      <job-xml>job.xml</job-xml>
      <configuration>
        <property>
          <name>mapred.map.output.compress</name>
          <value>false</value>
        </property>
        <property>
          <name>mapred.job.queue.name</name>
          <value>queuename</value>
        </property>
      </configuration>
    </map-reduce>
    <ok to='end' />
    <error to='fail' />
  </action>

and this is the same flow in groovy dsl

oozie {
first_map_reduce = [
            name: "first_map_reduce",
            type: "mapreduce",
            delete: ["${jobTracker}/pattern"],
            jobXML: "job.xml",
            ok: "end",
            error: "fail",
            configuration: [
                    "mapred.map.output.compress": "false",
                    "mapred.job.queue.name": "queuename"
            ]
    ]
}

see build.gradle and generated workflow.xml for a complete workflow

About

Groovy DSL for Apache oozie workflow

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published