mercoledì 30 luglio 2008

Grails - Dynamic Plugins - Isolated Classloader

To prevent a possible jars conflict during dynamic plugin execution (what I was talking about in my previous post), I found a simple solution that I'm pasting for you here:

class DefaultPluginJob {

static triggers = { }

static cachedClassLoader = [:]

def execute(context) {

def libraryFolder = context.mergedJobDataMap.get("libraryFolder")

if (!cachedClassLoader[context.mergedJobDataMap.get("jobName")]) {
log.debug "Constructing class loader for ${context.mergedJobDataMap.get("jobName")}"
def urls = []
libraryFolder?.eachFile {library ->
log.debug "Adding file ${library}"
urls.add(library.toURL())
}
cachedClassLoader[context.mergedJobDataMap.get("jobName")] = new URLClassLoader(urls as URL[], this.class.classLoader)
}

def file = context.mergedJobDataMap.get("scriptFile")

Binding binding = new Binding(context.mergedJobDataMap);
GroovyShell shell = new GroovyShell(cachedClassLoader[context.mergedJobDataMap.get("jobName")], binding);
def scriptResult = shell.evaluate(file.text);

}
}

A new URLClassLoader will be created for each script that need to be run from my job, and it will contain a copy of parent class loader and new jars needed for the script execution.

What you can find here is the current solution I'm using in my application: all classloaders are cached in a static map (thanks again to Sergey for suggestion ;)) and passed to GroovyShell for script exection. Using cache for classloader granted that I've only one classloader generation for each script (and not one for each job execution). You loose memory to get perfermance! ;)

Waiting for comments about this way to run a java web application! :)

sabato 26 luglio 2008

Grails - Dynamic Plugins for your applications

The new functionality offers from Groovy/Grails make you able to write a very dynamical application and the possibility to add some new features to your web application at run-time!! Naturally we are exiting a "little bit" from JEE standards where our library/scritps/class and so on, must be in your war or provided from application server.
What I think about this, is that this way to see the development is a little bit old and could bring java to death! I'm dramatic, I know, but there is lot of confusion inside java, java standard and all java project.
It's time to change!! :)

Anyway...

What I'll explain is the way that we are using to add plugins (some functions) to OpenSymbolic, after application installation and, if you want, after webserver startup (I don't know what JBoss thinks about this... I'll do some test when all will be ready).

Our goal was to obtain a way to add some schedulable functions (Quartz Job) to our application leaving to user the decision about which he needs and which not (in a future article we will see the real usage in Symbolic).

First step
Question: How can I add a Job Dynamically to my scheduler?
Answer: at the moment I can't! :(

Solution: I called Sergey Nebolsin, the Quartz plugin developer, exposing my problem, and in ONE NIGHT (underline well ONE), he send me the implementation.
Lesson learned: If I need something for the following day... I will have to call Sergey!! :P (Really thanks again for your work and help Sergey!!)

Second step
Writing a bit of code to see if I can really do what I'm thinking.

Here what I have.

All my plugins will be contained in a specific folder on my machine (configured in configuration file of my application).
/etc/symbolic/plugins
-> /nagios_plugin
-> /msn_plugin
-> /dont_know_plugin

Each of these folders contains the file I need: in my test a configuration file and one script file.
So... with a simple script I could try to scan these folders look for what I need.
import org.codehaus.groovy.grails.commons.ConfigurationHolder

class PluginService {

boolean transactional = false

static CONFIG_FILE_EXT = 'conf'
static SCRIPT_FILE_EXT = 'groovy'
static LIB_FOLDER = 'lib'

public void init() {
//Scan Plugins Folder
def pluginsFolder = ConfigurationHolder.config.plugin.folder
log.debug "Scanning plugin folder: ${pluginsFolder}"
if (pluginsFolder) {
new File(pluginsFolder).eachDir {dir ->
log.debug "Directory found: ${dir}"
def dataMap = [:]

//Read Plugin File and configuring it
dir.eachFile {file ->
if (file.isFile()) {
if (file.name.contains(CONFIG_FILE_EXT)) {
log.debug "File ${file.name} is the configuration file"
def pluginConfiguration = readConfigFile(file)
dataMap['pluginName'] = pluginConfiguration.get("job.name")
dataMap['cronString'] = pluginConfiguration.get("job.cron")
} else if (file.name.contains(SCRIPT_FILE_EXT)) {
log.debug "File ${file.name} is the script file"
dataMap['scriptFile'] = file
} else {
log.debug "File ${file.name} will be ignored!"
}
} else {
if (file.name.equals(LIB_FOLDER)) {
log.debug "Lib folder found... adding jars to classpath."
}
}
}
DefaultPluginJob.schedule(dataMap['cronString'], dataMap)
}
}

else {
logger.info "No plugins folder set. Nothing to load!"
}

}


def readConfigFile = {file ->
Properties prop = new Properties()
if (file) {
prop.load (new FileInputStream(file))
}
prop
}
}

It's just a simple test... there are many improvements to do! ;)

Class "DefaultPluginJob" is a simple Quartz Job, that you can create in a standard grails way, and, with a new plugin release made by Sergey, it has some static methods that you can use to add your job to your quartz scheduler!

Here is the code of Job:
import org.quartz.JobDataMap
import org.quartz.JobExecutionContext

class DefaultPluginJob {

static triggers = { }

def execute(context) {

String instName = context.getJobDetail().getName();
String instGroup = context.getJobDetail().getGroup();
def file = context.mergedJobDataMap.get("scriptFile")

Binding binding = new Binding();
GroovyShell shell = new GroovyShell(binding);

def scriptResult = shell.evaluate(file.text);

}
}

It seems very simple, no? :)

Third step
To create a real extension of your application, you may need to add also some libraries used by your script: you cannot add all existing java libraries to your application because someone would create a plugin that will use that libraries! ;)

A very simple way we have found to solve this problem is adding a lib sub-folder to your plugin folder, where you can put all your libraries.
The plugin server, scanning folders, will add your jars to root class loader in this way:
this.class.classLoader.rootLoader.addURL(new URL("${file}"))

By now you can use all classes contained in your added jar files! :D

Future...
What I need to solve now, is a way to prevent jar conflict. Add all to root class loader, in fact, could make some problems to your application or simply to your other plugins.
I think that a solution could be write something inside your job, that will add your library only to your job instances during (before) execution of the script!
If anyone has any ideas about that... is welcome! :P

giovedì 24 luglio 2008

Symbolic - Operations Runner

Symbolic engine, or OperationRunner, is something like what we have seen for the scripts: a couple of job/thread that going in polling over database, look for ready operation, or completed (success/failed) operations.
All this operation will be completely asynchronous both for user and symbolic application: a ControllerJob will call each "n" seconds to verify the state of ran operation.



  1. Select machine(s), decide which operation he want, and complete (if required) the parameters needed for the execution
  2. An entry is stored in database and set in "ready state"
  3. RunnerJob (is not the same job used to run scripts!) check, each "x" seconds, if there's any operation to run.
  4. When it finds something, using func api, he call func and change the operation status in database: running state
  5. Func, through func-transmit script, will call required minion creating an "async job" (the job id will be returned to RunnerJob and stored in database)
  6. Another job, the ControllerJob, verifies if there is any running operation (contacting database) and, using stored job id, it asks to Func the state of that job
  7. When it receives a "complete job" response (could be success or failed) store the information in database setting also the status to correct value (success/error)
  8. User can see the result for the operation he called.

What is symbolic operations?

As official guide illustrates, symbolic administrator can create function users could do, simply adding new operations. An operation is an object built using func module, method and parameters.

What administrator has to do to add a new operation is:
  • Assign a name that will be shown to users
  • Select a func operation, selecting module and method over that module. i.e. module: command, method: run
  • Add some optional parameters to complete the operation. For example if he has coded the operation over module command with method run, he need to decide which linux command function he want to execute.
  • Add some optional parameters that will be ask to user before operation run. To make completely dynamic the operation coding, in fact, something need to be added during the execution by the user. For example, administrator could decide that for a specific group of users, must be created an operation to make able the execution of all linux command. So he has to code an operation over "command run" and the add a parameter named "Command to run" that users will complete before operation running.

Authority assignment is the last operation he has to do to shown new operation to the user. In fact, all symbolic operations, machines and scripts are shown only to users that are enabled to use.

At runtime this operation will be completed with machine hostname (the one where user click before operation selection), the (optional) parameters that user must complete and all is stored in a table with "ready state" attached. Will be symbolic engine to select ready operation and call func for the execution.

mercoledì 23 luglio 2008

Symbolic: Synch Operation Pool

Most of the processes, script or operation, you can run using Symbolic are asynchronous, this means that multiprocess/thread control is delegated to func. There are some other situation where you may need run process in synchronous mode: some admin procedures should be run directly waiting for the response (i.e is needed to going on or to complete symbolic configuration).
Following the standard "running channel" used to run async script, can generate a bottleneck: my RunnerJob must wait the end of ran process and so it's impossible to run any other operation till the end of the async one.

To solve this problem Symbolic has a Pool Manager used and called from RunnerJob. Is a bit more than simple synchronized list of processes: you can decide how many "concurrent" synchronous processes you want and it exposes some method to "book" your process execution, send process and check process status.

Following diagram exposes how Symbolic manages synch operations.



  1. User or administrator "try" to run a synch operation
  2. User calls reach the RunnerJob that, calling PoolManager try to book a place for sync job execution
  3. If there is a place to process user request, PoolManager responds with the position assigned in pool list. If you want this is what happens when you want to book an hotel room or a theater seat: you call, if there's what you ask you received booking number, in any other case you have to wait.
  4. If JobRunner receives a useful position in the pool list, he start the process asked from the user and then send reference to running process and provided position to PoolManager. If JobRunner receives a "no free places found" it returns an answer the user with an error message.
  5. Process is queued in the pool list and it goes ahead with the execution
  6. Any "X" seconds there is another job, ControllerJob, that calls PoolManager asking if there is any process that has finished the execution
  7. If something is found a messege ControllerJob sends a message to User with the execution result.
This kind of implementation is something like asynchronous call for the Symbolic engine because there is no "internal" process that wait for the execution end. It's only an synch call for the user that can do nothing more on the application till the end of the process he ran.

martedì 22 luglio 2008

Symbolic - Scripts Runner

In this article we will see how Symbolic runs a pre-configured user script in asynchronous mode.

Why we say pre-configured?
In Symbolic application, the administrator can "upload" on the server a specific "well-formed" script, written in Pyhton, Groovy, Bash or Perl. After this procedure Symbolic recognizes the list of installed scripts and proposes them to enable users. (As we have seen in a previous article)

This schema illustrates the main steps follow to execute script and propose result to the user.



  1. First of all, a logged user has to select a script from the list
  2. This selection produces an entry in Symbolic Database with script information and state READY (like in a MicroProcessor state ready means that operation is waiting the execution).
  3. A "runner-process" (Quartz Job for Java Developers), launched at the application start-up, queries database in polling looking for new ready-state scripts. By default it is configured to wake up each 5 seconds, but the administrator could decide to change this polling time with a simple modification to Symbolic configuration file
  4. When the "runner-process" find a new ready-state script, runs it using a new "System" process (a process external to Symbolic application) and set the state to RUNNING. Whereas the script is to run in async mode, the runner-process job is finished.
  5. The script can communicate with Symbolic application through a provided Xml-RPC Server (embedded in Symbolic and reachable at "symbolic/api/xmlrpc" address).
    The access to the server xml-rpc is protected with username and password. This provides an other step in Symbolic security, making only well-formed script able to communicate with Symbolic application.
    By default a user "externalscript" is created during the Symbolic installation (with default password externalscript); the administrator could change this credentials so only "really-certified" scripts could communicate with that Symbolic instance.
    At the moment Xml-Rpc server exposes only two methods: getAllMachines and postInformation. The first one is used to get list of Symbolic certified and controlled machines; the second one is used to communicate the script execution result to Symbolic.
  6. When postInformation is invoked, xml-rpc server get the result and saved the correct status in the database. Each ran script knows the database ID that must be posted with the result information. This is the only way that make symbolic able to recognized information posted.
  7. If user try to get ran script status in this moment, it find script result information (with associated error or success result).

Well-Formed Symbolic Script

The first step to make a well-formed script is to add, at the beginning of the script file in a "commented area", a series of tag that will recognize from Symbolic and adapt the application accordingly to the script.

To write your own script you can choose between one of the four supported languages: Groovy, Python, Bash, Perl

The current accepted tag are:
  • @Name: the name for your script. Is used to shown to the users your saved script.
  • @Author: name and references of the script's author.
  • @Type: type of your script. Valid values are: python, groovy, bash, perl
  • @Description: a full description to inform users about what script really does.
Putting just this four simple tags at the beginning of your script, and copying your script in symbolic scripts folder, make Symbolic application able to recognized the script so that users can run it.

XML-RPC Communication
Symbolic exposes a service to which script can connect to get some useful information (like a Symbolic certified machines) and to post execution result.
All "user-runnable" scripts in fact are launched in a way that can we call "asynchronous": Symbolic does not wait the answer from each ran script; so the only way that we can use to communicate the result to Symbolic if calling it through a defined service.
In Symbolic there is an implementation of XML-RPC server that exposes a method to post result from script:postInformation(result). So in your scripts you have to put some lines of code that call XML-RPC server to post the result information.
When Symbolic call a script provides these parameters:

-a: asynchronous execution. Caller will not wait for script answer, so result must be posted through xml-rpc server
-p proccesID: is the Symbolic identification of ran script
-s serverAddress: xml-rpc server address.

For example if you have a python script, symbolic will call it using something like:
   python script.py -a -p 10 -s http://localhost:8080/symbolic/api/xmlrpc

So you need to get this parameters inside your script if you want to communicate with Symbolic.

The answer that Symbolic expects must be formatted as dictionary/map with these information:
   ["process_id":SYMBOLIC_PROC_ID,"status":process_status,"response":some_information]

process_id: is the process id provided during symbolic script invocation.
status: is the result of your script/process (0:Success, 1:Error)
response: what you want. It's better something Human-Readable because will be shown to user without any kind of parsing.

Authentication
The symbolic rpm server is secured behind a password protection. So when you create, inside your script, an instance of xmlrpc client you need to post BasicAuthentication username and password or you will get an "access forbidden error" from Symbolic.
As default setting there is an user that scripts can use to connect to server:
username: externalscript
password: externalscript

Symbolic administrator could change this information or create many other script-enable accounts. These account must have associated a custom script authority (like default created during installation) or root authority (it's better to do not use this authority to make script enable to communicate with symbolic!)

Make a Func Binding - How you can interact with func-transmit

Func-Transmit
Func-transmit operates over standard-in/standard-out with data formatted with JSON or YAML (you can choose during invocation) and it's used to make func calls from another language easier than CLI calls. You can call func-transmit making information over stdin to invoke a func call on a specific minion and receiving response over stdout.

How you can invoke func-transmit
For your func-transmit invocation you need to decide whitch kind of supported parser/serializer you want to use. At the moment func-trasmit accepts only JSON or YAML messages and responds using the same dialect you have chosen during invocation. This decision was made taking care to serializers and their different language spread. YAML and JSON have both libraries packaged for many languages, so it should be easy to create a func binding using these parsers.
You can invoke func-transmit using a command like this:
func-transmit --json < data

--yaml instead of --json if you want to use YAML parser (could be also -j or -y for json or yaml)
Data must be send to func-transmit as standard input ('<' in CLI syntax) and could be a file or directly the "String" with all information Information expected during func-transmit invocation
You have to package your data to invoke func-transmit in a dictionary/map. Here the list of all parameters accepted.

  • clients: (Required) list of client you want to call. Could be list,single string or special character ("*").
  • aysnc: (Optional-Default False) boolean that say if you want to call func in async mode (True) or in synch mode (False)
  • nforks: (Optional-Default 1) Number of forks you want to use to execute the calls to your clients. For example, if you have to execute a command on 10 different minions, you can use nforks=2 and each one call and get results on 5 minions.
  • module: (Required**) name of the module you want to call
  • method: (Required) method inside that module to invoke
  • parameters: (Optional-Default None) some additional parameters

** there are some specific "local" methods that, at the moment, do not need the module parameter (you can set what you want). But it's only to get information such as list_minions or something other local call.

Response returned
When func-transmit execution ends the response is packaged following the same structure of call. So you will receive a json/yaml message containing a dictionary with machinename and result.
{"host1": "result for your call", "host2":"result", ...}

The same structure is kept also if you call func-transmit with a single minion.

Async Calls
Asynchronous calls are an exception to what we have seen so far. In fact func-transmit responds to an invocation with a simple "String" containing the func process id (parsed with the selected dialect... but not really needed).
If you want to get results from a previous func-transmit call, you have to package a message containing:
clients: "*"
method: "job_status"
parameters: ["your process id"]

What you get as result of this call is a tuple with process_status and response packaged as "normal" func-transmit response:
[1, {host1:response, host2:response}]

Possible process statuses are:
0: your job is still running
1: finished
2: "lost in space". We cannot find job associated to provided pid
3: partial (executing but hasn't finished yet)
4: remote error

Generate a simple tree-menu in Grails

It's seems one of the many posts you can find in internet that helps you to generate a tree-menu using javascript/java/... What I want to illustrate is a method that use recursion in back-end (java code that generate the Tree structure) and front-end (a gsp that shown the tree calling itself!!).

In fact, the start point, is that our page is not a "simple" page but is written as grails templates.
Template is a grails way to get your front-end code structured and provide an highly re-usable mechanism that you can call simply using a defined taglib.

class TreeMenu {

def addNode = {nodeElement, machine, tagList ->
def nodes = [:]
nodes[machine.hostName] = machine
def newList = tagList - nodeElement
newList?.each {currentTag ->
nodes[currentTag.name] = addNode(currentTag, machine, newList)
}

nodes
}
}

Is an extraction of my program codes... in the original version the data structure is not a simple Map but I've a complex object, so I can do, for example, a check if there is a node with current provided name and so on.
What this code try to do, is to add a Machine to each Tree-Tag I'm sending to function as a List.
"nodeName" is the current node where I want my machine
"machine" is the object name I want in my tree
"tagList" is the list of all tree-node where my machine will be put

For example, I could have this situation

machine: "TryMachine"
tagList: ["A", "B"]

TreeMenu.addNode("Root", machine, tagList)

The result of this method invokation will be:
Root 
|-> A
| |->B
| | |TryMachine
| |TryMachine
|-> B
| |TryMachine
| |-> A
| | |TryMachine
| TryMachine

Display the tree
The extraordinary feature offers by grails is, as I said, the usage of recursion on the front-end, that make you able to create a page without insertion of some java codes: all just with default grails taglibs.
Here an example gets from my code:
<g:each in="${nodes}" var="element">
<g:if test="${element.value instanceof Machine}">
${element.name}
</g:if>
<g:else>
<g:machineList template="/templates/machineTree" data="${element}"/>
</g:else>
</g:each>

And in your page, where you want to put your tree, you can just simply call the template:
<g:machineList template="/templates/machineTree" data="${treeData}"/>

Is a just a simple example (and, in fact, I'm not sure that with mods I've done to create this post, all work well :P), If you want you can make some improvements to this code, attaching, for example, javascript functions to get your tree-node opened and closed, or some other kinds of object type.u