MOSES::MOBY - Start here! Documentation for the Perl extension for the automatic generation of BioMOBY web services!

  # to get started, run the install script
  moses-install.pl

  # generate a service implementation, for example HelloBiomobyWorld from samples.jmoby.net
  moses-generate-services.pl samples.jmoby.net HelloBiomobyWorld

  # add your business logic to the module services/HelloBiomobyWorld.pm

  # assuming that you have deployed it, test it!
  moses-testing-service.pl -e http://localhost/cgi-bin/MobyServer.cgi HelloBiomobyWorld

  # read the POD for more details!

This is the documentation for Perl MoSeS (Moby Services Support). If you are reading this from the perldoc utility, you may notice that some words are missing or that some phrases are incomplete. In order to view this documentation in the manner intended, please view the html version of this documentation that was installed duing make install or the version on CPAN.

First of all, it is assumed that you are familiar with BioMOBY. If this assumption is false, please go to the BioMOBY homepage (http://biomoby.org).

In addition, this module is used to generate code for BioMOBY web services that are already registered with the registry of your choosing. If the service doesn't exist already, please register it first before proceeding.

Hopefully, you have chosen to install this package so that you can create BioMOBY web services. BioMOBY is a community driven, interoperability project. While the concept is easy to grasp, implementation isn't!

Installation of this helpful perl package is straightforward!

On *nix machines, install as follows:

perl Makefile.PL

make test

make install

On Window machines, install as follows:

perl Makefile.PL

nmake test

nmake install

Important if you are upgrading to MoSeS-Perl version 0.82 or higher, please remove your moby-services.cfg and remember to re-install|"MoSeS Installation" MoSeS!

Assuming that you have already installed this package, the very first thing that you should do is run the script moses-install.

This script will do the following:

* Check for prerequisite modules

* Run you through some configuration for the Perl MoSeS modules

* Optionally create a cache directory and fill it for you

* Create the logging and service configuration files

Once the installation process is complete, you can create your first service!

Perl MoSeS, is a project aiming to help BioMoby service providers develop their code in Perl.

The basic design principles are the same: to provide libraries that allow a full object-oriented approach to BioMoby entities (data types and service instances) and shielding service developers from all XML wrappers and envelopes. The BioMoby entities are taken from the local cache, mirroring BioMoby registries, and thus allowing fast access to all needed details.

Because Perl offers more ways to do things (a freedom that one pays for by her own responsibility to do rather good than bad things) Perl Moses differs slightly from its Java brother - mainly by not insisting to create all necessary objects in advance but building them on-the-fly.

Some of the new features included in this release are:

* Support for Asynchronous CGI based moby services

* Support for CGI based moby services

* Support for Asynchronous based moby services

* Service developers can now enable the validation of namespaces

Perl Moses is a generator of Perl code - but this code does not need to be always stored in files, it can be generated on-the-fly every time a service is called. Also, there are up to four things to be generated: objects representing BioMoby data types, classes establishing bases for services, empty (but working) service implementation modules, and cgi-bin scripts, the entry points to BioMoby services.

However, before going to gory details, let's install Perl Moses, create and call the first service.

1. Download the MoSeS module, and install it.

2. Run the installation script for MoSeS

This will create/update your local cache of a BioMoby registry. Which may take several minutes when run the first time.

From the command prompt, enter:

moses-install

or,

moses-install.pl

This works, because part of the installation for the MoSeS module entails the installation of scripts that make MoSeS tasks more simple.

3. Generate a service.

You can pick up almost any service form the registered ones - just type its authority and name:

moses-generate-services samples.jmoby.net HelloBiomobyWorld

or,

moses-generate-services.pl samples.jmoby.net HelloBiomobyWorld

It creates a Perl module Service::HelloBiomobyWorld (in <your-home-directory>/Perl-MoSeS/services/), representing your first BioMoby Web Service . The service is empty - but is already able to accept BioMoby input requests and recognize its own data type, and it produces fake output data - but again, the output data are of the correct type (as registered for this service).

You can even generate more (or all) services for your authority:

moses-generate-services.pl samples.jmoby.net

4. Make your service available from your Web Server (this is called deploying).

The only thing you need to do is to tell your Web Server where there is a starting cgi-bin script. The script was already created during the installation in <your-home-directory>/Perl-MoSes/ directory. Make a symbolic link from a cgi-bin directory of your Web Server (e.g on some Linux distributions, using Apache Web server, the cgi-bin directory is /usr/lib/cgi-bin).

For example:

on windows, you can mimic the above|"How can I tell apache to execute MobyServer.cgi on Windows without moving the file to cgi-bin?"

If you cannot create Symbolic links, or apache is not allowed to follow them, try this workaround|"Cannot Create Symbolic links"

If you plan on developing asynchronous moby services, then don't forget to also create a symbolic link for AsyncMobyServer.cgi as well!

5. Last but not least: call your service.

There is a testing client that sends only empty data (so it can be used by any service):

Of course, you can also send real data, from a local file:

The output (the same with any input data) is not really that exciting:

You immediately notice that the returned value does not even have an expected "hello" greeting but only a plain "this is a value". Well, a fake output is just a fake output. You would need to add your own business logic into your service to do something meaningful (such as saying warmly "Hello, Biomoby").

But even with an empty input and fake output, you can see that the output knows about the HelloBiomobyWorld service (see the output type moby:String and the article name greeting, both coming from the registry knowledge).

Also, the service provider can see few log entries:

The number in square brackets is a process ID - it helps to find which response belongs to which request when a site gets more to many requests in the same time. And in parenthesis, there is the number of milliseconds since the program has started. More about the log format in logging|"Logging".

Many people will be tempted to ask "how does Perl Moses differ from CommonSubs.pm, a library already available (and heavily used) in BioMoby"?

An honest answer is "I do not know". Because I never wrote any BioMoby service based on CommonSubs.pm. As far as I know the CommonSubs.pm are good, reliable and used. I think (guessing), however, that Perl Moses allows few things that CommonSubs.pm have not aimed for:m

* Fully object-oriented approach,

 based on objects created on-the-fly (or pre-generated) from the local cache representing a Biomoby registry.

* More centralized

(and therefore easier changeable if/when needed) support for service configuration, for request logging, and even for protocol binding (for example, in your service implementation code there are no visible links to SOAP).

In other words, the services written with the Perl Moses as their back-end are more unified and less prone to be changed when their environment changes. They should differ really only in their business logic (what they do, and not how they communicate with the rest of the world).

Having said that it is fair also to add that CommonSubs.pm have been used for a while so people already have found and fixed their bugs, a process Perl Moses has yet to go through.

And, of course, the greatest motivation comes from the spirit of the Perl slogan of TMTOWTDI,There's more than one way to do it.

The other modules needed are (all available from the CPAN):

* File::Spec

* XML::LibXML - this is always needed because it creates XML BioMoby response. But other (perhaps more efficient) parsers can be used for parsing the input requests. More about it in the configuration|"Configuration" section.

* Log::Log4perl - a wonderful port of the famous log4j Java logging system.

* Template - a modern Perl Template Toolkit, a really fascinating tool.

* Config::Simple - for a simple service configuration

* IO::Stringy - a.k.a. IO::Scalar

* Unicode::String

* File::HomeDir

* File::ShareDir

* SOAP::Lite, of course

* FindBin

* IO::Prompt - try to install version 0.99.2 and not 0.99.4

* WSRF::Lite - for developing asynchronous moby services Optional

Perl Moses takes all information from a local cache that stores details about registered BioMoby data types and services. The cache is represented by several flat-files (they can contain details about several BioMoby registries). You can consider the cache as configuration files used by your Perl programs.

In order to start anything related to Perl Moses (except perhaps the script moses-config-status), you need to create, or update the cache. Take it as a requirement.

The script, moses-install intially creates the cache for you, while the scripts moses-generate-datatypes and moses-generate-services update the datatypes and services cache respectively. More on the Scripts|"Scripts"" below.

You need to tell Perl Moses (i.e. to tell your service implementation) where the cache is located. It is done through the Configuration|"Configuration".

Okay. Having said all that, have you heard about The Law of Leaky Abstractions? Because - in case our abstraction does not leak - you can move the above to a lower rack in your memory, and let the Perl Moses installation script deal with the local cache. But - as the article says - It's good to know.

The installation script is (as well as the other Perl Moses scripts) installed at module install time and is available from any command prompt. You should run it the first time, and you can run it anytime later again. The files that are already created are not overwritten - unless you want it to be, and the local cache will be updated only after your confirmation.

moses-install

This is an example of a typical conversation and output of the first installation:

All these things can be done manually, at any time. Installation script just makes it easier for the first comers. Here is what the installation does:

    * It checks if all needed third-party Perl modules are available. 
	Since you got this far, they most likely are! It does not help with 
	installing them, however. Perl has a CPAN mechanism in place to do 
	that. The required modules are listed in requirements. Installation
	stops if some module is not available.

    * It creates a directory called 'Perl-MoSeS' in your user directory.
	Perl MoSeS will stop working if you move this directory because it 
	contains vital configuration information inside it.

    * It creates two empty log files Perl-MoSeS/services.log and
	Perl-MoSeS/parser.log - unless they already exist. In any case,
	it changes their permissions to allow everybody to write to them.
	This helps later, when the same log files are written to by a Web
	Server. The purpose of these two files is described in logging.

    * It creates a Perl-MoSeS/log4perl.properties file from a distributed
	template, and updates their locations to reflect your local
	installation. Again, more about this file in logging.

    * It creates a Perl-MoSeS/MobyServer.cgi file from a distributed
	template, and updates their locations to reflect your local
	installation. This is a cgi-bin script that will be started by
	your web server when a BioMoby request comes. More about it in
	deploying.

	* It creates a Perl-MoSeS/AsyncMobyServer.cgi file from a distributed
	template, and updates their locations to reflect your local
	installation. This is a cgi-bin script that will be started by
	your web server when an asynchronous BioMoby request comes. More 
	about it in	deploying.

    * For Perl Moses, the most important configuration option is the location
	of your local cache (a place where a BioMoby registered entities are mirrored).
	The installation script asks for it, unless there already exists a configuration 
	file Perl-MoSeS/moby-services.cfg - in which case the installation scripts 
	tries to take this option from there.

    * Knowing where to cache BioMoby entities, the script asks whether you want
	to do so, and if yes which BioMoby registry to contact to. Then it starts
	updating the cache. Be patient. But even if you interrupt it, next time 
	it will start where it was interrupted, and not at the beginning. Depending
	how your logging in jMoby is configured, you may see the progress on the 
	screen, or in a log file. But usually you do not need it - just wait. If
	there is an error, it will be reported, do not worry.

    * Finally, it creates a configuration file Perl-MoSeS/moby-services.cfg 
	(unless it already exists). See more about how to further configure Perl 
	Moses in configuration.

If you wish to install from the scratch (the same way it was done the first time), start it by using a force option:

moses-install -F

In this mode, it overwrites the files moby-services.cfg, services.log, parser.log and log4perl.properties.

There is a little extra functionality going on behind the scenes: If the configuration file moby-services.cfg exists when you start the installation script, its values are used instead of default ones. It may be useful in cases when you plan to put all Perl Moses directories somewhere else (typically and for example, if your Web Server does not support symbolic links that can point to the current directories). In such cases, edit your moby-services.cfg, put the new locations inside it, and run moses-install again.

Perl Moses generates Perl code. Actually, up to four pieces of the code:

Each BioMoby data type, as registerd in a BioMoby registry, is represented by a Perl object MOSES::MOBY::Data::<moby-data-type-name>. For example, a GenericSequence object looks like this:

The BioMoby data type objects are generated either on-the-fly (more about it in a moment), or by using the moses-generate-datatypes script|"Scripts".

Each Perl Moses service implementation can benefit by inheriting some basic functionality from its base. These bases contain the code specific for the given service (e.g. they know who is the service authority and can, therefore, add it automatically into the response).

The service base takes care about:

    * Logging request/response.
    * Allowing to run a service locally, outside of the SOAP environment (good for early testing).
    * Catching and reporting exceptions if the input is wrong or incomplete.
    * Returning request at once if it is completely empty (this is the recently discussed ping feature).
    * Dealing with more queries (jobs) in a request.
    * Dealing with SOAP::Lite. 

You can see its code by running (for example):

moses-generate-services -sb samples.jmoby.net Mabuhay

Again, the services bases can be generated and loaded on-the-fly, or pre-generated in the files.

This is your playground! What is generated is only an empty service implementation - and you are supposed to add the meat - whatever your service is expected to do.

Well, it is not that empty, after all.

First, because it inherits from its base, it already knows how to do all the features listed in the paragraph above:

Second, it has the code that reads the input, using methods specific for this service. It does not do anything with the input, but the code shows you what methods you can use and how:

And finally, it produces a fake output (not related to the input at all). Which is good because you can call the service immediately, without writing a single line of code, and because you see what methods can be used to create the real output:

The service implementations are definitely not generated on-the-fly. They must be pre-generated into a file (because you have to edit them, donn't you?).

Again, the moses-generate-services script will do it. More in scripts|"Scripts".

A small but important piece of code is a dispatch table that contains all service names you wish to be used from your site, using the same launching cgi-bin script. By default, it is named SERVICES_TABLE, and it is updated every time you add (deploy) a new service. For example, this is a dispatch table for all services from the authority samples.jmoby.net:

The dispatch table is also not generated on-the-fly. It is updated every time a service implementation is generated. Again, the moses-generate-services scripts will do it. More in scripts|"Scripts".

So, how do all of these pieces fit together? Here we go:

Scripts

The scripts are small programs that generate pieces and that let you test things.

They share some basic features:

    * They are automatically installed with the perl module.

    * They can be started from anywhere.

    * They all are Perl programs, expecting Perl executable in /usr/bin/perl. If your perl is elsewhere, start them as:

        perl -w <script-name>

    * They all recognize an option -h, giving a short help. They also have options -v (verbose) and -d (debug) for setting the level of logging.

    * Usually, they also need additional information (such as where to find local cache) from the configuration file moby-services.cfg.

Here they are in the alphabetic order:

This script does not do much but gives you overview of your configuration and installation. You can run it to find how Perl Moses will behave when used. For example:

This script generates Perl objects representing BioMoby data types. Use this script if you wish to pre-generate some or all BioMoby data types. You do not need to - but you can. If your service is called and its data types are not pre-generated, the service knows how to generate and load them on-the-fly.

This will generate all data types (it does not take that long: 300 data types just about 2.4 seconds on my laptop):

moses-generate-datatypes

You may see the progress on your screen if your logging is in debug level, and directed to the screen (more about it in logging|"Logging").

You may generate also only named data types, of course. In which case, the script still asks the generator to generate also related data types (those representing the members of generated data types). It seems a reasonable assumption. For example, for:

moses-generate-datatypes -d DNASequenceWithGFFFeatures

the script reports to a log file (note the -d option to log in debug mode):

An obvious question is "where are the data types generated to"?

You can always determine this after generation by looking in the log file - the message has the INFO level which means it is almost always logged. But, if you want to know in advance here are the rules:

    1. If there is a generators.outdir parameter in the configuration file, it is used. It defines the directory where data types are created.

    2. Otherwise, program is trying to find an existing directory named 'generated' anywhere in the @INC (a set of directories used by Perl to locate its modules).

    3. If it fails, it creates a new directory 'generated' in the "current" directory.

You can use option -s to get the generated result directly on the screen (in that case no file is created).

You can use the option -u to update the datatype cache.

You can use the option -f to fill the datatype cache.

The -R option allows you to specify a registry endpoint. For instance, -R http://moby.ucalgary.ca/moby/MOBY-Central.pl would cause the script to use UCalgary registry.

The BioMoby primitive types (String, Integer, Float, Boolean and DateTime) are never generated. They were created manually.

For test-oriented geeks, here is how to check that the generated data types are syntactically correct (using Unix and bash commands):

This is the most important script. You may use only the moses-install and this one - and you will get all what you need. It generates services - all pieces belonging to services (except data types - for that, there is moses-generated-datatypes|"moses-generate-datatypes" script).

Usually, you generate code for one or only several services. And because each service belongs to an authority you need to tell both:

moses-generate-services samples.jmoby.net Mabuhay

If you specify only an authority the code for all services from this authority will be generated:

moses-generate-services samples.jmoby.net

Without any options (as shown above), it will generate service implementation classes, and it will update the dispatch table. However, it does not overwrite already existing service implementation - that would be dangerous because you may have already edited and added the real business logic:

[ There is an option to repress this cautious behaviour - look into the script itself.]

There are several configurables options|"Configuration" to influence the result:

generators.impl.outdir dictates where the code is to be generated. If this option does not exists, similar rules as described in moses-generate-datatypes are used (except the default name is services and not generated).

generators.impl.package.prefix tells what package name should be used (the package name always ends with the service name as it is registered in the BioMoby registry). Default is Service.

generators.impl.services.table is a name of a dispatch table. Default is SERVICES_TABLE.

With options, you can generated other Perl Moses pieces:

As with generated data types, here also you can use option -s to get the generated result directly on the screen (in that case no file is created).

For testing (and for fun) you can generate all services from all authorities (this time it is not that fast as data types, it takes almost 17 seconds - but who cares?):

moses-generate-services -Sa

In order to test syntax of all services, don't try the same trick as with data types, but look for the moses-universal-testing script.

This script is used for installation|"Installation".

The Perl Moses has a hard-coded list of known BioMoby registries. New entries can be added - check the comments in MOSES::MOBY::Cache::Registries. Each registry has an abbreviation (a synonym) that can be used in Perl Moses configuration - it is easier and less error-prone than using the long registry's endpoint. This script can tell you which registry has which synonym (and few other things about it).

moses-known-registries

At the time of writing this documentation, the response was (note that one of the registries is labeled as the "default" one):

This script does not have any options (nor the help).

The Perl Moses has a hard-coded list of known BioMoby registries. New entries can be added by using this script. Each registry has an abbreviation (a synonym) that can be used in Perl Moses configuration - it is easier and less error-prone than using the long registry's endpoint. This script can allow you to add or remove these registries and in combination with the moses-known-registries script, provide a wealth of information regarding the registries that you use.

moses-user-registries

Before we begin, it should be noted that you cannot remove the hard-coded list of known registries using this tool. Only those registries that you have personally added can be removed.

To begin a session of modification of your persistant user registry store, run the script!

When you start the script, you are given 3 choices.

Add a new persistent user registry

Remove a persistent user registry

Quit

If you choose to add a registry, you will be prompted to enter the various pieces of information need for storing your registry.

Alternatively, you may choose to remove a registry from the list. When you select this option, you will be given the list of registries available and you can attempt to remove them. Only those registries that you have added personally can be removed.

Obviously, choosing to Quit will end the session.

Local cache is explained in the local cache|"Local Cache of a BioMOBY Registry" section. This script can show what is in the cache. It may be useful for implementing a service because the generated service implementation uses article names as method names - and this script shows all article names.

Show a data type:

moses-local-cache -t DNASequence

As you see this did not show all children (members). If you want it, use the -c option instead:

moses-local-cache -c DNASequence

Option -r shows all related (used) data types:

moses-local-cache -r DNASequence

Options -s shows service definitions.

Both data types and services can be shown in XML, using the -x option. The XML is actually identical with the registration request for the given entity. This is just a side-effect - but perhaps it can be useful:

moses-local-cache -xs samples.jmoby.net getRandomImage

Option -l is for getting list of all names: of data types (when used together with the -t option), or of services (with -s option).

You can also see how many entities are currently cached in your local cache. It shows what is the current registry you are using (it takes it from the configuration option registry), and the numbers for all cached registries. Use option -i (as for information):

moses-local-cache -i

A debugging tool. It reads a BioMoby XML data, parses them into Perl Moses data type objects, and prints them or convert them back to XML. It always takes an XML file name as a parameter:

moses-testing-parser parser-test-input2.xml

moses-testing-parser -r parser-test-input2.xml

And interestingly, the -b parameter. It has the form:

-b < input-name>:<known-type>

and it indicates a backup data type that is used when an unknown XML top-level tag is encountered. This is not usually needed at all - only when your data type definitions, the generated data types, are not up-to-date. If such situation occurs the input data with article name <input-name> will use the <known-type>.

A script for the first testing of your service. It does not give you the comfort that you can get from other BioMoby clients (Taverna, Simple Panel in the Dashboard, etc.) - but it is well suited for immediate testing.

It calls a BioMoby service in one of the two modes (actually the two modes are completely separated to the point that this script could be two scripts):

* Calling the service before it is deployed (known) to a Web Server. This mode is useful for debugging. It sends a BioMoby XML input to a service, but without using any SOAP messages. Of course, the service can be called only locally in this mode.

* Calling the service for real, using the Web Server, its cgi-bin script and the SOAP envelope.

In both modes, the script can send an input XML file to the service - but if the file is not given, an empty input is created and sent to the service. Which is not particularly useful, but still it can help with some preliminary testing.

When calling the service locally, you may use the following options/parameters:

    * A mandatory package name - a full package name of the called service.

    * Option -l location can be used to specify a directory where is the called service stored. Default is src/Perl/services.

    * Options -v and -d make also sense in this mode (but not in the other one).

    * An optional input file name.

moses-testing-service -d Service::HelloBiomobyWorld

The output of this call was already shown in this documentation|"Quick Start - Five Steps to the First Service". Therefore, just look what debug messages were logged (notice the -d option used):

The full mode has the following options/parameters:

    * A mandatory service name (not a package name) of the called service.

    * A mandatory endpoint -e endpoint defining where is the service located. Actually, presence of this parameter decides which mode is used.

    * An optional input file name.

There are also few other behavioral differences between these two modes: If an input parsing error occurs (e.g. when an input has an unknown article name), it is reported directly to the standard error in the testing mode, but in a real mode it is properly included in the response as an exception. Or (and only if the logging is set to record debug messages), in testing mode the full input raw (not yet parsed) are logged, whereas in the real mode only the first 1000 characters are logged.

A testing tool - but not testing your services but the Perl Moses itself. It may have more functions later, but for now, it simply generates code for all services (except one or two black-listed), and then call all of them (with an empty input).

It can be called also for a subset of services, usually for services from an authority:

moses-universal-testing bioinfo.icapture.ubc.ca

For some services, it produces warnings - but they are just the consequence of the way how the example methods (in service implementation) were generated. They do not mean anything wrong. There may be, however, some other warnings, that are consequence of the fact that a service is registered with empty article names (which should not be, but there are still such services in some registries).

It has one interesting feature that can be useful outside of pure testing: It keeps all outputs from all services in a (temporary) directory. These outputs may have fake values but they are not empty, and they represent correct output data types. For example, a service provider tropgenedb.cirad.fr created this (quite complex) output:

Configuration means to avoid hard-coding local-specific things (such as file paths) into the code itself but hard-coding them in a separate file, a file that is not shared with other (CVS) users.

Perl Moses stores configuration in a file named moby-services.cfg. The file name is hard-coded (and cannot be changed without changing the MOSES::MOBY::Config module), but its location can be set using an environment variable BIOMOBY_CFG_DIR. Perl MoSeS looks for its configuration place in the following places, in this order:

   1. In the "current" directory (which is not that well defined when used from a Web Server).
   2. In the directory given by BIOMOBY_CFG_DIR environment variable.
   3. In the directory <your-user-dir>/Perl-MoSeS/.
   4. In one of the @INC directories (directories where Perl looks for its modules). 

Therefore, the best place is to keep the configuration file together where the installation script puts it anyway.

The Perl Moses internally uses Config::Simple CPAN module, but wraps it into its own MOSES::MOBY::Config. This allows expansion later, or even changing the underlying configuration system. The Config::Simple is simple (thus the name, and thus we selected it) but has few drawbacks that may be worth to work on later.

The file format is as defined by the Config::Simple. It can be actually of several formats. The most common is the one distributed in the moby-services.cfg.template. This is an example of a configuration file:

The names of the configuration parameters are created by concatenating the "section" name (the one in the square brackets) and the name itself. For example, the XML parser is specified by the parameter xml.parser. Parameters that are outside of any section (e.g. cachedir) has just their name, or they can be referred to as from the default section. For example, these two names are equivalent: default.cachedir and cachedir.

Blank lines are ignored, comments lines start with a hash (#), and boolean properties must have a value ('true' or 'false').

Obviously, important is to know what can be configured, and how. This document on various places already mentioned several configuration options. Here is their list (for more explanations about their purpose you may visit an appropriate section of this document):

cachedir - Directory with the local cache. No default.

registry - A synonym of a registry that will be used (when reading from a local cache - it can have multiple registries), or its endpoint. Default is 'default'.

generators.outdir - Directory where to generate data types and service bases. The default value for data types is 'generated', for service bases is 'services'.

generators.impl.outdir - Directory where to generate service implementations. Default is 'services'.

generators.impl.package.prefix - A beginning of the package name of the generated service implementations. Default is 'Service'. For example, a service Mabuhay will be represented by a Perl module Service::Mabuhay.

generators.impl.services.table - A name (without any path) of a file with a services dispatch table. Default is 'SERVICES_TABLE'.

generators.impl.async.services.table - A name (without any path) of a file with an async services dispatch table. Default is 'ASYNC_SERVICES_TABLE'.

generators.ignore.existing.types - A boolean property. If set to true ('true', 1, ' '+', 'yes', or 'ano') the data types generator will not check if a wanted data type module already exists on a disk but always generates into memory a new one. Default value is 'false' (meaning that the generator tries to use whatever already exists).

log.config - A full file name with the Log4perl properties. No default. If this parameter is given but the file is not existing or not readable, Perl Moses complains on STDERR (which may end up in the Web Server error.log file).

log.file - A full file name of a log file (where the log messages will be written to). No default. If the value is 'stderr' (case-insensitive) the messages will go to the STDERR. It is not clear what happens when it is used together with the above log.config.

log.level - A log level. Default is ERROR.

log.pattern - A format of the log messages. Default is '%d (%r) %p> [%x] %F{1}:%L - %m%n'.

xml.parser - A preferred XML SAX parser for reading input data. Perl finds available XML parser itself, but sometimes you prefer a different one. Of course, you need to install it first. Note that this parser is used only for reading BioMoby requests (which may be long so an ability to choose a good parser is meaningful). For other XML stuff (creating response and reading from tghe local cache) Perl Moses always uses XML::LibXML module.

The parameters just described are used by PerlMoses modules - but the configuration system is here also for your own services. You can invent any not-yet-taken name, and add your own parameter. In order not to clash with the future Perl Moses parameters, it is recommended to prefix your configuration properties with the service name. For example, the Mabuhay service needs to read a file with "hellos" in many languages, so it defines:

All configuration parameters are imported to a Perl namespace MOBYCFG. The imported names are changed to all-uppercase and dots are replaces by underscores. You can see this change if you run the config-status.cfg:

In your program, you can use the imported names. For example, here is how the Mabuhay service opens its resource file:

You can also change or add parameters during the run-time. For example, the script universal-testing.pl needs to overwrite existing parameters because it wants to create everything in a separate space, in a temporary directory, and within the 'Testing' package. Because the generators read from the configuration files, it is necessary to change it there:

More about how to communicate pragmatically with the configuration can be (hopefully) find in the Perl Modules Documentation|"Perl Modules Documentation".

The logging system is based on a splendid Perl module Log::Log4perl, a Perl port of the widely popular log4j logging package. The Log4perl is well documented (here is its POD documentation http://search.cpan.org/~mschilli/Log-Log4perl-1.06/lib/Log/Log4perl.pm).

How does it work in Perl Moses?

The logging is available from the moment when Perl Moses knows about the MOSES::MOBY::Base module. All generated service implementations inherit from this class, so all of them have immediate access to the logging system. By default, the MOSES::MOBY::Base creates a logger in a variable $LOG. Which means that in your service implementation you can log events in five different log levels:

The logger name is "services". (The name is used in the logging configuration file - see below).

You can create your own logger. Which may be good if you wish to have, for example, a different logging level for a particular service, or for a part of it (an example of such situation is in MOSES::MOBY::Parser.pm where the parser creates its own $PLOG logger). Here is what you need to do:

Then use the name "my_log_name" in the configuration to set its own properties. Which brings also us to the logging configuration.

The logging configuration can be done in three ways:

* Do nothing.

* Edit log4perl.properties file.

* Edit logging configuration options in moby-services.cfg.

If Perl Moses cannot find a log4perl.properties file, and if there are no logging options in moby-services.cfg, it assumes some defaults (check them in MOSES::MOBY::Base, in its BEGIN section, if you need-to-know).

The better way is to use log4perl.properties file. The file name can be actually different - it is specified by an option log.config in the moby-services.cfg configuration file. This is what PerlMoses installation creates there (of course, using your own path):

The log4perl.properties is created (in the installation time) from the log4perl.properties.template, by putting there your specific paths to log files. The log4perl (or log4j) documentation explains all details - here is just a brief example what is in the file and what it means:

It says: Log only INFO (and above) levels (so no DEBUG messages are logged) on the screen (meaning on the STDERR) and to a file. But because of the "screen appender" has defined a Threshold FATAL - the screen (STDERR) will get only FATAL messages. There is no threshold in the "file appender" so the file gets all the INFO, WARN, ERROR and FATAL messages. In both cases the format of the messages is defined by the "ConversionPattern".

Note that printing to STDERR means that the message will go to the error.log file of your Web Server.

To change the log level to DEBUG, replace INFO by DEBUG in the first line.

The message format (unless you change the Perl Moses default way) means:

Where:

The last option how to specify logging properties is to set few configuration options in the moby-service.cfg file. It was already mentioned that there is an option log.config that points to a full log property file. If this option exists, no other logging configuration options are considered. But if you comment it out, you can set the basics in the following options:

Where log.file defines a log file, log.level specifies what events will be logged (the one mentioned here and above), and the log.pattern creates the format of the log events.

This is meant for a fast change in the logging system (perhaps during the testing phase).

There are definitely more features in the Log4perl system to be explored:

For example, in the mod_perl mode it would be interesting to use the "Automatic reloading of changed configuration files". In this mode, Log::Log4perl will examine the configuration file every defined number of seconds for changes via the file's last modification time-stamp. If the file has been updated, it will be reloaded and replace the current logging configuration.

Or, one can explore additional log appenders (you will need to install additional Perl modules for that) allowing, for example, to rotate automatically log files when they reach a given size. See the Log4perl documentation for details.

Deploying means to make your BioMoby service visible via your Web Server.

By contrast to the deployment of the Java-based services, Perl services are invoked by a cgi-bin script directly from a Web Server - there is no other container, such as Tomcat in the Java world. Which makes the life slightly easier. Well, only slightly, because soon you will start to think about using mod_perl module, and it may make things complicated. To be done The Perl Moses was not tested in the mod_perl environment, at all. But it should be. I wonder if anybody can explore this a bit. In order to deploy a Perl Moses BioMoby service, you need:

You see above that the script "requires" SERVICES_TABLE file (its name was taken during installation form the configuration). This is a dispatch table with a list of all BioMoby services served by this script. Yes, one cgi-bin script can serve more services - meaning that more services are registered with the same endpoint (and they are distinguished by their service names - which is, surprisingly, not part of the endpoint; but that's the way how Perl does it).

These three things (a Web Server knowing about your cgi-bin scripts, a cgi-bin script knowing about Perl Moses code, and a dispatch table knowing what services to serve) are all you need to have your service deployed.

After reading so far you may still wonder: Okay, but what should I do in my implementation to gain all the benefits generated for me by Perl Moses? This section will try to answer it - but notice that some particular activities were already explained in corresponding sections about logging|"Logging" and configuration|"Configuration".

First of all, you need to have a service implementation, at least its starting (empty) phase. Generate it, using the moses=generated-services script. Depending on how you generate it (without any option, or using an -S option) generator enables one of the following options (not that it matters to your business logic code):

Secondly, you need to understand when and how your implementation code is called:

Every BioMoby request can have multiple queries (in the Moses world, called jobs). Your service implementation has to implement method process_it that is called for every individual job contained within every incoming request. The MOSES/MOBY/Service/ServiceBase has details about this method (what parameters it gets, how to deal with exceptions, etc.).

In the beginning of the generated process_it method is the code that tells you what methods are available for reading inputs, and at the end of the same method is the code showing how to fill the response. Feel free to remove the code, extend it, fill it, turn it upside-down, whatever. This is, after all, your implementation. And Perl Moses generator is clever enough not to overwrite the code once is generated. (It is not clever enough, however, to notice that it could be overwritten because you have not touched it yet.)

Perhaps the best way how to close this section is to show a full implementation of (so often mentioned) service Mabuhay:

When you go through the code above you notice how to do basic things that almost every service has to do. Which are:

Reading input data:

The possible methods were already pre-generated for you so you know what methods to use. But you should always check if the data are really there (the clients can send you rubbish, of course).

What was not pre-generated are the methods accessing ID and NAMESPACE. Their names are, not surprisingly, id and namespace. For example, the Mabuhay input is named language (as seen in the code above), so you can call:

The question is what to do if input (or anything else) is not complete or valid. This brings us to...

Reporting exceptions:

This immediately stops the processing of the input request (ignoring all remaining jobs if they are some still there), the text of the error message is put into the response as an exception with the code 600 ("INTERNAL_PROCESSING_ERROR"), the same message is logged as an error, and the response is sent back to the client.

Note, however, that the response may already contain some outputs from the previously processed jobs. If you do not like it, you can remove it (find them in the $context parameter).

Another, less drastic, option is to record an exception (and, usually, return):

This creates an exception in the response - you choose what code to use -, and it does not prevent processing of the remaining (if any) jobs.

In addition to using an eval{} block to handle exceptions (as shown above), you can also use a try-catch-finally block structure if Error.pm has been installed in your system. See documentation of MOSES::MOBY::Base for details and examples.

Creating output data:

Creating and adding cross-references:

Add the following to the end of your httpd.conf file:

Make sure to change /path/to/cgi-bin to be the real path to your cgi-bin directory!

If you cannot create symbolic links, another tested alternative would be to copy your file MobyServer.cgi to the cgi-bin directory of your web server.

It is highly recommended that you change the name of the file if you anticipate other users copying their files to the cgi-bin directory as well!

Once the file has been copied, change the ownership of the file to the web servers' user/group. Also, make sure that the path (and its parents) to all of the directories in the 'use lib ...' are readable by your web server.

That's all there is to it! Now when you test your services, remember that your file may no longer be called MobyServer.cgi, but something else that you named it!

This could mean that the package IO::Prompt is not installed properly.

What version do you have?

perl -MIO::Prompt -e'print "$IO::Prompt::VERSION\n";'

We have tested version 0.99.2 on both *nix machines and windows. Please make sure that you have that version. If you do not, please remove the one that you have (the cpan module CPAN Plus is very useful here) and install version 0.99.2! Version 0.99.4 doesnt seem to work too well and produces numerous warnings in our module. Other versions have yet to be tested.

There will be always bugs waiting to be fixed. Please let us know about them.

And there are features (and known) bugs that should or could be implemented (or fixed). Here are those I am aware of (B = bug, N = not yet implemented, F = potential future feature):

    * (B) Article names containing dashes and spaces are still "escaped" in the XML output. They should be kept as registered.

    * (N) Documentation of the Perl Modules is unfinished, and links to it are not yet included in this document. This is an important part of the documentation because it expands hints how to write your own service implementation.

    * (N) The generated service implementation could have a better Perl documentation, listing all available methods for inputs and outputs (the methods are already shown in the code, but having them also in the POD would help).

I will try to keep up-to-date the list of the recent changes in the Changes file included with the distribution.

The main developers (whom please direct your suggestions and reports to) are Martin Senger and Edward Kawas.

However, there would be no MoSeS without BioMoby - the BioMoby project was established through an award from Genome Prairie and Genome Alberta, in part through Genome Canada, a not-for-profit corporation leading Canada's national strategy on genomics. We acknowledge the support of the EPSRC through the myGrid (GR/R67743/01, EP/C536444/1, EP/D044324/1, GR/T17457/01) e-Science projects, the INB (Spanish National Institute for Bioinformatics), funded by Fundación Genoma España, and the Generation Challenge Programme (GCP; http://www.generationcp.org) of the CGIAR.

Martin Senger was developing the project in the frame of the Generation Challenge Programme, getting no-nonsense support from Richard Bruskiewich from the International Rice Research Institute in Philippines, and high motivation to work with Perl from Mathieu Rouard from INIBAP.

None by default.

For some tutorials on using Perl MoSeS:

MOBY discussion list http://www.biomoby.org/mailman/listinfo/moby-l

MOBY Developers List http://www.biomoby.org/mailman/listinfo/moby-dev

MOBY bugs discussion list http://www.biomoby.org/mailman/listinfo/moby-bugs

Please visit the BioMOBY website at http://biomoby.org!

 Edward Kawas (edward.kawas [at] gmail [dot] com)
 Martin Senger (martin.senger [at] gmail [dot] com)

Copyright (c) 2007 Martin Senger, Edward Kawas.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

This software is provided "as is" without warranty of any kind.