RPC::Serialized - Subroutine calls over the network using common serialization

version 1.110470

 # for the RPC server...

 # choose one of the supplied server types (NetServer is Net::Server)
 use RPC::Serialized::Server::NetServer;

 my $s = RPC::Serialized::Server::NetServer->new;
 $s->run;
     # server process is now looping and waiting for RPC (like Apache prefork)
     # the default port number for Net::Server is 20203

 # and so for the RPC client...

 use RPC::Serialized::Client::INET;

 my $c = RPC::Serialized::Client::INET->new({
     io_socket_inet => {PeerPort => 20203},
 });

 my $result = $c->remote_sub_name(qw/ some data /);
     # remote_sub_name gets mapped to an invocation on the RPC server
     # it's best to wrap this in an eval{} block

This module allows you to call a Perl subroutine in another process, possibly on another machine, using a very simple and extensible interface which ties together the features of other good modules from the CPAN.

There are lots of uses for RPC (remote procedure calls), so here are a couple of examples just to give you an idea:

Priveledge separation

If you have a web interface which is used to control a critical backend system, perhaps a key database or the settings on a firewall, you can use RPC to prevent security flaws on the web service from affecting the backend service. Only procedure calls which are permitted will be accepted from the web host, and it also offers a nice interface separation for your systems.

File or data access

To avoid sharing of filesystems over the network (SAMBA, NFS, etc), you can provide a restricted interface using RPC. For example a web service to search log files could send an RPC request with the search string to the log server, and display the results. There would be no need to run a network filesystem.

What makes this module different from another RPC implementation?

Data Serialization

This module uses Data::Serializer to construct its "on the wire" protocol. This means any Perl data structure can be sent or received, even Perl code itself in the case of some of the serialization modules (e.g. YAML supports this). You can also encrypt and compress data; all options to Data::Serializer are easily available through the configuration of this module.

Simple deployment

Each remote procedure is simply a perl subroutine in a module which is loaded by the server. You can let the server autoload everything as it is called, or specify each "handler" subroutine individually (or combine both!). Adding and modifying the available handlers is simple, meaning you think less about the RPC subsystem and more about your code and service provision.

Flexible configuration

All the modules used by RPC::Serialized can be fully configured as you wish, from one configuration file, or via options to new(). You saw an example of this in the SYNOPSIS section, above, for the IO::Socket::INET module.

The following sections take you through setting up an RPC server and client.

Both the client and server parts of this module share the same configuration system. There is a file, RPC::Serialized::Config, which contains the basic defaults suitable for most situations. Then, you can also specify the name of a configuration file as a parameter to any call to new(). Finally, a hash reference of options can be supplied directly to any call to new(). Let's go through these cases by example:

 # this is for a server, but the example applies equally to a client
 use RPC::Serialized::Server::NetServer;

 # no configuration at all - use the built in defaults
 $s = RPC::Serialized::Server::NetServer->new;

If you are happy with the settings in the source code of RPC::Serialized::Config, then no options are required. In the case of some types of server and client this is enough to get you going.

 # load a configuration file, using Config::Any  
 $s = RPC::Serialized::Server::NetServer->new('/path/to/config/file');

Alternatively, specify a file with configuration. We use Config::Any to load the file, and so it should contain a data structure in one of the formats supported by that module. Tip: make sure there is a filename suffix, e.g. .yml, to help Config::Any load the data. For details of the required structure of that data, read on...

 # pass some options directly
 $s = RPC::Serialized::Server::NetServer->new({
      net_server => {log_file => undef, port => 5233},
      rpc_serialized => {handler_namespaces => 'RPC::Serialized::Handler'},
 });

You can pass a hash reference to new(). Each key in this hash is the name of a module to configure. The module names are converted to lowercase, and the :: separator is replaced by an underscore. In the example above we are providing some options to Net::Server and this module, RPC::Serialized.

The value of each key is another anonymous hash, this time with any options as specified by that module's own manual page. Of course, this only works for modules which use key/value options themselves, but thankfully that is the case for each module used by RPC::Serialized.

Remember, in all of the examples in this manual page which show passing configuration settings to new(), you can also achieve the same thing using a configuration file by passing its name to new() instead.

As a final note on this topic, you can provide both a configuration filename, and an hash reference of options to the new() call. In fact, whatever and however many of these you provide, they will be read in with the later ones taking prescedent.

You do not have to know too much about internet servers to use this module. As well as providing a standard TCP and UDP server, there are also UNIX socket and Standard Input/Output servers that you might use for communicating with clients on the same host system.

In the main, we are dealing with a UNIX world here, so most of the description will make assumptions about that. If you get this module running on Windows, please let the author know!

A small perl script which starts the server running is all you need. This can be copied verbatim from the SYNOPSIS section above. For guidance on providing configuration to the server, see GENERAL CONFIGURATION, above.

When running a server process, it can either stay in the foreground, or detach from your shell. The default is to stay in the foreground, for two reasons. First, when you are developing you probably want to start the server and see what is happening on STDERR. Second, many people use Dan Bernstein's daemontools package to manage persistent servers, and this requires a process which does not detach from its parent process. If you are using the NetServer server, then it is easy to make it detach:

 $s = RPC::Serialized::Server::NetServer->new({
      net_server => { background => 1, setsid => 1 },
 });

To stop the server you will then have to issue a kill to the detached process.

There is a handful of alternative servers shipped with this module. For more details, please see the manual pages for each of them:

RPC::Serialized::Server::NetServer

This is a full-blown pre-forking internet server, with many many good features. You will have to install the Net::Server Perl module and dependencies to use this server. It supports TCP and UDP INET sockets, as well as UNIX domain sockets.

RPC::Serialized::Server::STDIO

This is a very simple server which processes one request at a time, accepting data on Standard Input and sending responses to Standard Output.

RPC::Serialized::Server::UCSPI::TCP

If you use Dan Bernstein's tcpserver (a.k.a. ucspi-tcp) then this is the option for you. It is based upon the STDIO server, above, and is designed to be fired up by tcpserver whenever an incoming connection is handled. There is an example script for this in the Perl distribution for this module.

RPC::Serialized::Server::UCSPI::IPC

This is similar to the option above, but for UNIX (i.e. local filesystem) sockets rather than INET sockets. It is designed for ucspi-ipc, producted by SuperScript Technology, Inc., and you can find more details by searching on Google.

Each RPC message which comes in "over the wire" is really just a Perl data structure, a hash. There is a "CALL" key, which has the name of the RPC method to invoke, and some "ARGS" to pass to it as arguments.

The server looks at the CALL and tries to load and execute the handler which maps to that CALL. If it fails it raises an Exception, and fires that back to the client. If the invocation is successful, then the RESPONSE is sent back to the client, again in a Perl data structure. It is all quite elegant and simple (i.e. not my design, see AUTHOR below for the acknowledgement!).

On the wire, if we switch off most of the Data::Serializer magic and set the Serializer to YAML, then it looks like this:

 ---
 CALL: localtime
 ARGS: []
 ...
 --- 
 RESPONSE: Sun Jul  8 21:57:28 2007
 ...

Note that ... is the record separator which tells the server when it can process the incoming request. In this example, the CALL was for a method called localtime, and there were no arguments so I passed an empty list. The RESPONSE was just the scalar output of Perl's localtime function.

You might find it interesting to note that, inside of RPC::Serialized, the methods used to send and receive data at the client and server are identical. In the example above, I entered the YAML document and trailing ... to make the method call, and the server responded with another YAML document and ... record terminator. Using one of the RPC::Serialized::Client family, it would look just the same.

First, know that there are three example handlers included in this distribution, so you can just go and look at them if you prefer reading code to documentation! See the modules under RPC::Serialized::Handler::.

So you might have guessed that the first step is to choose your package name for the handler. Each package contains one handler, or put another way, each handler lives in its own package. You can either have the handler's name be derived from the package's name, or set it manually. Using the localtime example from above, this is what the handler looks like:

 package RPC::Serialized::Handler::Localtime;

 use strict;
 use warnings FATAL => 'all';

 use base 'RPC::Serialized::Handler';

 sub invoke {
     my $self = shift;
     my $time = shift;

     $time = time unless defined $time;
     return scalar localtime($time);
 }

 1;

There is a "magic" method in the package, called invoke(), and it is this which is called by the RPC server to handle the incoming request.

By default all the servers will take the name of the requested method CALL, convert underscores to :: separators, convert initial letters to uppercase, and try to load a module in the RPC::Serialized::Handler:: namespace. For example, if I called the $c->frobnits_goo handler from a client, the server would try to load a package called RPC::Serialized::Handler::Frobnits::Goo and then call the <invoke()> method within that.

Alternatively you can see the OPTIONS FOR THIS MODULE section below to change that default namespace, or have your own mappings between client calls and loaded handler packages (or a mixture of both).

You should expect all arguments passed to your invoke() method to be in a plain perl list (i.e. @_); items in the list may be references to complex data structures. You should return a single scalar value from that method, and nothing else. The scalar value can, however, be a reference to an arbitrarily complex data structure.

As with the server, you don't need to know a lot about how networked services operate in order to set up the client. However you probably do need to know where your server is listening, to make contact with it!

Therefore, you will need to use the client which corresponds to your server. Please read the manual page for the appropriate module:

RPC::Serialized::Client::INET

Use this client package to communicate with either a NetServer server, or the UCSPI/TCP server.

RPC::Serialized::Client::UNIX

This client will make UNIX domain socket (i.e. local filesystem) connections to the NetServer or UCSPI/IPC servers.

RPC::Serialized::Client::STDIO

For testing purposes you can use this client, which communicates on Standard Input and Standard Output. Alternatively, use this package as a base to implement a new kind of client.

The basis of the client set-up is given in SYNOPSIS, above, but we will show another example here for completeness:

 #!/usr/bin/perl

 use strict;
 use warnings FATAL => 'all';

 use Readonly;
 use RPC::Serialized::Client::UNIX;

 Readonly my $SOCKET => '/var/run/rpc-serialized-example.socket';

 my $c = RPC::Serialized::Client::UNIX->new({
     io_socket_unix => { Peer => $SOCKET }
 });

 eval {
     my $response = $c->echo(qw/ a b c d /);
     print "echo: " . join( ":", @$res ) . "\n";
 };
 warn "$@\n" if $@;

 eval {
     my $now = $c->localtime;
     print "Localtime on the server is: $now\n";
 };
 warn "$@\n" if $@;

The above code uses the UNIX domain socket client to contact a server which is listening on the file mentioned in $SOCKET. Once the client object is set up, we can make a call to any method we wish on the remote server, just by specifying its name. Here, we call the echo and the localtime handlers.

For each call, you should specify the arguments as a plain perl list, although that list can include references to complex data structures. There is a single (scalar) return value from each call you make, although again this may be a reference to a data structure if you wish.

This example also shows how you should use eval{} constructs around the RPC calls. This is good practice for most network programming, as all kinds of things can go wrong. See the section ERROR HANDLING below for more information.

You should be aware that RPC::Serialized operates timeouts on all handler calls. By default, you get 30 seconds to make your request (i.e. make the call and pass any data to the server), another 30 seconds for the server to handle the call, and 30 seconds to transfer the response back to your client.

If any of this fails or times out, an exception will be raised and passed back to you if possible. Exceptions can be passed through RPC, but you don't need to know about how that works, only that you should be prepared to handle die using eval.

To alter the timeout setting, see the next section OPTIONS FOR THIS MODULE. To see examples of the eval construct see ERROR HANDLING, below.

There is actually only a small number of options for this module, as most of the heavy lifting within is done by other modules on the CPAN. In particular, we use Data::Serializer, and there is a section below which explains how to customize your serializer set-up.

Options are passed to the new() method in a hash reference. To see how this is done, take a look at the GENERAL CONFIGURATION or SYNOPSIS sections above, although here is a quick example:

 my $s = RPC::Serialized::Server::NetServer->new({
     rpc_serialized => { timeout => 15, trace => 1 },
 });

handler_namespaces

As explained above in How the server works, the server will try to auto-load the handler for a call, based on some naming conventions. This value sets the Perl package namespaces which are searched. You can set this either to a scalar string, the name of a single namespace, or an array reference containing a list of such namespaces. The default setting is RPC::Serialized::Handler, into which we supply the echo, localtime, and sleep handlers as examples. Setting this value to an array reference containing an empty list will disable auto-loading.

handlers

An alternative to handler_namespaces, this value allows you to map individual CALLs to a given package name. In this way you can alias or alter the "published" name of handlers, or restrict calls. It can be used in addition to handler_namespaces, but will take priority where both can be used to invoke an RPC handler. Set this value to an anonymous hash, where keys are calls made by the client (e.g. echo) and values are package names containing the handler (e.g. RPC::Serialized::Handler::Echo). No handlers are specified by default.

timeout

This is a scalar value which sets how long RPC::Serializer servers wait before timing out their connections. As explained above, it is used when receiving an RPC call, when dispatching to the handler of that call, and when replying to the client. Each phase is given the timeout in seconds to do its work. The default value is 30 seconds.

trace

This is a boolean (scalar) which sets whether logging of the content of RPC traffic is made using UNIX syslog. For more details see the section SERVER LOGGING, below. If set to a true value, logging will be enabled. The default value is false.

debug

This is a boolean (scalar) which disables the RPC::Serializer magic, meaning just the raw serialized data structures are sent between client and server. Normally, RPC::Serializer will perform compression, encryption, ASCII-armoring and hashing of the data it sends, if so configured. If set to a true value, debug prevents this. It can be very useful when combined with the STDIO client and server, to test operations, as you can type CALLs in by hand at the console. The default value is false.

The defaults for Data::Serializer, which is used to encode and decode your method calls and responses, are quite sane so you can safely leave this alone.

However you might prefer to override this and use a particular serialization format, or enable encryption, and this is quite straightforward to do. Passing a hash of options within the call to new() at either the client or server will do this, like so:

 my $c = RPC::Serialized::Client::STDIO->new({
     data_serializer => { serializer => 'YAML::Syck', encoding => 'b64' },
 });

The only option which you cannot alter is portable, and this is forced to true, meaning that Data::Serializer will ASCII-armor the a data structure (i.e. encode it in hexadecimal or base64). Of course, if you have enabled the debug option to RPC::Serialized then portable is ignored.

In most cases, the Data::Serializer module at the RPC server will auto-detect the settings used, and reply with a packet with the same settings. Where this might not work is in two cases: First make sure that the serializer used on the client is installed on the server. Second, make sure any keys and modules used for encryption on the client are available on the server. With a standard install of RPC::Serialized there should be no concern here, as it uses only core Perl modules, and encryption is not enabled.

For further details please see the Data::Serializer manual page.

The Net::Server binding shipped with this module has some defaults set, although none are enforced so you can override all options to that module.

The chosen personality is PreFork, and a Single personality is also available. If you want to use something else just copy the bundled binding module (RPC::Serialized::Server::NetServer) and modify as appropriate. Default settings which differ from those in the native Net::Server are as follows:

log_level is set to 4

syslog_facility is set to local1

background is set to undef

setsid is set to undef

This means that logging goes to STDERR from the parent server, but to send it to Syslog instead just do the following (after reading the Net::Server manual page):

 my $s = RPC::Serialized::Server::NetServer->new({
     net_server => { log_file => 'Sys::Syslog' },
 });

In addition the server does not fork or detach from the shell and go into the background. For further details please see the Net::Server manual page.

The system from which RPC::Serialized derives supports user-based authorization based on a calling username, the called handler, and the arguments passed to that handler.

In addition, Net::Server supports IP-based access control lists.

Both of these systems are available although by default disabled. Looking in the examples folder with this distribution you should find some sample ACLs for RPC::Serialized. You can also consult the Net::Server manual page for its options.

For the time being the authorization is not documented here, but it is hoped this will be remedied before too long! If you want help with authorization configuration, feel free to email the module author.

If you have enabled RPC server logging, using the trace option to new(), then output is sent via UNIX Syslog. The server will write out a serialized dump of the data sent and received, using whichever serializer you have set the server to use. This might not be the same serializer used in the transaction, however, as explained in the section CONFIGURING Data::Serializer, above. You will see the CALL, ARGS, RESPONSE and any EXCEPTIONs raised, in the log.

Logging uses the excellent Log::Dispatch module from CPAN, with its Syslog personality. The default settings are as follows:

name is set to rpc-serialized

min_level is set to info

facility is set to local0

callbacks is set to add a newline to each log message

You can override these settings in the configuration file, or the call to new(), like this:

 my $s = RPC::Serialized::Server::STDIO->new({
     log_dispatch_syslog => { facility => 'local7' },
 });

Log messages will be dispatched to your syslog subsystem at the level set in min_level. Note that the hash key used is log_dispatch_syslog, as above.

If you transmit sensitive data in the arguments to handler calls, but also wish to log a trace of the handler call+args, then the args_suppress_log configuration parameter will help.

This parameter takes a Hash reference where they keys are the names of handlers and the values are Array references of sensitive argument names. Naturally, this assumes treating of the args list as a Hash of keys/values by the handler and you would only be able to use this parameter in that situation. For example:

 $s = RPC::Serialized::Server::NetServer->new({
     rpc_serialized => { args_suppress_log => {
         login => [qw/ password /],
     }},
 });

Using the above configuration, the login handler when called would not log the value of the password named argument in its args. The text [suppressed] is output to the log in place of the named argument's value.

This module makes use of Exception::Class when it needs to raise a critical error, but don't fret if this makes no sense to you. The essential concept is that calls to this module might die, and you need to be able to deal with that.

The usual way is to wrap calls in an eval{} block to trap errors, like so:

 eval {
     my $now = $c->localtime;
     print "Localtime on the server is: $now\n";
 };
 warn "Remote procedure call failed: $@\n" if $@;

A nifty part of this module (courtesy of the original authors of the code) is that an exception can be raised in the server and delivered to the client. The exceptions are RPC::Serialized::X objects, derived from Class::Exception, of the following types:

RPC::Serialized::X::Protocol is for an RPC protocol error

RPC::Serialized::X::Parse is for a Data::Serializer parser failure

RPC::Serialized::X::Validation is for a data validation error

RPC::Serialized::X::System is for any system error

RPC::Serialized::X::Application is for application programming errors

RPC::Serialized::X::Authorization is for an authorization failure

Typically you want to check if it was RPC::Serialized having a problem, or some other issue:

 eval {
     my $num = $c->cabbages;
     print "Number of cabbages is: $num\n";
 };
 if ($@ and $@->isa('RPC::Serialized::X')) {
     print $@->message, "\n"; # "no handler for cabbages"
 }
 else { die $@ } # rethrow the exception

For further details please see the Class::Exception manual page.

Here is a list of the common error messages and exception types raised by this module, and probable causes:

Failed to create socket: ... in an X::System

The INET or UNIX client has failed to set up an IO::Socket::INET or IO::Socket::UNIX socket respectively.

Invalid or missing CALL in an X::Protocol

After de-serializing the incoming data message from the client, there appears to be no CALL parameter.

Invalid or missing ARGS in an X::Protocol

After de-serializing the incoming data message from the client, there appears to be no ARGS list.

Failed to load ... in an X::System

The server has attempted to load the handler specified for the current call, but failed. Did you specify the correct handler?

No handler for ... in an X::Application

After searching any manual handler mappings, or the auto-load namespaces, no suitable handler package was found for the current call.

... not a RPC::Serialized::Handler in an X::Application

Having found a package to load for the current call from the handler specification, it does not inherit from RPC::Serialized::Handler.

Cannot search for invalid name: ... in an X::Application

You are attempting to auto-load a handler whose package name would not be valid in perl. It must be letters, digits and underscores only.

Invalid or missing CLASS in an X::Protocol

An Exception class rasied by the server is not known to the client, so this exception is raised instead.

Object method called on class in an X::Application

You are attempting to invoke a call on the client module directly, rather than instantiating a new client object from it and then making the call on that.

Missing or invalid input handle in an X::Application

The server has not been passed a valid IO::Handle upon which to read data. The handle is passed in the call to new() or via the ifh accessor method on the server object.

Missing or invalid output handle in an X::Application

The server has not been passed a valid IO::Handle upon which to write data. The handle is passed in the call to new() or via the ofh accessor method on the server object.

Failed to load Log::Dispatch but trace is on: ... in an X::Application

You have enabled server logging using the trace option, but the Log::Dispatch or Log::Dispatch::Syslog module has failed to load.

Data not a hash reference in an X::Protocol

After de-serializing some data (from the client or server), the data structure appears not to be a hash reference.

Failed to send data: ... in an X::System

A system error has ocurred when sending data through the handle to the client or server.

Failed to read data: ... in an X::System

A system error has ocurred when reading data from the handle to the client or server.

Data::Serializer error: ... in an X::Protocol

An error has been thrown by the Data::Serializer module when initializing.

Serializer parse error in an X::Protocol

An error has been thrown by the Data::Serializer module when attempting to serialize or de-serialize data to or from the client or server.

In addition to the contents of the standard Perl 5.8.4 distribution, this module requires the following:

Data::Serializer

Exception::Class

Module::MultiConf

Readonly

Class::Accessor::Fast::Contained

To use some optional features, you may require the following:

Net::Server

Log::Dispatch

GDBM_File

This module is a derivative of YAML::RPC, written by pod and Ray Miller, at the University of Oxford Computing Services. Without their brilliant creation this system would not exist.

Oliver Gorwits <oliver@cpan.org>

This software is copyright (c) 2011 by University of Oxford.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.