An Introduction Into libmsgque Shell Integration

libmsgque was designed to act as a glue between different applications. For an overview about the basic concepts we are using the good old shell and using libmsgque to extend the usability of the well known pipe '|' syntax.

basic shell behaviour

A shell command-line is a collection of one or more commands linked together using the '|' symbol:

command1 | command2 | command3

command1, command2 and command3 are started by the shell and the stdout of command1 is the stdin of command2 and the stdout of command2 is the stdin of command3. The data send through the pipeline are strings and every command in the pipeline have to parse the string output of the previous command to extract the information's needed.

• advantage:
  • easy to use and human readable interface based on strings
• disadvantage:
  • every command has to re-parse the output of the previous command

additional shell behaviour using the libmsgque syntax

libmsgque is adding an additional link character '@' to the shell and the example from above looks like this:

alfacmd1 @ alfacmd2 @ alfacmd3

Only alfacmd1 is started by the shell and gets '@ alfacmd2 @ alfacmd3' as command-line arguments. libmsgque will start the both commands alfacmd2 and alfacmd3 and setup the message-queues:

1. alfacmd1 -> alfacmd2
2. alfacmd2 -> alfacmd3

Alfacmd2 receiving the output from alfacmd1 and alfacmd3 receiving the output from alfacmd2 without re-parsing the data again.

• advantage:
  • single parsing of output and direct access to columns and rows
  • the alfacmdX commands are independent processes and only connected by message queues
• disadvantage:
  • every command using the alfa syntax have to use the libmsgque library

interface between shell commands and alfa commands

For full integration of alfa commands into the shell syntax 2 additional interfaces are necessary

• advantage:
  • direct integration between shell and alfa commands
  • reusing of already available shell commands and adding new features by alfa commands

interface: shellcmd | alfacmd

To connect a shell with an alfa command the special alfa command split is used:

    shellcmd | atool split @ ...

The split command expect input data from stdin and is sending output data as package to an alfa command. For every input data string an output package is created by splitting the input string into output objects using the the delimiter -d.

interface: alfacmd | shellcmd

To connect an alfa with a shell command the special alfa command join is used.
If the libmsgque object was created by atool :

    ... @ join | shellcmd

or if the libmsgque object was not created by atool :

    ... @ atool join | shellcmd

The join tool expect data from a msgque client as input and create for every input package an stdout output string by joining the objects of the input package together using the delimiter -d .

command pipelines using multiple hosts

starthost:     alfacmd1 --tcp --host rmthost --port rmtport
endhost:       alfacmd2 --tcp --port rmtport

By default libmsgque is using unix-domain sockets (UDS) for communication but inet (TCP) sockets can be used as well. The data-flow is the same as above except that two hosts are involved using libmsgque over tcp sockets for connection. The tcp connection is buildup between alfacmd1 and alfacmd2.

• advantage:
  • multiple hosts can be used

a collection of examples should help to understand the software

example: this is a list of commands already available in this distribution

• atool - a tool to work like a swiss-knife for command-line operations
• aexec - a tool to setup a remote function call client and server

example 1 : just the famous hello world example

echo 'hello world with text' | atool split -d " " @ cut -f 0,1 @ join -D ":"

return: hello:world

example 2 : a little bit more features used

echo -e 'nobody 10 euro\nmanager 1000 dollar\nworker 100 pound' | \
    atool split -d " " @ sort -1 D @ join -d " : " -0 "position -> %-8s" \
        -1 "amount = %7.2f" -2 "%s"

with the output:

position -> nobody   : amount =   10.00 : euro
position -> worker   : amount =  100.00 : pound
position -> manager  : amount = 1000.00 : dollar

example 3 : use tcl to create a smart filter

The following tcl code total.tcl does 2 things:

1. convert the currencies into dollar ($)
2. calculate the total amount

package require TclMsgque
set total 0
proc FTR {ctx} {
  foreach {position amount currency} [$ctx ReadAll] break
  switch -exact $currency {
    euro    {set exchange 1.3541}
    pound   {set exchange 1.9896}
    default {set exchange 1}
  }
  set amount [expr {$amount * $exchange}]
  set currency dollar
  set ::total [expr {$::total + $amount}]
  $ctx SendSTART
  $ctx SendC $position
  $ctx SendD $amount
  $ctx SendFTR
}
proc EOF {ctx} {
  $ctx SendSTART
  $ctx SendC total
  $ctx SendD $::total
  $ctx SendFTR
}
set srv [tclmsgque MqS]
$srv ConfigSetFilterFTR FTR
$srv ConfigSetFilterEOF EOF
if {[catch {
  $srv LinkCreate {*}$argv
  $srv ProcessEvent -wait FOREVER
}]} {
  $srv ErrorSet
}
$srv Exit

using the following command pipeline:

echo -e "nobody 10 euro\nmanager 1000 dollar\nworker 100 pound" | \
    atool split -d " " @ sort -1 D @ tclsh total.tcl @ \
	atool join -d " : " -0 "%-8s" -1 "%5.2f$"

to create the following result:

nobody   : 13.54$
worker   : 198.96$
manager  : 1000.00$
total    : 1212.50$

example 4 : use the debug mode

The debugging mode is enabled with --debug followed by a number between 1 and 9

• debugging output from: client --num 1 --debug 9 @ server