vbmsgque

The VB.NET Application-Server Project

SYNOPSIS

Imports csmsgque

INDEX

OBJECT CREATION AND DELETION
OBJECT CONFIGURATION
LINK CREATION AND DELETION
ADDING SERVICES
SEND DATA
READ DATA
SLAVE OBJECT
BUFFER OBJECT
EXCEPTION OBJECT

INTRODUCTION

The msgque project is an infrastructure to link software together to act like a single software. To link mean distributing work from one software to an other software an wait or not wait for an answer. The linking is done using unix or inet domain sockets and is based on packages send from one software to an other software and back. The msgque project is used to handle all the different aspects for setup and maintain the link and is responsible for:

• starting and stopping the server application
• starting and stopping the communication interface
• sending and receiving package data
• reading and writing data from or into a package
• setup and maintain the event-handling for an asynchronous transfer
• propagate the error messages from the server to the client

OBJECT CREATION AND DELETION

MqS

The object is the user visible part of a vbmsgque application link.

Dim ctx As New MqS()

Create an object ready to be used with ctx.LinkCreate(String() argv) to setup an application link. Use the isConnected to check if a link is already available.

Private NotInheritable Class MyClass
  Inherits MqS
End Class
Sub Main(ByVal args() As String)
  Dim ctx AS New MyClass()
End Sub

deleted by Garbage Collection

Delete the entire object including the application link using ctx.LinkDelete(). The object can not be reused.

OBJECT CONFIGURATION

The configuration of an object is done using the functions starting with MqConfig... or by using command-line arguments.

static: Init(String() args)

Initialize the process startup argument prefix. The startup-prefix have to be the name of the executable found in the PATH and additional arguments like a script name or required startup options. The startup-prefix is used for two different purpose:

• To start a new entity using the --spawn command-line option
• To replace the SELF argument after the "@" command-line argument

The startup-prefix is initially set by vbmsgque during package loading.

command-line: --buffersize

type: Integer, get: ConfigGetBuffersize , set: ConfigSetBuffersize

The send and receive buffersize in bytes (default: 4KB)

command-line: --debug

type: Integer, get: ConfigGetDebug , set: ConfigSetDebug

Debug a message-queue application. Valid values are 0 <= NUM <= 9 using 0 for "no debugging" and 9 for "maximum debugging". (default: 0)

command-line: --timeout

type: Integer, get: ConfigGetTimeout , set: ConfigSetTimeout

User defined timeout used for connection setup and data sending (default: 90 seconds)

command-line: --name

type: String, get: ConfigGetName , set: ConfigSetName

Use STRING as basename of the new message-queue object. The name shows up in the debugging output and is used as prefix for the new created command. (default: the name of the executable)

command-line: --srvname

type: String, get: ConfigGetSrvName , set: ConfigSetSrvName

Use STRING as a client specific prefix in the server debugging output. This is used to link debugging and error messages on the server to a specific client connection. (default: empty)

command-line: --silent

type: Boolean, get: ConfigGetIsSilent , set: ConfigSetIsSilent

Write (false) or don't write (true) any output to stdout or stderr. (default: write to stdout or stderr)

command-line: --string

type: Boolean, get: ConfigGetIsString , set: ConfigSetIsString

Use (true), as much as possible, strings in the data-package. Convert any native data-type, like integer or double, into a string (sending) and back to native (receiving). (default: use binary)

command-line: --thread --spawn --fork

type: Integer, get: ConfigGetStartAs , set: ConfigSetStartAs (startAs)

Start a new server as thread or spawn a new process or fork a new process. This arguments are used with:

• a server-context to start a new instance after a client connection request
• on a server to create a worker with: ctx.SlaveWorker(Integer id, String() argv)
• on a client/server together with the "SELF" command-line argument: ctx.LinkCreate(String() argv)

Parameters:

[in]

startAs

0=default, 1=fork, 2=thread and 3=spawn

command-line: --tcp --host --port --myhost --myport

type: String, get: ConfigGetIoTcpHost/Port/MyHost/MyPort , set: ConfigSetIoTcp (host, port, myhost, myport)

Configure a tcp socket link.

Parameters:

	host	client: name of the remote host (default: localhost) server: name of the local interface (default: listen on all interfaces)
	port	client: name of the remote port server: name of the local port
	myhost	client: name of the local interface
	myport	client: name of the local port

Attention:

named ports are supported as well

command-line: --uds --file

type: String, get: ConfigGetIoUdsFile , set: ConfigSetIoUds (file)

Use a uds (http://en.wikipedia.org/wiki/Unix_domain_socket) socket link. The uds communication is usually 50% faster than a local tcp communication but only available on UNIX.

Parameters:

[in]

file

the name of the uds file

command-line: --pipe --socket

type: Integer, get: ConfigGetIoPipeSocket , set: ConfigSetIoPipe (socket)

Start a pipe server to listen on socket. This is the default mode to start a server. The socket option is special because it is used for internal purpose to submit the socket from the client to the server started as pipe by the client.

Parameters:

[in]

socket

the file-descriptor number. The only public usage for this option is to serve as interface for an existing tool like (x)inetd. The (x)inetd tool is a UNIX service to listen on a tcp/ip port and start for every connection the proper entry from the file /etc/(x)inetd.conf with the file-descriptor 0 as send/recv socket handle.

master

type: MqS, get: ConfigGetMaster , set: SLAVE OBJECT

return the master if ctx is a slave-context or NULL if ctx is not a slave-context. !Only a SLAVE has a master!

ctxId

type: Integer, get: ConfigGetCtxId , set: NO

return an identifier as integer and is unique per parent-context. The parent-context is always 0 and every new child-context get a new identifier by adding 1.

token

type: String, get: ConfigGetToken , set: NO

return the current TOKEN IDENTIFIER and is only useful in a service callback. This command is needed on the server to implement a generic service (A ctx.ServiceCreate(String token, AddressOf service) with different TOKEN IDENTIFIER but with the same service callback).

isConnected

type: Boolean, get: ConfigGetIsConnected , set: NO

Is the vbmsgque context connected ? A context is connected if the ctx.LinkCreate(String() argv) command was successful and a context is NOT connected if a) the object has just been created or b) the link was deleted with ctx.LinkDelete()

isServer

type: Boolean, get: ConfigGetIsServer , set: IServerSetup or IServerCleanup

True if object is a server object (default: false, be a client)

isParent

type: Boolean, get: ConfigGetIsParent , set: NO

True if object is a parent object (default: true, be a parent)

isSlave

type: Boolean, get: ConfigGetIsSlave , set: NO

True if object is a slave object (default: false, not be a slave)

IServerSetup

type: Interface, get: NO , set: csmsgque.IServerSetup.Call

Interface to define a server vbmsgque object. This Interface define the ServerSetup callback called at ctx.LinkCreate(String() argv) to define the services or to initialize context specific variables. This interface set the isServer configuration value to true.

IServerCleanup

type: Interface, get: NO , set: csmsgque.IServerCleanup.Call

Interface to define a server vbmsgque object. This Interface define the ServerCleanup callback called at ctx.LinkDelete() to free context specific variables. This interface set the isServer configuration value to true.

IFilterFTR

type: Interface, get: NO , set: csmsgque.IFilterFTR.Call

Interface required to define a filter data stream object. This Interface define the IFilterFTR callback. The callback is used to act on filter data rows. Every filter input data is a list of filter data rows and every row is a list of filter data columns. Every row is send from one filter-command to the following filter-command as FTR service request. This interface set the isServer configuration value to true. (read more at: FILTER MODE)

IFilterEOF

type: Interface, get: NO , set: csmsgque.IFilterEOF.Call

Interface required to define a filter data stream object. This Interface define the IFilterEOF callback. The callback is used to act on End-Of-Filter data and is called after all IFilterFTR data was send. Sometimes the filter data can not be served as IFilterFTR data (example: sorting of the input rows need to read all rows before the data can be send to the next filter command) and the IFilterEOF callback is used to continue send IFilterFTR data rows. This interface set the isServer configuration value to true. (read more at: FILTER MODE)

IFactory

type: Interface, get: NO , set: csmsgque.IFactory.Call

The factory pattern is used to create a new application object (read more at: MqSetupS::Factory) Without the Factory pattern only the initial startup context is available to serve incoming requests. In general every server need to provide a Factory pattern.

Private NotInheritable Class Server
  Inherits MqS
  Implements IFactory
  Public Function Factory() As csmsgque.MqS Implements csmsgque.IFactory.Call
    Return New Server()
  End Function
  ...
End Class

IBgError

type: Interface, get: NO , set: csmsgque.IBgError.Call

Define an asyncrone error handler. This handler is used for handle errors send with ctx.SendERROR(). Use Integer ex.num or ctx.ErrorGetNum() and String ex.txt or ctx.ErrorGetText() to access the error.

LINK CREATION AND DELETION

To create and to destroy a link is the main purpose of the vbmsgque library. A link is a client/server connection used to perform various tasks.

ctx.LinkCreate(String() argv)

Create a parent-context using the OBJECT CONFIGURATION. A parent-context is responsible to setup the client/server link:

• the client-parent-context create the server-parent-context
• the server-parent-context wait for a client-parent-context connection request

Parameters:

[in]	ctx	the object from OBJECT CREATION AND DELETION
[in]	args	command-line arguments including the "@" item for the --pipe setup

Attention:

if the first argument after the "@" item is the string "SELF" an independent server of the current server is started. This is not a SLAVE OBJECT. The "SELF" argument is replaced by an application default setting (if available) or by arguments set with static: Init(String() args).

ctx.LinkCreateChild(MqS parent, String() argv)

Create a child-context from a vbmsgque parent-context and command-line arguments. A child is using the same process or thread as the parent but a different namespace. With a different namespace a child is able to act on different services on the shared server.

Parameters:

[in]	ctx	the object from OBJECT CREATION AND DELETION
[in]	parent	the parent object
[in]	args	all command-line arguments including the "@" token

ctx.LinkDelete()

Close the client/server link and delete the underlying communication object. All depending objects will be deleted as well like depending child-context and slave-context local and on the remote site of the link.

ctx.Exit()

Shutdown the entire communication and exit the current process or thread.

ADDING SERVICES

To provide a service is the main purpose of a server. The ctx.ServiceCreate(String token, AddressOf service) method is usually added in the ServerSetup method of the IServerSetup.

Public Sub ServerSetup() Implements csmsgque.IServerSetup.Call
  ServiceCreate ("SRV1", AddressOf srv1);
End Sub

TOKEN IDENTIFIER

The token is a 4 byte string and identifies the service. The token is used in:

• ctx.SendEND(String token)
• ctx.SendEND_AND_WAIT(String token, [Integer timeout])
• ctx.SendEND_AND_CALLBACK(String token, AddressOf callback)
• ctx.ServiceCreate(String token, AddressOf service)
• ctx.ServiceDelete(string token)

SERVICE CALLBACK

A service is using a callback to act on an incoming service request. The callback is used on a server using ctx.ServiceCreate(String token, AddressOf service) or on a client using ctx.SendEND_AND_CALLBACK(String token, AddressOf callback).

The callback is implemented using Interface: IService method Service(MqS ctx) or using a delegate of type Sub Callback().

ctx.ServiceCreate(String token, AddressOf service)

Create a new service listen on TOKEN IDENTIFIER and start for every incoming request a service. (on error return an EXCEPTION OBJECT)

ctx.ServiceDelete(string token)

Delete a new service listen on TOKEN IDENTIFIER . (on error return an EXCEPTION OBJECT)

ctx.ProcessEvent([Integer timeout], [MqS.WAIT.(NO|ONCE|FOREVER)])

Start the Event-Loop and check for the next event. In client mode this command is usually used together with ctx.SendEND_AND_CALLBACK(String token, AddressOf callback) to process the results.

Parameters:

[in]	timeout	don't wait more than TIMEOUT seconds to get an answer from the server. If no answer is available an error is raised. (default: command-line: --timeout)
[in]	MqS.WAIT	use flag value to define the action (default: NO) NO, don't wait for an event do just the check ONCE, use timeout seconds to wait for exact one event FOREVER, wait forever and only come back on error or on exit

SEND DATA

The Send style methods are used to build and send data using the vbmsgque send data package. The data is typed using the TYPE IDENTIFIER and the methods can wait or not for an answer. During waiting the the application is still able to work on events like other vbmsgque client/server links.

ctx.SendSTART()
ctx.SendI(num)
ctx.SendL_START()
 ctx.SendD(balance)
 ctx.SendC("name of the owner")
 ctx.SendB(signature)
ctx.SendL_END()
ctx.SendEND_AND_WAIT("SRV1")

ctx.SendSTART()

Initialize a vbmsgque send data package. This method is required.

ctx.SendEND(String token)

Send the package using TOKEN IDENTIFIER without expecting an answer (FireAndForget). (on error return an EXCEPTION OBJECT)

Attention:

if an error is raised on the server during service-processing the error is send back as asynchronous event and can be raised sometime in the future. To be sure to get the error immediately use ctx.SendEND_AND_WAIT(String token, [Integer timeout])

ctx.SendEND_AND_WAIT(String token, [Integer timeout])

Send the package using TOKEN IDENTIFIER and wait timeout seconds for an answer. (on error return an EXCEPTION OBJECT)

ctx.SendEND_AND_CALLBACK(String token, AddressOf callback)

Send the package using TOKEN IDENTIFIER and don't wait for an answer. The answer will be handled by the SERVICE CALLBACK. (on error return an EXCEPTION OBJECT)

ctx.SendRETURN()

Answer an incoming service-call started with ctx.SendEND_AND_WAIT(String token, [Integer timeout]) or ctx.SendEND_AND_CALLBACK(String token, AddressOf callback). The return package is build-up like the previous service call package but with a ctx.SendRETURN() as final command. (on error return an EXCEPTION OBJECT)

ctx.SendERROR()

Send an asyncrone error to the link target. (on error return an EXCEPTION OBJECT)

ctx.SendFTR([Integer timeout])

Send a filter stream data package using FILTER MODE. (on error return an EXCEPTION OBJECT)

ctx.SendEOF([Integer timeout])

Send a filter end-of-file data package using FILTER MODE. (on error return an EXCEPTION OBJECT)

ctx.Send[TYPE](TYPE value)

Add an item using TYPE IDENTIFIER to the vbmsgque send data package. (example: SendY)

ctx.SendL_START()

Start to send an embedded list item.

ctx.SendL_END()

Finish to send an embedded list item.

READ DATA

Read data from an incoming vbmsgque read data package:

1. on a server to serve an incoming service-call from the client
2. on a client to process the return-data from a previous service-call

Reading data is a passive task in opposite to sending data which is an active task. Passive mean that the read is triggered by an incoming data-package and not by the software-workflow.

num = ctx.ReadI()
ctx.ReadL_START()
 balance = ctx.ReadD()
 owner = ctx.ReadC()
 signature = ctx.ReadB()
ctx.ReadL_END()

TYPE ctx.Read[TYPE]()

return an item using the TYPE IDENTIFIER from the vbmsgque read data package. (on error return an EXCEPTION OBJECT)

MqBufferS ctx.ReadU()

return a temporary BUFFER OBJECT. (on error return an EXCEPTION OBJECT)

Attention:

The object is only valid until a new vbmsgque read data package has arrived.

ctx.ReadProxy(MqS otherCtx)

Link two vbmsgque objects to direct pass a data item from one object to the other. (on error return an EXCEPTION OBJECT)

Integer ctx.ReadGetNumItems()

return the number of items left in the vbmsgque read data package.

Boolean ctx.ReadItemExists()

Check if are still items in the vbmsgque read data package.

ctx.ReadUndo()

Undo the last read operation. This is useful if an error was raised or if a buffer object (returned from MqBufferS ctx.ReadU()) has not the expected type. (on error return an EXCEPTION OBJECT)

ctx.ReadL_START()

Start to read an embedded list item. Use the current read package (buffer = NULL) or a buffer object. (on error return an EXCEPTION OBJECT)

ctx.ReadL_END()

Finish to read an embedded list item.

SLAVE OBJECT

The master/slave link is used to create a mesh of nodes by linking different PARENT context objects together. The master control the slave.
The link is used to perform the following tasks:

• report error messages from the slave to the master
• to create a slave-child-context if a master-child-context is created
• to delete a slave-context if a master-context is deleted

In difference to the client/server link the master/slave link connect two independent msgque-context in the same process or thread (e.g. node). This leads to the restriction that only the master-msgque-context can be a server because only one server per node is possible.

ctx.SlaveWorker(Integer id, String() argv)

Create a worker as slave of the ctx object using the image of the ctx object self and started as an independent process or thread based on the command-line: --thread --spawn --fork argument.

Parameters:

[in]	ctx	the master vbmsgque object as SERVER-PARENT without a CHILD
[in]	id	an integer used as unique identifer for the master/slave link
[in]	args	command-line arguments passed to the worker-client or the worker-server. all arguments prior the first @ token are added to the worker-client and the other arguments to the worker-server.

example create a worker with id 7 as thread.

ctx.SlaveCreate (7, "--thread", "--silent", "@", "WORKER", "--name", "MyWorker");

ctx.SlaveCreate(Integer id, MqS slave)

Create a master/slave link between the ctx object and the slave object. (on error return an EXCEPTION OBJECT)

Parameters:

[in]	ctx	the master vbmsgque object as PARENT without a CHILD
[in]	id	an integer used as unique identifer for the master/slave link
[in]	slave	the slave vbmsgque object as CLIENT-PARENT without a CHILD

ctx.SlaveDelete(Integer id)

Delete a slave-context from a former ctx.SlaveCreate(Integer id, MqS slave) command using the same identifier id. By default the slave-context object will be deleted if the master-context is deleted. In addition only the parent-slave-context can be deleted explicit. If id is invalid nothing will happen. (on error return an EXCEPTION OBJECT)

MqS ctx.SlaveGet(Integer id)

return the slave-context from the ctx.SlaveCreate(Integer id, MqS slave) using the same identifier id or NULL if the id is not valid or if ctx is not a master.

BUFFER OBJECT

The class MqBufferS object is created by MqBufferS ctx.ReadU() and id used to save a typeless temporary data item from the vbmsgque data package. The lifetime of the class MqBufferS is only the current callback up to the next read operation of the same context.

Dim buf As MqBufferS = ctx.ReadU()
If buf.Type == "C"c
  Console.WriteLine(buf.C)

TYPE IDENTIFIER

A vbmsgque buffer data package is type safe, this mean that every item has a type prefix and every TYPE ctx.Read[TYPE]() or TYPE val=buffer.[TYPE] have to match the previous ctx.Send[TYPE](TYPE value) with the same TYPE. One exception, the cast from and to string (TYPE=C) is allowed. The following type identifier's are available:

• Y : 1 byte signed character
• O : 1 byte boolean character using true for yes or true or false for no or wrong
• S : 2 byte signed short
• I : 4 byte signed integer
• W : 8 byte signed long long integer
• F : 4 byte float
• D : 8 byte double
• B : unsigned char array used for binary data
• C : string data using a \0 at the end
• L : list type able to hold a list of all items from above
• U : typeless buffer able to hold a single item from above

TYPE val=buffer.[TYPE]

return the data form the buffer using the TYPE IDENTIFIER. (on error return an EXCEPTION OBJECT)

Char buffer.Type

return the type as single character of the item stored in the buffer object.

EXCEPTION OBJECT

The exception object is used to transport a vbmsgque error using the class MqSException.

Try 
 i = ctx.ReadI()
Catch ex As Exception
 ErrorSet(ex)
 Console.WriteLine(ErrorGetText())
 ErrorReset()
End Try

String ex.txt or ctx.ErrorGetText()

return the error message from the error object.

Integer ex.num or ctx.ErrorGetNum()

return the error number from the error object. The number can be used as exit-code.

ctx.ErrorC(String prefix, Integer errnum, String message)

create a vbmsgque error object but do not raise the EXCEPTION OBJECT

ctx.ErrorSet(Exception ex)

convert a class MqSException into a vbmsgque conform error using the ctx object. This method is used to enable additional error processing capabilities from ctx.LinkDelete() or ctx.Exit() after the exception was caught and before the application exit or the object is deleted.

ctx.ErrorReset()

clear the vbmsgque error object.

ctx.ErrorRaise()

convert and raise an vbmsgque error object into a VB.NET EXCEPTION OBJECT.

ctx.ErrorPrint()

print an vbmsgque error object to stderr and clear the vbmsgque error

FILTER MODE

The filter mode is related to a special usage of the vbmsgque software called a command pipeline. A command pipeline is well known from the unix shell to link different commands together to act like a single application:

    command1 | command2 | command3

A command pipeline with vbmsgque queues is using the special token @ to link the commands together:

    msgcmd1 @ msgcmd2 @ msgcmd3

To create a vbmsgque filter use:

• IFilterFTR
• IFilterEOF

and to send data to the next filter command use:

• ctx.SendFTR([Integer timeout])
• ctx.SendEOF([Integer timeout])

PORTABILITY ISSUES

The library was tested on Linux, FreeBSD and WindowsXP using a x86_64, ppc processor or i686 processor from the VritualBox emulator.

EXAMPLES

1. create a server to multiplicate 2 double values and return the result

Imports System
Imports csmsgque

Public Module example
  Private Class mulserver
    Inherits MqS
    Implements IServerSetup
    Implements IFactory

    Public Function Factory() As csmsgque.MqS Implements csmsgque.IFactory.Call
      Return New mulserver()
    End Function

    Private Sub MMUL()
      SendSTART()
      SendD(ReadD() * ReadD())
      SendRETURN()
    End Sub

    Public Sub ServerSetup() Implements csmsgque.IServerSetup.Call
      ServiceCreate("MMUL", AddressOf MMUL)
    End Sub
  End Class

  Sub Main(ByVal args() As String)
    Dim srv As New mulserver()
    Try
      srv.ConfigSetName("MyMulServer")
      srv.LinkCreate(args)
      srv.ProcessEvent(MqS.WAIT.FOREVER)
    Catch ex As Exception
      srv.ErrorSet(ex)
    Finally
      srv.Exit()
    End Try
  End Sub
End Module

Start mulserver.exe using TCP port 7777 and create a thread for every incoming connection

> mono mulserver.exe --tcp --port 7777 --thread

2. in a client call the service from 1.

Imports System
Imports csmsgque

Public Module example
  Sub Main(ByVal args() As String)
    Dim ctx As New MqS()
    Try
      ctx.LinkCreate(args)
      ctx.SendSTART()
      ctx.SendD(3.67)
      ctx.SendD(22.3)
      ctx.SendEND_AND_WAIT("MMUL", 5)
      Console.WriteLine(ctx.ReadD())
    Catch ex As Exception
      ctx.ErrorSet(ex)
    Finally
      ctx.Exit()
    End Try
  End Sub
End Module

Use mulclient.exe to connect to mulserver.exe using TCP port 7777:

> mono mulclient.exe --tcp --port 7777

3. create a filter to wrap every column in a '<>' pair

Imports System
Imports csmsgque
Imports System.Collections.Generic

Public Module example
  Private Class manfilter
    Inherits MqS
    Implements IFilterFTR

    Public Sub FilterFTR() Implements csmsgque.IFilterFTR.Call
      SendSTART()
      While ReadItemExists()
        SendC("<" + ReadC() + ">")
      End While
      SendFTR(10)
    End Sub
  End Class

  Sub Main(ByVal args() As String)
    Dim srv As New manfilter()
    Try
      srv.ConfigSetName("ManFilter")
      srv.LinkCreate(args)
      srv.ProcessEvent(MqS.WAIT.FOREVER)
    Catch ex As Exception
      srv.ErrorSet(ex)
    Finally
      srv.Exit()
    End Try
  End Sub
End Module

Use manfilter.exe in a LibMsgque command pipeline:

> echo -e "1:2:3\na:b:c" | atool split -d : @ mono manfilter.exe @ atool join -d :

SEE ALSO

VB.NET, C: libmsgque(n), C++: ccmsgque(n), C#: csmsgque(n), JAVA: javamsgque(n), PYTHON: pymsgque(n), TCL: tclmsgque(n), PERL: perlmsgque(n)

KEYWORDS

VB.NET, unix, socket, message, msgque