Performance Comparison

Table Of Contents

• Introduction
• Summary
• Detail Setup Information:

  C	C++	C#	Java	Perl	Python	Tcl	VB.NET	PHP
  pipe	pipe	pipe	pipe	pipe	pipe	pipe	pipe	na.
  uds	uds	uds	uds	uds	uds	uds	uds	na.
  tcp	tcp	tcp	tcp	tcp	tcp	tcp	tcp	na.

Introduction

The performance is measured with the tool client and a programming language specific server in the subdirectory performance.

• performance/README

The performance test is separated in a test with thread support and without. A test with thread support is performed with a thread enabled base language and a thread enabled libmsgque library.

The performance-tests are divided into 2 major test-groups:

1. test the transaction performance (MqSendEND, MqSendEND_AND_CALLBACK, MqSendEND_AND_WAIT)
2. test the context creation performance (parent/child create/delete)

The server-setup is divided into 2 major setup-groups:

1. pipe , the server is started by the client using the client @ server syntax
2. tcp or uds , the server is started as a stand-alone server and the client is connecting to this server.

The following information is important:

• the test was done on Linux using the following CPU
  • Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz
• for C# the mono tools were used
• for JAVA the sun tools were used
• parent-create/delete using pipe or uds-spawn or tcp-spawn
  • create: An entire new process is created and the startup time of the new process is included.
    • process-create + language-interpreter-create + child-create
  • delete: The whole process is delete.
    • child-delete + language-interpreter-delete + process-delete
• parent-create using tcl and tcp-thread or uds-thread
  • Tcl allow multiple interpreter per process.
  • For every new parent-context one new thread is created.
  • For every new thread one new tcl interpreter is created.
• parent-create using java and tcp-thread or uds-thread
  • Java allow only one JVM per process.
  • For every new parent-context a new thread is created.
  • For every new thread the initial JVM is attached.
• parent-create using C# and tcp-thread or uds-thread
  • Using the same "design" as for JAVA was done
• tcl is using a different interpreter depending on thread used or not:
  • thread disabled: pipe, spawn , fork
  • thread enabled: thread
• the results are time-value per transaction -> as bigger, as slower
• PHP was not able to provide useful API documentation for Zend Engine 2.0
  • this is interesting because Zend is the commercial part of the PHP project
• VB.NET is using the CLR dll from C#
  • one goal was to prove if the CLR language compatibility works -> it does !!

Summary

    Tests from:       |   send     send     send    parent    parent   child    child  
 2009-11-09 14:30:30  |   END    CALLBACK   WAIT    create    delete   create   delete 
 -------------------  | -------- -------- -------- --------- -------- -------- --------

 pipe:
    C                 |     3.42     9.15    14.49    744.77    94.11    70.18    23.82
    C++               |     4.80     9.40    14.82   1313.18   184.30    79.33    24.53
    C#                |     4.10    10.85    16.12  30115.08  2579.47   127.86    24.16
    Java              |     5.46    13.99    19.62 108709.38  1762.69   157.10    24.44
    Perl              |     8.35    17.64    23.94 203871.66   549.07   113.83    23.67
    Python            |     5.25    16.97    22.72  29460.66  2226.39   123.88    24.90
    Tcl               |     6.87    16.04    22.39  10182.13   269.96   232.77    24.12
    VB.NET            |     3.99    10.92    16.13  29939.22  3076.39   153.56    24.59
    PHP               |      na.      na.      na.       na.      na.      na.      na.

 tcp_fork:
    C                 |     2.87     6.82    24.20    361.46   167.53    80.15    34.82
    C++               |     2.74     7.58    24.69    422.78   244.70    83.00    33.55
    C#                |      na.      na.      na.       na.      na.      na.      na.
    Java              |      na.      na.      na.       na.      na.      na.      na.
    Perl              |     8.48    13.99    33.01    962.26   874.05   124.78    33.09
    Python            |     4.98    13.40    32.34    955.55  3910.88   143.57    35.44
    Tcl               |     6.67    12.65    31.46    936.52   369.12   249.61    33.23
    VB.NET            |      na.      na.      na.       na.      na.      na.      na.
    PHP               |      na.      na.      na.       na.      na.      na.      na.

 tcp_thread:
    C                 |     3.10     7.16    24.61    215.28   113.75    80.32    34.36
    C++               |     3.52     7.93    25.37    207.05   113.30    85.08    33.19
    C#                |     3.44     8.34    26.43    362.29   546.10   201.66    33.71
    Java              |     5.59    11.13    29.05    369.42   129.03   158.25    33.16
    Perl              |    10.28    17.18    36.30   5399.62   290.68   138.53    34.58
    Python            |      na.      na.      na.       na.      na.      na.      na.
    Tcl               |     6.44    13.35    32.15   8486.40   535.58   246.47    32.79
    VB.NET            |     4.09     8.50    26.19    403.99   453.23   233.26    34.73
    PHP               |      na.      na.      na.       na.      na.      na.      na.

 tcp_spawn:
    C                 |     2.80     6.47    25.42    785.17   157.26    80.34    34.04
    C++               |     3.25     8.34    24.58   1354.51   208.97    84.05    33.69
    C#                |     4.22     8.50    26.14  29117.51  2602.97   137.87    33.21
    Java              |     5.57    10.59    28.55 106374.06  1713.39   176.35    33.43
    Perl              |     8.28    13.68    33.05 203311.67   557.60   126.64    36.65
    Python            |     4.88    13.48    32.53  29840.92  2216.77   135.81    32.91
    Tcl               |     5.95    12.77    31.80  10198.18   269.98   246.43    34.01
    VB.NET            |     3.47     8.16    26.10  29999.08  2894.43   167.23    34.56
    PHP               |      na.      na.      na.       na.      na.      na.      na.

 uds_fork:
    C                 |     3.36     9.53    14.79    294.96   119.75    69.97    23.76
    C++               |     3.38     9.71    14.82    325.15   210.83    77.54    23.72
    C#                |      na.      na.      na.       na.      na.      na.      na.
    Java              |      na.      na.      na.       na.      na.      na.      na.
    Perl              |     8.28    17.99    24.22    907.95   802.27   117.31    23.82
    Python            |     5.33    17.02    22.87    867.93  4442.21   129.51    24.96
    Tcl               |     6.52    16.33    22.37    860.80   320.76   235.97    24.38
    VB.NET            |      na.      na.      na.       na.      na.      na.      na.
    PHP               |      na.      na.      na.       na.      na.      na.      na.

 uds_thread:
    C                 |     3.65     9.77    15.08    140.56    77.82    74.89    25.37
    C++               |     3.76    10.01    15.34    143.65    64.28    75.06    24.63
    C#                |     4.11    11.02    16.22    286.66   417.40   193.38    23.99
    Java              |     5.56    14.18    19.72    323.03    85.78   151.76    24.28
    Perl              |    10.85    20.88    27.61   5287.58   293.73   130.69    24.60
    Python            |      na.      na.      na.       na.      na.      na.      na.
    Tcl               |     6.54    16.79    23.08   8413.21   506.79   235.65    24.43
    VB.NET            |     5.13    11.01    16.22    314.30   375.16   222.21    24.74
    PHP               |      na.      na.      na.       na.      na.      na.      na.

 uds_spawn:
    C                 |     4.40     9.40    14.67    735.16   115.10    71.23    24.28
    C++               |     4.81     9.63    15.07   1306.11   196.43    79.02    25.08
    C#                |     4.00    11.16    16.49  29787.25  2785.36   130.82    24.64
    Java              |     5.65    14.41    20.11 105689.31  1718.05   154.56    25.42
    Perl              |     8.69    18.25    24.56 204062.47   531.36   124.61    24.98
    Python            |     5.26    16.87    22.71  29482.46  2293.63   129.64    24.32
    Tcl               |     6.43    16.26    22.43  10070.69   253.78   237.41    24.10
    VB.NET            |     4.09    11.26    16.48  29828.93  2896.14   155.75    24.69
    PHP               |      na.      na.      na.       na.      na.      na.      na.

Detail C Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ nothread/libmsgque-3.8/tests/server 
  

Using UDS

• Using UDS and Spawn

  > nothread/libmsgque-3.8/tests/server  --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Fork

  > nothread/libmsgque-3.8/tests/server  --uds --file ./socket.uds --fork
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Thread

  > thread/libmsgque-3.8/tests/server  --uds --file ./socket.uds --thread
  > thread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Spawn

  > nothread/libmsgque-3.8/tests/server  --tcp --host localhost --port 7779 --spawn
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7779
  

• Using TCP and Fork

  > nothread/libmsgque-3.8/tests/server  --tcp --host localhost --port 7777 --fork
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7777
  

• Using TCP and Thread

  > thread/libmsgque-3.8/tests/server  --tcp --host localhost --port 7778 --thread
  > thread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7778
  

Detail C++ Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ nothread/libmsgque-3.8/tests/ccserver
  

Using UDS

• Using UDS and Spawn

  > nothread/libmsgque-3.8/tests/ccserver --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Fork

  > nothread/libmsgque-3.8/tests/ccserver --uds --file ./socket.uds --fork
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Thread

  > thread/libmsgque-3.8/tests/ccserver --uds --file ./socket.uds --thread
  > thread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Spawn

  > nothread/libmsgque-3.8/tests/ccserver --tcp --host localhost --port 7782 --spawn
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7782
  

• Using TCP and Fork

  > nothread/libmsgque-3.8/tests/ccserver --tcp --host localhost --port 7780 --fork
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7780
  

• Using TCP and Thread

  > thread/libmsgque-3.8/tests/ccserver --tcp --host localhost --port 7781 --thread
  > thread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7781
  

Detail C# Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ mono nothread/libmsgque-3.8/tests/csserver.exe
  

Using UDS

• Using UDS and Spawn

  > mono nothread/libmsgque-3.8/tests/csserver.exe --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Thread

  > mono thread/libmsgque-3.8/tests/csserver.exe --uds --file ./socket.uds --thread
  > thread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Spawn

  > mono nothread/libmsgque-3.8/tests/csserver.exe --tcp --host localhost --port 7791 --spawn
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7791
  

• Using TCP and Thread

  > mono thread/libmsgque-3.8/tests/csserver.exe --tcp --host localhost --port 7790 --thread
  > thread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7790
  

Detail Java Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ java example.Server
  

Using UDS

• Using UDS and Spawn

  > java example.Server --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Thread

  > java example.Server --uds --file ./socket.uds --thread
  > thread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Spawn

  > java example.Server --tcp --host localhost --port 7787 --spawn
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7787
  

• Using TCP and Thread

  > java example.Server --tcp --host localhost --port 7786 --thread
  > thread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7786
  

Detail Perl Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ perl nothread/libmsgque-3.8/tests/server.pl
  

Using UDS

• Using UDS and Fork

  > perl nothread/libmsgque-3.8/tests/server.pl --uds --file ./socket.uds --fork
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Spawn

  > perl nothread/libmsgque-3.8/tests/server.pl --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Thread

  > perl thread/libmsgque-3.8/tests/server.pl --uds --file ./socket.uds --thread
  > thread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Fork

  > perl nothread/libmsgque-3.8/tests/server.pl --tcp --host localhost --port 7794 --fork
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7794
  

• Using TCP and Spawn

  > perl nothread/libmsgque-3.8/tests/server.pl --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using TCP and Thread

  > perl thread/libmsgque-3.8/tests/server.pl --tcp --host localhost --port 7795 --thread
  > thread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7795
  

Detail Python Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ python nothread/libmsgque-3.8/tests/server.py
  

Using UDS

• Using UDS and Spawn

  > python nothread/libmsgque-3.8/tests/server.py --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Fork

  > python nothread/libmsgque-3.8/tests/server.py --uds --file ./socket.uds --fork
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Spawn

  > python nothread/libmsgque-3.8/tests/server.py --tcp --host localhost --port 7789 --spawn
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7789
  

• Using TCP and Fork

  > python nothread/libmsgque-3.8/tests/server.py --tcp --host localhost --port 7788 --fork
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7788
  

Detail Tcl Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ tclsh nothread/libmsgque-3.8/tests/server.tcl
  

Using UDS

• Using UDS and Spawn

  > tclsh nothread/libmsgque-3.8/tests/server.tcl --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Fork

  > tclsh nothread/libmsgque-3.8/tests/server.tcl --uds --file ./socket.uds --fork
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Thread

  > tclsh thread/libmsgque-3.8/tests/server.tcl --uds --file ./socket.uds --thread
  > thread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Spawn

  > tclsh nothread/libmsgque-3.8/tests/server.tcl --tcp --host localhost --port 7785 --spawn
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7785
  

• Using TCP and Fork

  > tclsh nothread/libmsgque-3.8/tests/server.tcl --tcp --host localhost --port 7783 --fork
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7783
  

• Using TCP and Thread

  > tclsh thread/libmsgque-3.8/tests/server.tcl --tcp --host localhost --port 7784 --thread
  > thread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7784
  

Detail VB.NET Setup

Using PIPE

• Using PIPE

  > nothread/libmsgque-3.8/tests/client --all @ mono nothread/libmsgque-3.8/tests/vbserver.exe
  

Using UDS

• Using UDS and Spawn

  > mono nothread/libmsgque-3.8/tests/vbserver.exe --uds --file ./socket.uds --spawn
  > nothread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

• Using UDS and Thread

  > mono thread/libmsgque-3.8/tests/vbserver.exe --uds --file ./socket.uds --thread
  > thread/libmsgque-3.8/tests/client --all --uds --file ./socket.uds
  

Using TCP

• Using TCP and Spawn

  > mono nothread/libmsgque-3.8/tests/vbserver.exe --tcp --host localhost --port 7793 --spawn
  > nothread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7793
  

• Using TCP and Thread

  > mono thread/libmsgque-3.8/tests/vbserver.exe --tcp --host localhost --port 7792 --thread
  > thread/libmsgque-3.8/tests/client --all --tcp --host localhost --port 7792