CIS 650 Final Project
Network Based Message Queue - Issues.
Amithaba Roy,
Sameer Shende,
Andreas Philipp,
Guoqing Weng
Abstract
The project will involve implementation of a message queue used for
interprocess communication between programs separated by a network. This
commuication construct will be similar in functionality to the System V
Unix IPC mechanism of message queues except that it would be extended to
operate over a network of nodes. It would include the design of the
software architecture, the analysis of the issues involved, the description of
the functionality, implementation of the user level library routines at the
client and server end, the design of the router program and its implementation,
and the performance monitor.
Introduction
Message queues provide a communication construct to allow programs to
communicate without establishing a point to point communication channel.
In using a message queue, a number of programs collectively participate in
the communication by sending messages over a common channel (like a bus) and
by tagging each message by a message type which is used by the peer process
to extract the message and send a reply message. The channel may be used for
full duplex form of communication. The advantage of such a scheme in
implementing client server systems is that the number of servers and clients
can be varied dynamically and be used efficiently in the implementation of
iterative servers (servers which are stateless and service multiple clients).
The obvious advantage is that here the number of servers can be varied according
to the load on the system (in terms of the number of messages).
In traditional messaging systems (like Unix IPC), a message queue typically
would be used by the same kind of server processes (as its not easy to determine
the number of messages of a particular type on the queue at a point in time) to
implement dynamic control over the number of server processes. This is
a restriction which leads to a homogeneous server access over a single message
queue.
The aim of the project would be to provide a means of communication between a
group of processes (which may or maynot be on the same node) borrowing ideas
from the message queue paradigm, but implementing it so that it can be used
effectively in distributed computing where the performance of the overall
system can be controlled by varying the number of server processes.
The system to be implemented would provide the network based queue
implementation which could be used by heterogeneous set of servers (as opposed
to the homogeneous case in the Unix implementation) by a set of tasks which
are geographically distributed.
Specifications, System Overview and Issues
The issues involved in designing the system are as follows :
- The system must provide a typed message delivery of messages with the
message type as a parameter that could be specified by the user (different
types of servers would use different types for specifying incoming messages in
case they wish to co-exist on the same distributed queue).
- Multiple servers should be able to specify the same type of the incoming
message (sent by one or more clients) and a message should reach only one server
in the queue.
- Sequencing of messages should be guaranteed on a per server basis - i.e.,
if a client manages to send two messages to the same server, then they should
not appear as out of sequence to the server process. However, two messages sent
by the client (having the same message type) could indeed be delivered to two
servers. So, essentially, to preserve the state of the server the application
would need to do this explicitly (although this would typically be used by
stateless servers).
- The number of clients connecting with the message queue should be scalable
and it should be possible to increase this number to a value which cannot
typically be used by an instance of an iterative server (on account of the
limited number of open "socket" descriptors - a restriction that the operating
system imposes - but can be tuned).
- Full duplex mode of communication should be provided by the system to
ensure two way communication between the clients and the server processes.
- The API for the message queue implementation should be compatible or similar
to that used by the existing Unix IPC message queue API so that the applications
that are currently using this mode of communication but are restricted to the
same node (and hence scalability means adding extra hardware to the same node),
can now be ported easily to operate over a network on independent nodes.
- Traditional message queue implementations have a problem with operating
within certain system boundaries (of the message size, and total no. of bytes
in the system) and use Flow control to jam the sender in case of large number
of messages on the message queue. This can lead to deadlocks (in case the
clients cannot put a message due to reaching the limit and the server process
- or the message consumer - cannot reply back to the client as it is
a producer of reply messages leading to a potential deadlock situation on a
two way communication channel when the number of messages reaches an upper
bound).
The distributed queue implementation should be free of this deadlock situation
and should be able to operate without such restrictions.
- Logging of messages should be an option that should be available to the
users - this is currently not implemented in the Unix Sys V IPC mechanisms.
- Performance monitors should be able to keep track of the state of the
message queue system and should be able to provide a means of querying on
the set of parameters dynamically. The nature of the queries should be
decided while designing the system but mechanisms should exist to extend this
operation later with minimal changes to the system.
- A reliable transport mechanism should be used to provide guaranteed delivery
of the message across the network (by using a connection oriented TCP/IP
service) within the system.
This gives the overview of the specifications and the issues of implementing the
network based message queue system.
Sameer Shende
<sameer@cs.uoregon.edu>
Last modified: Mon Oct 14 17:59:56 PDT