Queue :
A Queues is basically a list  of objects which can be accessed 
in the same order in which they are put. This is called a FIFO
, for first in first out.

In caravan an object of type queue is proovided which is persistent
(meaninng it will not vanish if server is shut down) . The queues
are used to schedule tasks that need to be done sequencially. 
For example, take the sitauation where a lot of messages have to
be send from one server to many other sites. The task which creates
the message usaully does not want to wait till the message is dispatched to
the destination. Dispatching the message can be done in a separate
thread. This thread can  poll for any new messages 
and send them one by one. But in case there are no messages it still
needs to poll for new messages. This is an unneccassary wastage of
CPU resources. The queue mechanism along with the concept of
event handler provides an elegant solution for such situations.

By creating a queue and the corresponding eventhandler it is possible to
dispatch tasks that are sporadic. Whenever a item is inserted into
the queue, the corresponding evendhandler is triggered and starts dispatching
the queued task. The best thing is that the what is inserted into the
the queue and how the task is handled is totally programable.

A Queue is identified by its name and optionally a type. This is because
usually a particular task is handled depending on some 
information that distinguishes it from other tasks -- eaxample, each message is 
distinguished by the destination address though the logic of sending the message
is the same. So in way one can think of the queue name to identify
the method of dispatching the task and the queue type to be a parameter
to distinguish individual tasks.

A queue object is used to create the queues of any name and type.
A queue object is also used to access an existing queue and
manipulate and view its status.

syntax :


queue objectname(name,[type]);// type is optional

Where name and type are either strings or variable data.
Examples decaring a queue object x:

1.queue x(myq,yahoo); // the queue name is myq and type is yahoo

2.queue x(y(name),y(typ));// the name and type are variable

3.queue x(myq);// the name is qname and applicable to all types

4.queue x(myq,*);// the name is qname and applicable to all types


Queue properties:

total : total items in queue
name : name of the queue

queueid : identifier for the queue entry

Since queue is a list, each entry is identified by its queueid

item : content of the queue entry
This is the content of the entry, it can be a string or a file

type : type of the queue entry

priority: priority of the queue entry

In caravan queues the item is not only in a fifo but also 
prioritised, this is a number and lower the value higher 
the priority, default is '0'

resume:

wakes up the corresponding eventhandler that had supended itself
using the 'suspend' statement.

eventhandler :

Event handler is a piece of caravan code that is linked to queue
very closely. It is created by adding a eventhandler statement
in a caravan template which is written to dispatch the queueed tasks.

The eventhandler statement is almost identical to the queue
declaration syntax.

syntax:
eventhandler(myq);// handle all tasks with name myq
eventhandler(myq,*);//handle all tasks with name myq
eventhandler(myq,?);// handle all tasks with name myq,
                   // all types, concurrently for each type
      
eventhandler(myq,mtyp)


We  will illustrate the use of queue and eventhandler through an example.

=========================================================================

Assume we have server recieving information from post operations coming from users
and other servers. The job of the server is to store this data in folders depending 
on the catagory information in the form. The form consists of 'message', 'catagory'' and
'forwardto' . 

The message is  some file of unknown size, which needs to be saved 
to some folder depending on the 'catagory' and the 'forwardto' are a list of servers
which should also be updated with this data. 

We have a data base called messages which contain three tables 

1. files:
    Holds the data on the local serrver. This is updated every time a form is
    accepted by our server.
2. catagories: holds category and folder information
    Contains two fields
    catagory : a string to denote a the category -- say news, graphics, messages
    folders : a list of folders in which this catagory has to be stored
3. servers:  Contains server information.
    servername, and ipaddress and port


The task is to accept the form, update our table 'messages.files', save the message into its respective
folder and forward it to the indicated servers.

We can write the code to in such a way that all the above are done in one process. The drawback
is that each thread will take an undefined amount of time and if there are many simultaneous requests
everything will slow down to a crawl. The code for doing it this way is given below.

Code for 'accept_message.html'

<caravan>
// this code is executed when a form is posted to caravan.

// first put the data into the 'files' table
table store=messages.files
store(message)=form(message)
store(category)=form(category)
store(forwardto)=form(forwardto)
store(insert)

// Now save the message in our folder

table cat=messages.categories

select from cat where category={category}
if cat(selected)> "0" ;// yes we have identified soe categories
  loop cloop (cat(selected)) ;// there can be multiple folders in which we have to save
    folder df=cat(folder)
    df(file)=form(message)
    cat(nextrecord)
  repeat cloop
endif

// now send to other servers

loop dloop (form(forwardto(00)))
  table servers=messages.servers
  var sql
  sql(servname)=form(forwardto(dloop(count)))
  select from servers where servername={servname}
  if servers(selected)="1";// ideally only there should only be one entry per server
     form myform
     myform(_server)=servers(ipaddress)
     myform(_port)=server(port)
     myform(_url)="accept_message.html"
     myform(message)=form(message)
     myform(category)=form(category)
     myform(post)
  endif
repeat dloop

// now send reply to the client who posted

"OK : recvd message"
</caravan>


The above code is a simplified form of a communication process. We can see that each part of the 
code can take much time to complete and the sender of the original form has to wait an undefined amount of time 
to get the response 'Ok : recvd message'.

Now we will split the task into three parts :

1. Receive data: 'accept_message.html'

<caravan>
// this code is executed when a form is posted to caravan.

// first put the data into the 'files' table
table store=messages.files
store(message)=form(message)
store(category)=form(category)
store(forwardto)=form(forwardto)
store(insert)

// Now queue the task

queue saveq("save") ;// a queue is created for saving the data into folders
saveq(item)=store(recordno)
queue sendq("send") ;// another queue is created to send the data to downstream servers
sendq(type)=store(forwardto)
sendq(item)=store(recordno)
"OK : recvd message" ;// reply ok to sender
</caravan>


In the above code all parts of the task is not completed, we have just updated our database and replied 'ok'
to sender. The other parts of the task are done by eventhandlers. We have used the queue object to create
two events which will trigger the respctive eventhandlers.

Event handler for saving the message in local file system:'save.html'

<caravan>
_eventhandler ("save");// this says that the follwing code is to be executed when a 'save' event is generated.

// there is a queue object '_event' which is already created when this eventhandler is triggered which
// is equivalent to the decaration:
// queue _event("save")

table store=messages.files
// retrieve the item
store(recordno)=_event(item) ;// now we are pointing to the record which contains the message
// rest of the code is similar to what we did before except that now we dont have the original
// object 'form' which was posted to us -- we have to get the message from our table
table cat=messages.categories
var sql
sql(category)=store(category)
select from cat where category={category}
if cat(selected)> "0" ;// yes we have identified some categories
  loop cloop (cat(selected)) ;// there can be multiple folders in which we have to save
    folder df=cat(folder)
    df(file)=store(message)
    cat(nextrecord)
  repeat cloop
endif
_event(deleteitem);// done with this item, delete it
// handler will restart with next item if any or gets suspended till another event occurs
// if no event occurs for a long time the thread dies and frees up the resources
</caravan>



For every message received the above code is executed and the message is saved to its
required folder without delaying our client. Note that by not specifying any 'type' for the
eventhandler we are ensuring that the messages are processed in the order they have landed,
in a sequential manner.

Now we need a event handler to send the messages to other servers. Here we will demostrate
a slightly different way of handling this task. Since a single message may have to go to
multiple servers we will need to send them concurrently -- else all servers may have to face
unacceptable delay.





Event handler for sending the message to a downstream server:'send.html'
====================================================================
<caravan>
_eventhandler(send,?);// create an instance of this eventhandler for each type
// there is a queue object '_event' which is already created when this eventhandler is triggered which
// is quivalenent to the decaration:
// queue _event("send",servername)
// the 'type' here is the servername

  table store=messages.files
// retrieve the item
  store(recordno)=_event(item) ;// now we are pointing to the record which contains the message
  table store=messages.files
  store(recordno)=_event(item)
  table servers=messages.servers
  var sql
  sql(servname)=_event(type)
  select from servers where servername={servname}
  if servers(selected)="1";// ideally only there should only be one entry per server
     form myform
     myform(_server)=servers(ipaddress)
     myform(_port)=server(port)
     myform(_url)="accept_message.html"
     myform(message)=form(message)
     myform(category)=form(category)
     myform(post)
     if _reply(error);// if there was error set a delay of 300 seconds before trying again.
       _event(wait)="300"
       goto skipdelete;// dont delete it now
     endif
  endif
  _event(deleteitem);// delete this item from 'send' queue
 label skipdelete
</caravan>


The above code illustrates how caravan can use the multithreading features of the OS to
speed up the response time and performance of your application. This way the message is send to one 
server per thread. So many threads can become active at once and finish the job much faster.

Also see :
Tables, form object and loop statements.