Project  : Netrek Metaserver Overhaul 1995
Document : Design Specification
Revision : BX1
Dated    : 28-Sep-1995 
Approved : (nobody)

Design Specification (Metaserver)

  1. single process listening on UDP port and specific TCP ports, handles UDP updates inline, forks to handle TCP connections so as to avoid stall by remote end of TCP connection.

  2. optional logging of incoming packets to a file with timestamps,

  3. optional logging of TCP requests to a file with timestamps,

  4. use httpd logging format so that httpd log analysis/graphing programs may be used,

  5. use textual UDP packet format with newline delimiters,

  6. provide one TCP port that acts like a httpd using HTTP/1.0 and provides two minute updates to a netscape client.

Design Specification (Server)

  1. allow server owner to specify list of metaservers to be used, including host name, port number, and five update frequency maximums;

  2. when player count changes, or on expiry of zero players maximum frequency timer, send an update _provided_ the time interval since the last update will not exceed the new update frequency maximum.

Design Specification (Protocol)

  1. Server will send UDP packets to Metaserver(s) under control of the server owner,

  2. Metaserver will not reply. (If packet loss occurs enough to be a problem, then we don't want players to go and make it worse.)

  3. Update packet from server will be limited to maximum practical UDP packet size, no follow-on packet. If truncation will occur, server may omit player details.

  4. Update packet will be in ASCII text with newline (\n) delimiters. Any delimiters found in text to be sent will be removed.

  5. Update packet format will be as follows;

            packet
                address\ntype\nport\nplayers\nfree\ntmode\ncomment\n
                player\nplayer\n [...]
            
            player
                slot\nteam\nclass\nrank\nname\nlogin
            
            So, an example packet would be;
            
                freddo.frog.com.au\nB\n2592\n14\n2\ny\nFine\n
                0\nF\nCA\nEnsign\nClueless\nuser0@host.host\n
                1\nF\nCA\nEnsign\nCluemore\nuser1@host.host\n
            
            address         server textual ip address
            type            server type;
                            B=Bronco, P=Paradise, R=Practice, H=Hockey, etc.
            port            numeric port number on server
            players         count of active players
            free            count of free slots (negative if queue positive)
            tmode           if t-mode is on or not (y/n)
            comment         free format text set by server owner
            player          information on a player, see below
            
            slot            player slot number
            team            team identifier
            class           ship class name (two letter format)
            rank            rank number or text format
            name            character name
            login           login and host name
    

Program Structure (Metaserver)

The program will consist of multiple C source modules linked by function calls. Each module will have a set of public definitions defined by a header file. Modules will be responsible for certain actions or data structures.

                Network
                Logging
                Data
                Incoming
                Reports
                Management

Modules within Metaserver

Network Module
Mainline of code. Responsible for interacting with the host network services. Waits for packets from servers. Waits for connections via TCP.

                main()
                {
                    /* read configuration */
                    man_initialise();
                    
                    /* open log file */
                    log_open();
                    
                    /* open network */
                    ...
                    
                    /* process packets and requests */
                    while ( TRUE )
                    {
                        /* process UDP packets */
                        inc_handle();
                        
                        /* process TCP connections */
                        rep_handle();
                    }
                } /* main() */
Logging Module
Manages the logging of events within the metaserver. A logging mask defines the classes of events that are to be logged. The output is sent to standard output, and includes timestamps. The format to be used will match the httpd server log format.

                
                log_open()
                {
                }
                
                log_send(char *something)
                {
                }
                
                log_close()
                {
                }
Data Module
Manages an in-memory database of information about servers. The database may also reside in a disk file. The database consists of packets received from servers, along with an indexing structure. The indexing is to provide the ability to replace old packets from the same server, and generate output in the appropriate order to the user(s).

Only the main thread will update the database. The forked threads for handling TCP connections will read the database. Some form of synchronisation must be used to ensure that a reader does not see a view of the database that is inconsistent due to concurrent updates. The packet format is human readable. For this reason, the data format in the disk file should also be human readable. Some information about the packet does not arrive in the packet, and must be stored with the packet. This includes actual source IP address, and date time of receipt. The internal representation of this data is to be described by a packet structure.

                
struct packet
{
    char *buffer;       /* pointer to packet buffer  */
    size_t length;      /* byte length of buffer     */
    time_t received;    /* date time packet received */
    addr_t source;      /* ip address of packet source */
    struct server       /* pointers within packet buffer to server items */
    {
        char *address, *type, *port, *players, *free, *tmode, *comment;
    } server;
    struct player       /* pointers within packet buffer to player items */
    {
        char *slot, *team, *class, *rank, *name, *login;
    } player[MAXPLAYER];
};
The external data represenation in the disk file is to be identical to the packet format as it arrived from the server, with the following exceptions;

The start of each record will be identified by a special character followed by the textual date time received and the textual IP address.

The index is to consist of a set of single dimension array of pointers into the packet structures. A primary index array will provide ordering by IP address and port number. Secondary index arrays, if required, will provide ordering in a request specific manner.

(bsearch() and qsort() will be used)

                
                /* Sets up empty packet list pointer array */
                dat_initialise()
                {
                }
                
                /* Find a packet with the same key as a received packet */
                struct packet *dat_find(struct packet *packet)
                {
                }
                
                /* Insert a new packet into the list */
                struct packet *dat_insert(struct packet *packet)
                {
                }
                
                /* Replace an old packet with a new packet */
                void dat_replace(struct packet *old, *new)
                {
                }
                
                /* Expire and delete old packets */
                void dat_expire()
                {
                }
                
                /* Traverse list for client request */
                void dat_traverse
                (
                    int ((*compar)(const struct packet *,
                        const struct packet *)),
                    int ((*action)(const struct packet *)
                )
                {
                    /* allocate a private copy of the packet list */
                    auto struct packet *ours[MAXSERVERS];
                    
                    /* copy the real one into our copy */
                    memcpy ( ours, packets,
                        sizeof ( struct packet * ) * MAXSERVERS);
                    
                    /* sort the copy using the caller's comparison routine */
                    qsort ( ours, MAXSERVERS, sizeof ( struct packet * ),
                        compar );
                    
                    /* call the caller's action routine for every packet */
                    for ( i=0; i<MAXSERVERS; i++ )
                        if ( ours[i] != NULL ) action ( ours[i] );
                }
                
                /* Save packet list structure to file */
                int dat_save()
                {
                }
                
                /* Load packet list structure from file */
                int dat_load()
                {
                }
Incoming Module (Probably a part of main)
        
                /*
                **  Handle an incoming UDP packet, validate it,
                **  and update the server packet database.
                */
                inc_handle( char *buffer, size_t length, addr_t source )
                {
                    struct *packet new, old;
                    
                    new = inc_decode ( buffer, length, source );
                    if ( new == NULL ) return;
                    
                    old = dat_find ( new );
                    if ( old == NULL )
                        dat_insert ( new );
                    else
                        dat_update ( old, new );
                }

/*
**  Build struct of pointers into data fields within packet.  Return either
**  NULL for an invalid packet, or a pointer to the allocated struct.
*/
struct *packet inc_decode ( char *buffer, size_t length, addr_t source )
{
    struct packet packet, *returned;
    
    packet.buffer = buffer;
    packet.length = length;
    packet.source = source;
    packet.received = time ( NULL );
    
    packet.server.address = strtok ( buffer, delimiter );
    if ( packet.server.address == NULL ) return NULL;
    
    packet.server.type    = strtok ( NULL,   delimiter );
    if ( packet.server.type    == NULL ) return NULL;
    
    packet.server.port    = strtok ( NULL,   delimiter );
    if ( packet.server.port    == NULL ) return NULL;
    
    packet.server.players = strtok ( NULL,   delimiter );
    if ( packet.server.players == NULL ) return NULL;
    
    packet.server.free    = strtok ( NULL,   delimiter );
    if ( packet.server.free    == NULL ) return NULL;
    
    packet.server.tmode   = strtok ( NULL,   delimiter );
    if ( packet.server.tmode   == NULL ) return NULL;
    
    packet.server.comment = strtok ( NULL,   delimiter );
    if ( packet.server.comment == NULL ) return NULL;
    
    /* ... player stuff ... */
    
    returned = malloc ( sizeof ( struct packet ) );
    if ( returned == NULL ) return NULL;
    
    memcpy ( returned, packet, sizeof ( struct packet ) );
    return returned;
}
Reports Module
Responsible for handling of TCP request from mainline. Causes fork of process. Explores the packet list to generate output to client. On completion, closes the connection and exits.

                        
                rep_handle(int socket, int port)
                {
                }
Management Module
                /* Read configuration file */
                man_initialise()
                {
                }

--
James Cameron
Digital Equipment Corporation (Australia) Pty. Ltd. A.C.N. 000 446 800
(cameron@stl.dec.com)