Logo Search packages:      
Sourcecode: djvulibre version File versions  Download package

DataPool Class Reference

#include <DataPool.h>

Inheritance diagram for DataPool:


List of all members.

Detailed Description

Thread safe data storage. The purpose of DataPool# is to provide a uniform interface for accessing data from decoding routines running in a multi-threaded environment. Depending on the mode of operation it may contain the actual data, may be connected to another DataPool# or may be mapped to a file. Regardless of the mode, the class returns data in a thread-safe way, blocking reading threads if there is no data of interest available. This blocking is especially useful in the networking environment (plugin) when there is a running decoding thread, which wants to start decoding as soon as there is just one byte available blocking if necessary.

Access to data in a DataPool# may be direct (Using {get_data}() function) or sequential (See {get_stream}() function).

If the DataPool# is not connected to anything, that is it contains some real data, this data can be added to it by means of two {add_data}() functions. One of them adds data sequentially maintaining the offset of the last block of data added by it. The other can store data anywhere. Thus it's important to realize, that there may be "white spots" in the data storage.

There is also a way to test if data is available for some given data range (See {has_data}()). In addition to this mechanism, there are so-called {trigger callbacks}, which are called, when there is all data available for a given data range.

Let us consider all modes of operation in details:

{enumerate} { Not connected DataPool#}. In this mode the DataPool# contains some real data. As mentioned above, it may be added by means of two functions {add_data}() operating independent of each other and allowing to add data sequentially and directly to any place of data storage. It's important to call function {set_eof}() after all data has been added.

Functions like {get_data}() or {get_stream}() can be used to obtain direct or sequential access to the data. As long as {is_eof}() is FALSE#, DataPool# will block every reader, which is trying to read unavailable data until it really becomes available. But as soon as {is_eof}() is TRUE#, any attempt to read non-existing data will read #0# bytes.

Taking into account the fact, that DataPool# was designed to store DjVu files, which are in IFF formats, it becomes possible to predict the size of the DataPool# as soon as the first #32# bytes have been added. This is invaluable for estimating download progress. See function {get_length}() for details. If this estimate fails (which means, that stored data is not in IFF format), {get_length}() returns #-1#.

Triggers may be added and removed by means of {add_trigger}() and {del_trigger}() functions. {add_trigger}() takes a data range. As soon as all data in that data range is available, the trigger callback will be called.

All trigger callbacks will be called when EOF# condition has been set.

{ DataPool# connected to another DataPool#}. In this {slave} mode you can map a given DataPool# to any offsets range inside another DataPool#. You can connect the slave DataPool# even if there is no data in the master DataPool#. Any {get_data}() request will be forwarded to the master DataPool#, and it will be responsible for blocking readers trying to access unavailable data.

The usage of {add_data}() functions is prohibited for connected DataPool::s.

The offsets range used to map a slave DataPool# can be fully specified (both start offset and length are positive numbers) or partially specified (the length is negative). In this mode the slave DataPool# is assumed to extend up to the end of the master DataPool#.

Triggers may be used with slave DataPool::s as well as with the master ones.

Calling {stop}() function of a slave will stop only the slave (and any other slave connected to it), but not the master.

{set_eof}() function is meaningless for slaves. They obtain the ByteStream::EndOfFile# status from their master.

Depending on the offsets range passed to the constructor, {get_length}() returns different values. If the length passed to the constructor was positive, then it is returned by {get_length}() all the time. Otherwise the value returned is either #-1# if master's length is still unknown (it didn't manage to parse IFF data yet) or it is calculated as masters_length-slave_start#.

{ DataPool# connected to a file}. This mode is quite similar to the case, when the DataPool# is connected to another DataPool#. Similarly, the DataPool# stores no data inside. It just forwards all {get_data}() requests to the underlying source (a file in this case). Thus these requests will never block the reader. But they may return #0# if there is no data available at the requested offset.

The usage of {add_data}() functions is meaningless and is prohibited.

{is_eof}() function always returns TRUE#. Thus {set_eof}() us meaningless and does nothing.

{get_length}() function always returns the file size.

Calling {stop}() function will stop this DataPool# and any other slave connected to it.

Trigger callbacks passed through {add_trigger}() function are called immediately.

This mode is useful to read and decode DjVu files without reading and storing them in full in memory. {enumerate}

Definition at line 227 of file DataPool.h.


static GP< DataPoolcreate (const GURL &url, int start=0, int length=-1)
static GP< DataPoolcreate (const GP< DataPool > &master_pool, int start=0, int length=-1)
static GP< DataPoolcreate (const GP< ByteStream > &str)
static GP< DataPoolcreate (void)
void connect (const GURL &url, int start=0, int length=-1)
void connect (const GP< DataPool > &master_pool, int start=0, int length=-1)
virtual ~DataPool ()

Public Member Functions

void clear_stream (const bool release=true)
int get_count (void) const
void load_file (void)
void stop (bool only_blocked=false)
Adding data.
Please note, that these functions are for not connected DataPool::s only. You can not add data to a DataPool#, which is connected to another DataPool# or to a file.

void add_data (const void *buffer, int offset, int size)
void add_data (const void *buffer, int size)
void set_eof (void)
Trigger callbacks.
{Trigger callbacks} are special callbacks called when all data for the given range of offsets has been made available. Since reading unavailable data may result in a thread block, which may be bad, the usage of {trigger callbacks} appears to be a convenient way to signal availability of data.

You can add a trigger callback in two ways: {enumerate} By specifying a range. This is the most general case By providing just one {threshold}. In this case the range is assumed to start from offset ZERO# and last for {threshold}+1 bytes. {enumerate}

void add_trigger (int thresh, void(*callback)(void *), void *cl_data)
void add_trigger (int start, int length, void(*callback)(void *), void *cl_data)
void del_trigger (void(*callback)(void *), void *cl_data)
Accessing data.
These functions provide direct and sequential access to the data of the DataPool#. If the DataPool# is not connected (contains some real data) then it handles the requests itself. Otherwise they are forwarded to the master DataPool# or the file.

int get_data (void *buffer, int offset, int size)
GP< ByteStreamget_stream (void)
State querying functions.

int get_length (void) const
int get_size (void) const
bool has_data (int start, int length)
bool is_connected (void) const
bool is_eof (void) const
Files #"DataPool.h"# and #"DataPool.cpp"# implement classes {DataPool} and {DataRange} used by DjVu decoder to access data.

The main goal of class {DataPool} is to provide concurrent access to the same data from many threads with a possibility to add data from yet another thread. It is especially important in the case of the Netscape plugin when data is not immediately available, but decoding should be started as soon as possible. In this situation it is vital to provide transparent access to the data from many threads possibly blocking readers that try to access information that has not been received yet.

When the data is local though, it can be accessed directly using standard IO mechanism. To provide a uniform interface for decoding routines, {DataPool} supports file mode as well.

Thread safe data storage

Andrei Erofeev <eaf@geocities.com>
DataPool.h,v 1.13 2007/05/19 03:07:33 leonb Exp

bool simple_compare (DataPool &pool) const

Static Public Member Functions

static void close_all (void)
static void load_file (const GURL &url)

Static Public Attributes

static const char * Stop = ERR_MSG("STOP")

Protected Attributes

volatile int count
 The reference counter.

Private Member Functions

void added_data (const int offset, const int size)
void analyze_iff (void)
void check_triggers (void)
int get_data (void *buffer, int offset, int size, int level)
int get_size (int start, int length) const
void init (void)
void restart_readers (void)
void trigger_cb (void)
void wait_for_data (const GP< Reader > &reader)
void wake_up_all_readers (void)

Static Private Member Functions

static void static_trigger_cb (void *)

Private Attributes

Counter * active_readers
int add_at
BlockList * block_list
GCriticalSection class_stream_lock
GP< ByteStreamdata
GCriticalSection data_lock
bool eof_flag
GP< OpenFiles_Filefstream
GURL furl
int length
GP< DataPoolpool
GPList< Reader > readers_list
GCriticalSection readers_lock
int start
bool stop_blocked_flag
bool stop_flag
GCriticalSection trigger_lock
GPList< Trigger > triggers_list
GCriticalSection triggers_lock


class FCPools


class  BlockList
class  Counter
class  Incrementor
class  OpenFiles
class  OpenFiles_File
class  Reader
class  Trigger

The documentation for this class was generated from the following files:

Generated by  Doxygen 1.6.0   Back to index