Here is what you do to start experimenting with the UnQLite database engine without having to do a lot of tedious reading and configuration:



download The Code

Get a copy of the last public release of UnQLite (A single 1.8 MB C file). Visit the download page for additional information.

Write Programs That Use UnQLite

The principal task of a database engine is to store and retrieve records. UnQLite support both structured and raw database record storage.





Structured data storage is presented to clients via the Document store interface to UnQLite. The document store itself is used to store JSON docs (i.e. Objects, Arrays, Strings, etc.) in the database and is powered by the Jx9 programming language. Jx9 is an embeddable scripting language also called extension language designed to support general procedural programming with data description facilities. Jx9 is a Turing-Complete, dynamically typed programming language based on JSON and implemented as a library in the UnQLite core. You can find more on Jx9 at the Introduction to Jx9 page.





Raw data storage is presented to clients via the Key/Value store interfaces.

UnQLite is a standard key/value store similar to Berkeley DB, Tokyo Cabinet, LevelDB, etc. but with a rich feature set including support for transactions (ACID), concurrent reader, etc. Under the KV store,

both keys and values are treated as simple arrays of bytes, so content can be anything from ASCII strings, binary blob and even disk files.

Introduction to the Key/Value Store C/C++ Interfaces

By far, the UnQLite Key/Value store layer is the simplest, yet the most flexible since it does not impose any structure or schema to the database records.

The Key/Value store layer is presented to clients via a set of interfaces, these includes: unqlite_kv_store(), unqlite_kv_append(), unqlite_kv_fetch_callback(), unqlite_kv_append_fmt(), and many more.

Below is a simple C program that demonstrates how to use the Key/Value C/C++ interface to UnQLite.

#include <unqlite.h> int i,rc; unqlite *pDb;

// Open our database; "test.db" ,UNQLITE_OPEN_CREATE); rc = unqlite_open (&pDb,,UNQLITE_OPEN_CREATE); if ( rc != UNQLITE_OK ){ return ; }

// Store some records rc = unqlite_kv_store(pDb, "test" , -1 , "Hello World" , 11); //test => 'Hello World' if ( rc != UNQLITE_OK ){ //Insertion fail, Hande error (See below)

return ; } // A small formatted string

rc = unqlite_kv_store_fmt(pDb, "date" ,-1, "Current date: %d:%d:%d" , 2013,06,07); if ( rc != UNQLITE_OK ){ //Insertion fail, Hande error (See below)

return ; }

//Switch to the append interface rc = unqlite_kv_append(pDb, "msg" ,-1, "Hello, " ,7); //msg => 'Hello, ' if ( rc == UNQLITE_OK ){ //The second chunk rc = unqlite_kv_append(pDb, "msg" ,-1, "Current time is: " ,17); //msg => 'Hello, Current time is: ' if ( rc == UNQLITE_OK ){ //The last formatted chunk rc = unqlite_kv_append_fmt(pDb, "msg" ,-1, "%d:%d:%d" ,10,16,53); //msg => 'Hello, Current time is: 10:16:53' } }

//Delete a record unqlite_kv_delete(pDb, "test" ,-1) ;

//Store 20 random records. for (i = 0 ; i < 20 ; ++i ){ char zKey[12]; //Random generated key char zData[34]; //Dummy data

// generate the random key unqlite_util_random_string(pDb,zKey, sizeof (zKey));

// Perform the insertion rc = unqlite_kv_store(pDb,zKey, sizeof (zKey),zData, sizeof (zData)); if ( rc != UNQLITE_OK ){ break ; } }

if ( rc != UNQLITE_OK ){ //Insertion fail, Handle error

const char *zBuf; int iLen; /* Something goes wrong, extract the database error log */ unqlite_config(pDb, UNQLITE_CONFIG_ERR_LOG , &zBuf , &iLen ); if ( iLen > 0 ){ puts(zBuf); } if ( rc != UNQLITE_BUSY && rc != UNQLITE_NOTIMPLEMENTED ){ /* Rollback */

unqlite_rollback(pDb); } }

//Auto-commit the transaction and close our handle. unqlite_close (pDb);



The database is opened on line 6 using a call to unqlite_open() . This is often the first UnQLite API call that an application makes and is a prerequisite in order to work with the database library.

Raw records storage via the Key/Value layer is performed on line 10 til line 29 using the various UnQLite interfaces such as unqlite_kv_store(), unqlite_kv_append(), unqlite_kv_store_fmt(), unqlite_kv_append_fmt(), etc.

On line 34, we remove a record from our database using the unqlite_kv_delete() interface.

On line 36 til line 48, we perform an insertion of 20 random records. The random key is obtained by the utility interface: unqlite_util_random_string() .

Error is handled on line 53 til line 62.

Finally, the transaction is auto-committed and our database is closed on line 66 using the unqlite_close() interface.







Under the KV store, since keys and values are treated as a simple array of bytes, you can even insert external files perhaps some XML or any other files you need in your UnQLite database and turn it (i.e. The UnQLite database) into a Tar archive with O(1) lookup. Here is a snippet.





#include <unqlite.h> // Usage example: ./unqlite_tar config.xml license.txt audio.wav splash.jpeg,... void *pMap; unqlite_int64 iSize; int rc,i; unqlite *pDb;

// Open our database; "test.db" ,UNQLITE_OPEN_CREATE); rc = unqlite_open (&pDb,,UNQLITE_OPEN_CREATE);

if ( rc != UNQLITE_OK ){ /* Handle error */ return ; }

// Store the given files in our database

for ( i = 1 ; i < argc ; ++i ){ const char *zName = argv[i]; //Name of the target file

// Obtain a read-only memory view of the target file; rc = unqlite_util_load_mmaped_file zName , &pMap , &iSize ); if ( rc != UNQLITE_OK ){ /* Handle error */ return ; }

// Store the whole file in our database; rc = unqlite_kv_store (pDb, zName ,-1, pMap , iSize ); if ( rc != UNQLITE_OK ){ /* Handle error */ return ; }

// Discard the memory view; unqlite_util_release_mmaped_file( pMap,iSize); pMap,iSize); }



//Auto-commit the transaction and close our handle unqlite_close (pDb);









The function calls to pay attention to here are the call to unqlite_util_load_mmaped_file() on line 18 which is used to obtain a read-only memory view of the whole file. After that, a standard KV insertion is done using a call to unqlite_kv_store() . Don't forget to release the memory view of the target file and closing your database.





Fetching data under the KV store is straightforward and involve only a single call to: unqlite_kv_fetch_callback() or unqlite_kv_fetch(). The unqlite_kv_fetch_callback() interface is the recommended one where the caller supply a simple callback responsible of consuming the record data perhaps redirecting it (i.e. Record data) to its standard output (STDOUT) or to the connected peer (See interface documentation for a working example). The unqlite_kv_fetch() interface should be familiar to everyone where it expects a user supplied buffer (either statistically or dynamically allocated) where record data is copied in.

Using Database Cursors



Cursors provide a mechanism by which you can iterate over the records in a database. Using cursors, you can seek, fetch, move, and delete database records.

For an introduction to the cursor interfaces, please refer to their API documentation available here.

Below is a simple C program that demonstrates how to use cursors. This program will iterate over the database from the last record to the first.





#include <unqlite.h>



int rc;

unqlite *pDb;

unqlite_kv_cursor *pCursor;

unqlite_int64 iData;



// Open our database;

"test.db" ,UNQLITE_OPEN_CREATE);

if ( rc != UNQLITE_OK ){ return ; }

rc = unqlite_open (&pDb,,UNQLITE_OPEN_CREATE);( rc != UNQLITE_OK ){; }

//Store some records unqlite_kv_store(), unqlite_kv_append() ...



/* Allocate a new cursor instance */ rc = unqlite_kv_cursor_init(pDb,&pCursor); if ( rc != UNQLITE_OK ){ return; }



/* Point to the last record */

rc =

rc = unqlite_kv_cursor_last_entry (pCursor); if ( rc != UNQLITE_OK ){ return ; }



/* Iterate over the records */ while ( unqlite_kv_cursor_valid_entry(pCursor) ){ /* Consume the key */ printf( "

Key ==>

\t" ); unqlite_kv_cursor_key_callback(pCursor,DataConsumerCallback,0);

/* Extract data length */ unqlite_kv_cursor_data(pCursor,NULL,&iData); printf( "

Data length ==> %lld

\t" ,iData);

/* Consume the data */ unqlite_kv_cursor_data_callback(pCursor,DataConsumerCallback,0); /* Point to the previous record */ unqlite_kv_cursor_prev_entry(pCursor); }

/* Finally, Release our cursor */ unqlite_kv_cursor_release(pDb,pCursor);

//Auto-commit the transaction and close our handle



unqlite_close (pDb);







Introduction to the Document-Store C/C++ Interfaces

Structured data storage is presented to clients via the Document store interface to UnQLite. The Document-store itself is used to store JSON docs (i.e. Objects, Arrays, Strings, etc.) in the database and is powered by the Jx9 programming language.

The following is the Jx9 script to compile by the C program defined below that demonstrates how to use the document store interfaces to UnQLite.



/* Create the collection 'users' */ if( !db_exists( 'users' ) ){ /* Try to create it */ $rc = db_create( 'users' ); if ( !$rc ){ //Handle error print db_errlog();

return; } } //The following is the JSON objects to be stored shortly in our 'users' collection $zRec = [ { name : 'james' , age : 27, mail : 'dude@example.com' }, { name : 'robert' , age : 35, mail : 'rob@example.com' }, { name : 'monji' , age : 47, mail : 'monji@example.com' },

{

name : 'barzini' , age : 52, mail : 'barz@mobster.com' } ];

//Store our records

$rc = db_store( 'users' ,$zRec );

if ( !$rc ){

//Handle error print db_errlog();

return; }

//One more record

$rc = db_store( 'users' , { name : 'alex' , age : 19, mail : 'alex@example.com' } );

if ( !$rc ){

//Handle error print db_errlog();

return; }

print "Total number of stored records: " ,db_total_records( 'users' ),JX9_EOL;







//Fetch data using db_fetch_all() db_fetch_by_id() and db_fetch()

Now, the C program which compile and executes the Jx9 script defined above.

#include <unqlite.h>

int rc; unqlite *pDb; unqlite_vm *pDb;

// Open our database; "test.db" ,UNQLITE_OPEN_CREATE); rc = unqlite_open (&pDb,,UNQLITE_OPEN_CREATE); if ( rc != UNQLITE_OK ){ /* Handle error */ return ;

/* Compile the Jx9 script defined above */ rc = unqlite_compile(pDb,JX9_PROG,sizeof(JX9_PROG)-1,&pVm); if ( rc != UNQLITE_OK ){ if ( rc == UNQLITE_COMPILE_ERR ){ const char *zBuf; int iLen; /* Compile-time error, extract the compiler error log */ unqlite_config(pDb,UNQLITE_CONFIG_JX9_ERR_LOG,&zBuf,&iLen); if ( iLen > 0 ){ puts(zBuf); } } return ; }

/* Install a VM output consumer callback */ rc = unqlite_vm_config(pVm,UNQLITE_VM_CONFIG_OUTPUT,OutputConsumer,0); if ( rc != UNQLITE_OK ){ return; }

/* Execute our script */ rc = unqlite_vm_exec(pVm); if ( rc != UNQLITE_OK ){ return; }

/* Finally, release our VM */ unqlite_vm_release(pVm);

unqlite_close (pDb);





The Document-Store interface to UnQLite works as follows:

Obtain a new database handle via unqlite_open() (line 8 in our example). Compile your Jx9 script (line 12 in our example) using one of the compile iterfaces such as unqlite_compile() or unqlite_compile_file() . On successful compilation, the engine will automatically create an instance of the structure unqlite_vm and a pointer to this structure is made available to the caller. When something goes wrong during compilation of the target Jx9 script due to a compile-time error, the caller must discard this pointer and fix its erroneous Jx9 code. Compile-time error logs can be extracted (line 18 in our example) via a call to unqlite_config() with a configuration verb set to UNQLITE_CONFIG_JX9_ERR_LOG . Optionally, configure the virtual machine using unqlite_vm_config() . Optionally, register one or more foreign functions or constants using the unqlite_create_function() or unqlite_create_constant() interfaces. Execute the compiled Jx9 program (line 33 in our example) by calling unqlite_vm_exec() . Optionally, extract the contents of one or more variables declared inside your Jx9 script using unqlite_vm_extract_variable(). Optionally, Reset the virtual machine using unqlite_vm_reset() , then go back to step 6. Do this zero or more times. Destroy the virtual machine (line 39 in our example) using unqlite_vm_release() .

Other useful Links to start with



Check out the Introduction To The UnQLite C/C++ Interface for an introductory overview and roadmap to the dozens of UnQLite interface functions. A separate document, The UnQLite C/C++ Interface , provides detailed specifications for all of the various C/C++ APIs for UnQLite. Once the reader understands the basic principles of operation for UnQLite, that document should be used as a reference guide.



Any questions, visit the Support Page for additional information .

Interested in Machine Learning & Computer Vision? try out Pixlab