Hoel
C Database abstraction library with json based language
Loading...
Searching...
No Matches
Hoel

.github/workflows/ccpp.yml

Database abstraction library written in C.

Simple and easy to use database access library. Works with SQLite 3, MariaDB and PostgreSQL databases. Uses a JSON-based language with jansson to execute simple queries based on one table.

Online documentation

See the online documentation for a doxygen format of the API documentation.

Installation

Prerequisites

Jansson

Install Jansson library for JSON manipulation, minimum version 2.4. On a Debian-based platform, run the following command:

$ sudo apt-get install libjansson-dev

Database libraries

Install Hoel database dependencies based on your requirements:

  • SQLite3: Install the package libsqlite3-dev
  • MariaDB: Install the package libmariadb-dev
  • PostgreSQL: Install the package libpq-dev

Distribution packages

Packaging status

Hoel is available in multiple distributions as official package. Check out your distribution documentation to install the package automatically.

$ # Example for Debian testing
$ sudo apt install libhoel-dev

Pre-compiled packages

You can install Hoel with a pre-compiled package available in the release pages. jansson, sqlite3, libmariadb-client and libpq development files packages are required to install Hoel. The packages files hoel-dev-full_* contain the libraries orcania, yder and hoel. It's also compiled with all the database backends.

For example, to install Hoel with the hoel-dev-full_2.3.0_Debian_stretch_x86_64.tar.gz package downloaded on the releases page, you must execute the following commands:

$ sudo apt install -y libjansson-dev libmariadb-dev libsqlite3-dev libpq-dev libsystemd-dev
$ wget https://github.com/babelouest/hoel/releases/download/v1.4.0/hoel-dev-full_1.4.0_Debian_stretch_x86_64.tar.gz
$ tar xf hoel-dev-full_1.4.0_Debian_stretch_x86_64.tar.gz
$ sudo dpkg -i liborcania-dev_1.2.0_Debian_stretch_x86_64.deb
$ sudo dpkg -i libyder-dev_1.2.0_Debian_stretch_x86_64.deb
$ sudo dpkg -i libhoel-dev_1.4.0_Debian_stretch_x86_64.deb

If there's no package available for your distribution, you can recompile it manually using CMake or Makefile.

Install from the source

CMake - Multi architecture

CMake minimum 3.5 is required.

Last Hoel release: https://github.com/babelouest/hoel/releases/latest/

Run the CMake script in a sub-directory, example:

$ cd <hoel_source>
$ mkdir build
$ cd build
$ cmake ..
$ make && sudo make install

The available options for CMake are:

  • -DWITH_SQLITE3=[on|off] (default on): Enable/disable SQLite3 database backend
  • -DWITH_MARIADB=[on|off] (default on): Enable/disable MariaDB database backend
  • -DWITH_PGSQL=[on|off] (default on): Enable/disable PostgreSQL database backend
  • -DWITH_JOURNALD=[on|off] (default on): Build with journald (SystemD) support for logging
  • -DBUILD_STATIC=[on|off] (default off): Build the static archive in addition to the shared library
  • -DBUILD_HOEL_TESTING=[on|off] (default off): Build unit tests
  • -DBUILD_HOEL_DOCUMENTATION=[on|off] (default off): Build the documentation, doxygen is required
  • -DINSTALL_HEADER=[on|off] (default on): Install header file hoel.h
  • -DBUILD_RPM=[on|off] (default off): Build RPM package when running make package
  • -DCMAKE_BUILD_TYPE=[Debug|Release] (default Release): Compile with debugging symbols or not

Good ol' Makefile

Clone, compile and install Orcania and Yder libraries.

Orcania (Miscellaneous functions)

Last Orcania release: https://github.com/babelouest/orcania/releases/latest/

$ cd orcania/src
$ make && sudo make install

Yder (simple logs library)

Last Yder release: https://github.com/babelouest/yder/releases/latest/

$ cd yder/src
$ make
$ sudo make install

Hoel

Last Hoel release: https://github.com/babelouest/hoel/releases/latest/

$ cd hoel/src
$ make
$ sudo make install

By default, Hoel is compiled with the 3 databases support. If you don't need one or more database, follow these instructions

SQLite 3

Add DISABLE_SQLITE=1 to the make command:

$ cd hoel/src
$ make DISABLE_SQLITE=1
$ sudo make install

MariaDB

Add DISABLE_MARIADB=1 to the make command:

$ cd hoel/src
$ make DISABLE_MARIADB=1
$ sudo make install

Postgre SQL

Add DISABLE_POSTGRESQL=1 to the make command:

$ cd hoel/src
$ make DISABLE_POSTGRESQL=1
$ sudo make install

Disable 2 backends

You can disable 2 databases backends to keep just one, simply add both parameters to the make command:

$ cd hoel/src
$ make DISABLE_MARIADB=1 DISABLE_POSTGRESQL=1
$ sudo make install

Installation folder

By default, the shared library and the header file will be installed in the /usr/local location. To change this setting, you can modify the DESTDIR value in the src/Makefile.

API Documentation

Header files and compilation

To use Hoel in your code, include the file hoel.h.

#include <hoel.h>
Hoel database abstraction library.

Use the flag -lhoel to include Hoel library in the linking process.

Return values

When specified, some functions return H_OK on success, and other values otherwise. H_OK is 0, other values are non-0 values. The defined errors list is the following:

#define H_OK 0 // No error
#define H_ERROR 1 // Generic error
#define H_ERROR_PARAMS 2 // Error in input parameters
#define H_ERROR_CONNECTION 3 // Error in database connection
#define H_ERROR_DISABLED 4 // Database connection is disabled
#define H_ERROR_QUERY 5 // Error executing query
#define H_ERROR_MEMORY 99 // Error allocating memory

Memory allocation

Some function return allocated values. When the value is not a structure, you must use the function h_free to clean it. Otherwise, use the dedicated functions.

void h_free(void * data);
void h_free(void *data)
Definition: hoel.c:32

Initialization

To create a connection to a database, use its dedicated function

struct _h_connection * h_connect_sqlite(const char * db_path);
struct _h_connection * h_connect_mariadb(const char * host, const char * user, const char * passwd, const char * db, const unsigned int port, const char * unix_socket);
struct _h_connection * h_connect_pgsql(char * conninfo);
struct _h_connection * h_connect_pgsql(const char *conninfo)
Definition: hoel-pgsql.c:410
struct _h_connection * h_connect_sqlite(const char *db_path)
Definition: hoel-sqlite.c:324
struct _h_connection * h_connect_mariadb(const char *host, const char *user, const char *passwd, const char *db, const unsigned int port, const char *unix_socket)
Definition: hoel-mariadb.c:437
Definition: hoel.h:87

All these functions return a struct _h_connection * on success. This pointer will be needed on every call to Hoel functions.

When you no longer need your connection, close it using the function h_close_db. This will close the connection to the database and free the memory allocated by the connection.

int h_close_db(struct _h_connection * conn);
int h_close_db(struct _h_connection *conn)
Definition: hoel.c:40

The connection must be cleaned when it's no longer needed.

int h_clean_connection(struct _h_connection * conn);
int h_clean_connection(struct _h_connection *conn)
Definition: hoel.c:633

Escape string

If you need to escape parameters, you can use the functions h_escape_string, the returned value must be h_free'd after use.

The meaning of existence of the function h_escape_string_with_quotes is because of PostgreSQL ways to escape an unsafe string.

TL;DR:

  • Use h_escape_string_with_quotes only if you use a PostgreSQL database.
  • You can use both h_escape_string_with_quotes and h_escape_string if you use only SQLite3 and/or MariaDB databases.

If the unsafe string contains backslashes "\"</tt>, the escaped string will have the pattern &lsquo;E&rsquo;<safe>'<tt>, otherwise it will have the pattern</tt>'<safe>'<tt>. Since the function</tt>h_escape_string<tt>must return the safe string only, without surroundings quotes</tt>'<tt>, the format</tt>E'<safe>'` is incompatible with Hoel API. Therefore, using <tt>h_escape_string</tt> with PostgreSQL could lead to undefined behaviour and more importantly exploitable bugs. To avoid this, on a PostgreSQL database, an escaped string using <tt>h_escape_string</tt> that should return the format &lsquo;E&rsquo;<safe>'<tt>will return</tt>NULL<tt>, and an escaped string using</tt>h_escape_string_with_quotes<tt>will return the format</tt>E'<safe>'`. @icode{c} /** * h_escape_string * Escapes a string * returned value must be h_free'd after use */ char * h_escape_string(const struct _h_connection * conn, const char * unsafe); /** * h_escape_string_with_quotes * Escapes a string and returns it ready to be inserted in the query * returned value must be h_h_free'd after use */ char * h_escape_string_with_quotes(const struct _h_connection * conn, const char * unsafe); @endicode @subsubsection autotoc_md25 Build a more complicated where clause When you need to run a query with a where clause using multiple parameters, such as &lsquo;WHERE col1='a&rsquo; AND (col2='b' OR col3=5) AND col4=42.3<tt>, you can use the operator</tt>raw`: @icode{JSON} { "table": "table1", "columns": ["col1", "col2"] "where": { " ": { "operator", "raw", "value", "col1='a' AND (col2='b' or col3='c') AND col4=5" } } } @endicode In some cases, you may need to build the where clause with multiple variables. In hoel 1.4.27, the function <tt>h_build_where_clause</tt> was introduced to help that. Please note that this function is still in Beta. @icode{C} /** * h_build_where_clause * Generates a where clause based on the pattern and the values given * @param conn the connection to the database * @param pattern the pattern to build the where clause * the pattern variables available are the following: * - %s: a string value to escape with quotes * - %S: a string value to escape without quotes * - %c: a string value not to escape with quotes * - %C: a string value not to escape without quotes * - %d: an integer value in json_int_t format * - %f: a double value * - %j: a json_t value, the value must be of the type JSON_INTEGER, JSON_REAL or JSON_STRING, string values will be escaped with quotes * - %%: the value '%' * @return a heap-allocated string * returned value must be h_free'd after use */ char * h_build_where_clause(const struct _h_connection * conn, const char * pattern, ...); @endicode Then, to build the where clause above using <tt>h_build_where_clause</tt>, you can use the following code: @icode{C} const char col1[] = "a", col2[] = "b"; json_int_t col3 = 5; double col4 = 42.3; char * where_clause = h_build_where_clause("col1=s AND (col2='S' OR col3=d) AND col4=f", col1, col2, col3, col4); json_t * j_query = json_pack("{sss[ss]s{s{ssss}}}", "table", "table1", "columns", "col1", "col2", "where", " ", "operator", "raw", "value", where_clause); h_free(where_clause); // Execute j_query @endicode Note that if you use constant litteral for integer or double values, you should cast them first: @icode{C} const char col1[] = "a", col2[] = "b"; char * where_clause = h_build_where_clause("col1=s AND (col2='S' OR col3=d) AND col4=f", col1, col2, (json_int_t)5, (double)42.3); @endicode @subsubsection autotoc_md26 Execute a SQL query To execute a SQL query, you can use the function <tt>h_execute_query</tt> which will run the query in the database specified by the parameter <tt>conn</tt>. If a <tt>result</tt> parameter is specified, the result of the query (if any) will be stored in the <tt>result</tt> structure. @icode{c} /** * h_execute_query * Execute a query, set the result structure with the returned values if available * if result is NULL, the query is executed but no value will be returned * options available * H_OPTION_NONE (0): no option * H_OPTION_SELECT: Execute a prepare statement (sqlite only) * H_OPTION_EXEC: Execute an exec statement (sqlite only) * return H_OK on success */ int h_execute_query(const struct _h_connection * conn, const char * query, struct _h_result * result, int options); @endicode @subsubsection autotoc_md27 Result structure The <tt>struct _h_result</tt> is a structure containing the values returned by a query. The definition of the structure is: @icode{c} /** * sql result structure */ struct _h_result { unsigned int nb_rows; unsigned int nb_columns; struct _h_data ** data; }; @endicode The data value is a 2 dimensional array with <tt>struct _h_data</tt> variables. A <tt>struct _h_data</tt> is defined as: @icode{c} /** * sql data container */ struct _h_data { int type; void * t_data; }; @endicode where <tt>type</tt> can be the following values: @icode{c} #define HOEL_COL_TYPE_INT 0 #define HOEL_COL_TYPE_DOUBLE 1 #define HOEL_COL_TYPE_TEXT 2 #define HOEL_COL_TYPE_DATE 3 #define HOEL_COL_TYPE_BLOB 4 #define HOEL_COL_TYPE_NULL 5 @endicode <tt>t_data</tt> will point to a <tt>struct _h_type_*</tt> corresponding to the type. The <tt>struct _h_type_*</tt> available are: @icode{c} /** * sql value integer type */ struct _h_type_int { int value; }; /** * sql value double type */ struct _h_type_double { double value; }; /** * sql value date/time type */ struct _h_type_datetime { struct tm value; }; /** * sql value string type */ struct _h_type_text { char * value; }; /** * sql value blob type */ struct _h_type_blob { size_t length; void * value; }; @endicode @subsubsection autotoc_md28 Clean results or data To clean a result or a data structure, you can use its dedicated functions: @icode{c} /** * h_clean_result * Free all the memory allocated by the struct _h_result * return H_OK on success */ int h_clean_result(struct _h_result * result); /** * h_clean_data * Free memory allocated by the struct _h_data * return H_OK on success */ int h_clean_data(struct _h_data * data); @endicode @subsubsection autotoc_md29 Get last id inserted If you need the last id generated after an insert query, you can use the following function: @icode{c} /** * h_query_last_insert_id * return the id of the last inserted value * return a pointer to `struct _h_data *` on success, NULL otherwise. */ struct _h_data * h_query_last_insert_id(const struct _h_connection * conn); @endicode @subsubsection autotoc_md30 Additional query functions You can use additional functions for specific needs. All these function will use <tt>h_execute_query</tt> but check input parameters before. @icode{c} /** * h_query_insert * Execute an insert query * return H_OK on success */ int h_query_insert(const struct _h_connection * conn, const char * query); /** * h_query_update * Execute an update query * return H_OK on success */ int h_query_update(const struct _h_connection * conn, const char * query); /** * h_query_delete * Execute an delete query * return H_OK on success */ int h_query_delete(const struct _h_connection * conn, const char * query); /** * h_execute_query * Execute a select query, set the result structure with the returned values * return H_OK on success */ int h_query_select(const struct _h_connection * conn, const char * query, struct _h_result * result); @endicode @subsubsection autotoc_md31 Simple JSON queries Hoel allows to use JSON objects for simple queries with <tt>jansson</tt> library. In the simple JSON queries, a JSON object called <tt>json_t * j_query</tt> is used to generate the query. All <tt>json_t *</tt> returned and updated values must be free after use. A <tt>j_query</tt> has the following form: @icode{javascript} * { * "table": "table_name" // String, mandatory, the table name where the query is executed * "columns": ["col1", "col2"] // Array of strings, available for h_select, optional. If not specified, * will be used, the columns values are not escaped by the library * "order_by": "col_name [asc|desc]" // String, available for h_select, specify the order by clause, optional, the value is not escaped by the library * "limit": integer_value // Integer, available for h_select, specify the limit value, optional * "offset" // Integer, available for h_select, specify the limit value, optional but available only if limit is set * "values": [{ // json object or json array of json objects, available for h_insert, mandatory, specify the values to update * "col1": "value1", // Generates col1='value1' for an update query * "col2": value_integer, // Generates col2=value_integer for an update query * "col3", "value3", // Generates col3='value3' for an update query * "col4", null // Generates col4=NULL for an update query * }] * "set": { // json object, available for h_update, mandatory, specify the values to update * "col1": "value1", // Generates col1='value1' for an update query * "col2": value_integer, // Generates col2=value_integer for an update query * "col3", "value3", // Generates col3='value3' for an update query * "col4", null // Generates col4=NULL for an update query * } * "where": { // json object, available for h_select, h_update and h_delete, mandatory, specify the where clause. All clauses are separated with an AND operator * "col1": "value1", // Generates col1='value1' * "col2": value_integer, // Generates col2=value_integer * "col3": null, // Generates col3=NULL * "col4", { // Generates col4<12 * "operator": "<", * "value": 12 * }, * "col5", { // Generates col5 IS NOT NULL * "operator": "NOT NULL" * }, * "col6", { // Generates col6 LIKE '%value6%' * "operator": "raw", * "value": "LIKE 'value6'" * } * } * } @endicode @paragraph autotoc_md32 Where clause construction A <tt>where</tt> clause is a JSON object containing a series of clauses. A clause can have 2 different forms: - <tt>col_name: value</tt> - <tt>col_name: {operator: "operator_value", value: value}</tt> In the first case, <tt>col_name: value</tt>, the clause becomes <tt>col_name = value</tt>. Value is always escaped. In the second case, <tt>col_name: {operator: "operator_value", value: value}</tt>, depending on the <tt>operator</tt> value, the clause can have different forms: - <tt>operator: "NOT NULL"</tt>, the clause becomes <tt>col_name IS NOT NULL</tt> - <tt>operator: "raw"</tt>, the <tt>value</tt> value becomes the clause itself, not escaped, for example in &lsquo;{ "operator": "raw", "value": "LIKE ’value6'" }<tt>, the clause becomes</tt>col6 LIKE '%value6%'<tt> - otherwise, the clause becomes</tt>col_name operator value`, value is escaped All clauses are separated by an <tt>AND</tt> operator. As en example, here is a JSON object and its generated where clause: JSON object: @icode{javascript} { "col1": "value1", "col2": 42, "col3": { "operator": ">=", "value": 55.5 }, "col4": { "operator": "raw", "value": "LIKE 'alu'" } }

SQL Where clause:

WHERE col1 = 'value1'
AND col2 = 42
AND col3 >= 55.5
AND col4 LIKE '%alu%'

If you need less simple clauses, you can build it on your own and use the h_execute_query or the h_execute_query_json functions.

The simple JSON queries functions are:

int h_select(const struct _h_connection * conn, const json_t * j_query, json_t ** j_result, char ** generated_query);
int h_insert(const struct _h_connection * conn, const json_t * j_query, char ** generated_query);
json_t * h_last_insert_id(const struct _h_connection * conn);
int h_update(const struct _h_connection * conn, const json_t * j_query, char ** generated_query);
int h_delete(const struct _h_connection * conn, const json_t * j_query, char ** generated_query);
int h_delete(const struct _h_connection *conn, const json_t *j_query, char **generated_query)
Definition: hoel-simple-json.c:762
json_t * h_last_insert_id(const struct _h_connection *conn)
Definition: hoel-simple-json.c:674
int h_select(const struct _h_connection *conn, const json_t *j_query, json_t **j_result, char **generated_query)
Definition: hoel-simple-json.c:458
int h_insert(const struct _h_connection *conn, const json_t *j_query, char **generated_query)
Definition: hoel-simple-json.c:604
int h_update(const struct _h_connection *conn, const json_t *j_query, char **generated_query)
Definition: hoel-simple-json.c:712

JSON last insert id

The function h_last_insert_id returns the last inserted id in a json_t * format.

json_t * h_last_insert_id(const struct _h_connection * conn);

Example source code

See examples folder for detailed sample source codes.