Global

parameter name, can be a string regexp to match dynamic parameters, matched pieces then can be used by the make property to build the final variable name.

parameter description, it is show when run bksh -help command

a valid config type:

• none - skip this parameter
• bool - converts to a boolean
• int, real, number - converts to a number
• map - convert key:value,key:value... pairs into an object, see delimiter/separator properties, if the value starts with { and ends with } the JSON parser is used instead, this is to support maps for simple objects and full JSON for objects with complex types like RegExp
• set, list - array type, set makes the list unique, splits strings by separator or ,|
• regexp - a RegExp
• regexpobj - add a regexp to the object that consist of list of patterns and compiled regexp, lib.testRegexpObj
• url - an object produces by URL.parse
• json, js - parse as JSON into an object/array
• path - resolves to an absolute sanitized path
• callback - calls the callback property only, does not save
• file - reads contents of a file

object name in the module where to store the value, otherwise the value is defined in the module, the obj name is stripped automatically from the variable name. The object name can contain dots to point to deep objects inside the module and use placeholders like $1, $2 if name has regexp parts

works with regexp names like name-([a-z]+)-(.+) to build the final parameter name, make the variable name from the matched pieces, the final variable is constructed by replacing every $1, $2, ... with corresponding matched piece from the name regexp.

if true prepend a value to the list, to remove an item prepend it with !!

for array: mode append a value

process only args that match the pass value

env variable name to apply before parsing config files

if true merge properties with existing object, 'obj' must be provided

if true and the name contains dots it is treated as deep object

default is auto to parse map values or can be any value type, for type map

convert name to this type using module:lib.toValue

only set new value to non-existing or existing property of the same type, to prevent overriding existing properties

skip if value is equal or included in the list, also works for merges

sort array params

only keep unique items in lists

characters to use when camelizing the name, default is "-"

do not camelize the name

do not save empty values

allows empty values for maps and regexps

detect type by using module:lib.autoType, if it is an object then each property defines a type by name/key, this is for complex parameters with obj

text with examples

function to call for the callback types for manully parsing and setting the value, (value, obj) the obj being current parameter

function to call at the end for additional processing as (value, obj)

separator to use for list and map items, for lists default is ,|, for maps it is :;

separator to split key:value pairs, default is ,

parsed but not saved, usually it is handled by onupdate callback

text to strip from the final variable name

only set this parameter once

only set new value to existing property

type mapping, convert lowercase type into other type supported by any specific database, e.g. "long": "integer"

ops mapping, convert lowercase op into other op supported by any specific database, e.g. "gt": ">="

Properties

supports IF EXISTS on table or indexes

supports for auto increment columns

supports NOT NULL constraint

actual column name

type from DbTableColumn

operator to use in comparison or update, lowercase and all underscores are converted into spaces

value passed for compare or update

join operator, AND is default

original name in case name contained _$ placeholder

existing column definition or undefined

name of the database pool where to execute this query. The difference with the high level functions that take a table name as their firt argument, this function must use pool explicitely if it is different from the default. Other functions can resolve the pool by table name if some tables are assigned to any specific pool by configuration parameters `db-pool-tables__.

call the callback via module:lib.tryCatch, in case of exception if tryCatch is a function it will be called with the raised error and same arguments, useful in case some continueation is required or returning an error from deep context back to the user.

api.app.get("/user/:id", (res, res, next) => {
    const dbopts = { tryCatch: next };

    db.get("users", { id: req.params.id }, dopts, (err, row) => {
        ... some processing raising TypeException

        context.reply(err, row);
    })
})

log results at the end with this level or debug by default

log errors with this level instead of error

clear errors occurred as it never happen, do not report in the log, if an array then only matched codes will be cleared

if true then skip post processing result rows, return the data as is, this will result in returning combined columns as it is

if true skip converting the data from the database format into Javascript data types, it uses column definitions

if true skip query preparation and columns processing, the req.query is passed as is, useful for syncing between pools for the table to convert values returned from the db into the the format defined by the column

if true then it is supposed to return only one record with property count, skip all post processing and convertion

to return the record just processed in the info object as query property, it will include all generated and updated columns like uuid...

to return the query record as result including all post processing and new generated columns, this is not what returning property does, it only returns the query record with new columns from memory like autogenerated uuid...

return record values after operation:

• * or new - return new record, update works for all databases, add only works for SQL, DynamoDB/ElasticSearch will return the input record, making it similar to result_query
• old - return old record values before the update

if true then run getCached version directly

disable caching even if configured for the table

operators to use for for properties, an object with column name and operator, for query methods like get, select ops is used for filter condition, for update methods like add/put/update/incr the ops define how a column is modified, see module:db for description about all comparison operators.

an object that defines ops by column type, for example typesOps: { list: "add" } will make sure all lists will have options.ops set to add if not specified explicitly

a list of columns or expressions to return or all columns if not specified, only existing columns will be returned

start position by primary keys for NoSQL, this is the next_token passed by the previous query from module.db.select or for SQL it is OFFSET position.

starting page number for pagination, uses count to find actual record to start, for SQL databases mostly

how many records to retrieve in a single batch or the size of a page for SQL

a convenient option to return the first record from the result or null (similar to db.get method)

similar to first but return the last record

how to join condition expressions, default is AND

operators to use to combine several expressions in case when an array of values is given, supports and|or|AND|OR__

sort by column(s). if not provided then no sorting must be done at all, records will be returned in the order they are kept in the DB.

• SQL: one or more column names to sort by, if a name starts with ! it means descending order, e.g. sort: ["name", "!time"]
• DynamoDB: this may affect the results if columns requsted are not projected in the index, sorting can only by done by indexed column
• Elasticsearch: by default page based pagination is used if no fullscan is provided with or without the sort option, can be an object in native ES format: { name: { order: "desc" }}, special name _random to return in random order using script

if sorting, do in descending order for all columns

exlicit key for caching, return from the cache or from the DB and then cache it with this key, works the same as get

a name of one of the cache keys to use, it must be defined by a db-cache-keys-table-name parameter

do not check for actual columns defined in the pool tables and add all properties from the obj, only will work for NoSQL dbs, by default all properties in the obj not described in the table definition for the given table will be ignored.

an object with the conditions for the update/incr, in SQL this is added to WHERE, it is used instead of or in addition to the primary keys in the query, a property named $OR/$AND/$NOT will be treated as a sub-expression if it is an object. For multiple OR/AND use $$OR, $$$OR,...

create a new record if it does not exist

triggers to use specific capacity, default is read

a factor to apply for the read or write capacity limit and trigger the capacity check, default is 0.9

use a different table for capacity throttling instead of the table, useful for cases when the row callback performs writes into that other table and capacity is different

a full capacity object to pass to select calls, used by module:db.checkCapacity

if true rowCallback will be called with all rows from the batch, used in module:db.scan not every row individually, batch size is defined by the count property, used in module:db.scan

as batch mode but the rowCallback is called synchronously as rowCallback(row, info), used in module:db.scan

how many rows or operations to process at the same time, used in module:db.scan, module:db.batch

total number of records to scan, used in module:db.scan

if true no scan will be performed if no primary keys are specified to prevent scanning large tables, used in module:db.scan

if true force to perform a full scan, this is slightly different for each database, used in module:db.scan:

• DynamoDB: used full table scan, the query condition still is checked, can be expensive for large table because pricing is by every object checked not by what is actually returned
• Elasticsearch: performs streaming scan, using native order but all conditions are checked still, very effective for large scans

skip column preprocessing and dynamic values for pool sync and backup restore, used in module:db.copy

column definitions

Common options to apply during table creation

Sqlite statements to append to create table statement, e.g. _$sqlite: { sql_extra: "STRICT" }

PostgreSQL statements to apply during table creation

DynamoDB options to apply during table creation

Elasticsearch options to apply during table creation

column type, supported types:

• int, bigint, log, real, float, number - numeric types
• bool, boolean - stored as boolean type
• text, varchar, str, string, keyword - text types
• date, time, timestamp - Date stored in database supported type, usually as text in SQL
• mtime - timestamp in milliseconds
• json - stored as JSON text but converted to/from native Javascript objects
• obj, object - native JSON object type
• array - native JSON array type
• list - store a list of primitive types, strings and/or numbers
• set - a unique set of string or numbers if supported natively, otherwise same as list
• random - generates a random number in module:db.add/module:db.put methods, uses optional .max/.min properties
• uuid, suuid, sfuuid - autogenerate the column value with UUID, optional prefix property will be prepended, { type: "uuid", prefix: "u" }_, see module:lib.uuid, module:lib.suuid, module:lib.tuuid
• now - defines a column to be automatically filled with the current timestamp in milliseconds
• counter - defines a columns that will be automatically incremented by the module:db.incr command, on creation it is set with 0

column is part of the primary key, for composite keys the number defines the place starting with 1

column is part of an index, the value is a number for the column position in the index, this creates a regular index

additonal indexes where N is a number, e.g. index1, index2, ...

• First column in the index may contain specific options about the index type, e.g.
  • id: { type: "int", index: 1, _$db: { index: "UNIQUE"} }
  • id: { type: "text", index1: 1, _$db: { index1: "INVERTED"} }
  • name: { index1: 1, _$pg: { index1: { sql: "CREATE INDEX @index@ ON @table@ USING GIN (@columns@)" } } }

default value to save if not provided

default value to return if not present in the db record

this column must be used as is, no parsing/stemming (Elasticsearch), same as type: "keyword" in case the type is specified already

only add/put operations will use the value, incr/update will ignore it, this happens in the app not database

prefix to be prepended for autogenerated columns: uuid, suuid, sfuuid

a list with property names that must be joined together before performing a db operation, it will use the given record to produce a new property, all properties must exist even if empty, any undefined value will skip the whole join

to be used as a separator for joins or lists converted into string

column length (SQL)

true if should be NOT NULL (SQL)

true for AUTO_INCREMENT column (SQL)

foreign key reference (SQL)

Properties

split returned value, options for module:lib.split, this is also used implicitly by all list types like list, set.

convert the value on save, all operations will be done in the order of definition

Properties

validation checks before save, in-app checks, supports a subset of module:lib.validate properties, all other properties are ignored

Properties

cleanup the row according to module:db.cleanupResult logic, if not defined this column will be removed from the result, false means do not cleanup explicitly.

Properties

Common pool specific instructions to apply to this column when creating a table

Sqlite specific instructions to apply to this column when creating a table, e.g.

• details: { type: "text", _$sqlite: { sql_extra: "HIDDEN" } }
• data: { type: "obj", _$sqlite: { sql: "@name@ UNINDEXED" } }

PostgreSQL specific instructions to apply to this column when creating a table

DynamoDB specific instructions to apply to this column when creating a table, e.g.

• id: { type: "text", index: 1, _$dynamodb: { projection1: ["name"], global: 1 } }

Elasticsearch properties to apply to this column when creating a table, e.g.

• data: { type: "obj", _$elasticsearch: { mapping: { enabled: false } } }

sorting number, auto created or explicit

HTTP method, empty for all methods

configured path to match

parts of the path

list of params to extract from path

middleware function(context, next)

Properties