module:api
- Description:
HTTP API to the server from the clients, this module implements the basic HTTP(S) API functionality with some common features. The API module incorporates the Express server which is exposed as
api.appobject, the server spawns Web workers which perform actual operations and monitors the worker processes if they die and restart them automatically. How many processes to spawn can be configured via-server-max-workersconfig parameter.When an HTTP request arrives it goes over Express middleware, but before processing any registered routes there are several steps performed:
- the
reqobject which is by convention is a Request object, assigned with common backend properties to be used later:- user - an empty object which will be filled after by signature verification method, if successful, properties from the
bk_usertable will be set - options - an object with internal state and control parameters. Every request always has an options object attached very
early with some properties always present:
- ip - cached IP address
- host - cached host header from the request
- path - parsed request url path
- apath - an array with the path split by /
- secure - if the request is encrypted, like https
- appTimezone - milliseconds offset from the UTC provided in the header by the app
- user - an empty object which will be filled after by signature verification method, if successful, properties from the
- access verification, can the request be satisfied without proper signature, i.e. is this a public request
- autherization, check the signature and other global or user specific checks
- when a API route found by the request url, it is called as any regular Express middlware
- if there are registered pre processing callback they will be called during access or autherization phases
- if inside the route a response was returned using
api.sendJSONmethod, registered post process callbacks will be called for such response
Every request has the
traceproperty, either fake or X-Ray depending on the config, see metrics for usage- the
Properties:
| Name | Type | Description |
|---|---|---|
port |
int | HTTP port to listen to for Express app |
bind |
string | listen on the specified local interfcae, 0.0.0.0 is default |
Members
(inner) args :Array.<ConfigOptions>
Type:
- Array.<ConfigOptions>
Methods
(static) checkProxy() → {string}
- Description:
Web proxy: checkProxy("web", req, res) WS proxy: checkProxy("ws", req, socket, head)
Returns:
| Type | Description |
|---|---|
| string |
|
(static) checkRateLimits()
- Description:
Perform rate limiting by specified property, if not given no limiting is done.
The following options properties can be used:
-
type - determines by which property to perform rate limiting, when using user properties the rate limiter should be called after the request signature has been parsed. Any other value is treated as custom type and used as is. If it is an array all items will be checked sequentially. This property is required.
The predefined types checked for every request:
-
ip - check every IP address
-
path - limit by path and IP address, * can be used at the end to match only the beginning, method can be placed before the path to use different rates for the same path by request method
-api-rlimits-rate-ip=100 -api-rlimits-rate-/api/path=2 -api-rlimits-rate-GET/api/path=10 -api-rlimits-rate-/api/path/=1 -api-rlimits-rate-/api/path/127.0.0.1=100 -api-rlimits-map-/api/=rate:100,interval:1000
-
-
ip - to use the specified IP address
-
max - max capacity to be used by default
-
rate - fill rate to be used by default
-
interval - interval in ms within which the rate is measured, default 1000 ms
-
message - more descriptive text to be used in the error message for the type, if not specified a generic error message is used
-
queue - which queue to use instead of the default, some limits is more useful with global queues like Redis instead of the default
-
delay - time in ms to delay the response, slowing down request rate
-
multiplier - multiply the interval after it consumed all tokens, subsequent checks use the increased interval, fractions supported, if the multiplier is positive then the interval will keep increasing indefinitely, if it is negative the interval will reset to the default value on first successful consumption
The metrics are kept in the LRU cache in the process by default or in cluster mode in the server process.
-
Example
api.checkRateLimits(req, { type: "ip", rate: 100, interval: 60000 }, (err, info) => {
if (err) return api.sendReply(err);
...
});
(static) checkRedirectPlaceholders()
- Description:
Replace redirect placeholders
(static) cleanupHeaders()
- Description:
For now just convert cookies into an object so logger can process
(static) cleanupResult()
- Description:
Process records and keep only public properties as defined in the table columns. This method is supposed to be used in the post process callbacks after all records have been processes and are ready to be returned to the client, the last step would be to cleanup all non public columns if necessary.
tablecan be a single table name or a list of table names which combined public columns need to be kept in the rows. List of request tables is kept in thereq.options.cleanupwhich by default is empty.By default primary keys are not kept and must be marked with
pubproperty in the table definition to be returned.If any column is marked with
privproperty is never returned.The
options.isInternalallows to return everything except 'priv' columnsColumns with the
pub_adminproperty will be returned only ifoptions.isAdminis truish, same with thepub_staffproperty, it requiresoptions.isStaff.To return data based on the current user roles a special property in the format
pub_typesmust be set as a string or an array with roles to be present in the current usertype`` field. This is checked only if the column is allowed, this is an additional restriction, i.e. a column must be allowed by thepub` property or other way.To retstrict by role define
priv_typesin a column with a list of roles which should be denied access to the field.The
req.options.poolproperty must match the actual rowset to be applied properly, in case the records have been retrieved for the different database pool.The
req.options.cleanup_strictwill enforce that all columns not present in the table definition will be skipped as well, by default all new columns or columns created on the fly are returned to the client.api.cleanupStrict=1can be configured globally.The
req.options.cleanup_rulescan be an object with property names and the values 0, or 1 forpubor2foradmin`` or3forstaff``The
req.options.cleanup_copymeans to return a copy of every modified record, the original data is preserved
(static) createWebServer(port, bindopt, restartopt, sslopt, timeoutopt, keepAliveTimeoutopt, requestTimeoutopt, maxRequestsPerSocketopt, maxHeaderSizeopt, reusePortopt, nameopt)
- Description:
Create a Web server with options and request handler, returns a server object.
Options can have the following properties:
Parameters:
| Name | Type | Attributes | Description |
|---|---|---|---|
port |
int | port number is required |
|
bind |
string |
<optional> |
address to bind |
restart |
string |
<optional> |
name of the processes to restart on address in use error, usually "web" |
ssl |
objext |
<optional> |
an object with SSL options for TLS createServer call |
timeout |
int |
<optional> |
number of milliseconds for the request timeout |
keepAliveTimeout |
int |
<optional> |
number of milliseconds to keep the HTTP connecton alive |
requestTimeout |
int |
<optional> |
number of milliseconds to receive the entire request from the client |
maxRequestsPerSocket |
int |
<optional> |
number of requests a socket can handle before closing keep alive connection |
maxHeaderSize |
int |
<optional> |
maximum length of request headers in bytes |
reusePort |
boolean |
<optional> |
allows multiple sockets on the same host to bind to the same port |
name |
string |
<optional> |
server name to be assigned |
(static) getResultPage()
- Description:
Return an object to be returned to the client as a page of result data with possibly next token if present in the info. This result object can be used for pagination responses.
(static) getTokenSecret()
- Description:
Return a secret to be used for enrypting tokens, it uses the user property if configured or the global API token to be used to encrypt data and pass it to the clients.
-api-query-token-secretcan be configured and if a column in thebk_userwith such name exists it is used as a secret, otherwise theapi.queryTokenSecretis used as a secret.
(static) handleBody()
- Description:
Parse incoming query parameters in the request body, this is default middleware called early before authenticatoion. Only methods in
-api-body-methodsprocessed, defaults are POST/PUT/PATCH. Store parsed parameters in thereq.body.
(static) handleCleanup()
- Description:
Call registered cleanup hooks and clear the request explicitly
(static) handleMetrics()
- Description:
Finish metrics collection about the current rquest
(static) handleMultipart()
- Description:
Parse multipart forms for uploaded files, this must be called explicitly by the endpoints that need uploads.
Example
api.app.post("/upload", api.handleMultipart, (req, res, next) => { if (req.files.file) .... })Another global way to handle uploads for many endpoints is to call it for all known paths at once before the actual upload handlers.
api.app.post(/^\/upload\//, api.handleMultipart, (req, res, next) => (next("route"))); ... api.app.post("/upload/icon", (req, res, next) => { ... api.app.post("/upload/icon", (req, res, next) => {The api module handles uploads automatically for configured paths via
-api-allow-multipartconfig parameter.
(static) init()
- Description:
Initialize API layer, this must be called before the
apimodule can be used but it is called by the server module automatically soapi.initis rearely need to called directly, only for new server implementation or if using in the shell for testing.During the init sequence, this function calls
configureMiddlewareandconfigureWebmethods of all modules.The api uses its own request parser that places query parameters into
req.queryorreq.bodydepending on the method.For GET method,
req.querycontains all url-encoded parameters, for POST methodreq.bodycontains url-encoded parameters or parsed JSON payload or multipart payload.The reason not to do this by default is that this may not be the always wanted case and distinguishing data coming in the request or in the body may be desirable, also, this will needed only for Express handlers
.all, when registering handler by method like.getor.postthen the handler needs to deal with only either source of the request data.
(static) prepareOptions()
- Description:
Parse or re-parse special headers about app version, language and timezone, it is called early to parse headers first and then right after the query parameters are available, query values have higher priority than headers.
(static) prepareRequest()
- Description:
Prepare request options that the API routes will merge with, can be used by pre process hooks, initialize required properties for subsequent use
(static) registerPreHeaders()
- Description:
Register a callback to be called just before HTTP headers are flushed, the callback may update response headers
- callback is a function(req, res, statusCode)
(static) registerRateLimits()
- Description:
Register access rate limit for a given name, all other rate limit properties will be applied as described in the
checkRateLimits
(static) sendFile()
- Description:
Send file back to the client, res is Express response object
(static) sendFormatted()
- Description:
Send result back formatting according to the options properties:
- format - json, csv, xml, JSON is default
- separator - a separator to use for CSV and other formats
(static) sendJSON(req, erropt, dataopt)
- Description:
Send result back with possibly executing post-process callback, this is used by all API handlers to allow custom post processing in the apps. If err is not null the error message is returned immediately with module:api.sendReply.
if
req.options.cleanupis defined it uses module:api.cleanupResult to remove not allowed properties according to the given table rules.
Parameters:
| Name | Type | Attributes | Description |
|---|---|---|---|
req |
object | Express request object |
|
err |
object | Error |
<optional> |
error object |
data |
object |
<optional> |
data to send back as JSON |
(static) sendReply()
- Description:
Send formatted JSON reply to an API client, if status is an instance of Error then error message with status 500 is sent back.
If the status is an object it is sent as is.
All Error objects will return a generic error message without exposing the real error message, it will log all error exceptions in the logger subject to log throttling configuration.
(static) sendStatus()
- Description:
Return reply to the client using the options object, it contains the following properties:
- status - defines the respone status code
- message - property to be sent as status line and in the body
- type - defines Content-Type header, the message will be sent in the body
- url - for redirects when status is 301, 302...
i18n Note:
The API server attaches fake i18n functions
req.__andres.__which are used automatically for themessageproperty before sending the response.With real i18n module these can/will be replaced performing actual translation without using
i18n.__method for messages explicitely in the application code forsendStatusorsendReplymethods.Replies can be delayed per status via
api.delaysif configured, to override any daly setreq.options.sendDelayto nonzero value, negative equals no delay
(static) shutdown()
- Description:
Gracefully close all connections, call the callback after that
(static) startMetrics()
- Description:
This is supposed to be called at the beginning of request processing to start metrics and install the handler which will be called at the end to finalize the metrics and call the cleanup handlers.
(static) toParams(req, schema, optionsopt) → {object|string}
- Description:
Parse body/query parameters according to the
schemaby usinglib.toParams, uses the req.body if present or req.query.
Parameters:
| Name | Type | Attributes | Description | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
req |
object | Express request |
|||||||||||||
schema |
module:lib.ParamsOptions | schema object |
|||||||||||||
options |
object |
<optional> |
Properties
|
Returns:
| Type | Description |
|---|---|
| object | string |
|
Example
var query = api.toParams(req, { q: { required: 1 } }, { null: 1 });
if (typeof query == "string") return api.sendReply(req, 400, query)