Neurelo Schema Language (NSL)

Introduction

The Neurelo Schema Language (NSL) is used to define entities, attributes, relationships, and constraints of a system. The language is based on the json-schema spec, with added capabilities to accurately describe the relationships and constraints within the schema.

Concepts

  • Object - An object represents an entity in your data source. Each object consists of a name, a collection of properties, and constraints. Each object maps to a table (e.g. in PostgreSQL or MySQL) or a collection (e.g. in MongoDB) in your data source.

  • Properties - A property represents the fields of an object in your data source e.g. a column in a table (e.g. PostgreSQL or MySQL) or a key in a key:value pair (e.g. in MongoDB). The property definition indicates how a value should be formed and validated, how it maps to the data source object (table or collection) and, optionally, if it represents a relationship to another object.

  • Enum - An enum is a list of named values that can be used to limit the values that can be assigned to a property.

  • Constraints - A constraint represents additional definitions or rules that should be applied to an object, like a multi-property index.

  • Functions - A function can be used when assigning default values to an object’s property e.g. instead of having a hard-coded value, you can rely on your data source capabilities to generate a value for each record.

  • Relationships - A relationship defines how objects interact between them. They are defined on a per-property basis and specify information such as the cardinality of the relation.

  • Inner Object - An inner object is used to define a custom property schema, and can be reused across multiple objects, or even to define properties of other inner objects. Inner objects are not mapped to a table or collection. Currently they are only supported when using MongoDB (embedded documents).

Top Level

The top level of a Neurelo Schema has the following format:

{
  "features": {...},
  "objects": {...},
  "innerObjects": {...},
  "enums": {...}
}

objects must be specified in the top level, however, features, enums and innerObjects are optional.

Objects

Each object is represented by a name and a set of properties:

{
  "objects": {
    "movies": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "name": {
          "type": "string"
        }
      }
    }
  }
}

Optionally, it can include a “sourceName” attribute, that indicates how an object is named in the data source:

{
  "objects": {
    "movies": {
      "sourceName": "films",
      "properties": {...}
    }
  }
}

And also an optional attribute “constraints” to indicate any extra indexes this entity may have in the data source:

{
  "objects": {
    "movies": {
      "sourceName": "films",
      "properties": {...},
      "constraints": [
        {
          "type": "identifier",
          "property": [
            "id",
            "name"
          ]
        }
      ]
    }
  }
}

The above example would create an index using multiple properties - id and name.

Enums

Each enum is represented by a name and a list of string values. Enums can be used to constrain the possible values for a property in a reusable way:

{
  "enums": {
    "Rating": [
      "G",
      "PG",
      "PG-13",
      "R"
    ]
  }
}

Properties

Properties define the values contained in an object e.g. in a column in a table (e.g. PostgreSQL or MySQL) or a key in a key:value pair (e.g. in MongoDB). Each property is defined by multiple attributes. These attributes define how the values for the property should be formed and how they map to the underlying data source.

Most of the properties accept the following attributes:

  • description - optional, string value to add any documentation about the property. Does not have an actual effect on how the property behaves. For example,

    {
      "objects": {
        "movies": {
          "properties": {
            "id": {
              "description": "Unique ID for a movie",
              "type": "integer",
              "identifier": true
            },
            "name": {
              "description": "Name of the movie",
              "type": "string"
            }
          }
        }
      }
    }
  • default- optional, object value that defines either a hardcoded value or a function that generates a default value for the property. For example,

    {
      "objects": {
        "movies": {
          "properties": {
            "id": {
              "type": "integer",
              "identifier": true,
              "default": {
                "function": "autoincrement"
              }
            },
            "name": {
              "description": "Name of the movie",
              "type": "string",
              "default": {
                "value": "Untitled"
              }
            }
          }
        }
      }
    }
  • sourceType- optional, string or array value to specify the underlying data source type for this property. For examples on sourceType, refer to the respective property’s documentation.

  • sourceName- optional, string value to specify how this property is named in the underlying data source. This is particularly useful in MongoDB where [_id field describes a unique objectId](https://www.mongodb.com/docs/v2.4/reference/object-id/), which is the primary key for every document. For example,

    {
      "objects": {
        "movies": {
          "properties": {
            "id": {
              "type": "string",
              "sourceType": "ObjectId",
              "sourceName": "_id",
              "identifier": true
            },
            "name": {
              "type": "string"
            }
          }
        }
      }
    }
  • nullable - optional, boolean value to specify if the property is nullable or not. By default, nullable is set as false which means it’s a required property. Here is example for setting nullable to true -

    {
      "objects": {
        "movies": {
          "properties": {
            "id": {
              "type": "integer",
              "identifier": true
            },
            "name": {
              "type": "string",
              "nullable": true
            }
          }
        }
      }
    }
  • identifier - optional, boolean value to specify if the property should be treated as a primary key in the underlying data source. For example,

    {
      "objects": {
        "movies": {
          "properties": {
            "id": {
              "type": "integer",
              "identifier": true
            },
            "name": {
              "type": "string"
            }
          }
        }
      }
    }
  • unique - optional, boolean value to specify if the property should be treated as a unique key in the underlying data source. For example,

    {
      "objects": {
        "movies": {
          "properties": {
            "id": {
              "type": "integer",
              "identifier": true
            },
            "name": {
              "type": "string",
              "unique": true
            }
          }
        }
      }
    }

Boolean

Required attributes: type

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

default

Default value can be one of:

  • Default true/false value. For instance, "default": { "value": true }.

  • dbgenerated function. Refer to Function section for further details on this.

sourceType

{
  "type": "boolean",
  "sourceType": "Boolean"
}

PostgreSQL

MySQL

MongoDB

String

Required attributes: type

Optional attributes: format, default, sourceType, description, nullable, sourceName, identifier, unique

default

String supports a wide range of defaults:

  • Default string value. For instance, "default": { "value": "SomeValue" }.

  • dbgenerated, auto, cuid, and uuid functions. Refer to Function section for further details on these.

sourceType

{
  "type": "string",
  "sourceType": ["VarChar", 255]
}

PostgreSQL

MySQL

MongoDB

Integer

Required attributes: type

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

default

Default value can be one of:

  • Default integer value. For instance, "default": { "value": 0 }.

  • dbgenerated function. Refer to Function section for further details on this.

sourceType

{
  "type": "integer",
  "sourceType": "SmallInt"
}

PostgreSQL

MySQL

MongoDB

Big Int

Required attributes: type, format

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

By default, Big Int maps to bigint in PostgreSQL, BIGINT in MySQL, and long in MongoDB. See sourceType sub-section for further details.

As big int is a special kind of string, to specify the usage of big int, "format": "bigInt" is required.

default

Default value can be one of:

  • Default big int value. For instance, "default": { "value": 0 }.

  • dbgenerated function. Refer to Function section for further details on this.

sourceType

{
  "type": "string",
  "format": "bigInt",
  "sourceType": "BigInt"
}

PostgreSQL

MySQL

MongoDB

Float

Required attributes: type, format

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

As float is a special kind of number, to specify the usage of float, "format": "float" is required.

default

Default value can be one of:

  • Default number value. For instance, "default": { "value": 0.0 }.

  • dbgenerated function. Refer to Function section for further details on this.

sourceType

{
  "type": "number",
  "format": "float",
  "sourceType": "Real"
}

PostgreSQL

MySQL

MongoDB

Decimal

Required attributes: type, format

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

As decimal is a special kind of string, to specify the usage of decimal, "format": "decimal" is required with "type": "string".

default

Default value can be one of:

  • Default decimal value. NSL matches decimal values with an optional minus sign, allowing for both integer and decimal parts. For instance, some strings that would match are: 123, -45, 0.314, .56, -123.456. A default value can be specified as "default": { "value": 0 }.

  • dbgenerated function. Refer to Function section for further details on this.

sourceType

{
  "type": "string",
  "format": "decimal",
  "sourceType": "Money"
}

PostgreSQL

MySQL

MongoDB

Date Time

Required attributes: type and format

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

For date time, the format supported is date-time. This is derived from RFC 3339, section 5.6, in accordance with the JSON schema standard.

  • date: 4 digit full year “-” 2 digit month “-” 2 digit day

    • For example, 9th January 2023 is represented as 2023-01-09

  • time: partial-time time-offset

    • partial-time - hour ":" minute ":" second

    • time-offset: "Z" / time-numoffset, where time-numoffset is ("+" / "-") hour-offset ":" minute-offset.

    • For example, 4:30 pm with a time offset of 15 minutes is represented as 16:30:00+00:15

  • date-time: date "T" time

    • For example, 12:30 pm on 4th July 1990 can be represented as 1990-07-04T12:30:00Z. Here, the Z suffix denotes a UTC offset of 00:00.

Note that the letters T and Z used in the date/time syntax must always be upper case.

Our APIs currently only accept complete timestamps. This means you must follow the date-time format (yyyy-mm-ddTHH:MM:SSZ) specified above, even if your underlying data source type is just a date or time type.

default

Date Time supports a wide range of defaults:

  • Default date-time value. For instance, "default": { "value": "2024-01-01T12:00:00Z", "format": "date-time" }.

  • now and dbgenerated functions. Refer to Function section for further details on these.

sourceType

{
  "format": "date-time",
  "type": "string",
  "sourceType": "Timestamptz"
}

{
  "format": "date-time",
  "type": "string",
  "sourceType": "Date"
}

{
  "format": "date-time",
  "type": "string",
  "sourceType": "Timetz"
}

PostgreSQL

MySQL

MongoDB

Bytes

Required attributes: type and format

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

As bytes is a special kind of string, to specify the usage of bytes, "format": "bytes" is required.

default

Default value can be one of:

  • Default byte value. For instance, "default": { "value": "V2VsY29tZSB0byBOZXVyZWxv" }.

  • dbgenerated function. Refer to Function section for further details on this.

sourceType

{
  "type": "string",
  "format": "bytes",
  "sourceType": "ByteA"
}

PostgreSQL

MySQL

MongoDB

JSON

Required attributes: type and format

Optional attributes: default, sourceType, description, nullable, sourceName, identifier, unique

As a JSON type can itself support a number of basic types, its type is represented as ["array", "boolean", "integer", "number", "object", "string"]. Furthermore, format is required to be of type json.

default

Default value can be one of:

  • Default JSON value. For instance, "default": { "value": ["apple gardens", "strawberry fields", "vineyards"] }.

  • dbgenerated function. Refer to Function section for further details on this.

sourceType

{
  "type": [
    "array",
    "boolean",
    "integer",
    "number",
    "object",
    "string"
  ],
  "format": "json",
  "sourceType": "Jsonb"
}

PostgreSQL

MySQL

MongoDB

In MongoDB, json is used indicate the flexible property type and that multiple types are supported for that property

Enum

Properties referencing an enum can be used to constrain its possible values to the ones defined in the enum.

The $ref property is used to reference a specific part of the schema by assigning it a value containing the path to the referenced part:

{
  "$ref": "#/enums/Rating",
  "default": {
    "value": "PG"
  }
}

The example above is referencing Rating enum defined in the Enums section.

Optionally, references to enums can define a default value for the property.

Array

Required attributes: type and items

Optional attributes: description and sourceName

An array field can be represented by specifying a "type": "array", followed by the shape of the individual items the array will contain:

{
  "type": "array",
  "items": {
    "type": "string",
    "format": "decimal"
  }
}

Note that sourceType can be specified in the items if necessary.

Relation

A reference in an object represents a relationship between the two of them. The relation property is used to indicate relationships between objects in the schema definition.

The relation property contains the following attributes:

  • attrKey - string or array of strings with the name of the object properties that contain the references to the foreign object.

  • foreignAttrKey - string or array of strings with the name of the foreign object properties that should match the ones defined in attrKey.

  • name - optional, used to disambiguate when multiple relationships between the same models are present.

  • sourceName - optional, the name that should be used in the datasource for the foreign key index

  • onUpdate - optional, side effect to use when a parent model is updated. Defaults to NoAction.

  • onDelete - optional, side effect to use when a parent model is deleted. Defaults to NoAction.

  • The actions supported by onUpdate and onDelete are:

    • Restrict: Prevents updation/deletion if any child model’s records exist.

    • NoAction: Similar to Restrict however the functionality of NoAction depends on the database being used. For instance,

      • PostgreSQL: As mentioned in PostgreSQL docs, NoAction behaves similar to Restrict except NoAction allows the check to be deferred until later in the transaction, whereas Restrict does not.

      • MySQL: For MySQL, [NoAction is equivalent to Restrict](https://dev.mysql.com/doc/refman/8.0/en/create-table-foreign-keys.html#foreign-key-referential-actions).

      • MongoDB: When a parent model’s record is updated/deleted, the child model’s records remain as is.

    • Cascade: Updating/deleting records from the parent model automatically triggers updation/deletion of corresponding records referenced in the child model.

    • SetNull: After updating/deleting record from the parent model, the corresponding records referenced in the child model are set to Null.

    • SetDefault: After updating/deleting record from the parent model, the corresponding records referenced in the child model are set to their default values.

One to One

In the following example we have two objects: authors and author_details. The author_details object contains the relationship definition, as it is the one that has the author_id foreign key property. On the other hand, the authors object only contains a property with items referencing the author_details object.

{
  "objects": {
    "authors": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "details": {
          "$ref": "#/objects/author_details",
          "nullable": true
        }
      }
    },
    "author_details": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "author_id": {
          "type": "integer",
          "unique": true
        },
        "author": {
          "$ref": "#/objects/authors",
          "relation": {
            "attrKey": "author_id",
            "foreignAttrKey": "id"
          }
        }
      }
    }
  }
}

One to Many

In the following example we have two objects: authors and books. The books object contains the relationship definition, as it is the one that has the author_id foreign key property. On the other hand, the authors object only contains an array property with items referencing the books object.

{
  "objects": {
    "authors": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "books": {
          "type": "array",
          "items": {
            "$ref": "#/objects/books"
          }
        }
      }
    },
    "books": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "author_id": {
          "type": "integer"
        },
        "author": {
          "$ref": "#/objects/authors",
          "relation": {
            "attrKey": "author_id",
            "foreignAttrKey": "id"
          }
        }
      }
    }
  }
}

Many to Many - implicit

In the following example we have two objects: books and libraries. A book can be found in many libraries, and each library contains many books. To define this type of relationship, where we are only interested in whether or not the records are related, we can use an implicit many to many definition.

{
  "objects": {
    "books": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "libraries": {
          "type": "array",
          "items": {
            "$ref": "#/objects/libraries"
          }
        }
      }
    },
    "libraries": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "books": {
          "type": "array",
          "items": {
            "$ref": "#/objects/books"
          }
        }
      }
    }
  }
}

Many to Many - explicit

Same scenario as the implicit version, but this time we need to keep track of additional information in the many to many pivot table, like the amount of books each library has purchased.

{
  "objects": {
    "books": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "libraries": {
          "type": "array",
          "items": {
            "$ref": "#/objects/library_books"
          }
        }
      }
    },
    "libraries": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "books": {
          "type": "array",
          "items": {
            "$ref": "#/objects/library_books"
          }
        }
      }
    },
    "library_books": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true
        },
        "book_id": {
          "type": "integer"
        },
        "library_id": {
          "type": "integer"
        },
        "purchased_amount": {
          "type": "integer"
        },
        "book": {
          "$ref": "#/objects/books",
          "relation": {
            "attrKey": "book_id",
            "foreignAttrKey": "id"
          }
        },
        "library": {
          "$ref": "#/objects/libraries",
          "relation": {
            "attrKey": "library_id",
            "foreignAttrKey": "id"
          }
        }
      }
    }
  }
}

Functions

Functions can be used to generate the default value for a property. The supported functions are:

  • auto - Only available for MongoDB. Generates an ObjectId.

  • autoincrement - Generates an automatically incremented integer value.

  • cuid - Generates a string value based on the CUID spec.

  • uuid - Generates a string value based on the UUID spec.

  • now - Generates a Date Time value with the current data source timestamp.

  • dbgenerated - Generates a value by executing the provided “rawDbValue”.

    • rawDbValue’s value is a database function. For instance, in PostgreSQL, gen_random_uuid() returns a random UUID that can be used for primary key values.

Functions can be used as follows:

{
  "objects": {
    "movies": {
      "properties": {
        "id": {
          "type": "integer",
          "identifier": true,
          "default": {
            "function": "autoincrement"
          }
        }
      }
    },
    "actors": {
      "properties": {
        "id": {
          "type": "string",
          "identifier": true,
          "default": {
            "function": "dbgenerated",
            "rawDbValue": "gen_random_uuid()"
          }
        }
      }
    }
  }
}

Constraints

Identifier

An identifier constraint can be used to define a multi-property primary key index for an object:

{
  "objects": {
    "movies": {
      "properties": {
        "name": {
          "type": "string"
        },
        "rating": {
          "$ref": "#/enums/Rating",
          "default": {
            "value": "G"
          }
        }
      },
      "constraints": [
        {
          "type": "identifier",
          "property": [
            "name",
            "rating"
          ],
          "alias": "name_rating_id",
          "sourceName": "name_rating_pk"
        }
      ]
    }
  },
  "enums": {
    "Rating": [
      "G"
    ]
  }
}

The optional “alias” property will be used in Neurelo’s APIs to allow filtering by all the properties of this constraint.

The optional “sourceName” property indicates how this index constraint is named in the data source.

Unique

A unique constraint can be used to define a multi-property unique index for an object:

{
  "objects": {
    "movies": {
      "properties": {
        "name": {
          "type": "string"
        },
        "rating": {
          "$ref": "#/enums/Rating",
          "default": {
            "value": "G"
          }
        }
      },
      "constraints": [
        {
          "type": "unique",
          "property": [
            "name",
            "rating"
          ],
          "alias": "name_rating_id",
          "sourceName": "name_rating_ux"
        }
      ]
    }
  },
  "enums": {
    "Rating": [
      "G"
    ]
  }
}

The optional “alias” property will be used in Neurelo’s APIs to allow filtering by all the properties of this constraint.

The optional “sourceName” property indicates how this index constraint is named in the data source.

NOTE: The “sort” property is not supported within a unique constraint.

Index

An index constraint can be used to define a single or multi-property index for an object:

{
  "objects": {
    "movies": {
      "properties": {
        "id": {
          "type": "string",
          "identifier": true
        },
        "name": {
          "type": "string"
        },
        "rating": {
          "$ref": "#/enums/Rating",
          "default": {
            "value": "G"
          }
        },
        "releaseDate": {
          "type": "string",
          "format": "date-time"
        }
      },
      "constraints": [
        {
          "type": "index",
          "property": [
            "name",
            "releaseDate"
          ],
          "alias": "nameReleaseDate",
          "kind": "BTree"
        }
      ]
    }
  },
  "enums": {
    "Rating": [
      "G"
    ]
  }
}

The optional “alias” property will be used in Neurelo’s APIs to allow filtering by all the properties of this constraint.

The optional “sourceName” property indicates how this index constraint is named in the data source.

The optional “sort” property indicates the order in which the entries of this index will be stored in the data source.

The optional “length” property indicates the maximum length of the value to be indexed. Only available for MySQL.

The optional “kind” property indicates the type of index to be used in the data source. Defaults to “BTree”. Available options are: Hash, Gist, SpGist, Gin, Brin. Only available for PostgreSQL.

Inner Objects

Only available for MongoDB. Inner Objects are defined in an almost identical way as objects, with the difference that the cannot have identifier or unique properties, nor constraints.

Inner objects can be used to describe complex schemas that can be reused across multiple objects properties via references. These properties will be represented in the underlying database as embedded documents.

{
  "objects": {
    "libraries": {
      "properties": {
        "id": {
          "type": "string",
          "sourceType": "ObjectId",
          "identifier": true,
          "sourceName": "_id",
          "default": {
            "function": "auto"
          }
        },
        "public_bookshelves": {
          "type": "array",
          "items": {
            "$ref": "#/innerObjects/bookshelf"
          }
        },
        "private_bookshelves": {
          "type": "array",
          "items": {
            "$ref": "#/innerObjects/bookshelf"
          }
        }
      }
    }
  },
  "innerObjects": {
    "bookshelf": {
      "properties": {
        "width": {
          "type": "number",
          "format": "float"
        },
        "height": {
          "type": "number",
          "format": "float"
        },
        "number_of_shelves": {
          "type": "integer"
        }
      }
    }
  }
}

Features

Features are extensions to the schema.

User Auth

For a guide on using this feature, seeUser Auth. To enable the feature you just have to specify the version (currently "1")

{
  "features": {
    "auth": {
      "version": "1"
    }
  },
  ...
}

Last updated