Schema Usage

Trivial applications are trivially configured; they are not the focus of this library.

Complex applications that are built up from many internal components are much more complicated to configure, because configuration is a cross-cutting concern that can induce leaky abstractions.

Either applications need to know a myriad of internal details of their components (implying that the components should cleanly expose those details, thus making them part of their API surface) or else the components need to know details of their application execution environment (leading to inappropriate inverse coupling and limiting reuse).

This library enables you to formalize your configuration data using composable schemas. Your application components should be able to trust that the configuration data it receives has already been properly transformed and validated.

To enable Configurator to manage configuration, you will use a Schema to define your configuration data model. The Schema will be compiled using a SchemaResolver into aCompiledSchema, which is then used to process and validate configuration data.

To avoid requiring "framework-like" dependencies throughout your application components, the Configurator library allows you to decide how deeply to integrate. You could decide to define the entire Schema hierarchy at the application tier. Alternatively, you could have your components each expose their own schemas, either as actual Schema instances, or just static "schema-shaped" data to avoid introducing dependencies. You could even define your component schemas in external sidecar files.

For the purposes of documentation, the example schemas will almost always be "monolithic" for simplicity, but in most real applications, they should be built by aggregating the individual schemas of each application component.

(The ModuleManager extension library builds on Configurator to provide this structure.) // todo - link

Defining Schemas

The Schema class lets you define your configuration data model. It can be simple:

import { Schema, Configurator } from '@versionzero/configurator';

const schema = new Schema('object')
        .property('debug', new Schema('boolean')
                .meta('description', 'enable debugging')
        )

const configuration = new Configurator({schema}).configure();

console.log(configuration);

// todo - run and document actual results

% node schema1.js --help

% node schema1.js -D

or more complex:

import { Schema, Configurator } from '@versionzero/configurator';

const schema = new Schema('object')
        .property('version', Schema.literal('1.0.1').meta('internal'))
        .property('cache', new Schema('object')
                .required()                
                .property('encrypted', new Schema('boolean')
                        .default(false)
                        .meta('description', 'enable cache encryption')
                )
                .property('key', new Schema('string')
                        .condition((value, configuration) => (configuration.encrypted === true))
                        .required()
                        .unionSchema('keyFile', new Schema('object')
                                .
                        )
                        .serializer(() => '*****')
                        .meta('description', 'encryption key (required only if encryption is enabled)')
                )
                .property('server', new Schema('string')
                        .default('localhost')
                        .validator('$reachable')
                        .meta('flagHint', 's')
                        .meta('description', 'cache server hostname or ip address')
                )
                .property('port', new Schema('number')
                        .default('9002')
                        .meta('advanced')
                        .validator('$port')
                )
                .property('credentials', new Schema('object')
                        .required(false)
                        .meta('description', 'cache credentials')
                        .property('userName', new Schema('string')
                                .required()
                                .meta('flagHint', 'u')
                                .meta('description', 'cache user')
                        )
                        .property('password', new Schema('string')
                                .serializer(() => '*****')                                
                                .meta('description', 'cache password')
                                .meta('flagHint', 'p')
                        )
                )
        );

const configuration = new Configurator({schema}).configure()
console.log(configuration);

// todo - run and show actual output 
% node schema2.js --help advanced

% node schema2.js -u qa -p test123 --cache-encrypted --ck=xyz

% node schema2.js --cache-user qa -cache-password test123 --ce --ck xyz --dump -  

The above examples demonstrate several core concepts of the

Unlike a generalized schema system focused on ontology, the Configurator schema is specialized to the task of enabling user-friendly application configuration. To wit,

The schema hierarchy defines a corresponding configuration model. This configuration model directly mirrors an "ideal" configuration file format for the application, ensuring that you can "round-trip" the serialized model to disk and load it back as an input config file.
Customizable properties need reasonable mappings from sources like command line options and environment variables. For this reason, all configurable data is assumed to potentially start out as a string.
To support multiple independent prioritized sources of configuration assignments, schema conditionals and lazy evaluation are necessary to avoid unnecessary processing related to overridden assignments.
Value normalization, transformation, and validation all support asynchronous processing in order to offload common configuration needs away from application logic.

The Schema class acts as the "schema builder", and is oriented around expressively modeling the configuration structure and constraints.

Schema Concepts

By default, a Schema simply passes any input directly to its output without modification.

const anySchema = new Schema()   // I am not opinionated

At the core of the Schema instance are options that define the schema's processing behavior, and metadata that defines how the schema interacts with users.

Options

Schema provides chainable fluent methods for setting options:

let directionSchema = new Schema().option('required', true)
                                  .option('values', ['north', 'south', 'east', 'west'])

For the most commonly used options, dedicated fluent methods are also available:

directionSchema = new Schema().required()
                              .values(['north', 'south', 'east', 'west'])

// or...

directionSchema = new Schema().required()
                              .value('north')
                              .value('south')
                              .value('east')
                              .value('west')

(See the Schema API for the list of available options. // TODO - link)

The schema's value processing is primarily defined by the normalizer, transformer, and validator handler options:

// Normalizers ensure that an assigned input value is of an expected type for later processors.

const myStringSchema = new Schema().normalizer(value => String(value) );

// Transformers take a normalized input and convert it to the final output format for the configuration.

const myBufferSchema = new Schema().transformer(value => {
  return (typeof value === 'string')? Buffer.from(value, 'base64') : Buffer.from(value)
})

// Validators check whether a configuration value meets all requirements. 

const myNumberSchema = new Schema()
        .normalizer(value => Number(value))
        .validator(value => {
          if (typeof value === 'number' && Number.isFinite(value)) {
            return value;
          }
          throw new Error(`Invalid number: "${value}`);
        })

Most value handlers only require access to the value being processed, but the full handler signature provides access to additional parameters for more complex processing. See the Processing section for details. // TODO - link

Metadata

Similarly, you can also attach metadata to the schema (but only a generic fluent method is provided):

const directionSchema = new Schema().required()
                                    .values(['north', 'south', 'east', 'west'])
                                    .meta('description', 'choose a direction')

The schema itself does not pay any attention to the metadata during processing, it is only used for user interaction.
For example, the description metadata is used by CommandLineSource as part of help text generation.

See the individual ConfigurationSource sections for details on how each particular source uses metadata. // TODO - link

Properties

Schemas define a hierarchy by defining child properties. Schemas are recursive: every property is itself defined by a schema.

// this:
const s = new Schema()
           .property('enabled', new Schema().default('true'))        
           .property('user', new Schema()
             .property('firstName', new Schema().required())
             .property('lastName', new Schema().required())
           )

// could match:
const o = {
  enabled: true,
  user: {
    firstName: 'John',
    lastName: 'Doe'
  }
} 

While it is possible to add options to each schema in this hierarchy to define what is a "valid value" for a given property, it would be better to build upon existing schemas that already provide this functionality.

See the next section on inheritance to learn how to accomplish this. // todo - link!

The Configurator uses the schema for two purposes:

process maps of property path assignments (dotted hierarchical property names with values) produced by configuration sources.
validate a candidate configuration object

Each concept is described separately.

For the following sections, we'll assume we've done this:

import { Schema } from '@versionzero/configurator';

const schema = new Schema('object');

// You can register your own schemas for reuse
resolver.registerSchema('awsCredentials',
        new Schema('object')
                .property('accessKeyId', new Schema('string')
                        .required()
                        .validator(/(A3IA|AKIA|AROA|ASIA|AGPA|AIDA)([A-Z0-9]{16})/)
                        .meta('description', 'Access Key ID')
                )
                .property('secretAccessKey', new Schema('string')
                        .required()
                        .validator(/[0-9a-zA-Z/+=]{40}/)
                        .meta('description', 'Secret Access Key')
                )
);

const schema = new Schema('object')
        .property('region', new Schema('string'))               // built in, but unresolved until compilation! 
        .property('credentials', new Schema('awsCredentials'))  // ...as is this

// You don't need to manually call this, Configurator does it for you:
// const compiledSchema = schemaResolver.compile(schema);

const configuration = new Configurator({resolver, schema}).configure({ default: { accessKeyId: '...', secretAccessKey: '...'}}) 

const requiredHostnameSchema = new Schema('string').required().validator('$hostname')

You can register your own schemas and validators with SchemaResolver for reuse.

Defining Schemas

Schema Concepts​

Options​

Metadata​

Properties​

Schema Concepts

Options

Metadata

Properties