Configuration options

This reference documentation details all available configuration options that can be specified in a collection’s configuration file to configure the Open Terms Archive engine, under the @opentermsarchive/engine key, and the issue reporter, under the @opentermsarchive/issue-reporter key.

As an example, see the production configuration file of the Demo collection.

Options

trackingSchedule string

Defines how often the engine should check for changes in terms. Uses standard cron syntax to set the schedule. By default, it runs every 12 hours at minute 30.

Default: 30 */12 * * *

collectionPath string

Path to the collection’s directory containing declarations directory and metadata file, relative to the engine execution location

Default: ./

Example ../collections/demo-declarations

Recorder

The recorder section manages how versions and snapshots of terms are stored, supporting multiple storage backends.

recorder.versions.storage object

Configuration for storing versions. Supports Git and MongoDB. See Storage Repositories for more information.

recorder.snapshots.storage object

Configuration for storing snapshots. Supports Git and MongoDB. See Storage Repositories for more information.

Tracking results

The tracking results section configures the repository in which the engine records the tracking status of each terms and the lifecycle of each tracking run. Tracking results are recorded by default; set tracking-results to null to disable them. To publish them, see how to create the tracking results repository.

tracking-results.storage.type string

Type of storage backend. Only Git is supported, as its history is what makes tracking results tamper-evident.

Default: git

Allowed values git

tracking-results.storage.git.path string

Path to the tracking results repository.

Default: ./data/tracking-results

tracking-results.storage.git.publish boolean

Set to true to push the tracking results to the origin remote of the repository at the end of each tracking run. Unlike versions and snapshots, tracking results are not published by default in production.

Default: false

tracking-results.storage.git.author.name string

Author name for changes.

Default: Open Terms Archive Bot

tracking-results.storage.git.author.email string

Author email for changes.

Default: bot@opentermsarchive.org

Fetcher

The fetcher section configures how the engine retrieves documents from the web.

fetcher.waitForElementsTimeout number

Maximum wait time for elements to appear in a page (milliseconds).

Default: 10000

fetcher.navigationTimeout number

Maximum wait time for a page to load (milliseconds).

Default: 30000

fetcher.language string

Comma-separated priority list of language tags used for the Accept-Language request header and the browser language, e.g. en or en-IE,en-GB,en. Quality factors (;q=…) are not supported.

Default: en

Logger

The logger section configures logging and error notification settings.

logger.smtp.host string

SMTP server hostname.

logger.smtp.port number

SMTP server port.

logger.smtp.username string

Username for SMTP server authentication.

logger.sendMailOnError.to string

Email address for error notifications.

Example admin@example.com

logger.sendMailOnError.from string

Sender email address for error notifications.

Example noreply@example.com

logger.sendMailOnError.sendWarnings boolean

Set to true to also send email in case of warning.

Default: false

logger.timestampPrefix boolean

Set to false to avoid duplicate timestamps if logs are managed by a process manager.

Default: true

Dataset

The dataset section configures how datasets are generated, stored and published. The latest generated dataset is stored locally and exposed through the Collection API. It can also be published to GitHub releases, GitLab releases, and/or data.gouv.fr. If both GitHub and GitLab tokens are configured, GitHub takes precedence.

dataset.title string required

Title of the dataset.

Example Contrib collection dataset

dataset.versionsRepositoryURL string required

Repository URL for dataset releases. Also used to generate the dataset README.

Example https://github.com/OpenTermsArchive/contrib-versions

dataset.publishingSchedule string

Cron expression for dataset publishing. By default, it runs every Monday at 8:30 AM. If publishing to data.gouv.fr, remember to update dataset.datagouv.frequency to match the actual publishing frequency.

Default: 30 8 * * MON

dataset.storagePath string

Path to the directory where the latest generated dataset archive is stored, alongside a metadata.json file describing it. This directory is managed by the engine, which deletes any .zip archive it does not reference at each generation, so it must not be shared with other files. Its content is exposed by the Collection API.

Default: ./data/datasets

data.gouv.fr publishing

The data.gouv.fr section configures publishing to the French government’s open data platform. Either datasetId or organizationIdOrSlug must be configured.

dataset.datagouv.datasetId string required

ID of an existing dataset on data.gouv.fr. Use this to publish to an existing dataset. Either this or organizationIdOrSlug is required.

Example 6914a64b17a0a91bb0a61222

dataset.datagouv.organizationIdOrSlug string required

ID or slug of the organization on data.gouv.fr. Use this to automatically create and publish a dataset. The dataset will be created with the title from dataset.title if it doesn’t exist. Either this or datasetId is required.

Example open-terms-archive

dataset.datagouv.frequency string required

Update frequency of the dataset. Used when creating or updating a dataset on data.gouv.fr. See data.gouv.fr API for all allowed values.

Example weekly

dataset.datagouv.useDemo boolean

Set to true to use the demo.data.gouv.fr environment for testing.

Default: false

Collection API

The collection API section sets the parameters for the API server.

collection-api.api.port number required

Port number for the API server.

Example 8080

collection-api.api.basePath string required

Base path for API endpoints.

Example /collection-api

collection-api.feed.limit number

Maximum number of entries returned by feed endpoints.

Default: 100

collection-api.feed.versionUrlTemplate string

Template for the alternate link of feed entries. %VERSION_ID will be replaced with the actual version ID.

Example https://github.com/OpenTermsArchive/demo-versions/commit/%VERSION_ID

Storage Repositories

The storage repositories section set the parameters for supported backends for storing versions and snapshots, supporting Git and MongoDB.

storage.type string

Type of storage backend.

Default: git

Allowed values git mongo

Git

The Git storage configuration allows to store versions in a Git repository.

storage.git.path string

Path to the versions database directory.

Default: ./data/versions

storage.git.publish boolean

Boolean to push changes to the origin.

Default: false

storage.git.snapshotIdentiferTemplate string

Template for snapshot ID reference. %SNAPSHOT_ID will be replaced with the actual snapshot ID.

Default: ./data/snapshots/%SNAPSHOT_ID

storage.git.author.name string

Author name for changes.

Default: Open Terms Archive Bot

storage.git.author.email string

Author email for changes.

Default: bot@opentermsarchive.org

MongoDB

The MongoDB storage configuration allows to store versions in a MongoDB database.

storage.mongo.connectionURI string

MongoDB connection URI.

Default: mongodb://127.0.0.1:27017

storage.mongo.database string

Database name.

Default: open-terms-archive

storage.mongo.collection string

Collection name.

Default: snapshots

Issue reporter

The following options configure the issue reporter, under the @opentermsarchive/issue-reporter key of the configuration file.

type string required

Type of forge hosting the declarations repository.

Allowed values github gitlab

Example github

repositories.declarations string required

Repository hosting the declarations, in which the issues are created, in the <owner>/<repo> format.

Example OpenTermsArchive/demo-declarations

repositories.versions string

Repository hosting the versions, to link the latest version of the terms from the issues.

Example OpenTermsArchive/demo-versions

repositories.snapshots string

Repository hosting the snapshots, to link the latest snapshots of the terms from the issues.

Example OpenTermsArchive/demo-snapshots

baseURL string

Base URL of the GitLab instance (if applicable).

Default: https://gitlab.com

apiBaseURL string

Base URL of the API of the GitLab instance (if applicable).

Default: https://gitlab.com/api/v4

collectionApi.url string required

URL of the Collection API of the collection, including its base path and version.

Example http://127.0.0.1:3000/collection-api/v1

schedule string

Cron expression of the checks for a new completed tracking run, when run with --schedule.

Default: */15 * * * *

logger.timestampPrefix boolean

Prefix log lines with a timestamp.

Default: true

logger.sendMailOnError object

Set to { "to": "<recipient>", "from": "<sender>" } to send errors by email, which requires the logger.smtp entries and the OTA_ISSUE_REPORTER_SMTP_PASSWORD environment variable.

Default: false

logger.smtp.host string

SMTP server sending the error emails.

Example smtp-relay.brevo.com

logger.smtp.port number

Port of the SMTP server.

Example 587

logger.smtp.username string

Username on the SMTP server.

Example admin@opentermsarchive.org