Change log of RosaeNLG

LF AI & Data branding

example showcases of the linguistic mechanisms implemented in RosaeNLG:

manages gender for ordinals in French: première and 1re

restaus plural exception in French

some English exceptions: an armlet (not a armlet)

English conjugations corrected: married (not marryed!), disagreed (not disagreeed!)

sonar "code smells" (some of it)

clearer license info in README.md

\#57 highlight js works again, reduced list of supported languages by highlight, and tried to remove favicon requests

\#58 gerund of 'to be' is 'being', not 'bing'

\#55 'to be' formally added to the list of irregular verbs in english-verbs-irregular (though is not not used to conjugate 'to be', as it is an auxiliary)

issues on Lambda version deployment

some security warnings by updating libraries versions

license for documentation files is now CC-BY-4.0

with a fixed random seed, output was not the same whether renderDebug was activated or not

in French, si il becomes s’il (as well as si ils which becomes s’ils, but there is no other case)

bug \#47: dates were not properly protected

configuration and doc for Vercel pkg to generate an .exe file for the server

governance in GOVERNANCE.md

.npmignore cleanup

wrong schema to test input in Lambda

some dependencies update

Italian contractions implemented: di il ⇒ del, su la ⇒ sulla

pug lint of yeoman boilerplate project

some more exceptions on French plurals

in French, support of de vs des when having an adjective: de bons restaurants

specific French plurals module rosaenlg-pluralize-fr forked from pluralize-fr

yeoman generator to easily bootstrap RosaeNLG projects

patreon funding link: https://www.patreon.com/ludan

in API, randomSeed is now automatically present in the output

boilerplate project is archived

new parameter renderDebug to follow the rendering path in the html output; see new debugging manual

performance monitoring in the CI

\#26: valueToSorP(val) was naive; now rules are language specific (in English 0 degrees, while in French 0 degré)

antora-ui, antora-playbook and docs-site, which were used to manage documentation publication using Antora, are now in main monorepo; old separate repos are archived

issue templates on GitHub

standard guides: README.md, DEVELOPER_GUIDE.md, CONTRIBUTING.md, COMMUNITY.md etc.

Observable HQ integration doc

doc on synonyms explaining best approach to generate all alternatives

browser IDE demo: German is blocked when using Firefox (https://github.com/RosaeNLG/rosaenlg/issues/3)

bug on context unstack

big bug on empty synonyms: empty alternatives could be triggered when they contained only spaces

render only browser packages are smaller (about 1/3 gain).

date-fns is now used to format dates instead of moment.

util.moment not available no more.

n2words is packaged more finely to reduce package size.

same as 1.20.1, but as Java version failed I need a new version number

sonar warnings

developer doc to add new languages

new organisation of language specific rules: per language classes vs switch/case everywhere, which will ease the addition of new languages

Java version now supports ouputData

updated Java version:

sonar alerts

github migration:

docker registry moved from registry.gitlab.com to hub.docker.com for cli, node.js server images and java server images

security update of dependencies

license change to Apache 2.0:

documentation in separate module rosaenlg-doc

updated logo (font is Soft Elegance)

code of conduct

bug in browser version of rosaenlg-packager

dependencies update (quite a lot of them)

getHasSaidCopy returns a copy of the hasSaid / recordSaid map as an object

in the API (lambda and node): added a outputData field so that specific computed data can be output by the API, and not only text

dependencies update

ability to complete or override default words and adjective lists in all languages

German dative definite article was wrong: denen instead of den

ROSAENLG_SHARED_DIR on node API to put shared templates in a separate folder

fixes 1.16.7 for DEFAULT_USER

in node API, when not using JWT, it is now possible to indicate which header contains the user

thanks to HB feedback: in English, new key useTheWhenPlural to output the when definite article in plural: #[+value('machine', {det:'DEFINITE', number:'P', useTheWhenPlural: true})] will output the machines

thanks to HB feedback: #[+subjectVerbAdj('machine', 'être', 'prêt', {det:'DEFINITE', number:'P'})] now produces les machines sont prêtes (instead of les machines est prêtes)

JSON schema is now enforced on input for Lambda API

stopword list was broken in the documentation

some errors in the swagger

larger timeout for lambda functions

countries library, for English and French

replaces 1.16.2 which had a little too much agressive contraction mechanism in French; always use 1.16.3 instead of 1.16.2.

in rosaenlg-packager, expandPackagedTemplateJson will transform a JSON packaged project into plain separate files

in swagger, added custom generic field

date contraction issues: de #[+value(new Date('1980-04-14'), {dateFormat:"MMMM"})] did output de avril instead of d’avril (in French)

security update of dependencies

node API and Lambda API: ability to set a user who has shared templates, which can be rendered by anyone

node API and Lambda API: render now also gives templateSha1 in the result

on both node.js and Lambda API:

in the prediction, corrected bug on empty element detection

Lambda API:

node.js API: CORS support

on the node.js API, reload is removed; now just using /templates/{templateId} will always try to reload and give the last SHA1 in the response

on both node.js and Lambda API, /templates/{templateId} will now systematically read on backend (and not use cache)

migration scripts in server-scripts folder

when rendering, in both node and lamba API: if the proper template/SHA1 is not in the cache but is found on backend, with a different SHA1: you now get a 308 redirect instead of an error (error was sha1 do not correspond, read sha1 is …​ while requested is …​)

French: in revamped module french-contractions:

clearer exception when value of something that is null or undefined

numerous tags now accept lists parameters instead of a single element, and will choose one randomly; use that feature instead of an embedded syn_fct call:

noSubject flag on subjectVerb and subjectVerbAdj mixins to silent the subject

verbs and adjectives are now properly extracted from subjectVerbAdj mixin, when embedding resources

there were sometimes inopportune ¤ in the html tags

there were spacing issues when parenthesis and quotes where mixed, like in bla ("bla") bla

Spanish support: verbs, words (gender, plural), adjectives, determiners

SEP_BEFORE and SEP_AFTER to provide separators when multiple adjectives; is practical to override default, e.g. value('árbol', { det:'DEFINITE', adj:{BEFORE:['primero', 'grande'], SEP_BEFORE:''} }) will generate el primer gran árbol

bug in BEFORE adjective list: value('homme', {det:'INDEFINITE', adj: { BEFORE:['vieux'] } }) now produces un vieil homme

bug in Lambda functions: forced embedded resources (in compile options) were ignored

bug in browser-ide-demo: JSON export was wrong (templateId in src and not a top level element)

bug in french-adjectives when adjective was placed before noun and plural: generated vieil alsaciens instead of vieux alsaciens

removed substantive mixin: use value instead with number param, see Value for Nominal Groups

english-ordinals is now a separate module

pluralize-fr removed from tiny runtime (automatically embeds a list of words with their plural instead)

french-adjectives is not embedded no more in tiny version (using lists instead)

doc organization for verb and value mixins

plugged to sonarqube, and thus some code style fixes

browser-ide-demo:

browser-ide-demo

in browser-ide-demo, save to compiled JavaScript with export option

fixed lambda create which did not save on S3 when the input already contained the compiled content

many changes in gulp-rosaenlg:

ability to include raw js files (e.g. include somejs.js), which is handy to be able to both lint and package these files in a json package

clearer exception and message when some raw js in the template is not properly compiled

gulp-rosaenlg now automatically includes all files to be included by reading the templates content

as a consequence, in the packageTemplateJson function:

dependencies update

removed sha1 external lib in favor of integrated crypto

dependencies update

rosaenlg-node-server issue in docker image

deprecated content-deduplicate and removed it from the repo

(almost) ready to deploy Lambda functions for AWS

common content for rosaenlg-node-server and rosaenlg-lambda in separate package rosaenlg-server-toolkit

tested (successfully) on different Node.js versions, see here

in pure rendering packages, essential libs were missing: numeral for all languages and french-adjectives for French, since 1.8.0

deprecated content-deduplicate

bullet lists for itemz and eachz, see doc

dynamic assemblies depending on the real length of non empty elements, also see doc

h1 h2 etc. were not considered as block level html elements

(much) better Italian ordinal support (up to 1 millions) thanks to Marco Riva

in value, use agree for ORDINAL_TEXTUAL in it_IT to have prima and not primo

rosaenlg-node-server: forgot to update docker tests

getRosaeNlgVersion() returns the version of RosaeNLG

rosaenlg-node-server: pre compiled templates are now accepted; when uploaded, templates are compiled only once and saved

rosaenlg-node-server: templates with include in subfolders had issues when importing

changed JSON packaging format, now separates src and comp fields

was able to analyze browser packages thanks to rollup-plugin-analyzer: removed a lot of unuseful stuff (particularly for rendering only packages)

big bug when choosebest was in an itemz structure; error message was choosebest cannot be imbricated

removed 2 patches thanks to n2words 1.3.2

more reliable a / an filtering in English: use of english-a-an and english-a-an-list instead of compromise in rosaenlg-filter

more reliable Englisn plurals: use of english-plurals and english-plurals-list instead of compromise

now using n2words for textual numbers in all languages, replacing written-number for French, write-int for German, compromise for English and italian-ordinals-cardinals for Italian textual

replaced compromise by n2words + custom code inspired by number-to-words for English ordinal numbers

completely removed compromise lib

added rosaenlg-parallel-poc in the repo to ease maintenance

added rosaenlg-browser-poc in the repo to ease maintenance

naming collisions on english-verbs on npm, renamed english-verbs-helper

en_US:

Italian and Other now available on browser IDE

use rollup.js to package (instead of browserify):

removed rosaenlg-pug-filters: brought few features and many security issues

package button in browser-ide

Rapid API documentation

mirror docker node.js server image on aws ecr

rosaenlg-node-server will now return a 404 on render when the template exists but the sha1 is not good

dependancies update

rosaenlg-node-server is now more ready to use on a cluster:

rosaenlg-node-server: lots of changes in the API, especially sha1 of the template is required to render a template (you get the sha1 when you load the template)

rosaenlg-node-server:

rosaenlg-node-server:

do not use 1.6.0 as there was an issue in rosaenlg-node-server: winston must not be devDependancy

rosaenlg-node-server:

rosaenlg-node-server:

rosaenlg-node-server:

rosaenlg-node-server: when using persistent templates, templates were not reloaded when server was restarted

rosaenlg-node-server: http codes are now more precise; like in Return 404 instead of 500 when template does not exist

rosaenlg-node-server: reload now uses PUT http method, and not GET no more

removed snyk dependancy

quotes ("…​") are now properly managed in terms of spacing

removed filter mixin: everything is filtered by default; use protect mixin to protect sections you don’t want to filter

removed disableFiltering option

in rosaenlg-node-server: ability to render a template directly, by sending the template and the data in the same request

the "missing space" issue, which lead to add lines with just | or | #{''}, is fixed (which is indeed a very good thing)

§…​§ structure is deprecated: please use protect tag instead which better cares for spacing

in rosaenlg-node-server: removed counter in the response (didn’t work properly, and was not really interesting)

new additional independant module content-deduplicate; it is not used by RosaeNLG while rendering texts but you can use it afterwards to assess the quality of your texts

in synonym global optimisation: html tags div p etc. are not considered no more as words; this can bring regressions on your projects, as synonyms are better chosen

French is better tokenized when there are apostrophes in the text

dependancies update

list of standard stop words in the documentation

Developer experience documentation

with CLI, templates can now be rendered in stdout (not systematically in a file)

compare with xSpin

examples of templates showing the added value of a Natural Language Generator compared to a plain template engine like Pug

itemz and eachz can be used with parameters: default will be {}

rosaenlg-cli used to output in files by default; now by default it outputs in stdout; add --out=./ to output in files as before

docker images (command line interface and server) are not built through main CI and available here

browser-ide-demo is now a package of the project

rosaenlg-node-server was not able to find openApiDocumentation_merged.json as path was not relative

rosaengl-cli (the command line interface):

rosaengl-node-server (server based on RosaeNLG):

removed @types/random-js dependancy which made a warning

NlgLib is now exposed. Useful when using compileFile (e.g. import {compileFile, NlgLib} from 'rosaenlg'),and mandatory for rosaenlg-node-server.

update of many dependancies

large reorganization of the documentation

more doc on how to integrate, Java Wrapper, Java Server etc.

#[+value('some string')] didn’t work no more

in the browser files supporting compilation (_comp.js), there was only the render function; added compile, compileFile, compileClient, compileFileClient, renderFile and an access to NlgLib

in gulp-rosaenlg, packageTemplateJson to bundle templates and included templates in a single Json file

removed 'fat js' version (was huge, long to produce & not really useful)

fat js is still as fat but at least works now.

fat js packaged version, with compilation ability on each language, to be used in GraalVM

recordSaid hasSaid and deleteSaid existed to safely record booleans; now there is recordValue getValue and deleteValue to safely record numbers or strings

compare with CoreNLG in the documentation

GraalVM runtime documentation

Dev dependancies security issues thanks to Snyk

More French contractions (j’ai, lorsqu’il, puisqu’elle, jusqu’à etc.) thanks to CoreNLG source code

new synonym mode once: will trigger each alternative randomly, but will try to trigger each one only once. Just use it instead of random!

README.md is back for npmjs.com

in French, on browser package, composed tenses raised an error, because verbs that always take ETRE aux were not correctly packaged

reduced size on npm thanks to .npmignore

when using subjectVerb in French, when aux is ETRE, the participle is automatically agreed with the subject

1.0.1 broke gulp-rosaenlg. Removed more deeply dependancy with uglify (which was used by Pug pre-RosaeNLG).

Browser compilation was broken. Fix + added automated tests.

search engine available in RosaeNLG doc thanks to Algolia DocSearch (awesome product and support btw)

outdated packages (yarn outdated) replaced by newer version

1.0.0 version at last!

new logo

open-source publication

nothing as it did never exist

nothing as it did never exist

security fixes: 0 vulnerabilities found - Packages audited: 57380

style for == and !=

usage of recast to parse js expressions when checking for linguistic resources

much better support of html when filtering

removed (some) deprecated libs

compare with CoreNLG (to be enriched with the feedback of the CoreNLG team!)

proper capitalization and spacing when mixin with html bold or italic

removed sqlite dependancy (because it was hard to build & not really useful)

adj in value mixin accepts more: can be a list adj:['beau', 'grand'], or an object adj:{ BEFORE: ['beau', 'intelligent'], AFTER: ['brun'] }

DEMONSTRATIVE determiner type in Italian

possessive adjective param in value possessiveAdj for Italian possessives

boilerplate documentation see boilerplate project

renamed for freenlg to rosaenlg

Indefinite articles in German simplified syntax <…​> parsing

Italian verbs

French past participles used as adjectives

Italian past participles and present participles used as adjectives

German present participles used as adjectives

better Yseop templates generation

first support of Italian (no verbs)

basic support on any language

INDEFINITE determiner type in German

Non-breaking space in French punctuation (before : ! ; and ?)

de #[+value(9000)] (or any number) would contract into d'9 000. Now generates de 9000 properly.

Punctuation on French and English when using ? or !.

French possessives issues.

Added dates formatting thanks to moment lib

Filtering can be explicitely disabled with disableFiltering: true in renderFile

Numbers formatting: use FORMAT in value to set a format directly used by numeral. See numeral.js formats. Very practical for currencies, %, etc. For instance +value(104000, {'FORMAT': '0a$'}) will output 104k€ when generating French.

det to add a determiner (French and German); current determiners are DEFINITE and DEMONSTRATIVE.

It is now easier to complete the params object with new values with addToParams: addToParams({xxx:yyy}) is equivalent to Object.assign({}, params, {xxx:yyy}.

adj property in value to add an adjective.

owner property in value to manage possessives. Does the same thing as thirdPossession.

recordSaid and deleteSaid do not need a `- ` before them no more.

Ability to globally choose the best synonymic alternatives with choosebest: see [choosebest].

Ability to compile and/or just render texts in browser, without using node.js. See [inbrowser].

Tutoriels can be run directly in the browser.

Improved French support, see below.

Partial support of German (de_DE), see below.

Ability to generate Yseop templates. Yseop is a NLG software vendor. See [yseop.adoc].

determiners and possessives (thirdPossession)

Some speed optimizations (which are not significant)

Refactoring: switched to TypeScript for most of the code

Some renaming: please impact your templates

Updated librairies and linguistic resources documentation: see index.adoc#resources

Replaced better-title-case, which was deprecated on npm, by better-title-case. Changed the test case (titlecase rules seem to be complex).

monthName(date) (which generated the name of the month january february etc.) is deprecated as we now have moment lib. Use #[+value(date, 'MMMM')] instead.

Removed format-number-french (replaced by numeral).

Removed jslingua (replaced by custom lefff extract).

valWithUnit is deprecated and removed.

loadDict param is removed; ressources a just loaded when necessary.

Removed registerSubst. Use setRefGenderNumber directly.

updated many libs: copyfiles mocha rimraf better-sqlite3 compromise moment written-number

no changes - just fixin Lerna

performance documentation

Sub modules are now referenced via their exact version number (no more ^…​). It will be easier to use fixed versions of RosaeNLG (with their dependencies) in the future.

French support for month names did not work - fixed

getNextRep (used in French) didn’t work properly: the chosen representant was often not the one that getNextRep had predicted.

anaphoras: after forcing referential representant output, the next one was still the representant; now it is the anaphoric one.

mostly refactoring

syno_sentences mixin deprecated, use synz > syn structure

assemble mixin deprecated, use itemz > item structure

global filtering is done automatically unless any filtering was been done during generation via filter mixin

noFilter parameter is deprecated

French support for months

refactoring, no feature change

pug render did not work - fixed.

doc is now also generated in one single large PDF file

to raise the probability of a specific synonymic alternative to be triggered, use syn {weight:4} (4 times higher chances - default is 1)

to force a specific synonym to trigger, use synz {force:3} (to trigger the 3rd one)

synz 'sequence' syntax is not valid no more: use synz {mode:'sequence'}

changed the random algorithm: it is still random but does not provide exactly the same numbers as before; you shall update your regression tests (and I’m sorry for that).

defaultSynoType renamed to defaultSynoMode

setSynoProps, setSize, getSize removed

semicolon (;) support in punctuation (and taking care of HTML entities which contain a semicolon)

spaces are automatically added before and after various mixins

titlecase is now a structure element and no more a mixin:

FYI the titlecase npm package, which is used in en_US, is no longer maintained

eachz structure as (preferred) an alternative to the foreach mixin:

main.pug is included automatically so do not include it yourself explicitely in your templates no nore.

new protect structure tag that acts like protectString, to exclude some text from the filtering process:

path issues to get RosaeNLG’s mixins

align versions between rosaenlg and rosaenlg-core

wrong French resources path

wrong path in include node_modules/rosaenlg/main.pug

easier integration:

automatic filtering unless noFilter is set to true

new boolean parameter noFilter if don’t want to automatically filter the whole outputs

new structure keywords:

a could be transformed into the by compromise in some situations

When listing elements in a foreach or in an assemble, the list of the non empty elements is made available in params.nonEmpty.

a/an specific cases, when a is:

…​ is automatically transformed into …

Proper punctuation around …: no space before and after etc. No automatic capitalisation after …, as it depends of the context.

mixin eatSpace that 'eats' the spaces around him.

a/an when uppercase: A apple now outputs An apple

missing French contractions when uppercase: De les→`Des`, De le→`Du`

in assemble or foreach with paragraph: true: paragraph parameter is deprecated, use either:

in assemblies, when mode is sentences or paragraphs:

a first version of the tutorial Tutorial

in assemble, when the mixin is not set, will automatically try to use value

in assemble and foreach: shuffle was done too late (after evaluation)

filter: remove spaces at the very end of the text

filter: capitalize the very beginning of the text

filter: capitalize after exclamation mark

filter: removes extra ponctuation when "!" and "." are combined

A shy start of French conjugation with jslingua lib. No support for group 3 verbs.

formatting of numbers in French in value thanks to format-number-french lib

plural of words in French: œil ⇒ yeux etc. thanks to pluralize-fr lib

title case (which is more difficult than it appears) in English thanks to titlecase lib, in French thanks to titlecase-french lib

</p> . </p> was not properly filtered

synonym mode is now a global parameter (random - default, or sequence), via defaultSynoType in the constructor ; can be overridden locally using setSynoType; see mixins_ref.adoc#synonyms_mode

a / an transformation was triggered in all languages, fixed to English only

a/an in English: a industry ⇒ an industry using compromise lib

function isNotEmpty() is deprecated because it brought side effects.

bad prediction in assemble and syno_sentences due to test context that was too narrow - see listOfParaTricky in test_assembly_para

uses compromise lib in en_US which was the basis for multiple new features in English:

direct access to compromise lib via util.compromise

empty prediction mechanism improvements:

randomSeed param of NlgLib constructor renamed to forceRandomSeed

list of words with their gender in French

french adjectives agreement rules (well, most of it)

adds spaces in assembly separators automatically if the user does not put them

notion of language: fr_FR or en_US