Programming Reference

This document covers each of the core components that constitute a skill. It is important to note that EBA follows an ontology based paradigm, meaning that it derives understanding about the world based on a relationship of concepts. This is in contrast to most other systems which are intent based and depend on predicate logic.

Ontology

At a high level, EBA is composed of an ontology of concepts and attributes that it is able to recognize and reason about. Natural language patterns help the assistant to perform natural language understanding, linking language tokens to appropriate concepts. In the process of reasoning about a question, the system considers all possible outcomes it can take given a set of actions and rules, ultimately selecting the best one or asking for user disambiguation in cases where it is fitting. You can learn more about each of these components as well as a few of our core concepts.

WARNING

Add a bit about agents (its always the obvious element that gets left out of discussion...) because it shows up in core concepts

We begin our understanding of ontology by first looking at the notion of a concept.

Concepts

A concept is simply an object in your domain that the assistant is able to recognize. For example, in the marketing domain, we have concepts for mailings, open-rates, click-rates, etc. A concept in denoted by a colon :. For example :Mailings denotes the mailings concepts. Concepts should additionally be prefixed with the domain of your skill. For example, marketing:Mailings signifies the mailings concept in the marketing domain.

Concepts and their relationship to other concepts are defined in an ontology. In fact, an ontology is simply a set of relationships between different concepts in rdf format, i.e. subject-predicate-object triples. Consider the following examples below.

:OpenRate subClassOf :Showable

Here we specify that :OpenRate is a concept which subclasses or derives from the :Showable concept. This system is able to visually render all :Showable objects, so we are effectively making our open rate concept viewable to the end user.

:Mailings isListOf :Mailing

Here we specify that the :Mailings concept is a list composed of individual :Mailing concepts. This is useful for actions that the assistant performs when working with collections.

Certain native predicates have significant meaning for our system. We will detail a few of them below.

subClassOf -- a subClassOf b means that a inherits all of the attributes which b contains. This is particularly useful in the cases that we need to model a hierarchical domain. For instance, :SalesOrder subClassOf :Order allows a particular entity to auto derive attributes from its base concept.
isListOf -- a listOf b means that a is a collection of b. This enables our system to automatically perform data operations on top of collections. For instance, :Orders isListOf :Order and :Order hasAttribute :Quantity, we immediately begin to ask questions such as 'show me all orders where quantity is above 500'.

Attributes

In EBA attributes are modeled via concepts, ontology, and semantic actions. Most of these components are auto-generated by EBA when an agent is loaded. For a developer there are only a few relations needed to be defined in the ontology:

An attribute concept has to be subclass of :Attribute;
An attribute concept has to be associated with an entity concept by defining an accessor (JSON field name).

All of this ontology definitions can be done at "Concepts" tab in EBA Dev Lab.

EBA handles attributes on the assumption that entities are represented by JSON objects and collections are represented by JSON array of JSON objects. EBA only supports primitive types as attribute values. To enable type-specific OOB semantic actions (like ranking for numerical attributes, fuzzy search for string attributes) you need to define additional ontology:

A numeric attribute concept has to be subclass of :NumAttribute;
A string attribute concept has to be subclass of :StrAttribute;
A date attribute concept has to be subclass of :DateAttribute.

Autogenerated concepts

For every attribute EBA automatically generates all the neccessary components (concepts, ontology, and semantic actions). You may use some of these autogenerated concepts in your semantic actions or visualizers if necessary. If we define the attribute example:Price as an attribute of example:Product EBA will generate the following ontology:

example:Product hasAttribute example:Price - this relation may be useful in polymorphic semantic actions or rewriting rules;
example:PriceValue, example:PriceValues isValueOf example:Price - for every attribute additional concepts are generated which represent actual values of attributes. EBA distinguish singular and plural forms of attribute values. You should refer to these concepts when you define custom visualization for your attributes;
attr:ProductPrice attributeOf example:Product, example:Products - for every attribute - entity relationship EBA generates an auxiliary concept to hold a JSON field name. These auxiliary concepts are used in OOB semantic actions to sort, filter or rank collections.

You can share attribute concepts among different business entities if necessary. For example you can define the attribute example:Address and then make it as an attribute of both example:Customer and example:Supplier. EBA will generate two auxiliary concepts to distinguish these attributes: attr:CustomerAddress and attr:SupplierAddress.

Core built-in concepts

EBA is an ecosystem of domain specific agents. In order to encapsulate and expose common functionality across our system, EBA contains a set of prebuiilt native agents. These agents, like any other agents, implement a set of concepts which can be utilized and extended by other agents within the ecosystem. In this article, we aim to highlight the key set of concepts which developers may want to familarize themselves with in order to better understand the generic functionalities they have available. The catalog below is not meant to be exhaustive, and, as with all our agents, you can view their further details within our dev lab at eba.ibm.com.

User interface concepts
Action concepts
Primitive data concepts
Conditional concepts
Aggregation concepts
Qualifier concepts
NLG concepts

User interface concepts

Concepts which represent a high level interface for capturing the role of your concept, e.g. a textual message or a showable data element.

:Message -- any subclass will be treated as type of message where underlying textual data is propagated as a response to the user. These concepts will be treated as directives to interact with the user rather than as composable data elements. This is particularly useful in cases where you want to interact with the user but not return any associated data, e.g. ns:DeleteProduct subClass :Message can enable the textual response 'I have deleted your product named abc' when data ns:DeleteProduct is produced.
:Showable -- any subclass will be treated as a showable entity. This entity can be visualized using our standard assets and displayed to the user in our chat, graph, and content views. If a entity is not :Showable, then it will not recieve a high score when used in conjunction with :ActionShow.

Action concepts

All action concepts are a subclass of :Message as they reprsent operations by the machine which require user interaction, e.g. show data elements or remove data elements will produce the appropriate message to the user. The following concepts are analogous to CRUD operations, all which are supported by our system.

:ActionCreate -- concept which represents a user's request to have data created. It will create data and post a message to the user.
:ActionShow -- concept which represents an user's request to have data shown. It will produce :Showable data.
:ActionModify -- concept which represents a user's request to have data updated. It will modify data and post a message to the user.
:ActionDelete -- concept which represents a user's request to have data removed. It will remove data and produce a textual response to the user.

Primitive data concepts

These concepts represent the basic types of data which our system provides OOB.

:UserString -- underlying data within quoted strings.
:Number -- a numeric value.
:Timeframe -- a timeframe element containing information such as the start and end of the timeframe as well as its level of grainularity.
:FreeText -- textual input form user's question which is not consumed by any other actions within a particular variant. This concept can be useful in cases where more free-formed input is expected.

Conditional concepts

:Contains -- condition which signifies that source value contains target value.
:StartsWith -- condition which signifies that source value starts with target value.
:EndsWith -- condition which signifies that source value ends with target value.
:Like -- condition which signifies that source value matches target value in a fuzzy search.
:GreaterThan -- condition which signifies that source value is greater than target value.
:LessThan -- condition which signifies that source value is less than target value.
:Equivalent -- condition which signifies that source value is strictly equal to the target value.

Aggregation concepts

These concepts represent aggregration and modifier operations on top of collections. Typically these concepts will be used in conjunction with certain predicates and collections, e.g. :SortedBy(:Quantity, :SalesOrders).

:TopN -- gets the top N elements within a collection
:SortedBy -- sorts a collection by a criteria
:Filter -- filter a collection
:First -- get the first element in a collection
:Largest -- get the largest element in a collection
:HighValue -- gets elements within a collection within the highest quartile
:LowValue -- gets elements within a collection within the lowest quartile
:Average -- get the average value across a collection
:Total -- get the sum total value across a collection
:Minimum -- get the minimum element within a collection
:Maximum -- get the maximum element within a collection
:MinimumBy -- get the minimum element by a particular predicate within a colleciton
:MaximumBy -- get the maximum element by a particular predicate within a colleciton

Qualifier concepts

These concepts represent certain language qualifiers for designated entities. Often times these qualifiers can be ommitted in natural language, and, consequently, they are often denoted as optional when used within semantic actions.

:WithName -- qualifier which references a value as being the name of an entity, e.g. 'show product named iPhone X.
:WithId -- qualifier which references a value as being the id of an entity, e.g. 'show product id 123.
:Relation -- qualifier which explictly denotes the relationship between two entities, e.g. show me contacts in this org.
:All -- qualifier which explictly denotes that all entities should be qualified, e.g. 'show me all sales orders'.
:Own -- qualifier which explictly denotes entities belonging to the user, e.g. 'show me my contacts'.

NLG Concepts

These concepts pertain to extending custom NLG capabilties.

nlg:PostModifier -- denotes that the spelling for a particular concept operates as a post modifier, e.g. 'show me complaints against this product'.
nlg:PreModifier -- denotes that the spelling for a particular concept operates as a pre modifier, e.g. 'show me late sales orders'.
nlg:PlainText -- used within NLToken interface to denote a plain text entry, where the data supplied to this concept will be spelled as is.

Patterns

Patterns are simply natural language text samples annotated with concepts. When the assistant receives a question from the user, it is able to tokenize and parse this input into a tree. By using natural language patterns, it is able to annotate this tree with the appropriate concepts. For example consider the patterns below.

show [trending](wmt:Trending) [products](wmt:Products)

This pattern tells the system that trending and products tokens correspond to the concepts wmt:Trending and wmt:Products respectively. From now on, the system will recognize and consider "trending" and "products" appropriately for all further input. Additionally, it is able to capture the syntactic part of speech tree associated with these tokens.

Best practices

It is best to enter patterns in a way that logically separates concepts. For instance, show me [product id](:ProductId) actually annotates two concepts as one, viz. "product" and "id". The better practice is to enter patterns for id and patterns for product separately, since they are actually two different concepts, viz. [id](wmt:Identifier) and [product](wmt:Product). In applications with multiple business entities and multiple ways to refer to these entities, you will find yourself entering many unnecessary patterns if you do not separate them appropriately.

Also note that individual patterns should not be placed within one entry, e.g. [\#](:IDSign), [id](:IDSign), [no](:IDSign), [no.](:IDSign), [num](:IDSign) will represent :IDSign as being equivalent to "# id no no. num". Rather, these patterns should be entered individually as a way to create equivalent references to the :IDSign concept.

Rules

Rewriting rules are a way to transform one single concept into a cluster of other related concepts. Each skill may contain a set of rules, where each rule is designated by a set of constraints, input and output. Often times natural language can be very short and allow for the omission of certain words or concepts. Rewriting rules enable the assistant to handle such cases effectively. While our concepts are well formed, we anticipate that users will use imprecise language to inquire about them. For example, we might expect a user to ask for "the best mailing" or "trending products". We can implement rewriting rules to reduce higher-level notions such as "best" and "trending" into a cluster of concepts that the assistant can more easily reason about. For example consider the rewriting rule below.

:TheBest(:Mailing) -> :HighValue(:ClickToOpenRate, :Mailings)

Here we have defined the best mailing to be translated into lower level concepts. :HighValue is a concept used by the system to return highest quartile data points and :ClickToOpenRate is an attribute of :Mailing. We are effectively programming our assistant to recognize "the best mailings" as being equivalent to "mailings with clickToOpenRate attribute in the highest quartile".

Constraints

Constraints are simply a way to qualify your concepts using rdf triples. As in the case of ontology, constraints follow the format of subject, predicate, object, where the subject and object must be either a concept or another symbol and predicate can be anything as long as it is used consistently within your configuration. Constraints are used within EBA to denote a polymorphic parameter. For instance, a subClassOf :List constrains the symbol a to be any list. Constraints only hold scope local to the signature they are defined in. Within our Lab, a warning will be issued if a constraint is defined but never actually used within the remainder of the signature. Likewise, a warning will be issued a constraint symbol is used within the signature but never declared. Hence, a subClassOf :RankingMetric => :Mailings -> data :Mailings contains a superfluous constraint, while :Mailings(a) -> :Mailings contains an undeclared symbol a. A valid signature would be the following a subClassOf :RankingMetric => :Mailings(a) -> data :Mailings. Additionally, constraints may make use of a wildcard symbol to denote that it accepts any type, e.g. a isListOf _ denotes a constraint a that is a list of anything.

In certain cases, it is possible to refer to a constraint parameter multiple times within a single signature, so it is necessary to distinguish between these references. For instance, a subClassOf :Showable => a(reference:Direct, context data a @src) makes uses of the constraint parameter a twice. To distinguish the second occurrence of the parameter, we add an alias: @src. Now, within the body of our action, we can require data by indexing our deps as "src".

Input and output

Both input and output to rules are represented as a tree of concepts. There are no paramTypes or queryTypes associated with these concepts. A rule will effectively translate the input (higher level) into the output (lower level).

Actions

Actions are the means by which the assistant can provide real data to corresponding concepts. Each skill will contain a set of actions, where each action is designated by a set of constraints, input, and output. An action's signature follows the format constraints => input -> output. Given an configuration of input concepts subject to certain constraints, the system will be able to produce the specified output after executing the body of the action. In addition to this signature, an action will have a name as well as a function to be invoked when the action is selected. For example consider the action below:

:Products(:Trending) -> data :Products

Note that this action does not have any constraints - constraints are optional. This action effectively states that if the assistant recognizes a :Products concept as well as a :Trending concept, then the system can produce real data for the :Products concepts. The data will be produced by the body of this action, which can query a database, call an api, etc.

Actions can also take data provided by other actions:

:Products(data :Category) -> data :Products

This action will expect a data node :Category which has to be created by another action. The action can be triggered once the user asks for products by category, for example:

Q: show me products for category "Jeans"

Inside the body you can access this data using params helpers:

const eba = require("eba")
module.exports.main = async function(params) {
	let p = new eba.Params(params)
	let category = await p.get(":Category")
	let products = ... get products by category here
	return new eba.Result().setData(":Products", products)
}

To get a category by it's name the following action can be used:

:Category(data :UserString) -> data :Category

Processing of quoted strings is available out of the box. Each quoted string will be annotated with :UserString concept and the corresponding data will be created. So you can just get this data in your action body:

let categoryName = await p.get(":UserString")

To clarify the sense of some input parameters we can add auxiliary concepts to the signature:

:Category(:WithName(data :UserString)) -> data :Category

In this case the action will be triggered ONLY when :UserString node has an auxiliary neighbor concept :WithName as in questions like:

Q: show me products for category named "Jeans"

Q: show me products for category with name "Jeans"

The auxiliary concept can be marked as optional:

:Category(optional :WithName(data :UserString)) -> data :Category

This action can be triggered in both questions with or without the concept :WithName.

The data parameters can be optional too:

:Products(optional data :Category) -> data :Products

In this case we can return products related to a certain category if we have category data or all the products otherwise.

The input parameters can be implicit:

:Products(implicit data :Category) -> data :Products

This action can be triggered without the :Category concept in the question thread (an optional parameter) but the agent will still be able to search for the :Category concept data in the context. For example the following scenario will work with implicit :Category parameter:

Q: show me category "Jeans"

A: Agent will get the category "Jeans"

Q: show me products

A: Agent will get products for category "Jeans"

If no concept found in the context the agent will try to recover this concept data using available actions so we will be able to ask questions like show me products for "Jeans".

Constraints

Constraints are simply a way to qualify your concepts using rdf triples. As in the case of ontology, constraints follow the format of subject, predicate, object, where the subject and object must be either a concept or another symbol and predicate can be anything as long as it is used consistently within your configuration. Constraints are used within EBA to denote a polymorphic parameter. For instance, a subClassOf :List constrains the symbol a to be any list. Constraints only hold scope local to the signature they are defined in. Within our Lab, a warning will be issued if a constraint is defined but never actually used within the remainder of the signature. Likewise, a warning will be issued a constraint symbol is used within the signature but never decalred. Hence, a subClassOf :RankingMetric => :Mailings -> data :Mailings contains a superfluous constraint, while :Mailings(a) -> :Mailings contains an undeclared symbol a. A valid signature would be the following a subClassOf :RankingMetric => :Mailings(a) -> data :Mailings.

ParamType and queryType

Concepts within a signature can be qualified with additional specifications to apply different semantics to your action. These specifications are applied through the use of paramTypes and queryTypes. paramTypes qualify the parameters within a given signature, while queryTypes qualify the actual query.

We support three paramTypes, viz. concept, data, and promise. concept is the default paramType and no keyword is required to denote it. It simply signifies that a signature requires a given concept to be present in the parse tree. data denotes the real data associated with an action (which is produced by another action). For instance, edi:Submit(data edi:Invoice) denotes a submission request that requires real invoice data. promise is used to denote a concept which we intend to cover with real data once some further preliminary data is acquired, e.g. promises to return weather data once the user's geo location is ascertained.

We support four queryTypes, viz. regular, optional, context, and implicit. regular is the default queryType and no keyword is required to denote it. It simply signifies that a query matches a concepts in the current question as expected. optional qualifies the query to accept an optional parameter, e.g. :Mailings(optional :All) means that our action can support questions such as "show me all mailings" or simply "show me mailings". context means that our query can search the conversational context for a parameter. For instance, edi:Modify (context data edi:Invoice) is an action which modifies an already invoice. implicit is like context but different insofar as it gives the assistant license to create new concepts through the use of other actions before executing the current action. For instance, if we have one action as :SendVolume(optional :Relation(implicit data :Mailings)) and another action as :Mailings -> data :Mailings, then, when answering a question about send volume, the assistant may execute the mailings action first in order to have data to complete the action for send volume. Mailings nodes will be tagged within the information space as virtual. Because implicit actions may create additional concepts as a side effect and thus significantly alter the complexity of answering a question, they should be used judiciously. In fact, you should try to avoid implicit parameters whenever possible. Both context and implicit should only be used to qualify data nodes.

Input

An action's input is represented as a tree of concepts. These concepts must be matched against in order for the action to be selected for execution. The concepts in this tree may be qualified with paramTypes and queryTypes. A common usage of paramTypes within input is to specify that an action requires real data. For instance, sc:Order(sc:Identifier) will fire when the system notices a request such as "show me order #1234", however, it will not supply the real data associated with the identifier, viz. 1234, to your action since it is only requiring the concept sc:Identifier to be present. To make use of this real data, i.e. to signify that sc:Idenifier is a parameter with data, input should be denoted as sc:Order(data sc:Identifier). A common usage of queryTypes within an input is to denote that a parameter may be taken from already existing context. For instance, :Confirmation(context data expo:DeleteProduct) requires real data associated with a deleted product to already exist within the context of the conversation--which makes sense given that this action is a confirmation request. As you can see from this example, both paramTypes and queryTypes can be used in conjunction with one another when defining an input signature.

Since input is represented as a tree, an ordering is imposed on the concepts. For instance, :News(:Relation(:Organization)) -> data :News implies the following ordering "news" -> "related to" -> "organization". EBA is able to understand such inherent dependencies when it builds the syntax tree for your question, enabling it to distinguish between a similar question such as "organization" -> "related to" -> "news".

Output

Whereas input was a tree, output is simply a flat list of concepts. These concepts can only be qualified with paramTypes. Most often the output of an action is strictly a single concept qualified as data, e.g. -> data Mailings.

Promises

As described in our action component, EBA supports a parameter type called promise, which enables data to be produced for an action only once some prelimenrary data is first acquired. This is typically accomplished by the use of follow up questions in a dialog series. For a live example of this feature, you may try out our Weather agent. In this article, we will walk you through the steps of implementing a few promise dialogs, using our Weather agent as a reference.

Asking for follow-up information

Let's say that you have a business entity which requires, as a necessary condition in order to fetch the entity, another data element(s). In our case, this is weather:Weather which requires both a city as well as a timeframe.

Or, let's say that you are resolving a user's question and the runtime execution indicates an ambiguous case which needs to be resolved by a follow up? For example, if the user asks for a certain entity but you need to establish which type or which class it should be belong to. In these cases, a follow up dialog can be used to resolve any disambiguation.

Of course, in all these cases a user can ask a fully qualified question, in which case no follow up dialog is required, e.g. 'show me the weather in San Fransisco two days from now'. However, we do not anticipate or assume that this will always be the case, and, as a result, we enable developers to handle less qualified requests for data.

To handle less qualified requests, the following strategy can be employed. When a reference to your entity occurs, you can implement an action which returns a promise node as output, rather than a data node. For instance, weather:Weather -> promise weather:Weather signifies an unqualified case (the users asks something general as 'what is the weather?'). We cannot produce data for weather:Weather until we first obtain a location. The code for such an action can look something like the following:

const {Result} = require('eba')

module.exports.main = (params) => {
  return new Result()
    .setName('weather:Weather', ':Question')
    .setData('weather:Weather', 'Which location should I get the weather for?')
}

From this code, we note two things. First, we rename our concept to :Question using the setName api detailed in our node helpers. Secondly, as data, we supply the follow up question itself. With these two elements in place, our system will be able to understand that weather:Weather cannot be resolved in the current execution; instead, it should display a follow up question.

Note that more complex cases can be modeled, e.g. our weather:Forecast requires both a timeframe as well as a location in order for data to be produced. We can either model this a 3-step sequence, viz. ask for forecast, ask for location, and, finally, ask for timeframe. Or we can model it as 2-step sequence, viz. ask for forecast and timeframe or location and then ask for the remaining element. Or we can be flexible enough to support both variants. This will depend largely on the agent's business requirements. In our implementation of weather:Forecast, we have followed the second route. Accordingly, you will find the following signatures in our implementation:

weather:Forecast(optional :Relation(data :Timeframe) -> promise weather:Forecast
weather:Forecast(optional :Relation(city)) -> promise weather:Forecast

You will find that the body of such actions all follow the similar pattern described above, viz. rename to :Question with supplied NL question.

Generating data

With a promise node in place, our system will attempt to resolve it, i.e. to convert it to a data node, whenever it is suitable. To enable such behavior, we should implement an additional action to produce real data. Of course, this action will contain as input all concepts required to resolve the entity at hand. In the case of weather, we have an action as weather:Forecast, we have an action weather:Forecast (optional :Relation (data city), optional :Relation (data :Timeframe)) -> data weather:Forecast. Note that this action requires both city and timeframe data and the output produces data weather:Forecast, meaning that it can provide the data which the user initially requested. The body of this action will use the supplied input to perform any api calls necessary. All actions for the weather agent as viewable to our users. Feel free to take a look and explore this feature in your own implementations.

# Programming Reference

# Ontology

# Concepts

# Attributes

# Autogenerated concepts

# Sharing the attributes

# Core built-in concepts

# User interface concepts

# Action concepts

# Primitive data concepts

# Conditional concepts

# Aggregation concepts

# Qualifier concepts

# NLG Concepts

# Patterns

# Best practices

# Rules

# Constraints

# Input and output

# Actions

# Constraints

# ParamType and queryType

# Input

# Output

# Promises

# Asking for follow-up information

# Generating data

Programming Reference

Ontology

Concepts

Attributes

Autogenerated concepts

Sharing the attributes

Core built-in concepts

User interface concepts

Action concepts

Primitive data concepts

Conditional concepts

Aggregation concepts

Qualifier concepts

NLG Concepts

Patterns

Best practices

Rules

Constraints

Input and output

Actions

Constraints

ParamType and queryType

Input

Output

Promises

Asking for follow-up information

Generating data