Lex SLU Integration

Turn speech or text input into structured meaning with the Jovo Framework integration for Amazon Lex.

Amazon Lex is a spoken language understanding (SLU) service that can do both automatic speech recognition (ASR) as well as natural language understanding (NLU).

Lex can be used for platforms that don't come with their own ASR and NLU capabilities, for example Jovo for Web.

Learn more in the sections below:

• Installation: How to add Lex to your Jovo project
• Configuration: All configuration options
• Entities: How tu use Lex with this.$entities
• Dialog Management: How to do slot filling with Lex

If you want to dig deeper, you can find the implementation here: LexSlu.ts.

You can install the plugin like this:

$ npm install @jovotech/slu-lex

SLU plugins need to be added to Jovo platform integrations. Here is an example how it can be added to the Jovo Core Platform in app.ts:

import { CorePlatform } from '@jovotech/platform-core';
import { LexSlu } from '@jovotech/slu-lex';
// ...

const app = new App({
  plugins: [
    new CorePlatform({
      plugins: [
        new LexSlu({
          bot: {
            id: '<your-bot-id>',
            aliasId: '<your-alias-id>',
          },
        }),
      ],
    }),
    // ...
  ],
});

For the integration to work, you need to add all configurations shown in the code snippet above. For more information, take a look at the configuration section.

The rest of this section provides an introduction to the steps you need to take depending on where you host your Jovo app:

• On AWS (e.g. Lambda)
• Outside AWS

If you host your app on AWS Lambda and want to use Lex v2 in the same region, you only need to add the bot id and aliasId to get started:

new LexSlu({
  bot: {
    id: '<your-bot-id>',
    aliasId: '<your-alias-id>',
  },
}),

If you want to use Lex v2 from outside AWS Lambda, you need to set it up for programmatic access. Learn more in the official guide by Amazon: Identity and access management for Amazon Lex V2 and Setting credentials in Node.js.

You can then add the necessary keys using the libraryConfig property:

new LexSlu({
  bot: {
    id: '<your-bot-id>',
    aliasId: '<your-alias-id>',
  },
  libraryConfig: {
    lexRuntimeV2Client: {
      region: 'us-east-1',
      credentials: {
        accessKeyId: 'myAccessKeyId',
        secretAccessKey: 'mySecretAccessKey',
      },
    }
  }
  // ...
}),

The following configurations can be added:

new LexSlu({
  bot: {
    id: '',
    aliasId: ''
  },
  fallbackLocale: 'en_US',
  localeMap: {
    'en': 'en_US',
    'es': 'es_ES',
  },  
  asr: true,
  nlu: true,
}),

• bot: Includes information about your created Lex bot.
• libraryConfig: Any configuration for the AWS Lex v2 SDK can be passed here. Learn more below.
• region: (deprecated: use libraryConfig) The AWS region of the Lex bot, for example us-east-1.
• credentials: (deprecated: use libraryConfig) Your AWS security credentials.
• fallbackLocale: Locale that should be used if none could be found in the request. Default: en_US.
• localeMap (optional): This is used to map a request locale to a Lex localeId.
• asr: Determines whether the Lex ASR capabilities should be used. Default: true.
• nlu: Determines whether the Lex NLU capabilities should be used. Default: true.

The libraryConfig property can be used to pass configurations to the AWS Lex v2 SDK that is used by this integration.

Currently, it includes options for the LexRuntimeV2Client:

new LexSlu({
  bot: { /* ... */ },
  libraryConfig: { /* ... */ }
  // ...
}),

For example, you can add credentials like this:

new LexSlu({
  libraryConfig: {
    lexRuntimeV2Client: {
      region: 'us-east-1',
      credentials: {
        accessKeyId: 'myAccessKeyId',
        secretAccessKey: 'mySecretAccessKey',
      },
    }
  }

}),

You can access Lex slots by using the $entities property. You can learn more in the Jovo Model and the $entities documentation.

The Lex slot values are translated into the following Jovo entity properties:

{
  value: slot.value.originalValue || slot.value.interpretedValue, // what the user said
  resolved: slot.value.resolvedValues?.[0] || slot.value.interpretedValue, // the resolved value
  id: slot.value.resolvedValues?.[0] || slot.value.interpretedValue, // same as resolved, since Lex doesn't support IDs
  native: { /* raw API response for this slot */ }
}

You can learn more about the Lex slot format in the official Lex documentation.

Lex allows you to manage sessions by specifying multiple slots that need to be filled before the intent can complete.

Learn more in the following sections:

• Dialog Management NLU Data
• Use Lex Dialog Management in a Handler

In addition to the properties that are usually part of $input.nlu (written into the $input object by NLU integrations), Lex can also add additional values for state, confirmationState, dialogAction and messages.

Here is an example:

{
  "intent": {
    "name": "BookHotel",
    "confidence": 1,
    "state": "InProgress",
    "confirmationState": "None"
  },
  "messages": [
    {
      "content": "How many nights will you be staying?",
      "contentType": "PlainText"
    }
  ],
  "entities": {
    "CheckInDate": {
      "id": "2022-08-09",
      "resolved": "2022-08-09",
      "value": "August 9th",
      "native": {
        "value": {
          "interpretedValue": "2022-08-09",
          "originalValue": "August 9th",
          "resolvedValues": ["2022-08-09"]
        }
      }
    },
    "Location": {
      "id": "los angeles",
      "resolved": "los angeles",
      "value": "Los Angeles",
      "native": {
        "value": {
          "interpretedValue": "Los Angeles",
          "originalValue": "Los Angeles",
          "resolvedValues": ["los angeles"]
        }
      }
    },
    "sessionState": {
      "dialogAction": {
        "slotToElicit": "Nights",
        "type": "ElicitSlot"
      }
    }
  }
}

Additional values are explained below:

• intent
  • state: Contains fulfillment information for the intent. Values are "InProgress", "ReadyForFulfillment", "Failed" and others.
  • confirmationState: Contains information about whether fulfillment of the intent has been confirmed. Values are "None", "Confirmed" and "Denied".
• sessionState
  • dialogAction: The next step that Amazon Lex V2 should take in the conversation with a user. The slotToElicit has its prompt in messages.
• messages: The next prompt to pass on to the user.

Here is an example of how a component handler can manage the dialog state:

@Intents(['BookHotel'])
bookRoom() {
  const nluData = this.$input.nlu as LexNluData;
  if (nluData.intent.state === 'ReadyForFulfillment') {
    // make api call

    return this.$send('The room has been booked!');
  }

  if (nluData.intent.state === 'Failed') {
    return this.$send('No worries. Maybe next time.');
  }

  const message = nluData.messages?.[0].content as string;
  let quickReplies;

  if (
    nluData.sessionState?.dialogAction?.type === 'ElicitSlot' &&
    nluData.sessionState?.dialogAction?.slotToElicit === 'RoomType'
  ) {
    quickReplies = ['king', 'queen', 'deluxe'];
  }

  return this.$send({ message, quickReplies });
}