We've Moved! Please visit our new and improved forum over at our new portal: https://portal.plumvoice.com/hc/en-us/community/topics

Previously deployed rev no longer works well

Questions and answers about Plum Fuse+

Moderators: admin, support

Post Reply
simonie
Posts: 3
Joined: Mon Jul 31, 2017 1:26 pm

Previously deployed rev no longer works well

Post by simonie »

I deployed a simple app that asks for a 16 digit number last week and had people call in. The speech rec operated fairly well. Today, many more people called in and most could not get a number recognized at all. Even the same people who called last week are having significant trouble. I built the app in Fuse+, but am using the following grammar to collect the number:

<grammar mode="voice" root="AcctNum">

<property name="sensitivity" value="0"/>
<property name="incompletetimeout" value="2000"/>

<rule id="AcctNum">
<item>
<item repeat="0-1"><ruleref uri="#AcctNumPre"/></item>
<item><ruleref uri="builtin:grammar/digits?language=en-US;minlength=16;maxlength=16"/>
</item>
</item>
</rule>

<rule id="AcctNumPre">
<one-of>
<item>
<item repeat="0-1">
<one-of>
<item>my</item>
<item>the</item>
</one-of>
</item>
<item repeat="0-1">sixteen digit</item>
<item>account</item>
<item repeat="0-1">number</item>
<item repeat="0-1">is</item>
</item>
<item>it is</item>
<item>its</item>
</one-of>
</rule>

<rule id="PossibleZipCodes">
<one-of>
<item>three zero three zero eight</item>

</one-of>
</rule>

</grammar>

The app name is PV US Bank Trial and the number is 888 352 8764.

support
Posts: 3632
Joined: Mon Jun 02, 2003 3:47 pm
Location: Boston, MA
Contact:

Re: Previously deployed rev no longer works well

Post by support »

Could you provide us a link to a successful call log from last week vs. a more recent successful call? Please feel free to PM us those links if you have sensitive data in your call logs.

simonie
Posts: 3
Joined: Mon Jul 31, 2017 1:26 pm

Re: Previously deployed rev no longer works well

Post by simonie »

I am using external grammars, shown below. This is the exact revision I had them test last week. They are now unable to get the first prompt to accept their digits, either interrupting them, or hearing the error prompt. Some are successful when they rapid fire the entire number, but it won't allow any pauses, even though I have manipulated the speech parameters several different ways for that express purpose. Can you check that speech parameters (specifically sensitivity and incomplete timeout) are being set correctly when I enter them? Mark said yesterday that one possibility might be that need a new number issued to deal with it.

Good log from July 27:

27 JUL 2017 02:54:44 PM -0500: start, [start] - Executing application "PV US Bank Trial" version "2.0"
27 JUL 2017 02:54:44 PM -0500: Welcome, [prompt] - Welcome to U S Bank.
27 JUL 2017 02:55:02 PM -0500: ThanksAcctNum, [prompt] - Thanks.
27 JUL 2017 02:55:06 PM -0500: GetZipGRXML, [custom_field] - Now, what's your zip code.?
27 JUL 2017 02:55:14 PM -0500: GetZipGRXML, [custom_field] - User input: three zero three zero eight (voice)
27 JUL 2017 02:55:14 PM -0500: CheckZip, [branch_on_var] - Value: three zero three zero eight
27 JUL 2017 02:55:14 PM -0500: ThanksZip, [prompt] - Thanks.
27 JUL 2017 02:55:17 PM -0500: MainMenu, [menu] - Main Menu. You can say Payments, Transactions, Account Changes, or General Information.
27 JUL 2017 02:55:30 PM -0500: MainMenu, [menu] - User input: General Information (voice)
27 JUL 2017 02:55:30 PM -0500: MMConf, [prompt] - General Information (say-as: null)
27 JUL 2017 02:55:30 PM -0500: MMConf, [prompt] - . Sure.
27 JUL 2017 02:55:35 PM -0500: MMOptions, [branch_on_var] - Value: General Information
27 JUL 2017 02:55:35 PM -0500: SubGeneral, [menu] - You can say rewards, balance details, mailing address, interest charges or annual percentage rate, or you can say start over or press 9.
27 JUL 2017 02:55:43 PM -0500: SubGeneral, [menu] - No Match x1
27 JUL 2017 02:55:43 PM -0500: SubGeneral, [menu] - Sorry.
27 JUL 2017 02:55:46 PM -0500: SubGeneral, [menu] - You can say rewards, balance details, mailing address, interest charges or annual percentage rate, or you can say start over or press 9.
27 JUL 2017 02:55:52 PM -0500: SubGeneral, [menu] - No Match x2
27 JUL 2017 02:55:52 PM -0500: SubGeneral, [menu] - Sorry. I didn't get that.
27 JUL 2017 02:55:56 PM -0500: SubGeneral, [menu] - You can say rewards, balance details, mailing address, interest charges or annual percentage rate, or you can say start over or press 9.
27 JUL 2017 02:56:00 PM -0500: SubGeneral, [menu] - Disconnecting call

Bad logs from the same user July 31:
31 JUL 2017 01:00:34 PM -0500: start, [start] - Executing application "PV US Bank Trial" version "2.10"
31 JUL 2017 01:00:35 PM -0500: Welcome, [prompt] - Welcome to U S Bank.
31 JUL 2017 01:01:45 PM -0500: start, [start] - Executing page "Welcome"
31 JUL 2017 01:01:45 PM -0500: Welcome, [prompt] - Welcome to U S Bank.
(took 71 seconds)

31 JUL 2017 12:40:20 PM -0500: start, [start] - Executing application "PV US Bank Trial" version "2.8"
31 JUL 2017 12:40:20 PM -0500: Welcome, [prompt] - Welcome to U S Bank.
(took 24 seconds)

31 JUL 2017 12:44:14 PM -0500: start, [start] - Executing application "PV US Bank Trial" version "2.8"
31 JUL 2017 12:44:15 PM -0500: Welcome, [prompt] - Welcome to U S Bank.
31 JUL 2017 12:44:31 PM -0500: ThanksAcctNum, [prompt] - Thanks.
31 JUL 2017 12:44:35 PM -0500: GetZipGRXML, [custom_field] - Now, what's your zip code.?
31 JUL 2017 12:44:37 PM -0500: GetZipGRXML, [custom_field] - Disconnecting call
(took 23 seconds)

31 JUL 2017 01:59:37 PM -0500: start, [start] - Executing application "PV US Bank Trial" version "0.13"
31 JUL 2017 01:59:37 PM -0500: Welcome, [prompt] - Thank you for calling U S Bank.
31 JUL 2017 02:00:25 PM -0500: start, [start] - Executing page "Welcome"
31 JUL 2017 02:00:25 PM -0500: Welcome, [prompt] - Thank you for calling U S Bank.
(took 57 seconds)

<grammar mode="voice" root="AcctNum">

<property name="sensitivity" value="0"/>
<property name="incompletetimeout" value="3s"/>

<rule id="AcctNum">
<item>
<item repeat="0-1"><ruleref uri="#AcctNumPre"/></item>
<item><ruleref uri="builtin:grammar/digits?language=en-US;minlength=16;maxlength=16"/>
</item>
</item>
</rule>

<rule id="AcctNumPre">
<one-of>
<item>
<item repeat="0-1">
<one-of>
<item>my</item>
<item>the</item>
</one-of>
</item>
<item repeat="0-1">sixteen digit</item>
<item>account</item>
<item repeat="0-1">number</item>
<item repeat="0-1">is</item>
</item>
<item>it is</item>
<item>its</item>
</one-of>
</rule>

</grammar>

support
Posts: 3632
Joined: Mon Jun 02, 2003 3:47 pm
Location: Boston, MA
Contact:

Re: Previously deployed rev no longer works well

Post by support »

You cannot manipulate sensitivity and incompletetimeout within a VXML grammar. These are properties that are declared outside the <grammar> tag. You may tweak timeout settings under app settings. These timeouts are global, for your entire application.

Sensitivity should never go below 0.2, as speech recognition becomes increasingly useless below 0.2. Your caller may have to yell to have their voice recognized below that level. However, you cannot control sensitivity in Fuse+, it is set at an industry default.

You have an unused PossibleZipCodes rule that you should get rid of from your grammar.

If you are only collecting digits, we highly recommend you use DTMF only instead of creating a custom voice grammar. As a general rule of thumb, you should avoid voice recognition as much as possible, as it introduces many factors that are beyond our control, like background noise, poor Bluetooth headsets, etc etc.

That said, we noticed some strangely behavior in the "bad" logs from July 31, which we are investigating. We'll let you know here as soon as we have updates.

simonie
Posts: 3
Joined: Mon Jul 31, 2017 1:26 pm

Re: Previously deployed rev no longer works well

Post by simonie »

incompletetimeout and others like maxspeechtimeout are not available to tweak in the settings on my app. Do I have to ask you to make those changes for me?

Is sensitivity is set to 0.2 in all the Fuse+ apps by defaults? I only tried to set it lower because callers were having so much trouble. If that is standard, that should be acceptable if we can achieve better ASR results in other ways.

The purpose of our application is requires both voice and DTMF, so limiting to DTMF is not an option. If your tool is not capable of handling speech input of this type in a robust manner, we will need to seek a different tool that is closer to industry standards for ASR. However, I only used an external grammar to try to manipulate parameters and some things that are more difficult to do in the app like large sets of grammar item synonyms and module based error handling.

Please let me know as soon as possible if the strange behavior reveals anything that will improve our caller experiences as well.

support
Posts: 3632
Joined: Mon Jun 02, 2003 3:47 pm
Location: Boston, MA
Contact:

Re: Previously deployed rev no longer works well

Post by support »

The setting "Final speech timeout" is incompletetimeout. maxspeechtimeout is not editable in Fuse+. Sensitivity is 0.5 for all Fuse+ apps. Granular control over properties is not available in Fuse+. We would recommend using Plum Dev if you want that level of control.

The industry standard for digit collection is DTMF. If you want more custom ASR grammars, we would also highly recommend Plum Dev, if at least for the development process. It will be far more efficient to develop your grammars and QA in Plum Dev, than in Fuse+.

Post Reply