Administrators & Analysts

This article defines the major metrics and categories used by Mavenoid's default analytics dashboards. You can also hover over metrics in the dashboard and then over over the 'i' icon to get specific information.

There are three default dashboards:

• Overview: for a quick understanding of overall performance of the assistant

• Improvements: for identifying impactful improvements for flows

• Live support: for tracking specifically the live chats between your users and your agents

General definitions

A few terms are useful to understand for multiple dashboards.

Conversation

A "conversation" occurs when a user opens the Mavenoid assistant and interacts with it in a meaningful way. If the assistant is embedded in a website or mobile app, this requires clicking/tapping a button (on a node that does not have "Step is triage" checked) or performing a search. If the assistant is in a Dynamic Help Center, where solution nodes may be presented as the result of an internet search, leaving the page open for ten seconds (on a non-autogenerated page) also counts as an interaction.

If the user opens the assistant but does not interact with it or visits a Dynamic Help Center page and closes it within ten seconds without interacting, this is considered a "bounce" and is not counted or tracked as a conversation.

Resolution

There are a few specific types of resolution detailed in the Overview dashboard's conversation breakdown, but in general a "resolution" occurs in a conversation when at least one of the following things happen:

• The user indicates that a Solution node has solved their issue, which takes the conversation through an implicit "Success" jump

• The conversation goes through an explicit "Success" jump

• The user abandons the conversation on a Solution node (or a node marked as a potential solution)

In the first two cases, this means there is an explicit indication that the issue has been solved. In the last case, the user has been given information that may solve their issue and hasn't given any further feedback to indicate that their problem is not solved, so this is considered a probable resolution.

Deflection

A "deflection" occurs when a conversation concludes without escalating to a ticket in your CRM or help desk system.

Any of the following outcomes are considered a deflection:

• Self-service resolution: The user was shown a solution during self-service and either confirmed that it resolved their request or abandoned without indicating their request was unresolved.

• Live support resolution: The user had a live chat with an agent and the conversation was then explicitly flagged as resolved.

• Abandoned during self-service: The user abandoned the conversation without completing self-service.

• Abandoned during escalation: The user began some kind of escalation but abandoned the conversation before finishing it.

Overview dashboard

This dashboard includes all conversations where a user interacted with the assistant such as by clicking/tapping a button (on a node that does not have "Step is triage" checked) or performing a search.

Conversation breakdown

All conversations that match the active filters are grouped into the following categories.

Self-service resolution

These are conversations where the user was shown a solution during self-service and either confirmed that it resolved their request or abandoned without indicating their request was unresolved.

Specifically, this category means:

• The conversation has not gone through an "Out of Solutions" jump

• The conversation has not gone through a Live support node

• At least one of:

  • The conversation has gone through a "Success" jump

  • The conversation was abandoned on a Solution node (or a node marked as a potential solution)

Live support resolution

These are conversations where the user had a live chat with an agent and the conversation was then explicitly flagged as resolved.

Specifically, this category means:

• The conversation has gone through a Live support node

• The conversation has gone through a "Success" jump

External escalation submitted

These are conversations where the end of the flow was reached but the request is unresolved.

Specifically, this category means:

• The conversation has gone through an "Out of Solutions" jump

• The conversation has not gone through a "Success" jump

• The conversation has reached a Feedback node or Flow end node

Abandoned during self service

These are conversations that were abandoned without completing self-service.

Specifically, this category means:

• The conversation has not gone through an "Out of Solutions" jump

• The conversation has not gone through a "Success" jump

• The conversation was abandoned on a node that was not a Solution node (or a node marked as a potential solution)

Abandoned during escalation

These are conversations where some kind of escalation has begun but was not finished.

Specifically, this category means:

• The conversation has gone through an "Out of Solutions" jump

• The conversation has not gone through a "Success" jump

• The conversation has not reached a Feedback node or Flow end node

Combined counts

• "Solved in Mavenoid" includes "Self-service resolution" and "Live support resolution".

• "Abandoned sessions" includes "Abandoned during self-service" and "Abandoned during escalation".

• "External escalations" includes only "External escalation submitted".

• "Deflections" includes "Self-service resolution", "Live support resolution", "Abandoned during self-service", and "Abandoned during escalation".

Improvements dashboard

This dashboard includes all conversations where a user interacted with the assistant such as by clicking/tapping a button (on a node that does not have "Step is triage" checked) or performing a search.

The purpose of this dashboard is to identify impactful improvements for flows. Several of the tables allow you to drill in on increasingly-specific sets of conversation. As such, it's best used by starting at the top and narrowing your results as you move down.

Self-service: Flow breakdown

This table lists your flows and several high-level metrics for each one. It can be sorted by any column.

Which metrics will matter for you will depend on your particular use cases and flow structure, but in many cases a low resolution rate, a high abandon rate, a high escalation rate, or a low percentage of positive ratings is a sign that it's worth looking into that flow.

Once you've identified a flow to look into, click its name to filter the dashboard to that flow and proceed to the next table.

Self-service: Customer request breakdown

This table shows conversations on the currently-selected flow(s), grouped by customer request. The request is determined by what search result or search shortcut the user selected. If the user did not use a search node, then the request is determined by the last searchable node the user visited.

For each request, there are a number of metrics available and the table can be sorted by any column.

The columns whose titles start with "Importance" are weighted averages based on the number of conversations for that request. For example, the "Importance - Abandons" column is based both on the rate at which conversations with that request were abandoned and what percentage of total conversations had that request. This is so that you can find the requests where improvements can have the greatest impact: a request with a low abandon rate may still be a good one to improve if the total number of conversations on that request is high enough. Note that the lower the value of this column, the greater the impact: an importance of "1" means this is the number one most important request for this particular metric.

Once you've identified a request to look into, click its name to filter the dashboard to that request. This may give you enough information to investigate the relevant parts of your flow on the canvas; alternately you can proceed to the next table to get insights into specific nodes.

Flow usage: Node-specific breakdown

This table shows conversations on the currently-selected flow(s) with the currently-selected customer request(s), grouped by the last nodes the conversation went through. The "Last answered question" is the last node to which the customer responded, while the "Last unanswered question" is the final node they saw without responding, which will either be the node they abandoned on or the Flow end node if the conversation was completed.

For each ending node pair, there are a number of metrics available and the table can be sorted by any column.

The columns whose titles start with "Importance" are weighted averages based on the number of conversations ending on those nodes. For example, the "Importance - Abandons" column is based both on the rate at which conversations were abandoned on those nodes and what percentage of total conversations ended on those nodes. This is so that you can find the nodes where improvements can have the greatest impact: a node with a low abandon rate may still be a good one to improve if the total number of conversations on that node is high enough. Note that the lower the value of this column, the greater the impact: an importance of "1" means this is the number one most important node for this particular metric.

Once you've identified a node to look into, you can investigate it on the canvas and consider its contents.

User feedback

This table shows submitted feedback for conversations on the currently-selected flow(s) with the currently-selected customer request(s). This allows you to see what your users are saying about your flows for particular use cases.

Live support dashboard

This dashboard focuses on conversations where the user requested live support.

Key metrics

Live support requests

The total number of conversations that passed through a Live support node, whether or not the user successfully connected with an agent.

Live support completed

The number of conversations that passed through a Live support node where an agent joined the chat and subsequently closed the chat.

Live support resolutions

The number of conversations that passed through a Live support node and subsequently a "Success" jump. (These are the same conversations categorized as live support resolution in the Overview dashboard.)

Resolution rate

The percentage of conversations that passed through a Live support node that also passed through a "Success" jump. (In other words, this is live support requests divided by live support resolutions.)

% Positive rating

Out of all the conversations that passed through a Live support node for which the user provided feedback, the percentage of them where the user rated their experience "Good" (3) or "Great" (4).