It's a good idea to introduce your AI agent in stages and monitor the impact as you go. Before you set Fin AI Agent live, you'll have plenty of opportunities to review the end-to-end experience for both your customers and your teammates.
Below are some ways to test Fin AI Agent internally and gradually roll Fin out to your customers.
Target Fin AI Agent at a specific audience
The audience rules for Fin AI Agent are configured differently depending on how you've set Fin up:
From a Workflow, or
From the Fin over chat setup
Audience rules from the Fin over chat setup are configured in the Who will see Fin section under AI & Automation > Fin AI Agent > Overview > Fin over chat.
From the Fin over email setup
Audience rules from the Fin over email setup are configured in the Who will see Fin section under AI & Automation > Fin AI Agent > Overview > Fin over email.
From a Workflow
Audience rules for a Workflow are configured in the trigger settings by clicking on the trigger or first step in the Workflows builder.
Note:
Audience rules will apply to all channels you’ve enabled for Fin AI Agent.
You'll need to have a live Messenger installed before you can test Fin on your website.
Set Fin AI Agent live for your team
To share Fin AI Agent with your team and allow them to test it themselves, you can set Fin live on your website and use audience rules to target your team only. This will also allow you to check that your handover rules are running as you intended.
For example, you could select ‘Users’ and use the filter ‘Email contains…’ to target teammates with your company email address.
Now Fin will be live for the users you specified. Those users will be able to interact with Fin by asking it questions in the Messenger.
Create a set of control questions to share with your team and ask them to note whether Fin provided the best response or not. This can also be a great way to identify gaps in your support content.
If you test Fin in your live environment, Fin will be billed per resolution.
However, if you or your teammate has asked to "Get more help" or speak to your team, the conversation is considered 'unresolved' and you would not be charged for this.
Roll Fin AI Agent out to customers
We recommend rolling Fin AI Agent out in stages by customer segment, such as starting with lower-tiered customers who would benefit the most from self-service, or by region. This approach allows you to supervise Fin closely and feel confident with how Fin works in your product.
Here are some ways you can identify and target customers:
Specific tiers
For example, you could enable Fin AI Agent for everyone except VIP customers by specifying ‘Users’ and using a custom data attribute you’re tracking such as ‘VIP customer is false’.
Or, target only self-serve customers or users on a specific price plan by using a custom data attribute such as ‘Plan is Free’.
An audience can become super targeted by combining And / Or filters. For example, you could use the ‘Last seen’ and ‘Region’ attributes to target only new users in a specific region.
Users or leads only
You can also make Fin AI Agent only available to logged in users, or only available to visitors to your website who reach out through the Messenger and become leads. Simply select this audience from the dropdown.
You can target users of a particular part of your product by selecting ‘Users’ and a custom data attribute such as ‘Projects is greater than 1’.
Specific page URLs
Enable Fin AI Agent only on specific pages by using the ‘Current Page URL’ attribute and entering the URL for that page. For example, you could target it at customers who are visiting your support page.
Or, if you want to see how much impact it has on reducing conversation volume for your team, you could target a page which drives a high volume of conversations.
Make sure you understand how audience rules and filters work to ensure you’re capturing the right audience and Fin AI Agent is able to get involved in these conversations.
See the impact of Fin AI Agent
Using reporting
You’ll have access to an AI agent performance report in Reports > Automation > Fin AI Agent.
In this report, you’ll be able to see:
The AI agent’s involvement in all conversations.
The answer rate, deflection rate, and resolution rate of conversations in which the AI agent is involved.
The AI agent’s impact over time.
Breakdown of resolved and unresolved conversations.
List of conversations the AI agent didn’t provide an answer for.
Identify gaps in content by reviewing unanswered conversations in the report and create new support content which covers these.
Learn more about the Fin AI Agent report.
Monitoring responses
The Fin AI Agent section in Help Desk allows you to monitor all conversations where your AI agent was involved. You can easily review these conversations and take note of any content gaps to update or create new articles.
Learn more about managing AI Agent conversations.
Need more help? Get support from our Community Forum
Find answers and get help from Intercom Support and Community Experts