| name | orchardcore-ai-chat-analytics |
| description | Skill for configuring AI Chat Session Analytics in Orchard Core with CrestApps modules. Covers session metrics tracking (unique visitors, handle time, containment rate, abandonment rate), dashboard reports (overview, time-of-day, day-of-week, user segments, performance, conversion, feedback), date range and profile filtering, CSV export, and per-profile analytics settings including conversion goals. Use it for CrestApps.OrchardCore.AI.Chat.Analytics, usage analytics, and related Orchard Core setup, extension, or troubleshooting work. |
| license | Apache-2.0 |
| metadata | {"author":"CrestApps Team","version":"1.0"} |
Orchard Core AI Chat Analytics - Prompt Templates
Configure AI Chat Analytics
You are an Orchard Core expert. Generate code, configuration, and recipes for adding AI chat session analytics and reporting to an Orchard Core application using CrestApps modules.
Guidelines
- The AI Chat Analytics feature (
CrestApps.OrchardCore.AI.Chat.Analytics) tracks chat session metrics and provides dashboard reporting with extensible display drivers.
- It depends on both the AI Chat feature (
CrestApps.OrchardCore.AI.Chat) and the AI Chat Core feature (CrestApps.OrchardCore.AI.Chat.Core).
- Metrics are collected automatically when analytics is enabled on an AI profile.
- Per-profile analytics settings control whether session metrics, AI resolution detection, and conversion metrics are active.
- Dashboard reports are rendered as display drivers on
AIChatAnalyticsReport, making them extensible through Orchard Core's display management.
- Filters are rendered as display drivers on
AIChatAnalyticsFilter, supporting date range and profile filtering.
- Session data is stored as
AIChatSessionEvent documents using YesSql with the AIChatSessionMetricsIndex for efficient queries.
- Install CrestApps packages in the web/startup project.
Feature ID
| Feature | ID | Category |
|---|
| AI Chat Session Analytics | CrestApps.OrchardCore.AI.Chat.Analytics | Artificial Intelligence |
Dependencies
| Dependency | Feature ID |
|---|
| AI Chat | CrestApps.OrchardCore.AI.Chat |
| AI Chat Core | CrestApps.OrchardCore.AI.Chat.Core |
NuGet Package
Install CrestApps.OrchardCore.AI.Chat in the web/startup project. The analytics feature is included in this module.
Enabling AI Chat Analytics
{
"steps": [
{
"name": "Feature",
"enable": [
"CrestApps.OrchardCore.AI",
"CrestApps.OrchardCore.AI.Chat",
"CrestApps.OrchardCore.AI.Chat.Analytics",
"CrestApps.OrchardCore.OpenAI"
],
"disable": []
}
]
}
Metrics Tracked
The analytics feature tracks the following metrics per chat session:
| Metric | Description |
|---|
| Session ID | Unique identifier for each chat session |
| Profile ID | The AI profile used for the session |
| Visitor ID | Unique visitor identifier (up to 64 characters) |
| User ID | Authenticated user identifier (if applicable) |
| Session start/end timestamps | When the session started and ended (UTC) |
| Message count | Total messages exchanged in the session |
| Handle time (seconds) | Duration of the chat session |
| Is resolved | Whether the session was resolved by AI |
| Input/output tokens | Total tokens consumed and generated |
| Average response latency (ms) | Mean AI response time across completions |
| Completion count | Number of AI completions in the session |
| Conversion score | Numeric score for conversion goal tracking |
| Thumbs up/down counts | User feedback on assistant messages |
Dashboard Reports
The analytics dashboard at Admin → Artificial Intelligence → Reports → AI Chat Session Analytics provides seven report sections, each rendered by its own display driver:
| Report | Shape Name | Description |
|---|
| Overview | ChatAnalyticsOverview | KPI cards showing total sessions, unique visitors, average handle time, containment rate %, abandonment rate %, resolved/abandoned sessions, returning user rate %, and average messages per session |
| Time-of-Day | ChatAnalyticsTimeOfDay | Hourly session distribution (0–23) |
| Day-of-Week | ChatAnalyticsDayOfWeek | Daily session distribution across the week |
| User Segments | ChatAnalyticsUserSegment | Breakdown of authenticated vs. anonymous sessions and unique users |
| Performance | ChatAnalyticsPerformance | Average response latency, total and average input/output tokens per session |
| Conversion | ChatAnalyticsConversion | AI resolution rate %, conversion score %, high-performing and low-performing session counts |
| Feedback | ChatAnalyticsFeedback | Thumbs up/down counts, percentages, and overall feedback rate |
Filtering Reports
Reports support two filters rendered as display drivers on AIChatAnalyticsFilter:
| Filter | Shape Name | Description |
|---|
| Date Range | ChatAnalyticsDateRangeFilter_Edit | Filter by start and end date (UTC) |
| Profile | ChatAnalyticsProfileFilter_Edit | Filter by AI profile from a dropdown |
CSV Export
The dashboard includes an Export action that generates a CSV file with session-level data including SessionId, ProfileId, VisitorId, UserId, IsAuthenticated, SessionStartedUtc, SessionEndedUtc, MessageCount, HandleTimeSeconds, and IsResolved.
Permissions
| Permission | Description |
|---|
ViewChatAnalytics | View AI Chat Analytics dashboard |
ExportChatAnalytics | Export AI Chat Analytics data to CSV |
Both permissions are granted to the Administrator role by default.
Admin Menu Entries
Enabling the analytics feature adds four entries under Artificial Intelligence → Reports:
- AI Chat Session Analytics
- AI Chat Extracted Data
- AI Usage Analytics
- AI Chat Conversion Goals
Per-Profile Analytics Settings
Analytics behavior is configured on each AI profile through the AnalyticsMetadata property:
| Setting | Description |
|---|
| Enable Session Metrics | Track session-level metrics (visitors, handle time, message count) |
| Enable AI Resolution Detection | Automatically detect whether AI resolved the user query |
| Enable Conversion Metrics | Track conversion goals defined on the profile |
Configuring Analytics on a Profile via Code
public sealed class AnalyticsProfileMigrations : DataMigration
{
private readonly IAIProfileManager _profileManager;
public AnalyticsProfileMigrations(IAIProfileManager profileManager)
{
_profileManager = profileManager;
}
public async Task<int> CreateAsync()
{
var profile = await _profileManager.NewAsync();
profile.Name = "support-chat";
profile.DisplayText = "Support Chat";
profile.Type = AIProfileType.Chat;
profile.Put(new AIProfileMetadata
{
SystemMessage = "You are a helpful support assistant.",
Temperature = 0.3f,
MaxTokens = 4096,
PastMessagesCount = 10,
});
profile.Put(new AnalyticsMetadata
{
EnableSessionMetrics = true,
EnableAIResolutionDetection = true,
EnableConversionMetrics = true,
ConversionGoals =
[
new ConversionGoal
{
Name = "issue-resolved",
Description = "The user's issue was fully resolved by the assistant.",
MinScore = 0,
MaxScore = 100,
},
],
});
await _profileManager.SaveAsync(profile);
return 1;
}
}
Key Services
| Service | Description |
|---|
AIChatSessionEventService | Core persistence service for recording session events (start, end, latency, tokens, resolution, conversion, ratings) |
AICompletionUsageService | Aggregates token usage analytics across sessions |
AnalyticsChatSessionHandler | IAIChatSessionHandler that records session start on first user message and tracks response latency |
AIChatSessionEventPostCloseObserver | Records session end with resolution status and conversion goal results |
Extending Reports with Custom Display Drivers
Reports are extensible through Orchard Core's display driver pattern. Register a custom display driver on AIChatAnalyticsReport to add a new report section:
public sealed class Startup : StartupBase
{
public override void ConfigureServices(IServiceCollection services)
{
services.AddDisplayDriver<AIChatAnalyticsReport, CustomAnalyticsDisplayDriver>();
}
}
Extending Filters with Custom Display Drivers
Add custom filters by registering a display driver on AIChatAnalyticsFilter:
public sealed class Startup : StartupBase
{
public override void ConfigureServices(IServiceCollection services)
{
services.AddDisplayDriver<AIChatAnalyticsFilter, CustomAnalyticsFilterDisplayDriver>();
}
}
The custom filter driver can append query conditions to AIChatAnalyticsFilter.Conditions to restrict the result set.
Usage Analytics
The AI Usage Analytics page (separate from session analytics) shows aggregated token usage data grouped by user, client, and model. It displays total API calls, sessions, chat interactions, and tokens with average latency per group.
