Recipe: User Intent Classification

Purpose

Design and implement an intelligent user intent classification system using LLMs to understand what users really want to accomplish, enabling more effective routing, personalization, and user experience optimization.

Context

Use when building conversational interfaces, customer support systems, search experiences, or any user-facing feature where understanding user intent can improve outcomes. Essential for chatbots, help systems, e-commerce platforms, and SaaS applications.

Complexity Level: 🟡 Intermediate
Track Focus: 🎯 LLM-in-Product
Phase: 🔍 Discover
Time Investment: 2-4 hours

Intent Classification Architecture

Implementation Patterns

1. Intent Classification Prompt Design

Multi-Level Classification Prompt

# System Message
You are an expert intent classifier for a SaaS platform. Analyze user inputs and classify them into primary and secondary intent categories.

## Intent Taxonomy

### Primary Intent Categories:
- **TRANSACTIONAL**: User wants to complete an action (purchase, signup, cancel, update)
- **INFORMATIONAL**: User seeks information or wants to learn something
- **NAVIGATIONAL**: User wants to find or access a specific feature/page
- **SUPPORT**: User needs help with a problem or has a question
- **SOCIAL**: User wants to communicate, share, or interact socially

### Secondary Intent Categories (Context-Specific):
- **ONBOARDING**: Getting started, setup, initial configuration
- **FEATURE_USAGE**: Using specific platform features or capabilities  
- **BILLING**: Payment, subscription, pricing inquiries
- **INTEGRATION**: Connecting with other tools or services
- **TROUBLESHOOTING**: Resolving technical issues or errors
- **FEEDBACK**: Providing suggestions, complaints, or praise

## Response Format
Respond with JSON:
{
  "primary_intent": "PRIMARY_CATEGORY",
  "secondary_intent": "SECONDARY_CATEGORY",
  "confidence": 0.95,
  "reasoning": "Brief explanation of classification logic",
  "urgency": "low|medium|high",
  "complexity": "simple|moderate|complex",
  "suggested_routing": "recommended_handler_type",
  "context_signals": ["signal1", "signal2"],
  "follow_up_likely": true|false
}

# User Input
{USER_INPUT}

# Context (if available)
- User Type: {USER_TYPE}
- Current Page: {CURRENT_PAGE}  
- Previous Actions: {RECENT_ACTIONS}
- Account Status: {ACCOUNT_STATUS}

Domain-Specific Classification Prompts

E-commerce Platform

# System Message  
You are classifying user intents for an e-commerce platform. Focus on purchase journey and customer service needs.

## E-commerce Intent Categories:
- **BROWSE**: Looking at products, categories, or collections
- **COMPARE**: Evaluating multiple products or options
- **PURCHASE**: Ready to buy or in checkout process
- **TRACK**: Checking order status or delivery information
- **RETURN**: Processing returns, exchanges, or refunds
- **RECOMMEND**: Seeking product recommendations or advice
- **ACCOUNT**: Managing profile, preferences, or settings

## Context Indicators:
- Cart Status: {CART_ITEMS}
- Purchase History: {ORDER_HISTORY}  
- Current Category: {PRODUCT_CATEGORY}
- Session Duration: {TIME_ON_SITE}

# User Input
{USER_INPUT}

SaaS Platform

# System Message
You are classifying user intents for a SaaS productivity platform. Focus on feature usage and customer success.

## SaaS Intent Categories:
- **SETUP**: Initial configuration, onboarding, getting started
- **LEARN**: Understanding features, tutorials, best practices
- **USE**: Actively using features, performing tasks
- **OPTIMIZE**: Improving workflows, advanced configurations
- **INTEGRATE**: Connecting with other tools or APIs
- **UPGRADE**: Exploring premium features or plan changes
- **SUPPORT**: Technical issues, bugs, or questions

## Success Indicators:
- Onboarding Stage: {ONBOARDING_PROGRESS}
- Feature Adoption: {FEATURES_USED}
- Plan Type: {SUBSCRIPTION_TIER}
- Usage Frequency: {LOGIN_FREQUENCY}

# User Input  
{USER_INPUT}

2. Context-Aware Classification

Context Assembly Service

interface UserContext {
  // User Profile
  userId: string;
  userType: 'new' | 'active' | 'churning' | 'returning';
  accountStatus: 'trial' | 'active' | 'suspended' | 'canceled';
  subscriptionTier: 'free' | 'pro' | 'enterprise';
  
  // Session Context
  currentPage: string;
  timeOnPage: number;
  navigationHistory: string[];
  previousQueries: string[];
  
  // Behavioral Context
  recentActions: UserAction[];
  featureUsage: Record<string, number>;
  supportTickets: number;
  lastInteraction: Date;
  
  // Temporal Context
  timeOfDay: 'morning' | 'afternoon' | 'evening' | 'night';
  dayOfWeek: string;
  timezone: string;
}

class ContextAssemblyService {
  async assembleContext(userId: string, sessionId: string): Promise<UserContext> {
    const [userProfile, sessionData, behaviorData] = await Promise.all([
      this.getUserProfile(userId),
      this.getSessionData(sessionId),
      this.getBehaviorData(userId)
    ]);
    
    return {
      // User Profile
      userId: userProfile.id,
      userType: this.classifyUserType(userProfile, behaviorData),
      accountStatus: userProfile.accountStatus,
      subscriptionTier: userProfile.subscriptionTier,
      
      // Session Context
      currentPage: sessionData.currentPage,
      timeOnPage: sessionData.timeOnPage,
      navigationHistory: sessionData.navigationHistory,
      previousQueries: sessionData.previousQueries,
      
      // Behavioral Context
      recentActions: behaviorData.recentActions,
      featureUsage: behaviorData.featureUsage,
      supportTickets: behaviorData.supportTicketCount,
      lastInteraction: behaviorData.lastInteraction,
      
      // Temporal Context
      timeOfDay: this.getTimeOfDay(),
      dayOfWeek: new Date().toLocaleDateString('en', { weekday: 'long' }),
      timezone: userProfile.timezone
    };
  }
  
  private classifyUserType(profile: any, behavior: any): UserContext['userType'] {
    const daysSinceSignup = this.daysSince(profile.createdAt);
    const daysSinceLastLogin = this.daysSince(behavior.lastLogin);
    const activityLevel = behavior.actionsLast30Days;
    
    if (daysSinceSignup <= 7) return 'new';
    if (daysSinceLastLogin > 30) return 'returning';
    if (activityLevel < 5) return 'churning';
    return 'active';
  }
}

Intent Classification Service

interface IntentClassification {
  primaryIntent: string;
  secondaryIntent: string;
  confidence: number;
  reasoning: string;
  urgency: 'low' | 'medium' | 'high';
  complexity: 'simple' | 'moderate' | 'complex';
  suggestedRouting: string;
  contextSignals: string[];
  followUpLikely: boolean;
  metadata: {
    processingTimeMs: number;
    modelVersion: string;
    contextUsed: boolean;
  };
}

class IntentClassificationService {
  async classifyIntent(userInput: string, context?: UserContext): Promise<IntentClassification> {
    const startTime = Date.now();
    
    try {
      // Build classification prompt with context
      const prompt = this.buildClassificationPrompt(userInput, context);
      
      // Call LLM for classification
      const response = await this.llmService.complete({
        prompt,
        temperature: 0.1, // Low temperature for consistent classification
        maxTokens: 300,
        model: 'gpt-4-turbo' // Use most capable model for accuracy
      });
      
      // Parse and validate response
      const classification = this.parseClassificationResponse(response);
      
      // Apply business rules and confidence adjustments
      const finalClassification = this.applyBusinessRules(classification, context);
      
      // Log for analytics and improvement
      await this.logClassification(userInput, context, finalClassification);
      
      return {
        ...finalClassification,
        metadata: {
          processingTimeMs: Date.now() - startTime,
          modelVersion: 'gpt-4-turbo',
          contextUsed: !!context
        }
      };
      
    } catch (error) {
      // Fallback to rule-based classification
      return this.fallbackClassification(userInput, context, error);
    }
  }
  
  private buildClassificationPrompt(userInput: string, context?: UserContext): string {
    const basePrompt = this.getBaseClassificationPrompt();
    
    if (!context) {
      return basePrompt.replace('{USER_INPUT}', userInput);
    }
    
    // Enrich with context
    const contextualPrompt = basePrompt
      .replace('{USER_INPUT}', userInput)
      .replace('{USER_TYPE}', context.userType)
      .replace('{CURRENT_PAGE}', context.currentPage)
      .replace('{RECENT_ACTIONS}', context.recentActions.map(a => a.type).join(', '))
      .replace('{ACCOUNT_STATUS}', context.accountStatus);
    
    return contextualPrompt;
  }
  
  private parseClassificationResponse(response: string): IntentClassification {
    try {
      const parsed = JSON.parse(response);
      
      // Validate required fields
      if (!parsed.primary_intent || !parsed.confidence) {
        throw new Error('Invalid classification response format');
      }
      
      return {
        primaryIntent: parsed.primary_intent,
        secondaryIntent: parsed.secondary_intent || 'GENERAL',
        confidence: Math.min(Math.max(parsed.confidence, 0), 1), // Clamp 0-1
        reasoning: parsed.reasoning || '',
        urgency: parsed.urgency || 'medium',
        complexity: parsed.complexity || 'moderate',
        suggestedRouting: parsed.suggested_routing || 'general_handler',
        contextSignals: parsed.context_signals || [],
        followUpLikely: parsed.follow_up_likely || false
      };
      
    } catch (error) {
      throw new Error(`Failed to parse classification response: ${error.message}`);
    }
  }
  
  private applyBusinessRules(classification: IntentClassification, context?: UserContext): IntentClassification {
    // Adjust confidence based on context availability
    if (context) {
      classification.confidence *= 1.1; // Boost confidence with context
      classification.confidence = Math.min(classification.confidence, 0.99);
    }
    
    // Adjust urgency based on user type
    if (context?.userType === 'churning' && classification.primaryIntent === 'SUPPORT') {
      classification.urgency = 'high';
      classification.suggestedRouting = 'priority_support';
    }
    
    // Route enterprise users to specialized handlers
    if (context?.subscriptionTier === 'enterprise') {
      classification.suggestedRouting = 'enterprise_' + classification.suggestedRouting;
    }
    
    return classification;
  }
}

3. Intent-Based Routing System

Smart Intent Router

interface RoutingRule {
  intentPattern: string | RegExp;
  handler: string;
  conditions?: {
    userType?: string[];
    confidence?: number;
    urgency?: string[];
    complexity?: string[];
  };
  priority: number;
}

class SmartIntentRouter {
  private routingRules: RoutingRule[] = [
    // High-priority transactional intents
    {
      intentPattern: 'TRANSACTIONAL',
      handler: 'transaction_handler',
      conditions: { confidence: 0.8, urgency: ['high', 'medium'] },
      priority: 1
    },
    
    // Support routing by user type
    {
      intentPattern: 'SUPPORT',
      handler: 'enterprise_support',
      conditions: { userType: ['enterprise'], confidence: 0.7 },
      priority: 2
    },
    
    {
      intentPattern: 'SUPPORT',
      handler: 'priority_support', 
      conditions: { userType: ['churning'], urgency: ['high'] },
      priority: 2
    },
    
    // Complex informational queries
    {
      intentPattern: 'INFORMATIONAL',
      handler: 'rag_system',
      conditions: { complexity: ['complex'], confidence: 0.8 },
      priority: 3
    },
    
    // Simple navigation
    {
      intentPattern: 'NAVIGATIONAL',
      handler: 'direct_navigation',
      conditions: { complexity: ['simple'], confidence: 0.9 },
      priority: 4
    },
    
    // Fallback
    {
      intentPattern: /.*/,
      handler: 'general_assistant',
      priority: 99
    }
  ];
  
  async routeIntent(
    classification: IntentClassification, 
    context?: UserContext
  ): Promise<RoutingDecision> {
    // Sort rules by priority
    const sortedRules = this.routingRules.sort((a, b) => a.priority - b.priority);
    
    for (const rule of sortedRules) {
      if (this.matchesRule(classification, context, rule)) {
        const handler = await this.getHandler(rule.handler);
        
        return {
          handler: rule.handler,
          handlerInstance: handler,
          routingReason: `Matched rule: ${rule.intentPattern}`,
          priority: rule.priority,
          estimatedLatency: handler.getEstimatedLatency(),
          fallbackOptions: this.getFallbackOptions(classification)
        };
      }
    }
    
    // Should never reach here due to fallback rule
    throw new Error('No routing rule matched - configuration error');
  }
  
  private matchesRule(
    classification: IntentClassification,
    context: UserContext | undefined,
    rule: RoutingRule
  ): boolean {
    // Check intent pattern match
    const intentMatch = typeof rule.intentPattern === 'string'
      ? classification.primaryIntent === rule.intentPattern
      : rule.intentPattern.test(classification.primaryIntent);
      
    if (!intentMatch) return false;
    
    // Check conditions
    const conditions = rule.conditions;
    if (!conditions) return true;
    
    // Check confidence threshold
    if (conditions.confidence && classification.confidence < conditions.confidence) {
      return false;
    }
    
    // Check user type
    if (conditions.userType && context && !conditions.userType.includes(context.userType)) {
      return false;
    }
    
    // Check urgency
    if (conditions.urgency && !conditions.urgency.includes(classification.urgency)) {
      return false;
    }
    
    // Check complexity
    if (conditions.complexity && !conditions.complexity.includes(classification.complexity)) {
      return false;
    }
    
    return true;
  }
}

4. Response Generation by Intent

Intent-Specific Response Generators

class TransactionalResponseGenerator {
  async generateResponse(
    classification: IntentClassification,
    context: UserContext,
    originalQuery: string
  ): Promise<IntentResponse> {
    
    switch (classification.secondaryIntent) {
      case 'PURCHASE':
        return this.generatePurchaseResponse(originalQuery, context);
      case 'ACCOUNT':
        return this.generateAccountResponse(originalQuery, context);
      case 'BILLING':
        return this.generateBillingResponse(originalQuery, context);
      default:
        return this.generateGenericTransactionResponse(originalQuery, context);
    }
  }
  
  private async generatePurchaseResponse(query: string, context: UserContext): Promise<IntentResponse> {
    // Analyze purchase intent and guide user through process
    const purchaseAnalysis = await this.analyzePurchaseIntent(query);
    
    return {
      type: 'GUIDED_FLOW',
      content: {
        message: `I can help you with that purchase. Let me guide you through the process.`,
        actionItems: [
          'Show product options',
          'Explain pricing',
          'Start checkout process'
        ],
        uiComponents: ['ProductSelector', 'PricingCalculator'],
        nextSteps: ['product_selection', 'pricing_review', 'checkout_start']
      },
      metadata: {
        intentHandled: true,
        conversionOpportunity: true,
        followUpRequired: true
      }
    };
  }
}

class InformationalResponseGenerator {
  async generateResponse(
    classification: IntentClassification,
    context: UserContext,
    originalQuery: string
  ): Promise<IntentResponse> {
    
    // Use RAG system for knowledge-based responses
    const knowledgeResults = await this.ragService.query(originalQuery, {
      userContext: context,
      maxResults: 5,
      confidenceThreshold: 0.7
    });
    
    if (knowledgeResults.length === 0) {
      return this.generateNoKnowledgeResponse(originalQuery, context);
    }
    
    // Generate contextual response using retrieved knowledge
    const response = await this.llmService.complete({
      system: this.buildInformationalSystemPrompt(context),
      user: originalQuery,
      context: knowledgeResults,
      temperature: 0.3
    });
    
    return {
      type: 'INFORMATIONAL',
      content: {
        message: response,
        sources: knowledgeResults.map(r => ({
          title: r.title,
          url: r.url,
          relevance: r.score
        })),
        relatedQuestions: await this.generateRelatedQuestions(originalQuery, knowledgeResults),
        suggestedActions: this.extractActionableItems(response)
      },
      metadata: {
        intentHandled: true,
        knowledgeSourcesUsed: knowledgeResults.length,
        followUpSuggested: true
      }
    };
  }
  
  private buildInformationalSystemPrompt(context: UserContext): string {
    return `
    You are a helpful assistant providing accurate information based on the retrieved knowledge.
    
    User Context:
    - User Type: ${context.userType}
    - Experience Level: ${this.inferExperienceLevel(context)}
    - Current Focus: ${context.currentPage}
    
    Instructions:
    1. Provide accurate, helpful information based on the retrieved sources
    2. Tailor the explanation to the user's experience level
    3. Include practical next steps when appropriate
    4. If information is incomplete, clearly state what's missing
    5. Cite sources when making specific claims
    `;
  }
}

Advanced Features

1. Confidence Calibration

class ConfidenceCalibrator {
  // Calibrate model confidence based on historical accuracy
  calibrateConfidence(rawConfidence: number, classification: IntentClassification, context?: UserContext): number {
    // Apply empirical corrections based on intent type
    const intentCalibration = this.getIntentCalibration(classification.primaryIntent);
    
    // Apply context-based adjustments
    const contextAdjustment = context ? this.getContextAdjustment(context) : 1.0;
    
    // Apply complexity-based adjustments
    const complexityAdjustment = this.getComplexityAdjustment(classification.complexity);
    
    const calibratedConfidence = rawConfidence * intentCalibration * contextAdjustment * complexityAdjustment;
    
    return Math.min(Math.max(calibratedConfidence, 0.1), 0.99);
  }
  
  private getIntentCalibration(primaryIntent: string): number {
    // Based on historical accuracy analysis
    const calibrationMap: Record<string, number> = {
      'TRANSACTIONAL': 1.1, // Model is typically conservative
      'NAVIGATIONAL': 0.9,  // Model is overconfident
      'INFORMATIONAL': 1.0, // Well-calibrated
      'SUPPORT': 0.95,      // Slightly overconfident
      'SOCIAL': 0.8         // Often misclassified
    };
    
    return calibrationMap[primaryIntent] || 1.0;
  }
}

2. Multi-Turn Intent Tracking

class ConversationIntentTracker {
  private conversationHistory: Map<string, ConversationState> = new Map();
  
  async trackIntentProgression(
    sessionId: string,
    currentClassification: IntentClassification,
    userInput: string
  ): Promise<IntentProgression> {
    
    const conversation = this.conversationHistory.get(sessionId) || {
      intents: [],
      startTime: new Date(),
      lastUpdate: new Date()
    };
    
    // Add current intent to history
    conversation.intents.push({
      classification: currentClassification,
      userInput,
      timestamp: new Date()
    });
    
    conversation.lastUpdate = new Date();
    
    // Analyze intent progression patterns
    const progression = this.analyzeIntentProgression(conversation);
    
    this.conversationHistory.set(sessionId, conversation);
    
    return progression;
  }
  
  private analyzeIntentProgression(conversation: ConversationState): IntentProgression {
    const intents = conversation.intents;
    const recentIntents = intents.slice(-5); // Last 5 intents
    
    return {
      totalIntents: intents.length,
      uniqueIntents: new Set(intents.map(i => i.classification.primaryIntent)).size,
      dominantIntent: this.getDominantIntent(recentIntents),
      intentShifts: this.countIntentShifts(recentIntents),
      satisfactionSignals: this.extractSatisfactionSignals(recentIntents),
      nextLikelyIntents: this.predictNextIntents(recentIntents),
      conversationStage: this.determineConversationStage(intents)
    };
  }
}

Validation Checklist

Functional Validation

• Classification accurately identifies primary intent categories (>85% accuracy)
• Secondary intent classification provides useful context
• Confidence scores correlate with actual classification accuracy
• Context enrichment improves classification quality
• Routing decisions are logical and effective
• Fallback mechanisms handle edge cases gracefully

Performance Validation

• Classification latency meets SLA requirements (<500ms p95)
• System handles expected concurrent load without degradation
• Memory usage remains stable over time
• Context assembly doesn't significantly impact performance
• Caching reduces redundant processing effectively

Quality Validation

• User feedback indicates satisfactory intent understanding
• A/B testing shows improvement over baseline routing
• Business metrics improve (conversion rates, support efficiency)
• Edge cases are handled appropriately
• Multi-turn conversations maintain context effectively

Business Impact Validation

• Support ticket deflection rates increase
• User engagement and satisfaction scores improve
• Conversion rates increase for transactional intents
• Time to resolution decreases for support intents
• Overall user experience metrics show positive trends

Implementation Checklist

Phase 1: Basic Classification (Week 1)

• Define intent taxonomy for your domain
• Create and test classification prompts
• Implement basic LLM classification service
• Add confidence scoring and validation
• Create simple routing based on primary intent

Phase 2: Context Integration (Week 2)

• Implement context assembly service
• Enhance prompts with contextual information
• Add context-aware routing rules
• Implement confidence calibration
• Test classification improvement with context

Phase 3: Advanced Routing (Week 3)

• Build sophisticated routing rule engine
• Create intent-specific response generators
• Implement multi-turn conversation tracking
• Add business rule integration
• Create fallback and error handling

Phase 4: Analytics & Optimization (Week 4)

• Implement comprehensive logging and analytics
• Create intent classification dashboard
• Set up A/B testing framework
• Build feedback collection mechanisms
• Optimize based on usage patterns and feedback

Success Metrics

Technical KPIs

• Classification Accuracy: >85% for primary intents, >75% for secondary intents
• Confidence Calibration: Model confidence correlates with actual accuracy (R² > 0.8)
• Response Time: <500ms p95 for classification and routing
• System Reliability: >99.9% uptime for classification service
• Context Improvement: 15-25% accuracy improvement with context vs. without

Business KPIs

• User Satisfaction: >80% positive feedback on intent understanding
• Task Completion: >90% of user intents resolved successfully
• Efficiency Gains: 30-50% improvement in routing accuracy vs. rules-based system
• Support Deflection: 25-40% reduction in unnecessary escalations
• Conversion Impact: 10-20% improvement in conversion rates for transactional intents

Ready to implement user intent classification? Start with Phase 1: Basic Classification and build systematically toward advanced analytics and optimization. Remember to validate with real users early and often!