🤖 Multi-Channel AI Agent System

graph TB
    subgraph orchestration["🎯 AGENTS"]
        Journey["🎯 Communication Orchestrator
───────────────
✓ Multi-step workflows
✓ Parallel multi-channel
✓ Context preservation
✓ State management"]
        PreCall["📞 Pre-Call Agent
───────────
✓ Confirmation Call
✓ Narrow Window
✓ On the Way Call"]
        FollowUp["📋 Follow-Up Agent
───────────
✓ Post-Service SMS
✓ Satisfaction Check
✓ Service Complete"]
        Comms["💬 Communications Hub
───────────
✓ 12 Scenarios
✓ Multi-Channel
✓ Templates"]
    end
    
    subgraph context["💾 CONTEXT & STATE MANAGEMENT"]
        StateManager["🧠 Conversation State Manager
───────────────────
1. Cross-channel tracking
2. Intent & sentiment
3. History & timeline
4. Next-best-action"]
        subgraph dataSources["📊 Data Sources"]
            direction LR
            MongoDB["🗄️ MongoDB Data Store
───────────────
1. Conversations
2. Customers
3. Interactions
4. Context
5. Analytics"]
            ServicePower["🔧 ServicePower/Snowflake
───────────────
1. Service orders
2. Appointments
3. Customer details
4. Schedules
5. Parts
6. Tech info"]
        end
    end
    
    subgraph channels["🤖 CHANNEL AGENTS LAYER"]
        subgraph voice["🎤 Voice Agents"]
            OutVoice["📤 Outbound Voice
──────────
1. Initiate calls
2. Real-time conv
3. Voicemail
4. Trigger actions"]
            InVoice["📥 Inbound Voice
──────────
1. Answer calls
2. Context-aware
3. KB-powered"]
        end
        
        subgraph sms["💬 SMS Agents"]
            OutSMS["📤 Outbound SMS
──────────
1. Send messages
2. Link sharing
3. Confirmations
4. Trigger actions"]
            InSMS["📥 Inbound SMS
──────────
1. Process replies
2. Context-aware
3. Determine intent
4. Auto-respond"]
        end
        
        subgraph email["📧 Email Agents"]
            OutEmail["📤 Outbound Email
──────────
1. Compose
2. Attachments
3. Follow-ups"]
            InEmail["📥 Inbound Email
──────────
1. Process replies
2. Extract info
3. Classify intent
4. Auto-respond"]
        end
    end
    
    subgraph integration["🔌 INTEGRATION LAYER"]
        OpenAI["🤖 OpenAI Realtime
──────────────
1. Speech-to-Speech
2. Function Calling
3. Voice Detection
4. Audio Streaming"]
        Twilio["📱 Twilio Voice/SMS
──────────────
1. Media Streams
2. TwiML
3. Messaging API
4. Webhooks"]
        Graph["📮 Microsoft Graph
──────────────
1. Mail API
2. Attachments"]
    end
    
    PreCall --> StateManager
    FollowUp --> StateManager
    Comms --> StateManager
    
    StateManager --> dataSources
    
    MongoDB ~~~ ServicePower
    
    dataSources --> OutVoice
    dataSources --> InVoice
    dataSources --> OutSMS
    dataSources --> InSMS
    dataSources --> OutEmail
    dataSources --> InEmail
    
    OutVoice --> Twilio
    InVoice --> Twilio
    OutVoice --> OpenAI
    InVoice --> OpenAI
    
    OutSMS --> Twilio
    InSMS --> Twilio
    
    OutEmail --> Graph
    InEmail --> Graph
    
    style orchestration fill:#e8f0f5,stroke:#1a1f3a,stroke-width:3px
    style context fill:#f5f0e8,stroke:#8b6f47,stroke-width:3px
    style channels fill:#f0e8f0,stroke:#4a2c5c,stroke-width:3px
    style integration fill:#e8f5f0,stroke:#2d5f4f,stroke-width:3px
    
    style dataSources fill:#f5ede5,stroke:#c9a961,stroke-width:2px
    
    style voice fill:#e8f0f5,stroke:#1e4d4d,stroke-width:2px
    style sms fill:#f5e8eb,stroke:#b76e79,stroke-width:2px
    style email fill:#f5f0e0,stroke:#d4a574,stroke-width:2px

sequenceDiagram
    participant Scheduler as ⏰ Scheduler
    participant PreCall as 📞 Home Manager
(Pre-Call Agent)
    participant DB as 🗄️ MongoDB
    participant GPT as 🤖 GPT-4o
    participant OutVoice as 📤 Outbound Voice
    participant Twilio as 📱 Twilio
    participant OpenAI as 🎤 OpenAI Realtime
    participant Customer as 👤 Customer
    participant SMS as 💬 SMS Agent

    Note over Scheduler,PreCall: 1-2 hours before appointment
    Scheduler->>PreCall: Trigger confirmation workflow
    
    PreCall->>DB: Load appointment data
    DB-->>PreCall: Service details, window, customer info
    
    PreCall->>GPT: Generate confirmation script
    Note right of GPT: • Technician name
• Narrowed window (4-5pm)
• Service type
• Payment ($89)
• Pet/access notes
    GPT-->>PreCall: Personalized script ready
    
    PreCall->>OutVoice: Initiate confirmation call
    OutVoice->>Twilio: Place call to +1234567890
    Twilio->>Customer: 📞 Calling...
    Customer-->>Twilio: Answer
    
    Twilio->>OpenAI: Connect audio stream
    OpenAI-->>Customer: 🗣️ "Hi! This is about your 4-5pm appointment..."
    Customer-->>OpenAI: 🗣️ "Yes, I'll be home"
    
    OpenAI->>Customer: 🗣️ "Adult 18+ present?"
    Customer->>OpenAI: 🗣️ "Yes"
    
    OpenAI->>Customer: 🗣️ "$89 trip charge, pets secured?"
    Customer->>OpenAI: 🗣️ "Got it, dog will be in backyard"
    
    OpenAI->>Customer: 🗣️ "I'll call when tech is 30 min away"
    Customer->>OpenAI: 🗣️ "Perfect, thanks!"
    
    OpenAI-->>OutVoice: Call completed (2 min 15 sec)
    OutVoice->>SMS: Send confirmation SMS
    SMS->>Customer: 📱 "Confirmed: Tech Tom, 4-5pm today. $89 trip charge."
    
    OutVoice->>DB: Log call details & confirmation
    DB-->>OutVoice: ✓ Saved
    
    Note over Scheduler,Customer: 30 minutes before arrival
    Scheduler->>PreCall: Trigger "on the way" call
    PreCall->>OutVoice: Make brief notification call
    OutVoice->>Twilio: Call customer
    Twilio->>OpenAI: Connect
    OpenAI->>Customer: 🗣️ "Tech finishing up, heading your way!"
    Customer->>OpenAI: 🗣️ "Great, ready!"
    OpenAI-->>OutVoice: Call complete (35 sec)
    
    OutVoice->>DB: Update: Customer ready ✓

sequenceDiagram
    participant Scheduler as ⏰ Scheduler
    participant Journey as 🎯 Communication Orchestrator
    participant OutVoice as 📤 Outbound Voice
    participant OutSMS as 📤 Outbound SMS
    participant OutEmail as 📤 Outbound Email
    participant DB as 🗄️ MongoDB State
    participant OpenAI as 🎤 OpenAI Realtime
    participant Twilio as 📱 Twilio
    participant Graph as 📮 MS Graph
    participant Customer as 👤 Customer

    Scheduler->>Journey: Trigger confirmation workflow
    Journey->>DB: Create communication state
    Note right of DB: State: workflow_initiated
Status: in_progress
Step: outbound_call
    
    Journey->>OutVoice: Initiate confirmation call
    OutVoice->>Twilio: Place call to customer
    Twilio->>Customer: 📞 Calling...
    
    alt Customer Answers Call
        Customer->>Twilio: ✓ Answers
        Twilio->>OpenAI: Connect audio stream
        OpenAI->>Customer: 🗣️ "Hi! This is about your appointment at 4-5pm today..."
        Customer->>OpenAI: 🗣️ "Yes, I'll be there!"
        OpenAI->>Customer: 🗣️ "Great! Tech Tom will arrive between 4-5pm. $89 trip charge."
        Customer->>OpenAI: 🗣️ "Perfect, see you then!"
        
        OpenAI-->>OutVoice: Call completed successfully
        OutVoice->>DB: Update: confirmed, workflow_complete
        Note right of DB: Status: confirmed
Channel: voice_outbound
Duration: 1m 45s
        Note over Journey,DB: ✅ SUCCESS - Workflow Complete
    
    else No Answer / Voicemail Detected
        Customer-->>Twilio: No answer (rings)
        Twilio->>OpenAI: Voicemail greeting detected
        Note right of OpenAI: AI detects:
• Beep sound
• Greeting patterns
• No human response
        
        OpenAI->>Customer: 🗣️ Leave voicemail message
        Note right of OpenAI: "Hi, this is Jessica from
Sears about your dishwasher
appointment today 4-5pm.
Please call back to confirm."
        
        OpenAI-->>OutVoice: Voicemail left (22 seconds)
        OutVoice->>DB: Update: voicemail_left
        Note right of DB: Status: voicemail_left
Timestamp: 13:30
        
        Note over Journey,OutEmail: Immediately send SMS and Email (parallel)
        
        par Send SMS
            OutVoice->>Journey: Trigger SMS
            Journey->>OutSMS: Send confirmation SMS
            OutSMS->>Twilio: Send message
            Twilio->>Customer: 💬 "Hi, we called about your appointment today at 4-5pm. Reply YES to confirm or call us back."
            OutSMS->>DB: Log: sms_sent
        and Send Email
            OutVoice->>Journey: Trigger Email
            Journey->>OutEmail: Send confirmation email
            OutEmail->>Graph: Send message
            Graph->>Customer: 📧 "Appointment Confirmation - Please Respond"
            OutEmail->>DB: Log: email_sent
        end
        
        Note right of DB: All channels contacted:
• Voicemail: 13:30
• SMS: 13:30
• Email: 13:30
        
        Journey->>Journey: Wait for response on ANY channel
        Note over Journey,DB: ⏳ Waiting for customer response
    end

sequenceDiagram
    participant Customer as 👤 Customer
    participant Twilio as 📱 Twilio
    participant InVoice as 📥 Inbound Voice
    participant DB as 🗄️ MongoDB State
    participant OpenAI as 🎤 OpenAI Realtime

    Note over Customer,InVoice: Customer received: Voicemail + SMS + Email
    
    Customer->>Twilio: 📞 Calls company number
    Twilio->>InVoice: Incoming call detected
    
    InVoice->>DB: Lookup by phone: +1234567890
    DB-->>InVoice: Load communication state
    Note right of DB: Context loaded:
• Voicemail left at 13:30
• SMS sent at 13:30
• Email sent at 13:30
• Appointment: 4-5pm today
    
    InVoice->>OpenAI: Connect with full context
    OpenAI->>Twilio: Ready for conversation
    Twilio->>Customer: ✓ Connected
    
    OpenAI->>Customer: 🗣️ "Hi! Thanks for calling back about your appointment today..."
    Customer->>OpenAI: 🗣️ "Yes, I got your messages. I'll be home."
    OpenAI->>Customer: 🗣️ "Perfect! Tech Tom will arrive between 4-5pm. $89 trip charge."
    Customer->>OpenAI: 🗣️ "Great, thanks!"
    
    OpenAI-->>InVoice: Call completed
    InVoice->>DB: Update: confirmed_via_callback, workflow_complete
    Note right of DB: Status: confirmed
Channel: voice_inbound
Response time: 25 min
    Note over InVoice,DB: ✅ SUCCESS - Workflow Complete

sequenceDiagram
    participant Customer as 👤 Customer
    participant Twilio as 📱 Twilio
    participant InSMS as 📥 Inbound SMS
    participant InVoice as 📥 Inbound Voice
    participant DB as 🗄️ MongoDB State
    participant OpenAI as 🎤 OpenAI Realtime

    Note over Customer,InSMS: Customer received: Voicemail + SMS + Email
    
    alt Customer Replies to SMS
        Customer->>Twilio: 💬 "YES"
        Twilio->>InSMS: Incoming SMS webhook
        
        InSMS->>DB: Load communication state
        DB-->>InSMS: Full context loaded
        Note right of DB: Complete history:
• Voicemail at 13:30
• SMS at 13:30
• Email at 13:30
• Customer replied
        
        InSMS->>InSMS: Analyze response: "YES" = confirmation
        InSMS->>Twilio: Send confirmation reply
        Twilio->>Customer: 💬 "Great! Tech Tom arriving 4-5pm. $89 trip charge. See you then!"
        
        InSMS->>DB: Update: confirmed_via_sms, workflow_complete
        Note right of DB: Status: confirmed
Channel: sms_inbound
Response time: 12 min
        Note over InSMS,DB: ✅ SUCCESS - Workflow Complete
    
    else Customer Calls Instead of SMS Reply
        Customer->>Twilio: 📞 Calls company number
        Twilio->>InVoice: Incoming call
        
        InVoice->>DB: Load communication state
        DB-->>InVoice: Full context loaded
        Note right of DB: Context:
• Voicemail at 13:30
• SMS at 13:30
• Email at 13:30
• Customer calling back
        
        InVoice->>OpenAI: Connect with full context
        OpenAI->>Customer: 🗣️ "Hi! Thanks for calling about your appointment today..."
        Customer->>OpenAI: 🗣️ "Yes, I can confirm. I'll be home at 4pm."
        OpenAI->>Customer: 🗣️ "Perfect! We'll see you between 4-5pm."
        
        OpenAI-->>InVoice: Call completed
        InVoice->>DB: Update: confirmed_via_callback, workflow_complete
        Note right of DB: Status: confirmed
Channel: voice_inbound
        Note over InVoice,DB: ✅ SUCCESS - Workflow Complete
    end

sequenceDiagram
    participant Customer as 👤 Customer
    participant Graph as 📮 MS Graph
    participant InEmail as 📥 Inbound Email
    participant Twilio as 📱 Twilio
    participant InVoice as 📥 Inbound Voice
    participant DB as 🗄️ MongoDB State
    participant OpenAI as 🎤 OpenAI Realtime

    Note over Customer,InEmail: Customer received: Voicemail + SMS + Email
    
    alt Customer Replies to Email
        Customer->>Graph: 📧 Reply: "Yes, I'll be there"
        Graph->>InEmail: Incoming email webhook
        
        InEmail->>DB: Load communication state
        DB-->>InEmail: Full context loaded
        Note right of DB: Complete history:
• Voicemail at 13:30
• SMS at 13:30
• Email at 13:30
• Customer replied
        
        InEmail->>InEmail: Analyze email: confirmation detected
        InEmail->>Graph: Send auto-reply
        Graph->>Customer: 📧 "Thank you! Confirmed for 4-5pm. Tech Tom will arrive with $89 trip charge."
        
        InEmail->>DB: Update: confirmed_via_email, workflow_complete
        Note right of DB: Status: confirmed
Channel: email_inbound
Response time: 35 min
        Note over InEmail,DB: ✅ SUCCESS - Workflow Complete
    
    else Customer Calls Instead of Email Reply
        Customer->>Twilio: 📞 Calls company number
        Twilio->>InVoice: Incoming call
        
        InVoice->>DB: Load communication state
        DB-->>InVoice: Complete communication history
        Note right of DB: All channels contacted:
• Voicemail at 13:30
• SMS at 13:30
• Email at 13:30
• Customer calling back
        
        InVoice->>OpenAI: Connect with full context
        OpenAI->>Customer: 🗣️ "Thanks for calling back about your appointment today!"
        Customer->>OpenAI: 🗣️ "Yes, I confirm, I'll be there at 4pm."
        OpenAI->>Customer: 🗣️ "Perfect! We'll see you between 4-5pm."
        
        OpenAI-->>InVoice: Call completed
        InVoice->>DB: Update: confirmed_via_callback, workflow_complete
        Note right of DB: Status: confirmed
Channel: voice_inbound
        Note over InVoice,DB: ✅ SUCCESS - Workflow Complete
    end

sequenceDiagram
    participant Customer as 👤 Customer
    participant Twilio as 📱 Twilio
    participant Webhook as 🔗 Webhook
    participant InVoice as 📥 Inbound Voice
    participant DB as 🗄️ MongoDB
    participant KB as 📚 Knowledge Base
    participant OpenAI as 🎤 OpenAI Realtime
    participant SMS as 💬 SMS Agent
    participant Email as 📧 Email Agent

    Customer->>Twilio: 📞 Calls company number
    Twilio->>Webhook: Incoming call webhook
    Webhook->>InVoice: Trigger inbound agent
    
    InVoice->>DB: Lookup by phone: +1234567890
    DB-->>InVoice: Customer ID: cust_xyz789
    
    InVoice->>DB: Get conversation history
    DB-->>InVoice: • AI called 2 hours ago for confirmation
• Left voicemail: "Appt tomorrow 2-4pm"
• Sent SMS + Email at same time
• Customer calling back
    
    InVoice->>KB: Search relevant context
    KB-->>InVoice: Rescheduling policies loaded
    
    InVoice->>OpenAI: Connect with full context
    Note right of OpenAI: Context includes:
• Confirmation call attempt
• Voicemail + SMS + Email sent
• Appt: Tomorrow 2-4pm
• Customer history + policies
    
    OpenAI->>Twilio: Ready for conversation
    Twilio->>Customer: ✓ Connected
    
    OpenAI->>Customer: 🗣️ "Hi! Thanks for calling back. I see we reached out about confirming your appointment tomorrow 2-4pm..."
    Customer->>OpenAI: 🗣️ "Yes, I got your message. I need to reschedule - I can't make it tomorrow"
    
    OpenAI->>Customer: 🗣️ "I can help! What date works better?"
    Customer->>OpenAI: 🗣️ "Next Thursday if possible"
    
    OpenAI->>DB: Check availability
    DB-->>OpenAI: Thursday 2-4pm available ✓
    
    OpenAI->>Customer: 🗣️ "Thursday 2-4pm is open, should I confirm?"
    Customer->>OpenAI: 🗣️ "Perfect!"
    
    Note over OpenAI,SMS: Mid-call actions
    OpenAI->>SMS: Send confirmation SMS
    SMS->>Customer: 📱 "Rescheduled to Thu 2-4pm. Confirmation details sent."
    
    OpenAI->>Email: Send confirmation email
    Email->>Customer: 📧 "Appointment Confirmation Details"
    
    OpenAI->>Customer: 🗣️ "Sent confirmation via SMS and email. Anything else?"
    Customer->>OpenAI: 🗣️ "No, that's all. Thanks!"
    
    OpenAI-->>InVoice: Call complete
    
    InVoice->>DB: Update conversation
    Note right of DB: • Call duration: 1m 45s
• Outcome: Rescheduled
• Sentiment: Positive
• Next action: None

graph LR
    A[🎧 Jessica
Home Manager calls] --> B[📞 Hi Sarah, about
your dishwasher today]
    B --> C[✓ Narrow window:
1-5pm → 4-5pm]
    C --> D[✓ Tech Tom
will arrive 4-5pm]
    D --> E[✓ Confirm: Adult 18+,
pets secured, payment]
    E --> F[✓ Promise: I'll call
when he's 30 min away]
    F --> G[💬 Send confirmation
SMS immediately]
    G --> H[💾 Log: confirmed,
customer prepared]
    
    style A fill:#e8f0f5,stroke:#1e4d4d,stroke-width:2px
    style C fill:#f5f0e8,stroke:#8b6f47,stroke-width:2px
    style G fill:#f5e8eb,stroke:#b76e79,stroke-width:2px
    style H fill:#e8f5f0,stroke:#2d5f4f,stroke-width:2px

graph LR
    A[⏰ 30 minutes
before arrival] --> B[🎧 Jessica calls
Sarah again]
    B --> C[📞 Tom finishing up,
heading your way]
    C --> D[📞 Should arrive
around 4pm]
    D --> E[👤 Customer:
All set! Cat's ready]
    E --> F[💾 Log: on_way_call
32 seconds]
    
    style A fill:#f5f0e8,stroke:#8b6f47,stroke-width:2px
    style B fill:#e8f0f5,stroke:#1a1f3a,stroke-width:2px
    style E fill:#e8f5f0,stroke:#2d5f4f,stroke-width:2px
    style F fill:#e8f5f0,stroke:#1e4d4d,stroke-width:2px

graph LR
    A[🚗 Tom arrives
NO CALL NEEDED] --> B[🚪 Knocks on
door]
    B --> C[👤 Sarah answers:
Hi, you must be Tom]
    C --> D[🔧 Tom: Thanks for
securing cat!]
    D --> E[🔧 Diagnoses:
Failed drain pump]
    E --> F[💵 Quote: $285 total
$89 + $196]
    F --> G[✅ Sarah approves
immediately]
    G --> H[🔧 Repair completed
tested working]
    H --> I[💳 Payment via
credit card]
    I --> J[📱 Auto SMS follow-up:
How did we do?]
    
    style A fill:#e8f0f5,stroke:#1e4d4d,stroke-width:2px
    style E fill:#f5f0e8,stroke:#8b6f47,stroke-width:2px
    style G fill:#e8f5f0,stroke:#2d5f4f,stroke-width:2px
    style J fill:#f5e8eb,stroke:#b76e79,stroke-width:2px

graph TB
    subgraph internet["🌐 Internet"]
        Users["👥 Users
(Customers)"]
        Twilio["📱 Twilio
Voice/SMS"]
        OpenAI["🤖 OpenAI
Realtime API"]
        MSGraph["📮 Microsoft
Graph API"]
    end
    
    subgraph aws["☁️ AWS Cloud"]
        subgraph network["VPC - Private Network"]
            ALB["⚖️ Application Load Balancer
──────────────
1. SSL/TLS Termination
2. Health Checks
3. Auto-scaling"]
            
            subgraph compute["Compute Layer (ECS/Fargate)"]
                API1["🚀 FastAPI Instance 1
──────────────
1. WebSocket Support
2. Async I/O
3. Agent Orchestration"]
                API2["🚀 FastAPI Instance 2
──────────────
1. WebSocket Support
2. Async I/O
3. Agent Orchestration"]
                API3["🚀 FastAPI Instance N
──────────────
1. Auto-scaled
2. On-demand"]
            end
            
            subgraph cache["Caching Layer"]
                Redis["🔴 Redis
──────────
1. Session Cache
2. Rate Limiting
3. Queue System"]
            end
        end
        
        S3["📦 S3 Bucket
──────────
1. Call Recordings
2. Attachments
3. Static Assets"]
    end
    
    subgraph database["🗄️ Database Layer"]
        MongoDB["🍃 MongoDB Atlas
(Managed Cluster)
──────────────
1. Conversations
2. Customers
3. Interactions
4. Context
5. Analytics
6. Knowledge_base"]
    end
    
    subgraph monitoring["📊 Monitoring & Logging"]
        CloudWatch["📈 AWS CloudWatch
──────────
1. Logs
2. Metrics
3. Alarms"]
        Grafana["📊 KAIros Pulse
──────────
1. Dashboard
2. Visualization"]
    end
    
    Users -->|HTTPS| ALB
    Twilio -->|Webhooks| ALB
    
    ALB --> API1
    ALB --> API2
    ALB --> API3
    
    API1 <--> Redis
    API2 <--> Redis
    API3 <--> Redis
    
    API1 --> MongoDB
    API2 --> MongoDB
    API3 --> MongoDB
    
    API1 --> S3
    API2 --> S3
    API3 --> S3
    
    API1 <-->|WebSocket| OpenAI
    API2 <-->|WebSocket| OpenAI
    
    API1 <-->|REST API| Twilio
    API2 <-->|REST API| Twilio
    
    API1 <-->|REST API| MSGraph
    API2 <-->|REST API| MSGraph
    
    API1 --> CloudWatch
    API2 --> CloudWatch
    API3 --> CloudWatch
    
    CloudWatch --> Grafana
    
    style internet fill:#e8f0f5,stroke:#1e4d4d,stroke-width:2px
    style aws fill:#f5f0e8,stroke:#8b6f47,stroke-width:3px
    style network fill:#f0e8f0,stroke:#4a2c5c,stroke-width:2px
    style compute fill:#e8f5f0,stroke:#2d5f4f,stroke-width:2px
    style cache fill:#f5e8eb,stroke:#6b2c3e,stroke-width:2px
    style database fill:#f5f0e0,stroke:#c9a961,stroke-width:3px
    style monitoring fill:#e8f0f0,stroke:#1e4d4d,stroke-width:2px