🌟 Solace AI: Your Empathetic Digital Confidant

An advanced mental health AI companion that understands your emotions and personality to provide personalized support through natural conversations. Powered by a sophisticated multi-agent architecture, Solace AI creates a judgment-free space where you can express yourself freely and receive compassionate guidance tailored specifically for your unique needs.

📋 Table of Contents

• ✨ Features

• 🎬 Demo

• 🔧 Technology Stack

• 📂 Project Structure
• 📥 Installation

• 🚀 Getting Started

• 🎙️ Voice Interaction

• 🧠 Advanced Features

• 🏗️ Project Architecture

• 👥 Contributing
• 📄 License
• 📬 Contact

🌟 Features

Sophisticated emotion detection from text and voice	Natural voice conversations with emotion recognition	Tailored support based on your unique traits
Evidence-based therapeutic approaches	Remembers conversations and adapts over time	Crisis detection and appropriate interventions

Core Capabilities

• 🎭 Emotional Intelligence: Analyzes emotions in both text and voice to respond appropriately using advanced sentiment analysis algorithms
• 🧠 Personality Adaptation: Tailors responses based on Big Five and MBTI personality assessments to match your communication style
• 🗣️ Voice Conversations: Natural speech recognition and response using Whisper V3 Turbo ASR with emotional tone detection
• 💬 Therapeutic Techniques: Integrates CBT, mindfulness, Solution-Focused Brief Therapy, and other evidence-based approaches
• 🔄 Contextual Memory: Employs vector databases and semantic memory to remember past interactions for more meaningful conversations
• 🛡️ Safety Protocols: Sophisticated safety agent monitors for crisis indicators and provides appropriate guidance
• 🎙️ Voice Style Customization: Multiple voice styles including warm, calm, professional, and more with celebrity voice cloning capabilities
• 🌐 Multi-Agent Architecture: Orchestrates specialized agents (emotion, safety, therapy, personality, diagnosis) for comprehensive support

🎬 Demo

Text-Based Interaction

🧠 Assistant: Hi there. I'm here to listen and support you in a safe, judgment-free space. What's on your mind today?
User: I've been feeling really overwhelmed lately with work and family stress
🤔 Processing...
🧠 Assistant: I can hear how overwhelmed you're feeling right now, juggling both work and family responsibilities. That's a lot to carry on your shoulders. It's completely natural to feel this way when multiple areas of life are demanding your attention and energy.
Based on what you've shared, it might help to create some boundaries between these different roles. Perhaps you could try a simple breathing technique when transitioning between work and family time - just taking 3-5 deep breaths while acknowledging that you're switching contexts.
Would you like to talk more about what specific aspects of work or family life are feeling most challenging right now?

Voice Interaction

🔧 Technology Stack

Key Library Dependencies

# Core NLP and AI
gemini-api==1.0.1        # Google Gemini API integration
torch==2.0.1             # PyTorch for deep learning
transformers==4.30.0     # Hugging Face Transformers for NLP models
langchain==0.0.267       # Framework for LLM application development
Voice Processing
whisper-v3-turbo==1.2.0  # Advanced speech recognition
speechrecognition==3.10.0 # Speech-to-text processing
pyttsx3==2.90            # Text-to-speech conversion
Memory and Data Management
chromadb==0.4.6          # Vector database for embeddings
sentence-transformers==2.2.2 # Text embeddings generation
UI and Visualization
streamlit==1.24.0        # Web interface creation
plotly==5.15.0           # Interactive visualizations

📂 Project Structure

Contextual-Chatbot/
├── app.py                   # Streamlit web application entry point
├── chat.py                  # Command-line chat interface
├── main.py                  # Main application initialization
├── config.yaml              # Main configuration file
├── celebrityvoiceclone_demo.py # Celebrity voice cloning demo
├── requirements.txt         # Core dependencies
├── requirements_voice.txt   # Voice-related dependencies
├── Dockerfile               # Docker containerization
├── setup_voice.py           # Voice capabilities setup
├── src/
│   ├── agents/              # Agent components
│   │   ├── agent_orchestrator.py  # Manages agent coordination
│   │   ├── base_agent.py    # Abstract agent class
│   │   ├── chat_agent.py    # Manages conversation flow
│   │   ├── emotion_agent.py # Emotion detection and response
│   │   ├── safety_agent.py  # Crisis detection and intervention
│   │   ├── therapy_agent.py # Therapeutic techniques
│   │   └── personality_agent.py # Personality adaptation
│   ├── api/                 # API endpoints
│   │   └── router.py        # API routing
│   ├── components/          # Core components
│   │   ├── audio_processor.py # Audio processing utilities
│   │   ├── text_processor.py  # Text processing utilities
│   │   └── response_generator.py # Response generation
│   ├── memory/              # Memory management
│   │   ├── vector_database.py # Vector storage for embeddings
│   │   ├── conversation_memory.py # Short-term memory
│   │   └── semantic_memory.py # Long-term memory
│   ├── models/              # LLM integration
│   │   ├── llm.py           # Language model interface
│   │   └── gemini_integration.py # Gemini-specific implementation
│   ├── personality/         # Personality assessment
│   │   ├── big_five.py      # Big Five personality model
│   │   └── mbti.py          # MBTI personality model
│   ├── diagnosis/           # Mental health assessment
│   │   ├── anxiety.py       # Anxiety screening tools
│   │   ├── depression.py    # Depression screening tools
│   │   └── stress.py        # Stress assessment tools
│   ├── knowledge/           # Therapeutic knowledge base
│   │   ├── cbt_techniques.py # Cognitive behavioral therapy
│   │   ├── mindfulness.py   # Mindfulness practices
│   │   └── solution_focused.py # Solution-focused brief therapy
│   ├── planner/             # Task planning
│   │   └── planner.py       # Task planning functionality
│   ├── ui/                  # Streamlit UI components
│   │   ├── sidebar.py       # Sidebar navigation
│   │   ├── chat_interface.py # Chat UI
│   │   └── visualizations.py # Data visualizations
│   └── utils/               # Helper utilities
│       ├── config.py        # Configuration management
│       ├── text_utils.py    # Text processing utilities
│       ├── audio.py         # Audio processing utilities
│       ├── file_utils.py    # File handling utilities
│       └── voicecloneintegration.py # Voice cloning utilities
├── tests/                   # Test suite
│   ├── unit/                # Unit tests
│   └── integration/         # Integration tests
├── scripts/                 # Utility scripts
│   └── setup_environment.sh # Environment setup script
└── docs/                    # Documentation

Each module is designed to be modular and reusable, enabling easy extension and customization of Solace AI's capabilities.

📥 Installation

Prerequisites

• Python 3.8 or higher
• pip (Python package installer)
• Git
• 8GB+ RAM recommended for optimal performance

Setup

• Clone the repository

git clone https://github.com/Rayyan9477/Contextual-Chatbot.git
   cd Contextual-Chatbot

• Create a virtual environment (recommended)

python -m venv venv
   source venv/bin/activate  # On Linux/macOS
   venv\Scripts\activate.bat  # On Windows

• Install core dependencies

pip install -r requirements.txt

• Setup voice capabilities (optional)

pip install -r requirements_voice.txt
   python setup_voice.py

• Set up environment variables

# Create a .env file with your API keys
   echo "GEMINIAPIKEY=yourapikey_here" > .env

Docker Installation (Alternative)

For containerized deployment:

• Build the Docker image

docker build -t solace-ai .

• Run the Docker container

docker run -p 5000:5000 solace-ai

🚀 Getting Started

API Server for Mobile App Integration

Start the API server for mobile app integration:

python launch_api.py

This launches the REST API server with:

• Complete API documentation at /docs
• Endpoints for chat, assessment, voice processing
• Mobile app integration capabilities
• User profile management

Command Line Interface

For a simple command-line experience:

python start.py

Optional flags:

• --voice-only: Use only voice input for conversation
• --text-only: Use only text input (disables voice features)
• --model turbo: Specify Whisper model version (options: base, small, medium, large)
• --voice-style warm: Choose voice style (options: warm, calm, professional, excited, sad)
• --personality-type supportive_counselor: Select chatbot personality
• --memory-size 10: Set the number of conversation turns to remember
• --debug: Enable debug mode with detailed logging

Configuration

You can customize Solace AI by editing the config.yaml file:

# Example configuration settings
agent_settings:
  emotion_threshold: 0.7
  safetycheckfrequency: 2
  therapyapproaches: ["cbt", "mindfulness", "solutionfocused"]
  
voice_settings:
  default_style: "warm"
  language: "en-US"
  
memory_settings:
  vectordbpath: "./data/memory"
  conversation_turns: 10

🎙️ Voice Interaction

Solace AI provides advanced voice interaction capabilities that create natural, emotionally-aware conversations:

Voice Recognition Architecture

• Whisper V3 Turbo: State-of-the-art speech recognition with 99% accuracy
• Real-time Processing: Low-latency voice recognition for fluid conversations
• Emotion Detection: Advanced algorithms analyze tone, pitch, and pacing to detect emotional states
• Multilingual Support: Works across 40+ languages with automatic language detection

Text-to-Speech System

The voice synthesis system features adaptive voice styles that respond to the emotional context of the conversation:

# Sample voice style customization
async def respondtoemotion(userinput, detectedemotion):
    if detected_emotion == "anxious":
        await voicemanager.speaktext(
            "I understand you're feeling anxious right now. Let's take a deep breath together.",
            style="calm",
            rate=0.85
        )
    elif detected_emotion == "sad":
        await voicemanager.speaktext(
            "I hear that you're feeling down. It's okay to feel this way.",
            style="warm",
            pitch_adjustment=-0.05
        )

Voice Styles and Parameters

Choose from multiple voice styles to match your preferences:

| Style | Description | Use Case | Parameters | |-------|-------------|----------|------------| | Warm | Compassionate and supportive | Emotional support | pitch=0.0, rate=1.0 | | Calm | Soothing and peaceful | Anxiety reduction | pitch=-0.1, rate=0.9 | | Professional | Clear and structured | Information delivery | pitch=0.05, rate=1.05 | | Excited | Energetic and motivating | Encouragement | pitch=0.15, rate=1.1 | | Sad | Empathetic for difficult moments | Grief support | pitch=-0.15, rate=0.9 |

Voice Integration Flow

sequenceDiagram
    User->>System: Speak (Audio Input)
    System->>Speech Recognition: Process Audio
    Speech Recognition->>Emotion Detector: Audio Features
    Speech Recognition->>Text Processor: Transcribed Text
    Emotion Detector->>Agent Orchestrator: Emotional State
    Text Processor->>Agent Orchestrator: Processed Text
    Agent Orchestrator->>Response Generator: Generate Response
    Response Generator->>Voice Synthesizer: Text Response
    Voice Synthesizer->>System: Select Voice Style
    System->>User: Speak (Audio Output)

Celebrity Voice Cloning Integration

Our advanced voice cloning system allows for personalized interactions with familiar voices:

# Example usage of celebrity voice cloning
python celebrityvoiceclone_demo.py --celebrity "Morgan Freeman" --text "Welcome to your mindfulness session. Let's begin with a moment of calm reflection."

This feature can be customized through the src/utils/voicecloneintegration.py module for creating new voice profiles.

🧠 Advanced Features

Celebrity Voice Cloning

Solace AI includes an advanced voice cloning system that can replicate celebrity voices for a more engaging and personalized experience:

# Clone a celebrity voice and generate audio
python celebrityvoiceclone_demo.py --celebrity "Morgan Freeman" --text "Hello, I am here to support you on your journey."
Clone with specific emotional tone
python celebrityvoiceclone_demo.py --celebrity "Oprah Winfrey" --text "Let's explore your thoughts together." --emotion "warm"
Save the generated audio to a file
python celebrityvoiceclonedemo.py --celebrity "David Attenborough" --text "The journey to mental wellness is remarkable." --output "meditationintro.mp3"

The voice cloning system uses a sophisticated neural network architecture to analyze and reproduce the unique characteristics of various voices while maintaining natural-sounding speech patterns.

Comprehensive Mental Health Assessment System

Solace AI includes an integrated assessment system that combines multiple evaluation methods:

The assessment system uses a combination of direct questioning, conversation analysis, and pattern recognition to build a comprehensive profile of the user's mental health needs and preferences.

Therapeutic Approach Integration

Solace AI integrates evidence-based therapeutic techniques from various psychological approaches:

graph TD
    A[User Input] --> B{Therapy Agent}
    B --> C[Approach Selection]
    C --> D[Cognitive Behavioral Therapy]
    C --> E[Mindfulness Practices]
    C --> F[Solution-Focused Brief Therapy]
    C --> G[Motivational Interviewing]
    C --> H[Acceptance and Commitment Therapy]
    D --> I[Response Generation]
    E --> I
    F --> I
    G --> I
    H --> I
    I --> J[User Output]

Therapeutic Techniques Available

• Cognitive Behavioral Therapy: Identifies and challenges negative thought patterns

# Example CBT implementation
  def cognitiverestructuring(negativethought):
      evidencefor = promptforevidence(negativethought, "supporting")
      evidenceagainst = promptforevidence(negativethought, "contradicting")
      alternativeperspective = generatebalancedthought(negativethought, evidencefor, evidenceagainst)
      return alternative_perspective

• Mindfulness and Meditation: Guided exercises for present-moment awareness
• Solution-Focused Brief Therapy: Focuses on solutions rather than problems
• Motivational Interviewing: Resolves ambivalence and increases motivation for change
• Acceptance and Commitment Therapy: Promotes psychological flexibility and values-based action

Real-time Emotion Tracking and Visualization

The system tracks emotional states throughout conversations and provides visualizations to help users understand their emotional patterns:

# Example emotion tracking visualization
def generateemotiontimeline(conversation_history):
    emotions = extractemotions(conversationhistory)
    timepoints = extracttimestamps(conversation_history)
    
    fig = plt.figure(figsize=(10, 6))
    plt.plot(time_points, emotions['joy'], 'g-', label='Joy')
    plt.plot(time_points, emotions['sadness'], 'b-', label='Sadness')
    plt.plot(time_points, emotions['anger'], 'r-', label='Anger')
    plt.plot(time_points, emotions['anxiety'], 'y-', label='Anxiety')
    plt.legend()
    plt.title('Emotional Journey')
    
    return fig

This feature enables users to gain insights into their emotional patterns over time, helping them develop greater emotional awareness and regulation strategies.

🏗️ Project Architecture

Solace AI uses a sophisticated multi-agent architecture designed for modularity, extensibility, and efficient task handling:

graph TD
    A[User Input] --> B(Agent Orchestrator)
    B --> C{Task Distribution}
    C --> D[Emotion Agent]
    C --> E[Safety Agent]
    C --> F[Chat Agent]
    C --> G[Therapy Agent]
    C --> H[Personality Agent]
    C --> I[Diagnosis Agent]
    D --> J{Vector Database}
    E --> J
    F --> J
    G --> J
    H --> J
    I --> J
    J --> K[Response Generation]
    K --> L[User Output]
    style B fill:#f9f,stroke:#333,stroke-width:2px
    style J fill:#ccf,stroke:#333,stroke-width:2px

Agent Components

• Agent Orchestrator: Central coordinator that manages task distribution, information flow between agents, and aggregating results into coherent responses. Handles the lifecycle of agents and their interactions.

• Emotion Agent: Detects and analyzes emotional content in text and voice inputs using sentiment analysis and tone recognition. Helps tailor responses to match the user's emotional state.

• Safety Agent: Continuously monitors conversations for crisis indicators, self-harm signals, or other safety concerns. Provides appropriate interventions and resources when needed.

• Chat Agent: Manages core conversation flow, ensuring natural dialogue and maintaining context. Interfaces directly with the LLM for response generation.

• Therapy Agent: Implements evidence-based therapeutic techniques including CBT, mindfulness, Solution-Focused Brief Therapy, and more. Selects appropriate therapeutic approaches based on user needs.

• Personality Agent: Adapts responses based on personality assessments (Big Five, MBTI) to match the user's communication style and preferences. Maintains a consistent personality profile.

• Diagnosis Agent: Performs mental health assessments and screening for anxiety, depression, and stress. Provides personalized insights based on recognized patterns.

Memory Systems Architecture

• Conversational Memory: Short-term memory that tracks immediate conversation history and context for coherent dialogue.

• Semantic Memory: Long-term storage for important user insights, preferences, and patterns. Enables the system to build a comprehensive understanding of the user over time.

• Vector Database: Employs embedding-based storage for efficient similarity search. Retrieves relevant context based on semantic meaning rather than keyword matching.

# Example of memory interaction
class VectorDB:
    def init(self, config):
        self.config = config
        self.collection = self.client.getorcreatecollection(name="usermemories")
        
    def add_data(self, data):
        self.collection.add(
            embeddings=[self.llm.get_embedding(d) for d in data],
            documents=data,
            ids=[str(uuid.uuid4()) for _ in data]
        )
def querydata(self, query, nresults=5):         return self.collection.query(             queryembeddings=[self.llm.getembedding(query)],             nresults=nresults         )

👥 Contributing

Contributions are welcome! Here's how to get started:

• Fork the repository
• Create a feature branch (git checkout -b feature/amazing-feature)
• Commit your changes (git commit -m 'Add amazing feature')
• Push to the branch (git push origin feature/amazing-feature)
• Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

📬 Contact

• GitHub: Rayyan9477
• LinkedIn: Rayyan Ahmed
• Email: rayyanahmed265@yahoo.com