Topics

#abap #llm #sap #sap-hana #saperp #sapr3 #tokenizer

Language breakdown

ABAP 96.0%

Python 3.8%

ABAP CDS 0.2%

▸ Files click to expand

README

ZLLM: A LangChain-lite for ABAP

ZLLM is a sophisticated "LangChain-lite" framework that brings the power of Large Language Model (LLM) orchestration to SAP/ABAP systems. Think of it as LangChain's enterprise-ready cousin, specifically designed for ABAP developers who need to build complex AI-powered workflows within their SAP landscape.

Why "LangChain-lite for ABAP"?

Like LangChain, ZLLM provides:

Chain Composition: Build complex workflows by chaining multiple LLM calls
Prompt Templates: Sophisticated template engine for dynamic prompt generation
Memory & State: Result propagation between steps maintains context
Modular Design: Steps and flows are composable building blocks

But optimized for enterprise ABAP:

Native ABAP: No external dependencies, runs entirely within SAP
Lazy Execution: Non-blocking async operations for better performance
Enterprise Features: Built-in caching, load balancing, and error handling
SAP Integration: Seamless integration with ABAP structures and tables

For a deep-dive into the framework and its capabilities, please see the GUIDE.md.

Core Features

🔗 Chaining & Orchestration

Sequential Flows: Chain multiple LLM calls with automatic result propagation
Parallel Execution: Process multiple inputs concurrently
Nested Flows: Compose flows within flows for complex workflows
Lazy Evaluation: Start operations without blocking, collect results when needed

🎯 Advanced Template Engine

Deep Structure Navigation: Access nested ABAP structures with dot notation ({T-CUSTOMER-NAME})
Table Processing: Automatically iterate over internal tables
JSON Integration: Seamless conversion between ABAP structures and JSON
Pattern Substitution: Dynamic prompt generation with result propagation

⚡ Performance & Reliability

Built-in Cache System: Intelligent caching reduces API calls and costs
Load Balancer: Route requests to different models based on complexity
Token Prediction: Estimate token usage without API calls (99.7% accuracy)
Error Recovery: Robust error handling with retry mechanisms

🔐 Security Features

Encoded Storage: Encryption is ENABLED by default - all virtual filesystem files are automatically encoded using symmetric encryption
Configurable Seeds: Set custom encryption seeds via ZLLM_CODEC parameter
API Key Protection: Credentials are never stored in plain text
User Isolation: Each user's data is encoded with their specific seed

🛠️ Developer Experience

Simple API: Intuitive interfaces for common tasks
REPL Environment: Interactive development and testing
Multiple LLM Support: Easy switching between different models
Environment Config: Manage configurations with .env files

Quick Start

Simple Query

" Create LLM instance and execute a simple query
DATA(lollm) = zclllm=>new( 'DEFAULT.ENV' ).
DATA(lostep) = zclllm00steplazy=>newfrom_string(
  iv_usr = 'Explain cloud computing in simple terms'
  iollm = lollm
).
DATA(lrresult) = lostep->exec( ).

Chained Workflow

" Create a two-step flow: generate content, then summarize it
DATA(lostep1) = zclllm00steplazy=>newfrom_string(
  iv_usr = 'Write a detailed explanation of quantum computing'
  iollm = lollm
).
DATA(lostep2) = zclllm00steplazy=>newfrom_string(   iv_usr = 'Summarize this in 3 bullet points: {T}'  " {T} contains result from step1   iollm = lollm ).
DATA(loflow) = zclllm00flow_lazy=>new(   VALUE #( ( lostep1 ) ( lostep2 ) ) ).
DATA(loresult) = loflow->exec( ).

Using Templates with ABAP Data

" Process structured data with templates
DATA: BEGIN OF ls_customer,
        name TYPE string VALUE 'ACME Corp',
        revenue TYPE p VALUE '1000000',
        industry TYPE string VALUE 'Technology',
      END OF ls_customer.
DATA(lopattern) = zclllm00pat=>new(    'Analyze this customer: Name: {T-NAME}, Revenue: ${T-REVENUE}, Industry: {T-INDUSTRY}' ).
DATA(lostep) = zclllm00steplazy=>newfrom_pat(   iopatusr = lo_pattern   iollm     = lollm ).
DATA(lrresult) = lostep->exec( REF #( ls_customer ) ).

Advanced Features

🚀 Parallel Processing

" Process multiple documents in parallel
DATA(loparallelstep) = zclllm00steplazy_parallel=>new(
  iostep = loanalysis_step
  iollm  = lollm
).
DATA(loresult) = loparallelstep->start( REF #( ltdocuments ) ).

💾 Smart Caching

" Caching is ENABLED by default - all LLM responses are automatically cached
DATA(lollm) = zclllm=>new( 'DEFAULT.ENV' ).  " Uses default cache
" To disable caching, inject a dummy cache: DATA(locachenever) = zclllm00cachenever=>new( ). DATA(lollmnocache) = zclllm=>new(    iv_config = 'DEFAULT.ENV'   iocache  = locache_never  " This disables caching ).
" To use custom cache with specific seed: DATA(locachecustom) = zclllm00cache=>new( ivseed = 42 ). DATA(lollmcustom) = zcl_llm=>new(    iv_config = 'DEFAULT.ENV'   iocache  = locache_custom ).

⚖️ Load Balancing

" Route to different models based on complexity
DATA(lollmcomposite) = zclllm00llmlazy_composite=>new(
  iollm       = lollm_mini      " For simple queries (<1000 tokens)
  iollmexp   = lollmmaxi      " For complex queries
  iv_threshold = 1000
).

PREDICTOKEN - Intelligent Token Prediction

ZLLM includes PREDICTOKEN, a sophisticated token prediction system that estimates token counts without API calls:

" Predict tokens before making expensive API calls
DATA(lvpredictedtokens) = zclllm00predictoken=>predicttokensgpt4( lvtext ).
" Use prediction for smart routing IF lvpredictedtokens < 500.   lollm = lollm_mini.  " Use cheaper model ELSE.   lollm = lollm_maxi.  " Use more capable model ENDIF.

| Method | R² Score | Accuracy | Speed | Cost | |--------|----------|----------|-------|------| | Simple heuristics | 0.70-0.83 | ±15-23% | Instant | Free | | PREDICTOKEN | 0.997 | ±3% | <1ms | Free | | Actual tokenizer | 1.000 | Perfect | Network latency | API cost |

Configuration

ZLLM supports multiple LLM providers through flexible .env configuration files:

OpenAI-Compatible APIs (Ollama, LM Studio, OpenAI)

# Local Ollama
API_MODEL=devstral
API_URL=http://192.168.8.107:11434/
API_KEY=ollama
APIMAXTOKEN=64000
APITOKENSPLIT_LIMIT=24000

Azure OpenAI

# Azure OpenAI Configuration
APIAZUREFULL_URL=https://yourdomain.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-01-01-preview
API_KEY=your-azure-api-key
#APIMAXTOKEN=4000
#API_MODEL=gpt-4o  # Optional: Override deployment name

OpenAI Direct

API_MODEL=gpt-4o-mini
API_URL=https://api.openai.com/v1/
API_KEY=sk-your-api-key
APIMAXTOKEN=16384

Getting Started

Run the Onboarding Program: ZLLM00ONBOARD

- Start with basic setup (single LLM configuration) - Press F4 for Expert Mode to configure multiple model variants - Use F3 to test and save your configurations automatically

Try the Demo: ZLLM00FLOW_DEMO - See basic chaining in action
Explore the REPL: ZLLM00REPL - Interactive development environment
Read the Guide: GUIDE.md - Comprehensive documentation with examples

Architecture Overview

ZLLM is built on a sophisticated multi-layered architecture designed for enterprise-grade LLM orchestration:

System Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Application Layer                         │
│  • Demo Programs (ONBOARD, FLOW_DEMO, REPL, SYNC)          │
│  • User Applications and Custom Workflows                   │
├─────────────────────────────────────────────────────────────┤
│                Flow Orchestration Layer                      │
│  • Steps (Lazy, Parallel) • Flows • Formulas • Patterns    │
├─────────────────────────────────────────────────────────────┤
│                 LLM Integration Layer                        │
│  • LLM Client • Load Balancer • Composite Router           │
│  • Response Handler • Retry Logic • Throttling             │
├─────────────────────────────────────────────────────────────┤
│                Support Services Layer                        │
│  • Cache System • File Abstraction • JSON Processing       │
│  • Token Prediction • Markdown Rendering • Encoding        │
├─────────────────────────────────────────────────────────────┤
│                   Data Model Layer                          │
│  • Graph Storage (NODE/EDGE) • Binary Storage (BIN)        │
│  • Cache Tables • Code Analytics • Documentation           │
└─────────────────────────────────────────────────────────────┘

Core Design Principles

Lazy Execution: Operations start immediately but don't block, allowing parallel execution
Composable Design: Steps and flows can be nested infinitely
Template Engine: Powerful pattern system that handles deep structures, tables, and JSON
Error Resilience: Built-in retry logic and error handling
Zero Dependencies: Pure ABAP implementation, no external libraries needed
Secure Storage: Virtual filesystem with automatic encoding using configurable symmetric encryption

Key Component Interactions

graph TB
    subgraph User["User Layer"]
        App[Application]
        Demo[Demo Programs]
    end
    
    subgraph Core["Core Framework"]
        Factory[ZCL_LLM<br/>Factory]
        Step[Step Components]
        Flow[Flow Engine]
        Pattern[Pattern Engine]
    end
    
    subgraph Services["Services"]
        Cache[Cache System]
        Predictor[Token Predictor]
        Files[File System]
    end
    
    subgraph Integration["LLM Integration"]
        Client[LLM Client]
        Balancer[Load Balancer]
        Response[Response Handler]
    end
    
    App --> Factory
    Demo --> Factory
    Factory --> Step
    Factory --> Flow
    Step --> Pattern
    Flow --> Step
    Step --> Client
    Client --> Cache
    Client --> Balancer
    Balancer --> Response
    Client --> Predictor

Database Schema

The framework leverages custom tables for persistence and analytics:

| Table | Purpose | Typical Size | |-------|---------|--------------| | ZLLM00NODE | Graph nodes for code objects | 15,000+ rows | | ZLLM00EDGE | Relationships between nodes | 500+ rows | | ZLLM00CACHE | Persistent cache with encoding | 750+ rows | | ZLLM00BIN | Binary file storage | 250+ rows | | ZLLM00CCLM | Code lifecycle management | Analytics | | ZLLM00DOC | Documentation storage | Metadata |

Key Components

Core Framework Classes

ZCL_LLM: Main factory class for creating LLM instances and components
ZCLLLM00LLMLAZY: HTTP-backed LLM client with caching, throttling, and retry logic
ZCLLLM00LLMLAZY_BALANCER: Intelligently routes requests across multiple LLM instances
ZCLLLM00LLMLAZY_COMPOSITE: Routes to different models based on token complexity

Flow Orchestration

ZCLLLM00STEPLAZY: Basic unit of LLM interaction with lazy execution
ZCLLLM00STEPLAZY_PARALLEL: Enables parallel processing of multiple inputs
ZCLLLM00FLOWLAZY: Chains steps sequentially with automatic result propagation
ZCLLLM00FLOWRESULT: Aggregates and manages flow execution results

Pattern & Template System

ZCLLLM00_PAT: Powerful template engine for dynamic prompt generation
ZCLLLM00PATLIST: Manages collections of patterns
ZCLLLM00_FORMULA: Combines system and user patterns for complex prompts

Supporting Infrastructure

ZCLLLM00_CACHE: Database-backed persistent cache with automatic encoding
ZCLLLM00_CODEC: Symmetric XOR-based encoding for secure storage
ZCLLLM00_PREDICTOKEN: High-accuracy token prediction without API calls
ZCLLLM00_MARKDOWN: Full-featured Markdown to HTML renderer
ZCLLLM00_JSON: Advanced JSON serialization/deserialization
ZCLLLM00_DOTENV: Environment configuration management

File System Abstraction

ZCLLLM00FILELIST_BIN: Binary file storage in database
ZCLLLM00FILELIST_LOCAL: Local file system access
ZCLLLM00FILEBIN: Individual binary file handling
ZCLLLM00FILEMOCK: In-memory file mock for testing

Documentation

GUIDE.md - Comprehensive usage guide with examples and component reference
IMPLEMENTATION.md - Technical implementation details of PREDICTOKEN
ARCHITECTURE.md - Detailed architecture diagrams and component documentation

Package Structure

$ZLLM_00/                       # Main package
├── Core/                       # Core framework classes
│   ├── ZCL_LLM                # Main factory
│   └── ZCLLLM00_*          # Core components
├── Flow/                       # Flow orchestration
│   ├── STEP              # Step components
│   └── FLOW              # Flow components
├── Pattern/                    # Template system
│   ├── _PAT                # Pattern engine
│   └── _FORMULA            # Formula system
├── Infrastructure/             # Supporting services
│   ├── _CACHE              # Cache system
│   ├── FILE              # File abstraction
│   └── _PREDICTOKEN        # Token prediction
└── Demo/                       # Demo programs
    ├── ZLLM00ONBOARD       # Setup wizard
    ├── ZLLM00FLOW_DEMO     # Basic examples
    ├── ZLLM00REPL          # Interactive env
    └── ZLLM00SYNC          # File sync utility

License

MIT License - see LICENSE file for details.

🔗 More in this category