openhuman Architecture: How It Works (Diagrams + Code Walkthrough)

Project Overview

OpenHuman is an innovative application designed to empower users with an intelligent AI agent that acts as a highly personalized digital assistant. Its core business problem is to bridge the gap between human intent and computer interaction by enabling an AI to 'see' (screen intelligence), 'hear' (voice processing), 'learn' (user profile, memory), and 'act' (tool integration) within a user's digital environment. It aims to automate complex workflows, provide proactive assistance, and adapt to individual user preferences, making computer usage more intuitive and efficient. Think of it as giving your AI assistant eyes, ears, and hands to truly understand and help you navigate your computer.

Category
tools
Difficulty
advanced
Tech Stack
Bash, rust
Tags
AI Agents

How openhuman Works

OpenHuman is an innovative application designed to empower users with an intelligent AI agent that acts as a highly personalized digital assistant. Its core business problem is to bridge the gap between human intent and computer interaction by enabling an AI to 'see' (screen intelligence), 'hear' (voice processing), 'learn' (user profile, memory), and 'act' (tool integration) within a user's digital environment. It aims to automate complex workflows, provide proactive assistance, and adapt to individual user preferences, making computer usage more intuitive and efficient. Think of it as giving your AI assistant eyes, ears, and hands to truly understand and help you navigate your computer.

Data Flow

Data in OpenHuman flows like a continuous feedback loop. It starts with sensory input: 'Screen Intelligence' streams visual data (like pixels, text, and UI elements from `screen_intelligence::vision.rs`) and the 'Voice Server' streams audio (from `voice::server.rs`). These raw inputs are processed, converted into structured observations, and fed into the AI agent's 'Memory Context' (`agent::tree_loader.rs`). The core AI agent (`agent::harness::session::builder.rs`), guided by LLMs, uses this memory, along with the 'User Profile' (`learning::user_profile.rs`) and 'Vault Data' (`vault::store.rs`), to reason about the user's intent and current state. It then decides whether to use an external 'Tool' (`tools::traits.rs`, `composio::tools.rs`) to perform an action (e.g., send an email, search the web) or to generate a direct response. Any changes to tasks or user data are stored in persistent layers like `agent::task_board.rs`, `people::store.rs`, or `vault::store.rs`. Communication between these modules and the UI happens in real-time via WebSockets (`socket::ws_loop.rs`), ensuring a dynamic and responsive experience. Throughout this process, data is often summarized by `context::summarizer.rs` to keep the agent's context manageable and cost-effective for LLMs, and embeddings are stored in `embeddings::store.rs` for efficient semantic retrieval.

Key Modules & Components

  • AI Agent's Brain: This is the central hub where the AI agent processes information, figures out what users want, plans its actions, and manages ongoing conversations. It's like the conductor of an orchestra, making sure all other parts of the AI work together smoothly to understand and respond to you.
    Key files: src/openhuman/mod.rs, src/openhuman/agent/harness/session/builder.rs, src/openhuman/agent/tree_loader.rs
  • AI Agent's Eyes and Ears: These modules allow the AI to 'see' what's on your computer screen and 'hear' your voice commands. It captures images and audio, then processes them so the AI can understand what's happening visually and what you're saying, giving it a sense of its digital environment.
    Key files: src/openhuman/screen_intelligence/vision.rs, src/openhuman/screen_intelligence/server.rs, src/openhuman/voice/server.rs
  • AI Agent's Hands and Tools: This part equips the AI with the ability to perform actions, much like you use different tools for different tasks. It defines how the AI can interact with other applications and services (like sending emails or browsing the web) and helps the AI manage its to-do list for you, keeping track of tasks it needs to complete.
    Key files: src/openhuman/tools/traits.rs, src/openhuman/composio/tools.rs, src/openhuman/agent/task_board.rs
  • AI Agent's Memory & Learning: This module allows the AI to learn about your preferences, habits, and important information over time. It helps the AI remember details about you and continuously adapt its behavior to be more helpful and personalized, making its assistance feel more natural and intuitive.
    Key files: src/openhuman/learning/user_profile.rs, src/openhuman/embeddings/store.rs
  • Private Data Keeper: This module is like a secure personal vault for your important data. It safely stores information about you (like your contacts) and also keeps track of specific files or documents you want the AI to manage, ensuring your personal and private data is organized and protected.
    Key files: src/openhuman/people/store.rs, src/openhuman/vault/store.rs
  • System's Communication Center: This is the application's central switchboard for sending and receiving messages in real-time, especially for talking to the user interface or other backend services. It makes sure messages get to the right place and handles connections even if there are temporary network issues, ensuring smooth interaction.
    Key files: src/openhuman/socket/ws_loop.rs, src/core/dispatch.rs, src/api/rest.rs
  • Foundational System Services: These are the essential background services that keep the entire OpenHuman application running smoothly. They handle things like making sure you're the right user (authentication), managing basic system settings, defining common data formats, and tracking operational costs, providing the core stability for the whole system.
    Key files: src/main.rs, src/core/auth.rs, src/core/types.rs

Related Architecture Deep Dives

Explore the full interactive analysis of openhuman on Revibe Codes — architecture diagrams, module flow, execution paths, and code-level insights.