SuperHarness: AI Agent Development Framework Benchmarking Claude Code

Introduction

In the AI Agent development field, Claude Code CLI and LangChain represent two different paradigms:

• Claude Code CLI: Terminal product, user-facing, ready to use
• LangChain: Development framework, developer-facing, flexible but steep learning curve

SuperHarness attempts to merge both: provide ready-to-use terminal product while offering concise Python SDK for developers to customize.

Project Positioning

Dual Product Strategy

Benchmarking Analysis

Six-Layer Architecture

┌─────────────────────────────────────────────┐
│ Layer 5: Application Layer (CLI/TUI)        │  ← User interaction
├─────────────────────────────────────────────┤
│ Layer 4: Agent Runtime                      │  ← Agent runtime
├─────────────────────────────────────────────┤
│ Layer 3: Tool System                        │  ← Tool system
├─────────────────────────────────────────────┤
│ Layer 2: Session & Memory                   │  ← Session & memory
├─────────────────────────────────────────────┤
│ Layer 1: Core Services                      │  ← Core services
├─────────────────────────────────────────────┤
│ Layer 0: Security Foundation                │  ← Security foundation
└─────────────────────────────────────────────┘

Layer Responsibilities

Python SDK

Concise API

from superharness import Agent
 
# Create Agent
agent = Agent(
    api_key="your-api-key",
    provider="anthropic",  # or "openai", "gemini", "custom"
    model="claude-sonnet-4-6"
)
 
# Simple conversation
response = agent.run("Hello, how are you?")
print(response)
 
# Streaming output
for chunk in agent.run_stream("Tell me a story"):
    print(chunk, end="", flush=True)

Tool Registration

@agent.tool
def calculate(expression: str) -> float:
    """Evaluate a mathematical expression."""
    return eval(expression)
 
# Use tool
response = agent.run("What is 123 * 456?")

CLI / TUI

Terminal Interaction

# Enter TUI directly
superharness
 
# Show help
superharness --help
 
# Show version
superharness --version

TUI Features

• Syntax highlighting
• Streaming output display
• Keyboard shortcuts
• Session management

Tech Stack

Development Progress

Phase 0: SDK/TUI Basic Features ✅ Complete

Total: 192 tests all passed

Phase 1: Production Features ⏳ In Progress

Design Philosophy

Core Principles

1. No MVP/Demo - Develop complete product directly
2. Benchmark Claude Code/LangChain - Match or exceed
3. Precise Tasks - Clear completion criteria and acceptance conditions
4. Real Verification - Verify with actual user workflows

Transparency Three Principles

• No Hidden Behavior - All operations visible
• State Visualization - Real-time progress display
• Explainable Decisions - Explain tool selection and execution

Project Structure

superharness/
├── python/                  # Python SDK
│   └── superharness_sdk/
│       ├── agent/          # Agent runtime
│       ├── llm/            # LLM client
│       ├── config/         # Configuration system
│       ├── tools/          # Tool system
│       └── memory/         # Memory system
├── rust/                    # Rust core
│   ├── layer0/             # Security foundation
│   ├── layer1/             # Core services
│   ├── layer2/             # Session management
│   └── layer3/             # Tool system
├── cli/                     # CLI/TUI
│   └── src/
│       ├── cli/            # Command line parsing
│       ├── tui/            # TUI interface
│       └── agent/          # Agent client
├── docs/                    # Documentation
└── tests/                   # Tests

Current Status

Phase 0 complete, 192 tests all passed. Phase 1 in progress, aiming to complete production features benchmarking Claude Code's full capabilities.

Related Links

• SuperHarness Project Page
• GitHub Repository(opens in a new tab)

Last updated: 2026-05-13