Hanzo Dev
36dd3959fb
- Add @hanzo/aci - Agent Computer Interface for cross-platform automation - Add @hanzo/mcp - Model Context Protocol server with 20+ tools - MCP now uses @hanzo/ai for AI tools (think, critic, consensus, agent) - LanceDB integration for multimodal vector storage - Complete feature parity with Python MCP implementation - Remove old dev package structure
2.4 KiB
2.4 KiB
@hanzo/aci
Agent Computer Interface - Cross-platform automation for AI agents.
Features
- 🖱️ Mouse Control: Click, drag, scroll, and move
- ⌨️ Keyboard Control: Type, press keys, and use hotkeys
- 📸 Screen Capture: Take screenshots of full screen or regions
- 🔍 Image Recognition: Find images on screen
- 📝 OCR: Read text from screen using Tesseract.js
- 🤖 MCP Integration: Ready-to-use MCP tools for AI agents
- 🌐 Cross-Platform: Works on Windows, macOS, and Linux
Installation
npm install @hanzo/aci
Usage
Basic Usage
import { ACI } from '@hanzo/aci';
const aci = new ACI();
// Take a screenshot
const screenshot = await aci.screenshot();
// Click at coordinates
await aci.click(100, 200);
// Type text
await aci.type('Hello, world!');
// Find and click on an image
await aci.clickImage('button.png');
// Read text from screen (requires OCR)
const aci = new ACI({ ocr: true });
const text = await aci.readText();
MCP Tools
import { aciTools } from '@hanzo/aci/mcp-tools';
// Use with your MCP server
const tools = aciTools;
API Reference
Mouse Operations
moveTo(x, y, duration?)- Move mouse to positionclick(x?, y?, button?)- Click at positiondoubleClick(x?, y?)- Double clickdrag(fromX, fromY, toX, toY)- Drag from one position to anotherscroll(clicks, x?, y?)- Scroll mouse wheel
Keyboard Operations
type(text, delay?)- Type textpress(key)- Press a single keyhotkey(...keys)- Press keyboard shortcut
Screen Operations
screenshot(region?)- Take screenshotgetScreenSize()- Get screen dimensionslocateOnScreen(imagePath, confidence?)- Find image on screenlocateAllOnScreen(imagePath, confidence?)- Find all occurrences
OCR Operations
readText(region?)- Read text using OCRfindText(text)- Find text location on screen
High-Level Operations
clickImage(imagePath, timeout?)- Click on an imageclickText(text, timeout?)- Click on textwaitForImage(imagePath, timeout?)- Wait for image to appearwaitForText(text, timeout?)- Wait for text to appear
Requirements
- Node.js 18+
- Platform-specific requirements:
- Windows: No additional requirements
- macOS: Accessibility permissions required
- Linux: X11 and dependencies for robotjs
License
MIT © Hanzo AI