Why wouldn't an AI agent harness be client-server / centralized?
Some reasons might apply...
Wiring an AI agent harness looks like today’s equivalent of “Hello, World” - everyone’s making one. And why wouldn’t they'? The cost of actually developing an idea in software is some $$, electricity, and time.
Looks like most of AI agents are firstly made for local users, then some kind of multiuser setup gets bolted on top. That’s how OpenClaw started, and people just followed the lead. But… do they have to?
While working with OpenClaw and trying to set it up for a company-wide use, it seems too difficult to separate user idiosyncrasies and company-wide knowledge. What if person A and B talk about essentially the same thing in a different way? What if they have similar needs / workflows, but not quite 100% exact? How do we even distinguish which person added which information to the agent’s memory pool, or the list of skills, and if needed, surgically alter or remove it?
I wanted to know what the problems are in this space, and I’m a big believer in learning by doing, so - I’ve started my own AI agent harness, and named it Harlequin.
First of all - you probably don’t want to run it. It’s a research project in very early development.
Some ideas I’m currently working on within it:
☑ Client-server architecture, where the server is the source of truth, it calls the LLM, manages user and shared data, keeps user isolation, handles authentication and authorization, etc.
☑A clear, well maintained separation of shared data and user-specific data
☑A conflict resolution mechanism that flags and resolved inconsistent data (e.g. shared data says the company name is Foo, user attempts to override with company name being Bar) using a hybrid of FTS, vector embeddings, and LLM as the final judgment.
☑Frugal use of LLM calls, I’m developing this for use with small local LLMs like
qwen3.6-35b-a3b.☑”Code-in-skills” - or skills templated with JS code. This is a detail which I think should help very much with some other plans, and which deserves a blog post on its own. For a preview, see the test “greeter” skill here.
The biggest of the known unknowns (well, up to a point) that I think will influence how usable this project actually becomes, is how it handles 3rd party tool integrations. Local agents have the option of executing local CLI commands, that read credentials and other auth data from local dot files, and if Harlequin is to support that, it will need to both store those secrets on the server side, and spawn a sandbox with that data populated, if it were to use CLI commands. MCP and REST API tools are easier, but share most of the security surface.
We’ll see what the future holds. I hope to make a series of blog posts on this project as I progress.