Council is a custom skill, I had setup a while ago, that let’s one spawn multiple visible panes in cmux (now in herdr as well) with each pane having an agent chair that can communicate over a problem statement and come to a conclusion together.

Right now it works with three coding agents: claude, codex and agy. Every chair gets a model and effort value to use. Each chair acts as an independent auditor/critique over a plan or an advisor over a problem. All the members of the council have full autonomy to research and go through the project and use that information to ask clarifying questions to the user or give the final output.

Defaults: Claude: latest opus model Codex: latest gpt sol model agy: latest gemini pro model

We have an orchestrator that the human chats with and invokes the council with. The orchestrator has the ability to pass in model and effort based on the problem statement but the defaults work most of the time. These defaults are controlled by something I call Agent Profiles.

Tldr on agent profiles: Agent profiles matter for skills that require multi agent orchestration for example, coder reviewer. Three profile have been setup right now, namely: gpt-heavy, claude-heavy, balanced. These names tell the story themselves but essentially it’s a global setting with which these agents get these model preferences set. In gpt-heavy profile we have the coder as claude and the reviewer as codex (reasoning is that a coder does not need a good model as long as coding plan is straightforward and reviewer needs to be a solid contender. That’s what heavy essentially means)

The orchestrator then frames the problem presented by the user to these chairs and ask them to share any questions they need clarified before they can proceed. The questions, if any, are surfaced to the user. When answered they are presented to the respective chair that asked that question. Once all the questions are resolved, the chairs go and do their research and get to a conclusion saying no-go, go-with-changes or go. After each round these chairs generate an md file and share the reference to it to the orchestrator. Then for the next round each chair gets the report shared by other council chairs and based on the report it researches to validate or refute the claims of the report. This goes on for a couple of rounds, generally 3 to 4. Then a final report is created by the orchestrator based on all the final reports shared by these council members.

The council seems to work so far but i’m not sure how well. It has helped me find issues and potential bugs that would have been shipped otherwise. I have not looked at studies that talk about multi agent collaboration where different families of models are involved. Maybe I’ll pick that up next to make the council better.

One concern I have is whether some chair is strong-arming other to not let it do its best work. I’ve not observed anything related to that but I think I should spend some time looking at those.

Will keep you guys posted. Lmk if you are working on something similar or dissimilar. Would love to talk.

Ps: This is one of my first attempt at writing something. Not using AI to write. So well done to myself?