Agent

Tool Calling

• Clearly define the functions (OpenAI uses TypeScript for definition).
• Only provide a few functions to the agent, functions should be of only a few clearly defined parameters.

Reasoning

LLMs have no internal monologue (by default).

• Prompt LLMs to reason on a problem with few-shot prompting, and preferably with fine-tuning.
• LLMs are zero-shot reasoners, simply tell the model to “let’s think step by step”.
• ReAct, tell the model to reply based on Thought, Act (tool calling), and Finish loop.
• Plan and solve prompting: “let’s first come up with a plan… then answer the question.”
• Reflexion: tell the model to review the work after finishing, and start over if not appropriate.
• Branch-and-merge: spawn multiple LLM solvers to attempt solving the problem, either with high temperature, or different aspects instructed, then combine the results.

Conversational Agent

• Structure of the context
  • Preamble: rules, tool definitions, few-shot examples (if any)
  • Prior conversation: previous conversations and artifacts.
  • Current exchange: the most recent request, tool calls and responses, attached artifacts.
  • Agent response
• Including artifacts: artifacts generated by tool calling should be preserved, as they’re helpful for the model to understand the tools; for long artifacts, adopt elastic snippets, or use RAG to summarize it first, and include a new tool, e.g. for details, call details('section 5').

sequenceDiagram
    participant User
    participant App
    participant Model
    participant Tool

    User->>App: Message
    Note over App: User message appended
    App->>Model: Prompt

    loop Zero or more tool invocations
        Model->>App: Function call
        Note over App: Parse tool invocation
        App->>Tool: Tool evaluation request
        Tool->>App: Tool evaluation response
        Note over App: Tool call added to prompt
        App->>Model: Prompt
    end

    Model->>App: Assistant message
    Note over App: Assistant message appended to thread
    App->>User: Assistant message

• Conversational agent are designed to interact with endusers — however, user proxy agent can be created to keep it running. AutoGen uses this approach.

Frameworks

• Representing a workflow as a DAG. This could be achieved with a traditional workflow tool such as Apache Airflow and Luigi. Or with a specialize framework such as LangGraph.
• Autogen, defines Assistant and UserProxy, where user proxy acts like a user to instruct and supervise other assistants doing the work to achieve some goal.
• CrewAI, defines multiple agents (crews) and orchestrate their collaboration.