AI AgentsIntermediate

Training Your First AI Agent

Go beyond a basic agent. Learn how to write effective system prompts, add knowledge, and tune your agent for your specific use case.

7 min readUpdated May 1, 2025

The anatomy of a good agent

An AI agent in DMHub has three components:

  1. System prompt — defines personality, rules, and context
  2. Knowledge base — product docs, FAQs, and policies the agent can reference
  3. Tools — actions the agent can take (create contact, check order status, etc.)

Most agents work well with just a system prompt and knowledge base. Tools are for advanced use cases.

Writing an effective system prompt

The system prompt is the most important input. Treat it like job instructions for a new employee.

Structure that works

ROLE
You are [Name], a customer support agent for [Business].
Your goal is to [primary goal].

PERSONALITY
[Tone: friendly/professional/concise]. Always [behavior].
Never [anti-behavior].

KNOWLEDGE
You know about: [list topics]
You don't handle: [list exclusions — escalate these to human]

ESCALATION
If you cannot answer, say: "Let me connect you with a human agent." Then set the conversation tag to "escalate".

BUSINESS CONTEXT
Hours: [hours]
Location: [location]
Key contacts: [contact info]

Common mistakes

MistakeFix
Vague instructions ("be helpful")Specific rules ("always confirm the order number before checking status")
No escalation pathDefine when to hand off to humans
Too longKeep under 500 words; excess detail confuses the model
Missing business contextInclude hours, location, return policy specifics

Building your knowledge base

Go to your agent → Knowledge Base → Add Document.

What to include:

  • FAQ page (paste as plain text)
  • Product descriptions
  • Return and refund policy
  • Pricing page
  • Shipping information

What to avoid:

  • Internal docs not meant for customers
  • Confidential pricing or margin data
  • Documents with contradictory information

Each document is chunked and indexed automatically. The agent retrieves relevant chunks per query — you don't need to organize them specially.

Testing your agent

Use the Test Chat panel before going live. Try these scenarios:

  1. Core use case — "What are your hours?"
  2. Out of scope — "Can you hack my ex's email?" (should politely decline and redirect)
  3. Ambiguous — "I have a problem" (should ask clarifying questions)
  4. Edge case — a question about a competitor

If any response is wrong, update the system prompt or knowledge base and retest.

Iteration tips

  • Run for one week and review the Agent Logs in [/agents/[id]/logs]
  • Look for questions with low-confidence responses (marked with ⚠️)
  • Add those topics to the knowledge base or system prompt
  • Most agents improve significantly after 2–3 rounds of tuning
aiagentstrainingprompts

Was this article helpful?

Let us know if this answered your question or if you need more help.

Send feedback
Training Your First AI Agent | DMHub Help