I Got Ghosted by My Own AI Agent
- Serah Rashidi (She/Her)
- 5 days ago
- 4 min read
Hey techies! I hope your agents are behaving and your systems aren’t silently ignoring half your instructions. Lately, I ran into something both funny and frustrating. My own agent, with all the tools, skills, and structure in place, decided to do its own thing. Turns out, getting ghosted is no longer just a social experience. It can happen with your own AI system too. In this blog, I'll break down why agents ignore instructions, what’s actually happening under the hood, and how to fix it.
The Plot
Something funny happened to me a while back. I realized that my own agent, my own MCP server, wasn’t actually abiding by everything in the instructions. And of course, I had to dig deeper into it, and apparently it just kept ignoring them. It kept ignoring and ignoring, even when I repeated myself multiple times. And then I realized this is actually very common.
Believe it or not, you can get ghosted by your own agent.
We used to get ghosted by friends, or Exs, and now, surprise, getting ghosted by your own agent is the new 2026 behavior. But before getting into what happened, let’s take a step back for a second.
What is an Agent?
An agent is basically a system powered by an LLM that takes a goal and decides how to act on it instead of just responding once. It can use tools, follow steps, and execute tasks.
What is an MCP Server?
A model context protocol (MCP) server is simply the layer that gives the agent access to these tools in a structured way, like a toolbox it can pull from. And skills are predefined behaviors or instructions the agent can reuse, like generating content or calling a function. So in theory, everything is structured. You define the system, give it tools, give it skills, and it should behave accordingly.
Under the Hood
Now back to what actually happened. We had an agent, connected to an MCP server, with clearly defined skills, instructions, and flows. Everything was there. And yet, it kept ignoring them. It would skip tools, bypass steps, generate outputs that had nothing to do with the instructions, and just behave on its own.
At first, I thought it was a bug, so naturally I did what most of us do, I added more instructions, more clarity, more rules. But instead of fixing it, it made things worse. And that’s when it clicked. The problem wasn’t that the agent was “ignoring” instructions, it was that the system itself wasn’t designed in a way that enforced them. When you overload an agent with instructions, it doesn’t strictly follow all of them, it kind of averages them. When you don’t define priorities, it guesses what matters most.
When your prompts are vague, it fills in the gaps. And when there are no strict constraints, it takes shortcuts. So it’s not disobedience, it’s just how these systems behave.
How to Actually Fix This
So how do you deal with it? Not by throwing more text at it, but by fixing the system itself, here are some tips:
1- Fixing the Architecture
Instead of relying on prompts alone, you need to control behavior at different layers, start by separating instruction layers clearly and do not mix everything in one place or the model will average them.
2- Enforce Mandatory Actions
If something must happen, do not “suggest” it in text. And ofcourse validate tool usage, check if required steps were executed and Block progression if conditions are not met; the system should fail, not continue incorrectly.
3- Define Clear Interfaces
Most agents fail because inputs and outputs are vague. You should define strict input parameters and the expected output format. Also, avoid generic instructions like “make it good”. The more structured the interface, the less room for drift.
4- Reduce Tool Ambiguity
If your MCP server exposes too many similar tools, the agent will choose randomly, avoid overlapping tool purposes and make tool descriptions precise. You should Limit available tools per task if needed.
5- Separate Planning from Execution
Do not let the model do everything in one step.
Instead:
• Step 1: decide what to do
• Step 2: select skill or tool
• Step 3: execute
• Step 4: validate output
6- Add Observability
You cannot fix what you cannot see, you should always track which tools were selected, which skills were ignored, what inputs were passed and where the flow broke. This is where most debugging actually happens.
The Takeaway
At the end of the day, the issue is rarely that your agent is “ignoring you.” It is that your system is too permissive. Agents do not follow instructions the way humans do, they operate within probabilities and constraints, constantly choosing what seems most likely rather than what you intended. If you leave gaps, they will fill them. If you allow shortcuts, they will take them. And if your system does not enforce structure, it will default to behavior that looks correct on the surface but breaks under inspection.
So the goal is not to write longer prompts or repeat instructions more aggressively. It is to design systems where expectations are explicit, behavior is constrained, and execution is verifiable. Because once you move from prompting to building agents, you are no longer just communicating with a model, you are defining how it is allowed to behave. And if your agent is ghosting you, it is not random and it is not personal, it is simply the system doing exactly what you designed it to do.
And with that, we reach the end of the blog. I hope you had a good read and learned a lot. Stay tuned as we'll cover more tech-related topics in future blogs.
In case of any questions or suggestions, feel free to reach out to me via LinkedIn. I'm always open to fruitful discussions.🍏🦜
Comments