Writing a skill that gets invoked at the right time

I have a skill for publishing HTML pages as shareable links. In late July I changed its description. Only the description: the publishing steps in the body stayed exactly as they were.

Before the change, it read:

Publish a generated HTML report to the online archive and return a fixed, shareable link. Use when the user asks to "generate an HTML report / output a summary page / give me an online link / publish the report / update that report."

The commit message recorded the problem: dashboards, prototypes, explainer pages, and demos didn't trigger the skill, because they weren't called "reports." I still got a local file path.

However detailed the publishing steps were, they didn't help. The agent never thought to use the skill.

The agent only sees the description

Here's how skills work. Normally the context holds only each skill's name and description. The agent compares them against the task in front of it, and only loads the body when one matches. At the moment it decides, it can't see how good the body is.

So the description isn't a summary for humans. It's a routing rule that decides when the skill comes to mind.

The problem was that the whole description revolved around the noun "report." It did include "give me an online link," but the opening said this was for publishing "HTML reports." "Report" was the word in my head. When the thing I'd made was a dashboard or a prototype, the agent concluded it wasn't this skill's job.

The new description:

Publish any HTML page as a shareable online link (summaries, audit reports, dashboards, prototypes, docs, demos all count). Triggers: the user wants "an online link / let me see it / publish it / deploy this page / don't give me a local file / give me a URL / update that page," or you just made an HTML page that the user needs to view in a browser or pass on.

Two things changed. First, the actual criteria were spelled out. The commit message put it as: "there are only two criteria: the output is HTML, and it needs to be viewed or shared." Second, the trigger phrases became things people really say. "Don't give me a local file" is much easier to match than "publish the report."

Say when not to use it

That change also added a line at the end:

Not for the product's own export feature, and not for deploying the app itself (that's the deploy skill).

The same repo has a deploy skill for putting the app on servers. "Deploy this page" and "deploy it" differ by a word or two. Without a boundary, two skills compete for the same sentence. Saying "not mine, that's deploy's" tells the agent which way to go.

When you list a case where the skill doesn't apply, point to where it does belong. "Not for X" tells the agent not to use this one. It still doesn't say which one to use.

The trigger can be the agent's own behavior

I have another skill for recording follow-ups, and its description works differently:

Mandatory, not optional: whenever you're about to write "known issue / do later / out of scope this time / optional backlog" in a reply, or any unhandled follow-up, risk, or business request comes up in the conversation, write it to this repo's ledger first, then continue.

It isn't waiting for something the user says. It's waiting for something the agent is about to say. The moment the agent is about to write "do this later" in a reply is exactly when the skill should step in. Users never say that, only the agent can notice it, and putting it in the description gives the agent one more cue to catch.

Still, on the day that skill shipped, I also put the same rule into the rules file. The commit message says "auto-triggering from the skill description isn't reliable enough; things get missed." However precise the description, invoking the skill is still the agent's judgment call, and judgment sometimes misses. If something must happen every time, don't rely on a skill alone; put it in the rules file (more on that in another essay).

One more thing: descriptions stay in context. In mid-September I turned off 34 unused skills in that repo. Every extra skill is another description the agent reads at the start of every session, and another candidate it might pick by mistake.

A version to copy

A good description covers four things: what it does, when to use it, what the user will say, and when not to use it plus what to use instead.

---
name: publish-page
description: >
  Publish any HTML page as a shareable online link (summaries,
  dashboards, prototypes, docs all count). Use when the user says
  "give me a link," "let me see it," "don't give me a local file,"
  or when you just made an HTML page the user needs to view in a
  browser or pass on. Not for the product's own export feature, and
  not for deploying the app itself (use the deploy skill).
---

Then check it against a few recent conversations where the skill should have kicked in. Does the description contain anything that matches what was actually said?

The publishing skill has been changed several times since, adding encryption and reworking the password flow. All of those changes were to the body. The description hasn't been touched since late July.