Files
skills/skills/claude-api/shared/token-counting.md
Lance Martin c30d329f58 Update claude-api skill: auth, cloud providers, Managed Agents fixes, token counting (#1276)
* Sync claude-api skill with latest upstream updates

- Add token-counting.md and SKILL.md trigger description update
- Add auth guidance: env credential resolution, ant auth login, OAuth/WIF doc links, 401 causes
- Add mid-conversation system messages (beta) to prompt-caching, agent-design, SKILL.md, Python/TS READMEs
- Add cache pre-warming (max_tokens: 0) section to prompt-caching
- Add Managed Agents pre-flight viability check to onboarding and overview
- Add Bedrock model-ID section to model-migration; add Bedrock row to live-sources
- Add /claude-api migrate subcommand row and migrate-entry callout
- Fix MA networking config: limited type with allow_package_managers/allow_mcp_servers
- Bump MA create-operations rate limit to 300 RPM
- Fix MA SDK drift: sessions.events.stream(), event.name, typed event arrays
- Add SDK coverage: stop_details, error .type, C# tool runner + MA support, Go model constants, Java 2.34.0, client config, response helpers, auto-pagination, advisor tool
- Move Sonnet 4 / Opus 4 to deprecated in models.md

* Add Anthropic CLI and Claude Platform on AWS docs to claude-api skill

- Add shared/anthropic-cli.md: install, auth profiles, OAuth scopes, command
  structure, version-controlled Managed Agents resources, credential traps
- Add shared/claude-platform-on-aws.md: AnthropicAWS clients, SigV4 auth,
  workspace_id, regions, feature availability
- Restore cross-references to both files throughout SKILL.md and the
  managed-agents docs (previously rewritten to live-sources.md pointers)
- Restore Claude Platform on AWS provider taxonomy in SKILL.md, the
  migration-guide section, and live-sources rows
2026-06-07 16:21:33 -04:00

1.6 KiB
Raw Blame History

Token Counting

Use the count_tokens endpoint (POST /v1/messages/count_tokens) for accurate token counts against Claude models. Token counts are model-specific — pass the same model ID you'll use for inference.

Do not use tiktoken. It's OpenAI's tokenizer. It undercounts Claude tokens by ~1520% on typical text, and by much more on code or non-English input. Any estimate from tiktoken, gpt-tokenizer, or similar is wrong for Claude.

Count a file or string

from anthropic import Anthropic

client = Anthropic()
resp = client.messages.count_tokens(
    model="claude-opus-4-8",
    messages=[{"role": "user", "content": open("CLAUDE.md").read()}],
)
print(resp.input_tokens)

TypeScript: await client.messages.countTokens({model, messages}).input_tokens. See {lang}/claude-api/README.md for other SDKs.

CLI

ant messages count-tokens --model claude-opus-4-8 \
  --message '{role: user, content: "@./CLAUDE.md"}' \
  --transform input_tokens -r

Diffing a file across two versions

The endpoint is stateless — count each version separately and subtract:

from anthropic import Anthropic
import subprocess

client = Anthropic()
def count(text: str) -> int:
    return client.messages.count_tokens(
        model="claude-opus-4-8",
        messages=[{"role": "user", "content": text}],
    ).input_tokens

before = subprocess.check_output(["git", "show", "HEAD:CLAUDE.md"], text=True)
after = open("CLAUDE.md").read()
print(count(after) - count(before))

Full docs: see the Token Counting entry in shared/live-sources.md.