Test an MCP server before you connect an agent: a probe for the handshake, the tool list and its context cost

Why test first

A tool-using client hides most failures. A server that crashes on startup, writes a banner to stdout, or takes too long to answer the handshake usually shows up as "no tools available", with nothing to say which of those happened. Talking to the server directly turns a vague symptom into a specific line of output. If you are still building one, the mechanics are laid out in building an MCP server from scratch, and what the protocol does is in what MCP actually does.

Source: the Model Context Protocol specification.

The probe

Give it the command that starts your server. It performs the handshake, lists the tools, and prints one line per finding. It uses a timeout, so a server that never answers is reported instead of hanging the check:

import json, select, subprocess, sys, time

# usage: mcp_probe.py <command> [args...]   (a stdio MCP server)
proc = subprocess.Popen(sys.argv[1:], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, bufsize=1)

def send(msg):
    proc.stdin.write(json.dumps(msg) + '\n'); proc.stdin.flush()

def read(want_id, timeout=10):
    end = time.time() + timeout
    while time.time() < end:
        if not select.select([proc.stdout], [], [], max(0, end - time.time()))[0]:
            return None          # nothing arrived before the deadline
        line = proc.stdout.readline()
        if not line:
            return None
        line = line.strip()
        if not line:
            continue
        try:
            msg = json.loads(line)
        except ValueError:
            print('FAIL  stdout line is not JSON (breaks the protocol): %r' % line[:80]); continue
        if msg.get('id') == want_id:
            return msg
    return None

t0 = time.time()
send({'jsonrpc': '2.0', 'id': 1, 'method': 'initialize', 'params': {'protocolVersion': '2025-06-18', 'capabilities': {}, 'clientInfo': {'name': 'probe', 'version': '0'}}})
init = read(1)
if not init or 'result' not in init:
    print('FAIL  no valid initialize response within 10 s'); proc.terminate(); sys.exit(1)
print('ok    initialize in %d ms; server %s %s; protocol %s' % ((time.time() - t0) * 1000, init['result']['serverInfo']['name'], init['result']['serverInfo'].get('version', '?'), init['result']['protocolVersion']))
print('      capabilities: %s' % ', '.join(init['result'].get('capabilities', {}).keys()))
send({'jsonrpc': '2.0', 'method': 'notifications/initialized'})
send({'jsonrpc': '2.0', 'id': 2, 'method': 'tools/list'})
tools = read(2)
if not tools or 'result' not in tools:
    print('FAIL  tools/list gave no result'); proc.terminate(); sys.exit(1)
lst = tools['result']['tools']
payload = json.dumps(tools['result'])
print('ok    %d tools, tools/list payload %d characters (about %d tokens at 4 chars/token, approximate)' % (len(lst), len(payload), len(payload) // 4))
problems = 0
for t in lst:
    size = len(json.dumps(t))
    issues = []
    if not t.get('description'): issues.append('no description')
    elif len(t['description']) < 40: issues.append('description under 40 chars')
    schema = t.get('inputSchema', {})
    if schema.get('type') != 'object': issues.append('inputSchema.type is not object')
    for name, p in schema.get('properties', {}).items():
        if not p.get('description'): issues.append('param %s has no description' % name)
    problems += bool(issues)
    print('%s %-28s %5d chars%s' % ('WARN ' if issues else 'ok   ', t['name'], size, ('  - ' + '; '.join(issues)) if issues else ''))
proc.terminate()
print('\n%d of %d tools have issues' % (problems, len(lst)))

Tested on real and broken servers

The hang case failed to be caught at first: my first version used a blocking read, so the timeout never fired, and the check itself hung. It now waits on the stream with a deadline. That is the sort of bug a probe is for, and it is why I tested it against a server that fails, not only one that works.

What the tool list costs

Clients send tool definitions to the model along with your prompt, so they count as input tokens whether or not a tool is called (Anthropic's tool use documentation describes tool definitions as part of the input). For the real server, the tools/list payload was 2,989 characters:

At about four characters per token that is roughly 750 tokens. I use that rule only to give a sense of scale: it is an approximation for English text, not the tokenizer any particular model uses. The useful reading is relative: one tool is 40% of the total, and a server with fifty tools would be a different decision. This is the same trade-off as other context costs in how coding agents choose what to read and prompt caching, where a stable block that is sent every turn can be cached.

What to fix

• Stdout noise: send logs to stderr, and check dependencies that print on import.
• Vague descriptions: say what the tool does, what it returns and when to use it instead of its neighbour.
• Undocumented parameters: add a description, with units and formats (a date range, a URL form).
• Heavy tools: trim schema text that repeats what the description already says, and split rarely used tools into a separate server.

What this does not cover