Sake.Test (Sake v0.4.3)

Copy Markdown View Source

Tests for a Sake server: a client connected to the server in memory, the MCP counterpart of Phoenix.ConnTest.

import Sake.Test

test "greets" do
  client = connect(MyApp.MCP, scope: user, scopes: ["greet"])
  assert {:ok, %{text: "Hello, Rui!"}} = call_tool(client, "greet", %{"name" => "Rui"})
  assert {:ok, tools} = list_tools(client)

  assert {:error, {:insufficient_scope, %{scope: ["issues:write"]}}} =
           call_tool(client, "close_issue", %{"id" => 1})
end

The client is a Sake.Client whose transport is the server itself: every request crosses as JSON into the dispatchers Sake.Plug and Sake.Stdio use, with a real Sake.Context, so argument validation, scopes, results and errors are the production ones, and every client feature works as against a real server (input requests and their retries, tasks, MCP Apps, progress and log notifications). The calls take and return what Sake.Client's do, without the session: a call returns {:ok, value} or {:error, reason}, a call with nothing to return :ok, and session/1 hands the session over when a test needs Sake.Client itself.

In test/test_helper.exs, start/1 starts what Sake needs when the application does not (see its documentation).

Options

connect/2 takes the options of Sake.Client.connect/2 (era:, legacy_version:, client_info:, capabilities:, on_input:, sampling:, roots:, on_notification:, log_level:, tasks:, task_timeout:, ui:), and:

  • :scope — the host's principal, what components see as ctx.scope (a value, or a function of no arguments called for every request). Default nil.
  • :scopes — the OAuth scopes of a simulated access token. With it, every request carries the token, and components are gated by their scope: as on an endpoint with authorization; without it there is no token and no gating, as on an endpoint without.
  • :subject — the token's subject, which owns the tasks it starts. Defaults to one derived from :scope, so two clients with different principals are different people; nil makes it a machine client's token (the client credentials grant), owned by :client_id.
  • :client_id — the token's client. Default "sake-test".
  • :acr, :auth_time — how and when the token's person signed in (RFC 9470): the authentication context class and a DateTime, for components with authentication:. Default nil (unknown), which meets no requirement: such a call answers {:error, {:insufficient_user_authentication, challenge}}.
  • :reraise — whether an exception raised by a component fails the test with that exception (the default), or becomes the error result a client would get (false).
  • :secret — signs cursors and request states. Default the Sake configuration's, else a fixed one shared by every test client.
  • :base_url — the endpoint's URL, what components see as ctx.base_url (see Sake.Context), for a tool that builds absolute URLs to its host. Default nil, as over stdio: the in-memory client has no request to take it from. Through a plug (below), Sake.Plug sets it from the request, as in production.

The era is modern unless era: :legacy asks for the handshake.

mcp_connect/2 is connect/2 under another name, for a test that also imports Phoenix.ChannelTest (a ChannelCase), whose connect macro has the same name and arity.

Answering the server

on_input:, sampling: and roots: answer the server's requests, and turn on their capabilities, exactly as for Sake.Client. Two shorter forms suit tests: on_input: may be a function of one request's params that returns {:accept, content}, :decline, :cancel or an answer map, and sampling: a function of the request's params that returns {:ok, result}, {:error, error} or the text of the assistant's reply.

Every request the server asks is recorded, for assert_asked/2:

client = connect(MyApp.MCP, on_input: fn _params -> {:accept, %{"name" => "Rui"}} end)
assert {:ok, %{text: "Hello, Rui!"}} = call_tool(client, "greet", %{})
assert_asked "elicitation/create", %{"message" => "Who should be greeted?"}

Notifications

The client asks for progress on every call, and every notification the server sends during one is recorded for assert_notification/2; pass log_level: to have the server send log messages too.

listen/2 records what a server broadcasts (Sake.Server.notify_list_changed/2, Sake.Server.resource_updated/2), for assert_list_changed/1 and assert_resource_updated/1. It sees only the broadcasts of the test process and of the processes it started, so tests can run with async: true; shared: true sees every broadcast of the server, from any process, for a test that cannot run concurrently with others.

Through a plug

plug: sends the requests through a plug instead, over HTTP as Sake.Client sends them, with Req running the plug in the test's process: the host's endpoint or router, with its pipelines and its authorization (plug: MyAppWeb.Endpoint, path: "/acme/mcp"), or Sake.Plug itself (plug: {Sake.Plug, server: MyApp.MCP}). path: is the endpoint's path (default "/mcp") and headers: go with every request. The stack computes the principal from the request, so scope: does not apply; scopes: issues a real access token for subject: (see access_token/1) and sends it as a bearer token, or, to a server that takes DPoP-bound tokens only (dpop: [required: true]), bound to a key the client signs a DPoP proof with for each request. That token is Sake.Auth's: an endpoint whose tokens come from another authorization server (Sake.Auth.External) does not take it, so send one that server accepts in headers: (in the test environment, a verify: callback that knows your test tokens), or test the server through the in-memory client, whose scopes: token stands for any issuer's. A legacy server's questions need the in-memory client, and a task is polled at the interval the server suggests, as over the network.

Tasks

A tool that runs as a task (see Sake.Tasks) runs as one here, in the background, when start/1 or the application started tasks. call_tool/3 follows it to its result, and call_tool_async/3 hands it back to drive with await_task/3, get_task/2, update_task/3 and cancel_task/2. Neither sleeps: the client sees the task's status change the moment it happens.

Summary

Functions

Issues an access token in the Sake.Auth store and returns it, for a request through a host's endpoint with authorization: connect/2 with plug: and scopes: sends one, and a Phoenix.ConnTest test can put it in an authorization header itself. Sake.Auth must be configured ({Sake, auth: ...}).

Asserts that the server asked this test's client something: "elicitation/create", "sampling/createMessage" or "roots/list", with params (a pattern).

Asserts that a server listen/2 follows broadcast a change of its list of :tools, :resources or :prompts (Sake.Server.notify_list_changed/2).

Asserts that the server sent a notification during a call of this test's client: "notifications/progress" (Sake.Progress.report/4) or "notifications/message" (Sake.Progress.log/3, with log_level:). params is a pattern.

Asserts that a server listen/2 follows broadcast a change of the resource at uri (Sake.Server.resource_updated/2). uri is a pattern.

Waits until a task (a task map or its ID) is no longer working, and returns it as tasks/get does then: input_required, completed, failed or cancelled. It wakes when the task's status changes, not on a timer. {:error, :timeout} after timeout ms.

Ends the client.

Connects a client to server, a Sake.Server module, in memory. See the module documentation for the options. Raises when the server refuses the connection.

Records the broadcasts of server (a Sake.Server module, or a client connected to one) for assert_list_changed/1 and assert_resource_updated/1, from now until the calling process ends.

connect/2 under another name, for a test that imports Phoenix.ChannelTest too, whose connect macro clashes with it

Refutes that the server asked such a thing (see assert_asked/2) so far.

Refutes that such a change (see assert_list_changed/1) was broadcast so far.

Refutes that the server sent such a notification (see assert_notification/2) so far.

Refutes that such a change (see assert_resource_updated/1) was broadcast so far.

The client's Sake.Client.Session as it is now, for Sake.Client itself.

Starts what Sake needs when the application does not start {Sake, ...} in the test environment: the stream registry (for listen/2, a legacy server's requests, and tasks) and tasks. What already runs is left alone. Call it once, in test/test_helper.exs

Functions

access_token(opts)

@spec access_token(keyword()) :: String.t()

Issues an access token in the Sake.Auth store and returns it, for a request through a host's endpoint with authorization: connect/2 with plug: and scopes: sends one, and a Phoenix.ConnTest test can put it in an authorization header itself. Sake.Auth must be configured ({Sake, auth: ...}).

Options: subject: (required: the ID your subject module's fetch/1 takes, or nil for a machine client's token), client_id: (default "sake-test"; a machine client's must be registered, see Sake.Auth.Clients.create/2), scopes: (default none), acr: and auth_time: (how and when the person signed in, RFC 9470; default unknown), resource: (the endpoint's URL, which the token is for; default "http://www.example.com/mcp", the endpoint of a test request at /mcp).

assert_asked(method, params \\ quote do _ end)

(macro)

Asserts that the server asked this test's client something: "elicitation/create", "sampling/createMessage" or "roots/list", with params (a pattern).

assert_asked "elicitation/create", %{"mode" => "form", "message" => "Who?"}

assert_list_changed(kind)

(macro)

Asserts that a server listen/2 follows broadcast a change of its list of :tools, :resources or :prompts (Sake.Server.notify_list_changed/2).

assert_notification(method, params \\ quote do _ end)

(macro)

Asserts that the server sent a notification during a call of this test's client: "notifications/progress" (Sake.Progress.report/4) or "notifications/message" (Sake.Progress.log/3, with log_level:). params is a pattern.

assert_notification "notifications/progress", %{"progress" => 3, "total" => 10}

assert_resource_updated(uri)

(macro)

Asserts that a server listen/2 follows broadcast a change of the resource at uri (Sake.Server.resource_updated/2). uri is a pattern.

await_task(client, task, timeout \\ 5000)

@spec await_task(Sake.Test.Client.t(), String.t() | map(), timeout()) ::
  {:ok, map()} | {:error, term()}

Waits until a task (a task map or its ID) is no longer working, and returns it as tasks/get does then: input_required, completed, failed or cancelled. It wakes when the task's status changes, not on a timer. {:error, :timeout} after timeout ms.

call_tool(client, name, arguments \\ %{})

@spec call_tool(Sake.Test.Client.t(), String.t(), map()) ::
  {:ok, Sake.Client.ToolResult.t()} | {:error, term()}

See Sake.Client.call_tool/3.

call_tool_async(client, name, arguments \\ %{})

@spec call_tool_async(Sake.Test.Client.t(), String.t(), map()) ::
  {:ok, Sake.Client.ToolResult.t() | {:task, map()}} | {:error, term()}

See Sake.Client.call_tool_async/3.

cancel_task(client, task_id)

@spec cancel_task(Sake.Test.Client.t(), String.t()) :: :ok | {:error, term()}

See Sake.Client.cancel_task/2.

close(client)

@spec close(Sake.Test.Client.t()) :: :ok

Ends the client.

complete(client, ref, argument, value)

@spec complete(
  Sake.Test.Client.t(),
  {:prompt, String.t()} | {:resource, String.t()},
  String.t(),
  String.t()
) :: {:ok, map()} | {:error, term()}

See Sake.Client.complete/4.

connect(server, opts \\ [])

@spec connect(module(), keyword()) :: Sake.Test.Client.t()

Connects a client to server, a Sake.Server module, in memory. See the module documentation for the options. Raises when the server refuses the connection.

get_prompt(client, name, arguments \\ %{})

@spec get_prompt(Sake.Test.Client.t(), String.t(), map()) ::
  {:ok, map()} | {:error, term()}

See Sake.Client.get_prompt/3.

get_task(client, task_id)

@spec get_task(Sake.Test.Client.t(), String.t()) :: {:ok, map()} | {:error, term()}

See Sake.Client.get_task/2.

list_prompts(client)

@spec list_prompts(Sake.Test.Client.t()) :: {:ok, [map()]} | {:error, term()}

See Sake.Client.list_prompts/1.

list_resource_templates(client)

@spec list_resource_templates(Sake.Test.Client.t()) ::
  {:ok, [map()]} | {:error, term()}

See Sake.Client.list_resource_templates/1.

list_resources(client)

@spec list_resources(Sake.Test.Client.t()) :: {:ok, [map()]} | {:error, term()}

See Sake.Client.list_resources/1.

list_tools(client)

@spec list_tools(Sake.Test.Client.t()) ::
  {:ok, [Sake.Tool.Spec.t()]} | {:error, term()}

See Sake.Client.list_tools/1.

listen(server, opts \\ [])

@spec listen(module() | Sake.Test.Client.t(), keyword()) :: :ok

Records the broadcasts of server (a Sake.Server module, or a client connected to one) for assert_list_changed/1 and assert_resource_updated/1, from now until the calling process ends.

Only the broadcasts made by the calling process and by the processes it started (a tool the test called, a task) are recorded, so concurrent tests do not see each other's. shared: true records every broadcast of the server, from any process: for a broadcast from a process the test did not start, in a test with async: false.

mcp_connect(server, opts \\ [])

@spec mcp_connect(module(), keyword()) :: Sake.Test.Client.t()

connect/2 under another name, for a test that imports Phoenix.ChannelTest too, whose connect macro clashes with it:

import Phoenix.ChannelTest
import Sake.Test, except: [connect: 1, connect: 2]

client = mcp_connect(MyApp.MCP, scope: user)

ping(client)

@spec ping(Sake.Test.Client.t()) :: :ok | {:error, term()}

See Sake.Client.ping/1.

read_resource(client, uri)

@spec read_resource(Sake.Test.Client.t(), String.t()) ::
  {:ok, [Sake.Content.t()]} | {:error, term()}

See Sake.Client.read_resource/2.

refute_asked(method, params \\ quote do _ end)

(macro)

Refutes that the server asked such a thing (see assert_asked/2) so far.

refute_list_changed(kind)

(macro)

Refutes that such a change (see assert_list_changed/1) was broadcast so far.

refute_notification(method, params \\ quote do _ end)

(macro)

Refutes that the server sent such a notification (see assert_notification/2) so far.

refute_resource_updated(uri)

(macro)

Refutes that such a change (see assert_resource_updated/1) was broadcast so far.

session(client)

The client's Sake.Client.Session as it is now, for Sake.Client itself.

set_log_level(client, level)

@spec set_log_level(Sake.Test.Client.t(), String.t() | atom()) ::
  :ok | {:error, term()}

See Sake.Client.set_log_level/2.

start(opts \\ [])

@spec start(keyword()) :: :ok

Starts what Sake needs when the application does not start {Sake, ...} in the test environment: the stream registry (for listen/2, a legacy server's requests, and tasks) and tasks. What already runs is left alone. Call it once, in test/test_helper.exs:

Sake.Test.start()

tasks: takes the Sake.Tasks options, or false for none.

update_task(client, task_id, responses)

@spec update_task(Sake.Test.Client.t(), String.t(), map()) :: :ok | {:error, term()}

See Sake.Client.update_task/3.