Compare commits
32
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bb0fb030c3 | ||
|
|
eed1d1d3f2 | ||
|
|
f201cb358b | ||
|
|
611a4571f3 | ||
|
|
d86610da8e | ||
|
|
bf6acaacc8 | ||
|
|
0938b3026e | ||
|
|
c6bc28442a | ||
|
|
7eaaffb63f | ||
|
|
c427e2432a | ||
|
|
2a78262754 | ||
|
|
2a4a054549 | ||
|
|
c179581d58 | ||
|
|
cec2f88375 | ||
|
|
1cedc1b927 | ||
|
|
3dfedd1f35 | ||
|
|
6c29a318e8 | ||
|
|
e8c6a5109e | ||
|
|
f8594bd3d8 | ||
|
|
7eace14042 | ||
|
|
35acd72e4e | ||
|
|
6b308d5a49 | ||
|
|
66ca3ab9be | ||
|
|
4fa36796b4 | ||
|
|
e283b6b62f | ||
|
|
f47459ba9b | ||
|
|
e410b5d4a5 | ||
|
|
4254cac826 | ||
|
|
2f61616a6f | ||
|
|
cc53da1dc7 | ||
|
|
a9a682dee0 | ||
|
|
8921b131ef |
@@ -25,9 +25,22 @@ Current state:
|
||||
`tools/list`, and authenticated read probe. They deliberately report an
|
||||
existing client connection as `not_observable`; a successful fresh probe
|
||||
does not claim that a previously opened Codex transport is connected.
|
||||
- Operator OAuth credentials remain in `.napseer/auth.json`. A repaired local
|
||||
gateway uses a separate `.napseer/gateway-auth.json` worker identity, so
|
||||
relay token renewal cannot rotate or overwrite the operator/MCP session.
|
||||
- `nap auth login` stores the shared account session in the user data folder
|
||||
(`~/.local/share/napseer/credentials/default.json` on Linux; platform user data directories
|
||||
on macOS/Windows). `NAPSEER_USER_DATA_DIR` overrides that location.
|
||||
`nap auth login --project` selects a repository override shared by its Git
|
||||
worktrees. Separate repositories retain separate overrides.
|
||||
- Private state lives entirely in user data. `credentials/` holds login grants;
|
||||
`repositories/<id>/` holds the shared signing key; `workspaces/<id>/` holds
|
||||
each worktree's selection, gateway identity, vault, index, locks, and logs.
|
||||
Git common-directory identity groups related worktrees; each worktree's Git
|
||||
directory identifies its runtime state, including after `git worktree move`.
|
||||
Nested project locators keep their own namespace. Ordinary directories use
|
||||
their canonical absolute path. Separate clones do not share project overrides.
|
||||
- `.napseer/project.json` is the commit-safe project locator. It contains only
|
||||
the schema, API origin, project UUID, and slug; it never contains tokens,
|
||||
account or worker identity, encryption state, keys, passphrases, or claim
|
||||
links. A fresh clone uses it to attach the intended project.
|
||||
- CLI terminal and schedule operations call the running gateway's
|
||||
loopback-only, CSRF-protected control API. PTYs therefore live in the daemon
|
||||
instead of disappearing when a CLI subprocess exits.
|
||||
@@ -35,6 +48,18 @@ Current state:
|
||||
Backend discovery publishes the release manifest and exact public source
|
||||
revision used for each bundle.
|
||||
|
||||
For an existing checkout, stop older MCP clients and local gateway/UI processes,
|
||||
update the runtime, and run `nap auth migrate` once from that checkout. This moves
|
||||
private files out of `.napseer`, preserving the public locator and ignore file.
|
||||
Migration checks running gateway PIDs, copies and verifies every file before
|
||||
removing originals, and refuses conflicting destinations. An interrupted copy
|
||||
can be retried; it never selects a different login to hide a conflict. Restart
|
||||
clients afterward. Migrate existing worktrees individually; conflicting older
|
||||
credentials or signing keys must be reconciled explicitly, never overwritten.
|
||||
|
||||
`NAPSEER_USER_DATA_DIR` and explicit credential files must point outside the
|
||||
checkout. There is no credential or private-state symlink inside the repository.
|
||||
|
||||
Configure Codex and other long-lived stdio clients to launch:
|
||||
|
||||
```sh
|
||||
@@ -46,24 +71,63 @@ directory, add a trusted project `.codex/config.toml` override with both
|
||||
`cwd` and `NAPSEER_PROJECT_ROOT`. MCP processes inherit their launch directory;
|
||||
they cannot infer a later client workspace switch over stdio.
|
||||
|
||||
If `NAPSEER_MCP_WORKER_PATH` points to a stable launcher that loads a separate,
|
||||
replaceable runtime, set `NAPSEER_MCP_WORKER_WATCH_PATHS` to the
|
||||
`os.pathsep`-separated runtime paths. The supervisor then reloads the launcher
|
||||
after any watched runtime is atomically replaced, preserving the same bounded,
|
||||
non-replaying request lifecycle used for direct workers.
|
||||
|
||||
MCP admits at most 32 unfinished requests by default, including cancelled work
|
||||
that is still cleaning up. `-32097` means a request was rejected before
|
||||
execution; retry after capacity is available. `-32098` means the result
|
||||
is uncertain: inspect current state before retrying a mutation. Cancellation
|
||||
prevents queued mutations from starting and attempts to release acquired locks.
|
||||
It cannot undo a mutation already accepted by the API.
|
||||
|
||||
The supervisor waits up to 60 seconds for a response, then requests cancellation.
|
||||
After 10 seconds without a cleanup acknowledgement, it terminates the worker and
|
||||
fails affected requests without replay. Worker input writes have a five-second
|
||||
deadline. Override these limits with `NAPSEER_MCP_RESPONSE_TIMEOUT_SECONDS`,
|
||||
`NAPSEER_MCP_CANCEL_GRACE_SECONDS`, `NAPSEER_MCP_WRITE_TIMEOUT_SECONDS`, and
|
||||
`NAPSEER_MCP_MAX_IN_FLIGHT_REQUESTS`. New source loads only after unfinished work
|
||||
has drained. `nap_contract` exposes the runtime's request lifecycle contract.
|
||||
These process recovery guarantees require the supervisor; the direct worker is
|
||||
for isolated protocol debugging. An already-running older supervisor needs a
|
||||
client reconnect after updating.
|
||||
|
||||
The public operator surface is intentionally small:
|
||||
|
||||
```text
|
||||
nap init | status | doctor | auth | project | mcp | gateway | update | version | help
|
||||
nap init | status | doctor | auth | project | files | export | mcp | gateway | update | version | help
|
||||
```
|
||||
|
||||
Authentication has one recovery verb: `nap auth repair`. Normal refresh is
|
||||
automatic. `nap gateway repair` creates the separate worker identity once;
|
||||
`--replace` is required to replace an existing identity.
|
||||
|
||||
Project-scoped MCP tools are zero-login by default. In a new workspace, the
|
||||
first such tool enrolls an anonymous worker and creates its first project
|
||||
automatically; `nap init` remains the explicit equivalent. Anonymous access
|
||||
and refresh-token or SSH-key recovery continue without operator
|
||||
authentication. Authentication through `nap project claim` is required only
|
||||
after the anonymous account reaches its service limit (currently one project
|
||||
or 100 stored nodes), or when its durable local enrollment identity is no
|
||||
longer recoverable.
|
||||
Agents start only after the user configures an account login or a delegated API
|
||||
key. With an account login, `nap init` creates a project owned by that account.
|
||||
A committed `.napseer/project.json` always selects the existing project; it
|
||||
never causes a replacement to be created. Concurrent refreshes share a bounded
|
||||
credential lock, while project overrides remain independent. Network failures
|
||||
preserve credentials and offer recovery through `nap auth repair`.
|
||||
|
||||
`nap auth keys --help` describes scoped API key issuance;
|
||||
`nap auth api-key --env VARIABLE_NAME` installs a delegated key privately.
|
||||
Keys can expire or be revoked and cannot approve agent decisions. Use separate
|
||||
keys when agents need separate identities or permissions.
|
||||
|
||||
Upgrading from 0.2.x to 0.3.0 requires the current bootstrap once because old
|
||||
installers have a fixed list of runtime files. `nap update` on those versions
|
||||
rejects the new bundle before changing the active installation. Run:
|
||||
|
||||
```sh
|
||||
curl -fsSL https://api.napseer.com/install | python3 -
|
||||
```
|
||||
|
||||
The bootstrap verifies every asset and switches the runtime atomically without
|
||||
changing credentials or project locators. Restart existing MCP client sessions
|
||||
after this upgrade; subsequent releases use `nap update` normally.
|
||||
|
||||
In Bash, run commands as `nap update`, `nap auth login`, and so on. A leading
|
||||
`!` is shell history expansion: `!nap update` replays the most recent command
|
||||
|
||||
@@ -6,6 +6,7 @@ Usage:
|
||||
nap init
|
||||
nap status
|
||||
nap doctor
|
||||
nap reindex
|
||||
nap auth login
|
||||
nap project create
|
||||
nap project attach
|
||||
@@ -42,6 +43,7 @@ SCRIPT_NAMES = (
|
||||
"napseer_mcp_server.py",
|
||||
"napseer_mcp_supervisor.py",
|
||||
"napseer_spake2.py",
|
||||
"napseer_credentials.py",
|
||||
"terminal_init.py",
|
||||
"terminal_protocol.py",
|
||||
"terminal_pty_manager.py",
|
||||
@@ -56,6 +58,7 @@ LOCAL_HOST = "127.0.0.1"
|
||||
DEFAULT_GATEWAY_PORT = os.environ.get("NAPSEER_GATEWAY_PORT")
|
||||
RUNTIME_SCRIPT_NAMES = (
|
||||
"napseer_spake2.py",
|
||||
"napseer_credentials.py",
|
||||
"terminal_init.py",
|
||||
"terminal_protocol.py",
|
||||
"terminal_pty_manager.py",
|
||||
@@ -63,20 +66,23 @@ RUNTIME_SCRIPT_NAMES = (
|
||||
"napseer_mcp_supervisor.py",
|
||||
)
|
||||
MAX_SERVICE_LOG_BYTES = int(os.environ.get("NAPSEER_MAX_SERVICE_LOG_BYTES", str(5 * 1024 * 1024)))
|
||||
CLI_RELEASE_VERSION = "0.2.2"
|
||||
CLI_DISTRIBUTION_CONTRACT_VERSION = "2026-08-01"
|
||||
CLI_MINIMUM_CONTRACT_VERSION = "2026-05-19"
|
||||
CLI_RELEASE_VERSION = "0.5.0"
|
||||
CLI_DISTRIBUTION_CONTRACT_VERSION = "2026-09-29"
|
||||
# Older installers have a fixed asset list and would omit the credentials module.
|
||||
# Reject them before activation; the published bootstrap performs the upgrade.
|
||||
CLI_MINIMUM_CONTRACT_VERSION = "2026-09-10"
|
||||
CLI_BUNDLE_SCHEMA_VERSION = "napseer.cli.bundle.v1"
|
||||
INSTALL_PATHS = {
|
||||
"nap_install.py": "nap_install.py",
|
||||
"napseer_mcp_server.py": "napseer_mcp_server.py",
|
||||
"napseer_mcp_supervisor.py": "napseer_mcp_supervisor.py",
|
||||
"napseer_spake2.py": "napseer_spake2.py",
|
||||
"napseer_credentials.py": "napseer_credentials.py",
|
||||
"terminal_init.py": "terminal/__init__.py",
|
||||
"terminal_protocol.py": "terminal/protocol.py",
|
||||
"terminal_pty_manager.py": "terminal/pty_manager.py",
|
||||
}
|
||||
CLI_GENERATED_SOURCE_REPO = os.environ.get("NAPSEER_CLI_SOURCE_REPO", "https://github.com/napseer/nap-cli")
|
||||
CLI_GENERATED_SOURCE_REPO = os.environ.get("NAPSEER_CLI_SOURCE_REPO", "https://gitea.powc-r.com/napseer/nap-cli")
|
||||
CLI_GENERATED_SOURCE_REVISION = os.environ.get("NAPSEER_CLI_SOURCE_REVISION", "unresolved")
|
||||
CLI_GENERATED_SOURCE_REVISION_STATUS = os.environ.get("NAPSEER_CLI_SOURCE_REVISION_STATUS", "unresolved")
|
||||
HELP_TOKENS = {"help", "-h", "--help"}
|
||||
@@ -84,8 +90,11 @@ CANONICAL_COMMANDS = (
|
||||
("init", "Initialize Napseer in the current folder."),
|
||||
("status", "Show concise account, project, and runtime state."),
|
||||
("doctor", "Diagnose local setup without changing it."),
|
||||
("reindex", "Rebuild the current project's local memory index."),
|
||||
("auth", "Authenticate an account or repair credentials."),
|
||||
("project", "Create, attach, claim, or inspect a project."),
|
||||
("files", "List, upload, download, or archive project files."),
|
||||
("export", "Export project documentation and files to a new ZIP archive."),
|
||||
("mcp", "Install, update, inspect, or serve the local MCP runtime."),
|
||||
("gateway", "Set up and operate the local gateway."),
|
||||
("update", "Update the installed CLI and runtime bundle."),
|
||||
@@ -96,8 +105,11 @@ COMMAND_METADATA = {
|
||||
"init": {"mutates": True, "visible": True},
|
||||
"status": {"mutates": False, "visible": True},
|
||||
"doctor": {"mutates": False, "visible": True},
|
||||
"reindex": {"mutates": True, "visible": True},
|
||||
"auth": {"mutates": True, "visible": True},
|
||||
"project": {"mutates": True, "visible": True},
|
||||
"files": {"mutates": True, "visible": True},
|
||||
"export": {"mutates": True, "visible": True},
|
||||
"mcp": {"mutates": True, "visible": True},
|
||||
"gateway": {"mutates": True, "visible": True},
|
||||
"update": {"mutates": True, "visible": True},
|
||||
@@ -500,6 +512,7 @@ def activate_release(release_dir):
|
||||
"napseer_mcp_server.py",
|
||||
"napseer_mcp_supervisor.py",
|
||||
"napseer_spake2.py",
|
||||
"napseer_credentials.py",
|
||||
):
|
||||
path = INSTALL_DIR / filename
|
||||
preserve_legacy_path(path)
|
||||
@@ -617,28 +630,34 @@ def run_wrapper(command_name, args):
|
||||
|
||||
|
||||
def preferred_state_dir():
|
||||
return pathlib.Path.cwd() / ".napseer"
|
||||
from napseer_credentials import WorkspacePaths
|
||||
return WorkspacePaths().state
|
||||
|
||||
|
||||
def cwd_state_dir():
|
||||
preferred = preferred_state_dir()
|
||||
preferred.mkdir(exist_ok=True)
|
||||
preferred.mkdir(parents=True, exist_ok=True)
|
||||
chmod_best_effort(preferred, 0o700)
|
||||
return preferred
|
||||
|
||||
|
||||
def state_dir_status():
|
||||
preferred = preferred_state_dir()
|
||||
from napseer_credentials import CredentialStore, WorkspacePaths
|
||||
try: auth_path, credential_source = CredentialStore(WorkspacePaths().legacy).selected()
|
||||
except RuntimeError: auth_path, credential_source = CredentialStore(WorkspacePaths().legacy).user, "invalid_selection"
|
||||
return {
|
||||
"credential_source": credential_source,
|
||||
"migration_required": bool(WorkspacePaths().migration_sources()),
|
||||
"active_state_dir": str(preferred),
|
||||
"active_auth_path": str(preferred / "auth.json"),
|
||||
"active_auth_path": str(auth_path),
|
||||
"state_dir": str(preferred),
|
||||
"auth_path": str(preferred / "auth.json"),
|
||||
"auth_path": str(auth_path),
|
||||
"state_dir_exists": preferred.exists(),
|
||||
"state_dir_message": (
|
||||
".napseer directory is active."
|
||||
"Private Napseer state is stored in user data."
|
||||
if preferred.exists()
|
||||
else ".napseer directory is not initialized."
|
||||
else "Private Napseer state is not initialized."
|
||||
),
|
||||
}
|
||||
|
||||
@@ -845,6 +864,8 @@ def validate_cli_options(args, *, flags=(), value_options=(), command_label):
|
||||
|
||||
|
||||
def start_service(kind, args):
|
||||
from napseer_credentials import WorkspacePaths
|
||||
WorkspacePaths().require_migrated()
|
||||
wrapper = runtime_asset_path("napseer_mcp_server.py")
|
||||
require_runtime_assets()
|
||||
existing = read_service_state(kind)
|
||||
@@ -1021,8 +1042,12 @@ def print_gateway_help(topic=None):
|
||||
nap gateway schedule delete [ID | --schedule ID]""")
|
||||
return 0
|
||||
if topic == "vault":
|
||||
print("Usage: nap gateway vault [status|process|rotate-secret] [--kind memory] [--all] [--project-id ID] [--vault-passphrase TEXT]")
|
||||
print(" status Show local gateway state and pending setup requests.")
|
||||
print("Usage: nap gateway vault [status|setup|forget|process|rotate-secret] [--kind memory] [--all] [--project-id ID] [--vault-passphrase TEXT]")
|
||||
print(" status Show local gateway state, memory secret provisioning, and pending setup requests.")
|
||||
print(" setup Store the project memory secret in local encrypted storage for this worktree.")
|
||||
print(" Prompts once for the project vault passphrase; the passphrase is never stored.")
|
||||
print(" The CLI and MCP workers then load the secret automatically.")
|
||||
print(" forget Remove the locally stored memory secret for this worktree.")
|
||||
print(" process Upload opaque client-wrapped key bundle records for backend-owned HashiCorp storage.")
|
||||
print(" rotate-secret Rotate the memory project secret from the gateway worker path.")
|
||||
return 0
|
||||
@@ -1033,6 +1058,7 @@ def print_gateway_help(topic=None):
|
||||
print(" nap gateway repair Create dedicated gateway worker credentials.")
|
||||
print(" nap gateway terminal help Show terminal lifecycle commands.")
|
||||
print(" nap gateway schedule help Show schedule lifecycle commands.")
|
||||
print(" nap gateway vault setup Store the E2E memory secret locally (prompts once).")
|
||||
print(" nap gateway vault help Show vault commands.")
|
||||
return 0
|
||||
|
||||
@@ -1066,12 +1092,25 @@ def print_command_help(command):
|
||||
"Usage: nap doctor\n"
|
||||
"Run read-only checks and print exact repair commands."
|
||||
),
|
||||
"reindex": (
|
||||
"Usage: nap reindex [--limit N]\n"
|
||||
"Rebuild the current project's local search and graph index."
|
||||
),
|
||||
"auth": (
|
||||
"Usage: nap auth [status|login|repair]\n"
|
||||
" login Authenticate an account; does not select a project.\n"
|
||||
" login Authenticate an account and attach the project named by .napseer/project.json.\n"
|
||||
" status Show credential state without credential values.\n"
|
||||
" repair Recover failed automatic credential renewal."
|
||||
" repair Recover failed automatic credential renewal.\n"
|
||||
" logout [--project] Revoke the selected login.\n"
|
||||
" login --project Store a separate project login in user data.\n"
|
||||
" migrate Move legacy private state into user data after stopping older clients/gateways.\n"
|
||||
" use-user Explicitly select the general login for this folder.\n"
|
||||
" api-key --env NAME Configure project access from a named environment variable.\n"
|
||||
" keys list|create|revoke Manage named, expiring delegated access."
|
||||
|
||||
),
|
||||
"files": "Usage: nap files list|upload|download|archive [--path FILE] [--id UUID] [--node-id UUID] [--cursor CURSOR] [--limit N]",
|
||||
"export": "Usage: nap export --path NEW_ARCHIVE.zip\nExports active project nodes and files; refuses changes during export or an existing destination.",
|
||||
"project": (
|
||||
"Usage: nap project [init|create|attach|claim|status|encryption]\n"
|
||||
" init Initialize a fresh folder (same workflow as `nap init`).\n"
|
||||
@@ -1274,6 +1313,10 @@ def mcp_runtime_probe(timeout_seconds=6):
|
||||
"verified": False,
|
||||
},
|
||||
"remote_project_access": "not_tested",
|
||||
"memory_secret": (local_payload or {}).get("memory_secret"),
|
||||
"deprecations": [
|
||||
item.get("code") for item in (local_payload or {}).get("deprecations") or [] if isinstance(item, dict)
|
||||
],
|
||||
}
|
||||
except Exception as exc:
|
||||
return {
|
||||
@@ -1335,26 +1378,31 @@ def doctor_status():
|
||||
auth_path = pathlib.Path(state["active_auth_path"])
|
||||
auth = {}
|
||||
auth_error = None
|
||||
if auth_path.exists():
|
||||
try:
|
||||
auth = json.loads(auth_path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
auth_error = "auth file is unreadable or invalid JSON"
|
||||
try:
|
||||
from napseer_credentials import CredentialStore, WorkspacePaths, read_json
|
||||
store=CredentialStore(WorkspacePaths().legacy)
|
||||
auth=store.read()
|
||||
locator=read_json(WorkspacePaths().locator)
|
||||
if not auth.get("project_id"): auth["project_id"]=locator.get("project_id")
|
||||
except RuntimeError:
|
||||
auth_error="credential state is unreadable; run nap auth login or inspect the selected override"
|
||||
missing = runtime_assets_missing()
|
||||
issues = []
|
||||
setup = []
|
||||
if auth_error:
|
||||
if state.get("migration_required"):
|
||||
issues.append({"code": "state_migration_required", "message": "Stop older MCP clients and local gateways, then migrate private state into user data.", "next": "nap auth migrate"})
|
||||
elif auth_error:
|
||||
issues.append({"code": "auth_file_invalid", "message": auth_error, "next": "nap auth repair"})
|
||||
elif not auth_path.exists():
|
||||
setup.append({
|
||||
"code": "project_not_initialized",
|
||||
"message": "current folder is not initialized; anonymous setup is available without login",
|
||||
"next": "nap init or invoke a project-scoped MCP tool",
|
||||
"message": "current folder is not initialized; configure a user login with nap auth login",
|
||||
"next": "nap auth login",
|
||||
})
|
||||
elif not auth.get("project_id"):
|
||||
setup.append({
|
||||
"code": "project_not_configured",
|
||||
"message": "no project is configured for this folder; anonymous setup is available without login",
|
||||
"message": "no project is configured for this folder; nap init uses the configured user login",
|
||||
"next": "nap init",
|
||||
})
|
||||
if missing:
|
||||
@@ -1378,6 +1426,18 @@ def doctor_status():
|
||||
else "inspect the failed_stage field and local project binding"
|
||||
),
|
||||
})
|
||||
if probe.get("memory_secret") == "rotated":
|
||||
setup.append({
|
||||
"code": "memory_secret_rotated",
|
||||
"message": "the project memory secret was rotated after it was stored locally",
|
||||
"next": "nap gateway vault setup",
|
||||
})
|
||||
if "vault_passphrase_env_deprecated" in (probe.get("deprecations") or []):
|
||||
setup.append({
|
||||
"code": "vault_passphrase_env_deprecated",
|
||||
"message": "NAPSEER_VAULT_PASSPHRASE / NAPSEER_MASTER_PASSPHRASE are deprecated for agents; store the memory secret locally instead",
|
||||
"next": "nap gateway vault setup",
|
||||
})
|
||||
status = "repair_required" if issues else ("setup_available" if setup else "ok")
|
||||
return {
|
||||
"status": status,
|
||||
@@ -1431,6 +1491,10 @@ def version_status():
|
||||
|
||||
def main(argv):
|
||||
invoked_as_nap = pathlib.Path(argv[0]).name in {"nap", "nap.cmd"}
|
||||
# The published /install endpoint is executed as `python3 -`. Keep that
|
||||
# bootstrap separate from the removed operator-facing `nap install` alias.
|
||||
if len(argv) == 1 and not invoked_as_nap:
|
||||
return handle_mcp(["install"])
|
||||
command = argv[1] if len(argv) > 1 else ("help" if invoked_as_nap else "install")
|
||||
args = argv[2:]
|
||||
if command in HELP_TOKENS:
|
||||
@@ -1445,6 +1509,8 @@ def main(argv):
|
||||
if command == "doctor":
|
||||
print_json(doctor_status())
|
||||
return 0
|
||||
if command == "reindex":
|
||||
return run_wrapper("reindex", args)
|
||||
if command == "mcp":
|
||||
return handle_mcp(args)
|
||||
if command == "version":
|
||||
@@ -1457,6 +1523,8 @@ def main(argv):
|
||||
return 0
|
||||
if command == "gateway":
|
||||
return handle_gateway(args)
|
||||
if command in {"files", "export"}:
|
||||
return run_wrapper(command, args)
|
||||
if command == "status":
|
||||
return run_wrapper("configure", args)
|
||||
if command == "project":
|
||||
@@ -1469,7 +1537,7 @@ def main(argv):
|
||||
if args and args[0] in {"login", "login-local", "operator-login"}:
|
||||
validate_cli_options(
|
||||
args[1:],
|
||||
flags={"--no-browser"},
|
||||
flags={"--no-browser", "--project"},
|
||||
value_options={
|
||||
"--frontend-url",
|
||||
"--api-base-url",
|
||||
|
||||
@@ -0,0 +1,433 @@
|
||||
"""User-owned credentials, separate from repository identity and gateway state.
|
||||
|
||||
No network, enrollment, or credential output belongs in this module.
|
||||
"""
|
||||
from contextlib import contextmanager
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
|
||||
|
||||
class CredentialError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
CREDENTIAL_KEYS = {
|
||||
"base_url", "account_id", "token", "access_token", "refresh_token",
|
||||
"token_expires_at", "refresh_expires_at", "account_mode", "oauth_client_id",
|
||||
"oauth_scope", "logged_out", "credential_kind",
|
||||
}
|
||||
SELECTION_KEYS = {
|
||||
"project_id", "project_slug", "project_name", "project_encryption_state",
|
||||
"project_signing_key_fingerprint", "credential_profile",
|
||||
}
|
||||
_registry_lock = threading.Lock()
|
||||
_thread_locks = {}
|
||||
_held = threading.local()
|
||||
|
||||
|
||||
def user_data_dir():
|
||||
if os.environ.get("NAPSEER_USER_DATA_DIR"):
|
||||
path = Path(os.environ["NAPSEER_USER_DATA_DIR"]).expanduser()
|
||||
if not path.is_absolute():
|
||||
raise CredentialError("NAPSEER_USER_DATA_DIR must be an absolute path outside the checkout.")
|
||||
return path.resolve()
|
||||
if sys.platform == "win32":
|
||||
return Path(os.environ.get("LOCALAPPDATA", Path.home() / "AppData" / "Local")) / "napseer"
|
||||
if sys.platform == "darwin":
|
||||
return Path.home() / "Library" / "Application Support" / "napseer"
|
||||
xdg = Path(os.environ.get("XDG_DATA_HOME", Path.home() / ".local" / "share"))
|
||||
return (xdg if xdg.is_absolute() else Path.home() / ".local" / "share") / "napseer"
|
||||
|
||||
|
||||
class WorkspacePaths:
|
||||
"""Pure path resolution: Git identity survives worktree moves, not copies.
|
||||
|
||||
The public locator stays in Git. Repository credentials/signing keys are
|
||||
shared by related worktrees; daemon/vault/cache state belongs to one worktree.
|
||||
No credentials are read and no directory is created during path discovery.
|
||||
"""
|
||||
def __init__(self, root=None):
|
||||
start = Path(root or os.environ.get("NAPSEER_PROJECT_ROOT") or Path.cwd()).expanduser().resolve()
|
||||
self.root = start
|
||||
git_dir = common = None
|
||||
environment = {k:v for k,v in os.environ.items() if k not in
|
||||
{"GIT_DIR", "GIT_WORK_TREE", "GIT_COMMON_DIR", "GIT_INDEX_FILE"}}
|
||||
try:
|
||||
result = subprocess.run(["git", "-C", str(start), "rev-parse", "--path-format=absolute",
|
||||
"--show-toplevel", "--git-dir", "--git-common-dir"],
|
||||
capture_output=True, text=True, timeout=3, env=environment)
|
||||
if result.returncode == 0:
|
||||
top, git_dir, common = result.stdout.strip().splitlines()
|
||||
self.root = Path(top).resolve()
|
||||
except (OSError, ValueError, subprocess.TimeoutExpired):
|
||||
pass
|
||||
# A nested project's explicit locator is a separate namespace within Git.
|
||||
for candidate in (start, *start.parents):
|
||||
if (candidate / ".napseer/project.json").is_file():
|
||||
self.root = candidate
|
||||
break
|
||||
if git_dir and candidate == self.root:
|
||||
break
|
||||
suffix = str(self.root.relative_to(Path(top).resolve())) if git_dir else ""
|
||||
repo_identity = str(Path(common).resolve()) + ":" + suffix if common else str(self.root)
|
||||
worktree_identity = str(Path(git_dir).resolve()) + ":" + suffix if git_dir else str(self.root)
|
||||
self.repository_id = hashlib.sha256(repo_identity.encode()).hexdigest()
|
||||
self.workspace_id = hashlib.sha256(worktree_identity.encode()).hexdigest()
|
||||
self.data = user_data_dir().expanduser().resolve()
|
||||
if self.data.is_relative_to(self.root) and self.root != Path.home().resolve():
|
||||
raise CredentialError("Napseer user data must be outside the project checkout.")
|
||||
self.legacy = self.root / ".napseer"
|
||||
self.locator = self.legacy / "project.json"
|
||||
self.repository = self.data / "repositories" / self.repository_id
|
||||
self.state = self.data / "workspaces" / self.workspace_id
|
||||
self.signing_key = self.repository / "id_ed25519"
|
||||
self.project_credentials = self.data / "credentials/projects" / (self.repository_id + ".json")
|
||||
old_id = hashlib.sha256(str(self.root).encode()).hexdigest()
|
||||
self.old_credentials = self.data / "credentials/projects" / (old_id + ".json")
|
||||
|
||||
def migration_sources(self):
|
||||
if self.legacy.is_symlink():
|
||||
raise CredentialError("Legacy Napseer state must not be a symbolic link.")
|
||||
sources = []
|
||||
if self.legacy.exists():
|
||||
sources = [p for p in self.legacy.iterdir() if p.name not in {"project.json", ".gitignore"}]
|
||||
if self.old_credentials != self.project_credentials and self.old_credentials.exists():
|
||||
sources.append(self.old_credentials)
|
||||
return sources
|
||||
|
||||
def require_migrated(self):
|
||||
if self.migration_sources():
|
||||
raise CredentialError("Repository-local Napseer state needs migration. Stop older MCP clients and local gateways, then run `nap auth migrate`.")
|
||||
|
||||
def migrate(self, *, gateway=False):
|
||||
"""Copy/verify/publish before removing originals; never overwrite conflicts.
|
||||
|
||||
Older processes must be stopped before this explicit operation. Existing
|
||||
daemon PIDs are checked. A partial copy can be retried safely; all source
|
||||
bytes remain until every destination has been verified.
|
||||
"""
|
||||
with credential_lock(self.data / "migration"):
|
||||
sources = self.migration_sources()
|
||||
if not sources:
|
||||
return {"status":"already_migrated", "state_dir":str(self.state)}
|
||||
for name in ("gateway.json", "ui.json"):
|
||||
state = read_json(self.legacy / name)
|
||||
pid = state.get("pid")
|
||||
if isinstance(pid, int) and pid > 0:
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except ProcessLookupError:
|
||||
continue
|
||||
except OSError:
|
||||
pass
|
||||
raise CredentialError("Stop the running local gateway/UI before `nap auth migrate`.")
|
||||
entries = []
|
||||
for source in sources:
|
||||
if source.is_symlink() or not source.is_file():
|
||||
raise CredentialError("Migration requires regular files; original state was preserved.")
|
||||
destination = self.state / source.name
|
||||
if source.name in {"id_ed25519", "id_ed25519.pub"}:
|
||||
destination = self.repository / source.name
|
||||
if source.name == "auth.json" or source == self.old_credentials:
|
||||
destination = (self.state / "auth.json") if gateway and source.name == "auth.json" else self.project_credentials
|
||||
original = file_fingerprint(source)
|
||||
copied = None
|
||||
if source.name in {"auth.json", "gateway-auth.json", "state.json"} or source == self.old_credentials:
|
||||
data = read_json(source)
|
||||
for field, old, new in (
|
||||
("ssh_key_path", self.legacy / "id_ed25519", self.signing_key),
|
||||
("vault_path", self.legacy / "vault.json", self.state / "vault.json"),
|
||||
):
|
||||
value = data.get(field)
|
||||
if isinstance(value, str):
|
||||
candidate = Path(value).expanduser()
|
||||
candidate = candidate if candidate.is_absolute() else self.root / candidate
|
||||
if candidate.resolve() == old.resolve(): data[field] = str(new)
|
||||
copied = (json.dumps(data, indent=2) + "\n").encode()
|
||||
if destination.is_symlink():
|
||||
raise CredentialError("Migration destination must not be a symbolic link.")
|
||||
expected = (len(copied), hashlib.sha256(copied).hexdigest()) if copied is not None else original
|
||||
if destination.exists() and file_fingerprint(destination) != expected:
|
||||
raise CredentialError("Migration conflicts with existing private state; nothing was overwritten. Reconcile the two states before retrying.")
|
||||
if any(dst == destination and digest != expected for _,dst,_,_,digest in entries):
|
||||
raise CredentialError("Legacy credentials conflict; original state was preserved.")
|
||||
entries.append((source, destination, original, copied, expected))
|
||||
# Publish credentials last, after their keys and other state exist.
|
||||
entries.sort(key=lambda e: e[1] in {self.project_credentials, self.state / "auth.json"})
|
||||
for source, destination, original, copied, expected in entries:
|
||||
private_directory(destination.parent, self.data)
|
||||
if not destination.exists():
|
||||
fd, temporary = tempfile.mkstemp(prefix=".migrate-", dir=destination.parent)
|
||||
try:
|
||||
with os.fdopen(fd, "wb") as stream:
|
||||
os.fchmod(stream.fileno(), 0o600)
|
||||
if copied is not None:
|
||||
stream.write(copied)
|
||||
else:
|
||||
with source.open("rb") as original_stream:
|
||||
while chunk := original_stream.read(1024 * 1024):
|
||||
stream.write(chunk)
|
||||
stream.flush(); os.fsync(stream.fileno())
|
||||
# Publish without replacing another login or migration's file.
|
||||
try:
|
||||
os.link(temporary, destination)
|
||||
except FileExistsError:
|
||||
if file_fingerprint(destination) != expected:
|
||||
raise CredentialError("Migration destination changed; original state was preserved.") from None
|
||||
finally:
|
||||
if os.path.exists(temporary): os.unlink(temporary)
|
||||
if file_fingerprint(destination) != expected or file_fingerprint(source) != original:
|
||||
raise CredentialError("State changed during migration; original files were preserved. Stop older clients and retry.")
|
||||
for source, destination, original, copied, expected in entries:
|
||||
if file_fingerprint(source) != original or file_fingerprint(destination) != expected:
|
||||
raise CredentialError("Migration verification failed; remaining originals were preserved.")
|
||||
# Make published directory entries durable before removing originals.
|
||||
# Windows does not support opening directories for fsync this way.
|
||||
if os.name != "nt":
|
||||
parents = set()
|
||||
for _, destination, _, _, _ in entries:
|
||||
for parent in (destination.parent, *destination.parent.parents):
|
||||
parents.add(parent)
|
||||
if parent == self.data: break
|
||||
for parent in sorted(parents, key=lambda p: len(p.parts), reverse=True):
|
||||
fd = os.open(parent, os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
|
||||
try: os.fsync(fd)
|
||||
finally: os.close(fd)
|
||||
for source, _, _, _, _ in entries:
|
||||
source.unlink()
|
||||
return {"status":"migrated", "state_dir":str(self.state), "files":len(entries)}
|
||||
|
||||
|
||||
def private_directory(path, boundary):
|
||||
path, boundary = Path(path), Path(boundary)
|
||||
if not path.is_relative_to(boundary):
|
||||
raise CredentialError("Private state directory is outside its user data boundary.")
|
||||
for parent in (path, *path.parents):
|
||||
if parent.is_symlink():
|
||||
raise CredentialError("Private Napseer state must not use symlink directories.")
|
||||
if parent == boundary: break
|
||||
path.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
for parent in (path, *path.parents):
|
||||
parent.chmod(0o700)
|
||||
if parent == boundary: break
|
||||
|
||||
|
||||
def file_fingerprint(path):
|
||||
digest = hashlib.sha256()
|
||||
fd = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0))
|
||||
with os.fdopen(fd, "rb") as stream:
|
||||
if not stat.S_ISREG(os.fstat(stream.fileno()).st_mode):
|
||||
raise CredentialError("Migration requires regular files.")
|
||||
size = 0
|
||||
while chunk := stream.read(1024 * 1024):
|
||||
digest.update(chunk); size += len(chunk)
|
||||
return size, digest.hexdigest()
|
||||
|
||||
|
||||
def read_json(path):
|
||||
try:
|
||||
fd = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0))
|
||||
except FileNotFoundError:
|
||||
return {}
|
||||
except OSError:
|
||||
raise CredentialError("Credential state cannot be opened safely.") from None
|
||||
try:
|
||||
if not stat.S_ISREG(os.fstat(fd).st_mode):
|
||||
raise CredentialError("Credential state must be a regular file.")
|
||||
with os.fdopen(fd, "r", encoding="utf-8") as stream:
|
||||
fd = None
|
||||
raw = stream.read(65537)
|
||||
if len(raw) > 65536:
|
||||
raise CredentialError("Credential state exceeds the size limit.")
|
||||
result = json.loads(raw)
|
||||
if not isinstance(result, dict):
|
||||
raise ValueError()
|
||||
return result
|
||||
except (ValueError, UnicodeError, OSError):
|
||||
raise CredentialError("Credential state is unreadable; run nap auth login to repair it.") from None
|
||||
finally:
|
||||
if fd is not None:
|
||||
os.close(fd)
|
||||
|
||||
|
||||
def atomic_json(path, data):
|
||||
path = Path(path)
|
||||
path.parent.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
if path.is_symlink():
|
||||
raise CredentialError("Credential state must not be a symbolic link.")
|
||||
fd, temporary = tempfile.mkstemp(prefix=f".{path.name}.", dir=path.parent)
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as stream:
|
||||
os.fchmod(stream.fileno(), 0o600)
|
||||
json.dump(data, stream, indent=2)
|
||||
stream.write("\n")
|
||||
stream.flush()
|
||||
os.fsync(stream.fileno())
|
||||
os.replace(temporary, path)
|
||||
finally:
|
||||
if os.path.exists(temporary):
|
||||
os.unlink(temporary)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def credential_lock(path, timeout=30):
|
||||
"""Bounded process lock; nested calls in the same thread reuse the lock."""
|
||||
path = Path(path).absolute()
|
||||
with _registry_lock:
|
||||
thread_lock = _thread_locks.setdefault(str(path), threading.RLock())
|
||||
if not thread_lock.acquire(timeout=timeout):
|
||||
raise CredentialError("Credential update is busy; retry after the other login or refresh finishes.")
|
||||
try:
|
||||
held = getattr(_held, "paths", {})
|
||||
if str(path) in held:
|
||||
yield
|
||||
return
|
||||
path.parent.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
try:
|
||||
fd = os.open(str(path) + ".lock", os.O_RDWR | os.O_CREAT | getattr(os, "O_NOFOLLOW", 0), 0o600)
|
||||
except OSError:
|
||||
raise CredentialError("Credential lock cannot be opened safely.") from None
|
||||
try:
|
||||
if not stat.S_ISREG(os.fstat(fd).st_mode):
|
||||
raise CredentialError("Credential lock must be a regular file.")
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
try:
|
||||
if os.name == "nt":
|
||||
import msvcrt
|
||||
os.lseek(fd, 0, os.SEEK_SET)
|
||||
msvcrt.locking(fd, msvcrt.LK_NBLCK, 1)
|
||||
else:
|
||||
import fcntl
|
||||
fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
break
|
||||
except (BlockingIOError, OSError):
|
||||
if time.monotonic() >= deadline:
|
||||
raise CredentialError("Credential update is busy; retry after the other login or refresh finishes.") from None
|
||||
time.sleep(0.025)
|
||||
held[str(path)] = fd
|
||||
_held.paths = held
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
held.pop(str(path), None)
|
||||
finally:
|
||||
os.close(fd)
|
||||
finally:
|
||||
thread_lock.release()
|
||||
|
||||
|
||||
class CredentialStore:
|
||||
def __init__(self, state_dir, legacy_path=None):
|
||||
self.workspace = WorkspacePaths(Path(state_dir).parent)
|
||||
self.state_dir = self.workspace.state
|
||||
self.legacy = Path(legacy_path or self.workspace.legacy / "auth.json").expanduser().absolute()
|
||||
self.user = user_data_dir() / "credentials" / "default.json"
|
||||
self.project = self.workspace.project_credentials
|
||||
self.selection = self.state_dir / "state.json"
|
||||
|
||||
def selected(self):
|
||||
bound = getattr(_held,"bindings",{}).get(str(self.state_dir.absolute()))
|
||||
if bound: return bound
|
||||
explicit = os.environ.get("NAPSEER_AUTH_FILE")
|
||||
if explicit:
|
||||
path = Path(explicit).expanduser().absolute()
|
||||
if path.is_symlink():
|
||||
raise CredentialError("Explicit credentials must not be a symbolic link.")
|
||||
if path.resolve().is_relative_to(self.workspace.root) and self.workspace.root != Path.home().resolve():
|
||||
raise CredentialError("NAPSEER_AUTH_FILE must be outside the checkout; migrate private state first.")
|
||||
return path, "explicit"
|
||||
if self.legacy != self.workspace.legacy / "auth.json":
|
||||
if self.legacy.resolve().is_relative_to(self.workspace.root) and self.workspace.root != Path.home().resolve():
|
||||
raise CredentialError("Explicit credentials must be outside the checkout; migrate private state first.")
|
||||
return self.legacy, "explicit"
|
||||
self.workspace.require_migrated()
|
||||
if read_json(self.selection).get("credential_profile") == "user":
|
||||
return self.user, "user"
|
||||
if self.project.exists() or self.project.is_symlink():
|
||||
return self.project, "project"
|
||||
return self.user, "user"
|
||||
|
||||
@contextmanager
|
||||
def transaction(self):
|
||||
"""Pin one credential source through read/refresh/write despite new overrides."""
|
||||
identity = str(self.state_dir.absolute())
|
||||
chosen = self.selected()
|
||||
with credential_lock(chosen[0]):
|
||||
bindings = getattr(_held,"bindings",{})
|
||||
previous = bindings.get(identity)
|
||||
bindings[identity] = chosen; _held.bindings = bindings
|
||||
try: yield
|
||||
finally:
|
||||
if previous: bindings[identity] = previous
|
||||
else: bindings.pop(identity,None)
|
||||
|
||||
def use_user(self):
|
||||
current = read_json(self.user)
|
||||
if not current.get("token") or current.get("logged_out"):
|
||||
raise CredentialError("Run nap auth login before selecting the general user login.")
|
||||
with credential_lock(self.selection):
|
||||
selection = read_json(self.selection)
|
||||
selection["credential_profile"] = "user"
|
||||
atomic_json(self.selection, selection)
|
||||
return self.user
|
||||
|
||||
def migrate(self):
|
||||
self.workspace.migrate()
|
||||
return self.project
|
||||
|
||||
def read(self):
|
||||
path, source = self.selected()
|
||||
credentials = read_json(path)
|
||||
selection = {k: v for k, v in read_json(self.selection).items() if k in SELECTION_KEYS}
|
||||
# Project-bound grants retain their issued project. General grants only
|
||||
# consume repository selection; selection cannot confer authority.
|
||||
return {**selection, **credentials}
|
||||
|
||||
def write(self, data):
|
||||
path, source = self.selected()
|
||||
with credential_lock(path):
|
||||
if source == "explicit":
|
||||
atomic_json(path, data)
|
||||
return
|
||||
credentials = {k: v for k, v in data.items() if k in CREDENTIAL_KEYS}
|
||||
if source == "project":
|
||||
# Preserve project-bound worker/recovery/key metadata during
|
||||
# refresh after legacy migration. Profile selection stays local
|
||||
# to the worktree and must not change sibling worktrees.
|
||||
credentials = {k: v for k, v in data.items() if k != "credential_profile"}
|
||||
current = read_json(path)
|
||||
# Reading project metadata must not churn or overwrite shared
|
||||
# credentials, including after another process logged out.
|
||||
if credentials != current:
|
||||
atomic_json(path, credentials)
|
||||
selection = {**read_json(self.selection), **{k: v for k, v in data.items() if k in SELECTION_KEYS}}
|
||||
if selection != read_json(self.selection):
|
||||
atomic_json(self.selection, selection)
|
||||
|
||||
def login(self, credentials, project=False):
|
||||
target = self.project if project else self.user
|
||||
with credential_lock(target):
|
||||
atomic_json(target, credentials)
|
||||
if project:
|
||||
with credential_lock(self.selection):
|
||||
selection = read_json(self.selection); selection["credential_profile"] = "project"
|
||||
atomic_json(self.selection, selection)
|
||||
return target
|
||||
|
||||
def logout(self, project=False):
|
||||
target = self.selected()[0] if project else self.user
|
||||
with credential_lock(target):
|
||||
previous = read_json(target)
|
||||
# Keep a tombstone: removing an override would silently restore the
|
||||
# more privileged user session on the next request.
|
||||
atomic_json(target, {"base_url": previous.get("base_url"), "logged_out": True})
|
||||
return target
|
||||
+7497
-816
File diff suppressed because it is too large
Load Diff
@@ -8,11 +8,12 @@ session. This supervisor owns those client-facing pipes while treating
|
||||
- valid requests are forwarded without head-of-line blocking and responses
|
||||
are correlated by JSON-RPC request id;
|
||||
- notifications never create client-facing responses;
|
||||
- cancellation releases the matching pending request and forwards the
|
||||
notification to the worker;
|
||||
- cancellation stops waiting for a result but retains unfinished work until
|
||||
the worker acknowledges cleanup, with bounded grace before termination;
|
||||
- a worker exit fails every affected in-flight request once and never replays
|
||||
it automatically;
|
||||
- a replaced worker is reloaded at the next quiescent request boundary.
|
||||
- a replaced worker, or an explicitly configured runtime dependency behind a
|
||||
stable launcher, is reloaded at the next quiescent request boundary.
|
||||
|
||||
The supervisor never logs MCP payloads, tool results, auth state, or child
|
||||
stderr. Protocol stdout contains JSON-RPC messages only.
|
||||
@@ -25,10 +26,12 @@ import json
|
||||
import os
|
||||
import pathlib
|
||||
import queue
|
||||
import select
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import BinaryIO, Callable
|
||||
|
||||
@@ -39,6 +42,11 @@ WORKER_PATH = pathlib.Path(
|
||||
pathlib.Path(__file__).resolve().with_name("napseer_mcp_server.py"),
|
||||
)
|
||||
).expanduser()
|
||||
WORKER_WATCH_PATHS = tuple(
|
||||
pathlib.Path(value).expanduser()
|
||||
for value in os.environ.get("NAPSEER_MCP_WORKER_WATCH_PATHS", "").split(os.pathsep)
|
||||
if value.strip()
|
||||
)
|
||||
RESPONSE_TIMEOUT_SECONDS = max(
|
||||
1,
|
||||
int(os.environ.get("NAPSEER_MCP_RESPONSE_TIMEOUT_SECONDS", "60")),
|
||||
@@ -48,6 +56,10 @@ MAX_IN_FLIGHT_REQUESTS = max(
|
||||
int(os.environ.get("NAPSEER_MCP_MAX_IN_FLIGHT_REQUESTS", "32")),
|
||||
)
|
||||
WORKER_RESTARTED_ERROR = -32098
|
||||
REQUEST_REJECTED_ERROR = -32097
|
||||
CANCEL_GRACE_SECONDS = max(1, int(os.environ.get("NAPSEER_MCP_CANCEL_GRACE_SECONDS", "10")))
|
||||
WRITE_TIMEOUT_SECONDS = max(1, int(os.environ.get("NAPSEER_MCP_WRITE_TIMEOUT_SECONDS", "5")))
|
||||
REQUEST_FINISHED = "notifications/napseer/requestFinished"
|
||||
_CANCELLED = object()
|
||||
|
||||
|
||||
@@ -55,11 +67,24 @@ class WorkerUnavailable(RuntimeError):
|
||||
"""The replaceable MCP worker could not complete a request."""
|
||||
|
||||
|
||||
class RequestRejected(WorkerUnavailable):
|
||||
"""The request was not sent to the worker and cannot have executed."""
|
||||
|
||||
|
||||
def source_identity(path: pathlib.Path) -> tuple[int, int, int]:
|
||||
stat = path.stat()
|
||||
return stat.st_ino, stat.st_size, stat.st_mtime_ns
|
||||
|
||||
|
||||
def runtime_identity(
|
||||
worker_path: pathlib.Path,
|
||||
watch_paths: tuple[pathlib.Path, ...] = (),
|
||||
) -> tuple[tuple[int, int, int], ...]:
|
||||
"""Identify every source whose replacement changes the worker runtime."""
|
||||
|
||||
return tuple(source_identity(path) for path in (worker_path, *watch_paths))
|
||||
|
||||
|
||||
def parse_message(message: bytes):
|
||||
try:
|
||||
payload = json.loads(message)
|
||||
@@ -97,16 +122,16 @@ def cancelled_request_id(message: bytes):
|
||||
return True, params.get("requestId")
|
||||
|
||||
|
||||
def worker_error(message: bytes) -> bytes:
|
||||
def worker_error(message: bytes, *, rejected=False) -> bytes:
|
||||
payload = {
|
||||
"jsonrpc": "2.0",
|
||||
"id": request_id(message),
|
||||
"error": {
|
||||
"code": WORKER_RESTARTED_ERROR,
|
||||
"code": REQUEST_REJECTED_ERROR if rejected else WORKER_RESTARTED_ERROR,
|
||||
"message": (
|
||||
"Napseer MCP could not complete this request. The outcome may be "
|
||||
"uncertain; inspect state before retrying a mutation."
|
||||
),
|
||||
) if not rejected else "Napseer MCP is busy or this request id is still active. This request was not started; retry later.",
|
||||
},
|
||||
}
|
||||
return json.dumps(payload, separators=(",", ":")).encode("utf-8") + b"\n"
|
||||
@@ -118,13 +143,21 @@ class PendingRequest:
|
||||
key: str
|
||||
generation: int
|
||||
response: queue.Queue = field(default_factory=lambda: queue.Queue(maxsize=1))
|
||||
cancelled: bool = False
|
||||
cancel_timer: threading.Timer | None = None
|
||||
|
||||
|
||||
class Worker:
|
||||
def __init__(self, path: pathlib.Path, unsolicited: Callable[[bytes], None] | None = None):
|
||||
def __init__(
|
||||
self,
|
||||
path: pathlib.Path,
|
||||
unsolicited: Callable[[bytes], None] | None = None,
|
||||
watch_paths: tuple[pathlib.Path, ...] = (),
|
||||
):
|
||||
self.path = path
|
||||
self.watch_paths = tuple(watch_paths)
|
||||
self.process: subprocess.Popen[bytes] | None = None
|
||||
self.identity: tuple[int, int, int] | None = None
|
||||
self.identity: tuple[tuple[int, int, int], ...] | None = None
|
||||
self.generation = 0
|
||||
self.unsolicited = unsolicited
|
||||
self._lifecycle_lock = threading.RLock()
|
||||
@@ -135,7 +168,7 @@ class Worker:
|
||||
|
||||
def _discard_stderr(self, stream: BinaryIO) -> None:
|
||||
try:
|
||||
for _line in stream:
|
||||
while stream.read(4096):
|
||||
pass
|
||||
finally:
|
||||
stream.close()
|
||||
@@ -159,6 +192,8 @@ class Worker:
|
||||
if pending.generation != generation:
|
||||
continue
|
||||
self._pending.pop(key, None)
|
||||
if pending.cancel_timer is not None:
|
||||
pending.cancel_timer.cancel()
|
||||
failed.append(pending)
|
||||
for pending in failed:
|
||||
self._signal(pending, error)
|
||||
@@ -168,18 +203,32 @@ class Worker:
|
||||
if payload is None:
|
||||
return False
|
||||
normalized = response if response.endswith(b"\n") else response + b"\n"
|
||||
if payload.get("method") == REQUEST_FINISHED:
|
||||
params = payload.get("params") or {}
|
||||
if isinstance(params, dict) and "requestId" in params:
|
||||
key = request_key(params["requestId"])
|
||||
with self._pending_lock:
|
||||
pending = self._pending.get(key)
|
||||
if pending is not None and pending.generation == generation and pending.cancelled:
|
||||
self._pending.pop(key, None)
|
||||
if pending.cancel_timer is not None:
|
||||
pending.cancel_timer.cancel()
|
||||
return True
|
||||
if "id" in payload:
|
||||
key = request_key(payload.get("id"))
|
||||
with self._pending_lock:
|
||||
pending = self._pending.get(key)
|
||||
if pending is not None and pending.generation == generation:
|
||||
self._pending.pop(key, None)
|
||||
if pending.cancel_timer is not None:
|
||||
pending.cancel_timer.cancel()
|
||||
else:
|
||||
pending = None
|
||||
if pending is not None:
|
||||
self._signal(pending, normalized)
|
||||
if not pending.cancelled:
|
||||
self._signal(pending, normalized)
|
||||
return True
|
||||
if "method" in payload and self.unsolicited is not None:
|
||||
if generation == self.generation and "method" in payload and self.unsolicited is not None:
|
||||
self.unsolicited(normalized)
|
||||
return True
|
||||
|
||||
@@ -219,12 +268,15 @@ class Worker:
|
||||
if process is None:
|
||||
return
|
||||
if process.poll() is None:
|
||||
process.terminate()
|
||||
try:
|
||||
process.terminate()
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
try:
|
||||
process.wait(timeout=2)
|
||||
except subprocess.TimeoutExpired:
|
||||
process.kill()
|
||||
process.wait()
|
||||
process.wait(timeout=2)
|
||||
for stream in (process.stdin, process.stdout):
|
||||
if stream is not None:
|
||||
try:
|
||||
@@ -237,13 +289,14 @@ class Worker:
|
||||
with self._lifecycle_lock:
|
||||
self._stop_locked()
|
||||
try:
|
||||
identity = source_identity(self.path)
|
||||
identity = runtime_identity(self.path, self.watch_paths)
|
||||
process = subprocess.Popen(
|
||||
[sys.executable, "-u", str(self.path)],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
bufsize=0,
|
||||
env={**os.environ, "NAPSEER_MCP_SUPERVISED": "1"},
|
||||
)
|
||||
except (OSError, subprocess.SubprocessError) as exc:
|
||||
raise WorkerUnavailable("unable to start Napseer MCP worker") from exc
|
||||
@@ -254,6 +307,7 @@ class Worker:
|
||||
self.generation += 1
|
||||
generation = self.generation
|
||||
self.process = process
|
||||
os.set_blocking(process.stdin.fileno(), False)
|
||||
self.identity = identity
|
||||
threading.Thread(
|
||||
target=self._discard_stderr,
|
||||
@@ -275,7 +329,7 @@ class Worker:
|
||||
def ensure_current(self) -> None:
|
||||
with self._lifecycle_lock:
|
||||
try:
|
||||
current_identity = source_identity(self.path)
|
||||
current_identity = runtime_identity(self.path, self.watch_paths)
|
||||
except OSError as exc:
|
||||
self._stop_locked()
|
||||
raise WorkerUnavailable("Napseer MCP worker is missing") from exc
|
||||
@@ -288,14 +342,30 @@ class Worker:
|
||||
self.start()
|
||||
|
||||
def _write(self, process: subprocess.Popen[bytes], message: bytes) -> None:
|
||||
with self._stdin_lock:
|
||||
deadline = time.monotonic() + WRITE_TIMEOUT_SECONDS
|
||||
if not self._stdin_lock.acquire(timeout=WRITE_TIMEOUT_SECONDS):
|
||||
raise WorkerUnavailable("Napseer MCP worker input timed out")
|
||||
try:
|
||||
if self.process is not process or process.poll() is not None or process.stdin is None:
|
||||
raise WorkerUnavailable("Napseer MCP worker is not running")
|
||||
try:
|
||||
process.stdin.write(message if message.endswith(b"\n") else message + b"\n")
|
||||
process.stdin.flush()
|
||||
except (BrokenPipeError, OSError) as exc:
|
||||
remaining = memoryview(message if message.endswith(b"\n") else message + b"\n")
|
||||
while remaining:
|
||||
timeout = deadline - time.monotonic()
|
||||
if timeout <= 0 or self.process is not process:
|
||||
raise WorkerUnavailable("Napseer MCP worker input timed out")
|
||||
fd = process.stdin.fileno()
|
||||
if not select.select([], [fd], [], timeout)[1]:
|
||||
raise WorkerUnavailable("Napseer MCP worker input timed out")
|
||||
try:
|
||||
count = os.write(fd, remaining)
|
||||
remaining = remaining[count:]
|
||||
except BlockingIOError:
|
||||
continue
|
||||
except (BrokenPipeError, OSError, ValueError) as exc:
|
||||
raise WorkerUnavailable("Napseer MCP worker input closed") from exc
|
||||
finally:
|
||||
self._stdin_lock.release()
|
||||
|
||||
def submit(self, message: bytes) -> PendingRequest | None:
|
||||
self.ensure_current()
|
||||
@@ -305,16 +375,20 @@ class Worker:
|
||||
if process is None:
|
||||
raise WorkerUnavailable("Napseer MCP worker is not running")
|
||||
if is_notification(message):
|
||||
self._write(process, message)
|
||||
try:
|
||||
self._write(process, message)
|
||||
except WorkerUnavailable:
|
||||
self.stop()
|
||||
raise
|
||||
return None
|
||||
|
||||
key = request_key(request_id(message))
|
||||
pending = PendingRequest(message=message, key=key, generation=generation)
|
||||
with self._pending_lock:
|
||||
if key in self._pending:
|
||||
raise WorkerUnavailable("duplicate in-flight JSON-RPC request id")
|
||||
raise RequestRejected("duplicate in-flight JSON-RPC request id")
|
||||
if len(self._pending) >= MAX_IN_FLIGHT_REQUESTS:
|
||||
raise WorkerUnavailable("too many in-flight Napseer MCP requests")
|
||||
raise RequestRejected("too many unfinished Napseer MCP requests")
|
||||
self._pending[key] = pending
|
||||
try:
|
||||
self._write(process, message)
|
||||
@@ -322,26 +396,54 @@ class Worker:
|
||||
with self._pending_lock:
|
||||
if self._pending.get(key) is pending:
|
||||
self._pending.pop(key, None)
|
||||
# A partial write has an uncertain outcome and corrupts framing.
|
||||
# Stop this generation rather than appending another request to it.
|
||||
self.stop()
|
||||
raise
|
||||
return pending
|
||||
|
||||
def cancel(self, value) -> bool:
|
||||
def cancel(self, value, *, expected: PendingRequest | None = None) -> bool:
|
||||
key = request_key(value)
|
||||
with self._pending_lock:
|
||||
pending = self._pending.get(key)
|
||||
if expected is not None and pending is not expected:
|
||||
return False
|
||||
if pending is None or request_method(pending.message) == "initialize":
|
||||
return False
|
||||
self._pending.pop(key, None)
|
||||
if pending.cancelled:
|
||||
return True
|
||||
pending.cancelled = True
|
||||
pending.cancel_timer = threading.Timer(CANCEL_GRACE_SECONDS, self._expire_cancelled, args=(pending,))
|
||||
pending.cancel_timer.daemon = True
|
||||
timer = pending.cancel_timer
|
||||
self._signal(pending, _CANCELLED)
|
||||
timer.start()
|
||||
process = self.process
|
||||
if process is not None and pending.generation == self.generation:
|
||||
try:
|
||||
self._write(process, json.dumps({"jsonrpc": "2.0", "method": "notifications/cancelled",
|
||||
"params": {"requestId": value}}).encode("utf-8"))
|
||||
except WorkerUnavailable:
|
||||
self._expire_cancelled(pending)
|
||||
return True
|
||||
|
||||
def _expire_cancelled(self, pending: PendingRequest) -> None:
|
||||
with self._lifecycle_lock:
|
||||
with self._pending_lock:
|
||||
unfinished = self._pending.get(pending.key) is pending
|
||||
if unfinished and pending.generation == self.generation:
|
||||
self._stop_locked()
|
||||
|
||||
def wait(self, pending: PendingRequest) -> bytes | None:
|
||||
try:
|
||||
result = pending.response.get(timeout=RESPONSE_TIMEOUT_SECONDS)
|
||||
except queue.Empty as exc:
|
||||
with self._pending_lock:
|
||||
if self._pending.get(pending.key) is pending:
|
||||
self._pending.pop(pending.key, None)
|
||||
if request_method(pending.message) == "initialize":
|
||||
# Initialize is not cancellable, but a stuck initialization
|
||||
# must not permanently consume the worker generation.
|
||||
self._expire_cancelled(pending)
|
||||
else:
|
||||
self.cancel(request_id(pending.message), expected=pending)
|
||||
raise WorkerUnavailable("Napseer MCP worker response timed out") from exc
|
||||
if result is _CANCELLED:
|
||||
return None
|
||||
@@ -361,7 +463,11 @@ def supervise() -> int:
|
||||
sys.stdout.buffer.write(response if response.endswith(b"\n") else response + b"\n")
|
||||
sys.stdout.buffer.flush()
|
||||
|
||||
worker = Worker(WORKER_PATH, unsolicited=emit)
|
||||
worker = Worker(
|
||||
WORKER_PATH,
|
||||
unsolicited=emit,
|
||||
watch_paths=WORKER_WATCH_PATHS,
|
||||
)
|
||||
waiters = concurrent.futures.ThreadPoolExecutor(
|
||||
max_workers=MAX_IN_FLIGHT_REQUESTS,
|
||||
thread_name_prefix="napseer-mcp-request",
|
||||
@@ -391,16 +497,12 @@ def supervise() -> int:
|
||||
cancelled, value = cancelled_request_id(message)
|
||||
if cancelled:
|
||||
worker.cancel(value)
|
||||
try:
|
||||
worker.submit(message)
|
||||
except WorkerUnavailable:
|
||||
pass
|
||||
continue
|
||||
try:
|
||||
pending = worker.submit(message)
|
||||
except WorkerUnavailable:
|
||||
except WorkerUnavailable as exc:
|
||||
if not is_notification(message):
|
||||
emit(worker_error(message))
|
||||
emit(worker_error(message, rejected=isinstance(exc, RequestRejected)))
|
||||
continue
|
||||
if pending is not None:
|
||||
waiters.submit(finish, message, pending)
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
"""All tests use disposable user credential state, including legacy test scripts."""
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolated_user_credentials(monkeypatch, tmp_path, tmp_path_factory):
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setenv("NAPSEER_USER_DATA_DIR", str(tmp_path_factory.mktemp("napseer-user")))
|
||||
for name in ("NAPSEER_API_KEY", "NAPSEER_TOKEN", "NAPSEER_REFRESH_TOKEN", "NAPSEER_AUTH_FILE", "NAPSEER_PROJECT_ROOT", "NAPSEER_PROJECT_ID", "NAPSEER_BASE_URL"):
|
||||
monkeypatch.delenv(name, raising=False)
|
||||
@@ -0,0 +1,127 @@
|
||||
"""Public worker/CLI entry points against synthetic binary HTTP storage."""
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import uuid
|
||||
import zipfile
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import urlsplit, parse_qs
|
||||
|
||||
import pytest
|
||||
from test_user_credentials import worker, outcome, SCRIPT
|
||||
from napseer_credentials import CredentialStore, atomic_json
|
||||
|
||||
PROJECT="11111111-1111-4111-8111-111111111111"
|
||||
NODE="22222222-2222-4222-8222-222222222222"
|
||||
|
||||
@pytest.fixture
|
||||
def storage(tmp_path):
|
||||
state={"files":{},"corrupt":False,"keys":[],"projects":[]}
|
||||
node={"id":NODE,"name":"Guide","full_path":"/services/guide","type":"documentation","updated_at":"2026-09-09T00:00:00Z","content_text":"Recovery procedure","metadata":{},"links":[]}
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self,*_args): pass
|
||||
def reply(self,status,value,binary=False):
|
||||
body=value if binary else json.dumps(value).encode()
|
||||
self.send_response(status);self.send_header("Content-Length",str(len(body)));self.end_headers();self.wfile.write(body)
|
||||
def do_GET(self):
|
||||
path=urlsplit(self.path).path
|
||||
if path=='/v1/projects':return self.reply(200,{'items':state['projects']})
|
||||
if path=='/v1/api-keys':return self.reply(200,{'items':state['keys']})
|
||||
if path.endswith('/nodes'): return self.reply(200,{"items":[node]})
|
||||
if path.endswith('/nodes/'+NODE):return self.reply(200,node)
|
||||
if path.endswith('/files'): return self.reply(200,{"items":[entry[0] for entry in state["files"].values()]})
|
||||
identity=path.split('/')[-2] if path.endswith('/metadata') else path.split('/')[-1]
|
||||
entry=state["files"].get(identity)
|
||||
if not entry:return self.reply(404,{"error":{"code":"not_found"}})
|
||||
if path.endswith('/metadata'):return self.reply(200,entry[0])
|
||||
return self.reply(200,b"corrupted" if state["corrupt"] else entry[1],True)
|
||||
def do_POST(self):
|
||||
if self.path=='/v1/projects':
|
||||
payload=json.loads(self.rfile.read(int(self.headers['Content-Length'])))
|
||||
project={**payload,'id':str(uuid.uuid4()),'encryption_state':'standard'}
|
||||
state['projects'].append(project);return self.reply(200,project)
|
||||
if self.path=='/v1/api-keys':
|
||||
payload=json.loads(self.rfile.read(int(self.headers['Content-Length'])))
|
||||
key={**payload,'id':str(uuid.uuid4())};state['keys'].append(key)
|
||||
return self.reply(200,{**key,'api_key':'npk_'+uuid.uuid4().hex})
|
||||
query=parse_qs(urlsplit(self.path).query)
|
||||
body=self.rfile.read(int(self.headers['Content-Length']))
|
||||
identity=query['id'][0]
|
||||
info={"id":identity,"name":query['name'][0],"node_id":query.get('node_id',[None])[0],"size_bytes":len(body),"sha256":hashlib.sha256(body).hexdigest()}
|
||||
state['files'][identity]=(info,body)
|
||||
return self.reply(200,info)
|
||||
def do_DELETE(self):
|
||||
identity=urlsplit(self.path).path.split('/')[-1];state['files'].pop(identity,None)
|
||||
return self.reply(200,{"archived":True})
|
||||
server=ThreadingHTTPServer(('127.0.0.1',0),Handler)
|
||||
thread=threading.Thread(target=server.serve_forever,daemon=True);thread.start()
|
||||
store=CredentialStore(tmp_path/'.napseer')
|
||||
store.login({"base_url":f"http://127.0.0.1:{server.server_port}","token":"synthetic-access","account_mode":"operator_account"})
|
||||
atomic_json(store.selection,{"project_id":PROJECT})
|
||||
try:yield state
|
||||
finally:server.shutdown();server.server_close();thread.join(timeout=2)
|
||||
|
||||
|
||||
def test_agent_upload_retries_download_and_export_preserve_binary_and_references(tmp_path,storage):
|
||||
body=bytes([0,255,128,13,10,65]);source=tmp_path/'proof.bin';source.write_bytes(body)
|
||||
args={"local_path":str(source),"node_id":NODE}
|
||||
first=outcome(worker(tmp_path,f"m.call_tool_impl('nap_file_upload',{args!r})"))
|
||||
assert first['ok'],first.get('code')
|
||||
identity=first['result']['id']
|
||||
again=outcome(worker(tmp_path,f"m.call_tool_impl('nap_file_upload',{args!r})"))
|
||||
assert again['result']['id']==identity
|
||||
assert len(storage['files'])==1
|
||||
destination=tmp_path/'download.bin'
|
||||
result=outcome(worker(tmp_path,f"m.call_tool_impl('nap_file_download',{{'id':{identity!r},'local_path':{str(destination)!r}}})"))
|
||||
assert result['ok'];assert destination.read_bytes()==body
|
||||
rejected=outcome(worker(tmp_path,f"m.call_tool_impl('nap_file_download',{{'id':{identity!r},'local_path':{str(source)!r}}})"))
|
||||
assert rejected['code']=='destination_unavailable';assert source.read_bytes()==body
|
||||
target=tmp_path/'export.zip'
|
||||
env=dict(os.environ,NAPSEER_PROJECT_ROOT=str(tmp_path),NAPSEER_TELEMETRY='0')
|
||||
for name in ['NAPSEER_AUTH_FILE','NAPSEER_BASE_URL','NAPSEER_PROJECT_ID']:env.pop(name,None)
|
||||
exported=subprocess.run([sys.executable,str(SCRIPT),'export','--path',str(target)],cwd=tmp_path,env=env,capture_output=True,text=True,timeout=15)
|
||||
assert exported.returncode==0
|
||||
assert json.loads(exported.stdout)['status']=='exported'
|
||||
with zipfile.ZipFile(target) as archive:
|
||||
manifest=json.loads(archive.read('manifest.json'))
|
||||
assert manifest['files'][0]['node_id']==NODE
|
||||
assert archive.read('files/'+identity)==body
|
||||
assert json.loads(archive.read('nodes/'+NODE+'.json'))['content_text']=='Recovery procedure'
|
||||
assert not any('auth' in name for name in archive.namelist())
|
||||
storage['corrupt']=True
|
||||
rejected=outcome(worker(tmp_path,f"m.export_project({{'local_path':{str(tmp_path/'broken.zip')!r}}})"))
|
||||
assert rejected['code']=='file_integrity_failed'
|
||||
assert not (tmp_path/'broken.zip').exists()
|
||||
assert not list(tmp_path.glob('.napseer-export-*'))
|
||||
assert outcome(worker(tmp_path,f"m.call_tool_impl('nap_file_archive',{{'id':{identity!r}}})"))['ok']
|
||||
assert not storage['files']
|
||||
|
||||
|
||||
def test_cli_key_issuance_writes_private_credential_without_printing_it(tmp_path,storage):
|
||||
env=dict(os.environ,NAPSEER_PROJECT_ROOT=str(tmp_path),NAPSEER_TELEMETRY='0')
|
||||
for name in ['NAPSEER_AUTH_FILE','NAPSEER_BASE_URL','NAPSEER_PROJECT_ID']:env.pop(name,None)
|
||||
created=subprocess.run([sys.executable,str(SCRIPT),'auth','keys','create','--project-id',PROJECT,'--name','Test delegate'],cwd=tmp_path,env=env,capture_output=True,text=True,timeout=15)
|
||||
assert created.returncode==0
|
||||
assert 'npk_' not in created.stdout
|
||||
result=json.loads(created.stdout);path=Path(result['credential_file'])
|
||||
assert path.stat().st_mode & 0o777 == 0o600
|
||||
assert json.loads(path.read_text())['token'].startswith('npk_')
|
||||
assert not (tmp_path/'.napseer/auth.json').exists()
|
||||
|
||||
|
||||
def test_two_agents_bootstrap_one_project_with_general_login(tmp_path,storage):
|
||||
store=CredentialStore(tmp_path/'.napseer');store.selection.unlink()
|
||||
first=worker(tmp_path,"m.resolve_project_id({})")
|
||||
second=worker(tmp_path,"m.resolve_project_id({})")
|
||||
one,two=outcome(first),outcome(second)
|
||||
assert one['ok'] and two['ok']
|
||||
assert one['result']==two['result']
|
||||
assert len(storage['projects'])==1
|
||||
assert json.loads(store.user.read_text())['account_mode']=='operator_account'
|
||||
assert 'project_id' not in json.loads(store.user.read_text())
|
||||
assert not store.legacy.exists()
|
||||
assert (tmp_path/'.napseer/project.json').exists()
|
||||
@@ -29,37 +29,18 @@ def anonymous_auth(mod):
|
||||
}
|
||||
|
||||
|
||||
def test_initial_enrollment_persists_refresh_credential():
|
||||
def test_unconfigured_agent_requires_user_login_without_enrollment():
|
||||
mod = load_module()
|
||||
mod.TOKEN = None
|
||||
mod.AUTH = anonymous_auth(mod)
|
||||
mod.BASE_URL = mod.AUTH["base_url"]
|
||||
mod.DEFAULT_PROJECT_ID = None
|
||||
saved = []
|
||||
requests = []
|
||||
mod.save_auth = lambda payload: saved.append(dict(payload))
|
||||
|
||||
def request(method, path, payload=None, token_required=False):
|
||||
requests.append((method, path, payload, token_required))
|
||||
return {
|
||||
"token": {
|
||||
"access_token": "access",
|
||||
"refresh_token": "refresh",
|
||||
"expires_at": "2099-01-01T00:00:00Z",
|
||||
"refresh_expires_at": "",
|
||||
},
|
||||
"worker": {"id": "worker", "agent_id": "agent-id"},
|
||||
}
|
||||
|
||||
mod.request_json = request
|
||||
mod.ensure_enrolled({"slug": "example"})
|
||||
|
||||
assert saved[0]["refresh_token"] == "refresh"
|
||||
assert "refresh_expires_at" in saved[0]
|
||||
assert [(method, path) for method, path, _, _ in requests] == [
|
||||
("POST", "/v1/enrollment/token")
|
||||
]
|
||||
assert requests[0][3] is False
|
||||
mod.request_json = lambda *args, **kwargs: requests.append(args)
|
||||
try:
|
||||
mod.ensure_enrolled({"slug": "example"})
|
||||
except mod.SafeToolError as exc:
|
||||
assert exc.code == "auth_required"
|
||||
else:
|
||||
raise AssertionError("unconfigured agents must not enroll")
|
||||
assert not requests
|
||||
|
||||
|
||||
def test_failed_anonymous_refresh_uses_ssh_recovery():
|
||||
@@ -107,65 +88,28 @@ def test_failed_anonymous_refresh_uses_ssh_recovery():
|
||||
assert saved[0]["refresh_token"] == "recovered-refresh"
|
||||
|
||||
|
||||
def test_failed_mixed_state_refresh_uses_ssh_recovery():
|
||||
def test_oauth_failure_never_falls_back_to_anonymous_identity():
|
||||
mod = load_module()
|
||||
mod.AUTH = {
|
||||
**anonymous_auth(mod),
|
||||
"account_mode": "operator_project",
|
||||
"oauth_client_id": "nap-cli",
|
||||
"project_id": "project",
|
||||
"ssh_private_key": "test-private-key",
|
||||
"ssh_public_key": "ssh-ed25519 test-public-key",
|
||||
}
|
||||
mod.BASE_URL = mod.AUTH["base_url"]
|
||||
mod.DEFAULT_PROJECT_ID = "project"
|
||||
mod.REFRESH_TOKEN = "stale-refresh"
|
||||
mod.REFRESH_EXPIRES_AT = None
|
||||
mod.AUTH = {**anonymous_auth(mod), "account_mode": "operator_project"}
|
||||
mod.TOKEN = "expired-access"
|
||||
mod.REFRESH_TOKEN = "revoked-refresh"
|
||||
mod.refresh_public_auth_state = lambda: mod.AUTH
|
||||
mod.auth_public_key = lambda: "ssh-ed25519 test-public-key"
|
||||
mod.sign_with_auth_key = lambda *args: "test-signature"
|
||||
saved = []
|
||||
mod.save_auth = lambda payload: saved.append(dict(payload))
|
||||
|
||||
def request_form(method, path, payload=None, token_required=False):
|
||||
assert (method, path, token_required) == ("POST", "/v1/oauth/token", False)
|
||||
raise RuntimeError("HTTP 401")
|
||||
|
||||
def request(method, path, payload=None, token_required=False):
|
||||
if path == "/v1/enrollment/refresh":
|
||||
raise RuntimeError("HTTP 401")
|
||||
if path == "/v1/enrollment/challenges":
|
||||
return {"challenge_id": "challenge", "challenge_text": "sign-me"}
|
||||
if path == "/v1/enrollment/verify":
|
||||
return {
|
||||
"token": {
|
||||
"access_token": "recovered-access",
|
||||
"refresh_token": "recovered-refresh",
|
||||
"expires_at": "2099-01-01T00:00:00Z",
|
||||
"refresh_expires_at": "2099-12-31T00:00:00Z",
|
||||
},
|
||||
"worker": {
|
||||
"id": "worker",
|
||||
"agent_id": "agent-id",
|
||||
"name": "agent",
|
||||
"device_fingerprint": "device",
|
||||
"root_path": "/repo",
|
||||
"project_id": "project",
|
||||
},
|
||||
}
|
||||
raise AssertionError(path)
|
||||
|
||||
mod.request_form_json = request_form
|
||||
mod.request_json = request
|
||||
result = mod.renew_auth()
|
||||
|
||||
assert result["method"] == "anonymous_ssh_recovery"
|
||||
assert result["project_id"] == "project"
|
||||
assert saved[0]["refresh_token"] == "recovered-refresh"
|
||||
calls = []
|
||||
def rejected(*args, **kwargs):
|
||||
raise mod.SafeToolError("http_401", "Login rejected", status=401)
|
||||
mod.request_form_json = rejected
|
||||
mod.request_json = lambda *args, **kwargs: calls.append(args)
|
||||
try:
|
||||
mod.renew_auth()
|
||||
except mod.SafeToolError as exc:
|
||||
assert exc.code == "auth_required"
|
||||
else:
|
||||
raise AssertionError("OAuth revocation must require user login")
|
||||
assert not calls
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_initial_enrollment_persists_refresh_credential()
|
||||
test_unconfigured_agent_requires_user_login_without_enrollment()
|
||||
test_failed_anonymous_refresh_uses_ssh_recovery()
|
||||
test_failed_mixed_state_refresh_uses_ssh_recovery()
|
||||
test_oauth_failure_never_falls_back_to_anonymous_identity()
|
||||
print("ok: anonymous auth recovery tests passed")
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Exercise the published `curl /install | python3 -` entry point over HTTP."""
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
|
||||
|
||||
def test_stdin_bootstrap_installs_verified_runtime_and_preserves_credentials(tmp_path):
|
||||
source = Path(__file__).resolve().parents[1] / "resources/scripts"
|
||||
spec = importlib.util.spec_from_file_location("bootstrap_contract", source / "nap_install.py")
|
||||
installer = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(installer)
|
||||
revision = "f" * 40
|
||||
items, payloads = [], {}
|
||||
for name in installer.SCRIPT_NAMES:
|
||||
content = (source / installer.INSTALL_PATHS[name]).read_text()
|
||||
item = {"name": name, "version": installer.CLI_RELEASE_VERSION,
|
||||
"bytes": len(content.encode()), "sha256": hashlib.sha256(content.encode()).hexdigest(),
|
||||
"source_repo": "https://gitea.powc-r.com/napseer/nap-cli", "source_revision": revision,
|
||||
"source_revision_status": "resolved", "contract_version": installer.CLI_DISTRIBUTION_CONTRACT_VERSION,
|
||||
"minimum_contract_version": installer.CLI_MINIMUM_CONTRACT_VERSION,
|
||||
"install_path": installer.INSTALL_PATHS[name], "mode": "0755"}
|
||||
items.append(item)
|
||||
payloads["/v1/scripts/" + name] = {**item, "content": content}
|
||||
payloads["/v1/scripts"] = {"schema_version": installer.CLI_BUNDLE_SCHEMA_VERSION,
|
||||
"bundle_id": "synthetic-bootstrap", "release_version": installer.CLI_RELEASE_VERSION,
|
||||
"contract": {"current": installer.CLI_DISTRIBUTION_CONTRACT_VERSION, "minimum_supported": installer.CLI_MINIMUM_CONTRACT_VERSION},
|
||||
"source": {"repo": "https://gitea.powc-r.com/napseer/nap-cli", "revision": revision, "revision_status": "resolved"}, "items": items}
|
||||
|
||||
class Provider(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def do_GET(self):
|
||||
assert self.path in payloads, "bootstrap must only download its declared bundle"
|
||||
raw = json.dumps(payloads[self.path]).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(raw)))
|
||||
self.end_headers()
|
||||
self.wfile.write(raw)
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Provider)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
workspace = tmp_path / "workspace"
|
||||
workspace.mkdir()
|
||||
user_data = tmp_path / "user-data"
|
||||
user_data.mkdir(exist_ok=True)
|
||||
auth = user_data / "auth.json"
|
||||
auth.write_text('{"refresh_token":"synthetic-preserved"}')
|
||||
env = {"PATH": os.environ["PATH"], "NAPSEER_HOME": str(tmp_path / "runtime"),
|
||||
"NAPSEER_BIN_DIR": str(tmp_path / "bin"), "NAPSEER_USER_DATA_DIR": str(user_data),
|
||||
"NAPSEER_PROJECT_ROOT": str(workspace), "NAPSEER_BASE_URL": f"http://127.0.0.1:{server.server_port}"}
|
||||
try:
|
||||
result = subprocess.run([sys.executable, "-"], input=(source / "nap_install.py").read_text(),
|
||||
cwd=tmp_path, env=env, capture_output=True, text=True, timeout=30)
|
||||
assert result.returncode == 0, "the advertised stdin bootstrap must install without an explicit subcommand"
|
||||
assert json.loads(result.stdout)["status"] == "installed"
|
||||
for name in installer.SCRIPT_NAMES:
|
||||
installed = tmp_path / "runtime/current" / installer.INSTALL_PATHS[name]
|
||||
assert installed.read_bytes() == (source / installer.INSTALL_PATHS[name]).read_bytes()
|
||||
assert auth.read_text() == '{"refresh_token":"synthetic-preserved"}'
|
||||
# Import the actual activated modules, including the newly added dependency.
|
||||
env["PYTHONPATH"] = str(tmp_path / "runtime/current")
|
||||
imported = subprocess.run([sys.executable, "-c", "import napseer_credentials, napseer_mcp_server"],
|
||||
cwd=tmp_path, env=env, capture_output=True, timeout=10)
|
||||
assert imported.returncode == 0, "activated runtime must load all dependencies"
|
||||
result = subprocess.run([sys.executable, str(tmp_path / "bin/nap"), "update"],
|
||||
cwd=tmp_path, env=env, capture_output=True, text=True, timeout=30)
|
||||
assert result.returncode == 0
|
||||
assert auth.read_text() == '{"refresh_token":"synthetic-preserved"}'
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
@@ -1,8 +1,10 @@
|
||||
import importlib.util
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import pathlib
|
||||
import time
|
||||
import sys
|
||||
import pytest
|
||||
|
||||
|
||||
@@ -15,6 +17,8 @@ SCRIPT_PATH = (
|
||||
|
||||
|
||||
def load_installer():
|
||||
if str(SCRIPT_PATH.parent) not in sys.path:
|
||||
sys.path.insert(0, str(SCRIPT_PATH.parent))
|
||||
spec = importlib.util.spec_from_file_location("nap_install_command_system_test", SCRIPT_PATH)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
@@ -33,7 +37,7 @@ def test_root_help_exposes_only_canonical_commands(capsys):
|
||||
output = capsys.readouterr().out
|
||||
visible = [name for name, metadata in module.COMMAND_METADATA.items() if metadata["visible"]]
|
||||
assert visible == [name for name, _summary in module.CANONICAL_COMMANDS]
|
||||
assert len(visible) == 10
|
||||
assert len(visible) == 13
|
||||
for name in visible:
|
||||
assert f" {name}" in output
|
||||
for hidden in ("chat", "plan", "lineage", "agent", "feedback", "where", "install"):
|
||||
@@ -101,6 +105,24 @@ def test_init_and_auth_repair_normalize_to_existing_handlers(monkeypatch):
|
||||
]
|
||||
|
||||
|
||||
def test_reindex_routes_to_worker_and_has_side_effect_free_help(monkeypatch, capsys):
|
||||
module = load_installer()
|
||||
calls = []
|
||||
monkeypatch.setattr(
|
||||
module,
|
||||
"run_wrapper",
|
||||
lambda command, args: calls.append((command, args)) or 0,
|
||||
)
|
||||
|
||||
assert module.main(["nap", "reindex", "--limit", "50"]) == 0
|
||||
assert calls == [("reindex", ["--limit", "50"])]
|
||||
|
||||
calls.clear()
|
||||
assert module.main(["nap", "reindex", "--help"]) == 0
|
||||
assert calls == []
|
||||
assert "Usage: nap reindex [--limit N]" in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_update_does_not_overwrite_repository_source(monkeypatch, capsys):
|
||||
module = load_installer()
|
||||
calls = []
|
||||
@@ -171,6 +193,10 @@ def test_doctor_reports_presence_without_exposing_values(tmp_path, monkeypatch):
|
||||
},
|
||||
)
|
||||
|
||||
from napseer_credentials import CredentialStore
|
||||
store = CredentialStore(state_dir)
|
||||
store.migrate()
|
||||
auth_path = store.project
|
||||
result = module.doctor_status()
|
||||
serialized = json.dumps(result)
|
||||
|
||||
@@ -374,7 +400,7 @@ def fake_bundle(module, revision="a" * 40, broken_name=None):
|
||||
"version": module.CLI_RELEASE_VERSION,
|
||||
"bytes": len(content.encode("utf-8")),
|
||||
"sha256": digest,
|
||||
"source_repo": "https://github.com/napseer/nap-cli",
|
||||
"source_repo": "https://gitea.powc-r.com/napseer/nap-cli",
|
||||
"source_revision": revision,
|
||||
"source_revision_status": "resolved",
|
||||
"contract_version": module.CLI_DISTRIBUTION_CONTRACT_VERSION,
|
||||
@@ -397,7 +423,7 @@ def fake_bundle(module, revision="a" * 40, broken_name=None):
|
||||
"minimum_supported": module.CLI_MINIMUM_CONTRACT_VERSION,
|
||||
},
|
||||
"source": {
|
||||
"repo": "https://github.com/napseer/nap-cli",
|
||||
"repo": "https://gitea.powc-r.com/napseer/nap-cli",
|
||||
"revision": revision,
|
||||
"revision_status": "resolved",
|
||||
},
|
||||
@@ -418,7 +444,7 @@ def test_bundle_install_stages_validates_and_activates_atomically(tmp_path, monk
|
||||
|
||||
result = module.install_assets()
|
||||
|
||||
assert result["cli_distribution"]["release_version"] == "0.2.2"
|
||||
assert result["cli_distribution"]["release_version"] == module.CLI_RELEASE_VERSION
|
||||
assert (install_dir / "current").is_symlink()
|
||||
assert (install_dir / "current" / "bundle-manifest.json").is_file()
|
||||
assert (install_dir / "current" / "napseer_mcp_server.py").is_file()
|
||||
@@ -475,10 +501,10 @@ def test_manifest_accepts_a_future_compatible_release():
|
||||
module = load_installer()
|
||||
manifest, _payloads = fake_bundle(module)
|
||||
manifest["release_version"] = "0.2.2"
|
||||
manifest["contract"]["current"] = "2026-08-02"
|
||||
manifest["contract"]["current"] = "2026-09-11"
|
||||
for item in manifest["items"]:
|
||||
item["version"] = "0.2.2"
|
||||
item["contract_version"] = "2026-08-02"
|
||||
item["contract_version"] = "2026-09-11"
|
||||
|
||||
items = module.validated_manifest_items(manifest)
|
||||
|
||||
@@ -488,11 +514,16 @@ def test_manifest_accepts_a_future_compatible_release():
|
||||
def test_manifest_rejects_a_minimum_contract_newer_than_installer():
|
||||
module = load_installer()
|
||||
manifest, _payloads = fake_bundle(module)
|
||||
manifest["contract"]["current"] = "2026-08-02"
|
||||
manifest["contract"]["minimum_supported"] = "2026-08-02"
|
||||
# One day past this installer's own contract, whatever it currently is.
|
||||
newer = (
|
||||
datetime.date.fromisoformat(module.CLI_DISTRIBUTION_CONTRACT_VERSION)
|
||||
+ datetime.timedelta(days=1)
|
||||
).isoformat()
|
||||
manifest["contract"]["current"] = newer
|
||||
manifest["contract"]["minimum_supported"] = newer
|
||||
for item in manifest["items"]:
|
||||
item["contract_version"] = "2026-08-02"
|
||||
item["minimum_contract_version"] = "2026-08-02"
|
||||
item["contract_version"] = newer
|
||||
item["minimum_contract_version"] = newer
|
||||
|
||||
try:
|
||||
module.validated_manifest_items(manifest)
|
||||
@@ -502,6 +533,31 @@ def test_manifest_rejects_a_minimum_contract_newer_than_installer():
|
||||
raise AssertionError("incompatible minimum contract must be rejected")
|
||||
|
||||
|
||||
def test_legacy_update_preserves_active_runtime_and_bootstrap_recovers(tmp_path, monkeypatch):
|
||||
module = load_installer()
|
||||
monkeypatch.setattr(module, "INSTALL_DIR", tmp_path / "install")
|
||||
monkeypatch.setattr(module, "BIN_DIR", tmp_path / "bin")
|
||||
manifest, payloads = fake_bundle(module)
|
||||
previous = module.INSTALL_DIR / "releases" / "previous"
|
||||
previous.mkdir(parents=True)
|
||||
(previous / "nap_install.py").write_text("# existing release\n")
|
||||
(module.INSTALL_DIR / "current").symlink_to(previous, target_is_directory=True)
|
||||
credentials = module.INSTALL_DIR / "auth.json"
|
||||
credentials.write_text('{"refresh_token":"synthetic-preserved"}')
|
||||
monkeypatch.setattr(module, "bundle_manifest", lambda: manifest)
|
||||
monkeypatch.setattr(module, "fetch_script", lambda name: payloads[name])
|
||||
current_contract = module.CLI_DISTRIBUTION_CONTRACT_VERSION
|
||||
monkeypatch.setattr(module, "CLI_DISTRIBUTION_CONTRACT_VERSION", "2026-08-01")
|
||||
with pytest.raises(RuntimeError, match="newer bootstrap installer"):
|
||||
module.main(["nap", "update"])
|
||||
assert (module.INSTALL_DIR / "current").resolve() == previous
|
||||
assert list((module.INSTALL_DIR / "releases").iterdir()) == [previous]
|
||||
monkeypatch.setattr(module, "CLI_DISTRIBUTION_CONTRACT_VERSION", current_contract)
|
||||
assert module.main(["nap_install.py", "mcp", "install"]) == 0
|
||||
assert (module.INSTALL_DIR / "current" / "napseer_credentials.py").is_file()
|
||||
assert credentials.read_text() == '{"refresh_token":"synthetic-preserved"}'
|
||||
|
||||
|
||||
def test_cli_log_compaction_preserves_live_writer_inode(tmp_path):
|
||||
module = load_installer()
|
||||
log_path = tmp_path / "gateway.log"
|
||||
|
||||
@@ -36,6 +36,22 @@ def expect_unknown(mod, argv, expected_fragment):
|
||||
raise AssertionError(f"expected RuntimeError for {argv}")
|
||||
|
||||
|
||||
def test_reindex_cli_rebuilds_with_bounded_page_size():
|
||||
mod = load_module()
|
||||
calls = []
|
||||
mod.reindex_project = lambda args: calls.append(dict(args)) or {
|
||||
"indexed": 7,
|
||||
"graph_complete": True,
|
||||
}
|
||||
|
||||
output = io.StringIO()
|
||||
with contextlib.redirect_stdout(output):
|
||||
mod.cli_main(["napseer_mcp_server.py", "reindex", "--limit", "50"])
|
||||
|
||||
assert calls == [{"limit": 50}]
|
||||
assert '"indexed": 7' in output.getvalue()
|
||||
|
||||
|
||||
def test_auth_login_rejects_history_expansion_tail_before_browser():
|
||||
mod = load_module()
|
||||
mod.operator_account_login = lambda _args: (_ for _ in ()).throw(
|
||||
@@ -172,6 +188,13 @@ def test_gateway_vault_aliases_rejected():
|
||||
expect_unknown(mod, args, "gateway vault action: list")
|
||||
|
||||
|
||||
def test_gateway_vault_unlock_verbs_point_to_setup_and_forget():
|
||||
"""The memory secret is provisioned once, not unlocked per session."""
|
||||
mod = load_module()
|
||||
for action in ("unlock", "lock", "provision", "remove", "reset"):
|
||||
expect_unknown(mod, ["napseer_mcp_server.py", "gateway", "vault", action], "nap gateway vault setup")
|
||||
|
||||
|
||||
def test_gateway_rotate_passphrase_aliases_rejected():
|
||||
mod = load_module()
|
||||
for args in [
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
"""Login in a checkout with a committed locator rebinds its project without a
|
||||
second, project-scoped consent, and never replaces a still-valid override."""
|
||||
import importlib.util
|
||||
import json
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import parse_qs
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parents[1] / "resources/scripts/napseer_mcp_server.py"
|
||||
sys.path.insert(0, str(SCRIPT.parent))
|
||||
from napseer_credentials import CredentialStore, read_json # noqa: E402
|
||||
|
||||
PROJECT_ID = "33333333-3333-3333-3333-333333333333"
|
||||
PROJECT = {"id": PROJECT_ID, "slug": "committed-project", "name": "Committed Project",
|
||||
"encryption_state": "standard"}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def provider():
|
||||
state = {"refresh": [], "project_reads": [], "token_status": 401}
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def reply(self, status, value):
|
||||
data = json.dumps(value).encode()
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
def do_GET(self):
|
||||
if self.path == f"/v1/projects/{PROJECT_ID}":
|
||||
state["project_reads"].append(self.headers.get("Authorization"))
|
||||
if self.headers.get("Authorization") == "Bearer account-access":
|
||||
return self.reply(200, PROJECT)
|
||||
self.reply(401, {"error": {"code": "unauthorized"}})
|
||||
|
||||
def do_POST(self):
|
||||
form = parse_qs(self.rfile.read(int(self.headers["Content-Length"])).decode())
|
||||
assert self.path == "/v1/oauth/token"
|
||||
state["refresh"].append(form.get("refresh_token"))
|
||||
if state["token_status"] == 503:
|
||||
return self.reply(503, {"error": {"code": "unavailable"}})
|
||||
self.reply(401, {"error": {"code": "invalid_grant"}})
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
try:
|
||||
yield f"http://127.0.0.1:{server.server_port}", state
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
def checkout(tmp_path, monkeypatch, url, project_credential):
|
||||
root = tmp_path / "checkout"
|
||||
(root / ".napseer").mkdir(parents=True)
|
||||
(root / ".napseer/project.json").write_text(json.dumps({
|
||||
"schema": "napseer.project.v1", "base_url": url,
|
||||
"project_id": PROJECT_ID, "project_slug": PROJECT["slug"]}) + "\n")
|
||||
monkeypatch.setenv("NAPSEER_PROJECT_ROOT", str(root))
|
||||
monkeypatch.setenv("NAPSEER_BASE_URL", url)
|
||||
monkeypatch.chdir(root)
|
||||
spec = importlib.util.spec_from_file_location(f"committed_login_{id(root)}", SCRIPT)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
store = module.credential_store()
|
||||
if project_credential is not None:
|
||||
store.login({"base_url": url, "account_mode": "operator_project", "oauth_client_id": "nap-cli",
|
||||
"project_id": PROJECT_ID, **project_credential}, project=True)
|
||||
oauth_calls = []
|
||||
|
||||
def consent(args):
|
||||
oauth_calls.append(args["flow"])
|
||||
assert args["flow"] == "operator_login", "a committed project must not need project consent"
|
||||
return {"api_base_url": url, "access_token": "account-access", "refresh_token": "account-refresh",
|
||||
"expires_in": 3600, "scope": args["scope"]}
|
||||
|
||||
module.oauth_loopback_authorize = consent
|
||||
return module, store, oauth_calls
|
||||
|
||||
|
||||
def test_login_replaces_revoked_project_grant_and_attaches_committed_project(tmp_path, monkeypatch, provider):
|
||||
url, state = provider
|
||||
module, store, oauth_calls = checkout(tmp_path, monkeypatch, url, {
|
||||
"token": "dead-access", "refresh_token": "dead-refresh",
|
||||
"token_expires_at": "2020-01-01T00:00:00Z", "refresh_expires_at": "2099-01-01T00:00:00Z"})
|
||||
|
||||
result = module.operator_account_login({"base_url": url})
|
||||
|
||||
assert oauth_calls == ["operator_login"]
|
||||
assert state["refresh"] == [["dead-refresh"]], "the dead grant is proven dead before it is replaced"
|
||||
assert result["replaced_project_credential"] == "refresh_rejected"
|
||||
assert result["credential_source"] == "user"
|
||||
assert result["project_id"] == PROJECT_ID
|
||||
assert store.selected()[1] == "user"
|
||||
assert read_json(store.selection)["project_id"] == PROJECT_ID
|
||||
assert state["project_reads"][-1] == "Bearer account-access"
|
||||
assert "dead-refresh" not in json.dumps(result)
|
||||
|
||||
|
||||
def test_login_keeps_valid_project_override(tmp_path, monkeypatch, provider):
|
||||
url, state = provider
|
||||
module, store, oauth_calls = checkout(tmp_path, monkeypatch, url, {
|
||||
"token": "live-access", "refresh_token": "live-refresh",
|
||||
"token_expires_at": "2099-01-01T00:00:00Z", "refresh_expires_at": "2099-01-01T00:00:00Z"})
|
||||
|
||||
result = module.operator_account_login({"base_url": url})
|
||||
|
||||
assert oauth_calls == ["operator_login"]
|
||||
assert result["credential_source"] == "project"
|
||||
assert "use-user" in result["message"]
|
||||
assert store.selected()[1] == "project"
|
||||
assert state["refresh"] == [] and state["project_reads"] == []
|
||||
|
||||
|
||||
def test_login_does_not_replace_override_when_refresh_outcome_is_unknown(tmp_path, monkeypatch, provider):
|
||||
url, state = provider
|
||||
state["token_status"] = 503
|
||||
module, store, _calls = checkout(tmp_path, monkeypatch, url, {
|
||||
"token": "dead-access", "refresh_token": "dead-refresh",
|
||||
"token_expires_at": "2020-01-01T00:00:00Z", "refresh_expires_at": "2099-01-01T00:00:00Z"})
|
||||
|
||||
result = module.operator_account_login({"base_url": url})
|
||||
|
||||
assert result["credential_source"] == "project"
|
||||
assert store.selected()[1] == "project"
|
||||
|
||||
|
||||
def test_login_without_override_attaches_committed_project(tmp_path, monkeypatch, provider):
|
||||
url, state = provider
|
||||
module, store, oauth_calls = checkout(tmp_path, monkeypatch, url, None)
|
||||
|
||||
result = module.operator_account_login({"base_url": url})
|
||||
|
||||
assert oauth_calls == ["operator_login"]
|
||||
assert result["project_id"] == PROJECT_ID
|
||||
assert read_json(store.selection)["project_id"] == PROJECT_ID
|
||||
|
||||
|
||||
def test_attach_uses_account_login_when_project_grant_was_logged_out(tmp_path, monkeypatch, provider):
|
||||
url, state = provider
|
||||
module, store, oauth_calls = checkout(tmp_path, monkeypatch, url, None)
|
||||
store.login({"base_url": url, "token": "account-access", "refresh_token": "account-refresh",
|
||||
"account_mode": "operator_account", "oauth_client_id": "nap-cli",
|
||||
"token_expires_at": "2099-01-01T00:00:00Z"})
|
||||
store.login({"base_url": url, "logged_out": True}, project=True)
|
||||
|
||||
result = module.operator_project_attach({})
|
||||
|
||||
assert oauth_calls == [], "no browser consent is needed for a committed project"
|
||||
assert result["mode"] == "operator_account"
|
||||
assert result["project_id"] == PROJECT_ID
|
||||
assert store.selected()[1] == "user"
|
||||
|
||||
|
||||
def test_project_grant_rejection_explains_reattach(tmp_path, monkeypatch, provider):
|
||||
url, _state = provider
|
||||
module, _store, _calls = checkout(tmp_path, monkeypatch, url, {
|
||||
"token": "dead-access", "refresh_token": "dead-refresh",
|
||||
"token_expires_at": "2020-01-01T00:00:00Z", "refresh_expires_at": "2099-01-01T00:00:00Z"})
|
||||
module.refresh_public_auth_state()
|
||||
with pytest.raises(module.SafeToolError) as error:
|
||||
module.renew_auth()
|
||||
assert "project login" in error.value.safe_message
|
||||
assert ".napseer/project.json" in error.value.safe_message
|
||||
@@ -0,0 +1,152 @@
|
||||
"""AES-256-GCM backend selection: known answers, interop and tamper detection.
|
||||
|
||||
Both the optional native backend (`cryptography`) and the stdlib-only
|
||||
pure-Python backend run the same vectors through the public
|
||||
aes_gcm_encrypt/aes_gcm_decrypt API used by node and bundle encryption.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import pathlib
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = pathlib.Path(__file__).resolve().parents[1] / "resources/scripts/napseer_mcp_server.py"
|
||||
|
||||
h = bytes.fromhex
|
||||
GCM_KEY = h("feffe9928665731c6d6a8f9467308308feffe9928665731c6d6a8f9467308308")
|
||||
GCM_IV = h("cafebabefacedbaddecaf888")
|
||||
GCM_P = h(
|
||||
"d9313225f88406e5a55909c5aff5269a86a7a9531534f7da2e4c303d8a318a72"
|
||||
"1c3c0c95956809532fcf0e2449a6b525b16aedf5aa0de657ba637b391aafd255"
|
||||
)
|
||||
GCM_C = h(
|
||||
"522dc1f099567d07f47f37a32a84427d643a8cdcbfe5c0c97598a2bd2555d1aa"
|
||||
"8cb08e48590dbb3da7b08b1056828838c5f61e6393ba7a0abcc9f662898015ad"
|
||||
)
|
||||
# The GCM specification (McGrew & Viega) test cases 13-16, AES-256.
|
||||
KNOWN_ANSWERS = [
|
||||
("tc13", bytes(32), bytes(12), b"", b"", b"", h("530f8afbc74536b9a963b4f1c4cb738b")),
|
||||
("tc14", bytes(32), bytes(12), b"", bytes(16), h("cea7403d4d606b6e074ec5d3baf39d18"),
|
||||
h("d0d1c8a799996bf0265b98b5d48ab919")),
|
||||
("tc15", GCM_KEY, GCM_IV, b"", GCM_P, GCM_C, h("b094dac5d93471bdec1a502270e3cc6c")),
|
||||
("tc16", GCM_KEY, GCM_IV, h("feedfacedeadbeeffeedfacedeadbeefabaddad2"), GCM_P[:60], GCM_C[:60],
|
||||
h("76fc6ece0f4e1768cddf8853bb2d551b")),
|
||||
]
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def mod():
|
||||
spec = importlib.util.spec_from_file_location("crypto_backend_test_worker", SCRIPT)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
import sys
|
||||
sys.path.insert(0, str(SCRIPT.parent))
|
||||
spec.loader.exec_module(module)
|
||||
yield module
|
||||
module.select_crypto_backend()
|
||||
|
||||
|
||||
def backends(mod):
|
||||
names = [mod.CRYPTO_BACKEND_PURE]
|
||||
if mod.native_crypto_available():
|
||||
names.append(mod.CRYPTO_BACKEND_NATIVE)
|
||||
return names
|
||||
|
||||
|
||||
def use(mod, name):
|
||||
assert mod.select_crypto_backend(name) == name
|
||||
return name
|
||||
|
||||
|
||||
def test_native_backend_is_selected_when_cryptography_is_importable(mod, monkeypatch):
|
||||
pytest.importorskip("cryptography")
|
||||
monkeypatch.delenv("NAPSEER_CRYPTO_BACKEND", raising=False)
|
||||
assert mod.select_crypto_backend() == mod.CRYPTO_BACKEND_NATIVE
|
||||
monkeypatch.setenv("NAPSEER_CRYPTO_BACKEND", "pure_python")
|
||||
assert mod.select_crypto_backend() == mod.CRYPTO_BACKEND_PURE
|
||||
assert mod.crypto_backend_status()["crypto_backend"] == "pure_python"
|
||||
|
||||
|
||||
def test_pure_backend_is_used_when_native_is_unavailable(mod, monkeypatch):
|
||||
monkeypatch.setattr(mod, "_NativeAESGCM", None)
|
||||
monkeypatch.delenv("NAPSEER_CRYPTO_BACKEND", raising=False)
|
||||
try:
|
||||
assert mod.select_crypto_backend() == mod.CRYPTO_BACKEND_PURE
|
||||
status = mod.crypto_backend_status()
|
||||
assert status["native_crypto_available"] is False
|
||||
assert "cryptography" in status["crypto_backend_note"]
|
||||
finally:
|
||||
monkeypatch.undo()
|
||||
mod.select_crypto_backend()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("case", KNOWN_ANSWERS, ids=[case[0] for case in KNOWN_ANSWERS])
|
||||
def test_known_answer_vectors_through_every_backend(mod, case):
|
||||
_name, key, iv, aad, plaintext, ciphertext, tag = case
|
||||
for backend in backends(mod):
|
||||
use(mod, backend)
|
||||
assert mod.aes_gcm_encrypt(key, iv, aad, plaintext) == ciphertext + tag, backend
|
||||
assert mod.aes_gcm_decrypt(key, iv, aad, ciphertext + tag) == plaintext, backend
|
||||
mod.select_crypto_backend()
|
||||
|
||||
|
||||
def test_cross_backend_interop(mod):
|
||||
pytest.importorskip("cryptography")
|
||||
key = bytes(range(32))
|
||||
nonce = bytes(range(12))
|
||||
aad = b"napseer interop aad"
|
||||
for size in (0, 1, 15, 16, 17, 1000):
|
||||
plaintext = bytes((index * 7) & 0xFF for index in range(size))
|
||||
use(mod, mod.CRYPTO_BACKEND_PURE)
|
||||
pure = mod.aes_gcm_encrypt(key, nonce, aad, plaintext)
|
||||
use(mod, mod.CRYPTO_BACKEND_NATIVE)
|
||||
native = mod.aes_gcm_encrypt(key, nonce, aad, plaintext)
|
||||
assert pure == native
|
||||
assert mod.aes_gcm_decrypt(key, nonce, aad, pure) == plaintext
|
||||
use(mod, mod.CRYPTO_BACKEND_PURE)
|
||||
assert mod.aes_gcm_decrypt(key, nonce, aad, native) == plaintext
|
||||
mod.select_crypto_backend()
|
||||
|
||||
|
||||
def test_tamper_detection_raises_the_same_error_in_every_backend(mod):
|
||||
key = bytes(range(32))
|
||||
nonce = bytes(12)
|
||||
for backend in backends(mod):
|
||||
use(mod, backend)
|
||||
packed = mod.aes_gcm_encrypt(key, nonce, b"aad", b"synthetic payload")
|
||||
tampered_body = bytes([packed[0] ^ 1]) + packed[1:]
|
||||
tampered_tag = packed[:-1] + bytes([packed[-1] ^ 0x80])
|
||||
wrong_key = bytes([key[0] ^ 1]) + key[1:]
|
||||
for label, attempt in (
|
||||
("body", lambda: mod.aes_gcm_decrypt(key, nonce, b"aad", tampered_body)),
|
||||
("tag", lambda: mod.aes_gcm_decrypt(key, nonce, b"aad", tampered_tag)),
|
||||
("aad", lambda: mod.aes_gcm_decrypt(key, nonce, b"other", packed)),
|
||||
("key", lambda: mod.aes_gcm_decrypt(wrong_key, nonce, b"aad", packed)),
|
||||
):
|
||||
with pytest.raises(RuntimeError, match="^AES-GCM authentication failed$"):
|
||||
attempt()
|
||||
|
||||
with pytest.raises(RuntimeError, match="missing tag"):
|
||||
mod.aes_gcm_decrypt(key, nonce, b"aad", b"short")
|
||||
with pytest.raises(RuntimeError, match="nonce must be 12 bytes"):
|
||||
mod.aes_gcm_encrypt(key, bytes(16), b"", b"x")
|
||||
with pytest.raises(RuntimeError, match="key must be 32 bytes"):
|
||||
mod.aes_gcm_encrypt(bytes(16), nonce, b"", b"x")
|
||||
mod.select_crypto_backend()
|
||||
|
||||
|
||||
def test_hkdf_sha256_matches_rfc5869_vectors(mod):
|
||||
# RFC 5869 Appendix A, test cases 1 and 3.
|
||||
okm = mod.hkdf_sha256(b"\x0b" * 22, bytes(range(13)), bytes(range(0xF0, 0xFA)), 42)
|
||||
assert okm.hex() == (
|
||||
"3cb25f25faacd57a90434f64d0362f2a2d2d0a90cf1a5a4c5db02d56ecc4c5bf34007208d5b887185865"
|
||||
)
|
||||
okm = mod.hkdf_sha256(b"\x0b" * 22, b"", b"", 42)
|
||||
assert okm.hex() == (
|
||||
"8da4e775a563c18f715f802a063c5a31b8a11f5c5ee1879ec3454e5f3c738d2d9d201395faa4b61a96c8"
|
||||
)
|
||||
|
||||
|
||||
def test_diagnostics_report_the_backend_without_key_material(mod):
|
||||
status = mod.crypto_backend_status()
|
||||
assert status["crypto_backend"] in {"native", "pure_python"}
|
||||
assert set(status) <= {"crypto_backend", "native_crypto_available", "crypto_backend_note"}
|
||||
@@ -0,0 +1,397 @@
|
||||
"""Effective node types and release-terminal statuses in the CLI wrapper.
|
||||
|
||||
Decision /decisions/2026-09-29-node-type-status-vocabulary: creates infer a
|
||||
type from the folder when the caller omits one, type filters match effective
|
||||
types (a generic `note` resolves from its folder), and active-only lists also
|
||||
exclude release-style terminal statuses. Older APIs do none of this, so the
|
||||
wrapper must behave the same on its own. Writes go through the real request
|
||||
path to a fake transport; discovery runs against a disposable HTTP provider
|
||||
and the real local index.
|
||||
"""
|
||||
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = pathlib.Path(__file__).resolve().parents[1] / "resources" / "scripts" / "napseer_mcp_server.py"
|
||||
|
||||
|
||||
def load_module(name):
|
||||
sys.path.insert(0, str(SCRIPT.parent))
|
||||
spec = importlib.util.spec_from_file_location(name, SCRIPT)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
# -- Writes -----------------------------------------------------------------
|
||||
|
||||
|
||||
class FakeResponse:
|
||||
def __init__(self, payload):
|
||||
self._body = json.dumps(payload).encode("utf-8")
|
||||
|
||||
def read(self, *_args):
|
||||
return self._body
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *_exc):
|
||||
return False
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def writer(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "all")
|
||||
mod = load_module("napseer_effective_types_write_test")
|
||||
mod.DEFAULT_PROJECT_ID = "project-1"
|
||||
mod.TOKEN = "synthetic-access"
|
||||
mod.TOKEN_EXPIRES_AT = "2099-01-01T00:00:00Z"
|
||||
mod.AUTH = {**mod.AUTH, "account_mode": "operator_account", "project_id": "project-1"}
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
mod.refresh_public_auth_state = lambda: mod.AUTH
|
||||
mod.project_memory_encryption_active = lambda _project_id: False
|
||||
mod.AUTH_DIR = tmp_path / "state"
|
||||
mod.INDEX_PATH = tmp_path / "state" / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = tmp_path / "state" / "index.lock"
|
||||
mod.resolve_project_id = lambda _args: "project-1"
|
||||
mod.acquire_project_lock = lambda args=None, **kwargs: {"id": "lock-1", "lease_token": "lease-1"}
|
||||
mod.cleanup_project_lock = lambda project_id, lock: None
|
||||
sent = []
|
||||
|
||||
def fake_urlopen(request, timeout=None):
|
||||
body = json.loads(request.data.decode("utf-8")) if request.data else None
|
||||
sent.append((request.get_method(), urlsplit(request.full_url).path, body))
|
||||
folder = (body or {}).get("folder_path") or "/"
|
||||
name = (body or {}).get("name") or "node"
|
||||
return FakeResponse({
|
||||
"id": f"node-{len(sent)}", "project_id": "project-1",
|
||||
"full_path": f"{folder.rstrip('/')}/{name}", "folder_path": folder, "name": name,
|
||||
# The fake server applies the historical default; the payload is what is asserted.
|
||||
"type": (body or {}).get("type") or "note",
|
||||
"content_text": (body or {}).get("content_text", ""), "metadata": {},
|
||||
"updated_at": "2026-09-29T12:00:00Z",
|
||||
})
|
||||
|
||||
mod.api_urlopen = fake_urlopen
|
||||
return mod, sent
|
||||
|
||||
|
||||
def created_payload(sent):
|
||||
posts = [body for method, path, body in sent if method == "POST" and path.endswith("/nodes")]
|
||||
assert len(posts) == 1, sent
|
||||
return posts[0]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("folder_path", "expected"),
|
||||
[
|
||||
("/plans", "plan"),
|
||||
("/plans/2026", "plan"),
|
||||
("/decisions", "decision"),
|
||||
("/product/decisions", "decision"),
|
||||
("/implementation-notes", "implementation_note"),
|
||||
("/deployments", "deployment"),
|
||||
("/runbooks", "runbook"),
|
||||
("/rules", "rule"),
|
||||
("/indexes", "index"),
|
||||
("/analysis", "analysis"),
|
||||
("/product/analysis", "analysis"),
|
||||
("/documentation/security", "documentation"),
|
||||
("/kanban/doing", "kanban_card"),
|
||||
# The server normalizes folder segments before storing; inference agrees.
|
||||
("/Implementation Notes/", "implementation_note"),
|
||||
],
|
||||
)
|
||||
def test_create_without_type_sends_the_folder_type(writer, folder_path, expected):
|
||||
mod, sent = writer
|
||||
mod.call_tool_impl("nap_create_node", {"folder_path": folder_path, "name": "record", "content_text": "body"})
|
||||
assert created_payload(sent)["type"] == expected
|
||||
|
||||
|
||||
@pytest.mark.parametrize("folder_path", ["/notes", "/plansx", "/product", "/services/plans"])
|
||||
def test_create_without_type_outside_typed_folders_leaves_the_server_default(writer, folder_path):
|
||||
mod, sent = writer
|
||||
mod.call_tool_impl("nap_create_node", {"folder_path": folder_path, "name": "record", "content_text": "body"})
|
||||
assert "type" not in created_payload(sent)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("given", "expected"),
|
||||
[
|
||||
("implementation-note", "implementation_note"),
|
||||
("Kanban Card", "kanban_card"),
|
||||
("product-spec", "product_spec"),
|
||||
("decision", "decision"),
|
||||
# An explicit generic note stays a note even inside a typed folder.
|
||||
("note", "note"),
|
||||
# Unknown custom types are kept exactly as given.
|
||||
("security-documentation", "security-documentation"),
|
||||
("Custom Type", "Custom Type"),
|
||||
],
|
||||
)
|
||||
def test_create_with_explicit_type_only_canonicalizes_known_spellings(writer, given, expected):
|
||||
mod, sent = writer
|
||||
mod.call_tool_impl(
|
||||
"nap_create_node",
|
||||
{"folder_path": "/plans", "name": "record", "content_text": "body", "type": given},
|
||||
)
|
||||
assert created_payload(sent)["type"] == expected
|
||||
|
||||
|
||||
def test_create_node_schema_advertises_optional_type(writer):
|
||||
mod, _sent = writer
|
||||
tool = next(tool for tool in mod.tools() if tool["name"] == "nap_create_node")
|
||||
schema = tool["inputSchema"]
|
||||
assert schema["properties"]["type"]["type"] == "string"
|
||||
assert "type" not in schema.get("required", [])
|
||||
|
||||
|
||||
def test_bulk_creates_infer_types_per_item(writer):
|
||||
mod, sent = writer
|
||||
mod.try_get_node_by_path = lambda path, allow_agent=False: None
|
||||
mod.call_tool_impl("nap_bulk", {"nodes": [
|
||||
{"path": "/decisions/choose-store", "content_text": "a"},
|
||||
{"path": "/scratch/idea", "content_text": "b"},
|
||||
{"path": "/runbooks/restore", "content_text": "c", "type": "product-spec"},
|
||||
{"path": "/plans/custom", "content_text": "d", "type": "Custom Type"},
|
||||
]})
|
||||
posts = [body for method, path, body in sent if method == "POST" and path.endswith("/nodes")]
|
||||
assert [body.get("type") for body in posts] == ["decision", None, "product_spec", "Custom Type"]
|
||||
assert "type" not in posts[1]
|
||||
|
||||
|
||||
def test_bulk_update_without_type_does_not_retype_the_stored_node(writer):
|
||||
mod, sent = writer
|
||||
existing = {"id": "node-9", "project_id": "project-1", "full_path": "/plans/legacy",
|
||||
"folder_path": "/plans", "name": "legacy", "type": "note", "content_text": "old"}
|
||||
mod.get_node_by_id = lambda args: existing
|
||||
mod.call_tool_impl("nap_bulk", {"nodes": [{"node_id": "node-9", "content_text": "new"}]})
|
||||
patches = [body for method, path, body in sent if method == "PATCH"]
|
||||
assert patches and "type" not in patches[0]
|
||||
|
||||
|
||||
# -- Local discovery ---------------------------------------------------------
|
||||
|
||||
|
||||
def stored_node(folder, name, node_type, *, minute, status=None, content=None):
|
||||
stamp = f"2026-09-01T12:{minute:02d}:00Z"
|
||||
return {
|
||||
"id": name, "project_id": "project-1", "full_path": f"{folder}/{name}",
|
||||
"folder_path": folder, "name": name, "type": node_type,
|
||||
"metadata": {"status": status} if status else {}, "tags": [], "aliases": [], "links": [],
|
||||
"content_text": content or f"{name} vocabularyterm", "updated_at": stamp, "archived_at": None,
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def local(tmp_path, monkeypatch):
|
||||
nodes = {}
|
||||
events = []
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def reply(self, status, payload):
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps(payload).encode())
|
||||
|
||||
def do_GET(self):
|
||||
url = urlsplit(self.path)
|
||||
query = parse_qs(url.query)
|
||||
if url.path.endswith("/changes"):
|
||||
after = query.get("after", ["head"])[0]
|
||||
start = len(events) if after == "head" else int(after[1:])
|
||||
return self.reply(200, {"items": [], "next_cursor": f"c{start}", "has_more": False})
|
||||
if url.path.endswith("/nodes"):
|
||||
rows = sorted((copy.deepcopy(n) for n in nodes.values()),
|
||||
key=lambda n: (n["updated_at"], n["id"]), reverse=True)
|
||||
return self.reply(200, {"items": rows, "next_cursor": None})
|
||||
node_id = url.path.rsplit("/", 1)[-1]
|
||||
if node_id not in nodes:
|
||||
return self.reply(404, {"error": {"code": "not_found", "message": "missing"}})
|
||||
return self.reply(200, copy.deepcopy(nodes[node_id]))
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "all")
|
||||
mod = load_module("napseer_effective_types_local_test")
|
||||
mod.AUTH_DIR = tmp_path / "project-state"
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.lock"
|
||||
mod.BASE_URL = f"http://127.0.0.1:{server.server_port}"
|
||||
mod.TOKEN = "disposable-test-token"
|
||||
mod.AUTH = {"account_id": "test-account"}
|
||||
mod.DEFAULT_PROJECT_ID = "project-1"
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
mod.require_unlocked = lambda _name: None
|
||||
mod.resolve_project_id = lambda args: args.get("project_id", "project-1")
|
||||
mod.project_encryption_state = lambda _project_id: "standard"
|
||||
|
||||
def seed(*items):
|
||||
for item in items:
|
||||
nodes[item["id"]] = copy.deepcopy(item)
|
||||
events.append(item["id"])
|
||||
assert mod.reindex_project({})["graph_complete"] is True
|
||||
|
||||
try:
|
||||
yield mod, seed
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
TYPED_FIXTURE = [
|
||||
stored_node("/plans", "legacy-plan", "note", minute=1),
|
||||
stored_node("/plans/2026", "nested-plan", "note", minute=2),
|
||||
stored_node("/plans", "typed-plan", "plan", minute=3),
|
||||
stored_node("/notes", "scratch", "note", minute=4),
|
||||
stored_node("/implementation-notes", "dashed-note", "implementation-note", minute=5),
|
||||
stored_node("/implementation-notes", "generic-note", "note", minute=6),
|
||||
stored_node("/services", "misplaced-plan", "Plan", minute=7),
|
||||
stored_node("/services", "custom", "security-documentation", minute=8),
|
||||
# A differently spelled generic note still resolves from its folder.
|
||||
stored_node("/decisions", "capital-note", "Note", minute=9),
|
||||
]
|
||||
|
||||
|
||||
def discover(mod, **args):
|
||||
return mod.call_tool_impl("nap_discover", {"mode": "local", "limit": 50, **args})
|
||||
|
||||
|
||||
@pytest.mark.parametrize("q", [None, "vocabularyterm"])
|
||||
def test_local_type_filter_matches_effective_types(local, q):
|
||||
mod, seed = local
|
||||
seed(*TYPED_FIXTURE)
|
||||
extra = {"q": q} if q else {}
|
||||
|
||||
plans = discover(mod, type="plan", **extra)
|
||||
by_id = {item["node_id"]: item for item in plans["items"]}
|
||||
assert set(by_id) == {"legacy-plan", "nested-plan", "typed-plan", "misplaced-plan"}
|
||||
# The stored type is reported unchanged; effective_type is added when it differs.
|
||||
assert (by_id["legacy-plan"]["type"], by_id["legacy-plan"]["effective_type"]) == ("note", "plan")
|
||||
assert (by_id["nested-plan"]["type"], by_id["nested-plan"]["effective_type"]) == ("note", "plan")
|
||||
assert (by_id["misplaced-plan"]["type"], by_id["misplaced-plan"]["effective_type"]) == ("Plan", "plan")
|
||||
assert by_id["typed-plan"]["type"] == "plan"
|
||||
assert "effective_type" not in by_id["typed-plan"]
|
||||
|
||||
notes = discover(mod, type="implementation_note", **extra)
|
||||
by_id = {item["node_id"]: item for item in notes["items"]}
|
||||
assert set(by_id) == {"dashed-note", "generic-note"}
|
||||
assert by_id["dashed-note"]["type"] == "implementation-note"
|
||||
assert {item["effective_type"] for item in notes["items"]} == {"implementation_note"}
|
||||
|
||||
# A spelling variant in the filter canonicalizes the same way.
|
||||
dashed = discover(mod, type="implementation-note", **extra)
|
||||
assert {item["node_id"] for item in dashed["items"]} == {"dashed-note", "generic-note"}
|
||||
|
||||
decisions = discover(mod, type="decision", **extra)
|
||||
assert [item["node_id"] for item in decisions["items"]] == ["capital-note"]
|
||||
assert (decisions["items"][0]["type"], decisions["items"][0]["effective_type"]) == ("Note", "decision")
|
||||
|
||||
custom = discover(mod, type="security_documentation", **extra)
|
||||
assert [item["node_id"] for item in custom["items"]] == ["custom"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("q", [None, "vocabularyterm"])
|
||||
@pytest.mark.parametrize("requested", ["note", "Note", " NOTE "])
|
||||
def test_local_note_filter_is_effective_type_only(local, q, requested):
|
||||
mod, seed = local
|
||||
seed(*TYPED_FIXTURE)
|
||||
extra = {"q": q} if q else {}
|
||||
# Only the note under /notes is a note; notes under /plans, /decisions and
|
||||
# /implementation-notes take their folder's type and are excluded.
|
||||
notes = discover(mod, type=requested, **extra)
|
||||
assert [item["node_id"] for item in notes["items"]] == ["scratch"]
|
||||
assert "effective_type" not in notes["items"][0]
|
||||
|
||||
|
||||
def test_unfiltered_local_rows_carry_effective_type_and_facets(local):
|
||||
mod, seed = local
|
||||
seed(*TYPED_FIXTURE)
|
||||
result = discover(mod, view="paths", facets=["type", "effective_type"])
|
||||
by_id = {item["node_id"]: item for item in result["items"]}
|
||||
assert by_id["legacy-plan"]["type"] == "note"
|
||||
assert by_id["legacy-plan"]["effective_type"] == "plan"
|
||||
assert "effective_type" not in by_id["scratch"]
|
||||
assert by_id["custom"]["effective_type"] == "security_documentation"
|
||||
counts = result["facets"]["counts"]
|
||||
# The stored-type facet keeps its historical meaning.
|
||||
assert {"value": "note", "count": 4} in counts["type"]
|
||||
assert {"value": "plan", "count": 4} in counts["effective_type"]
|
||||
assert {"value": "implementation_note", "count": 2} in counts["effective_type"]
|
||||
|
||||
|
||||
RELEASE_TERMINAL = ["deployed", "released", "released_verified_production", "folded"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("q", [None, "vocabularyterm"])
|
||||
def test_local_active_only_excludes_release_terminal_statuses(local, q):
|
||||
mod, seed = local
|
||||
seed(
|
||||
stored_node("/deployments", "live", "note", minute=1, status="active"),
|
||||
*[
|
||||
stored_node("/deployments", f"release-{status}", "note", minute=2 + index, status=status)
|
||||
for index, status in enumerate(RELEASE_TERMINAL)
|
||||
],
|
||||
stored_node("/plans", "finished", "note", minute=9, status="completed"),
|
||||
)
|
||||
extra = {"q": q} if q else {}
|
||||
active = discover(mod, active_only=True, **extra)
|
||||
assert [item["node_id"] for item in active["items"]] == ["live"]
|
||||
# Without active_only the release records remain discoverable.
|
||||
everything = discover(mod, **extra)
|
||||
assert {f"release-{status}" for status in RELEASE_TERMINAL} <= {
|
||||
item["node_id"] for item in everything["items"]
|
||||
}
|
||||
|
||||
|
||||
def test_server_rows_are_post_filtered_on_effective_type(local):
|
||||
mod, _seed = local
|
||||
# An older server matches stored types only, so type=note returns every
|
||||
# stored note; the wrapper keeps just the effective notes.
|
||||
rows = [
|
||||
{"id": "under-plans", "full_path": "/plans/legacy", "folder_path": "/plans", "type": "note"},
|
||||
{"id": "under-decisions", "full_path": "/decisions/choice", "type": "Note"},
|
||||
{"id": "under-notes", "full_path": "/notes/scratch", "folder_path": "/notes", "type": "note"},
|
||||
]
|
||||
requests = []
|
||||
|
||||
def fake_request(method, path, payload=None, **kwargs):
|
||||
requests.append(path)
|
||||
return {"items": copy.deepcopy(rows), "next_cursor": None}
|
||||
|
||||
mod.request_json = fake_request
|
||||
notes = mod.list_project_nodes({"type": "note", "view": "paths"})
|
||||
assert "type=note" in requests[-1]
|
||||
assert [item["node_id"] for item in notes["items"]] == ["under-notes"]
|
||||
|
||||
rows[:] = [{"id": "typed", "full_path": "/plans/typed", "type": "plan"},
|
||||
{"id": "legacy", "full_path": "/plans/legacy", "type": "note"}]
|
||||
plans = mod.list_project_nodes({"type": "plan", "view": "paths"})
|
||||
assert [item["node_id"] for item in plans["items"]] == ["typed", "legacy"]
|
||||
|
||||
|
||||
def test_active_only_filter_on_server_rows_excludes_release_statuses(local):
|
||||
mod, _seed = local
|
||||
rows = [{"id": status, "metadata": {"status": status}} for status in [*RELEASE_TERMINAL, "active", "done"]]
|
||||
filters = mod.status_filter_from_args({"active_only": True})
|
||||
# "done" keeps its existing CLI handling (already excluded).
|
||||
assert [row["id"] for row in mod.apply_status_filters(rows, filters)] == ["active"]
|
||||
explicit = mod.status_filter_from_args({"status": "deployed"})
|
||||
assert explicit["warnings"] == []
|
||||
@@ -15,6 +15,8 @@ def load_module():
|
||||
|
||||
def test_renew_auth_serializes_rotating_refresh_tokens():
|
||||
mod = load_module()
|
||||
mod.TOKEN = "synthetic-configured-access"
|
||||
mod.REFRESH_TOKEN = "synthetic-configured-refresh"
|
||||
active = 0
|
||||
maximum = 0
|
||||
guard = threading.Lock()
|
||||
|
||||
@@ -15,6 +15,8 @@ def load_module():
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
module.load_project_locator = lambda: None
|
||||
module.require_project_locator_service = lambda _locator: None
|
||||
return module
|
||||
|
||||
|
||||
@@ -53,7 +55,7 @@ def run():
|
||||
mod.DEFAULT_PROJECT_ID = None
|
||||
assert mod.resolve_project_id({}) == "project-1"
|
||||
|
||||
current = {}
|
||||
current = {"token": "configured-account", "account_mode": "operator_account"}
|
||||
mod.DEFAULT_PROJECT_ID = "stale-project"
|
||||
bootstrap_calls = []
|
||||
|
||||
@@ -73,7 +75,7 @@ def run():
|
||||
}
|
||||
]
|
||||
|
||||
current = {}
|
||||
current = {"token": "configured-account", "account_mode": "operator_account"}
|
||||
mod.DEFAULT_PROJECT_ID = None
|
||||
|
||||
def limited_project(_args):
|
||||
|
||||
@@ -0,0 +1,303 @@
|
||||
import importlib.util
|
||||
import pathlib
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def load_module():
|
||||
script_path = (
|
||||
pathlib.Path(__file__).resolve().parents[1]
|
||||
/ "resources"
|
||||
/ "scripts"
|
||||
/ "napseer_mcp_server.py"
|
||||
)
|
||||
sys.path.insert(0, str(script_path.parent))
|
||||
spec = importlib.util.spec_from_file_location("napseer_mcp_batch_test", script_path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
module.DEFAULT_PROJECT_ID = "project-1"
|
||||
module.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
return module
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mod(monkeypatch):
|
||||
monkeypatch.delenv("NAPSEER_TOOL_PROFILES", raising=False)
|
||||
return load_module()
|
||||
|
||||
|
||||
def action(action_id, tool="nap_node_get", *, depends_on=None, arguments=None):
|
||||
return {
|
||||
"id": action_id,
|
||||
"tool": tool,
|
||||
"reason": f"Run {action_id} for batch verification",
|
||||
"arguments": arguments or {},
|
||||
"depends_on": depends_on or [],
|
||||
}
|
||||
|
||||
|
||||
def test_batch_is_a_conservatively_annotated_core_tool(mod):
|
||||
tool = next(item for item in mod.tools() if item["name"] == "nap_batch")
|
||||
|
||||
assert tool["annotations"] == {
|
||||
"title": "nap_batch",
|
||||
"readOnlyHint": False,
|
||||
"destructiveHint": True,
|
||||
"idempotentHint": False,
|
||||
"openWorldHint": False,
|
||||
}
|
||||
assert tool["napseer"]["lock_policy"] == "per-action-mutation-barrier"
|
||||
assert tool["napseer"]["idempotency"] == "mixed-no-replay"
|
||||
schema = tool["inputSchema"]
|
||||
assert schema["properties"]["actions"]["maxItems"] == 32
|
||||
assert schema["properties"]["max_parallel"]["maximum"] == 8
|
||||
assert set(schema["properties"]["actions"]["items"]["required"]) == {
|
||||
"id", "tool", "reason", "arguments",
|
||||
}
|
||||
|
||||
|
||||
def test_independent_reads_overlap_and_results_preserve_input_order(mod, monkeypatch):
|
||||
state_lock = threading.Lock()
|
||||
gate = threading.Barrier(2)
|
||||
active = 0
|
||||
max_active = 0
|
||||
|
||||
def fake_call(name, arguments):
|
||||
nonlocal active, max_active
|
||||
with state_lock:
|
||||
active += 1
|
||||
max_active = max(max_active, active)
|
||||
gate.wait(timeout=1)
|
||||
if arguments["value"] == "first":
|
||||
time.sleep(0.04)
|
||||
with state_lock:
|
||||
active -= 1
|
||||
return {"tool": name, "value": arguments["value"]}
|
||||
|
||||
monkeypatch.setattr(mod, "call_tool_impl", fake_call)
|
||||
result = mod.execute_batch({
|
||||
"max_parallel": 2,
|
||||
"actions": [
|
||||
action("first", arguments={"value": "first"}),
|
||||
action("second", tool="nap_context", arguments={"value": "second"}),
|
||||
],
|
||||
})
|
||||
|
||||
assert result["ok"] is True
|
||||
assert result["failure_policy"] == "continue_on_error"
|
||||
assert result["parallel_read_groups"] == 1
|
||||
assert max_active == 2
|
||||
assert [item["id"] for item in result["results"]] == ["first", "second"]
|
||||
assert [item["result"]["value"] for item in result["results"]] == ["first", "second"]
|
||||
|
||||
|
||||
def test_mutation_is_an_exclusive_array_order_barrier(mod, monkeypatch):
|
||||
state_lock = threading.Lock()
|
||||
active_reads = 0
|
||||
active_writes = 0
|
||||
max_reads = 0
|
||||
violations = []
|
||||
completed = []
|
||||
|
||||
def fake_call(name, arguments):
|
||||
nonlocal active_reads, active_writes, max_reads
|
||||
is_read = mod.tool_side_effect(name) == "none"
|
||||
with state_lock:
|
||||
if is_read:
|
||||
if active_writes:
|
||||
violations.append("read overlapped write")
|
||||
active_reads += 1
|
||||
max_reads = max(max_reads, active_reads)
|
||||
else:
|
||||
if active_reads or active_writes:
|
||||
violations.append("write was not exclusive")
|
||||
active_writes += 1
|
||||
time.sleep(0.04)
|
||||
with state_lock:
|
||||
if is_read:
|
||||
active_reads -= 1
|
||||
else:
|
||||
active_writes -= 1
|
||||
completed.append(arguments["value"])
|
||||
return {"value": arguments["value"]}
|
||||
|
||||
monkeypatch.setattr(mod, "call_tool_impl", fake_call)
|
||||
result = mod.execute_batch({
|
||||
"actions": [
|
||||
action("read-a", arguments={"value": "read-a"}),
|
||||
action("read-b", tool="nap_context", arguments={"value": "read-b"}),
|
||||
action("write", tool="nap_create_node", arguments={"value": "write"}),
|
||||
action("read-c", arguments={"value": "read-c"}),
|
||||
action("read-d", tool="nap_context", arguments={"value": "read-d"}),
|
||||
],
|
||||
})
|
||||
|
||||
assert result["ok"] is True
|
||||
assert result["failure_policy"] == "stop_on_error"
|
||||
assert max_reads == 2
|
||||
assert violations == []
|
||||
assert completed.index("write") > max(completed.index("read-a"), completed.index("read-b"))
|
||||
assert completed.index("write") < min(completed.index("read-c"), completed.index("read-d"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("actions", "error_code"),
|
||||
[
|
||||
(
|
||||
[
|
||||
action("a", depends_on=["b"]),
|
||||
action("b", tool="nap_context", depends_on=["a"]),
|
||||
],
|
||||
"batch_dependency_cycle",
|
||||
),
|
||||
([action("a", depends_on=["missing"])], "unknown_batch_dependency"),
|
||||
([action("a", tool="nap_gateway_status")], "batch_tool_not_allowed"),
|
||||
([action("a", tool="nap_batch")], "batch_tool_not_allowed"),
|
||||
],
|
||||
)
|
||||
def test_invalid_graphs_and_capabilities_fail_before_execution(
|
||||
mod, monkeypatch, actions, error_code
|
||||
):
|
||||
calls = []
|
||||
monkeypatch.setattr(mod, "call_tool_impl", lambda *args: calls.append(args))
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as error:
|
||||
mod.execute_batch({"actions": actions})
|
||||
|
||||
assert error.value.code == error_code
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_dependency_failure_skips_dependents_but_runs_independent_reads(
|
||||
mod, monkeypatch
|
||||
):
|
||||
calls = []
|
||||
|
||||
def fake_call(_name, arguments):
|
||||
calls.append(arguments["value"])
|
||||
if arguments["value"] == "fails":
|
||||
raise mod.SafeToolError("fixture_failure", "fixture failed safely")
|
||||
return {"value": arguments["value"]}
|
||||
|
||||
monkeypatch.setattr(mod, "call_tool_impl", fake_call)
|
||||
result = mod.execute_batch({
|
||||
"failure_policy": "continue_on_error",
|
||||
"actions": [
|
||||
action("fails", arguments={"value": "fails"}),
|
||||
action("dependent", depends_on=["fails"], arguments={"value": "dependent"}),
|
||||
action("independent", tool="nap_context", arguments={"value": "independent"}),
|
||||
],
|
||||
})
|
||||
|
||||
assert result["ok"] is False
|
||||
assert result["stopped_at"] is None
|
||||
assert [item["status"] for item in result["results"]] == [
|
||||
"failed", "skipped", "succeeded",
|
||||
]
|
||||
assert set(calls) == {"fails", "independent"}
|
||||
|
||||
|
||||
def test_stop_on_error_does_not_schedule_a_later_read_wave(mod, monkeypatch):
|
||||
calls = []
|
||||
|
||||
def fake_call(_name, arguments):
|
||||
calls.append(arguments["value"])
|
||||
raise mod.SafeToolError("fixture_failure", "fixture failed safely")
|
||||
|
||||
monkeypatch.setattr(mod, "call_tool_impl", fake_call)
|
||||
result = mod.execute_batch({
|
||||
"failure_policy": "stop_on_error",
|
||||
"max_parallel": 1,
|
||||
"actions": [
|
||||
action("first", arguments={"value": "first"}),
|
||||
action("second", tool="nap_context", arguments={"value": "second"}),
|
||||
],
|
||||
})
|
||||
|
||||
assert calls == ["first"]
|
||||
assert result["stopped_at"] == "first"
|
||||
assert [item["status"] for item in result["results"]] == ["failed", "skipped"]
|
||||
|
||||
|
||||
def test_dry_run_validates_without_invoking_actions(mod, monkeypatch):
|
||||
calls = []
|
||||
monkeypatch.setattr(mod, "call_tool_impl", lambda *args: calls.append(args))
|
||||
|
||||
result = mod.execute_batch({
|
||||
"dry_run": True,
|
||||
"actions": [
|
||||
action("read", arguments={"node_id": "node-1"}),
|
||||
action("write", tool="nap_create_node", arguments={"name": "note"}),
|
||||
],
|
||||
})
|
||||
|
||||
assert calls == []
|
||||
assert result["failure_policy"] == "dry_run"
|
||||
assert result["contains_mutations"] is True
|
||||
assert [item["status"] for item in result["results"]] == ["planned", "planned"]
|
||||
|
||||
|
||||
def test_cancellation_prevents_unscheduled_actions_from_starting(mod, monkeypatch):
|
||||
cancelled = threading.Event()
|
||||
calls = []
|
||||
|
||||
def fake_call(_name, arguments):
|
||||
calls.append(arguments["value"])
|
||||
cancelled.set()
|
||||
return {"ok": True}
|
||||
|
||||
monkeypatch.setattr(mod, "call_tool_impl", fake_call)
|
||||
mod.MCP_REQUEST_LOCAL.cancelled = cancelled
|
||||
try:
|
||||
with pytest.raises(mod.SafeToolError) as error:
|
||||
mod.execute_batch({
|
||||
"max_parallel": 1,
|
||||
"actions": [
|
||||
action("first", arguments={"value": "first"}),
|
||||
action("second", tool="nap_context", arguments={"value": "second"}),
|
||||
],
|
||||
})
|
||||
finally:
|
||||
del mod.MCP_REQUEST_LOCAL.cancelled
|
||||
|
||||
assert error.value.code == "request_cancelled"
|
||||
assert calls == ["first"]
|
||||
|
||||
|
||||
def test_protocol_lists_and_dry_runs_batch(mod):
|
||||
listed = mod.handle({"jsonrpc": "2.0", "id": 1, "method": "tools/list"})
|
||||
names = [item["name"] for item in listed["result"]["tools"]]
|
||||
assert "nap_batch" in names
|
||||
|
||||
called = mod.handle({
|
||||
"jsonrpc": "2.0",
|
||||
"id": 2,
|
||||
"method": "tools/call",
|
||||
"params": {
|
||||
"name": "nap_batch",
|
||||
"arguments": {
|
||||
"dry_run": True,
|
||||
"actions": [action("read", arguments={"node_id": "node-1"})],
|
||||
},
|
||||
},
|
||||
})
|
||||
payload = called["result"]["structuredContent"]
|
||||
assert called["result"]["isError"] is False
|
||||
assert payload["ok"] is True
|
||||
assert payload["results"][0]["status"] == "planned"
|
||||
|
||||
|
||||
def test_profile_gated_tool_is_rejected_before_execution(mod, monkeypatch):
|
||||
mod.CONFIGURED_TOOL_PROFILES = set()
|
||||
calls = []
|
||||
monkeypatch.setattr(mod, "call_tool_impl", lambda *args: calls.append(args))
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as error:
|
||||
mod.execute_batch({"actions": [action("plan", tool="nap_plan_list_active")]})
|
||||
|
||||
assert error.value.code == "tool_profile_required"
|
||||
assert calls == []
|
||||
@@ -0,0 +1,502 @@
|
||||
"""Encrypted local E2E index shard against a disposable HTTP provider.
|
||||
|
||||
E2E nodes are encrypted with a synthetic memory secret using the wrapper's
|
||||
own envelope code. The real reindex, feed sync, search and shard persistence
|
||||
code run; only the key cache is seeded (standing in for an unlocked vault).
|
||||
"""
|
||||
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import stat
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = pathlib.Path(__file__).resolve().parents[1] / "resources/scripts/napseer_mcp_server.py"
|
||||
PROJECT = "project-1"
|
||||
# Synthetic fixture strings; none may appear in any file in the state dir.
|
||||
BODY_TERM = "zephyrquartz"
|
||||
BODY = f"calibrate the {BODY_TERM} manifold before launch"
|
||||
META_VALUE = "obsidianfalcon"
|
||||
PLAINTEXT_MARKERS = (BODY_TERM.encode(), META_VALUE.encode(), b"manifold before launch", b"brandnewsecret")
|
||||
|
||||
|
||||
def memory_secret(version=3, secret_bytes=bytes(range(32))):
|
||||
return {
|
||||
"project_id": PROJECT, "secret_kind": "memory", "version": version,
|
||||
"wrapping_epoch": 1, "bundle_version": version, "data_key_epoch": version,
|
||||
"secret_bytes": secret_bytes, "created_at": "2026-09-01T00:00:00Z", "last_rotated_at": "2026-09-01T00:00:00Z",
|
||||
}
|
||||
|
||||
|
||||
def cache_secret(mod, secret, active=True):
|
||||
slot = (PROJECT, "memory", secret["version"], secret["wrapping_epoch"], secret["bundle_version"], secret["data_key_epoch"])
|
||||
mod.MEMORY_SECRET_CACHE[slot] = secret
|
||||
if active:
|
||||
mod.ACTIVE_MEMORY_SECRET_VERSIONS[PROJECT] = slot
|
||||
|
||||
|
||||
def lock(mod):
|
||||
mod.e2e_shards_close(flush=True)
|
||||
mod.MEMORY_SECRET_CACHE.clear()
|
||||
mod.ACTIVE_MEMORY_SECRET_VERSIONS.clear()
|
||||
mod.PROJECT_VAULT_PASSPHRASE = None
|
||||
|
||||
|
||||
def unlock(mod, secret=None):
|
||||
cache_secret(mod, secret or memory_secret())
|
||||
|
||||
|
||||
def plain_node(name, minute, content=None, folder="/notes"):
|
||||
stamp = f"2026-09-01T12:{minute:02d}:00Z"
|
||||
return {
|
||||
"id": name, "project_id": PROJECT, "full_path": f"{folder}/{name}", "folder_path": folder,
|
||||
"name": name, "type": "note", "metadata": {}, "tags": [], "aliases": [], "links": [],
|
||||
"content_text": content if content is not None else name, "updated_at": stamp, "archived_at": None,
|
||||
}
|
||||
|
||||
|
||||
def e2e_node(mod, name, minute, content, metadata=None, secret=None, archived=False, folder="/notes"):
|
||||
secret = secret or memory_secret()
|
||||
node = plain_node(name, minute, folder=folder)
|
||||
context = {"full_path": node["full_path"]}
|
||||
node.update({
|
||||
"content_text": "", "metadata": {}, "encryption_state": "encrypted",
|
||||
"project_secret_version": secret["version"],
|
||||
"encrypted_content_envelope": mod.encrypt_memory_bytes(
|
||||
PROJECT, secret, "node_content", content.encode(), context, "text/plain; charset=utf-8"),
|
||||
"encrypted_metadata_envelope": mod.encrypt_memory_bytes(
|
||||
PROJECT, secret, "node_metadata", mod.stable_json_bytes(metadata or {}), context,
|
||||
"application/json; charset=utf-8") if metadata else None,
|
||||
})
|
||||
if archived:
|
||||
node["archived_at"] = node["updated_at"]
|
||||
return node
|
||||
|
||||
|
||||
class Provider:
|
||||
def __init__(self):
|
||||
self.nodes = {}
|
||||
self.events = []
|
||||
self.requests = []
|
||||
|
||||
def put(self, node, kind):
|
||||
self.nodes[node["id"]] = copy.deepcopy(node)
|
||||
self.events.append({"type": kind, "node_id": node["id"], "full_path": node["full_path"],
|
||||
"new_revision": node["updated_at"]})
|
||||
|
||||
def delete(self, node_id):
|
||||
node = self.nodes.pop(node_id)
|
||||
self.events.append({"type": "node_deleted", "node_id": node_id, "full_path": node["full_path"],
|
||||
"new_revision": None})
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def env(tmp_path, monkeypatch):
|
||||
provider = Provider()
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def reply(self, status, payload):
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps(payload).encode())
|
||||
|
||||
def do_GET(self):
|
||||
url = urlsplit(self.path)
|
||||
query = parse_qs(url.query)
|
||||
kind = "changes" if url.path.endswith("/changes") else "list" if url.path.endswith("/nodes") else "get"
|
||||
provider.requests.append((kind, url.path))
|
||||
if "/vault/secrets/" in url.path:
|
||||
return self.reply(503, {"error": {"code": "test_failure", "message": "no vault in test"}})
|
||||
if kind == "changes":
|
||||
after = query.get("after", ["head"])[0]
|
||||
start = len(provider.events) if after == "head" else int(re.fullmatch(r"c(\d+)", after).group(1))
|
||||
limit = int(query.get("limit", ["100"])[0])
|
||||
page = provider.events[start:start + limit]
|
||||
items = [{**event, "cursor": f"c{start + i + 1}", "project_id": PROJECT, "changed_fields": [],
|
||||
"created_at": "2026-09-01T12:00:00Z"} for i, event in enumerate(page)]
|
||||
end = start + len(page)
|
||||
return self.reply(200, {"items": items, "next_cursor": f"c{end}", "has_more": end < len(provider.events)})
|
||||
if kind == "list":
|
||||
include_archived = query.get("include_archived", ["false"])[0] in {"true", "True", "1"}
|
||||
rows = [copy.deepcopy(n) for n in provider.nodes.values() if include_archived or not n["archived_at"]]
|
||||
rows.sort(key=lambda n: (n["updated_at"], n["id"]), reverse=True)
|
||||
return self.reply(200, {"items": rows, "next_cursor": None})
|
||||
node_id = url.path.rsplit("/", 1)[-1]
|
||||
if node_id not in provider.nodes:
|
||||
return self.reply(404, {"error": {"code": "not_found", "message": "missing"}})
|
||||
return self.reply(200, copy.deepcopy(provider.nodes[node_id]))
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
sqlite_tmp = tmp_path / "sqlite-tmp"
|
||||
sqlite_tmp.mkdir()
|
||||
monkeypatch.setenv("SQLITE_TMPDIR", str(sqlite_tmp))
|
||||
monkeypatch.setenv("TMPDIR", str(sqlite_tmp))
|
||||
for name in ("NAPSEER_VAULT_PASSPHRASE", "NAPSEER_MASTER_PASSPHRASE", "NAPSEER_CRYPTO_BACKEND"):
|
||||
monkeypatch.delenv(name, raising=False)
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "all")
|
||||
monkeypatch.syspath_prepend(str(SCRIPT.parent))
|
||||
spec = importlib.util.spec_from_file_location("e2e_shard_test_worker", SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
mod.AUTH_DIR = tmp_path / "project-state"
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.lock"
|
||||
mod.BASE_URL = f"http://127.0.0.1:{server.server_port}"
|
||||
mod.TOKEN = "disposable-test-token"
|
||||
mod.AUTH = {"account_id": "test-account"}
|
||||
mod.DEFAULT_PROJECT_ID = PROJECT
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
mod.require_unlocked = lambda _name: None
|
||||
mod.resolve_project_id = lambda args: args.get("project_id", PROJECT)
|
||||
mod.project_encryption_state = lambda _project_id: "encrypted"
|
||||
mod.MEMORY_SECRET_CACHE.clear()
|
||||
mod.ACTIVE_MEMORY_SECRET_VERSIONS.clear()
|
||||
unlock(mod)
|
||||
try:
|
||||
yield mod, provider, sqlite_tmp
|
||||
finally:
|
||||
mod.e2e_shards_close(flush=False)
|
||||
mod.select_crypto_backend()
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
def seed(mod, provider):
|
||||
provider.put(e2e_node(mod, "secretnote", 1, BODY, metadata={"owner": META_VALUE}), "node_created")
|
||||
provider.put(plain_node("plainnote", 2, "ordinary plain body text"), "node_created")
|
||||
return mod.reindex_project({})
|
||||
|
||||
|
||||
def local_search(mod, text):
|
||||
return mod.search_local_index({"q": text, "limit": 10})
|
||||
|
||||
|
||||
def hit_ids(result):
|
||||
return [item["id"] for item in result["items"]]
|
||||
|
||||
|
||||
def codes(result):
|
||||
return [warning["code"] for warning in result.get("warnings") or []]
|
||||
|
||||
|
||||
def shard_file(mod):
|
||||
return mod.e2e_shard_path(PROJECT)
|
||||
|
||||
|
||||
def assert_no_plaintext_on_disk(mod, *extra_dirs):
|
||||
for directory in (mod.AUTH_DIR, *extra_dirs):
|
||||
for path in directory.rglob("*"):
|
||||
if path.is_file():
|
||||
raw = path.read_bytes()
|
||||
for marker in PLAINTEXT_MARKERS:
|
||||
assert marker not in raw, f"{marker!r} leaked into {path.name}"
|
||||
|
||||
|
||||
def test_e2e_body_term_is_findable_with_snippet_while_unlocked(env):
|
||||
mod, provider, _ = env
|
||||
result = seed(mod, provider)
|
||||
assert result["e2e_index"]["status"] == "applied"
|
||||
|
||||
local = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(local) == ["secretnote"]
|
||||
item = local["items"][0]
|
||||
assert item["search_source"] == "local_e2e"
|
||||
assert BODY_TERM in item["snippet"]
|
||||
assert BODY_TERM in item["matched_terms"]
|
||||
assert hit_ids(local_search(mod, META_VALUE)) == ["secretnote"]
|
||||
# Non-E2E results are unchanged and not marked as shard hits.
|
||||
plain = local_search(mod, "ordinary")
|
||||
assert hit_ids(plain) == ["plainnote"] and "search_source" not in plain["items"][0]
|
||||
|
||||
discovered = mod.discover_memory({"q": BODY_TERM, "mode": "local"})
|
||||
assert [item["node_id"] for item in discovered["items"]] == ["secretnote"]
|
||||
assert discovered["items"][0]["search_source"] == "local_e2e"
|
||||
assert "local_e2e" in discovered["sources"]
|
||||
assert not discovered.get("warnings")
|
||||
|
||||
|
||||
def test_e2e_shard_hits_filter_on_effective_type(env):
|
||||
mod, provider, _ = env
|
||||
provider.put(e2e_node(mod, "legacyplan", 1, BODY, folder="/plans"), "node_created")
|
||||
provider.put(e2e_node(mod, "scratchnote", 2, BODY), "node_created")
|
||||
assert mod.reindex_project({})["e2e_index"]["status"] == "applied"
|
||||
|
||||
# Bodies are only in the shard, so these hits come from the E2E path.
|
||||
plans = mod.search_local_index({"q": BODY_TERM, "limit": 10, "type": "plan"})
|
||||
assert hit_ids(plans) == ["legacyplan"]
|
||||
assert plans["items"][0]["search_source"] == "local_e2e"
|
||||
assert (plans["items"][0]["type"], plans["items"][0]["effective_type"]) == ("note", "plan")
|
||||
notes = mod.search_local_index({"q": BODY_TERM, "limit": 10, "type": "note"})
|
||||
assert hit_ids(notes) == ["scratchnote"]
|
||||
assert notes["items"][0]["search_source"] == "local_e2e"
|
||||
|
||||
|
||||
def test_unprovisioned_secret_yields_no_e2e_hits_and_an_explicit_warning(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
lock(mod)
|
||||
|
||||
local = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(local) == []
|
||||
assert codes(local) == ["e2e_index_not_provisioned"]
|
||||
warning = local["warnings"][0]
|
||||
assert warning["memory_secret"] == "not_provisioned"
|
||||
assert warning["next_action"] == "nap gateway vault setup"
|
||||
response = mod.search_memory({"q": BODY_TERM, "mode": "local", "view": "summary"})
|
||||
assert response["items"] == []
|
||||
assert "e2e_index_not_provisioned" in codes(response)
|
||||
# Identity fields of the E2E node remain searchable from the main index.
|
||||
assert hit_ids(local_search(mod, "secretnote")) == ["secretnote"]
|
||||
# An unprovisioned reindex never prompts and writes no shard.
|
||||
shard_file(mod).unlink()
|
||||
assert "e2e_index" in mod.reindex_project({})
|
||||
assert not shard_file(mod).exists()
|
||||
|
||||
unlock(mod)
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == [] # shard not rebuilt yet: reported, not guessed
|
||||
assert "e2e_index_incomplete" in codes(local_search(mod, BODY_TERM))
|
||||
synced = mod.sync_local_index({})
|
||||
assert synced["e2e_index"]["status"] == "complete"
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == ["secretnote"]
|
||||
|
||||
|
||||
def test_shard_file_holds_no_plaintext_and_no_side_files_exist(env):
|
||||
mod, provider, sqlite_tmp = env
|
||||
seed(mod, provider)
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == ["secretnote"]
|
||||
shard = mod.e2e_shard_loaded(PROJECT)
|
||||
with shard.lock:
|
||||
assert shard.conn.execute("PRAGMA journal_mode").fetchone()[0] == "memory"
|
||||
assert shard.conn.execute("PRAGMA temp_store").fetchone()[0] == 2
|
||||
databases = shard.conn.execute("PRAGMA database_list").fetchall()
|
||||
assert [(row[1], row[2]) for row in databases] == [("main", "")]
|
||||
mod.e2e_shards_close(flush=True)
|
||||
|
||||
path = shard_file(mod)
|
||||
assert path.exists() and stat.S_IMODE(path.stat().st_mode) == 0o600
|
||||
assert path.read_bytes().startswith(mod.E2E_SHARD_MAGIC)
|
||||
names = {entry.name for entry in mod.AUTH_DIR.iterdir()}
|
||||
assert names <= {"index.sqlite", "index.sqlite-wal", "index.sqlite-shm", "index.lock", path.name}
|
||||
assert not any(name.endswith(".tmp") or "journal" in name for name in names)
|
||||
assert list(sqlite_tmp.iterdir()) == []
|
||||
assert_no_plaintext_on_disk(mod, sqlite_tmp)
|
||||
# The shard lives in the private state dir, never in the checkout.
|
||||
assert path.parent == mod.AUTH_DIR
|
||||
|
||||
|
||||
def test_tampered_shard_fails_closed_rebuilds_and_reports(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
mod.e2e_shards_close(flush=True)
|
||||
path = shard_file(mod)
|
||||
blob = bytearray(path.read_bytes())
|
||||
blob[-20] ^= 0x01
|
||||
path.write_bytes(bytes(blob))
|
||||
|
||||
response = mod.search_memory({"q": BODY_TERM, "mode": "local", "view": "summary"})
|
||||
|
||||
discarded = [w for w in response["warnings"] if w["code"] == "e2e_index_discarded"]
|
||||
assert discarded and discarded[0]["reason"] == "tampered"
|
||||
assert [item["node_id"] for item in response["items"]] == ["secretnote"] # rebuilt from source
|
||||
mod.e2e_shards_close(flush=True)
|
||||
rebuilt = path.read_bytes()
|
||||
assert rebuilt != bytes(blob)
|
||||
# The rebuilt file loads cleanly in a fresh process state.
|
||||
local = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(local) == ["secretnote"] and not local.get("warnings")
|
||||
|
||||
|
||||
def test_truncated_or_foreign_shard_file_is_discarded(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
mod.e2e_shards_close(flush=True)
|
||||
shard_file(mod).write_bytes(b"not a shard")
|
||||
result = mod.sync_local_index({})
|
||||
assert [w["reason"] for w in result["e2e_index"]["warnings"]] == ["format"]
|
||||
assert result["e2e_index"]["status"] == "complete"
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == ["secretnote"]
|
||||
|
||||
|
||||
def test_wrong_key_fails_closed_and_reports_undecryptable_nodes(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
lock(mod)
|
||||
unlock(mod, memory_secret(secret_bytes=bytes(range(1, 33))))
|
||||
|
||||
response = mod.search_memory({"q": BODY_TERM, "mode": "local", "view": "summary"})
|
||||
|
||||
assert response["items"] == []
|
||||
by_code = {w["code"]: w for w in response["warnings"]}
|
||||
assert by_code["e2e_index_discarded"]["reason"] == "wrong_key"
|
||||
assert by_code["e2e_index_undecryptable"]["nodes"] == 1
|
||||
assert_no_plaintext_on_disk(mod)
|
||||
|
||||
|
||||
def test_secret_version_change_discards_and_rebuilds_under_the_new_key(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == ["secretnote"]
|
||||
|
||||
# Rotation while loaded: the in-memory shard is re-keyed by discarding it.
|
||||
cache_secret(mod, memory_secret(version=4, secret_bytes=bytes(range(100, 132))))
|
||||
response = mod.search_memory({"q": BODY_TERM, "mode": "local", "view": "summary"})
|
||||
reasons = [w.get("reason") for w in response["warnings"] if w["code"] == "e2e_index_discarded"]
|
||||
assert reasons == ["key_rotated"]
|
||||
assert [item["node_id"] for item in response["items"]] == ["secretnote"] # v3 node, v4 shard key
|
||||
mod.e2e_shards_close(flush=True)
|
||||
header = shard_file(mod).read_bytes()
|
||||
assert b'"secret_version":4' in header
|
||||
|
||||
# Rotation between processes: a v4 file opened with a v5 active secret.
|
||||
cache_secret(mod, memory_secret(version=5, secret_bytes=bytes(range(50, 82))))
|
||||
result = mod.sync_local_index({})
|
||||
assert [w["reason"] for w in result["e2e_index"]["warnings"]] == ["key_rotated"]
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == ["secretnote"]
|
||||
|
||||
|
||||
def test_feed_update_archive_and_delete_update_the_shard(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
provider.put(e2e_node(mod, "othernote", 3, "second brandnewsecret body"), "node_created")
|
||||
mod.sync_local_index_from_feed(PROJECT)
|
||||
assert hit_ids(local_search(mod, "brandnewsecret")) == ["othernote"]
|
||||
|
||||
provider.put(e2e_node(mod, "secretnote", 10, "rewritten body mentions heliotrope"), "node_updated")
|
||||
result = mod.sync_local_index_from_feed(PROJECT)
|
||||
assert result["e2e_index"]["indexed"] == 1
|
||||
assert hit_ids(local_search(mod, "heliotrope")) == ["secretnote"]
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == []
|
||||
|
||||
provider.delete("othernote")
|
||||
provider.put(e2e_node(mod, "secretnote", 11, "archived text", archived=True), "node_archived")
|
||||
mod.sync_local_index_from_feed(PROJECT)
|
||||
assert hit_ids(local_search(mod, "brandnewsecret")) == []
|
||||
assert hit_ids(local_search(mod, "heliotrope")) == []
|
||||
shard = mod.e2e_shard_loaded(PROJECT)
|
||||
with shard.lock:
|
||||
assert shard.conn.execute("SELECT COUNT(*) FROM e2e_nodes").fetchone()[0] == 0
|
||||
# A reloaded shard reflects the removals as well.
|
||||
mod.e2e_shards_close(flush=True)
|
||||
assert hit_ids(local_search(mod, "brandnewsecret")) == []
|
||||
|
||||
|
||||
def test_local_write_path_indexes_and_removes_shard_text(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
written = e2e_node(mod, "writtennote", 20, "locally written umbraquill text")
|
||||
mod.index_node(written)
|
||||
assert hit_ids(local_search(mod, "umbraquill")) == ["writtennote"]
|
||||
mod.remove_indexed_node("writtennote")
|
||||
assert hit_ids(local_search(mod, "umbraquill")) == []
|
||||
|
||||
|
||||
def test_stale_shard_row_is_not_served_and_is_refetched(env):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
mod.e2e_shards_close(flush=True)
|
||||
# Another process advanced the main index without this shard file.
|
||||
updated = e2e_node(mod, "secretnote", 30, "fresh body with vermilionkite")
|
||||
provider.put(updated, "node_updated")
|
||||
with mod.index_connect() as conn:
|
||||
mod.index_node(updated, connection=conn)
|
||||
local = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(local) == [] and "e2e_index_incomplete" in codes(local)
|
||||
mod.sync_local_index({})
|
||||
assert hit_ids(local_search(mod, "vermilionkite")) == ["secretnote"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("writer,reader", [("native", "pure_python"), ("pure_python", "native")])
|
||||
def test_crypto_backends_read_each_others_shard(env, writer, reader):
|
||||
mod, provider, _ = env
|
||||
if not mod.native_crypto_available():
|
||||
pytest.skip("cryptography is not installed")
|
||||
mod.select_crypto_backend(writer)
|
||||
seed(mod, provider)
|
||||
mod.e2e_shards_close(flush=True)
|
||||
mod.select_crypto_backend(reader)
|
||||
local = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(local) == ["secretnote"] and not local.get("warnings")
|
||||
|
||||
|
||||
def test_pure_backend_large_shard_loads_in_background_and_warns(env, monkeypatch):
|
||||
mod, provider, _ = env
|
||||
seed(mod, provider)
|
||||
mod.e2e_shards_close(flush=True)
|
||||
mod.select_crypto_backend("pure_python")
|
||||
monkeypatch.setattr(mod, "E2E_SHARD_PURE_SYNC_LOAD_MAX_BYTES", 1)
|
||||
|
||||
first = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(first) == []
|
||||
slow = [w for w in first["warnings"] if w["code"] == "e2e_index_slow_crypto"]
|
||||
assert slow and slow[0]["state"] == "loading"
|
||||
loader = mod.e2e_shard_get(PROJECT).loading
|
||||
if loader is not None:
|
||||
loader.join(timeout=30)
|
||||
assert hit_ids(local_search(mod, BODY_TERM)) == ["secretnote"]
|
||||
|
||||
|
||||
def test_pure_backend_does_not_persist_oversized_shard(env, monkeypatch):
|
||||
mod, provider, _ = env
|
||||
mod.select_crypto_backend("pure_python")
|
||||
monkeypatch.setattr(mod, "E2E_SHARD_PURE_MAX_PERSIST_BYTES", 1)
|
||||
result = seed(mod, provider)
|
||||
assert result["e2e_index"]["persist"] == "skipped_slow_crypto"
|
||||
assert not shard_file(mod).exists()
|
||||
local = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(local) == ["secretnote"]
|
||||
assert [w.get("state") for w in local["warnings"]] == ["not_persisted"]
|
||||
|
||||
|
||||
def test_unsupported_sqlite_disables_e2e_search_with_reason(env, monkeypatch):
|
||||
mod, provider, _ = env
|
||||
monkeypatch.setattr(mod, "e2e_shard_supported", lambda: (False, "sqlite_serialize_unavailable"))
|
||||
seed(mod, provider)
|
||||
local = local_search(mod, BODY_TERM)
|
||||
assert hit_ids(local) == []
|
||||
assert codes(local) == ["e2e_index_unsupported"]
|
||||
assert not shard_file(mod).exists()
|
||||
|
||||
|
||||
def test_project_without_e2e_nodes_creates_no_shard_or_warnings(env):
|
||||
mod, provider, _ = env
|
||||
provider.put(plain_node("plainnote", 2, "ordinary plain body text"), "node_created")
|
||||
result = mod.reindex_project({})
|
||||
assert "e2e_index" not in result
|
||||
local = local_search(mod, "ordinary")
|
||||
assert hit_ids(local) == ["plainnote"] and "warnings" not in local
|
||||
assert not shard_file(mod).exists()
|
||||
assert mod.index_status({})["e2e_index"]["loaded"] is False
|
||||
|
||||
|
||||
def test_shard_key_is_bound_to_project_and_secret_identity(env):
|
||||
mod, _provider, _ = env
|
||||
secret = memory_secret()
|
||||
identity, key, key_check = mod.e2e_shard_key_material(PROJECT, secret)
|
||||
other_project = mod.e2e_shard_key_material("project-2", secret)
|
||||
other_version = mod.e2e_shard_key_material(PROJECT, {**secret, "data_key_epoch": 4})
|
||||
assert len(key) == 32 and key not in {other_project[1], other_version[1]}
|
||||
blob = mod.e2e_shard_encode(PROJECT, identity, key, key_check, b"synthetic")
|
||||
assert mod.e2e_shard_decode(blob, PROJECT, identity, key, key_check) == b"synthetic"
|
||||
assert blob != mod.e2e_shard_encode(PROJECT, identity, key, key_check, b"synthetic") # fresh nonce
|
||||
with pytest.raises(mod.E2EShardFileInvalid) as info:
|
||||
mod.e2e_shard_decode(blob, "project-2", identity, key, key_check)
|
||||
assert info.value.reason == "project_mismatch"
|
||||
# Rewriting the header (AAD) without the key is detected.
|
||||
forged = blob.replace(b'"index_schema":2', b'"index_schema":2 ')
|
||||
assert forged != blob
|
||||
with pytest.raises(mod.E2EShardFileInvalid):
|
||||
mod.e2e_shard_decode(forged, PROJECT, identity, key, key_check)
|
||||
@@ -0,0 +1,404 @@
|
||||
"""Change-feed driven local index freshness against a disposable HTTP provider.
|
||||
|
||||
The provider implements the node list, single-node read and change-feed
|
||||
routes with synthetic data. The real request, index and search code run.
|
||||
"""
|
||||
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = pathlib.Path(__file__).resolve().parents[1] / "resources/scripts/napseer_mcp_server.py"
|
||||
|
||||
|
||||
def make_node(name, minute, *, content=None, archived=False):
|
||||
stamp = f"2026-09-01T12:{minute:02d}:00Z"
|
||||
return {
|
||||
"id": name, "project_id": "project-1", "full_path": f"/notes/{name}",
|
||||
"folder_path": "/notes", "name": name, "type": "note",
|
||||
"metadata": {}, "tags": [], "aliases": [], "links": [],
|
||||
"content_text": content if content is not None else name,
|
||||
"updated_at": stamp, "archived_at": stamp if archived else None,
|
||||
}
|
||||
|
||||
|
||||
class Provider:
|
||||
def __init__(self):
|
||||
self.nodes = {}
|
||||
self.events = []
|
||||
self.requests = []
|
||||
self.fail = {} # route kind -> HTTP status
|
||||
self.on_list = None
|
||||
|
||||
def put(self, node, kind):
|
||||
self.nodes[node["id"]] = copy.deepcopy(node)
|
||||
self.events.append({"type": kind, "node_id": node["id"], "full_path": node["full_path"],
|
||||
"new_revision": node["updated_at"]})
|
||||
|
||||
def delete(self, node_id):
|
||||
node = self.nodes.pop(node_id)
|
||||
self.events.append({"type": "node_deleted", "node_id": node_id, "full_path": node["full_path"],
|
||||
"new_revision": None})
|
||||
|
||||
def count(self, kind):
|
||||
return sum(1 for item in self.requests if item[0] == kind)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def feed(tmp_path, monkeypatch):
|
||||
provider = Provider()
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def reply(self, status, payload):
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps(payload).encode())
|
||||
|
||||
def do_GET(self):
|
||||
url = urlsplit(self.path)
|
||||
query = parse_qs(url.query)
|
||||
if url.path.endswith("/changes"):
|
||||
kind = "changes"
|
||||
elif url.path.endswith("/nodes"):
|
||||
kind = "list"
|
||||
else:
|
||||
kind = "get"
|
||||
provider.requests.append((kind, url.path, query))
|
||||
if kind in provider.fail:
|
||||
status = provider.fail[kind]
|
||||
return self.reply(status, {"error": {"code": "test_failure", "message": "injected"}})
|
||||
if kind == "changes":
|
||||
after = query.get("after", ["head"])[0]
|
||||
if after == "head":
|
||||
start = len(provider.events)
|
||||
elif re.fullmatch(r"c\d+", after):
|
||||
start = int(after[1:])
|
||||
else:
|
||||
return self.reply(400, {"error": {"code": "invalid_cursor", "message": "bad cursor"}})
|
||||
limit = int(query.get("limit", ["100"])[0])
|
||||
page = provider.events[start:start + limit]
|
||||
items = [{**event, "cursor": f"c{start + index + 1}", "project_id": "project-1",
|
||||
"changed_fields": [], "created_at": "2026-09-01T12:00:00Z"}
|
||||
for index, event in enumerate(page)]
|
||||
end = start + len(page)
|
||||
return self.reply(200, {"items": items, "next_cursor": f"c{end}",
|
||||
"has_more": end < len(provider.events)})
|
||||
if kind == "list":
|
||||
if provider.on_list is not None:
|
||||
hook, provider.on_list = provider.on_list, None
|
||||
hook()
|
||||
rows = [copy.deepcopy(n) for n in provider.nodes.values() if not n["archived_at"]]
|
||||
rows.sort(key=lambda n: (n["updated_at"], n["id"]), reverse=True)
|
||||
after = query.get("updated_after", [""])[0]
|
||||
if after:
|
||||
rows = [n for n in provider.nodes.values() if n["updated_at"] > after]
|
||||
return self.reply(200, {"items": rows, "next_cursor": None})
|
||||
node_id = url.path.rsplit("/", 1)[-1]
|
||||
if node_id not in provider.nodes:
|
||||
return self.reply(404, {"error": {"code": "not_found", "message": "missing"}})
|
||||
return self.reply(200, copy.deepcopy(provider.nodes[node_id]))
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "all")
|
||||
monkeypatch.syspath_prepend(str(SCRIPT.parent))
|
||||
spec = importlib.util.spec_from_file_location("feed_sync_test_worker", SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
mod.AUTH_DIR = tmp_path / "project-state"
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.lock"
|
||||
mod.BASE_URL = f"http://127.0.0.1:{server.server_port}"
|
||||
mod.TOKEN = "disposable-test-token"
|
||||
mod.AUTH = {"account_id": "test-account"}
|
||||
mod.DEFAULT_PROJECT_ID = "project-1"
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
mod.require_unlocked = lambda _name: None
|
||||
mod.resolve_project_id = lambda args: args.get("project_id", "project-1")
|
||||
mod.project_encryption_state = lambda _project_id: "standard"
|
||||
try:
|
||||
yield mod, provider
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
def local_ids(mod, text):
|
||||
return [item["id"] for item in mod.search_local_index({"q": text, "limit": 10})["items"]]
|
||||
|
||||
|
||||
def seed(mod, provider, *names):
|
||||
for index, name in enumerate(names):
|
||||
provider.put(make_node(name, index), "node_created")
|
||||
result = mod.reindex_project({})
|
||||
assert result["graph_complete"] is True
|
||||
return result
|
||||
|
||||
|
||||
def test_reindex_takes_head_cursor_before_listing_and_replays_idempotently(feed):
|
||||
mod, provider = feed
|
||||
provider.put(make_node("alpha", 1), "node_created")
|
||||
|
||||
def concurrent_write():
|
||||
provider.put(make_node("alpha", 2, content="alpha concurrent revision"), "node_updated")
|
||||
|
||||
provider.on_list = concurrent_write
|
||||
mod.reindex_project({})
|
||||
|
||||
kinds = [item[0] for item in provider.requests]
|
||||
assert kinds[:2] == ["changes", "list"]
|
||||
assert provider.requests[0][2]["after"] == ["head"]
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == "c1"
|
||||
assert local_ids(mod, "concurrent") == ["alpha"]
|
||||
|
||||
result = mod.sync_local_index_from_feed("project-1")
|
||||
|
||||
assert (result["events"], result["skipped_current"], result["indexed"]) == (1, 1, 0)
|
||||
assert provider.count("get") == 0 # revision already indexed: no refetch
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == "c2"
|
||||
|
||||
|
||||
def test_feed_applies_create_update_archive_restore_and_delete(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "keepme", "archiveme", "deleteme", "restoreme")
|
||||
provider.put(make_node("restoreme", 5, archived=True), "node_archived")
|
||||
provider.put(make_node("freshnode", 10, content="brandnewterm"), "node_created")
|
||||
provider.put(make_node("keepme", 11, content="keepme revisedterm"), "node_updated")
|
||||
provider.put(make_node("archiveme", 12, archived=True), "node_archived")
|
||||
provider.delete("deleteme")
|
||||
provider.put(make_node("restoreme", 13), "node_restored")
|
||||
|
||||
result = mod.sync_local_index_from_feed("project-1")
|
||||
|
||||
assert result["status"] == "synced"
|
||||
assert local_ids(mod, "brandnewterm") == ["freshnode"]
|
||||
assert local_ids(mod, "revisedterm") == ["keepme"]
|
||||
assert local_ids(mod, "archiveme") == []
|
||||
assert local_ids(mod, "deleteme") == []
|
||||
assert local_ids(mod, "restoreme") == ["restoreme"]
|
||||
with mod.index_connect() as conn:
|
||||
ids = {row[0] for row in conn.execute("SELECT node_id FROM local_index_nodes")}
|
||||
# Archived rows match a full rebuild, which lists only active nodes.
|
||||
assert ids == {"keepme", "freshnode", "restoreme"}
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == f"c{len(provider.events)}"
|
||||
|
||||
|
||||
def test_feed_failure_never_advances_cursor_and_retry_recovers(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "stable")
|
||||
before = mod.local_index_feed_state("project-1")["feed_cursor"]
|
||||
provider.put(make_node("newcomer", 10, content="latecomerterm"), "node_created")
|
||||
provider.fail["get"] = 503
|
||||
|
||||
with pytest.raises(mod.SafeToolError):
|
||||
mod.sync_local_index_from_feed("project-1")
|
||||
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == before
|
||||
assert local_ids(mod, "latecomerterm") == []
|
||||
del provider.fail["get"]
|
||||
mod.sync_local_index_from_feed("project-1")
|
||||
assert local_ids(mod, "latecomerterm") == ["newcomer"]
|
||||
|
||||
|
||||
def test_invalid_feed_cursor_triggers_full_reindex(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "stable")
|
||||
with mod.index_connect() as conn:
|
||||
conn.execute("UPDATE local_index_state SET feed_cursor = 'expired-cursor'")
|
||||
provider.nodes["hardgone"] = make_node("hardgone", 3) # present only in the full listing
|
||||
|
||||
result = mod.sync_local_index({})
|
||||
|
||||
assert result["mode"] == "full_reindex"
|
||||
assert result["reason"] == "feed_cursor_invalid"
|
||||
assert local_ids(mod, "hardgone") == ["hardgone"]
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == f"c{len(provider.events)}"
|
||||
|
||||
|
||||
def test_old_server_without_feed_falls_back_to_timestamp_sync(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "stable")
|
||||
provider.fail["changes"] = 404
|
||||
provider.nodes["later"] = make_node("later", 30, content="timestampterm")
|
||||
|
||||
result = mod.sync_local_index({})
|
||||
|
||||
assert result["mode"] == "incremental"
|
||||
assert result["feed"] == "unsupported"
|
||||
assert local_ids(mod, "timestampterm") == ["later"]
|
||||
|
||||
|
||||
def test_reindex_on_old_server_without_feed_still_rebuilds(feed):
|
||||
mod, provider = feed
|
||||
provider.fail["changes"] = 404
|
||||
provider.put(make_node("legacy", 1), "node_created")
|
||||
|
||||
result = mod.reindex_project({})
|
||||
|
||||
assert result["graph_complete"] is True
|
||||
assert local_ids(mod, "legacy") == ["legacy"]
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] is None
|
||||
|
||||
|
||||
def test_search_auto_syncs_from_feed_and_throttles(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "stable")
|
||||
provider.put(make_node("autosync", 10, content="autosyncterm"), "node_created")
|
||||
|
||||
first = mod.call_tool_impl("nap_discover", {"q": "autosyncterm", "mode": "local"})
|
||||
provider.put(make_node("throttled", 11, content="throttledterm"), "node_created")
|
||||
second = mod.call_tool_impl("nap_discover", {"q": "throttledterm", "mode": "local"})
|
||||
|
||||
assert [item["node_id"] for item in first["items"]] == ["autosync"]
|
||||
assert first["freshness"]["stale"] is False
|
||||
assert first["freshness"]["feed_synced_at"]
|
||||
assert second["items"] == [] # within the throttle window no second feed read
|
||||
assert provider.count("changes") == 2 # reindex head + one automatic sync
|
||||
assert second["freshness"]["feed_sync"] == "throttled"
|
||||
|
||||
|
||||
def test_search_answers_locally_and_marks_stale_when_feed_unavailable(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "stable")
|
||||
provider.fail["changes"] = 503
|
||||
|
||||
result = mod.call_tool_impl("nap_discover", {"q": "stable", "mode": "local"})
|
||||
|
||||
assert [item["node_id"] for item in result["items"]] == ["stable"]
|
||||
assert result["freshness"]["stale"] is True
|
||||
assert result["freshness"]["reason"] == "server_unavailable"
|
||||
assert result["warnings"][0]["code"] == "server_unavailable"
|
||||
|
||||
|
||||
def test_context_local_read_auto_syncs_from_feed(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "stable")
|
||||
provider.put(make_node("contextnew", 10), "node_created")
|
||||
|
||||
result = mod.call_tool_impl("nap_context", {"paths": ["/notes/contextnew"], "mode": "hybrid"})
|
||||
|
||||
assert provider.count("changes") == 2
|
||||
assert "contextnew" in json.dumps(result)
|
||||
assert result["freshness"]["feed_sync"] == "synced"
|
||||
|
||||
|
||||
def cursorless_complete_index(mod, provider, *names):
|
||||
for index, name in enumerate(names):
|
||||
node = make_node(name, index)
|
||||
provider.put(node, "node_created")
|
||||
mod.index_node(node)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
|
||||
def test_cursorless_index_bootstraps_on_first_search_then_refreshes(feed):
|
||||
mod, provider = feed
|
||||
cursorless_complete_index(mod, provider, "manual")
|
||||
provider.nodes["gapnode"] = make_node("gapnode", 20, content="gapterm") # changed before tracking
|
||||
|
||||
first = mod.call_tool_impl("nap_discover", {"q": "gapterm", "mode": "local"})
|
||||
|
||||
assert [kind for kind, *_ in provider.requests] == ["changes", "list"]
|
||||
assert provider.requests[0][2]["after"] == ["head"]
|
||||
assert [item["node_id"] for item in first["items"]] == ["gapnode"]
|
||||
assert first["freshness"]["feed_sync"] == "tracking_initialized"
|
||||
assert not first.get("warnings")
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == "c1"
|
||||
|
||||
provider.put(make_node("manual", 30, content="manual remoteupdateterm"), "node_updated")
|
||||
throttled = mod.call_tool_impl("nap_discover", {"q": "remoteupdateterm", "mode": "local"})
|
||||
assert throttled["items"] == [] and len(provider.requests) == 2 # no HTTP inside the window
|
||||
|
||||
mod.FEED_AUTO_SYNC_STATE.clear() # the throttle window elapses
|
||||
later = mod.call_tool_impl("nap_discover", {"q": "remoteupdateterm", "mode": "local"})
|
||||
assert [item["node_id"] for item in later["items"]] == ["manual"]
|
||||
assert later["freshness"]["feed_sync"] == "synced"
|
||||
|
||||
|
||||
def test_bootstrap_failure_answers_locally_warns_and_does_not_retry(feed):
|
||||
mod, provider = feed
|
||||
cursorless_complete_index(mod, provider, "manual")
|
||||
provider.fail["changes"] = 503
|
||||
|
||||
first = mod.call_tool_impl("nap_discover", {"q": "manual", "mode": "local"})
|
||||
second = mod.call_tool_impl("nap_discover", {"q": "manual", "mode": "local"})
|
||||
|
||||
for result in (first, second):
|
||||
assert [item["node_id"] for item in result["items"]] == ["manual"]
|
||||
assert [w["code"] for w in result["warnings"]] == ["index_not_tracking"]
|
||||
assert result["warnings"][0]["next_action"] == "nap_index_sync"
|
||||
assert result["freshness"]["stale"] is True
|
||||
assert provider.count("changes") == 1 # one attempt per retry window
|
||||
assert provider.count("list") == 0
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] is None
|
||||
|
||||
|
||||
def test_bootstrap_budget_exhaustion_stays_untracked_and_applies_nothing(feed, monkeypatch):
|
||||
mod, provider = feed
|
||||
cursorless_complete_index(mod, provider, "manual")
|
||||
provider.nodes["gapnode"] = make_node("gapnode", 20, content="gapterm")
|
||||
monkeypatch.setattr(mod, "FEED_AUTO_SYNC_MAX_PAGES", 0)
|
||||
|
||||
result = mod.call_tool_impl("nap_discover", {"q": "gapterm", "mode": "local"})
|
||||
|
||||
assert result["items"] == []
|
||||
assert [w["code"] for w in result["warnings"]] == ["index_not_tracking"]
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] is None
|
||||
|
||||
|
||||
def test_bootstrap_on_old_server_marks_unsupported_without_retry(feed):
|
||||
mod, provider = feed
|
||||
cursorless_complete_index(mod, provider, "manual")
|
||||
provider.fail["changes"] = 404
|
||||
|
||||
first = mod.call_tool_impl("nap_discover", {"q": "manual", "mode": "local"})
|
||||
mod.call_tool_impl("nap_discover", {"q": "manual", "mode": "local"})
|
||||
|
||||
assert first["freshness"]["feed_sync"] == "unsupported"
|
||||
assert not first.get("warnings")
|
||||
assert provider.count("changes") == 1 and provider.count("list") == 0
|
||||
|
||||
|
||||
def test_feed_sync_is_bounded_by_pages_and_resumes(feed):
|
||||
mod, provider = feed
|
||||
seed(mod, provider, "stable")
|
||||
start = len(provider.events)
|
||||
for index in range(3):
|
||||
provider.put(make_node(f"bounded{index}", 10 + index), "node_created")
|
||||
|
||||
first = mod.sync_local_index_from_feed("project-1", max_pages=2, page_limit=1)
|
||||
|
||||
assert (first["status"], first["pages"], first["has_more"]) == ("partial", 2, True)
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == f"c{start + 2}"
|
||||
second = mod.sync_local_index_from_feed("project-1", max_pages=5, page_limit=1)
|
||||
assert second["status"] == "synced"
|
||||
assert local_ids(mod, "bounded2") == ["bounded2"]
|
||||
|
||||
|
||||
def test_explicit_sync_initializes_feed_tracking_for_cursorless_index(feed):
|
||||
mod, provider = feed
|
||||
provider.put(make_node("manual", 1), "node_created")
|
||||
mod.index_node(make_node("manual", 1))
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
result = mod.sync_local_index({})
|
||||
|
||||
assert result["mode"] == "incremental"
|
||||
assert result["feed"] == "initialized"
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] == "c1"
|
||||
provider.delete("manual")
|
||||
mod.sync_local_index({})
|
||||
assert local_ids(mod, "manual") == [] # hard delete observed through the feed
|
||||
@@ -0,0 +1,347 @@
|
||||
"""Public memory-tool regressions with disposable SQLite and an HTTP provider."""
|
||||
|
||||
import copy
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def node(name, minute, *, archived=False):
|
||||
stamp = f"2026-09-01T12:{minute:02d}:00Z"
|
||||
return {
|
||||
"id": name, "project_id": "project-1", "full_path": f"/notes/{name}",
|
||||
"folder_path": "/notes", "name": name, "type": "custom-existing-type",
|
||||
"metadata": {"custom": "retained"}, "tags": [], "aliases": [], "links": [],
|
||||
"content_text": name, "updated_at": stamp,
|
||||
"archived_at": stamp if archived else None,
|
||||
}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def memory(tmp_path, monkeypatch):
|
||||
provider = {"nodes": [], "requests": [], "fail_page": None, "override": None}
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def do_GET(self):
|
||||
if urlsplit(self.path).path.endswith("/changes"):
|
||||
# Model a server without the change feed; the rebuild must not depend on it.
|
||||
self.send_response(404)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'{"error":{"code":"not_found","message":"no change feed"}}')
|
||||
return
|
||||
query = parse_qs(urlsplit(self.path).query)
|
||||
provider["requests"].append(query)
|
||||
offset = int(query.get("cursor", ["0"])[0])
|
||||
if offset == provider["fail_page"] or len(provider["requests"]) > 20:
|
||||
self.send_response(503)
|
||||
self.end_headers()
|
||||
self.wfile.write(b'{"error":{"code":"unavailable","message":"test outage"}}')
|
||||
return
|
||||
rows = copy.deepcopy(provider["nodes"])
|
||||
after = query.get("updated_after", [""])[0]
|
||||
before = query.get("updated_before", [""])[0]
|
||||
rows = [n for n in rows if n["updated_at"] > after
|
||||
and (not before or n["updated_at"] < before)
|
||||
and (query.get("include_archived") == ["true"] or not n["archived_at"])]
|
||||
rows.sort(key=lambda n: (n["updated_at"], n["id"]), reverse=True)
|
||||
limit = int(query.get("limit", ["200"])[0])
|
||||
end = offset + limit
|
||||
page = {"items": rows[offset:end], "next_cursor": str(end) if end < len(rows) else None}
|
||||
if provider["override"] is not None:
|
||||
page = provider["override"](page, query)
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.end_headers()
|
||||
self.wfile.write(json.dumps(page).encode())
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "all")
|
||||
script = pathlib.Path(__file__).resolve().parents[1] / "resources/scripts/napseer_mcp_server.py"
|
||||
monkeypatch.syspath_prepend(str(script.parent))
|
||||
|
||||
def worker():
|
||||
spec = importlib.util.spec_from_file_location("rebuild_recovery_test_worker", script)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
mod.AUTH_DIR = tmp_path / "project-state"
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.lock"
|
||||
mod.BASE_URL = f"http://127.0.0.1:{server.server_port}"
|
||||
mod.TOKEN = "disposable-test-token"
|
||||
mod.AUTH = {"account_id": "test-account"}
|
||||
mod.DEFAULT_PROJECT_ID = "project-1"
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
# Identity is synthetic; the real request, dispatch and index code run.
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
mod.require_unlocked = lambda _name: None
|
||||
mod.resolve_project_id = lambda args: args.get("project_id", "project-1")
|
||||
return mod
|
||||
|
||||
try:
|
||||
yield worker(), provider, worker
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
def rebuild(mod, capsys, *, limit=200):
|
||||
mod.cli_main(["napseer_mcp_server.py", "reindex", "--limit", str(limit)])
|
||||
return json.loads(capsys.readouterr().out)
|
||||
|
||||
|
||||
def found(mod, text):
|
||||
return [n["id"] for n in mod.call_tool_impl(
|
||||
"nap_discover", {"q": text, "mode": "local"}
|
||||
)["items"]]
|
||||
|
||||
|
||||
def seed(mod, provider, capsys):
|
||||
provider["nodes"] = [node("initial", 0)]
|
||||
rebuild(mod, capsys)
|
||||
|
||||
|
||||
def test_rebuild_finds_remote_changes_despite_newer_cached_write(memory, capsys):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
remote, own = node("remoteunique", 5), node("ownwrite", 10)
|
||||
provider["nodes"].extend([remote, own])
|
||||
mod.index_node(own)
|
||||
|
||||
result = rebuild(mod, capsys)
|
||||
|
||||
assert result["graph_complete"] is True
|
||||
assert found(mod, "remoteunique") == ["remoteunique"]
|
||||
|
||||
|
||||
def test_failed_page_preserves_snapshot_and_restart_recovers(memory, capsys):
|
||||
mod, provider, worker = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["nodes"].extend([node("remoteunique", 5), node("newest", 20)])
|
||||
provider["fail_page"] = 1
|
||||
with pytest.raises(mod.SafeToolError):
|
||||
rebuild(mod, capsys, limit=1)
|
||||
assert found(mod, "newest") == []
|
||||
own = node("ownwrite", 30)
|
||||
mod.index_node(own)
|
||||
provider["nodes"].append(own)
|
||||
provider["fail_page"] = None
|
||||
|
||||
restarted = worker()
|
||||
rebuild(restarted, capsys, limit=1)
|
||||
|
||||
assert found(restarted, "remoteunique") == ["remoteunique"]
|
||||
|
||||
|
||||
def test_existing_records_need_no_new_fields_to_rebuild(memory, capsys):
|
||||
mod, provider, _ = memory
|
||||
mod.index_node(node("ownwrite", 30))
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
provider["nodes"] = [node("remoteunique", 5), node("ownwrite", 30)]
|
||||
|
||||
rebuild(mod, capsys)
|
||||
|
||||
assert "updated_after" not in provider["requests"][-1]
|
||||
assert found(mod, "remoteunique") == ["remoteunique"]
|
||||
|
||||
|
||||
def test_apply_failure_rolls_back_batch_and_retry_recovers(memory, capsys):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["nodes"].extend([node("newest", 20), node("failsave", 5)])
|
||||
with mod.index_connect() as conn:
|
||||
conn.execute("CREATE TRIGGER fail_sync BEFORE INSERT ON local_index_nodes "
|
||||
"WHEN NEW.node_id = 'failsave' BEGIN SELECT RAISE(ABORT, 'injected'); END")
|
||||
with pytest.raises(sqlite3.IntegrityError):
|
||||
rebuild(mod, capsys)
|
||||
assert found(mod, "newest") == []
|
||||
assert found(mod, "initial") == ["initial"]
|
||||
assert mod.local_index_diagnostics("project-1")["graph_complete"] is True
|
||||
with mod.index_connect() as conn:
|
||||
conn.execute("DROP TRIGGER fail_sync")
|
||||
|
||||
rebuild(mod, capsys)
|
||||
assert found(mod, "failsave") == ["failsave"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("page", [{}, {"items": {}}, {"items": [], "next_cursor": 3},
|
||||
{"items": [None]}, {"items": [], "has_more": True},
|
||||
{"items": [{"id": "broken"}]},
|
||||
{"items": [{**node("other", 5), "project_id": "project-2"}]}])
|
||||
def test_malformed_rebuild_page_preserves_index_and_returns_safe_failure(memory, capsys, page):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["override"] = lambda *_: page
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as error:
|
||||
rebuild(mod, capsys)
|
||||
|
||||
assert error.value.code == "invalid_index_response"
|
||||
assert found(mod, "initial") == ["initial"]
|
||||
|
||||
|
||||
def test_repeated_cursor_is_rejected_before_mutating_the_index(memory, capsys):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["override"] = lambda *_: {"items": [], "next_cursor": "1"}
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as error:
|
||||
rebuild(mod, capsys)
|
||||
|
||||
assert error.value.code == "invalid_index_response"
|
||||
assert len(provider["requests"]) == 3 # seed + two pages, no unbounded loop
|
||||
assert found(mod, "initial") == ["initial"]
|
||||
|
||||
|
||||
def test_readers_see_previous_snapshot_until_replacement_commits(memory, capsys, monkeypatch):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["nodes"] = [node("replacement", 20), node("second", 10)]
|
||||
entered, release = threading.Event(), threading.Event()
|
||||
errors = []
|
||||
original = mod.index_node
|
||||
|
||||
def paused_write(item, **kwargs):
|
||||
original(item, **kwargs)
|
||||
if item["id"] == "replacement":
|
||||
entered.set()
|
||||
assert release.wait(timeout=3)
|
||||
|
||||
def run():
|
||||
try:
|
||||
mod.cli_main(["napseer_mcp_server.py", "reindex"])
|
||||
except Exception as error:
|
||||
errors.append(error)
|
||||
|
||||
monkeypatch.setattr(mod, "index_node", paused_write)
|
||||
thread = threading.Thread(target=run)
|
||||
thread.start()
|
||||
try:
|
||||
assert entered.wait(timeout=3)
|
||||
assert found(mod, "initial") == ["initial"]
|
||||
assert found(mod, "replacement") == []
|
||||
assert mod.local_index_diagnostics("project-1")["graph_complete"] is True
|
||||
finally:
|
||||
release.set()
|
||||
thread.join(timeout=3)
|
||||
|
||||
assert not thread.is_alive()
|
||||
assert not errors
|
||||
assert json.loads(capsys.readouterr().out)["graph_complete"] is True
|
||||
assert found(mod, "initial") == []
|
||||
assert found(mod, "replacement") == ["replacement"]
|
||||
|
||||
|
||||
def test_cancelled_replacement_rolls_back_and_releases_writer_lock(memory, capsys, monkeypatch):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["nodes"] = [node("replacement", 20), node("second", 10)]
|
||||
cancelled = threading.Event()
|
||||
mod.MCP_REQUEST_LOCAL.cancelled = cancelled
|
||||
original = mod.index_node
|
||||
|
||||
def cancel_after_write(item, **kwargs):
|
||||
original(item, **kwargs)
|
||||
cancelled.set()
|
||||
|
||||
monkeypatch.setattr(mod, "index_node", cancel_after_write)
|
||||
with pytest.raises(mod.SafeToolError) as error:
|
||||
rebuild(mod, capsys)
|
||||
assert error.value.code == "request_cancelled"
|
||||
cancelled.clear()
|
||||
assert found(mod, "initial") == ["initial"]
|
||||
assert found(mod, "replacement") == []
|
||||
assert mod.local_index_diagnostics("project-1")["graph_complete"] is True
|
||||
|
||||
monkeypatch.setattr(mod, "index_node", original)
|
||||
rebuild(mod, capsys)
|
||||
assert found(mod, "replacement") == ["replacement"]
|
||||
|
||||
|
||||
def test_rebuild_preserves_other_projects_and_existing_node_facets(memory, capsys):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
mod.index_node({**node("otherproject", 30), "project_id": "project-2"})
|
||||
kept = node("kept", 5)
|
||||
kept.update(tags=["existing-tag"], aliases=["secondaryhandle"],
|
||||
links=[{"path": "/notes/target", "relation": "custom-relation"}])
|
||||
mod.index_node(kept)
|
||||
assert found(mod, "secondaryhandle") == ["kept"], "baseline before rebuild"
|
||||
provider["nodes"] = [kept, node("target", 6), node("retired", 8, archived=True)]
|
||||
|
||||
result = rebuild(mod, capsys, limit=1)
|
||||
|
||||
assert result["indexed"] == 2
|
||||
assert found(mod, "initial") == []
|
||||
assert found(mod, "retired") == []
|
||||
assert found(mod, "secondaryhandle") == ["kept"]
|
||||
with mod.index_connect() as conn:
|
||||
assert conn.execute("SELECT COUNT(*) FROM local_index_nodes WHERE project_id='project-2'").fetchone()[0] == 1
|
||||
assert conn.execute("SELECT node_type FROM local_index_nodes WHERE node_id='kept'").fetchone()[0] == "custom-existing-type"
|
||||
assert json.loads(conn.execute("SELECT aliases_json FROM local_index_nodes WHERE node_id='kept'").fetchone()[0]) == ["secondaryhandle"]
|
||||
assert conn.execute("SELECT relation FROM local_index_edges WHERE source_node_id='kept'").fetchone()[0] == "custom-relation"
|
||||
assert conn.execute("SELECT tag FROM local_index_node_tags WHERE node_id='kept'").fetchone()[0] == "existing-tag"
|
||||
|
||||
|
||||
def test_valid_empty_inventory_commits_an_empty_complete_project(memory, capsys):
|
||||
mod, provider, _ = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["nodes"] = []
|
||||
|
||||
result = rebuild(mod, capsys)
|
||||
|
||||
assert result["indexed"] == 0
|
||||
assert result["graph_complete"] is True
|
||||
assert mod.local_index_diagnostics("project-1")["count"] == 0
|
||||
|
||||
|
||||
def test_process_exit_during_rebuild_preserves_snapshot_after_restart(memory, capsys):
|
||||
mod, provider, worker = memory
|
||||
seed(mod, provider, capsys)
|
||||
provider["nodes"] = [node("replacement", 20), node("second", 10)]
|
||||
child = r'''
|
||||
import importlib.util, os, pathlib, sys
|
||||
script, state, base = sys.argv[1:]
|
||||
sys.path.insert(0, str(pathlib.Path(script).parent))
|
||||
spec = importlib.util.spec_from_file_location("rebuild_crash_worker", script)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
mod.AUTH_DIR = pathlib.Path(state)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.lock"
|
||||
mod.BASE_URL = base
|
||||
mod.TOKEN = "disposable-test-token"
|
||||
mod.DEFAULT_PROJECT_ID = "project-1"
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
mod.resolve_project_id = lambda _: "project-1"
|
||||
original = mod.index_node
|
||||
def crash_after_write(item, **kwargs):
|
||||
original(item, **kwargs)
|
||||
os._exit(71)
|
||||
mod.index_node = crash_after_write
|
||||
mod.cli_main([script, "reindex"])
|
||||
'''
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-c", child, mod.__file__, str(mod.AUTH_DIR), mod.BASE_URL],
|
||||
capture_output=True, timeout=8, check=False,
|
||||
)
|
||||
assert result.returncode == 71
|
||||
restarted = worker()
|
||||
assert found(restarted, "initial") == ["initial"]
|
||||
assert found(restarted, "replacement") == []
|
||||
assert restarted.local_index_diagnostics("project-1")["graph_complete"] is True
|
||||
rebuild(restarted, capsys)
|
||||
assert found(restarted, "replacement") == ["replacement"]
|
||||
@@ -147,6 +147,34 @@ def test_node_by_path_returns_canonical_identity_uri(mod):
|
||||
assert "Path lookup is index/route resolution only" in result["warnings"][0]
|
||||
|
||||
|
||||
def test_node_by_path_resolves_project_anchor_shortcut(mod):
|
||||
def fake_request(method, path, payload=None, **kwargs):
|
||||
assert method == "GET"
|
||||
assert path.endswith("/nodes/by-path?path=%40project")
|
||||
return {
|
||||
"identity": {
|
||||
"node_id": "anchor-1",
|
||||
"canonical_uri": "nap://napseer/project-1/anchor-1",
|
||||
"project_id": "project-1",
|
||||
"lookup_kind": "shortcut",
|
||||
"lookup_value": "@project",
|
||||
"legacy_full_path": "/routes/project/core",
|
||||
},
|
||||
"route": {
|
||||
"id": "anchor-1",
|
||||
"project_id": "project-1",
|
||||
"full_path": "/routes/project/core",
|
||||
},
|
||||
}
|
||||
|
||||
mod.request_json = fake_request
|
||||
result = mod.node_by_path({"shortcut": "@Project"})
|
||||
|
||||
assert result["identity"]["lookup_kind"] == "shortcut"
|
||||
assert result["identity"]["lookup_value"] == "@project"
|
||||
assert result["route"]["full_path"] == "/routes/project/core"
|
||||
|
||||
|
||||
def test_path_resolver_then_read_uses_node_id_route(mod):
|
||||
calls = []
|
||||
|
||||
@@ -255,7 +283,7 @@ def test_node_get_rejects_path_reads(mod):
|
||||
|
||||
def test_tools_include_node_by_path_descriptor(mod):
|
||||
tool_by_name = {item["name"]: item for item in mod.tools()}
|
||||
assert len(tool_by_name) == 51
|
||||
assert len(tool_by_name) == 60
|
||||
assert len(tool_by_name) == len(mod.tools())
|
||||
|
||||
get_tool = next(item for item in mod.tools() if item["name"] == "nap_node_get")
|
||||
@@ -263,9 +291,19 @@ def test_tools_include_node_by_path_descriptor(mod):
|
||||
assert "node_id" in get_tool["inputSchema"]["properties"]
|
||||
assert "uri" in get_tool["inputSchema"]["properties"]
|
||||
assert "path" not in get_tool["inputSchema"]["properties"]
|
||||
assert "node_id returned by nap_discover" in get_tool["description"]
|
||||
|
||||
discover_tool = next(item for item in mod.tools() if item["name"] == "nap_discover")
|
||||
assert "node_id on every row" in discover_tool["description"]
|
||||
assert "directly to nap_node_get" in discover_tool["description"]
|
||||
|
||||
tool = next(item for item in mod.tools() if item["name"] == "nap_node_by_path")
|
||||
assert tool["inputSchema"]["required"] == ["path"]
|
||||
assert tool["inputSchema"].get("required") is None
|
||||
assert tool["inputSchema"]["oneOf"] == [
|
||||
{"required": ["path"]},
|
||||
{"required": ["shortcut"]},
|
||||
]
|
||||
assert "shortcut" in tool["inputSchema"]["properties"]
|
||||
assert "preview_chars" not in tool["inputSchema"]["properties"]
|
||||
assert "does not read full node content" in tool["description"]
|
||||
|
||||
@@ -425,9 +463,9 @@ def test_tee_and_bulk_create_by_path_update_by_identity(mod):
|
||||
"content_text": payload.get("content_text", existing["content_text"]),
|
||||
}
|
||||
mod.index_node = lambda node: None
|
||||
mod.acquire_project_lock = lambda args: {"id": "lock-1", "lease_token": "lease-1"}
|
||||
mod.acquire_project_lock = lambda args, **kwargs: {"id": "lock-1", "lease_token": "lease-1"}
|
||||
mod.lock_headers = lambda lock: {"X-Lock": lock["id"]}
|
||||
mod.release_project_lock = lambda args: None
|
||||
mod.release_project_lock = lambda args, **kwargs: None
|
||||
mod.request_json = lambda method, path, payload=None, **kwargs: writes.append((method, path, payload, ())) or {
|
||||
**existing,
|
||||
"content_text": payload.get("content_text", existing["content_text"]) if payload else existing["content_text"],
|
||||
@@ -591,6 +629,39 @@ def test_context_and_related_use_identity_inputs(mod):
|
||||
mod.find_related_nodes({"path": "/notes/root"})
|
||||
|
||||
|
||||
def test_hybrid_context_resolves_path_inside_complete_local_snapshot(mod):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
state_dir = pathlib.Path(tmp)
|
||||
mod.AUTH_DIR = state_dir
|
||||
mod.INDEX_PATH = state_dir / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = state_dir / "index.lock"
|
||||
indexed = node("/notes/local-root", "active", node_type="note")
|
||||
indexed["id"] = "local-root-123"
|
||||
mod.index_node(indexed)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
def unexpected_request(*_args, **_kwargs):
|
||||
raise AssertionError("complete local context must not make an HTTP request")
|
||||
|
||||
mod.request_json = unexpected_request
|
||||
result = mod.get_memory_context({
|
||||
"path": "/notes/local-root",
|
||||
"depth": 0,
|
||||
"mode": "hybrid",
|
||||
})
|
||||
|
||||
assert result["source"] == "local_index"
|
||||
assert result["seed_node_ids"] == ["local-root-123"]
|
||||
assert result["items"][0]["full_path"] == "/notes/local-root"
|
||||
with mod.index_connect() as conn:
|
||||
plan = conn.execute(
|
||||
"EXPLAIN QUERY PLAN SELECT node_id FROM local_index_nodes "
|
||||
"WHERE project_id = ? AND full_path = ? AND archived = 0",
|
||||
("project-1", "/notes/local-root"),
|
||||
).fetchall()
|
||||
assert any("local_index_nodes_project_path" in row["detail"] for row in plan)
|
||||
|
||||
|
||||
def test_local_helper_ui_forms_submit_identity_for_existing_node_actions():
|
||||
source = (pathlib.Path(__file__).resolve().parents[1] / "resources" / "scripts" / "napseer_mcp_server.py").read_text()
|
||||
assert "name='node_id' value='{html.escape(item.get('id') or '')}'" in source
|
||||
@@ -643,6 +714,7 @@ def test_recent_memory_defaults_to_paths(mod):
|
||||
assert result["ok"] is True
|
||||
assert result["view"] == "paths"
|
||||
assert result["items"] == [{
|
||||
"node_id": "id-recent-title",
|
||||
"full_path": "/notes/recent-title",
|
||||
"type": "note",
|
||||
"status": "active",
|
||||
@@ -671,6 +743,7 @@ def test_recent_memory_accepts_date_aliases_and_returns_dates(mod):
|
||||
mod.request_json = fake_request
|
||||
result = mod.recent_memory({"since": "2026-05-05", "until": "2026-05-06"})
|
||||
assert result["items"] == [{
|
||||
"node_id": "id-recent-title",
|
||||
"full_path": "/notes/recent-title",
|
||||
"type": "note",
|
||||
"status": "active",
|
||||
@@ -699,6 +772,7 @@ def test_find_defaults_to_paths_and_requests_compact_rest_view(mod):
|
||||
assert result["ok"] is True
|
||||
assert result["view"] == "paths"
|
||||
assert result["items"] == [{
|
||||
"node_id": "id-found",
|
||||
"full_path": "/plans/found",
|
||||
"type": "plan",
|
||||
"status": "planned",
|
||||
@@ -721,6 +795,8 @@ def test_discover_uses_search_when_keywords_present(mod):
|
||||
assert calls and calls[-1]["q"] == "gateway relay"
|
||||
assert calls[-1]["view"] == "summary"
|
||||
assert "date" in result["awareness"]["date_fields_in_rows"]
|
||||
assert "item.node_id" in result["awareness"]["next_step"]
|
||||
assert "outside discovery" in result["awareness"]["next_step"]
|
||||
|
||||
|
||||
def test_classify_query_points_exact_paths_to_context(mod):
|
||||
@@ -748,7 +824,11 @@ def test_title_view_requests_summary_rest_view_without_content(mod):
|
||||
result = mod.list_project_nodes({"view": "titles"})
|
||||
assert result["ok"] is True
|
||||
assert result["view"] == "titles"
|
||||
assert result["items"] == [{"title": "Readable Title", "full_path": "/notes/titled"}]
|
||||
assert result["items"] == [{
|
||||
"node_id": "id-titled",
|
||||
"title": "Readable Title",
|
||||
"full_path": "/notes/titled",
|
||||
}]
|
||||
assert "content_text" not in result["items"][0]
|
||||
|
||||
|
||||
@@ -781,7 +861,8 @@ def test_index_status_reports_sync_high_water_mark(mod):
|
||||
status = mod.index_status({})
|
||||
assert status["count"] == 1
|
||||
assert status["latest_node_updated_at"] == "2026-05-06T12:00:00Z"
|
||||
assert status["sync_recommended"] is False
|
||||
assert status["graph_complete"] is False
|
||||
assert status["sync_recommended"] is True
|
||||
|
||||
|
||||
def test_index_sync_fetches_incremental_changes_and_removes_archived_nodes(mod):
|
||||
@@ -794,6 +875,7 @@ def test_index_sync_fetches_incremental_changes_and_removes_archived_nodes(mod):
|
||||
existing["id"] = "existing-node"
|
||||
existing["updated_at"] = "2026-05-06T12:00:00Z"
|
||||
mod.index_node(existing)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
fresh = node("/notes/fresh", "active", content="fresh sync needle")
|
||||
fresh["id"] = "fresh-node"
|
||||
@@ -804,6 +886,9 @@ def test_index_sync_fetches_incremental_changes_and_removes_archived_nodes(mod):
|
||||
calls = []
|
||||
|
||||
def fake_request(method, path, payload=None, **kwargs):
|
||||
if "/changes?" in path:
|
||||
# Server without the change feed: sync uses the timestamp path.
|
||||
raise mod.SafeToolError("not_found", "no change feed", status=404)
|
||||
calls.append(path)
|
||||
assert method == "GET"
|
||||
assert "view=full" in path
|
||||
@@ -817,6 +902,8 @@ def test_index_sync_fetches_incremental_changes_and_removes_archived_nodes(mod):
|
||||
assert result["seen"] == 2
|
||||
assert result["indexed"] == 1
|
||||
assert result["removed"] == 1
|
||||
assert result["mode"] == "incremental"
|
||||
assert result["index"]["graph_complete"] is True
|
||||
assert calls
|
||||
local = mod.search_local_index({"q": "fresh needle"})
|
||||
assert [item["id"] for item in local["items"]] == ["fresh-node"]
|
||||
|
||||
@@ -116,6 +116,8 @@ def run():
|
||||
mod = load_module()
|
||||
test_state = tempfile.TemporaryDirectory()
|
||||
mod.AUTH_PATH = pathlib.Path(test_state.name) / "auth.json"
|
||||
mod.MEMORY_SECRET_VAULT_PATH = pathlib.Path(test_state.name) / "memory-secrets.vault.json"
|
||||
mod.LOCAL_VAULT_KEY_PATH = pathlib.Path(test_state.name) / "vault.local-key"
|
||||
mod.AUTH = {"account_id": ACCOUNT_ID, "token": "token", "token_expires_at": "later"}
|
||||
mod.PROJECT_VAULT_PASSPHRASE = PASSPHRASE
|
||||
mod.DEFAULT_PROJECT_ID = "project-1"
|
||||
@@ -194,8 +196,13 @@ def run():
|
||||
"precondition": {"revision": "r1", "read_fingerprint": "fp1"},
|
||||
"content_op": {"op": "replace_all", "content_text": "plaintext"},
|
||||
})
|
||||
except RuntimeError as exc:
|
||||
assert "cannot safely patch encrypted content or metadata" in str(exc)
|
||||
except mod.SafeToolError as exc:
|
||||
assert exc.code == "encrypted_patch_requires_full_replacement"
|
||||
assert "cannot safely patch client-E2E encrypted content or metadata" in str(exc)
|
||||
assert "nap_bulk" in str(exc)
|
||||
assert "fresh read" in str(exc)
|
||||
assert "nap_tee" not in str(exc)
|
||||
assert "nap_patch" not in str(exc)
|
||||
else:
|
||||
raise AssertionError("nap_node_patch content updates must fail closed for encrypted projects")
|
||||
|
||||
@@ -296,7 +303,14 @@ def run():
|
||||
active = mod.list_project_nodes({"folder_path": "/plans", "active_only": True})
|
||||
assert [item["full_path"] for item in active["items"]] == ["/plans/active"]
|
||||
status_filtered = mod.list_project_nodes({"folder_path": "/plans", "status": ["completed"], "view": "paths"})
|
||||
assert status_filtered["items"] == [{"full_path": "/plans/done", "type": "note", "status": "completed"}]
|
||||
assert status_filtered["items"] == [{
|
||||
"node_id": "done",
|
||||
"full_path": "/plans/done",
|
||||
"type": "note",
|
||||
# A generic note under /plans reports its effective type as well.
|
||||
"effective_type": "plan",
|
||||
"status": "completed",
|
||||
}]
|
||||
|
||||
def search_response(method, path, payload=None, **kwargs):
|
||||
assert "q=" in path
|
||||
@@ -383,9 +397,9 @@ def run():
|
||||
assert patch_body["encrypted_content_envelope"]["aad_subject"] == "node_id:node-1"
|
||||
|
||||
bulk_writes = []
|
||||
mod.acquire_project_lock = lambda args: {"id": "lock-1", "lease_token": "lease-1"}
|
||||
mod.acquire_project_lock = lambda args, **kwargs: {"id": "lock-1", "lease_token": "lease-1"}
|
||||
mod.lock_headers = lambda lock: {"X-Lock": lock["id"]}
|
||||
mod.release_project_lock = lambda args: None
|
||||
mod.release_project_lock = lambda args, **kwargs: None
|
||||
mod.try_get_node_by_path = lambda path, allow_agent=False: None
|
||||
mod.request_json = lambda method, path, payload=None, **kwargs: bulk_writes.append((method, path, payload, kwargs)) or {
|
||||
"id": "node-bulk",
|
||||
@@ -462,8 +476,24 @@ def run():
|
||||
indexed["metadata"] = {"private": "secret metadata"}
|
||||
mod.index_node(indexed)
|
||||
assert mod.search_local_index({"q": "alpha", "limit": 5})["items"]
|
||||
assert mod.search_local_index({"q": "needle", "limit": 5})["items"] == []
|
||||
assert mod.search_local_index({"q": "metadata", "limit": 5})["items"] == []
|
||||
# E2E body/metadata text never reaches the on-disk index; with the
|
||||
# memory secret unlocked it is found only via the encrypted shard.
|
||||
for term in ("needle", "metadata"):
|
||||
hits = mod.search_local_index({"q": term, "limit": 5})["items"]
|
||||
assert hits and all(item.get("search_source") == "local_e2e" for item in hits)
|
||||
mod.e2e_shards_close(flush=True)
|
||||
for path in state_dir.iterdir():
|
||||
if path.is_file():
|
||||
raw = path.read_bytes()
|
||||
assert b"secret needle" not in raw and b"secret metadata" not in raw, path.name
|
||||
mod.MEMORY_SECRET_CACHE.clear()
|
||||
mod.ACTIVE_MEMORY_SECRET_VERSIONS.clear()
|
||||
mod.PROJECT_VAULT_PASSPHRASE = None
|
||||
# Earlier passphrase unwraps persisted the secret locally; remove it.
|
||||
mod.memory_secret_vault_forget("project-1")
|
||||
locked = mod.search_local_index({"q": "needle", "limit": 5})
|
||||
assert locked["items"] == []
|
||||
assert [warning["code"] for warning in locked["warnings"]] == ["e2e_index_not_provisioned"]
|
||||
|
||||
print("ok: local MCP memory node encryption smoke passed")
|
||||
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
"""Contract tests for the reduced, bounded agent-memory surface."""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
@@ -24,15 +25,19 @@ def load_module():
|
||||
def mod(monkeypatch):
|
||||
monkeypatch.delenv("NAPSEER_TOOL_PROFILES", raising=False)
|
||||
module = load_module()
|
||||
module._auth_state = tempfile.TemporaryDirectory()
|
||||
module.AUTH_PATH = pathlib.Path(module._auth_state.name) / "auth.json"
|
||||
module.AUTH = {"account_id": "acct-1", "token": "token"}
|
||||
module.CONFIGURED_TOOL_PROFILES = None
|
||||
return module
|
||||
|
||||
|
||||
def test_default_surface_is_exactly_the_core_fourteen(mod):
|
||||
def test_default_surface_includes_knowledge_files_and_coordination(mod):
|
||||
assert [tool["name"] for tool in mod.tools()] == [
|
||||
"nap_apropos", "nap_man", "nap_doctor", "nap_whoami",
|
||||
"nap_discover", "nap_context", "nap_node_by_path", "nap_node_get",
|
||||
"nap_create_node", "nap_node_patch", "nap_bulk", "nap_ln",
|
||||
"nap_operation_list", "nap_operation", "nap_file_list", "nap_file_upload", "nap_file_download", "nap_file_archive",
|
||||
"nap_changes_since", "nap_discover", "nap_context", "nap_node_by_path", "nap_node_get",
|
||||
"nap_create_node", "nap_node_patch", "nap_bulk", "nap_batch", "nap_ln",
|
||||
"nap_mv", "nap_rm",
|
||||
]
|
||||
|
||||
@@ -64,6 +69,67 @@ def test_pruned_and_profile_gated_calls_are_actionable(mod):
|
||||
assert "workflow" in gated.value.safe_message
|
||||
|
||||
|
||||
def test_removed_write_tools_and_unknown_manual_lookups_are_actionable(mod):
|
||||
tee = mod.man_tool({"tool": "nap_tee"})
|
||||
assert tee["removed"] is True
|
||||
assert "nap_create_node" in tee["replacement"]
|
||||
assert "nap_bulk" in tee["replacement"]
|
||||
|
||||
patch = mod.man_tool({"tool": "nap_patch"})
|
||||
assert patch["removed"] is True
|
||||
assert "nap_node_patch" in patch["replacement"]
|
||||
assert "nap_bulk" in patch["replacement"]
|
||||
|
||||
for name in ("nap_tee", "nap_patch"):
|
||||
with pytest.raises(mod.SafeToolError) as removed:
|
||||
mod.call_tool_impl(name, {})
|
||||
assert removed.value.code == "tool_removed"
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as unknown:
|
||||
mod.man_tool({"tool": "not_a_real_tool"})
|
||||
assert unknown.value.code == "unknown_tool"
|
||||
assert "nap_apropos" in unknown.value.safe_message
|
||||
|
||||
|
||||
def test_node_patch_rejects_malformed_arguments_before_project_or_node_reads(mod):
|
||||
mod.resolve_project_id = lambda args: (_ for _ in ()).throw(
|
||||
AssertionError("invalid arguments must fail before project resolution")
|
||||
)
|
||||
mod.get_node_by_id = lambda args: (_ for _ in ()).throw(
|
||||
AssertionError("invalid arguments must fail before node reads")
|
||||
)
|
||||
base = {
|
||||
"node_id": "node-1",
|
||||
"precondition": {"revision": "r1", "read_fingerprint": "fp1"},
|
||||
}
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as legacy_metadata:
|
||||
mod.guarded_node_patch({**base, "metadata_patch": {"set": {"status": "done"}}})
|
||||
assert legacy_metadata.value.code == "invalid_arguments"
|
||||
assert "metadata_patch" in legacy_metadata.value.safe_message
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as legacy_content:
|
||||
mod.guarded_node_patch({
|
||||
**base,
|
||||
"content_op": {"kind": "replace_all", "content_text": "updated"},
|
||||
})
|
||||
assert legacy_content.value.code == "invalid_arguments"
|
||||
assert "use 'op'" in legacy_content.value.safe_message
|
||||
|
||||
|
||||
def test_node_patch_argument_validation_accepts_canonical_shapes(mod):
|
||||
mod.validate_node_patch_arguments({
|
||||
"node_id": "node-1",
|
||||
"precondition": {"revision": "r1", "read_fingerprint": "fp1"},
|
||||
"content_op": {"op": "replace_all", "content_text": "updated"},
|
||||
"merge_metadata": {"status": "done"},
|
||||
"tag_ops": {"add": ["verified"]},
|
||||
"link_ops": [
|
||||
{"op": "add", "path": "/plans/example", "relation": "implements"}
|
||||
],
|
||||
})
|
||||
|
||||
|
||||
def test_context_and_node_read_schemas_are_bounded(mod):
|
||||
by_name = {tool["name"]: tool for tool in mod.all_tools()}
|
||||
context = by_name["nap_context"]["inputSchema"]["properties"]
|
||||
@@ -73,6 +139,10 @@ def test_context_and_node_read_schemas_are_bounded(mod):
|
||||
assert "relations" in context
|
||||
assert "max_edges" in context
|
||||
assert "max_per_level" in context
|
||||
assert context["node_ids"]["maxItems"] == 10
|
||||
assert context["paths"]["maxItems"] == 10
|
||||
assert context["shortcuts"]["maxItems"] == 10
|
||||
assert context["uris"]["maxItems"] == 10
|
||||
|
||||
node_get = by_name["nap_node_get"]["inputSchema"]["properties"]
|
||||
assert node_get["view"]["enum"] == ["content", "outline", "edit"]
|
||||
@@ -83,10 +153,47 @@ def test_tool_text_does_not_duplicate_large_structured_payload(mod):
|
||||
payload = {"ok": True, "items": [{"content_text": "x" * 20_000}]}
|
||||
result = mod.tool_result(payload)
|
||||
assert result["structuredContent"] == payload
|
||||
assert len(result["content"][0]["text"]) < 200
|
||||
assert len(result["content"][0]["text"]) < 2_000
|
||||
assert "20000" not in result["content"][0]["text"]
|
||||
|
||||
|
||||
def test_tool_text_keeps_small_structured_results_useful_for_text_only_clients(mod):
|
||||
payload = {
|
||||
"status": "current",
|
||||
"full_path": "/docs/useful",
|
||||
"view": "content",
|
||||
"content_text": "A small useful body.",
|
||||
}
|
||||
|
||||
result = mod.tool_result(payload)
|
||||
|
||||
assert json.loads(result["content"][0]["text"]) == payload
|
||||
|
||||
|
||||
def test_large_graph_text_fallback_contains_bounded_node_and_edge_previews(mod):
|
||||
payload = {
|
||||
"ok": True,
|
||||
"nodes": [
|
||||
{"id": f"node-{index}", "full_path": f"/docs/{index}", "title": f"Node {index}"}
|
||||
for index in range(100)
|
||||
],
|
||||
"edges": [
|
||||
{"source": f"node-{index}", "target": f"node-{index + 1}", "relation": "references"}
|
||||
for index in range(99)
|
||||
],
|
||||
"warnings": ["Use first-class links."],
|
||||
}
|
||||
|
||||
fallback = json.loads(mod.tool_result(payload)["content"][0]["text"])
|
||||
|
||||
assert fallback["nodes_count"] == 100
|
||||
assert fallback["edges_count"] == 99
|
||||
assert len(fallback["nodes_preview"]) == 5
|
||||
assert len(fallback["edges_preview"]) == 5
|
||||
assert fallback["warnings"] == ["Use first-class links."]
|
||||
assert fallback["structured_result"] is True
|
||||
|
||||
|
||||
def test_local_context_returns_title_radius_without_node_bodies(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_state = state
|
||||
@@ -123,6 +230,315 @@ def test_local_context_returns_title_radius_without_node_bodies(mod):
|
||||
assert result["source"] == "local_index"
|
||||
|
||||
|
||||
def test_local_context_resolves_anchor_shortcut_without_extra_tool(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_shortcut_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
mod.index_node({
|
||||
"id": "anchor", "project_id": "project-1",
|
||||
"full_path": "/routes/project/core", "folder_path": "/routes/project",
|
||||
"name": "core", "type": "anchor_route", "aliases": ["shortcut.project"],
|
||||
"metadata": {"title": "Project"}, "tags": [],
|
||||
"links": [{"node_id": "definition", "relation": "defines"}],
|
||||
"content_text": "route body", "updated_at": "2026-08-16T12:00:00Z",
|
||||
})
|
||||
mod.index_node({
|
||||
"id": "definition", "project_id": "project-1",
|
||||
"full_path": "/documentation/product/what-we-are-building",
|
||||
"folder_path": "/documentation/product", "name": "what-we-are-building",
|
||||
"type": "definition", "aliases": [], "metadata": {"title": "Product"},
|
||||
"tags": [], "links": [], "content_text": "definition body",
|
||||
"updated_at": "2026-08-16T12:01:00Z",
|
||||
})
|
||||
|
||||
result = mod.get_memory_context({
|
||||
"shortcut": "@Project", "mode": "local", "depth": 1,
|
||||
"direction": "outgoing",
|
||||
})
|
||||
|
||||
assert result["seed_node_ids"] == ["anchor"]
|
||||
assert [item["id"] for item in result["nodes"]] == ["anchor", "definition"]
|
||||
assert all("content_text" not in item for item in result["nodes"])
|
||||
|
||||
|
||||
def test_duplicate_local_anchor_shortcuts_fail_closed(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_duplicate_shortcut_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
for index in range(2):
|
||||
mod.index_node({
|
||||
"id": f"anchor-{index}", "project_id": "project-1",
|
||||
"full_path": f"/routes/project/{index}", "folder_path": "/routes/project",
|
||||
"name": str(index), "type": "anchor_route", "aliases": ["shortcut.project"],
|
||||
"metadata": {}, "tags": [], "links": [], "content_text": "",
|
||||
"updated_at": f"2026-08-16T12:0{index}:00Z",
|
||||
})
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as ambiguous:
|
||||
mod.get_memory_context({"shortcut": "@project", "mode": "local"})
|
||||
|
||||
assert ambiguous.value.code == "anchor_shortcut_ambiguous"
|
||||
|
||||
|
||||
def test_anchor_integrity_reports_budgets_duplicates_and_broken_links(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_anchor_health_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
for index in range(2):
|
||||
mod.index_node({
|
||||
"id": f"anchor-{index}", "project_id": "project-1",
|
||||
"full_path": f"/routes/project/{index}", "folder_path": "/routes/project",
|
||||
"name": str(index), "type": "anchor_route", "aliases": ["shortcut.project"],
|
||||
"metadata": {}, "tags": [],
|
||||
"links": ([{"path": "/missing", "relation": "routes-to"}] if index == 0 else []),
|
||||
"content_text": "", "updated_at": f"2026-08-16T12:0{index}:00Z",
|
||||
})
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
report = mod.local_anchor_integrity()
|
||||
|
||||
assert report["ok"] is False
|
||||
assert report["graph_complete"] is True
|
||||
assert report["duplicate_shortcuts"][0]["shortcut"] == "@project"
|
||||
assert report["broken_anchor_links"] == [{
|
||||
"source": "/routes/project/0",
|
||||
"target": "/missing",
|
||||
"relation": "routes-to",
|
||||
}]
|
||||
|
||||
|
||||
def test_local_context_reports_partial_and_total_missing_seeds_actionably(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_missing_seed_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
mod.index_node({
|
||||
"id": "root", "project_id": "project-1", "full_path": "/docs/root",
|
||||
"folder_path": "/docs", "name": "root", "type": "note",
|
||||
"metadata": {"title": "Root"}, "tags": [], "links": [],
|
||||
"content_text": "body", "updated_at": "2026-08-16T12:00:00Z",
|
||||
})
|
||||
|
||||
partial = mod.get_memory_context({
|
||||
"node_ids": ["root", "missing"], "mode": "local", "depth": 0,
|
||||
})
|
||||
|
||||
assert partial["requested_seed_node_ids"] == ["root", "missing"]
|
||||
assert partial["seed_node_ids"] == ["root"]
|
||||
assert partial["missing_seed_node_ids"] == ["missing"]
|
||||
assert any("active project" in warning for warning in partial["warnings"])
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as missing:
|
||||
mod.get_memory_context({"node_id": "missing", "mode": "local"})
|
||||
assert missing.value.code == "context_seed_not_found"
|
||||
assert "nap_whoami" in missing.value.safe_message
|
||||
assert "nap_discover" in missing.value.safe_message
|
||||
|
||||
|
||||
def test_context_translates_server_not_found_into_seed_diagnostic(mod):
|
||||
def missing_request(*_args, **_kwargs):
|
||||
raise mod.SafeToolError(
|
||||
"http_404",
|
||||
"Napseer GET request failed with HTTP 404.",
|
||||
status=404,
|
||||
service_code="not_found",
|
||||
)
|
||||
|
||||
mod.request_json = missing_request
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as missing:
|
||||
mod.get_memory_context({"node_id": "missing", "mode": "server"})
|
||||
|
||||
assert missing.value.code == "context_seed_not_found"
|
||||
assert missing.value.status == 404
|
||||
assert mod.safe_exception_payload(missing.value)["service_code"] == "not_found"
|
||||
|
||||
|
||||
def test_context_rejects_cross_project_uri_before_graph_reads(mod):
|
||||
with pytest.raises(mod.SafeToolError) as mismatch:
|
||||
mod.get_memory_context({
|
||||
"uri": "nap://other-project/project-2/node-1",
|
||||
"mode": "server",
|
||||
})
|
||||
|
||||
assert mismatch.value.code == "context_project_mismatch"
|
||||
assert "nap_whoami" in mismatch.value.safe_message
|
||||
|
||||
|
||||
def test_local_context_projects_requested_view_and_bounds_frontier(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_view_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
children = []
|
||||
for index in range(5):
|
||||
children.append({
|
||||
"id": f"child-{index}", "project_id": "project-1",
|
||||
"full_path": f"/docs/child-{index}", "folder_path": "/docs",
|
||||
"name": f"child-{index}", "type": "note", "metadata": {},
|
||||
"tags": [], "links": [], "content_text": "",
|
||||
"updated_at": f"2026-08-16T12:0{index + 1}:00Z",
|
||||
})
|
||||
mod.index_node({
|
||||
"id": "root", "project_id": "project-1", "full_path": "/docs/root",
|
||||
"folder_path": "/docs", "name": "root", "type": "note",
|
||||
"metadata": {"title": "Root"}, "tags": [],
|
||||
"links": [
|
||||
{"node_id": child["id"], "relation": "references"}
|
||||
for child in children
|
||||
],
|
||||
"content_text": "", "updated_at": "2026-08-16T12:00:00Z",
|
||||
})
|
||||
for child in children:
|
||||
mod.index_node(child)
|
||||
|
||||
result = mod.get_memory_context({
|
||||
"node_id": "root", "mode": "local", "depth": 1,
|
||||
"max_nodes": 2, "max_per_level": 1, "view": "paths",
|
||||
})
|
||||
|
||||
assert result["view"] == "paths"
|
||||
assert "title" not in result["nodes"][0]
|
||||
assert set(result["nodes"][0]) == {
|
||||
"id", "full_path", "type", "status", "updated_at", "depth"
|
||||
}
|
||||
assert len(result["frontier"]) <= 1
|
||||
assert "max_frontier" in result["truncation_reasons"]
|
||||
assert result["node_count"] == 2
|
||||
assert result["edge_count"] == 1
|
||||
|
||||
|
||||
def test_context_explains_empty_first_class_graph(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_empty_graph_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
mod.index_node({
|
||||
"id": "root", "project_id": "project-1", "full_path": "/docs/root",
|
||||
"folder_path": "/docs", "name": "root", "type": "note",
|
||||
"metadata": {"title": "Root"}, "tags": [], "links": [],
|
||||
"content_text": "mentions /docs/other only in prose",
|
||||
"updated_at": "2026-08-16T12:00:00Z",
|
||||
})
|
||||
|
||||
result = mod.get_memory_context({"node_id": "root", "mode": "local", "depth": 1})
|
||||
|
||||
assert any("Prose mentions are not traversed" in warning for warning in result["warnings"])
|
||||
assert "nap_ln" in result["next_action"]
|
||||
|
||||
|
||||
def test_hybrid_context_falls_back_until_local_graph_coverage_is_complete(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_fallback_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
mod.index_node({
|
||||
"id": "root", "project_id": "project-1", "full_path": "/docs/root",
|
||||
"folder_path": "/docs", "name": "root", "type": "note",
|
||||
"metadata": {"title": "Root"}, "tags": [], "links": [],
|
||||
"content_text": "body", "updated_at": "2026-08-16T12:00:00Z",
|
||||
})
|
||||
calls = []
|
||||
|
||||
def server_context(_args, seed_ids):
|
||||
calls.append(seed_ids)
|
||||
return {"ok": True, "nodes": [{"id": "root"}], "edges": [], "source": "server"}
|
||||
|
||||
mod.server_memory_context = server_context
|
||||
|
||||
result = mod.get_memory_context({"node_id": "root", "mode": "hybrid"})
|
||||
|
||||
assert result["source"] == "server"
|
||||
assert result["local_fallback"]
|
||||
assert calls == [["root"]]
|
||||
|
||||
|
||||
def test_local_context_does_not_emit_edges_beyond_requested_depth(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_depth_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
nodes = [
|
||||
{
|
||||
"id": "root", "project_id": "project-1", "full_path": "/docs/root",
|
||||
"folder_path": "/docs", "name": "root", "type": "note", "metadata": {},
|
||||
"tags": [], "links": [
|
||||
{"node_id": "left", "relation": "references"},
|
||||
{"node_id": "right", "relation": "references"},
|
||||
], "content_text": "", "updated_at": "2026-08-16T12:00:00Z",
|
||||
},
|
||||
{
|
||||
"id": "left", "project_id": "project-1", "full_path": "/docs/left",
|
||||
"folder_path": "/docs", "name": "left", "type": "note", "metadata": {},
|
||||
"tags": [], "links": [{"node_id": "right", "relation": "explains"}],
|
||||
"content_text": "", "updated_at": "2026-08-16T12:01:00Z",
|
||||
},
|
||||
{
|
||||
"id": "right", "project_id": "project-1", "full_path": "/docs/right",
|
||||
"folder_path": "/docs", "name": "right", "type": "note", "metadata": {},
|
||||
"tags": [], "links": [], "content_text": "",
|
||||
"updated_at": "2026-08-16T12:02:00Z",
|
||||
},
|
||||
]
|
||||
for item in nodes:
|
||||
mod.index_node(item)
|
||||
|
||||
result = mod.get_memory_context({
|
||||
"node_id": "root", "mode": "local", "depth": 1, "direction": "both",
|
||||
})
|
||||
|
||||
assert {item["id"] for item in result["nodes"]} == {"root", "left", "right"}
|
||||
assert result["edges"] == [
|
||||
{"source": "root", "target": "left", "relation": "references"},
|
||||
{"source": "root", "target": "right", "relation": "references"},
|
||||
]
|
||||
|
||||
|
||||
def test_index_sync_upgrades_pre_graph_index_with_full_reindex(mod):
|
||||
state = tempfile.TemporaryDirectory()
|
||||
mod._surface_upgrade_state = state
|
||||
mod.AUTH_DIR = pathlib.Path(state.name)
|
||||
mod.INDEX_PATH = mod.AUTH_DIR / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = mod.AUTH_DIR / "index.sqlite.lock"
|
||||
mod.index_node({
|
||||
"id": "old", "project_id": "project-1", "full_path": "/docs/old",
|
||||
"folder_path": "/docs", "name": "old", "type": "note",
|
||||
"metadata": {}, "tags": [], "links": [], "content_text": "old",
|
||||
"updated_at": "2026-08-16T12:00:00Z",
|
||||
})
|
||||
fresh = {
|
||||
"id": "fresh", "project_id": "project-1", "full_path": "/docs/fresh",
|
||||
"folder_path": "/docs", "name": "fresh", "type": "note",
|
||||
"metadata": {}, "tags": [],
|
||||
"links": [{"node_id": "target", "relation": "references"}],
|
||||
"content_text": "fresh", "updated_at": "2026-08-16T12:01:00Z",
|
||||
}
|
||||
mod.request_json = lambda *_args, **_kwargs: {"items": [fresh], "next_cursor": None}
|
||||
|
||||
result = mod.sync_local_index({})
|
||||
|
||||
assert result["mode"] == "full_reindex"
|
||||
assert result["reason"] == "local_graph_index_incomplete"
|
||||
assert result["graph_complete"] is True
|
||||
assert result["index"]["graph_complete"] is True
|
||||
with mod.index_connect() as conn:
|
||||
assert conn.execute(
|
||||
"SELECT COUNT(*) FROM local_index_edges WHERE project_id = ?",
|
||||
("project-1",),
|
||||
).fetchone()[0] == 1
|
||||
|
||||
|
||||
def test_discovery_sort_and_returned_item_facets(mod):
|
||||
mod.list_project_nodes = lambda args: {
|
||||
"items": [
|
||||
|
||||
@@ -0,0 +1,721 @@
|
||||
"""Regressions for write idempotency, stale local reads, atomic claim-next,
|
||||
indexed link counts, and committed .napseer-tools.json tool profiles."""
|
||||
|
||||
import importlib.util
|
||||
import io
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import urllib.error
|
||||
import uuid
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def load_module():
|
||||
script_path = (
|
||||
pathlib.Path(__file__).resolve().parents[1]
|
||||
/ "resources"
|
||||
/ "scripts"
|
||||
/ "napseer_mcp_server.py"
|
||||
)
|
||||
sys.path.insert(0, str(script_path.parent))
|
||||
spec = importlib.util.spec_from_file_location("napseer_mcp_reliability_test", script_path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mod(monkeypatch, tmp_path):
|
||||
monkeypatch.delenv("NAPSEER_TOOL_PROFILES", raising=False)
|
||||
module = load_module()
|
||||
module.DEFAULT_PROJECT_ID = "project-1"
|
||||
module.TOKEN = "synthetic-access"
|
||||
module.REFRESH_TOKEN = "synthetic-refresh"
|
||||
module.TOKEN_EXPIRES_AT = "2099-01-01T00:00:00Z"
|
||||
module.REFRESH_EXPIRES_AT = None
|
||||
module.AUTH = {**module.AUTH, "account_mode": "operator_account", "project_id": "project-1"}
|
||||
module.refresh_public_auth_state = lambda: module.AUTH
|
||||
module.project_memory_encryption_active = lambda _project_id: False
|
||||
module.AUTH_DIR = tmp_path / "state"
|
||||
module.INDEX_PATH = tmp_path / "state" / "index.sqlite"
|
||||
module.INDEX_LOCK_PATH = tmp_path / "state" / "index.lock"
|
||||
module.resolve_project_id = lambda _args: "project-1"
|
||||
return module
|
||||
|
||||
|
||||
class FakeResponse:
|
||||
def __init__(self, payload):
|
||||
self._body = json.dumps(payload).encode("utf-8")
|
||||
|
||||
def read(self, *_args):
|
||||
return self._body
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *_exc):
|
||||
return False
|
||||
|
||||
|
||||
def http_error(status, body):
|
||||
return urllib.error.HTTPError(
|
||||
"https://api.example.test/x", status, "error", {}, io.BytesIO(json.dumps(body).encode("utf-8"))
|
||||
)
|
||||
|
||||
|
||||
def stub_project_lock(mod):
|
||||
mod.acquire_project_lock = lambda args=None, **kwargs: {"id": "lock-1", "lease_token": "lease-1"}
|
||||
mod.cleanup_project_lock = lambda project_id, lock: None
|
||||
|
||||
|
||||
# -- 1. Idempotency keys ----------------------------------------------------
|
||||
|
||||
|
||||
def test_create_node_idempotency_key_survives_auth_retry_and_is_returned(mod):
|
||||
stub_project_lock(mod)
|
||||
seen = []
|
||||
|
||||
def fake_urlopen(request, timeout=None):
|
||||
seen.append((request.get_method(), request.full_url, request.get_header("Idempotency-key"),
|
||||
request.get_header("Authorization")))
|
||||
if len(seen) == 1:
|
||||
raise http_error(401, {"error": {"code": "unauthorized"}})
|
||||
return FakeResponse({"id": "node-new", "project_id": "project-1", "full_path": "/notes/a",
|
||||
"folder_path": "/notes", "name": "a", "type": "note", "content_text": "x"})
|
||||
|
||||
def fake_renew(stale_token=None):
|
||||
mod.TOKEN = "renewed-access"
|
||||
return {"status": "renewed"}
|
||||
|
||||
mod.api_urlopen = fake_urlopen
|
||||
mod.renew_auth = fake_renew
|
||||
|
||||
result = mod.call_tool_impl("nap_create_node", {"folder_path": "/notes", "name": "a", "content_text": "x"})
|
||||
|
||||
assert len(seen) == 2
|
||||
generated = seen[0][2]
|
||||
assert generated and str(uuid.UUID(generated)) == generated
|
||||
# One logical request: the internal auth replay reuses the same key.
|
||||
assert seen[1][2] == generated
|
||||
assert result["idempotency_key"] == generated
|
||||
|
||||
seen.clear()
|
||||
mod.api_urlopen = lambda request, timeout=None: seen.append(request.get_header("Idempotency-key")) or FakeResponse(
|
||||
{"id": "node-new", "project_id": "project-1", "full_path": "/notes/a"}
|
||||
)
|
||||
result = mod.call_tool_impl(
|
||||
"nap_create_node", {"folder_path": "/notes", "name": "a", "content_text": "x", "idempotency_key": "caller-key-1"}
|
||||
)
|
||||
assert seen == ["caller-key-1"]
|
||||
assert result["idempotency_key"] == "caller-key-1"
|
||||
|
||||
|
||||
def test_node_patch_sends_header_not_body_and_bulk_items_get_keys(mod):
|
||||
stub_project_lock(mod)
|
||||
existing = {"id": "node-123", "project_id": "project-1", "full_path": "/notes/current",
|
||||
"folder_path": "/notes", "name": "current", "type": "note", "content_text": "old"}
|
||||
mod.get_node_by_id = lambda args: existing
|
||||
mod.try_get_node_by_path = lambda path, allow_agent=False: None
|
||||
mod.index_node = lambda node: None
|
||||
writes = []
|
||||
|
||||
def fake_request(method, path, payload=None, **kwargs):
|
||||
writes.append((method, path, payload, dict(kwargs.get("extra_headers") or {})))
|
||||
if path.endswith("/patch"):
|
||||
return {"ok": True, "data": {"node": existing, "revision": "r2", "read_fingerprint": "f2"}}
|
||||
return {**existing, "id": "node-new" if method == "POST" else existing["id"]}
|
||||
|
||||
mod.request_json = fake_request
|
||||
|
||||
patched = mod.guarded_node_patch({
|
||||
"node_id": "node-123",
|
||||
"precondition": {"revision": "r1", "read_fingerprint": "f1"},
|
||||
"merge_metadata": {"status": "active"},
|
||||
"idempotency_key": "patch-key-1",
|
||||
})
|
||||
method, path, payload, headers = writes[-1]
|
||||
assert (method, path) == ("PATCH", "/v1/projects/project-1/nodes/node-123/patch")
|
||||
assert "idempotency_key" not in payload
|
||||
assert headers["Idempotency-Key"] == "patch-key-1"
|
||||
assert headers["X-Napseer-Lock-Id"] == "lock-1"
|
||||
assert patched["idempotency_key"] == "patch-key-1"
|
||||
|
||||
generated = mod.guarded_node_patch({
|
||||
"node_id": "node-123",
|
||||
"precondition": {"revision": "r1", "read_fingerprint": "f1"},
|
||||
"merge_metadata": {"status": "active"},
|
||||
})
|
||||
assert generated["idempotency_key"] == writes[-1][3]["Idempotency-Key"]
|
||||
uuid.UUID(generated["idempotency_key"])
|
||||
|
||||
writes.clear()
|
||||
bulk = mod.bulk_upsert_nodes({
|
||||
"idempotency_key": "bulk-base",
|
||||
"nodes": [
|
||||
{"path": "/notes/new", "content_text": "new"},
|
||||
{"node_id": "node-123", "content_text": "updated", "idempotency_key": "item-own"},
|
||||
],
|
||||
})
|
||||
assert [(w[0], w[3]["Idempotency-Key"]) for w in writes] == [("POST", "bulk-base:0"), ("PATCH", "item-own")]
|
||||
assert [item["idempotency_key"] for item in bulk["items"]] == ["bulk-base:0", "item-own"]
|
||||
|
||||
for name in ("nap_create_node", "nap_node_patch", "nap_bulk"):
|
||||
assert mod.tool_contract_metadata(name)["idempotency"] == "same-idempotency-key"
|
||||
tools = {tool["name"]: tool for tool in mod.raw_tools()}
|
||||
assert "idempotency_key" in tools["nap_create_node"]["inputSchema"]["properties"]
|
||||
assert "idempotency_key" in tools["nap_bulk"]["inputSchema"]["properties"]
|
||||
assert "idempotency_key" in tools["nap_bulk"]["inputSchema"]["properties"]["nodes"]["items"]["properties"]
|
||||
|
||||
|
||||
# -- 2. Stale local reads and whoami auth precedence -----------------------
|
||||
|
||||
|
||||
def indexed(path, links=None):
|
||||
folder, name = path.rsplit("/", 1)
|
||||
return {
|
||||
"id": f"id-{name}",
|
||||
"project_id": "project-1",
|
||||
"full_path": path,
|
||||
"folder_path": folder,
|
||||
"name": name,
|
||||
"type": "note",
|
||||
"tags": [],
|
||||
"aliases": [],
|
||||
"links": links or [],
|
||||
"metadata": {"status": "active"},
|
||||
"content_text": f"body of {name} gateway reliability",
|
||||
"encryption_state": "legacy_plaintext",
|
||||
"updated_at": "2026-08-22T00:00:00Z",
|
||||
}
|
||||
|
||||
|
||||
def build_complete_index(mod):
|
||||
mod.index_node(indexed("/plans/alpha", links=[{"path": "/plans/beta", "relation": "references"}]))
|
||||
mod.index_node(indexed("/plans/beta"))
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
|
||||
def record_revoked_grant(mod):
|
||||
def rejected(*_args, **_kwargs):
|
||||
raise mod.SafeToolError("http_401", "rejected", status=401)
|
||||
|
||||
mod.request_form_json = rejected
|
||||
with pytest.raises(mod.SafeToolError) as caught:
|
||||
mod.renew_auth()
|
||||
assert caught.value.code == "auth_required"
|
||||
assert "expired or revoked" in caught.value.safe_message
|
||||
|
||||
|
||||
def test_local_answers_warn_when_login_grant_is_revoked(mod):
|
||||
build_complete_index(mod)
|
||||
|
||||
def empty_feed(_method, path, *_args, **_kwargs):
|
||||
if "/changes?" in path:
|
||||
return {"items": [], "next_cursor": "c0", "has_more": False}
|
||||
return {"items": [], "next_cursor": None}
|
||||
|
||||
# A healthy read starts change-feed tracking for the cursorless index.
|
||||
mod.request_json = empty_feed
|
||||
healthy = mod.discover_memory({"folder_path": "/plans", "view": "paths"})
|
||||
assert not healthy.get("warnings")
|
||||
assert not (healthy.get("freshness") or {}).get("stale")
|
||||
assert healthy["freshness"]["feed_sync"] == "tracking_initialized"
|
||||
mod.request_json = lambda *a, **k: (_ for _ in ()).throw(AssertionError("no HTTP expected"))
|
||||
|
||||
record_revoked_grant(mod)
|
||||
state_text = (mod.AUTH_DIR / "auth-failure.json").read_text()
|
||||
assert "synthetic-refresh" not in state_text and "synthetic-access" not in state_text
|
||||
|
||||
browsed = mod.discover_memory({"folder_path": "/plans", "view": "paths"})
|
||||
assert [w["code"] for w in browsed["warnings"]] == ["auth_required"]
|
||||
assert browsed["warnings"][0]["indexed_at"]
|
||||
assert browsed["freshness"]["stale"] is True
|
||||
assert browsed["freshness"]["graph_complete"] is True
|
||||
|
||||
context = mod.get_memory_context({"node_id": "id-alpha", "depth": 1})
|
||||
codes = [w["code"] for w in context["warnings"] if isinstance(w, dict)]
|
||||
assert codes == ["auth_required"]
|
||||
assert context["freshness"]["stale"] is True
|
||||
assert context["edge_count"] >= 1
|
||||
|
||||
status = mod.gateway_status()
|
||||
assert status["status"] == "auth_required"
|
||||
assert status["auth_problem"]["reason"] == "auth_required"
|
||||
|
||||
# A new login invalidates the recorded failure without clearing files.
|
||||
mod.TOKEN = "fresh-access"
|
||||
mod.REFRESH_TOKEN = "fresh-refresh"
|
||||
assert mod.current_auth_problem() is None
|
||||
assert mod.gateway_status()["status"] != "auth_required"
|
||||
|
||||
|
||||
def test_hybrid_search_falls_back_to_local_with_server_unavailable_warning(mod):
|
||||
build_complete_index(mod)
|
||||
|
||||
def server_down(*_args, **_kwargs):
|
||||
raise mod.SafeToolError("http_503", "Napseer GET request failed with HTTP 503.", status=503)
|
||||
|
||||
mod.request_json = server_down
|
||||
result = mod.discover_memory({"q": "gateway reliability", "limit": 25})
|
||||
assert {item["full_path"] for item in result["items"]} == {"/plans/alpha", "/plans/beta"}
|
||||
assert [w["code"] for w in result["warnings"]] == ["server_unavailable"]
|
||||
assert result["freshness"]["stale"] is True
|
||||
assert result["freshness"]["reason"] == "server_unavailable"
|
||||
|
||||
# A non-recoverable server error is not masked by the local answer.
|
||||
mod.request_json = lambda *a, **k: (_ for _ in ()).throw(mod.SafeToolError("http_403", "forbidden", status=403))
|
||||
with pytest.raises(mod.SafeToolError):
|
||||
mod.discover_memory({"q": "gateway reliability", "limit": 25})
|
||||
|
||||
|
||||
def test_whoami_reports_expired_access_without_refresh_ahead_of_gateway_state(mod):
|
||||
mod.REFRESH_TOKEN = None
|
||||
mod.TOKEN_EXPIRES_AT = "2020-01-01T00:00:00Z"
|
||||
status = mod.gateway_status()
|
||||
assert status["status"] == "auth_required"
|
||||
assert status["auth_problem"]["reason"] == "access_token_expired_without_refresh"
|
||||
|
||||
|
||||
# -- 3. Atomic claim-next ---------------------------------------------------
|
||||
|
||||
|
||||
def kanban_card(name, priority, rank):
|
||||
return {
|
||||
"id": f"id-{name}",
|
||||
"project_id": "project-1",
|
||||
"full_path": f"/kanban/todo/{name}",
|
||||
"folder_path": "/kanban/todo",
|
||||
"name": name,
|
||||
"type": "kanban_card",
|
||||
"tags": ["kanban"],
|
||||
"links": [],
|
||||
"metadata": {"status": "todo", "column": "todo", "priority": priority, "rank": rank, "blocked": False},
|
||||
"content_text": "card",
|
||||
"updated_at": "2026-05-06T12:00:00Z",
|
||||
"archived_at": None,
|
||||
"encryption_state": "plaintext",
|
||||
}
|
||||
|
||||
|
||||
def kanban_backend(mod, claim_outcomes):
|
||||
cards = [kanban_card("first", "urgent", "00001024"), kanban_card("second", "high", "00001024"),
|
||||
kanban_card("third", "normal", "00001024")]
|
||||
claims = []
|
||||
|
||||
def fake_request(method, path, payload=None, **kwargs):
|
||||
if method == "GET":
|
||||
return {"items": cards, "next_cursor": None}
|
||||
assert (method, path) == ("POST", "/v1/projects/project-1/operations")
|
||||
claims.append(payload)
|
||||
outcome = claim_outcomes[len(claims) - 1]
|
||||
if isinstance(outcome, Exception):
|
||||
raise outcome
|
||||
return {**outcome, "id": payload["id"], "node_id": payload["node_id"]}
|
||||
|
||||
mod.request_json = fake_request
|
||||
return claims
|
||||
|
||||
|
||||
def held_error(mod, legacy=False):
|
||||
mod_error = mod.safe_http_error(
|
||||
http_error(409, {}),
|
||||
operation="POST request",
|
||||
body_text=json.dumps(
|
||||
{"error": {"code": "conflict", "message": "Another execution holds this work until it expires."}}
|
||||
if legacy else
|
||||
{"error": {"code": "operation_held", "message": "held", "details": {"active_operation_id": "op-active"}}}
|
||||
),
|
||||
)
|
||||
return mod_error
|
||||
|
||||
|
||||
def test_claim_next_skips_held_cards_and_claims_in_pick_order(mod):
|
||||
claims = kanban_backend(mod, [held_error(mod), held_error(mod, legacy=True), {"state": "running"}])
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"workflow"}
|
||||
|
||||
result = mod.call_tool_impl("nap_kanban_claim_next", {"holder": "agent-a", "ttl_seconds": 120})
|
||||
|
||||
assert [c["node_id"] for c in claims] == ["id-first", "id-second", "id-third"]
|
||||
assert len({c["id"] for c in claims}) == 3
|
||||
assert all(c["holder"] == "agent-a" and c["ttl_seconds"] == 120 for c in claims)
|
||||
assert result["claimed"]["full_path"] == "/kanban/todo/third"
|
||||
assert result["operation"]["state"] == "running"
|
||||
assert result["operation_id"] == claims[-1]["id"]
|
||||
assert [s["reason"] for s in result["skipped"]] == ["operation_held", "operation_held"]
|
||||
assert result["skipped"][0]["active_operation_id"] == "op-active"
|
||||
|
||||
|
||||
def test_claim_next_returns_null_when_all_held_and_stops_on_ambiguous_error(mod):
|
||||
kanban_backend(mod, [held_error(mod)] * 3)
|
||||
none = mod.claim_next_kanban_card({"holder": "agent-a"})
|
||||
assert none["ok"] is True and none["claimed"] is None
|
||||
assert none["reason"] == "all_candidates_held"
|
||||
|
||||
claims = kanban_backend(mod, [held_error(mod), mod.SafeToolError("request_timeout", "timed out")])
|
||||
with pytest.raises(mod.SafeToolError) as caught:
|
||||
mod.claim_next_kanban_card({"holder": "agent-a"})
|
||||
# Stops at the ambiguous claim; never retries or moves on.
|
||||
assert len(claims) == 2
|
||||
payload = mod.safe_exception_payload(caught.value)
|
||||
assert payload["code"] == "kanban_claim_unresolved"
|
||||
assert payload["details"]["attempted_operation_id"] == claims[1]["id"]
|
||||
assert payload["details"]["cause"]["code"] == "request_timeout"
|
||||
assert claims[1]["id"] in payload["message"]
|
||||
|
||||
claims = kanban_backend(mod, [held_error(mod), held_error(mod)])
|
||||
capped = mod.claim_next_kanban_card({"holder": "agent-a", "max_candidates": 2})
|
||||
assert len(claims) == 2 and capped["reason"] == "max_candidates_held"
|
||||
|
||||
with pytest.raises(mod.SafeToolError):
|
||||
mod.claim_next_kanban_card({})
|
||||
|
||||
|
||||
def test_claim_next_is_registered_as_workflow_mutation(mod):
|
||||
assert "nap_kanban_claim_next" in mod.CANONICAL_TOOL_NAMES
|
||||
assert mod.tool_profile("nap_kanban_claim_next") == "workflow"
|
||||
assert mod.tool_category("nap_kanban_claim_next") == "kanban"
|
||||
metadata = mod.tool_contract_metadata("nap_kanban_claim_next")
|
||||
assert metadata["side_effect"] == "remote-write"
|
||||
assert metadata["lock_policy"] == "atomic-execution-lease"
|
||||
assert "nap_kanban_claim_next" not in mod.READ_ONLY_TOOLS
|
||||
|
||||
|
||||
# -- 4. links_count from indexed edges -------------------------------------
|
||||
|
||||
|
||||
def test_local_discovery_rows_report_indexed_link_counts(mod):
|
||||
build_complete_index(mod)
|
||||
mod.request_json = lambda *a, **k: (_ for _ in ()).throw(AssertionError("no HTTP expected"))
|
||||
|
||||
searched = mod.search_local_index({"q": "gateway reliability", "limit": 10})
|
||||
by_path = {item["full_path"]: item for item in searched["items"]}
|
||||
assert by_path["/plans/alpha"]["links_count"] == 1
|
||||
assert by_path["/plans/beta"]["backlinks_count"] == 1
|
||||
|
||||
rows = mod.discover_memory({"q": "gateway reliability", "view": "summary", "mode": "local"})["items"]
|
||||
rows = {item["full_path"]: item for item in rows}
|
||||
assert rows["/plans/alpha"]["links_count"] == 1
|
||||
assert rows["/plans/beta"]["links_count"] == 0
|
||||
assert rows["/plans/beta"]["backlinks_count"] == 1
|
||||
|
||||
# Unknown counts are omitted rather than reported as zero.
|
||||
summary = mod.summarize_node({"id": "x", "full_path": "/a/x", "name": "x"}, view="summary")
|
||||
assert "links_count" not in summary
|
||||
|
||||
|
||||
# -- 5. Tool profiles from the committed root .napseer-tools.json ----------
|
||||
|
||||
|
||||
def write_locator(mod, **extra):
|
||||
path = mod.project_locator_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps({
|
||||
"schema": "napseer.project.v1",
|
||||
"base_url": "https://api.example.test",
|
||||
"project_id": "11111111-1111-4111-8111-111111111111",
|
||||
"project_slug": "example",
|
||||
**extra,
|
||||
}))
|
||||
|
||||
|
||||
def write_tools_file(mod, profiles=None, *, raw=None):
|
||||
path = mod.project_tools_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if raw is None:
|
||||
raw = json.dumps({"schema": "napseer.tools.v1", "tool_profiles": profiles}) + "\n"
|
||||
path.write_bytes(raw if isinstance(raw, bytes) else raw.encode("utf-8"))
|
||||
return path
|
||||
|
||||
|
||||
def doctor_warnings(mod):
|
||||
mod.BASE_URL = "https://api.example.test"
|
||||
mod.local_anchor_integrity = lambda: {}
|
||||
mod.index_status = lambda args: {}
|
||||
return mod.doctor({}).get("warnings", [])
|
||||
|
||||
|
||||
def test_tools_file_profiles_apply_when_env_unset(mod):
|
||||
mod.CONFIGURED_TOOL_PROFILES = None
|
||||
write_locator(mod)
|
||||
assert mod.tool_profile_configuration() == (set(), "default", [])
|
||||
assert mod.project_tools_path() == mod.project_locator_path().parent.parent / ".napseer-tools.json"
|
||||
# The retired in-.napseer location is never read.
|
||||
(mod.project_locator_path().parent / "tools.json").write_text(
|
||||
json.dumps({"schema": "napseer.tools.v1", "tool_profiles": ["workflow"]}))
|
||||
assert mod.tool_profile_configuration() == (set(), "default", [])
|
||||
|
||||
write_tools_file(mod, ["workflow", " Operations "])
|
||||
assert mod.tool_profile_configuration() == ({"workflow", "operations"}, "tools_file", [])
|
||||
assert "nap_kanban_claim_next" in mod.enabled_tool_names()
|
||||
apropos = mod.explore_tools({})
|
||||
assert apropos["profile_source"] == "tools_file"
|
||||
assert apropos["active_profiles"] == ["operations", "workflow"]
|
||||
assert "warnings" not in apropos
|
||||
contract = mod.contract_profile({"include_tools": False})["tool_profiles"]
|
||||
assert contract["source"] == "tools_file"
|
||||
assert contract["tools_file"] == str(mod.project_tools_path())
|
||||
assert doctor_warnings(mod) == []
|
||||
# The committed locator still carries identity only.
|
||||
assert set(mod.load_project_locator()) == {"schema", "base_url", "project_id", "project_slug"}
|
||||
|
||||
|
||||
def test_env_overrides_tools_file(mod, monkeypatch):
|
||||
mod.CONFIGURED_TOOL_PROFILES = None
|
||||
write_tools_file(mod, ["workflow"])
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "maintenance")
|
||||
assert mod.tool_profile_configuration() == ({"maintenance"}, "environment", [])
|
||||
assert "nap_kanban_claim_next" not in mod.enabled_tool_names()
|
||||
assert mod.explore_tools({})["profile_source"] == "environment"
|
||||
|
||||
# A non-empty environment value wins without consulting a broken file.
|
||||
write_tools_file(mod, raw="{not json")
|
||||
assert mod.tool_profile_configuration() == ({"maintenance"}, "environment", [])
|
||||
|
||||
# An empty or blank value is unset: the committed file applies again.
|
||||
write_tools_file(mod, ["workflow"])
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", " ")
|
||||
assert mod.tool_profile_configuration() == ({"workflow"}, "tools_file", [])
|
||||
|
||||
|
||||
def test_tools_file_unknown_profile_names_are_ignored_with_warning(mod):
|
||||
mod.CONFIGURED_TOOL_PROFILES = None
|
||||
write_tools_file(mod, ["workflow", "bogus", 7])
|
||||
profiles, source, warnings = mod.tool_profile_configuration()
|
||||
assert (profiles, source) == ({"workflow"}, "tools_file")
|
||||
assert [w["code"] for w in warnings] == ["unknown_tool_profile"]
|
||||
assert warnings[0]["ignored"] == ["bogus", "7"]
|
||||
apropos = mod.explore_tools({})
|
||||
assert apropos["warnings"][0]["code"] == "unknown_tool_profile"
|
||||
assert [w["code"] for w in doctor_warnings(mod)] == ["unknown_tool_profile"]
|
||||
|
||||
write_tools_file(mod, ["workflow"] * 20)
|
||||
assert [w["code"] for w in mod.tool_profile_configuration()[2]] == ["tool_profiles_truncated"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"case",
|
||||
["invalid_json", "not_object", "wrong_schema", "missing_schema", "profiles_not_list",
|
||||
"oversized", "not_utf8", "symlink", "directory"],
|
||||
)
|
||||
def test_unusable_tools_file_warns_and_falls_back_to_core(mod, tmp_path_factory, case):
|
||||
mod.CONFIGURED_TOOL_PROFILES = None
|
||||
marker = "synthetic-marker-not-echoed"
|
||||
raw = {
|
||||
"invalid_json": '{"schema": "napseer.tools.v1", "tool_profiles": ["workflow"], ' + marker,
|
||||
"not_object": json.dumps(["workflow", marker]),
|
||||
"wrong_schema": json.dumps({"schema": "napseer.tools.v9", "tool_profiles": ["workflow"]}),
|
||||
"missing_schema": json.dumps({"tool_profiles": ["workflow"]}),
|
||||
"profiles_not_list": json.dumps({"schema": "napseer.tools.v1", "tool_profiles": "workflow"}),
|
||||
"oversized": json.dumps({"schema": "napseer.tools.v1", "tool_profiles": ["workflow"]}) + " " * 4096,
|
||||
"not_utf8": b'{"schema": "napseer.tools.v1", "tool_profiles": ["\xff"]}',
|
||||
}.get(case)
|
||||
path = mod.project_tools_path()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
if case == "symlink":
|
||||
outside = tmp_path_factory.mktemp("outside-project") / ".napseer-tools.json"
|
||||
outside.write_text(json.dumps({"schema": "napseer.tools.v1", "tool_profiles": ["all"]}))
|
||||
path.symlink_to(outside)
|
||||
elif case == "directory":
|
||||
path.mkdir()
|
||||
else:
|
||||
write_tools_file(mod, raw=raw)
|
||||
|
||||
profiles, source, warnings = mod.tool_profile_configuration()
|
||||
assert (profiles, source) == (set(), "default")
|
||||
assert [w["code"] for w in warnings] == ["tools_file_invalid"]
|
||||
assert marker not in warnings[0]["message"]
|
||||
assert mod.enabled_tool_names() == set(mod.CORE_TOOL_NAMES)
|
||||
assert mod.explore_tools({})["warnings"][0]["code"] == "tools_file_invalid"
|
||||
assert [w["code"] for w in doctor_warnings(mod)] == ["tools_file_invalid"]
|
||||
with pytest.raises(mod.SafeToolError) as excinfo:
|
||||
mod._call_tool_impl_dispatch("nap_kanban_claim_next", {})
|
||||
assert excinfo.value.code == "tool_profile_required"
|
||||
assert ".napseer-tools.json" in excinfo.value.safe_message
|
||||
|
||||
|
||||
def test_tools_file_parser_recursion_is_contained(mod, monkeypatch):
|
||||
# Older Python JSON decoders raise RecursionError on deep nesting that
|
||||
# still fits the size bound; it must degrade to core, not fail every call.
|
||||
mod.CONFIGURED_TOOL_PROFILES = None
|
||||
write_tools_file(mod, ["workflow"])
|
||||
|
||||
def too_deep(*_args, **_kwargs):
|
||||
raise RecursionError("synthetic")
|
||||
|
||||
with monkeypatch.context() as patched:
|
||||
patched.setattr(mod.json, "loads", too_deep)
|
||||
profiles, source, warnings = mod.tool_profile_configuration()
|
||||
assert (profiles, source) == (set(), "default")
|
||||
assert [w["code"] for w in warnings] == ["tools_file_invalid"]
|
||||
|
||||
|
||||
def test_locator_tool_profiles_field_is_still_rejected(mod):
|
||||
# tool_profiles was never a released locator field: the locator keeps its
|
||||
# exact-field schema and the field is neither accepted nor applied.
|
||||
mod.CONFIGURED_TOOL_PROFILES = None
|
||||
write_locator(mod, tool_profiles=["workflow"])
|
||||
with pytest.raises(mod.SafeToolError) as excinfo:
|
||||
mod.load_project_locator()
|
||||
assert excinfo.value.code == "project_locator_invalid"
|
||||
assert "unsupported or missing fields" in excinfo.value.safe_message
|
||||
with pytest.raises(mod.SafeToolError) as excinfo:
|
||||
mod.write_project_locator({"id": "11111111-1111-4111-8111-111111111111", "slug": "example"},
|
||||
base_url="https://api.example.test")
|
||||
assert excinfo.value.code == "project_locator_invalid"
|
||||
assert "tool_profiles" in json.loads(mod.project_locator_path().read_text())
|
||||
assert mod.tool_profile_configuration() == (set(), "default", [])
|
||||
assert "nap_kanban_claim_next" not in mod.enabled_tool_names()
|
||||
|
||||
|
||||
# -- 6. Change feed cursor --------------------------------------------------
|
||||
|
||||
|
||||
def changes_backend(mod, responses):
|
||||
calls = []
|
||||
|
||||
def fake_request(method, path, payload=None, **kwargs):
|
||||
calls.append((method, path))
|
||||
outcome = responses[len(calls) - 1]
|
||||
if isinstance(outcome, Exception):
|
||||
raise outcome
|
||||
return outcome
|
||||
|
||||
mod.request_json = fake_request
|
||||
return calls
|
||||
|
||||
|
||||
def test_changes_since_starts_at_head_then_advances_only_on_success(mod, tmp_path):
|
||||
change = {"type": "node_updated", "cursor": "c1.5.9", "event_id": "e1", "project_id": "project-1",
|
||||
"node_id": "n1", "full_path": "/plans/a", "changed_fields": ["content_text"]}
|
||||
calls = changes_backend(mod, [
|
||||
{"items": [], "next_cursor": "c1.5.8", "has_more": False},
|
||||
{"items": [change], "next_cursor": "c1.5.9", "has_more": True},
|
||||
mod.SafeToolError("http_503", "unavailable", status=503),
|
||||
{"items": [], "next_cursor": "c1.9.9", "has_more": False},
|
||||
])
|
||||
|
||||
first = mod.call_tool_impl("nap_changes_since", {})
|
||||
assert calls[0] == ("GET", "/v1/projects/project-1/changes?after=head&limit=100")
|
||||
assert first["started_at_head"] is True and first["items"] == []
|
||||
stored = json.loads((mod.AUTH_DIR / "changes-cursor.json").read_text())
|
||||
assert stored["project_id"] == "project-1" and stored["cursor"] == "c1.5.8"
|
||||
assert not (tmp_path / ".napseer" / "changes-cursor.json").exists()
|
||||
|
||||
second = mod.changes_since({"limit": 50})
|
||||
assert calls[1] == ("GET", "/v1/projects/project-1/changes?after=c1.5.8&limit=50")
|
||||
assert second["items"] == [change] and second["has_more"] is True and second["cursor_advanced"] is True
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as caught:
|
||||
mod.changes_since({})
|
||||
assert caught.value.details["cursor_advanced"] is False
|
||||
assert mod.read_changes_cursor("project-1") == "c1.5.9"
|
||||
|
||||
peek = mod.changes_since({"advance": False})
|
||||
assert peek["cursor_advanced"] is False
|
||||
assert mod.read_changes_cursor("project-1") == "c1.5.9"
|
||||
|
||||
|
||||
def test_changes_since_errors_project_binding_and_reset(mod):
|
||||
mod.write_changes_cursor("other-project", "c1.1.1")
|
||||
calls = changes_backend(mod, [
|
||||
mod.safe_http_error(http_error(404, {}), body_text="{}"),
|
||||
mod.safe_http_error(http_error(400, {}), body_text=json.dumps({"error": {"code": "validation_error"}})),
|
||||
{"items": [], "next_cursor": "c1.2.2", "has_more": False},
|
||||
])
|
||||
with pytest.raises(mod.SafeToolError) as unsupported:
|
||||
mod.changes_since({})
|
||||
# A cursor stored for another project is never reused.
|
||||
assert "after=head" in calls[0][1]
|
||||
assert unsupported.value.code == "unsupported_server"
|
||||
|
||||
with pytest.raises(mod.SafeToolError) as invalid:
|
||||
mod.changes_since({"after": "bogus"})
|
||||
assert invalid.value.code == "invalid_changes_cursor"
|
||||
|
||||
reset = mod.changes_since({"reset": True})
|
||||
assert "after=head" in calls[2][1] and reset["started_at_head"] is True
|
||||
assert mod.read_changes_cursor("project-1") == "c1.2.2"
|
||||
|
||||
mod.REFRESH_TOKEN = None
|
||||
mod.TOKEN_EXPIRES_AT = "2020-01-01T00:00:00Z"
|
||||
with pytest.raises(mod.SafeToolError) as auth:
|
||||
mod.changes_since({})
|
||||
assert auth.value.code == "auth_required"
|
||||
assert auth.value.details["auth_problem"]["reason"] == "access_token_expired_without_refresh"
|
||||
assert len(calls) == 3
|
||||
assert mod.read_changes_cursor("project-1") == "c1.2.2"
|
||||
|
||||
|
||||
def test_changes_since_is_a_core_read_tool(mod):
|
||||
assert "nap_changes_since" in mod.CORE_TOOL_NAMES
|
||||
assert mod.tool_category("nap_changes_since") == "memory"
|
||||
assert mod.tool_contract_metadata("nap_changes_since")["side_effect"] == "none"
|
||||
|
||||
|
||||
# -- 7. Idempotency keys on remaining node writes --------------------------
|
||||
|
||||
|
||||
def node_write_backend(mod, nodes):
|
||||
stub_project_lock(mod)
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
mod.get_node_by_id = lambda args: nodes[args.get("node_id") or args.get("id")]
|
||||
mod.index_node = lambda node: None
|
||||
mod.remove_indexed_node = lambda node_id: None
|
||||
writes = []
|
||||
|
||||
def fake_request(method, path, payload=None, **kwargs):
|
||||
if method == "GET":
|
||||
return {"items": list(nodes.values()) if "/nodes?" in path else [], "next_cursor": None}
|
||||
writes.append((method, path, dict(kwargs.get("extra_headers") or {})))
|
||||
node_id = path.rsplit("/", 1)[-1]
|
||||
return {**nodes.get(node_id, next(iter(nodes.values()))), **(payload or {})}
|
||||
|
||||
mod.request_json = fake_request
|
||||
return writes
|
||||
|
||||
|
||||
def test_link_and_kanban_writes_send_scoped_idempotency_keys(mod):
|
||||
card = kanban_card("first", "high", "00001024")
|
||||
card["id"] = "card-1"
|
||||
target = {"id": "target-1", "project_id": "project-1", "full_path": "/plans/t", "links": []}
|
||||
writes = node_write_backend(mod, {"card-1": card, "target-1": target})
|
||||
|
||||
linked = mod.call_tool_impl("nap_ln", {"source_node_id": "card-1", "target_node_id": "target-1"})
|
||||
assert writes[-1][0:2] == ("PATCH", "/v1/projects/project-1/nodes/card-1")
|
||||
assert linked["idempotency_key"] == writes[-1][2]["Idempotency-Key"]
|
||||
uuid.UUID(linked["idempotency_key"])
|
||||
|
||||
started = mod.call_tool_impl("nap_kanban_start", {"node_id": "card-1", "idempotency_key": "start-1"})
|
||||
assert writes[-1][2]["Idempotency-Key"] == "start-1"
|
||||
assert started["idempotency_key"] == "start-1"
|
||||
|
||||
tools = {tool["name"]: tool for tool in mod.all_tools()}
|
||||
for name in ("nap_ln", "nap_kanban_start", "nap_kanban_update", "nap_agent_tee", "nap_mv_folder"):
|
||||
assert "idempotency_key" in tools[name]["inputSchema"]["properties"], name
|
||||
assert mod.tool_contract_metadata(name)["idempotency"] == "same-idempotency-key"
|
||||
|
||||
|
||||
def test_multi_write_tools_derive_keys_and_surface_them_on_failure(mod):
|
||||
nodes = {
|
||||
"a": {"id": "a", "project_id": "project-1", "full_path": "/old/a", "folder_path": "/old", "name": "a"},
|
||||
"b": {"id": "b", "project_id": "project-1", "full_path": "/old/b", "folder_path": "/old", "name": "b"},
|
||||
}
|
||||
writes = node_write_backend(mod, nodes)
|
||||
moved = mod.call_tool_impl("nap_mv_folder", {"from_folder": "/old", "to_folder": "/new", "idempotency_key": "mvf"})
|
||||
assert [w[2]["Idempotency-Key"] for w in writes] == ["mvf", "mvf:1"]
|
||||
assert moved["idempotency_key"] == "mvf" and moved["idempotency_keys"] == ["mvf", "mvf:1"]
|
||||
|
||||
def ambiguous(method, path, payload=None, **kwargs):
|
||||
raise mod.SafeToolError("request_timeout", "timed out")
|
||||
|
||||
mod.request_json = ambiguous
|
||||
with pytest.raises(mod.SafeToolError) as caught:
|
||||
mod.call_tool_impl("nap_create_node", {"folder_path": "/notes", "name": "x", "content_text": "x"})
|
||||
uuid.UUID(caught.value.details["idempotency_key"])
|
||||
|
||||
assert mod.node_write_idempotency_headers("POST", "/v1/projects/p/nodes/n/restore") == {}
|
||||
assert mod.node_write_idempotency_headers("DELETE", "/v1/projects/p/nodes/n") == {}
|
||||
assert "Idempotency-Key" in mod.node_write_idempotency_headers("PATCH", "/v1/projects/p/nodes/n/patch")
|
||||
@@ -1,7 +1,11 @@
|
||||
import importlib.util
|
||||
import pathlib
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def load_module():
|
||||
@@ -30,7 +34,14 @@ def test_natural_language_discovery_uses_one_backend_search_request(tmp_path):
|
||||
|
||||
def request(method, path, payload=None, **_kwargs):
|
||||
calls.append((method, path, payload))
|
||||
return {"items": [], "next_cursor": None, "query_analysis": {"mode": "terms"}}
|
||||
return {
|
||||
"items": [],
|
||||
"next_cursor": None,
|
||||
"query_analysis": {"mode": "terms"},
|
||||
"search_strategy": "identity_fts",
|
||||
"search_coverage": "identity_only",
|
||||
"search_projection": "postgres_identity_v1",
|
||||
}
|
||||
|
||||
mod.request_json = request
|
||||
|
||||
@@ -46,6 +57,9 @@ def test_natural_language_discovery_uses_one_backend_search_request(tmp_path):
|
||||
assert len(calls) == 1
|
||||
assert calls[0][0] == "GET"
|
||||
assert "/nodes?" in calls[0][1]
|
||||
assert result["search_strategy"] == "identity_fts"
|
||||
assert result["search_coverage"] == "identity_only"
|
||||
assert result["search_projection"] == "postgres_identity_v1"
|
||||
|
||||
|
||||
def test_sparse_index_search_latency_does_not_scale_with_query_variants(tmp_path):
|
||||
@@ -74,3 +88,455 @@ def test_sparse_index_search_latency_does_not_scale_with_query_variants(tmp_path
|
||||
elapsed = time.monotonic() - started
|
||||
assert len(calls) == 1
|
||||
assert elapsed < 0.15
|
||||
|
||||
|
||||
def test_complete_local_search_skips_remote_encryption_preflight_and_search(tmp_path):
|
||||
mod = load_module()
|
||||
mod.INDEX_PATH = tmp_path / "index.sqlite"
|
||||
mod.INDEX_PATH.touch()
|
||||
mod.resolve_project_id = lambda _args: "project-1"
|
||||
mod.project_encryption_state = lambda _project_id: "unknown"
|
||||
items = [
|
||||
{
|
||||
"id": f"node-{index}",
|
||||
"project_id": "project-1",
|
||||
"full_path": f"/notes/result-{index}",
|
||||
"name": f"result-{index}",
|
||||
"type": "note",
|
||||
"tags": [],
|
||||
"updated_at": "2026-08-22T00:00:00Z",
|
||||
}
|
||||
for index in range(5)
|
||||
]
|
||||
mod.local_index_diagnostics = lambda _project_id: {
|
||||
"exists": True,
|
||||
"graph_complete": True,
|
||||
}
|
||||
mod.search_local_index = lambda _args: {
|
||||
"items": items,
|
||||
"query": {"terms": ["result"]},
|
||||
"search_strategies": [],
|
||||
"index": {"exists": True, "graph_complete": True, "used": True},
|
||||
}
|
||||
|
||||
def unexpected_request(*_args, **_kwargs):
|
||||
raise AssertionError("complete local search must not make an HTTP request")
|
||||
|
||||
mod.request_json = unexpected_request
|
||||
result = mod.search_memory({"q": "result", "limit": 5, "view": "summary"})
|
||||
|
||||
assert result["sources"] == ["local"]
|
||||
assert result["search_strategy"] == "local_fts"
|
||||
assert result["search_coverage"] == "identity_and_local_body"
|
||||
assert result["search_projection"] == "sqlite_fts5_current"
|
||||
assert len(result["items"]) == 5
|
||||
assert all(item["node_id"].startswith("node-") for item in result["items"])
|
||||
|
||||
|
||||
def indexed_node(path, *, node_type="note", tags=None, status="active", content=""):
|
||||
name = path.rsplit("/", 1)[-1]
|
||||
return {
|
||||
"id": f"node-{path.strip('/').replace('/', '-')}",
|
||||
"project_id": "project-1",
|
||||
"full_path": path,
|
||||
"folder_path": path.rsplit("/", 1)[0] or None,
|
||||
"name": name,
|
||||
"title": name,
|
||||
"type": node_type,
|
||||
"tags": tags or [],
|
||||
"aliases": [],
|
||||
"links": [],
|
||||
"metadata": {"status": status},
|
||||
"content_text": content,
|
||||
"encryption_state": "legacy_plaintext",
|
||||
"updated_at": "2026-08-22T00:00:00Z",
|
||||
}
|
||||
|
||||
|
||||
def feed_provider(calls):
|
||||
"""Synthetic server: empty change feed and no newer nodes."""
|
||||
|
||||
def request(method, path, payload=None, **_kwargs):
|
||||
calls.append((method, path))
|
||||
if "/changes?" in path:
|
||||
return {"items": [], "next_cursor": "c0", "has_more": False}
|
||||
return {"items": [], "next_cursor": None}
|
||||
|
||||
return request
|
||||
|
||||
|
||||
def configure_local_index(mod, tmp_path):
|
||||
mod.AUTH_DIR = tmp_path
|
||||
mod.INDEX_PATH = tmp_path / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = tmp_path / "index.lock"
|
||||
mod.resolve_project_id = lambda _args: "project-1"
|
||||
|
||||
|
||||
def test_single_pass_ranking_prefers_exact_canonical_identity(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/rules/memory-source-policy",
|
||||
node_type="rule",
|
||||
tags=["memory", "policy"],
|
||||
content="Canonical memory source rule.",
|
||||
)
|
||||
)
|
||||
for index in range(12):
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
f"/work/current/incidental-{index}",
|
||||
content="memory source policy " * (index + 2),
|
||||
)
|
||||
)
|
||||
|
||||
result = mod.search_local_index({"q": "memory source policy", "limit": 5})
|
||||
|
||||
assert result["items"][0]["full_path"] == "/rules/memory-source-policy"
|
||||
assert "exact_identity" in result["items"][0]["match_reason"]
|
||||
assert "full_path" in result["items"][0]["matched_fields"]
|
||||
assert result["search_strategies"] == [
|
||||
{
|
||||
"strategy": "single_pass_weighted_candidates",
|
||||
"matched": 13,
|
||||
"returned": 5,
|
||||
"query_count": 1,
|
||||
"candidate_limit": 30,
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_complete_local_compact_browse_uses_keyset_cursor_without_http(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
for index in range(7):
|
||||
item = indexed_node(
|
||||
f"/plans/item-{index}",
|
||||
node_type="plan",
|
||||
tags=["performance"],
|
||||
status="active",
|
||||
)
|
||||
item["updated_at"] = f"2026-08-2{index}T00:00:00Z"
|
||||
mod.index_node(item)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
calls = []
|
||||
mod.TOKEN = "disposable-test-token" # synthetic login; requests go to feed_provider only
|
||||
mod.request_json = feed_provider(calls)
|
||||
# The first read starts change-feed tracking for this cursorless index.
|
||||
mod.discover_memory({"folder_path": "/plans", "limit": 1, "view": "paths"})
|
||||
assert [path.split("?")[0].rsplit("/", 1)[-1] for _method, path in calls] == ["changes", "nodes"]
|
||||
calls.clear()
|
||||
|
||||
first = mod.discover_memory(
|
||||
{
|
||||
"folder_path": "/plans",
|
||||
"tag": "performance",
|
||||
"limit": 3,
|
||||
"view": "paths",
|
||||
}
|
||||
)
|
||||
second = mod.discover_memory(
|
||||
{
|
||||
"folder_path": "/plans",
|
||||
"tag": "performance",
|
||||
"limit": 3,
|
||||
"view": "paths",
|
||||
"cursor": first["next_cursor"],
|
||||
}
|
||||
)
|
||||
|
||||
assert first["source"] == "local_index"
|
||||
assert first["search_strategy"] == "local_snapshot_browse"
|
||||
assert first["search_projection"] == "sqlite_compact_identity_v1"
|
||||
assert first["freshness"]["graph_complete"] is True
|
||||
assert first["next_cursor"].startswith(mod.LOCAL_BROWSE_CURSOR_PREFIX)
|
||||
assert [item["updated_at"] for item in first["items"]] == sorted(
|
||||
(item["updated_at"] for item in first["items"]), reverse=True
|
||||
)
|
||||
assert not ({item["node_id"] for item in first["items"]} & {item["node_id"] for item in second["items"]})
|
||||
assert all(item["node_id"] for item in first["items"] + second["items"])
|
||||
# Steady state inside the refresh throttle window: no HTTP at all.
|
||||
assert calls == []
|
||||
assert first["freshness"]["feed_sync"] == "throttled"
|
||||
|
||||
|
||||
def test_local_compact_browse_cursor_is_bound_to_filters(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
for index in range(3):
|
||||
mod.index_node(indexed_node(f"/plans/item-{index}", node_type="plan"))
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
first = mod.discover_memory({"folder_path": "/plans", "limit": 1, "view": "summary"})
|
||||
with pytest.raises(RuntimeError, match="cursor filters do not match"):
|
||||
mod.discover_memory(
|
||||
{
|
||||
"folder_path": "/decisions",
|
||||
"limit": 1,
|
||||
"view": "summary",
|
||||
"cursor": first["next_cursor"],
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_local_compact_browse_title_order_is_global_across_pages(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
for title in ["Zulu", "alpha", "Mike", "bravo"]:
|
||||
item = indexed_node(f"/notes/{title}")
|
||||
item["title"] = title
|
||||
mod.index_node(item)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
first = mod.discover_memory({"limit": 2, "view": "titles", "sort": "title_asc"})
|
||||
second = mod.discover_memory(
|
||||
{"limit": 2, "view": "titles", "sort": "title_asc", "cursor": first["next_cursor"]}
|
||||
)
|
||||
|
||||
assert [item["title"] for item in first["items"] + second["items"]] == [
|
||||
"alpha",
|
||||
"bravo",
|
||||
"Mike",
|
||||
"Zulu",
|
||||
]
|
||||
|
||||
|
||||
def test_compact_browse_falls_back_once_when_local_contract_is_incomplete(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
mod.index_node(indexed_node("/plans/local", node_type="plan"))
|
||||
calls = []
|
||||
|
||||
def request(method, path, payload=None, **_kwargs):
|
||||
calls.append((method, path, payload))
|
||||
return {"items": [], "next_cursor": None}
|
||||
|
||||
mod.request_json = request
|
||||
result = mod.discover_memory({"folder_path": "/plans", "limit": 5, "view": "paths"})
|
||||
|
||||
assert result["ok"] is True
|
||||
assert len(calls) == 1
|
||||
assert calls[0][0] == "GET"
|
||||
assert "/nodes?" in calls[0][1]
|
||||
with pytest.raises(RuntimeError, match="complete active-node index"):
|
||||
mod.discover_memory(
|
||||
{"folder_path": "/plans", "limit": 5, "view": "paths", "mode": "local"}
|
||||
)
|
||||
|
||||
|
||||
def test_local_search_reads_overlap_without_writer_file_lock(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
for index in range(4):
|
||||
mod.index_node(indexed_node(f"/notes/result-{index}", content="shared needle"))
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
def unexpected_writer_lock(*_args, **_kwargs):
|
||||
raise AssertionError("read-only local search must not acquire the writer file lock")
|
||||
|
||||
mod.index_file_lock = unexpected_writer_lock
|
||||
original_rank = mod.rank_search_items
|
||||
rendezvous = threading.Barrier(2)
|
||||
|
||||
def overlapping_rank(items, analysis, limit=None):
|
||||
rendezvous.wait(timeout=1)
|
||||
return original_rank(items, analysis, limit=limit)
|
||||
|
||||
mod.rank_search_items = overlapping_rank
|
||||
with ThreadPoolExecutor(max_workers=2) as executor:
|
||||
futures = [
|
||||
executor.submit(mod.search_local_index, {"q": "shared needle", "limit": 2})
|
||||
for _ in range(2)
|
||||
]
|
||||
results = [future.result(timeout=2) for future in futures]
|
||||
|
||||
assert all(len(result["items"]) == 2 for result in results)
|
||||
|
||||
|
||||
def test_index_schema_initializes_once_and_revalidates_replacement(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
original_initialize = mod.initialize_index_schema
|
||||
calls = []
|
||||
|
||||
def counted_initialize(conn):
|
||||
calls.append(str(mod.INDEX_PATH))
|
||||
return original_initialize(conn)
|
||||
|
||||
mod.initialize_index_schema = counted_initialize
|
||||
with mod.index_connect():
|
||||
pass
|
||||
with mod.index_connect():
|
||||
pass
|
||||
assert len(calls) == 1
|
||||
|
||||
replacement = tmp_path / "replacement.sqlite"
|
||||
replacement.touch()
|
||||
replacement.replace(mod.INDEX_PATH)
|
||||
with mod.index_connect():
|
||||
pass
|
||||
assert len(calls) == 2
|
||||
|
||||
|
||||
def test_discover_contract_advertises_local_routing_and_pagination():
|
||||
mod = load_module()
|
||||
tool = next(item for item in mod.tools() if item["name"] == "nap_discover")
|
||||
properties = tool["inputSchema"]["properties"]
|
||||
|
||||
assert "cursor" in properties
|
||||
assert properties["mode"]["enum"] == ["hybrid", "server", "local"]
|
||||
|
||||
|
||||
def test_query_planner_normalizes_plurals_and_reports_intent():
|
||||
mod = load_module()
|
||||
|
||||
analysis = mod.analyze_search_query(
|
||||
"where did we decide how search queries and policies should work"
|
||||
)
|
||||
|
||||
assert "query" in analysis["terms"]
|
||||
assert "querie" not in analysis["terms"]
|
||||
assert analysis["inferred_intent"] == "decision"
|
||||
assert len(mod.fts_query_variants(analysis["raw"])[1]) == 1
|
||||
|
||||
|
||||
def test_local_search_pushes_structured_filters_into_candidate_query(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
nodes = [
|
||||
indexed_node(
|
||||
"/decisions/search-governance",
|
||||
node_type="decision",
|
||||
tags=["search"],
|
||||
content="search governance",
|
||||
),
|
||||
indexed_node(
|
||||
"/decisions/old-search-governance",
|
||||
node_type="decision",
|
||||
tags=["search"],
|
||||
status="superseded",
|
||||
content="search governance",
|
||||
),
|
||||
indexed_node(
|
||||
"/plans/search-governance",
|
||||
node_type="plan",
|
||||
tags=["search"],
|
||||
content="search governance",
|
||||
),
|
||||
indexed_node(
|
||||
"/decisions/other-search",
|
||||
node_type="decision",
|
||||
tags=["other"],
|
||||
content="search governance",
|
||||
),
|
||||
]
|
||||
for node in nodes:
|
||||
mod.index_node(node)
|
||||
|
||||
result = mod.search_local_index(
|
||||
{
|
||||
"q": "search governance",
|
||||
"folder_path": "/decisions",
|
||||
"tag": "search",
|
||||
"type": "decision",
|
||||
"status": "active",
|
||||
"limit": 10,
|
||||
}
|
||||
)
|
||||
|
||||
assert [item["full_path"] for item in result["items"]] == [
|
||||
"/decisions/search-governance"
|
||||
]
|
||||
assert result["index"]["query_count"] == 1
|
||||
assert set(result["index"]["filters_applied"]) >= {
|
||||
"folder_path",
|
||||
"tag",
|
||||
"type",
|
||||
"status",
|
||||
}
|
||||
|
||||
|
||||
def test_explicit_intent_boosts_without_filtering(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/decisions/gateway-relay-authentication",
|
||||
node_type="decision",
|
||||
content="gateway relay authentication",
|
||||
)
|
||||
)
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/work/current/gateway-relay-authentication",
|
||||
content="gateway relay authentication",
|
||||
)
|
||||
)
|
||||
|
||||
result = mod.search_local_index(
|
||||
{"q": "gateway relay authentication", "intent": "decision", "limit": 10}
|
||||
)
|
||||
|
||||
assert result["items"][0]["full_path"].startswith("/decisions/")
|
||||
assert len(result["items"]) == 2
|
||||
|
||||
|
||||
def test_complete_filtered_local_search_stays_off_network(tmp_path):
|
||||
mod = load_module()
|
||||
configure_local_index(mod, tmp_path)
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/documentation/facturacion-electronica",
|
||||
tags=["billing"],
|
||||
content="facturación electrónica",
|
||||
)
|
||||
)
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/work/current/facturacion-migration",
|
||||
tags=["migration"],
|
||||
content="facturación electrónica",
|
||||
)
|
||||
)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
calls = []
|
||||
mod.TOKEN = "disposable-test-token" # synthetic login; requests go to feed_provider only
|
||||
mod.request_json = feed_provider(calls)
|
||||
mod.search_memory({"q": "facturación", "tag": "billing", "limit": 1, "view": "summary"})
|
||||
assert len(calls) == 2 # one-time change-feed bootstrap: head + catch-up
|
||||
calls.clear()
|
||||
|
||||
result = mod.search_memory(
|
||||
{
|
||||
"q": "facturación",
|
||||
"tag": "billing",
|
||||
"limit": 1,
|
||||
"view": "summary",
|
||||
}
|
||||
)
|
||||
|
||||
assert result["sources"] == ["local"]
|
||||
assert [item["full_path"] for item in result["items"]] == [
|
||||
"/documentation/facturacion-electronica"
|
||||
]
|
||||
assert calls == [] # a complete, tracked, recently refreshed index stays off the network
|
||||
assert not result.get("warnings")
|
||||
|
||||
|
||||
def test_discovery_schema_documents_dynamic_sort_and_intent():
|
||||
mod = load_module()
|
||||
tool = next(item for item in mod.tools() if item["name"] == "nap_discover")
|
||||
properties = tool["inputSchema"]["properties"]
|
||||
|
||||
assert properties["intent"]["enum"] == [
|
||||
"any",
|
||||
"decision",
|
||||
"rule",
|
||||
"plan",
|
||||
"task",
|
||||
"implementation",
|
||||
"documentation",
|
||||
]
|
||||
assert "default" not in properties["sort"]
|
||||
assert "relevance" in properties["sort"]["description"]
|
||||
|
||||
@@ -0,0 +1,308 @@
|
||||
"""Local search quality: FTS schema v2, weighted ranking and query snippets.
|
||||
|
||||
Every test runs the real index, query planner and ranking code against a
|
||||
disposable SQLite file; no network is used.
|
||||
"""
|
||||
|
||||
import importlib.util
|
||||
import pathlib
|
||||
import sqlite3
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = pathlib.Path(__file__).resolve().parents[1] / "resources" / "scripts" / "napseer_mcp_server.py"
|
||||
|
||||
|
||||
def load_module(tmp_path):
|
||||
sys.path.insert(0, str(SCRIPT.parent))
|
||||
spec = importlib.util.spec_from_file_location("napseer_mcp_server_search_quality_test", SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
mod.AUTH_DIR = tmp_path
|
||||
mod.INDEX_PATH = tmp_path / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = tmp_path / "index.lock"
|
||||
mod.resolve_project_id = lambda _args: "project-1"
|
||||
mod.project_encryption_state = lambda _project_id: "standard"
|
||||
|
||||
def unexpected_request(*_args, **_kwargs):
|
||||
raise AssertionError("local search quality tests must not use the network")
|
||||
|
||||
mod.request_json = unexpected_request
|
||||
return mod
|
||||
|
||||
|
||||
def indexed_node(path, *, content="", status="active", updated_at="2026-08-22T00:00:00Z", **extra):
|
||||
name = path.rsplit("/", 1)[-1]
|
||||
node = {
|
||||
"id": f"node-{path.strip('/').replace('/', '-')}",
|
||||
"project_id": "project-1",
|
||||
"full_path": path,
|
||||
"folder_path": path.rsplit("/", 1)[0] or None,
|
||||
"name": name,
|
||||
"title": name,
|
||||
"type": "note",
|
||||
"tags": [],
|
||||
"aliases": [],
|
||||
"links": [],
|
||||
"metadata": {"status": status},
|
||||
"content_text": content,
|
||||
"encryption_state": "legacy_plaintext",
|
||||
"updated_at": updated_at,
|
||||
}
|
||||
node.update(extra)
|
||||
return node
|
||||
|
||||
|
||||
def paths(result):
|
||||
return [item["full_path"] for item in result["items"]]
|
||||
|
||||
|
||||
# --- FTS schema v2 ---------------------------------------------------------
|
||||
|
||||
|
||||
def test_stemming_recalls_other_inflections(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(indexed_node("/notes/release-log", content="The deployment pipeline promotes images."))
|
||||
|
||||
assert paths(mod.search_local_index({"q": "deploying", "limit": 5})) == ["/notes/release-log"]
|
||||
assert paths(mod.search_local_index({"q": "configured deployments", "limit": 5})) == ["/notes/release-log"]
|
||||
|
||||
|
||||
def test_prefix_query_uses_prefix_index_and_matches(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(indexed_node("/notes/cluster", content="kubernetes control plane"))
|
||||
|
||||
with mod.index_connect() as conn:
|
||||
sql = conn.execute(
|
||||
"SELECT sql FROM sqlite_master WHERE name = 'local_index_fts'"
|
||||
).fetchone()["sql"]
|
||||
assert "porter unicode61 remove_diacritics 2" in sql
|
||||
assert "prefix = '2 3'" in sql
|
||||
assert paths(mod.search_local_index({"q": "kuber", "limit": 5})) == ["/notes/cluster"]
|
||||
|
||||
|
||||
def test_two_character_terms_are_searchable(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(indexed_node("/notes/note-a", content="db migration runbook for postgres cutover steps"))
|
||||
mod.index_node(indexed_node("/notes/note-b", content="migration", updated_at="2026-09-01T00:00:00Z"))
|
||||
|
||||
analysis = mod.analyze_search_query("db migration")
|
||||
result = mod.search_local_index({"q": "db migration", "limit": 5})
|
||||
|
||||
assert "db" in analysis["terms"]
|
||||
assert paths(result)[0] == "/notes/note-a"
|
||||
|
||||
|
||||
def test_domain_words_are_not_stop_words():
|
||||
mod = load_module(pathlib.Path("/nonexistent"))
|
||||
concepts = mod.analyze_search_query("plan work decide")["concepts"]
|
||||
|
||||
assert {"plan", "work", "decide"} <= set(concepts)
|
||||
|
||||
|
||||
def test_single_node_fts_delete_uses_rowid_not_scan(tmp_path, monkeypatch):
|
||||
mod = load_module(tmp_path)
|
||||
node = indexed_node("/notes/rowid-target", content="first version")
|
||||
mod.index_node(node)
|
||||
statements = []
|
||||
original_open = mod.open_index_connection
|
||||
|
||||
def traced_open():
|
||||
conn = original_open()
|
||||
conn.set_trace_callback(statements.append)
|
||||
return conn
|
||||
|
||||
monkeypatch.setattr(mod, "open_index_connection", traced_open)
|
||||
mod.index_node({**node, "content_text": "second version"})
|
||||
mod.remove_indexed_node(node["id"])
|
||||
|
||||
fts_deletes = [s for s in statements if "DELETE FROM local_index_fts" in s]
|
||||
assert fts_deletes
|
||||
assert all("rowid" in s and "node_id" not in s for s in fts_deletes)
|
||||
with mod.index_connect() as conn:
|
||||
assert conn.execute("SELECT COUNT(*) FROM local_index_fts").fetchone()[0] == 0
|
||||
assert conn.execute("SELECT COUNT(*) FROM local_index_nodes").fetchone()[0] == 0
|
||||
|
||||
|
||||
def test_reindexing_a_node_replaces_its_single_fts_row(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
node = indexed_node("/notes/replaced", content="alpha")
|
||||
mod.index_node(node)
|
||||
mod.index_node({**node, "content_text": "omega"})
|
||||
|
||||
with mod.index_connect() as conn:
|
||||
rows = conn.execute("SELECT rowid, content_text FROM local_index_fts").fetchall()
|
||||
mapped = conn.execute("SELECT fts_rowid FROM local_index_nodes").fetchone()[0]
|
||||
assert [row["content_text"] for row in rows] == ["omega"]
|
||||
assert rows[0]["rowid"] == mapped
|
||||
|
||||
|
||||
V1_SCHEMA = [
|
||||
"""CREATE TABLE local_index_nodes (
|
||||
node_id TEXT PRIMARY KEY, project_id TEXT NOT NULL, full_path TEXT NOT NULL,
|
||||
name TEXT NOT NULL, folder_path TEXT, node_type TEXT NOT NULL, tags_json TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL, indexed_at TEXT NOT NULL, title TEXT, status TEXT,
|
||||
archived INTEGER NOT NULL DEFAULT 0, aliases_json TEXT NOT NULL DEFAULT '[]')""",
|
||||
"""CREATE TABLE local_index_state (
|
||||
project_id TEXT PRIMARY KEY, graph_complete INTEGER NOT NULL DEFAULT 0, graph_completed_at TEXT)""",
|
||||
"""CREATE VIRTUAL TABLE local_index_fts USING fts5(
|
||||
node_id UNINDEXED, project_id UNINDEXED, full_path, name, folder_path, node_type,
|
||||
tags, aliases, links, metadata_text, content_text)""",
|
||||
]
|
||||
|
||||
|
||||
def test_v1_index_upgrades_in_place_and_keeps_completeness(tmp_path):
|
||||
index_path = tmp_path / "index.sqlite"
|
||||
conn = sqlite3.connect(str(index_path))
|
||||
for statement in V1_SCHEMA:
|
||||
conn.execute(statement)
|
||||
conn.execute(
|
||||
"INSERT INTO local_index_nodes VALUES ('n1','project-1','/notes/legacy','legacy','/notes','note','[]',"
|
||||
"'2026-08-01T00:00:00Z','2026-08-01T00:00:00Z','Legacy title','active',0,'[]')"
|
||||
)
|
||||
conn.execute(
|
||||
"INSERT INTO local_index_fts VALUES ('n1','project-1','/notes/legacy','legacy','/notes','note','','','',"
|
||||
"'','the deployment runbook body')"
|
||||
)
|
||||
conn.execute("INSERT INTO local_index_state VALUES ('project-1', 1, '2026-08-01T00:00:00Z')")
|
||||
conn.commit()
|
||||
conn.close()
|
||||
|
||||
mod = load_module(tmp_path)
|
||||
result = mod.search_local_index({"q": "deploying", "limit": 5})
|
||||
|
||||
assert paths(result) == ["/notes/legacy"]
|
||||
with mod.index_connect() as conn:
|
||||
assert conn.execute("PRAGMA user_version").fetchone()[0] == mod.INDEX_SCHEMA_VERSION
|
||||
assert conn.execute("SELECT COUNT(*) FROM local_index_fts").fetchone()[0] == 1
|
||||
assert "porter" in conn.execute(
|
||||
"SELECT sql FROM sqlite_master WHERE name = 'local_index_fts'"
|
||||
).fetchone()[0]
|
||||
assert conn.execute("SELECT fts_rowid FROM local_index_nodes").fetchone()[0] is not None
|
||||
assert mod.local_index_diagnostics("project-1")["graph_complete"] is True
|
||||
# The upgrade drops no rows and a legacy-migrated index has no feed cursor yet.
|
||||
assert mod.local_index_feed_state("project-1")["feed_cursor"] is None
|
||||
|
||||
|
||||
# --- ranking --------------------------------------------------------------
|
||||
|
||||
|
||||
def test_body_full_coverage_beats_authority_path_partial_match(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(indexed_node("/rules/quasar", content="unrelated governance text"))
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/work/current/notes-a",
|
||||
content="The quasar rollout checklist is kept here with every gate listed.",
|
||||
)
|
||||
)
|
||||
|
||||
result = mod.search_local_index({"q": "quasar rollout checklist", "limit": 5})
|
||||
|
||||
assert paths(result) == ["/work/current/notes-a", "/rules/quasar"]
|
||||
|
||||
|
||||
def test_superseded_status_is_demoted_unless_requested(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(
|
||||
indexed_node("/decisions/retry-a", content="gateway retry budget", status="superseded",
|
||||
updated_at="2026-09-10T00:00:00Z")
|
||||
)
|
||||
mod.index_node(
|
||||
indexed_node("/decisions/retry-b", content="gateway retry budget", status="active",
|
||||
updated_at="2026-08-01T00:00:00Z")
|
||||
)
|
||||
|
||||
default = mod.search_local_index({"q": "gateway retry budget", "limit": 5})
|
||||
requested = mod.search_local_index({"q": "superseded gateway retry budget", "limit": 5})
|
||||
|
||||
assert paths(default) == ["/decisions/retry-b", "/decisions/retry-a"]
|
||||
assert paths(requested)[0] == "/decisions/retry-a"
|
||||
|
||||
|
||||
def test_explicit_exact_identity_stays_on_top_of_requested_intent(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(indexed_node("/notes/cache-invalidation", content="cache invalidation"))
|
||||
mod.index_node(indexed_node("/decisions/cache-layer", content="cache invalidation " * 5))
|
||||
|
||||
result = mod.search_local_index({"q": "cache invalidation", "intent": "decision", "limit": 5})
|
||||
|
||||
assert paths(result)[0] == "/notes/cache-invalidation"
|
||||
assert "exact_identity" in result["items"][0]["match_reason"]
|
||||
|
||||
|
||||
# --- snippets -------------------------------------------------------------
|
||||
|
||||
|
||||
def test_local_search_returns_query_centred_snippet(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
filler = "Unrelated operational background sentence. " * 20
|
||||
mod.index_node(
|
||||
indexed_node("/notes/long-note", content=f"{filler}The zanzibar failover drill runs weekly. {filler}")
|
||||
)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
local = mod.search_local_index({"q": "zanzibar", "limit": 5})
|
||||
envelope = mod.search_memory({"q": "zanzibar", "limit": 5, "view": "summary", "mode": "local"})
|
||||
|
||||
assert "zanzibar" in local["items"][0]["snippet"].lower()
|
||||
assert "zanzibar" in envelope["items"][0]["snippet"].lower()
|
||||
assert "zanzibar" in envelope["items"][0]["preview"].lower()
|
||||
|
||||
|
||||
def test_e2e_nodes_have_no_snippet_or_body_index(tmp_path):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/notes/zanzibar-sealed",
|
||||
content="plaintext-secret-marker zanzibar",
|
||||
encryption_state="encrypted",
|
||||
encrypted_content_envelope={"ciphertext": "opaque"},
|
||||
)
|
||||
)
|
||||
|
||||
assert mod.search_local_index({"q": "plaintext-secret-marker", "limit": 5})["items"] == []
|
||||
result = mod.search_local_index({"q": "zanzibar", "limit": 5})
|
||||
assert paths(result) == ["/notes/zanzibar-sealed"]
|
||||
assert not result["items"][0].get("snippet")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("query", ["memory-source-policy", "memory source policy"])
|
||||
def test_hyphenated_identity_query_matches_tokenized_path(tmp_path, query):
|
||||
mod = load_module(tmp_path)
|
||||
mod.index_node(indexed_node("/rules/memory-source-policy", content="Canonical memory rule."))
|
||||
mod.index_node(indexed_node("/notes/other", content="policy about memory elsewhere"))
|
||||
|
||||
assert paths(mod.search_local_index({"q": query, "limit": 5}))[0] == "/rules/memory-source-policy"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"query, expected",
|
||||
[
|
||||
("existing-alias", "node-notes-first"),
|
||||
("secondaryhandle", "node-notes-second"),
|
||||
("/notes/existing-work", "node-notes-existing-work"),
|
||||
("existing-work", "node-notes-existing-work"),
|
||||
],
|
||||
)
|
||||
def test_discover_local_literal_alias_or_path_is_top_exact_identity(tmp_path, query, expected):
|
||||
mod = load_module(tmp_path)
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
mod.require_unlocked = lambda _name: None
|
||||
mod.index_node(indexed_node("/notes/first", content="body", aliases=["existing-alias"]))
|
||||
mod.index_node(indexed_node("/notes/second", content="body", aliases=["secondaryhandle"]))
|
||||
mod.index_node(indexed_node("/notes/existing-work", content="body"))
|
||||
# Authoritative distractor whose body repeats the words of every literal.
|
||||
mod.index_node(
|
||||
indexed_node(
|
||||
"/rules/distractor",
|
||||
content="existing alias existing work secondaryhandle notes " * 4,
|
||||
)
|
||||
)
|
||||
mod.set_local_graph_index_complete("project-1", True)
|
||||
|
||||
result = mod.call_tool_impl("nap_discover", {"q": query, "mode": "local", "view": "summary"})
|
||||
|
||||
assert result["items"][0]["node_id"] == expected
|
||||
assert "exact_identity" in result["items"][0]["match_reason"]
|
||||
@@ -70,10 +70,26 @@ for line in sys.stdin:
|
||||
|
||||
|
||||
class SupervisorClient:
|
||||
def __init__(self, worker_path: pathlib.Path, project_root: pathlib.Path | None = None):
|
||||
def __init__(
|
||||
self,
|
||||
worker_path: pathlib.Path,
|
||||
project_root: pathlib.Path | None = None,
|
||||
watch_paths: tuple[pathlib.Path, ...] = (),
|
||||
response_timeout: int = 5,
|
||||
cancel_grace: int = 5,
|
||||
max_in_flight: int = 32,
|
||||
write_timeout: int = 5,
|
||||
):
|
||||
environment = os.environ.copy()
|
||||
environment["NAPSEER_MCP_WORKER_PATH"] = str(worker_path)
|
||||
environment["NAPSEER_MCP_RESPONSE_TIMEOUT_SECONDS"] = "5"
|
||||
if watch_paths:
|
||||
environment["NAPSEER_MCP_WORKER_WATCH_PATHS"] = os.pathsep.join(
|
||||
str(path) for path in watch_paths
|
||||
)
|
||||
environment["NAPSEER_MCP_RESPONSE_TIMEOUT_SECONDS"] = str(response_timeout)
|
||||
environment["NAPSEER_MCP_CANCEL_GRACE_SECONDS"] = str(cancel_grace)
|
||||
environment["NAPSEER_MCP_MAX_IN_FLIGHT_REQUESTS"] = str(max_in_flight)
|
||||
environment["NAPSEER_MCP_WRITE_TIMEOUT_SECONDS"] = str(write_timeout)
|
||||
environment["NAPSEER_TELEMETRY"] = "0"
|
||||
if project_root is not None:
|
||||
environment["NAPSEER_PROJECT_ROOT"] = str(project_root)
|
||||
@@ -83,6 +99,7 @@ class SupervisorClient:
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=environment,
|
||||
bufsize=0,
|
||||
)
|
||||
|
||||
def request(self, request_id: int, method: str, params=None):
|
||||
@@ -272,6 +289,33 @@ def test_reloads_replaced_worker_between_requests(tmp_path):
|
||||
client.close()
|
||||
|
||||
|
||||
def test_reloads_stable_launcher_when_watched_runtime_is_replaced(tmp_path):
|
||||
runtime_path = tmp_path / "runtime.py"
|
||||
launcher_path = tmp_path / "launcher.py"
|
||||
write_worker(runtime_path, 1)
|
||||
launcher_path.write_text(
|
||||
"import runpy\n"
|
||||
f"runpy.run_path({str(runtime_path)!r}, run_name='__main__')\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
client = SupervisorClient(launcher_path, watch_paths=(runtime_path,))
|
||||
try:
|
||||
first = client.request(1, "initialize")
|
||||
launcher_identity = launcher_path.stat().st_mtime_ns
|
||||
|
||||
time.sleep(0.01)
|
||||
write_worker(runtime_path, 2)
|
||||
os.utime(runtime_path, None)
|
||||
|
||||
second = client.request(2, "tools/list")
|
||||
|
||||
assert first["result"]["version"] == 1
|
||||
assert second["result"]["version"] == 2
|
||||
assert launcher_path.stat().st_mtime_ns == launcher_identity
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_worker_crash_returns_error_without_closing_transport(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
write_worker(worker_path, 1)
|
||||
@@ -285,3 +329,201 @@ def test_worker_crash_returns_error_without_closing_transport(tmp_path):
|
||||
assert client.process.poll() is None
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def write_lifecycle_worker(path, version=1):
|
||||
# The actual worker dispatcher owns cancellation/completion; the handler
|
||||
# supplies deterministic slow and uncooperative work without credentials.
|
||||
path.write_text(f'''import json, os, pathlib, runpy, sys, time
|
||||
sys.path.insert(0, {str(REAL_WORKER.parent)!r})
|
||||
mod = runpy.run_path({str(REAL_WORKER)!r})
|
||||
root = pathlib.Path({str(path.parent)!r})
|
||||
def handle(message):
|
||||
name = (message.get("params") or {{}}).get("name")
|
||||
if name in ("slow", "stuck"):
|
||||
(root / (str(message["id"]) + ".started")).write_text("started")
|
||||
try:
|
||||
deadline = time.monotonic() + 10
|
||||
while time.monotonic() < deadline:
|
||||
if name == "slow":
|
||||
mod["raise_if_mcp_request_cancelled"]()
|
||||
time.sleep(0.01)
|
||||
finally:
|
||||
time.sleep(0.3)
|
||||
(root / (str(message["id"]) + ".cleaned")).write_text("cleaned")
|
||||
return mod["rpc_result"](message.get("id"), {{"pid": os.getpid(), "version": {version}}})
|
||||
runtime = mod["McpRequestRuntime"](handler=handle)
|
||||
try:
|
||||
for line in sys.stdin:
|
||||
runtime.dispatch(json.loads(line))
|
||||
finally:
|
||||
runtime.close()
|
||||
''')
|
||||
|
||||
|
||||
def wait_for_path(path, timeout=3):
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
if path.exists():
|
||||
return
|
||||
time.sleep(0.01)
|
||||
raise AssertionError("worker did not reach the expected lifecycle boundary")
|
||||
|
||||
|
||||
def test_cancelled_work_counts_until_cleanup_before_source_replacement(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
write_lifecycle_worker(worker_path)
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path)
|
||||
try:
|
||||
client.send(1, "tools/call", {"name": "slow"})
|
||||
wait_for_path(tmp_path / "1.started")
|
||||
client.notify("notifications/cancelled", {"requestId": 1})
|
||||
write_lifecycle_worker(worker_path, version=2)
|
||||
during_cleanup = client.request(2, "tools/list")
|
||||
assert during_cleanup["result"]["version"] == 1
|
||||
wait_for_path(tmp_path / "1.cleaned")
|
||||
time.sleep(0.05)
|
||||
after_cleanup = client.request(3, "tools/list")
|
||||
assert after_cleanup["result"]["version"] == 2
|
||||
assert after_cleanup["result"]["pid"] != during_cleanup["result"]["pid"]
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_timeout_cancels_work_and_keeps_transport_usable(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
write_lifecycle_worker(worker_path)
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path, response_timeout=1)
|
||||
try:
|
||||
before = client.request(1, "tools/list")
|
||||
failed = client.request(2, "tools/call", {"name": "slow"})
|
||||
assert failed["error"]["code"] == -32098
|
||||
wait_for_path(tmp_path / "2.cleaned")
|
||||
recovered = client.request(3, "tools/list")
|
||||
assert recovered["result"]["pid"] == before["result"]["pid"]
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_cancelled_work_cannot_bypass_admission_limit(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
write_lifecycle_worker(worker_path)
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path, max_in_flight=2)
|
||||
try:
|
||||
client.send(1, "tools/call", {"name": "stuck"})
|
||||
wait_for_path(tmp_path / "1.started")
|
||||
client.notify("notifications/cancelled", {"requestId": 1})
|
||||
client.send(2, "tools/call", {"name": "stuck"})
|
||||
wait_for_path(tmp_path / "2.started")
|
||||
client.send(3, "tools/call", {"name": "stuck"})
|
||||
rejected = client.receive(timeout=1)
|
||||
assert rejected["id"] == 3
|
||||
assert rejected["error"]["code"] == -32097
|
||||
assert not (tmp_path / "3.started").exists()
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_uncooperative_timeout_restarts_worker_with_no_replay(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
write_lifecycle_worker(worker_path)
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path,
|
||||
response_timeout=1, cancel_grace=1)
|
||||
try:
|
||||
before = client.request(1, "tools/list")
|
||||
failed = client.request(2, "tools/call", {"name": "stuck"})
|
||||
assert failed["error"]["code"] == -32098
|
||||
time.sleep(1.3)
|
||||
recovered = client.request(3, "tools/list")
|
||||
assert recovered["result"]["pid"] != before["result"]["pid"]
|
||||
assert not (tmp_path / "2.cleaned").exists()
|
||||
assert client.process.poll() is None
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_reused_id_is_rejected_until_cancelled_request_finishes(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
write_lifecycle_worker(worker_path)
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path)
|
||||
try:
|
||||
client.send(1, "tools/call", {"name": "slow"})
|
||||
wait_for_path(tmp_path / "1.started")
|
||||
client.notify("notifications/cancelled", {"requestId": 1})
|
||||
rejected = client.request(1, "tools/list")
|
||||
assert rejected["error"]["code"] == -32097
|
||||
wait_for_path(tmp_path / "1.cleaned")
|
||||
time.sleep(0.05)
|
||||
assert client.request(1, "tools/list")["result"]["version"] == 1
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_blocked_worker_stdin_has_a_deadline_and_recovers(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
worker_path.write_text("import time\ntime.sleep(60)\n")
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path, write_timeout=1)
|
||||
try:
|
||||
client.send(1, "tools/call", {"name": "large", "arguments": {"data": "x" * 262144}})
|
||||
failed = client.receive(timeout=3)
|
||||
assert failed["error"]["code"] == -32098
|
||||
write_worker(worker_path, 2)
|
||||
assert client.request(2, "tools/list")["result"]["version"] == 2
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_eof_stops_worker_with_unfinished_requests(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
write_lifecycle_worker(worker_path)
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path)
|
||||
try:
|
||||
worker_pid = client.request(1, "tools/list")["result"]["pid"]
|
||||
client.send(2, "tools/call", {"name": "stuck"})
|
||||
wait_for_path(tmp_path / "2.started")
|
||||
client.close()
|
||||
assert client.process.returncode == 0
|
||||
try:
|
||||
os.kill(worker_pid, 0)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("worker survived supervisor EOF")
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_initialize_timeout_retires_uncancellable_worker(tmp_path):
|
||||
worker_path = tmp_path / "worker.py"
|
||||
worker_path.write_text('''import json, os, sys, time
|
||||
for line in sys.stdin:
|
||||
msg = json.loads(line)
|
||||
if msg.get("method") == "initialize":
|
||||
time.sleep(10)
|
||||
print(json.dumps({"jsonrpc": "2.0", "id": msg["id"], "result": {"pid": os.getpid()}}), flush=True)
|
||||
''')
|
||||
client = SupervisorClient(worker_path, project_root=tmp_path, response_timeout=1)
|
||||
try:
|
||||
before = client.request(1, "tools/list")
|
||||
failed = client.request(2, "initialize")
|
||||
assert failed["error"]["code"] == -32098
|
||||
recovered = client.request(3, "tools/list")
|
||||
assert recovered["result"]["pid"] != before["result"]["pid"]
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_real_worker_contract_exposes_recovery_without_private_notifications(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "maintenance")
|
||||
client = SupervisorClient(REAL_WORKER, project_root=tmp_path)
|
||||
try:
|
||||
assert "result" in client.request(1, "initialize")
|
||||
response = client.request(2, "tools/call", {"name": "nap_contract", "arguments": {}})
|
||||
contract = response["result"]["structuredContent"]["request_lifecycle"]
|
||||
assert contract["max_unfinished_requests"] == 32
|
||||
assert contract["supervised"] is True
|
||||
assert contract["rejected_error_code"] == -32097
|
||||
assert contract["uncertain_error_code"] == -32098
|
||||
assert not (tmp_path / ".napseer").exists()
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
@@ -4,6 +4,8 @@ import sys
|
||||
import threading
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def load_module():
|
||||
script_path = (
|
||||
@@ -164,7 +166,8 @@ def test_mutating_tools_are_serialized_while_reads_can_overlap(monkeypatch):
|
||||
|
||||
def test_concurrent_first_reads_share_one_project_bootstrap(tmp_path, monkeypatch):
|
||||
mod = load_module()
|
||||
mod.AUTH_PATH = tmp_path / "missing-auth.json"
|
||||
mod.TOKEN = "synthetic-configured-access"
|
||||
mod.AUTH_PATH = mod.AUTH_DIR / "missing-auth.json"
|
||||
mod.DEFAULT_PROJECT_ID = None
|
||||
monkeypatch.setattr(mod, "refresh_public_auth_state", lambda: None)
|
||||
calls = []
|
||||
@@ -242,3 +245,160 @@ def test_auth_renewal_reuses_token_refreshed_by_another_request(monkeypatch):
|
||||
"refresh_expires_at": "later",
|
||||
"project_id": "project-1",
|
||||
}
|
||||
|
||||
|
||||
def test_tool_call_notification_cannot_execute_a_mutation(monkeypatch):
|
||||
mod = load_module()
|
||||
calls = []
|
||||
monkeypatch.setattr(mod, "call_tool", lambda *args: calls.append(args))
|
||||
assert mod.handle({"jsonrpc": "2.0", "method": "tools/call", "params": {
|
||||
"name": "nap_create_node", "arguments": {"name": "unacknowledged"},
|
||||
}}) is None
|
||||
assert calls == []
|
||||
|
||||
|
||||
def test_worker_admission_is_bounded_and_recovers():
|
||||
mod = load_module()
|
||||
release = threading.Event()
|
||||
emitted = []
|
||||
executed = []
|
||||
|
||||
def handler(message):
|
||||
executed.append(message["id"])
|
||||
release.wait(2)
|
||||
return mod.rpc_result(message["id"], {"ok": True})
|
||||
|
||||
runtime = mod.McpRequestRuntime(handler=handler, sender=emitted.append,
|
||||
max_workers=2, max_in_flight=2)
|
||||
try:
|
||||
for request_id in (1, 2, 3):
|
||||
runtime.dispatch({"jsonrpc": "2.0", "id": request_id, "method": "tools/call"})
|
||||
wait_until(lambda: any(item["id"] == 3 for item in emitted))
|
||||
rejection = next(item for item in emitted if item["id"] == 3)
|
||||
assert rejection["error"]["code"] == -32097
|
||||
assert 3 not in executed
|
||||
release.set()
|
||||
wait_until(lambda: len(emitted) == 3)
|
||||
runtime.dispatch({"jsonrpc": "2.0", "id": 4, "method": "tools/call"})
|
||||
wait_until(lambda: any(item["id"] == 4 for item in emitted))
|
||||
finally:
|
||||
release.set()
|
||||
runtime.close()
|
||||
|
||||
|
||||
def test_cancelled_mutation_waiting_for_lock_never_starts(monkeypatch):
|
||||
mod = load_module()
|
||||
cancelled = threading.Event()
|
||||
waiting = threading.Event()
|
||||
calls = []
|
||||
errors = []
|
||||
monkeypatch.setattr(mod, "call_tool_impl", lambda *args: calls.append(args))
|
||||
monkeypatch.setattr(mod, "send_telemetry_event_async", lambda *a, **kw: None)
|
||||
|
||||
def run():
|
||||
mod.MCP_REQUEST_LOCAL.cancelled = cancelled
|
||||
waiting.set()
|
||||
try:
|
||||
mod.call_tool("nap_create_node", {})
|
||||
except mod.SafeToolError as exc:
|
||||
errors.append(exc)
|
||||
|
||||
with mod.MCP_MUTATION_LOCK:
|
||||
thread = threading.Thread(target=run)
|
||||
thread.start()
|
||||
assert waiting.wait(1)
|
||||
cancelled.set()
|
||||
thread.join(2)
|
||||
assert not thread.is_alive()
|
||||
assert calls == []
|
||||
assert len(errors) == 1
|
||||
|
||||
|
||||
def test_cancelled_queue_entries_remain_bounded_until_consumed():
|
||||
mod = load_module()
|
||||
release = threading.Event()
|
||||
executed = []
|
||||
emitted = []
|
||||
|
||||
def handler(message):
|
||||
executed.append(message["id"])
|
||||
release.wait(2)
|
||||
return mod.rpc_result(message["id"], {"ok": True})
|
||||
|
||||
runtime = mod.McpRequestRuntime(handler=handler, sender=emitted.append,
|
||||
max_workers=2, max_in_flight=3)
|
||||
try:
|
||||
for request_id in (1, 2, 3):
|
||||
runtime.dispatch({"id": request_id, "method": "tools/call"})
|
||||
wait_until(lambda: len(executed) == 2)
|
||||
runtime.cancel(3)
|
||||
runtime.dispatch({"id": 4, "method": "tools/call"})
|
||||
wait_until(lambda: any(item["id"] == 4 for item in emitted))
|
||||
assert emitted[0]["error"]["code"] == -32097
|
||||
release.set()
|
||||
wait_until(lambda: len(emitted) == 3)
|
||||
assert executed == [1, 2]
|
||||
finally:
|
||||
release.set()
|
||||
runtime.close()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("cancel_at", ["acquire", "operation"])
|
||||
def test_cancelled_write_releases_its_original_project_lock(tmp_path, monkeypatch, cancel_at):
|
||||
import http.server
|
||||
import json
|
||||
|
||||
monkeypatch.setenv("NAPSEER_PROJECT_ROOT", str(tmp_path))
|
||||
mod = load_module()
|
||||
cancelled = threading.Event()
|
||||
requests = []
|
||||
mutation_ran = []
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
def do_POST(self):
|
||||
self.rfile.read(int(self.headers.get("Content-Length", 0)))
|
||||
requests.append(("POST", self.path))
|
||||
if cancel_at == "acquire":
|
||||
cancelled.set()
|
||||
body = json.dumps({"id": "lock-test", "lease_token": "synthetic-lease"}).encode()
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_DELETE(self):
|
||||
requests.append(("DELETE", self.path))
|
||||
self.send_response(204)
|
||||
self.end_headers()
|
||||
|
||||
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
mod.BASE_URL = f"http://127.0.0.1:{server.server_port}"
|
||||
mod.TOKEN = "synthetic-access"
|
||||
mod.DEFAULT_PROJECT_ID = "original-project"
|
||||
monkeypatch.setattr(mod, "refresh_public_auth_state", lambda: None)
|
||||
mod.MCP_REQUEST_LOCAL.cancelled = cancelled
|
||||
|
||||
def operation(_headers):
|
||||
mutation_ran.append(True)
|
||||
mod.DEFAULT_PROJECT_ID = "different-project"
|
||||
cancelled.set()
|
||||
mod.raise_if_mcp_request_cancelled()
|
||||
|
||||
try:
|
||||
with pytest.raises(mod.SafeToolError):
|
||||
mod.with_project_lock({}, operation)
|
||||
assert requests == [
|
||||
("POST", "/v1/projects/original-project/locks"),
|
||||
("DELETE", "/v1/projects/original-project/locks/lock-test"),
|
||||
]
|
||||
assert mutation_ran == ([] if cancel_at == "acquire" else [True])
|
||||
finally:
|
||||
del mod.MCP_REQUEST_LOCAL.cancelled
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(2)
|
||||
|
||||
@@ -0,0 +1,674 @@
|
||||
"""Project memory secret provisioned once into the local vault, loaded automatically.
|
||||
|
||||
Runs the real wrapper (in-process and as fresh CLI/MCP processes) against a
|
||||
disposable HTTP provider that serves a memory secret wrapped with a fixture
|
||||
passphrase. The fixture secret and passphrase are random per test and every
|
||||
tool output, error, log line and state file is scanned for them (raw, base64,
|
||||
hex) without ever printing them.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import pty
|
||||
import queue
|
||||
import re
|
||||
import select
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import termios
|
||||
import threading
|
||||
import time
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import parse_qs, urlsplit
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = pathlib.Path(__file__).resolve().parents[1] / "resources/scripts/napseer_mcp_server.py"
|
||||
PROJECT = "project-vault-1"
|
||||
ACCOUNT = "22222222-2222-2222-2222-222222222222"
|
||||
BODY_TERM = "zephyrquartz"
|
||||
BODY = f"calibrate the {BODY_TERM} manifold before launch"
|
||||
SETUP_COMMAND = "nap gateway vault setup"
|
||||
|
||||
|
||||
# -- fixture material ---------------------------------------------------------
|
||||
|
||||
class Material:
|
||||
def __init__(self):
|
||||
self.passphrase = "fixture-" + os.urandom(12).hex()
|
||||
self.secrets = {}
|
||||
|
||||
def secret(self, version):
|
||||
return self.secrets.setdefault(version, os.urandom(32))
|
||||
|
||||
def forms(self):
|
||||
forms = [self.passphrase.encode("utf-8")]
|
||||
for value in self.secrets.values():
|
||||
encoded = base64.b64encode(value)
|
||||
forms += [value, encoded, encoded.rstrip(b"="), base64.urlsafe_b64encode(value).rstrip(b"="),
|
||||
value.hex().encode("ascii")]
|
||||
return forms
|
||||
|
||||
|
||||
def assert_no_leak(material, label, blob):
|
||||
data = blob if isinstance(blob, bytes) else json.dumps(blob, default=str).encode("utf-8") \
|
||||
if not isinstance(blob, str) else blob.encode("utf-8")
|
||||
for form in material.forms():
|
||||
if form in data:
|
||||
pytest.fail(f"fixture secret material leaked into {label}") # never print the value
|
||||
|
||||
|
||||
def assert_state_dir_clean(material, *roots):
|
||||
for root in roots:
|
||||
for path in pathlib.Path(root).rglob("*"):
|
||||
if path.is_file():
|
||||
assert_no_leak(material, f"state file {path.name}", path.read_bytes())
|
||||
|
||||
|
||||
# -- disposable provider ------------------------------------------------------
|
||||
|
||||
def secret_record(material, mod, version, wrapping_epoch=1):
|
||||
previous = mod.PROJECT_VAULT_PASSPHRASE
|
||||
mod.PROJECT_VAULT_PASSPHRASE = material.passphrase
|
||||
try:
|
||||
bundle = mod.generate_project_data_key_bundle(PROJECT, ACCOUNT, data_key_epoch=version)
|
||||
bundle["keys"]["memory"]["versions"][str(version)]["key_b64"] = base64.b64encode(
|
||||
material.secret(version)).decode("ascii")
|
||||
wrapped = mod.wrapped_project_key_bundle_record(
|
||||
PROJECT, ACCOUNT, "memory", bundle,
|
||||
wrapping_epoch=wrapping_epoch, bundle_version=version, data_key_epoch=version)
|
||||
finally:
|
||||
mod.PROJECT_VAULT_PASSPHRASE = previous
|
||||
return {
|
||||
"secret_kind": "memory", "version": version, "project_secret_version": version,
|
||||
"wrapping_epoch": wrapping_epoch, "bundle_version": version, "data_key_epoch": version,
|
||||
"wrapped_key_bundle": wrapped, "created_at": "2026-09-01T00:00:00Z", "last_rotated_at": "2026-09-01T00:00:00Z",
|
||||
}
|
||||
|
||||
|
||||
def e2e_node(mod, material, name, version, content, minute=1):
|
||||
secret = {
|
||||
"project_id": PROJECT, "secret_kind": "memory", "version": version, "wrapping_epoch": 1,
|
||||
"bundle_version": version, "data_key_epoch": version, "secret_bytes": material.secret(version),
|
||||
}
|
||||
stamp = f"2026-09-01T12:{minute:02d}:00Z"
|
||||
return {
|
||||
"id": name, "project_id": PROJECT, "full_path": f"/notes/{name}", "folder_path": "/notes",
|
||||
"name": name, "type": "note", "metadata": {}, "tags": [], "aliases": [], "links": [],
|
||||
"content_text": "", "updated_at": stamp, "archived_at": None, "encryption_state": "encrypted",
|
||||
"project_secret_version": version,
|
||||
"encrypted_content_envelope": mod.encrypt_memory_bytes(
|
||||
PROJECT, secret, "node_content", content.encode("utf-8"), {"full_path": f"/notes/{name}"},
|
||||
"text/plain; charset=utf-8"),
|
||||
"encrypted_metadata_envelope": None,
|
||||
}
|
||||
|
||||
|
||||
class Provider:
|
||||
def __init__(self):
|
||||
self.nodes = {}
|
||||
self.events = []
|
||||
self.requests = []
|
||||
self.active = None
|
||||
self.versions = {}
|
||||
self.vault_down = False
|
||||
|
||||
def publish(self, record):
|
||||
self.active = record
|
||||
self.versions[record["version"]] = record
|
||||
|
||||
def put(self, node):
|
||||
self.nodes[node["id"]] = node
|
||||
self.events.append({"type": "node_created", "node_id": node["id"], "full_path": node["full_path"],
|
||||
"new_revision": node["updated_at"]})
|
||||
|
||||
|
||||
def start_provider(provider):
|
||||
vault_re = re.compile(rf"/v1/projects/{PROJECT}/vault/secrets/memory/(active|versions/(\d+))")
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def reply(self, status, payload):
|
||||
body = json.dumps(payload).encode()
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_POST(self):
|
||||
provider.requests.append(("POST", urlsplit(self.path).path))
|
||||
self.reply(404, {"error": {"code": "not_found", "message": "not served"}})
|
||||
|
||||
def do_GET(self):
|
||||
url = urlsplit(self.path)
|
||||
query = parse_qs(url.query)
|
||||
provider.requests.append(("GET", url.path))
|
||||
match = vault_re.fullmatch(url.path)
|
||||
if match:
|
||||
if provider.vault_down:
|
||||
return self.reply(503, {"error": {"code": "unavailable", "message": "vault down"}})
|
||||
record = provider.active if match.group(1) == "active" else provider.versions.get(int(match.group(2)))
|
||||
if record is None:
|
||||
return self.reply(404, {"error": {"code": "not_found", "message": "no such secret"}})
|
||||
return self.reply(200, record)
|
||||
if url.path.endswith("/encryption/status"):
|
||||
return self.reply(200, {"state": "encrypted"})
|
||||
if url.path.endswith("/changes"):
|
||||
after = query.get("after", ["head"])[0]
|
||||
start = len(provider.events) if after == "head" else int(after[1:])
|
||||
page = provider.events[start:]
|
||||
items = [{**event, "cursor": f"c{start + i + 1}", "project_id": PROJECT, "changed_fields": [],
|
||||
"created_at": "2026-09-01T12:00:00Z"} for i, event in enumerate(page)]
|
||||
return self.reply(200, {"items": items, "next_cursor": f"c{start + len(page)}", "has_more": False})
|
||||
if url.path == f"/v1/projects/{PROJECT}/nodes":
|
||||
rows = sorted(provider.nodes.values(), key=lambda n: (n["updated_at"], n["id"]), reverse=True)
|
||||
return self.reply(200, {"items": rows, "next_cursor": None})
|
||||
node_id = url.path.rsplit("/", 1)[-1]
|
||||
if "/nodes/" in url.path and node_id in provider.nodes:
|
||||
return self.reply(200, provider.nodes[node_id])
|
||||
return self.reply(404, {"error": {"code": "not_found", "message": "not served"}})
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
return server, thread
|
||||
|
||||
|
||||
def load_module(name, state_dir, base_url):
|
||||
spec = importlib.util.spec_from_file_location(name, SCRIPT)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
mod.AUTH_DIR = state_dir
|
||||
mod.INDEX_PATH = state_dir / "index.sqlite"
|
||||
mod.INDEX_LOCK_PATH = state_dir / "index.lock"
|
||||
mod.VAULT_PATH = state_dir / "vault.json"
|
||||
mod.LOCAL_VAULT_KEY_PATH = state_dir / "vault.local-key"
|
||||
mod.MEMORY_SECRET_VAULT_PATH = state_dir / "memory-secrets.vault.json"
|
||||
mod.BASE_URL = base_url
|
||||
mod.TOKEN = "disposable-test-token"
|
||||
mod.AUTH = {**mod.AUTH, "account_id": ACCOUNT, "project_id": PROJECT}
|
||||
mod.DEFAULT_PROJECT_ID = PROJECT
|
||||
mod.CONFIGURED_TOOL_PROFILES = {"all"}
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
mod.resolve_project_id = lambda args: args.get("project_id", PROJECT)
|
||||
mod.project_encryption_state = lambda _project_id: "encrypted"
|
||||
return mod
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def env(tmp_path, monkeypatch):
|
||||
for name in ("NAPSEER_VAULT_PASSPHRASE", "NAPSEER_MASTER_PASSPHRASE", "NAPSEER_LOCAL_VAULT_KEY"):
|
||||
monkeypatch.delenv(name, raising=False)
|
||||
monkeypatch.setenv("NAPSEER_TOOL_PROFILES", "all")
|
||||
monkeypatch.setenv("NAPSEER_TELEMETRY", "0")
|
||||
sqlite_tmp = tmp_path / "sqlite-tmp"
|
||||
sqlite_tmp.mkdir()
|
||||
monkeypatch.setenv("SQLITE_TMPDIR", str(sqlite_tmp))
|
||||
monkeypatch.syspath_prepend(str(SCRIPT.parent))
|
||||
provider = Provider()
|
||||
server, thread = start_provider(provider)
|
||||
base_url = f"http://127.0.0.1:{server.server_port}"
|
||||
state_dir = tmp_path / "state"
|
||||
mod = load_module("memory_vault_worker_a", state_dir, base_url)
|
||||
material = Material()
|
||||
provider.publish(secret_record(material, mod, 1))
|
||||
provider.put(e2e_node(mod, material, "secretnote", 1, BODY))
|
||||
try:
|
||||
yield mod, provider, material, state_dir, base_url
|
||||
finally:
|
||||
mod.e2e_shards_close(flush=False)
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
def fresh_process_state(mod):
|
||||
"""What a new worker starts with: no cached secret, no passphrase."""
|
||||
mod.e2e_shards_close(flush=True)
|
||||
mod.MEMORY_SECRET_CACHE.clear()
|
||||
mod.ACTIVE_MEMORY_SECRET_VERSIONS.clear()
|
||||
mod.MEMORY_SECRET_ROTATION_OBSERVED.clear()
|
||||
mod.PROJECT_VAULT_PASSPHRASE = None
|
||||
|
||||
|
||||
def error_of(callable_, *args):
|
||||
with pytest.raises(Exception) as info:
|
||||
callable_(*args)
|
||||
return info.value
|
||||
|
||||
|
||||
# -- in-process behavior ------------------------------------------------------
|
||||
|
||||
def test_setup_stores_secret_not_passphrase_and_loads_without_it(env, capsys):
|
||||
mod, provider, material, state_dir, _ = env
|
||||
before = error_of(mod.active_memory_secret, PROJECT)
|
||||
assert getattr(before, "code", None) == "memory_secret_not_provisioned"
|
||||
assert mod.memory_secret_status(PROJECT)["memory_secret"] == "not_provisioned"
|
||||
|
||||
result = mod.memory_secret_setup({"vault_passphrase": material.passphrase})
|
||||
assert result["status"] == "provisioned" and result["memory_secret"] == "provisioned"
|
||||
assert result["passphrase_stored"] is False and result["passphrase_source"] == "argument"
|
||||
assert result["active_identity"] == {"version": 1, "wrapping_epoch": 1, "bundle_version": 1, "data_key_epoch": 1}
|
||||
assert mod.PROJECT_VAULT_PASSPHRASE is None # not retained after setup
|
||||
|
||||
vault = mod.MEMORY_SECRET_VAULT_PATH
|
||||
assert stat.S_IMODE(vault.stat().st_mode) == 0o600
|
||||
assert stat.S_IMODE(mod.LOCAL_VAULT_KEY_PATH.stat().st_mode) == 0o600
|
||||
assert json.loads(vault.read_text())["schema"] == "napseer.memory-secret-vault.v1"
|
||||
|
||||
fresh_process_state(mod)
|
||||
requests_before = len(provider.requests)
|
||||
secret = mod.active_memory_secret(PROJECT)
|
||||
assert secret["secret_bytes"] == material.secret(1)
|
||||
# Identity comes from the server; no unwrap or passphrase was involved.
|
||||
assert provider.requests[requests_before:] == [("GET", f"/v1/projects/{PROJECT}/vault/secrets/memory/active")]
|
||||
whoami = mod.gateway_status()
|
||||
assert whoami["memory_secret"] == "provisioned" and "deprecations" not in whoami
|
||||
assert whoami["memory_secret_versions"] == [1]
|
||||
|
||||
captured = capsys.readouterr()
|
||||
for label, blob in (("setup result", result), ("whoami", whoami), ("stdout", captured.out),
|
||||
("stderr", captured.err)):
|
||||
assert_no_leak(material, label, blob)
|
||||
assert_state_dir_clean(material, state_dir)
|
||||
|
||||
|
||||
def test_e2e_search_uses_provisioned_secret_and_reports_not_provisioned(env):
|
||||
mod, provider, material, state_dir, _ = env
|
||||
mod.reindex_project({})
|
||||
unprovisioned = mod.search_local_index({"q": BODY_TERM, "limit": 5})
|
||||
assert unprovisioned["items"] == []
|
||||
warning = unprovisioned["warnings"][0]
|
||||
assert warning["code"] == "e2e_index_not_provisioned"
|
||||
assert warning["memory_secret"] == "not_provisioned" and warning["next_action"] == SETUP_COMMAND
|
||||
|
||||
mod.memory_secret_setup({"vault_passphrase": material.passphrase})
|
||||
fresh_process_state(mod)
|
||||
mod.sync_local_index({})
|
||||
found = mod.search_local_index({"q": BODY_TERM, "limit": 5})
|
||||
assert [item["id"] for item in found["items"]] == ["secretnote"]
|
||||
assert found["items"][0]["search_source"] == "local_e2e" and not found.get("warnings")
|
||||
mod.e2e_shards_close(flush=True)
|
||||
assert_state_dir_clean(material, state_dir)
|
||||
|
||||
|
||||
def test_rotation_is_reported_older_versions_still_read_and_passphrase_reprovisions(env):
|
||||
mod, provider, material, state_dir, _ = env
|
||||
mod.memory_secret_setup({"vault_passphrase": material.passphrase})
|
||||
provider.publish(secret_record(material, mod, 2))
|
||||
provider.put(e2e_node(mod, material, "newnote", 2, "second body", minute=2))
|
||||
|
||||
fresh_process_state(mod)
|
||||
rotated = error_of(mod.active_memory_secret, PROJECT)
|
||||
assert rotated.code == "memory_secret_rotated"
|
||||
assert rotated.details == {"memory_secret": "rotated", "next_action": SETUP_COMMAND, "active_version": 2}
|
||||
assert mod.gateway_status()["memory_secret"] == "rotated"
|
||||
assert mod.gateway_status()["memory_secret_next_action"] == SETUP_COMMAND
|
||||
# The stored entry still serves nodes written under the older version...
|
||||
assert mod.memory_secret_for_read(PROJECT, 1)["secret_bytes"] == material.secret(1)
|
||||
assert mod.decrypt_node_for_return(provider.nodes["secretnote"], project_id=PROJECT)["content_text"] == BODY
|
||||
# ...but never the rotated active version.
|
||||
assert error_of(mod.memory_secret_for_read, PROJECT, 2).code == "memory_secret_rotated"
|
||||
mod.reindex_project({})
|
||||
search = mod.search_local_index({"q": BODY_TERM, "limit": 5})
|
||||
assert search["items"] == []
|
||||
assert [(w["code"], w.get("memory_secret")) for w in search["warnings"]] == [("e2e_index_not_provisioned", "rotated")]
|
||||
|
||||
# A doctor in a fresh process detects the rotation from the server identity.
|
||||
fresh_process_state(mod)
|
||||
mod.local_anchor_integrity = lambda: {}
|
||||
doctor = mod.doctor({})
|
||||
assert doctor["memory_secret"]["memory_secret"] == "rotated"
|
||||
assert any(SETUP_COMMAND in warning for warning in doctor["warnings"])
|
||||
|
||||
# A passphrase supplied through the tool-argument path re-provisions automatically.
|
||||
mod.set_project_vault_passphrase(material.passphrase)
|
||||
assert mod.active_memory_secret(PROJECT)["version"] == 2
|
||||
fresh_process_state(mod)
|
||||
assert mod.active_memory_secret(PROJECT)["secret_bytes"] == material.secret(2)
|
||||
assert mod.gateway_status()["memory_secret"] == "provisioned"
|
||||
assert mod.gateway_status()["memory_secret_versions"] == [2, 1]
|
||||
for blob in (rotated.details, str(rotated), doctor):
|
||||
assert_no_leak(material, "rotation output", blob)
|
||||
mod.e2e_shards_close(flush=True)
|
||||
assert_state_dir_clean(material, state_dir)
|
||||
|
||||
|
||||
def test_setup_after_rotation_stores_active_and_older_versions(env):
|
||||
mod, provider, material, _state_dir, _ = env
|
||||
provider.publish(secret_record(material, mod, 2))
|
||||
provider.publish(secret_record(material, mod, 3))
|
||||
del provider.versions[1] # an unavailable older version is reported, not fatal
|
||||
result = mod.memory_secret_setup({"vault_passphrase": material.passphrase})
|
||||
assert result["active_identity"]["version"] == 3
|
||||
assert result["older_versions_stored"] == [2] and result["older_versions_unavailable"] == [1]
|
||||
fresh_process_state(mod)
|
||||
assert mod.memory_secret_for_read(PROJECT, 2)["secret_bytes"] == material.secret(2)
|
||||
|
||||
|
||||
def test_forget_revokes_access_in_this_and_other_running_processes(env):
|
||||
mod, provider, material, state_dir, base_url = env
|
||||
mod.memory_secret_setup({"vault_passphrase": material.passphrase})
|
||||
mod.reindex_project({})
|
||||
assert [item["id"] for item in mod.search_local_index({"q": BODY_TERM})["items"]] == ["secretnote"]
|
||||
mod.e2e_shards_close(flush=True)
|
||||
assert mod.e2e_shard_path(PROJECT).exists()
|
||||
|
||||
# Another running worker has the secret cached in memory.
|
||||
other = load_module("memory_vault_worker_b", state_dir, base_url)
|
||||
other.MEMORY_SECRET_ROTATION_OBSERVED.clear()
|
||||
assert other.active_memory_secret(PROJECT)["secret_bytes"] == material.secret(1)
|
||||
|
||||
removed = mod.memory_secret_forget({})
|
||||
assert removed["status"] == "removed" and removed["removed_entries"] == 1
|
||||
assert removed["memory_secret"] == "not_provisioned"
|
||||
assert not mod.MEMORY_SECRET_VAULT_PATH.exists()
|
||||
assert not mod.e2e_shard_path(PROJECT).exists()
|
||||
|
||||
assert error_of(mod.active_memory_secret, PROJECT).code == "memory_secret_not_provisioned"
|
||||
# The other worker drops its cached copy on the next memory access.
|
||||
assert error_of(other.active_memory_secret, PROJECT).code == "memory_secret_not_provisioned"
|
||||
assert other.e2e_cached_memory_secret(PROJECT) is None
|
||||
search = mod.search_local_index({"q": BODY_TERM})
|
||||
assert search["items"] == [] and search["warnings"][0]["code"] == "e2e_index_not_provisioned"
|
||||
assert mod.memory_secret_forget({})["status"] == "not_provisioned"
|
||||
assert_no_leak(material, "forget output", removed)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("damage", ["tampered", "wrong_local_key", "missing_local_key", "garbage"])
|
||||
def test_damaged_vault_fails_closed_without_leaking(env, capsys, damage):
|
||||
mod, provider, material, state_dir, _ = env
|
||||
mod.memory_secret_setup({"vault_passphrase": material.passphrase})
|
||||
vault = mod.MEMORY_SECRET_VAULT_PATH
|
||||
if damage == "tampered":
|
||||
envelope = json.loads(vault.read_text())
|
||||
ciphertext = bytearray(base64.b64decode(envelope["cipher"]["ciphertext"]))
|
||||
ciphertext[5] ^= 0x01
|
||||
envelope["cipher"]["ciphertext"] = base64.b64encode(bytes(ciphertext)).decode()
|
||||
vault.write_text(json.dumps(envelope))
|
||||
elif damage == "wrong_local_key":
|
||||
mod.LOCAL_VAULT_KEY_PATH.write_text(base64.b64encode(os.urandom(32)).decode() + "\n")
|
||||
elif damage == "missing_local_key":
|
||||
mod.LOCAL_VAULT_KEY_PATH.unlink()
|
||||
else:
|
||||
vault.write_bytes(b"{not json")
|
||||
|
||||
fresh_process_state(mod)
|
||||
failure = error_of(mod.active_memory_secret, PROJECT)
|
||||
assert failure.code == "memory_secret_not_provisioned"
|
||||
status = mod.gateway_status()
|
||||
assert status["memory_secret"] == "not_provisioned"
|
||||
assert status["memory_secret_detail"] == "local_vault_unreadable"
|
||||
mod.reindex_project({})
|
||||
search = mod.search_local_index({"q": BODY_TERM})
|
||||
assert search["items"] == [] and search["warnings"][0]["code"] == "e2e_index_not_provisioned"
|
||||
payload = mod.safe_exception_payload(failure)
|
||||
captured = capsys.readouterr()
|
||||
for label, blob in (("error", payload), ("status", status), ("search", search),
|
||||
("stdout", captured.out), ("stderr", captured.err)):
|
||||
assert_no_leak(material, label, blob)
|
||||
|
||||
# An explicit setup replaces the unreadable vault.
|
||||
mod.memory_secret_setup({"vault_passphrase": material.passphrase})
|
||||
fresh_process_state(mod)
|
||||
assert mod.active_memory_secret(PROJECT)["secret_bytes"] == material.secret(1)
|
||||
|
||||
|
||||
def test_wrong_passphrase_stores_nothing(env):
|
||||
mod, _provider, material, state_dir, _ = env
|
||||
failure = error_of(mod.memory_secret_setup, {"vault_passphrase": "not-" + material.passphrase})
|
||||
assert failure.code == "memory_secret_unwrap_failed"
|
||||
assert not mod.MEMORY_SECRET_VAULT_PATH.exists()
|
||||
assert_no_leak(material, "unwrap failure", mod.safe_exception_payload(failure))
|
||||
|
||||
|
||||
def test_env_passphrase_still_works_reports_deprecation_and_is_persisted(env, monkeypatch, capsys):
|
||||
mod, provider, material, state_dir, _ = env
|
||||
monkeypatch.setenv("NAPSEER_VAULT_PASSPHRASE", material.passphrase)
|
||||
assert mod.active_memory_secret(PROJECT)["secret_bytes"] == material.secret(1)
|
||||
status = mod.gateway_status()
|
||||
assert status["memory_secret"] == "provisioned"
|
||||
assert [item["code"] for item in status["deprecations"]] == ["vault_passphrase_env_deprecated"]
|
||||
assert status["deprecations"][0]["variables"] == ["NAPSEER_VAULT_PASSPHRASE"]
|
||||
assert status["deprecations"][0]["next_action"] == SETUP_COMMAND
|
||||
mod.local_anchor_integrity = lambda: {}
|
||||
doctor = mod.doctor({})
|
||||
assert any("deprecated" in warning for warning in doctor["warnings"])
|
||||
log = capsys.readouterr().err
|
||||
assert '"event":"vault_passphrase_env_deprecated"' in log
|
||||
|
||||
# Later processes no longer need the variable.
|
||||
monkeypatch.delenv("NAPSEER_VAULT_PASSPHRASE")
|
||||
fresh_process_state(mod)
|
||||
assert mod.active_memory_secret(PROJECT)["secret_bytes"] == material.secret(1)
|
||||
assert "deprecations" not in mod.gateway_status()
|
||||
for label, blob in (("status", status), ("doctor", doctor), ("log", log)):
|
||||
assert_no_leak(material, label, blob)
|
||||
assert_state_dir_clean(material, state_dir)
|
||||
|
||||
|
||||
def test_mcp_mode_never_prompts_even_on_a_tty(env, monkeypatch):
|
||||
mod, _provider, _material, _state_dir, _ = env
|
||||
|
||||
class Tty:
|
||||
def isatty(self):
|
||||
return True
|
||||
|
||||
def forbidden_prompt(*_args, **_kwargs):
|
||||
raise AssertionError("MCP must never prompt")
|
||||
|
||||
monkeypatch.setattr(mod.sys, "stdin", Tty())
|
||||
monkeypatch.setattr(mod.getpass, "getpass", forbidden_prompt)
|
||||
mod.MCP_SERVING = True
|
||||
assert error_of(mod.active_memory_secret, PROJECT).code == "memory_secret_not_provisioned"
|
||||
assert "NAPSEER_VAULT_PASSPHRASE" in str(error_of(mod.project_bundle_passphrase))
|
||||
assert error_of(mod.memory_secret_setup, {}).code == "passphrase_required"
|
||||
assert error_of(mod.decrypt_node_for_return, _provider.nodes["secretnote"], PROJECT).code == \
|
||||
"memory_secret_not_provisioned"
|
||||
|
||||
|
||||
# -- fresh processes ----------------------------------------------------------
|
||||
|
||||
def child_env(user_data, base_url, **extra):
|
||||
env = {key: value for key, value in os.environ.items()
|
||||
if not key.startswith("NAPSEER_") or key == "NAPSEER_CRYPTO_BACKEND"}
|
||||
env.update({
|
||||
"NAPSEER_USER_DATA_DIR": str(user_data),
|
||||
"NAPSEER_BASE_URL": base_url,
|
||||
"NAPSEER_TOKEN": "disposable-test-token",
|
||||
"NAPSEER_PROJECT_ID": PROJECT,
|
||||
"NAPSEER_ACCOUNT_ID": ACCOUNT,
|
||||
"NAPSEER_TELEMETRY": "0",
|
||||
"NAPSEER_TOOL_PROFILES": "all",
|
||||
"PYTHONDONTWRITEBYTECODE": "1",
|
||||
})
|
||||
env.update(extra)
|
||||
return env
|
||||
|
||||
|
||||
def no_echo_pty():
|
||||
master, slave = pty.openpty()
|
||||
attrs = termios.tcgetattr(slave)
|
||||
attrs[3] &= ~termios.ECHO
|
||||
termios.tcsetattr(slave, termios.TCSANOW, attrs)
|
||||
return master, slave
|
||||
|
||||
|
||||
def run_interactive_setup(env, cwd, passphrase):
|
||||
"""`gateway vault setup` on a terminal: the passphrase is typed at the prompt."""
|
||||
master, slave = no_echo_pty()
|
||||
process = subprocess.Popen(
|
||||
[sys.executable, str(SCRIPT), "gateway", "vault", "setup"],
|
||||
stdin=slave, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=cwd, env=env, start_new_session=True,
|
||||
)
|
||||
os.close(slave)
|
||||
stderr = b""
|
||||
deadline = time.monotonic() + 30
|
||||
try:
|
||||
while b"passphrase" not in stderr.lower():
|
||||
remaining = deadline - time.monotonic()
|
||||
ready, _, _ = select.select([process.stderr], [], [], max(0.0, remaining))
|
||||
if not ready:
|
||||
raise AssertionError("setup did not prompt for the passphrase")
|
||||
chunk = os.read(process.stderr.fileno(), 4096)
|
||||
if not chunk:
|
||||
break
|
||||
stderr += chunk
|
||||
os.write(master, passphrase.encode("utf-8") + b"\n")
|
||||
stdout, rest = process.communicate(timeout=60)
|
||||
finally:
|
||||
if process.poll() is None:
|
||||
process.kill()
|
||||
process.wait()
|
||||
os.close(master)
|
||||
return process.returncode, stdout, stderr + rest
|
||||
|
||||
|
||||
class McpProcess:
|
||||
def __init__(self, env, cwd, tty_stdin=False):
|
||||
self.master = None
|
||||
if tty_stdin:
|
||||
self.master, slave = no_echo_pty()
|
||||
stdin = slave
|
||||
else:
|
||||
stdin = subprocess.PIPE
|
||||
self.process = subprocess.Popen(
|
||||
[sys.executable, "-u", str(SCRIPT)], stdin=stdin, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
|
||||
cwd=cwd, env=env, start_new_session=True,
|
||||
)
|
||||
if tty_stdin:
|
||||
os.close(slave)
|
||||
self.lines = queue.Queue()
|
||||
self.stdout = []
|
||||
self.stderr = b""
|
||||
threading.Thread(target=self._read_stdout, daemon=True).start()
|
||||
|
||||
def _read_stdout(self):
|
||||
for line in self.process.stdout:
|
||||
self.stdout.append(line)
|
||||
self.lines.put(line)
|
||||
|
||||
def send(self, message):
|
||||
data = (json.dumps(message) + "\n").encode("utf-8")
|
||||
if self.master is not None:
|
||||
os.write(self.master, data)
|
||||
else:
|
||||
self.process.stdin.write(data)
|
||||
self.process.stdin.flush()
|
||||
|
||||
def request(self, request_id, method, params=None, timeout=60):
|
||||
self.send({"jsonrpc": "2.0", "id": request_id, "method": method, "params": params or {}})
|
||||
deadline = time.monotonic() + timeout
|
||||
while True:
|
||||
try:
|
||||
message = json.loads(self.lines.get(timeout=max(0.01, deadline - time.monotonic())))
|
||||
except queue.Empty:
|
||||
raise AssertionError(f"no MCP response to {method}") from None
|
||||
if message.get("id") == request_id:
|
||||
return message
|
||||
|
||||
def call(self, request_id, name, arguments=None):
|
||||
return self.request(request_id, "tools/call", {"name": name, "arguments": arguments or {}})["result"]
|
||||
|
||||
def start(self):
|
||||
self.request(1, "initialize", {"protocolVersion": "2025-11-25", "capabilities": {},
|
||||
"clientInfo": {"name": "memory-vault-test", "version": "1"}})
|
||||
self.send({"jsonrpc": "2.0", "method": "notifications/initialized", "params": {}})
|
||||
|
||||
def close(self):
|
||||
if self.master is not None:
|
||||
os.close(self.master)
|
||||
elif self.process.stdin:
|
||||
self.process.stdin.close()
|
||||
try:
|
||||
self.process.wait(timeout=10)
|
||||
except subprocess.TimeoutExpired:
|
||||
self.process.kill()
|
||||
self.process.wait()
|
||||
self.stderr = self.process.stderr.read()
|
||||
return b"".join(self.stdout), self.stderr
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def processes(env, tmp_path):
|
||||
mod, provider, material, _state_dir, base_url = env
|
||||
user_data = tmp_path / "user-data"
|
||||
workdir = tmp_path / "work"
|
||||
workdir.mkdir()
|
||||
return mod, provider, material, user_data, workdir, child_env(user_data, base_url)
|
||||
|
||||
|
||||
def test_interactive_setup_then_fresh_mcp_process_searches_e2e_automatically(processes):
|
||||
mod, provider, material, user_data, workdir, environment = processes
|
||||
code, stdout, stderr = run_interactive_setup(environment, workdir, material.passphrase)
|
||||
assert code == 0, stderr.decode(errors="replace")[-2000:]
|
||||
result = json.loads(stdout)
|
||||
assert result["status"] == "provisioned" and result["passphrase_source"] == "prompt"
|
||||
assert result["passphrase_stored"] is False
|
||||
assert_no_leak(material, "setup stdout", stdout)
|
||||
assert_no_leak(material, "setup stderr", stderr)
|
||||
|
||||
# A new CLI process builds the encrypted E2E index without any passphrase.
|
||||
reindexed = subprocess.run([sys.executable, str(SCRIPT), "reindex"], cwd=workdir, env=environment,
|
||||
capture_output=True, timeout=120, stdin=subprocess.DEVNULL)
|
||||
assert reindexed.returncode == 0, reindexed.stderr.decode(errors="replace")[-2000:]
|
||||
assert json.loads(reindexed.stdout)["e2e_index"]["status"] == "applied"
|
||||
assert_no_leak(material, "reindex output", reindexed.stdout + reindexed.stderr)
|
||||
|
||||
# A new MCP worker: no passphrase anywhere in its environment.
|
||||
client = McpProcess(environment, workdir)
|
||||
try:
|
||||
client.start()
|
||||
whoami = client.call(2, "nap_whoami")["structuredContent"]
|
||||
assert whoami["memory_secret"] == "provisioned" and "deprecations" not in whoami
|
||||
found = client.call(3, "nap_discover", {"q": BODY_TERM, "mode": "local"})
|
||||
assert found["isError"] is False, found["structuredContent"]
|
||||
found = found["structuredContent"]
|
||||
assert [item["node_id"] for item in found["items"]] == ["secretnote"]
|
||||
assert found["items"][0]["search_source"] == "local_e2e" and not found.get("warnings")
|
||||
node = client.call(4, "nap_node_get", {"node_id": "secretnote"})["structuredContent"]
|
||||
assert BODY in json.dumps(node)
|
||||
finally:
|
||||
out, err = client.close()
|
||||
assert_no_leak(material, "mcp stdout", out)
|
||||
assert_no_leak(material, "mcp stderr", err)
|
||||
assert_state_dir_clean(material, user_data, workdir)
|
||||
assert not (workdir / ".napseer").exists()
|
||||
|
||||
# Removal through the CLI revokes access for the next process.
|
||||
forgot = subprocess.run([sys.executable, str(SCRIPT), "gateway", "vault", "forget"], cwd=workdir,
|
||||
env=environment, capture_output=True, timeout=60, stdin=subprocess.DEVNULL)
|
||||
assert forgot.returncode == 0 and json.loads(forgot.stdout)["status"] == "removed"
|
||||
client = McpProcess(environment, workdir)
|
||||
try:
|
||||
client.start()
|
||||
failed = client.call(2, "nap_node_get", {"node_id": "secretnote"})
|
||||
assert failed["isError"] is True
|
||||
assert failed["structuredContent"]["error"]["code"] == "memory_secret_not_provisioned"
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_mcp_worker_on_a_tty_never_prompts(processes):
|
||||
_mod, _provider, material, user_data, workdir, environment = processes
|
||||
client = McpProcess(environment, workdir, tty_stdin=True)
|
||||
try:
|
||||
client.start()
|
||||
failed = client.call(2, "nap_node_get", {"node_id": "secretnote"})
|
||||
assert failed["isError"] is True
|
||||
error = failed["structuredContent"]["error"]
|
||||
assert error["code"] == "memory_secret_not_provisioned"
|
||||
assert error["details"]["next_action"] == SETUP_COMMAND
|
||||
# stdin was not consumed by a prompt: the worker keeps answering.
|
||||
assert client.call(3, "nap_whoami")["structuredContent"]["memory_secret"] == "not_provisioned"
|
||||
finally:
|
||||
_out, err = client.close()
|
||||
assert b"passphrase:" not in err.lower()
|
||||
assert_state_dir_clean(material, user_data)
|
||||
@@ -36,6 +36,11 @@ def run():
|
||||
|
||||
assert mod.handle_gateway(["vault", "rotate-secret", "--kind", "memory"]) == 0
|
||||
assert calls[-1] == ("gateway", ["vault", "rotate-secret", "--kind", "memory"])
|
||||
# Memory secret provisioning/removal route to the wrapper, which owns the prompt.
|
||||
assert mod.handle_gateway(["vault", "setup"]) == 0
|
||||
assert calls[-1] == ("gateway", ["vault", "setup"])
|
||||
assert mod.handle_gateway(["vault", "forget"]) == 0
|
||||
assert calls[-1] == ("gateway", ["vault", "forget"])
|
||||
|
||||
try:
|
||||
mod.handle_gateway(["process", "--all"])
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
"""Smoke-test existing-project OAuth attachment for folder bootstrap."""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
@@ -17,6 +18,10 @@ def load_module():
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
module.load_project_locator = lambda: None
|
||||
module.require_project_locator_service = lambda _locator: None
|
||||
module.write_project_locator = lambda *_args, **_kwargs: {"declared": False}
|
||||
module.project_locator_status = lambda _locator=None: {"declared": False}
|
||||
return module
|
||||
|
||||
|
||||
@@ -55,12 +60,12 @@ def run():
|
||||
|
||||
result = mod.operator_project_attach({"open_browser": False})
|
||||
|
||||
saves.insert(0, json.loads(pathlib.Path(result["auth_path"]).read_text()))
|
||||
assert saves[0]["token"] == "oauth-token"
|
||||
assert saves[0]["refresh_token"] == "oauth-refresh-token"
|
||||
assert saves[0]["project_id"] == PROJECT_ID
|
||||
assert saves[0]["account_mode"] == "operator_project"
|
||||
assert replacements[0] == mod.WORKER_BINDING_AUTH_KEYS
|
||||
assert "project_id" not in replacements[0]
|
||||
assert mod.WORKER_BINDING_AUTH_KEYS.isdisjoint(saves[0])
|
||||
assert "napseer.projects.read" in saves[0]["oauth_scope"]
|
||||
assert saves[1]["project_slug"] == "existing-project"
|
||||
assert result["project_id"] == PROJECT_ID
|
||||
@@ -77,6 +82,7 @@ def run():
|
||||
"scope": args["scope"],
|
||||
}
|
||||
result = mod.operator_account_login({"open_browser": False})
|
||||
saves.insert(0, json.loads(pathlib.Path(result["auth_path"]).read_text()))
|
||||
assert saves[0]["refresh_token"] == "account-refresh-token"
|
||||
assert saves[0]["account_mode"] == "operator_account"
|
||||
assert "project_id" not in saves[0]
|
||||
|
||||
@@ -59,6 +59,7 @@ def test_account_login_requests_create_and_runtime_scopes():
|
||||
|
||||
result = mod.operator_account_login({"open_browser": False})
|
||||
|
||||
saves.append(json.loads(pathlib.Path(result["auth_path"]).read_text()))
|
||||
requested = set(captured[0]["scope"].split())
|
||||
assert "napseer.projects.write" in requested
|
||||
assert set(mod.LOCAL_PROJECT_OAUTH_SCOPE.split()).issubset(requested)
|
||||
@@ -84,23 +85,27 @@ def test_legacy_account_scope_fails_before_http_with_recovery():
|
||||
assert not requests
|
||||
|
||||
|
||||
def test_successful_create_transitions_local_state_to_project_mode():
|
||||
def test_successful_create_preserves_general_account_mode():
|
||||
mod = load_module()
|
||||
configure_account_state(mod, mod.OPERATOR_ACCOUNT_OAUTH_SCOPE)
|
||||
saves = []
|
||||
locators = []
|
||||
mod.request_json = lambda method, path, payload=None: {
|
||||
"id": PROJECT_ID,
|
||||
"slug": payload["slug"],
|
||||
"name": payload["name"],
|
||||
}
|
||||
mod.save_auth = lambda updates: saves.append(dict(updates))
|
||||
mod.write_project_locator = lambda project: locators.append(dict(project))
|
||||
|
||||
project, status, _message = mod.create_project_with_state(project_args())
|
||||
|
||||
assert status == "created"
|
||||
assert project["id"] == PROJECT_ID
|
||||
assert saves[0]["project_id"] == PROJECT_ID
|
||||
assert saves[0]["account_mode"] == "operator_project"
|
||||
assert "account_mode" not in saves[0]
|
||||
assert mod.AUTH["account_mode"] == "operator_account"
|
||||
assert locators == [project]
|
||||
|
||||
|
||||
def test_server_scope_denial_is_actionable():
|
||||
@@ -159,7 +164,7 @@ def test_anonymous_limits_are_actionable_without_exposing_response_details():
|
||||
if __name__ == "__main__":
|
||||
test_account_login_requests_create_and_runtime_scopes()
|
||||
test_legacy_account_scope_fails_before_http_with_recovery()
|
||||
test_successful_create_transitions_local_state_to_project_mode()
|
||||
test_successful_create_preserves_general_account_mode()
|
||||
test_server_scope_denial_is_actionable()
|
||||
test_anonymous_limits_are_actionable_without_exposing_response_details()
|
||||
print("ok: project create OAuth lifecycle passed")
|
||||
|
||||
@@ -16,6 +16,10 @@ def load_module():
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
module.load_project_locator = lambda: None
|
||||
module.require_project_locator_service = lambda _locator: None
|
||||
module.write_project_locator = lambda *_args, **_kwargs: {"declared": False}
|
||||
module.project_locator_status = lambda _locator=None: {"declared": False}
|
||||
return module
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Contract tests for the committed, non-secret Napseer project locator."""
|
||||
|
||||
import importlib.util
|
||||
import json
|
||||
import pathlib
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
|
||||
PROJECT_ID = "33333333-3333-3333-3333-333333333333"
|
||||
OTHER_PROJECT_ID = "44444444-4444-4444-4444-444444444444"
|
||||
|
||||
|
||||
def load_module():
|
||||
script_path = pathlib.Path(__file__).resolve().parents[1] / "resources" / "scripts" / "napseer_mcp_server.py"
|
||||
sys.path.insert(0, str(script_path.parent))
|
||||
spec = importlib.util.spec_from_file_location("napseer_mcp_server_project_locator_test", script_path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
assert spec and spec.loader
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def configure_state(mod, root):
|
||||
state_dir = pathlib.Path(root) / ".napseer"
|
||||
mod.AUTH_DIR = state_dir
|
||||
mod.PROJECT_DIR = state_dir
|
||||
mod.AUTH_PATH = state_dir / "auth.json"
|
||||
mod.BASE_URL = "https://api.example.test"
|
||||
mod.TOKEN = None
|
||||
mod.DEFAULT_PROJECT_ID = None
|
||||
mod.AUTH = {}
|
||||
return state_dir
|
||||
|
||||
|
||||
def test_locator_is_deterministic_committable_and_secret_free():
|
||||
mod = load_module()
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = pathlib.Path(directory)
|
||||
subprocess.run(["git", "init", "-q"], cwd=root, check=True)
|
||||
state_dir = configure_state(mod, root)
|
||||
mod.DEFAULT_PROJECT_ID = PROJECT_ID
|
||||
|
||||
result = mod.write_project_locator({"id": PROJECT_ID, "slug": "portable-project"})
|
||||
locator = json.loads((state_dir / "project.json").read_text(encoding="utf-8"))
|
||||
|
||||
assert locator == {
|
||||
"schema": "napseer.project.v1",
|
||||
"base_url": "https://api.example.test",
|
||||
"project_id": PROJECT_ID,
|
||||
"project_slug": "portable-project",
|
||||
}
|
||||
assert result["declared"] is True
|
||||
assert result["matches_local_auth"] is True
|
||||
forbidden = {
|
||||
"token", "access_token", "refresh_token", "account_id", "agent_id",
|
||||
"worker_id", "claim_token", "claim_url", "passphrase", "encryption_state",
|
||||
}
|
||||
assert forbidden.isdisjoint(locator)
|
||||
|
||||
(state_dir / "auth.json").write_text('{"token":"private"}\n', encoding="utf-8")
|
||||
(state_dir / "gateway-auth.json").write_text('{"token":"private"}\n', encoding="utf-8")
|
||||
assert subprocess.run(
|
||||
["git", "check-ignore", "--quiet", ".napseer/auth.json"], cwd=root
|
||||
).returncode == 0
|
||||
assert subprocess.run(
|
||||
["git", "check-ignore", "--quiet", ".napseer/gateway-auth.json"], cwd=root
|
||||
).returncode == 0
|
||||
assert subprocess.run(
|
||||
["git", "check-ignore", "--quiet", ".napseer/project.json"], cwd=root
|
||||
).returncode == 1
|
||||
assert subprocess.run(
|
||||
["git", "check-ignore", "--quiet", ".napseer/.gitignore"], cwd=root
|
||||
).returncode == 1
|
||||
|
||||
|
||||
def test_locator_rejects_extra_secret_fields_and_mismatched_auth():
|
||||
mod = load_module()
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
state_dir = configure_state(mod, directory)
|
||||
state_dir.mkdir(parents=True)
|
||||
(state_dir / "project.json").write_text(
|
||||
json.dumps({
|
||||
"schema": "napseer.project.v1",
|
||||
"base_url": "https://api.example.test",
|
||||
"project_id": PROJECT_ID,
|
||||
"project_slug": "portable-project",
|
||||
"token": "must-not-be-accepted",
|
||||
}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
try:
|
||||
mod.load_project_locator()
|
||||
except mod.SafeToolError as exc:
|
||||
assert exc.code == "project_locator_invalid"
|
||||
else:
|
||||
raise AssertionError("locator with an extra secret field must be rejected")
|
||||
|
||||
(state_dir / "project.json").unlink()
|
||||
mod.write_project_locator({"id": PROJECT_ID, "slug": "portable-project"})
|
||||
mod.DEFAULT_PROJECT_ID = OTHER_PROJECT_ID
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
try:
|
||||
mod.resolve_project_id({})
|
||||
except mod.SafeToolError as exc:
|
||||
assert exc.code == "project_locator_mismatch"
|
||||
else:
|
||||
raise AssertionError("local auth/locator mismatch must fail closed")
|
||||
|
||||
|
||||
def test_fresh_clone_requires_attach_and_never_bootstraps_anonymous_project():
|
||||
mod = load_module()
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
configure_state(mod, directory)
|
||||
mod.write_project_locator({"id": PROJECT_ID, "slug": "portable-project"})
|
||||
enrollment_calls = []
|
||||
bootstrap_calls = []
|
||||
mod.ensure_enrolled = lambda args: enrollment_calls.append(dict(args))
|
||||
mod.bootstrap_project = lambda args: bootstrap_calls.append(dict(args))
|
||||
|
||||
result = mod.cli_project_init([])
|
||||
assert result["status"] == "project_access_required"
|
||||
assert result["project_id"] == PROJECT_ID
|
||||
assert result["next"]["attach_project"] == "nap project attach"
|
||||
assert enrollment_calls == []
|
||||
assert bootstrap_calls == []
|
||||
|
||||
mod.refresh_public_auth_state = lambda: None
|
||||
try:
|
||||
mod.resolve_project_id({})
|
||||
except mod.SafeToolError as exc:
|
||||
assert exc.code == "project_access_required"
|
||||
assert "will not create a duplicate anonymous project" in exc.safe_message
|
||||
else:
|
||||
raise AssertionError("first MCP project resolution must require attach")
|
||||
|
||||
|
||||
def test_attach_forwards_and_verifies_repository_project_hint():
|
||||
mod = load_module()
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
configure_state(mod, directory)
|
||||
mod.write_project_locator({"id": PROJECT_ID, "slug": "portable-project"})
|
||||
oauth_calls = []
|
||||
saves = []
|
||||
|
||||
def oauth(args):
|
||||
oauth_calls.append(dict(args))
|
||||
return {
|
||||
"api_base_url": "https://api.example.test",
|
||||
"access_token": "local-test-token",
|
||||
"refresh_token": "local-test-refresh",
|
||||
"expires_in": 3600,
|
||||
"scope": args["scope"],
|
||||
"project_id": PROJECT_ID,
|
||||
}
|
||||
|
||||
mod.oauth_loopback_authorize = oauth
|
||||
mod.replace_public_auth_state = lambda updates, clear_keys=(): saves.append(dict(updates))
|
||||
mod.save_auth_file_credentials = lambda updates: saves.append(dict(updates))
|
||||
mod.request_json = lambda method, path: {
|
||||
"id": PROJECT_ID,
|
||||
"slug": "portable-project",
|
||||
"name": "Portable Project",
|
||||
"encryption_state": "standard",
|
||||
}
|
||||
|
||||
result = mod.operator_project_attach({"open_browser": False})
|
||||
assert oauth_calls[0]["project_id_hint"] == PROJECT_ID
|
||||
assert result["project_id"] == PROJECT_ID
|
||||
assert result["project_locator"]["project_id"] == PROJECT_ID
|
||||
credentials = json.loads(pathlib.Path(result["auth_path"]).read_text())
|
||||
assert credentials["project_id"] == PROJECT_ID
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
test_locator_is_deterministic_committable_and_secret_free()
|
||||
test_locator_rejects_extra_secret_fields_and_mismatched_auth()
|
||||
test_fresh_clone_requires_attach_and_never_bootstraps_anonymous_project()
|
||||
test_attach_forwards_and_verifies_repository_project_hint()
|
||||
print("ok: committed project locator contract passed")
|
||||
@@ -23,16 +23,17 @@ SMOKES = [
|
||||
"test_project_attach_oauth.py",
|
||||
"test_project_encryption_enable_payload.py",
|
||||
"test_project_init_wizard.py",
|
||||
"test_project_locator.py",
|
||||
"test_two_passphrase_boundary.py",
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("filename", SMOKES)
|
||||
def test_standalone_smoke(filename):
|
||||
def test_standalone_smoke(filename, tmp_path):
|
||||
test_dir = pathlib.Path(__file__).resolve().parent
|
||||
result = subprocess.run(
|
||||
[sys.executable, str(test_dir / filename)],
|
||||
cwd=test_dir.parent,
|
||||
cwd=tmp_path,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=90,
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
"""Real workers and synthetic HTTP provider: user sessions across repositories."""
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from urllib.parse import parse_qs
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPT = Path(__file__).resolve().parents[1] / "resources/scripts/napseer_mcp_server.py"
|
||||
sys.path.insert(0, str(SCRIPT.parent))
|
||||
from napseer_credentials import CredentialStore, CredentialError, atomic_json, read_json
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def provider():
|
||||
state = {"refreshes": 0, "requests": 0, "failure": None, "revoked": False}
|
||||
mutex = threading.Lock()
|
||||
state["refresh_started"] = threading.Event()
|
||||
state["refresh_release"] = threading.Event()
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self, *_args):
|
||||
pass
|
||||
|
||||
def reply(self, status, value):
|
||||
data = json.dumps(value).encode()
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(data)))
|
||||
self.end_headers()
|
||||
self.wfile.write(data)
|
||||
|
||||
def do_GET(self):
|
||||
with mutex:
|
||||
state["requests"] += 1
|
||||
if self.headers.get("Authorization") == "Bearer fresh-access" and not state["revoked"]:
|
||||
self.reply(200, {"ok": True})
|
||||
else:
|
||||
self.reply(401, {"error": {"code": "unauthorized"}})
|
||||
|
||||
def do_POST(self):
|
||||
form = parse_qs(self.rfile.read(int(self.headers["Content-Length"])).decode())
|
||||
if self.path == "/v1/oauth/revoke":
|
||||
state["revoked"] = True
|
||||
return self.reply(200, {})
|
||||
assert self.path == "/v1/oauth/token", "no enrollment fallback is allowed"
|
||||
if state["failure"] == "transport":
|
||||
return self.reply(503, {"error": {"code": "unavailable"}})
|
||||
if state["failure"] == "malformed":
|
||||
return self.reply(200, {"access_token": "incomplete"})
|
||||
if form.get("refresh_token") != ["old-refresh"] or state["refreshes"]:
|
||||
return self.reply(401, {"error": {"code": "invalid_grant"}})
|
||||
if state.get("hold_refresh"):
|
||||
state["refresh_started"].set()
|
||||
assert state["refresh_release"].wait(5)
|
||||
with mutex:
|
||||
state["refreshes"] += 1
|
||||
self.reply(200, {"access_token": "fresh-access", "refresh_token": "fresh-refresh",
|
||||
"expires_in": 3600, "refresh_expires_at": "2099-01-01T00:00:00Z"})
|
||||
|
||||
server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
thread = threading.Thread(target=server.serve_forever, daemon=True)
|
||||
thread.start()
|
||||
try:
|
||||
yield f"http://127.0.0.1:{server.server_port}", state
|
||||
finally:
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join(timeout=2)
|
||||
|
||||
|
||||
def seed(root, url):
|
||||
root.mkdir(exist_ok=True)
|
||||
store = CredentialStore(root / ".napseer")
|
||||
store.login({"base_url": url, "token": "old-access", "refresh_token": "old-refresh",
|
||||
"account_mode": "operator_account", "oauth_client_id": "nap-cli"})
|
||||
return store
|
||||
|
||||
|
||||
def worker(root, expression):
|
||||
code = "import importlib.util, json, sys\n" + (
|
||||
"spec = importlib.util.spec_from_file_location('worker', sys.argv[1])\n"
|
||||
"m = importlib.util.module_from_spec(spec)\nspec.loader.exec_module(m)\n"
|
||||
"try:\n result = " + expression + "\n print(json.dumps({'ok': True, 'result': result}))\n"
|
||||
"except Exception as exc:\n print(json.dumps({'ok': False, 'code': getattr(exc, 'code', type(exc).__name__)}))\n"
|
||||
)
|
||||
env = dict(os.environ, NAPSEER_PROJECT_ROOT=str(root), NAPSEER_TELEMETRY="0")
|
||||
for key in ("NAPSEER_AUTH_FILE", "NAPSEER_BASE_URL", "NAPSEER_PROJECT_ID"):
|
||||
env.pop(key, None)
|
||||
return subprocess.Popen([sys.executable, "-c", code, str(SCRIPT)], cwd=root, env=env,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
||||
|
||||
|
||||
def outcome(process):
|
||||
out, _err = process.communicate(timeout=15)
|
||||
assert process.returncode == 0, "worker must exit without exposing credential state"
|
||||
return json.loads(out)
|
||||
|
||||
|
||||
def test_shared_login_refreshes_once_across_real_processes_and_survives_restart(tmp_path, provider):
|
||||
url, state = provider
|
||||
first, second = tmp_path / "first", tmp_path / "second"
|
||||
store = seed(first, url)
|
||||
second.mkdir()
|
||||
processes = [worker(root, "m.request_json('GET', '/v1/projects')") for root in [first, second]]
|
||||
assert all(outcome(p)["ok"] for p in processes)
|
||||
assert state["refreshes"] == 1
|
||||
assert outcome(worker(second, "m.request_json('GET', '/v1/projects')"))["ok"]
|
||||
assert state["refreshes"] == 1
|
||||
assert store.user.stat().st_mode & 0o777 == 0o600
|
||||
assert not (first / ".napseer/auth.json").exists()
|
||||
assert not (second / ".napseer/auth.json").exists()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("failure", ["transport", "malformed"])
|
||||
def test_refresh_failure_preserves_credentials_and_recovers(tmp_path, provider, failure):
|
||||
url, state = provider
|
||||
store = seed(tmp_path, url)
|
||||
before = store.user.read_bytes()
|
||||
state["failure"] = failure
|
||||
assert not outcome(worker(tmp_path, "m.request_json('GET', '/v1/projects')"))["ok"]
|
||||
assert store.user.read_bytes() == before
|
||||
state["failure"] = None
|
||||
assert outcome(worker(tmp_path, "m.request_json('GET', '/v1/projects')"))["ok"]
|
||||
|
||||
|
||||
def test_project_override_does_not_fall_back_to_user_authority(tmp_path, provider):
|
||||
url, state = provider
|
||||
store = seed(tmp_path, url)
|
||||
store.login({"base_url": url, "token": "npk_expired", "account_mode": "api_key"}, project=True)
|
||||
result = outcome(worker(tmp_path, "m.request_json('GET', '/v1/projects')"))
|
||||
assert not result["ok"]
|
||||
assert state["refreshes"] == 0
|
||||
assert read_json(store.user)["token"] == "old-access"
|
||||
store.logout(project=True)
|
||||
assert not outcome(worker(tmp_path, "m.request_json('GET', '/v1/projects')"))["ok"]
|
||||
assert state["refreshes"] == 0
|
||||
|
||||
|
||||
def test_unconfigured_agent_never_enrolls_or_creates_project(tmp_path):
|
||||
result = outcome(worker(tmp_path, "m.resolve_project_id({})"))
|
||||
assert result == {"ok": False, "code": "auth_required"}
|
||||
assert not (tmp_path / ".napseer").exists()
|
||||
|
||||
|
||||
def test_logout_revokes_and_next_worker_observes_tombstone(tmp_path, provider):
|
||||
url, state = provider
|
||||
store = seed(tmp_path, url)
|
||||
assert outcome(worker(tmp_path, "m.operator_logout({})"))["ok"]
|
||||
assert state["revoked"]
|
||||
assert read_json(store.user)["logged_out"]
|
||||
assert not outcome(worker(tmp_path, "m.request_json('GET', '/v1/projects')"))["ok"]
|
||||
assert state["requests"] == 0
|
||||
|
||||
|
||||
def test_global_credentials_keep_repository_selections_independent(tmp_path, provider):
|
||||
url, _ = provider
|
||||
first = tmp_path / "first"
|
||||
store = seed(first, url)
|
||||
other = CredentialStore(tmp_path / "second/.napseer")
|
||||
store.write({**store.read(), "project_id": "first"})
|
||||
other.write({**other.read(), "project_id": "second"})
|
||||
assert "project_id" not in read_json(store.user)
|
||||
assert store.read()["project_id"] == "first"
|
||||
assert other.read()["project_id"] == "second"
|
||||
|
||||
|
||||
def test_logout_waits_for_inflight_rotation_and_cannot_be_undone(tmp_path, provider):
|
||||
url,state=provider
|
||||
store=seed(tmp_path,url)
|
||||
state["hold_refresh"]=True
|
||||
refresh=worker(tmp_path,"m.request_json('GET','/v1/projects')")
|
||||
assert state["refresh_started"].wait(5)
|
||||
logout=worker(tmp_path,"m.operator_logout({})")
|
||||
state["refresh_release"].set()
|
||||
outcome(refresh)
|
||||
assert outcome(logout)["ok"]
|
||||
assert state["revoked"]
|
||||
assert read_json(store.user).get("logged_out")
|
||||
assert not outcome(worker(tmp_path,"m.request_json('GET','/v1/projects')"))["ok"]
|
||||
|
||||
|
||||
def test_refresh_pins_credential_source_when_override_appears(tmp_path, provider):
|
||||
url,state=provider
|
||||
store=seed(tmp_path,url)
|
||||
state["hold_refresh"]=True
|
||||
refresh=worker(tmp_path,"m.request_json('GET','/v1/projects')")
|
||||
assert state["refresh_started"].wait(5)
|
||||
override={"base_url":url,"token":"npk_project_test","account_mode":"api_key"}
|
||||
store.login(override,project=True)
|
||||
state["refresh_release"].set()
|
||||
assert not outcome(refresh)["ok"], "retry observes the explicit override"
|
||||
assert read_json(store.project)==override
|
||||
assert read_json(store.user)["token"]=="fresh-access"
|
||||
|
||||
|
||||
def test_legacy_migration_preserves_authority_and_general_selection_is_explicit(tmp_path,provider):
|
||||
url,_=provider
|
||||
store=seed(tmp_path,url)
|
||||
legacy={"base_url":url,"token":"legacy-project","account_mode":"operator_project","project_id":"test-project"}
|
||||
atomic_json(store.legacy,legacy)
|
||||
with pytest.raises(CredentialError, match="needs migration"):
|
||||
store.selected()
|
||||
store.migrate()
|
||||
assert not store.legacy.exists()
|
||||
assert store.read()["token"]=="legacy-project"
|
||||
store.use_user()
|
||||
assert store.read()["token"]=="old-access"
|
||||
assert read_json(store.project)==legacy
|
||||
|
||||
|
||||
def test_failed_login_initialization_releases_actual_loopback_socket(tmp_path,monkeypatch):
|
||||
import socket
|
||||
spec=importlib.util.spec_from_file_location("cleanup_worker",SCRIPT)
|
||||
module=importlib.util.module_from_spec(spec);spec.loader.exec_module(module)
|
||||
reserved=socket.socket();reserved.bind(("127.0.0.1",0));port=reserved.getsockname()[1];reserved.close()
|
||||
monkeypatch.setattr(module,"request_json",lambda *a,**k: (_ for _ in ()).throw(RuntimeError("synthetic unavailable")))
|
||||
with pytest.raises(RuntimeError):
|
||||
module.oauth_loopback_authorize({"scope":"test","flow":"operator_login","port":port,"open_browser":False})
|
||||
probe=socket.socket()
|
||||
try: probe.bind(("127.0.0.1",port))
|
||||
finally: probe.close()
|
||||
|
||||
|
||||
def test_native_login_uses_requested_service_pkce_callback_and_shared_store(tmp_path,monkeypatch):
|
||||
import base64
|
||||
import hashlib
|
||||
import urllib.request
|
||||
from urllib.parse import urlsplit, urlencode
|
||||
state={}
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
def log_message(self,*_args): pass
|
||||
def do_POST(self):
|
||||
raw=self.rfile.read(int(self.headers['Content-Length']))
|
||||
if self.path.endswith('/authorizations'):
|
||||
state['authorization']=json.loads(raw)
|
||||
payload={"authorization_url":"http://synthetic-browser.invalid/consent"}
|
||||
else:
|
||||
form=parse_qs(raw.decode());state['exchanged']=True
|
||||
expected=base64.urlsafe_b64encode(hashlib.sha256(form['code_verifier'][0].encode()).digest()).decode().rstrip('=')
|
||||
assert expected==state['authorization']['code_challenge']
|
||||
assert form['code']==['synthetic-one-time-code']
|
||||
payload={"access_token":"synthetic-native-access","refresh_token":"synthetic-native-refresh","expires_in":3600}
|
||||
body=json.dumps(payload).encode();self.send_response(200);self.end_headers();self.wfile.write(body)
|
||||
server=ThreadingHTTPServer(('127.0.0.1',0),Handler)
|
||||
thread=threading.Thread(target=server.serve_forever,daemon=True);thread.start()
|
||||
spec=importlib.util.spec_from_file_location('native_login_worker',SCRIPT)
|
||||
module=importlib.util.module_from_spec(spec);spec.loader.exec_module(module)
|
||||
monkeypatch.setattr(module,'AUTH_DIR',tmp_path/'.napseer')
|
||||
monkeypatch.setattr(module,'AUTH_PATH',tmp_path/'.napseer/auth.json')
|
||||
monkeypatch.setattr(module,'BASE_URL','http://127.0.0.1:1')
|
||||
store = CredentialStore(tmp_path/'.napseer')
|
||||
store.user.parent.mkdir(parents=True,exist_ok=True)
|
||||
store.user.write_text('{broken synthetic state')
|
||||
def consent(_url):
|
||||
grant=state['authorization']
|
||||
callback=grant['redirect_uri']+'?'+urlencode({'code':'synthetic-one-time-code','state':grant['state']})
|
||||
with urllib.request.urlopen(callback,timeout=3) as response: assert response.status==200
|
||||
return True
|
||||
monkeypatch.setattr(module.webbrowser,'open',consent)
|
||||
try:
|
||||
result=module.operator_account_login({'base_url':f'http://127.0.0.1:{server.server_port}','timeout_seconds':5})
|
||||
assert result['status']=='authenticated';assert state['exchanged']
|
||||
store=CredentialStore(tmp_path/'.napseer')
|
||||
assert store.user.exists();assert not store.legacy.exists()
|
||||
assert read_json(store.user)['base_url']==f'http://127.0.0.1:{server.server_port}'
|
||||
finally:server.shutdown();server.server_close();thread.join(timeout=2)
|
||||
|
||||
|
||||
def test_corrupt_credentials_allow_help_but_never_authenticated_fallback(tmp_path):
|
||||
store=CredentialStore(tmp_path/'.napseer')
|
||||
store.user.parent.mkdir(parents=True,exist_ok=True)
|
||||
store.user.write_text('{broken synthetic state')
|
||||
result=outcome(worker(tmp_path,"m.request_json('GET','/v1/projects')"))
|
||||
assert result['code']=='CredentialError'
|
||||
assert store.user.read_text()=='{broken synthetic state'
|
||||
env=dict(os.environ,NAPSEER_PROJECT_ROOT=str(tmp_path))
|
||||
help_result=subprocess.run([sys.executable,str(SCRIPT),'auth','--help'],cwd=tmp_path,env=env,capture_output=True,text=True,timeout=10)
|
||||
assert help_result.returncode==0
|
||||
assert 'nap auth login' in help_result.stdout
|
||||
@@ -0,0 +1,212 @@
|
||||
"""Real Git worktrees and worker processes exercise private-state ownership."""
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
SCRIPTS = Path(__file__).resolve().parents[1] / 'resources/scripts'
|
||||
sys.path.insert(0, str(SCRIPTS))
|
||||
from napseer_credentials import CredentialStore, atomic_json, read_json
|
||||
from test_user_credentials import provider, worker, outcome
|
||||
|
||||
# Released CLI 0.4.0 source revision (pinned by the Napseer deploy workflow).
|
||||
RELEASED_0_4_0 = 'c427e2432ac85038a4f00b2b888bbdb7c851ada1'
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def worktrees(tmp_path):
|
||||
main, other = tmp_path/'main', tmp_path/'other'
|
||||
main.mkdir()
|
||||
def git(*args):
|
||||
subprocess.run(['git', '-C', str(main), *args], check=True, capture_output=True)
|
||||
git('init', '-q')
|
||||
atomic_json(main/'.napseer/project.json', {'schema':'napseer.project.v1',
|
||||
'base_url':'https://api.example.test', 'project_id':'33333333-3333-3333-3333-333333333333',
|
||||
'project_slug':'shared'})
|
||||
git('add', '.napseer/project.json')
|
||||
git('-c', 'user.name=Test', '-c', 'user.email=test@example.test', 'commit', '-qm', 'locator')
|
||||
git('worktree', 'add', '-qb', 'other', str(other))
|
||||
return main, other
|
||||
|
||||
|
||||
def test_worker_keeps_private_paths_outside_checkout_and_stable_from_subdirectory(worktrees):
|
||||
main, other = worktrees
|
||||
sub = main/'nested'; sub.mkdir()
|
||||
expression = "{'state':str(m.local_state_dir()),'key':str(m.DEFAULT_KEY_PATH),'locator':str(m.project_locator_path())}"
|
||||
a,b,c = [outcome(worker(root, expression))['result'] for root in (main,other,sub)]
|
||||
assert not Path(a['state']).is_relative_to(main)
|
||||
assert not Path(b['state']).is_relative_to(other)
|
||||
assert a['state'] == c['state']
|
||||
assert a['state'] != b['state']
|
||||
assert a['key'] == b['key']
|
||||
assert a['locator'] == str(main/'.napseer/project.json')
|
||||
assert c['locator'] == a['locator']
|
||||
|
||||
|
||||
def test_project_login_is_shared_across_worktrees_and_refreshes_once(worktrees, provider):
|
||||
main, other = worktrees
|
||||
url, state = provider
|
||||
first, second = [CredentialStore(root/'.napseer') for root in (main,other)]
|
||||
first.login({'base_url':url,'token':'old-access','refresh_token':'old-refresh',
|
||||
'account_mode':'operator_project','oauth_client_id':'nap-cli'}, project=True)
|
||||
assert first.project == second.project
|
||||
assert first.selection != second.selection
|
||||
assert not first.selection.is_relative_to(main)
|
||||
requests=[worker(root, "m.request_json('GET','/v1/projects')") for root in (main,other)]
|
||||
assert all(outcome(p)['ok'] for p in requests)
|
||||
assert state['refreshes'] == 1
|
||||
assert outcome(worker(other, "m.request_json('GET','/v1/projects')"))['ok']
|
||||
assert state['refreshes'] == 1
|
||||
|
||||
|
||||
def test_migration_preserves_keys_and_locator_and_refuses_conflicting_destination(worktrees):
|
||||
from napseer_credentials import WorkspacePaths, CredentialError
|
||||
main, _ = worktrees
|
||||
paths = WorkspacePaths(main)
|
||||
old = main/'.napseer'
|
||||
locator=(old/'project.json').read_bytes()
|
||||
(old/'vault.local-key').write_bytes(b'synthetic-vault-key\n')
|
||||
(old/'vault.json').write_bytes(b'{"synthetic":"ciphertext"}\n')
|
||||
(old/'id_ed25519').write_bytes(b'synthetic-signing-key\n')
|
||||
atomic_json(old/'auth.json', {'token':'synthetic-token','ssh_key_path':str(old/'id_ed25519')})
|
||||
paths.state.mkdir(parents=True)
|
||||
(paths.state/'vault.local-key').write_bytes(b'conflicting-key')
|
||||
with pytest.raises(CredentialError): paths.migrate()
|
||||
assert (old/'vault.local-key').read_bytes() == b'synthetic-vault-key\n'
|
||||
assert (old/'auth.json').exists()
|
||||
(paths.state/'vault.local-key').unlink()
|
||||
paths.migrate()
|
||||
assert (paths.state/'vault.local-key').read_bytes() == b'synthetic-vault-key\n'
|
||||
assert paths.signing_key.read_bytes() == b'synthetic-signing-key\n'
|
||||
assert read_json(CredentialStore(old).project)['ssh_key_path'] == str(paths.signing_key)
|
||||
assert (old/'project.json').read_bytes() == locator
|
||||
assert set(p.name for p in old.iterdir()) == {'project.json'}
|
||||
paths.migrate() # retry after a completed migration is harmless
|
||||
assert paths.signing_key.stat().st_mode & 0o777 == 0o600
|
||||
|
||||
|
||||
def test_other_repositories_do_not_share_project_override(tmp_path):
|
||||
first=CredentialStore(tmp_path/'one/.napseer')
|
||||
second=CredentialStore(tmp_path/'two/.napseer')
|
||||
first.login({'token':'synthetic-project-token'}, project=True)
|
||||
assert first.project != second.project
|
||||
assert not second.read().get('token')
|
||||
|
||||
|
||||
def test_linked_worktree_move_keeps_external_state_identity(worktrees):
|
||||
from napseer_credentials import WorkspacePaths
|
||||
main, other = worktrees
|
||||
before = WorkspacePaths(other)
|
||||
moved = other.with_name('moved')
|
||||
subprocess.run(['git','-C',str(main),'worktree','move',str(other),str(moved)], check=True, capture_output=True)
|
||||
after = WorkspacePaths(moved)
|
||||
assert before.state == after.state
|
||||
assert before.project_credentials == after.project_credentials
|
||||
|
||||
|
||||
def test_concurrent_worktree_signing_key_creation_is_serialized(worktrees):
|
||||
main, other = worktrees
|
||||
processes = [worker(root, 'str(m.ensure_local_files())') for root in (main,other)]
|
||||
results = [outcome(p) for p in processes]
|
||||
assert all(r['ok'] for r in results)
|
||||
assert results[0]['result'] == results[1]['result']
|
||||
key = Path(results[0]['result'])
|
||||
assert key.is_file() and Path(str(key)+'.pub').is_file()
|
||||
assert key.stat().st_mode & 0o777 == 0o600
|
||||
assert all(not (root/'.napseer/id_ed25519').exists() for root in (main,other))
|
||||
|
||||
|
||||
def test_interrupted_migration_preserves_originals_and_retries(worktrees, monkeypatch):
|
||||
from napseer_credentials import WorkspacePaths
|
||||
main, _ = worktrees
|
||||
paths=WorkspacePaths(main)
|
||||
for name in ('vault.local-key','vault.json'):(paths.legacy/name).write_bytes(b'synthetic-'+name.encode())
|
||||
original_link=os.link
|
||||
calls=[]
|
||||
def interrupted(source,destination):
|
||||
calls.append(destination)
|
||||
if len(calls)==2:raise OSError('synthetic interrupted publication')
|
||||
return original_link(source,destination)
|
||||
monkeypatch.setattr(os,'link',interrupted)
|
||||
with pytest.raises(OSError):paths.migrate()
|
||||
assert all((paths.legacy/name).exists() for name in ('vault.local-key','vault.json'))
|
||||
monkeypatch.setattr(os,'link',original_link)
|
||||
paths.migrate()
|
||||
assert all((paths.state/name).read_bytes()==b'synthetic-'+name.encode() for name in ('vault.local-key','vault.json'))
|
||||
|
||||
|
||||
def test_cli_migration_reports_running_gateway_and_recovers(worktrees):
|
||||
main, _ = worktrees
|
||||
atomic_json(main/'.napseer/gateway.json', {'pid':os.getpid()})
|
||||
key=main/'.napseer/vault.local-key';key.write_text('synthetic-key')
|
||||
env=dict(os.environ,NAPSEER_PROJECT_ROOT=str(main),NAPSEER_TELEMETRY='0')
|
||||
command=[sys.executable,str(SCRIPTS/'napseer_mcp_server.py'),'auth','migrate']
|
||||
result=subprocess.run(command,cwd=main,env=env,capture_output=True,text=True,timeout=15)
|
||||
assert result.returncode!=0 and 'Stop the running' in result.stderr
|
||||
assert 'synthetic-key' not in result.stderr and key.exists()
|
||||
atomic_json(main/'.napseer/gateway.json', {'pid':0})
|
||||
result=subprocess.run(command,cwd=main,env=env,capture_output=True,text=True,timeout=15)
|
||||
assert result.returncode==0
|
||||
report=json.loads(result.stdout)
|
||||
assert report['status']=='migrated'
|
||||
assert (Path(report['state_dir'])/'vault.local-key').read_text()=='synthetic-key'
|
||||
assert not key.exists()
|
||||
|
||||
|
||||
def test_private_state_cannot_be_redirected_into_checkout(worktrees, monkeypatch):
|
||||
from napseer_credentials import WorkspacePaths, CredentialError
|
||||
main,_=worktrees
|
||||
monkeypatch.setenv('NAPSEER_USER_DATA_DIR',str(main/'private-state'))
|
||||
with pytest.raises(CredentialError,match='outside'):WorkspacePaths(main)
|
||||
|
||||
|
||||
|
||||
def test_root_tools_file_resolves_like_locator_and_is_invisible_to_migration(worktrees, monkeypatch):
|
||||
from napseer_credentials import WorkspacePaths
|
||||
main, _ = worktrees
|
||||
monkeypatch.delenv('NAPSEER_TOOL_PROFILES', raising=False)
|
||||
tools = main/'.napseer-tools.json'
|
||||
committed = b'{"schema": "napseer.tools.v1", "tool_profiles": ["workflow"]}\n'
|
||||
tools.write_bytes(committed)
|
||||
sub = main/'nested'; sub.mkdir()
|
||||
# A worker started below the project root finds the file at the root that
|
||||
# owns .napseer/project.json, the same way it finds the locator.
|
||||
expression = ("{'path':str(m.project_tools_path()),'profiles':sorted(m.tool_profile_configuration()[0]),"
|
||||
"'source':m.tool_profile_configuration()[1],"
|
||||
"'pending':sorted(p.name for p in m.WORKSPACE.migration_sources())}")
|
||||
result = outcome(worker(sub, expression))['result']
|
||||
assert result == {'path': str(tools), 'profiles': ['workflow'], 'source': 'tools_file', 'pending': []}
|
||||
assert set(p.name for p in (main/'.napseer').iterdir()) <= {'project.json', '.gitignore'}
|
||||
paths = WorkspacePaths(main)
|
||||
assert paths.migration_sources() == []
|
||||
paths.require_migrated()
|
||||
paths.migrate()
|
||||
assert tools.read_bytes() == committed
|
||||
|
||||
|
||||
def test_root_tools_file_is_invisible_to_released_0_4_0_migration_check(worktrees, tmp_path):
|
||||
"""CLI 0.4.0 treats any extra .napseer/ entry as private state to migrate."""
|
||||
import importlib.util
|
||||
released = subprocess.run(
|
||||
['git', '-C', str(SCRIPTS.parent.parent), 'show', f'{RELEASED_0_4_0}:resources/scripts/napseer_credentials.py'],
|
||||
capture_output=True)
|
||||
if released.returncode != 0:
|
||||
pytest.skip(f'released 0.4.0 source {RELEASED_0_4_0} is not in this checkout')
|
||||
source = tmp_path/'napseer_credentials_0_4_0.py'
|
||||
source.write_bytes(released.stdout)
|
||||
spec = importlib.util.spec_from_file_location('napseer_credentials_0_4_0', source)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
main, _ = worktrees
|
||||
(main/'.napseer-tools.json').write_text('{"schema": "napseer.tools.v1", "tool_profiles": ["workflow"]}\n')
|
||||
paths = module.WorkspacePaths(main)
|
||||
assert paths.migration_sources() == []
|
||||
paths.require_migrated()
|
||||
# Control: the same file inside .napseer/ would block 0.4.0 clients.
|
||||
(main/'.napseer/tools.json').write_text('{}\n')
|
||||
assert [p.name for p in paths.migration_sources()] == ['tools.json']
|
||||
with pytest.raises(module.CredentialError):
|
||||
paths.require_migrated()
|
||||
Reference in New Issue
Block a user