End-to-End Testing Guide

This guide walks through verifying that the full policy delivery chain works: CLI → API → device agent picks up the change.

Prerequisites

agent-charley-cli installed (see Getting Started)
An API key with read_write scope
At least one active enrolled device (see Devices)

Step 1: Verify Authentication

agent-charley-cli --verbose plan --file /dev/null 2>&1 | head -5

You should see your base URL and key prefix in the output. If you get 401, the key is wrong or expired. A simpler smoke test — pull current server policies:

agent-charley-cli pull --output /tmp/current.yaml && echo "Auth OK"

Step 2: Get Your Device ID

From the dashboard: Devices → find your test device → copy the UUID shown under the name. Or via the API:

curl -s https://dash.charlemagnelabs.ai/api/devices \
  -H "x-api-key: $CHARLEY_API_KEY" | python3 -m json.tool | grep device_id

Set it in your shell:

export TEST_DEVICE_ID=<your-device-uuid>

Step 3: Check the Current Effective Policy

agent-charley-cli effective $TEST_DEVICE_ID

Note the current values — you’ll compare against these after applying.

Step 4: Create a Test Policy File

Create test-policy.yaml with a change you can verify on the device. A safe, visible test is toggling suppress_frontend_ui or flow_logs_enabled:

- name: Test Policy
  type: app_config
  scope: device
  target: <your-device-uuid>
  locked: false
  data:
    flow_logs_enabled: true
    suppress_frontend_ui: false

Replace <your-device-uuid> with your $TEST_DEVICE_ID.

Step 5: Plan

agent-charley-cli plan --file test-policy.yaml

Expected output:

Plan: 1 to create, 0 to update, 0 to delete, 0 unchanged.

--- /dev/null
+++ Test Policy (device/<uuid>/app_config)
@@ ...

If you see 0 unchanged with no diff, the policy already matches what’s on the server.

Step 6: Apply

agent-charley-cli apply --file test-policy.yaml -y

Expected output:

Plan: 1 to create, 0 to update, 0 to delete, 0 unchanged.
Done.

Step 7: Confirm Delivery to Device

The device agent polls for policy updates periodically. After applying, verify that the device has received the new policy. Option A — Check via CLI:

agent-charley-cli effective $TEST_DEVICE_ID

The response shows what the API will serve to the device. If your new field appears here, the API has it correctly. Option B — Check on the device (macOS): The agent stores its current policy at:

~/Library/Application Support/AgentCharley/dlp_watchdog_policy.json

You can read it:

cat ~/Library/Application\ Support/AgentCharley/dlp_watchdog_policy.json

Option C — Check in the dashboard: Go to Policies, select the policy type, switch to Device override, and select your test device. You should see the values you applied.

Step 8: Test a DLP Policy

To test DLP end-to-end on a device where Agent Charley is running:

Apply a DLP policy with mode: warn and builtins: [credit_card].
On the test device, open any text editor or browser form.
Copy a test card number (use a Luhn-valid test number, e.g. 4111 1111 1111 1111) to the clipboard.
Paste it.
You should see a warning toast notification from Agent Charley.

Switch mode to block and repeat — the paste should be prevented and the clipboard cleared (if clear_clipboard_on_block: true).

Step 9: Clean Up

Delete the test policy after confirming delivery:

agent-charley-cli apply --file /dev/null --prune -y

--prune will delete all server policies not in the file. If your org has other policies, pass an empty file only in a test org, or delete the specific policy by listing all policies (agent-charley-cli pull) and removing just the test one.

Or delete it from the dashboard: Policies → Device override → select device → delete.

Common Issues

Symptom	Likely Cause
`401 Unauthorized`	Wrong or expired API key
`403 Forbidden`	Read-only key used for `apply`
`404 Not Found` on group target	Group doesn’t exist yet — run `apply --groups groups.yaml` first
Policy shows in `effective` but device hasn’t changed	Agent is still on its poll interval; wait ~60 seconds
`Failed to load policies`	YAML parse error — check indentation (use 2-space indent)
Plan shows no changes when you expect some	The local file already matches server state — pull to verify

Testing Against Dev Environment

agent-charley-cli --env dev plan --file test-policy.yaml
agent-charley-cli --env dev apply --file test-policy.yaml -y

Or set CHARLEY_ENV=dev to avoid repeating the flag.

Overview

Dashboard

Policy as Code

Integrations

API Reference

Self-Hosting

End-to-End Testing Guide

Prerequisites

Step 1: Verify Authentication

Step 2: Get Your Device ID

Step 3: Check the Current Effective Policy

Step 4: Create a Test Policy File

Step 5: Plan

Step 6: Apply

Step 7: Confirm Delivery to Device

Step 8: Test a DLP Policy

Step 9: Clean Up

Common Issues

Testing Against Dev Environment

​Prerequisites

​Step 1: Verify Authentication

​Step 2: Get Your Device ID

​Step 3: Check the Current Effective Policy

​Step 4: Create a Test Policy File

​Step 5: Plan

​Step 6: Apply

​Step 7: Confirm Delivery to Device

​Step 8: Test a DLP Policy

​Step 9: Clean Up

​Common Issues

​Testing Against Dev Environment

Prerequisites

Step 1: Verify Authentication

Step 2: Get Your Device ID

Step 3: Check the Current Effective Policy

Step 4: Create a Test Policy File

Step 5: Plan

Step 6: Apply

Step 7: Confirm Delivery to Device

Step 8: Test a DLP Policy

Step 9: Clean Up

Common Issues

Testing Against Dev Environment