Miji 弥济 logo

Developer docs

Testing in the Lab: build a suite from a file, run it on a draft, and check a deploy

A test suite in Miji is a set of cases for one workflow: each case is an input, the answers its outside steps get, the decisions its approval steps take, and what the run should produce. A test run runs one TEST run of the workflow per case and judges each against its expectations as it ends. This guide goes from a CSV file to a deploy decision, through the REST API and through the MCP tools an AI assistant uses, which wrap the same routes.

A TEST run sends nothing outside Miji. A read such as a credit pull is answered from the case, the suite or the stub library; a write, an email or any other outside call is held with what it would have sent; an approval step takes the case's scripted decision or stops the case as needing a person, and no approval is raised. Test runs are kept out of every live list, count and alert.

In short

  • Eight calls take you from a CSV file to a deploy decision: store the data, create a suite, map the columns, import, run, read the results, compare, check the deploy.
  • Every route needs the permission it names: testing.view to read, testing.manage to build, testing.run to run. An API key holds them only when its grant names them.
  • Results come a page at a time. Through MCP, test_run_results returns 50 cases a page and never more than 64 KB.
  • A deploy to Production asks the workflow's deploy check: the suites it requires must have passed on that exact version.

Before you start

  • An API key. Mint one at Settings › API keys with the scopes experiments.run, testing.view, testing.manage and testing.run, and a role of Operator or above. The suite and run routes also ask the Lab's experiments.run and the Operator role.
  • The base URL. Every route is under /v1, for example https://<your-miji>/v1. The examples read it from MIJI_API_URL and the key from MIJI_API_KEY.
  • A workflow and its Test environment. The suite tests one workflow. A run of a version needs that version deployed to the Test environment; a run of the draft needs nothing deployed.
  • Or the MCP server. Add @mijios/mcp-server to your assistant with the same URL and key. Its test_* tools take the same fields as the routes below.
What each permission allows
PermissionAllows
testing.viewReading datasets, cases, stubs, run results, comparisons and the deploy check.
testing.manageStoring datasets, creating and changing suites and cases, saving a mapping, importing, writing stubs, accepting results as expected.
testing.runStarting and cancelling a test run.
testing.deploy_overrideDeploying past a Block deploy check, with a reason. Admin roles only.

A key without the permission a route needs is refused with 403 and the permission named: Missing permission(s): testing.run.

1. Store the test data

Send rows as objects, CSV text with a header row, or JSON text. origin says whether the data is made up or real; real data comes from past Production runs unless the workspace allows uploads. caseKey names the column that keys each row, so a later import updates a case rather than adding one. At most 10,000 rows.

curl
curl -s -X POST "$MIJI_API_URL"/test-datasets -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json" -d '{
  "origin": "made_up",
  "caseKey": "applicant_id",
  "csv": "applicant_id,credit_file,expected_decision\napp-7,\"{\"\"creditScore\"\": 712}\",Approved\napp-8,\"{\"\"creditScore\"\": 610}\",Declined\n"
}'
# → { "datasetId": "…", "columns": [...], "rowCount": 2, "preview": [...] }
MCP
test_dataset_create { "origin": "made_up", "caseKey": "applicant_id", "csv": "applicant_id,credit_file,expected_decision\n…" }

test_dataset_from_runs makes a dataset from a workflow's past live runs instead: one row per run, with what its outside steps got back and what its people decided.

2. Create a suite

A suite names its workflow and the environment its runs belong to, and what every case is judged on. Here each case is judged on the run's named result, translated from the file's words through a value map.

curl
curl -s -X POST "$MIJI_API_URL"/scenario-sets -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json" -d '{
  "name": "Credit decisions",
  "workflowId": "<workflow id>",
  "environmentId": "<Test environment id>",
  "valueMaps": { "decision": { "Approved": "approved", "Declined": "declined" } },
  "expectations": [{
    "id": "result", "label": "Result", "target": { "result": true },
    "comparator": { "name": "translate_equal", "map": "decision" },
    "expected": { "column": "expected_decision" }
  }]
}'
# → { "id": "<suite id>", "cases": [], ... }
MCP
test_suite_create { "name": "Credit decisions", "workflowId": "<workflow id>", "environmentId": "<Test environment id>", "valueMaps": {...}, "expectations": [...] }

3. Map the columns to the input

A mapping says how a row becomes a case. Each rule writes one target from one or more columns: a path in the input, the answer an outside step returns, or an expected value. Here the credit file becomes the bureau step's answer, so the run never calls the bureau. Rules run no AI.

curl
curl -s -X PUT "$MIJI_API_URL"/scenario-sets/<suite id>/mapping -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json" -d '{
  "mode": "fields",
  "rules": [
    { "target": { "path": "applicantId" }, "source": "applicant_id", "transform": "copy" },
    { "target": { "answer": "pull-credit" }, "source": "credit_file", "transform": "copy" }
  ]
}'
# → { "version": 1, "mode": "fields", ... }
MCP
test_suite_mapping_save { "id": "<suite id>", "mode": "fields", "rules": [...] }

4. Import the rows into cases

The import runs every row through the saved mapping in one transaction. A row whose key matches a case updates it; a row with problems becomes a draft case, which never runs, with its problems listed. Preview first to see the counts and the first 20 cases without writing anything.

curl
curl -s -X POST "$MIJI_API_URL"/scenario-sets/<suite id>/import/preview -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json" -d '{ "datasetId": "<dataset id>" }'
curl -s -X POST "$MIJI_API_URL"/scenario-sets/<suite id>/import -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json" -d '{ "datasetId": "<dataset id>" }'
# → { "rows": 2, "new": 2, "updated": 0, "unchanged": 0, "ready": 2, "drafts": [] }
MCP
test_suite_import { "id": "<suite id>", "datasetId": "<dataset id>", "preview": true }
test_suite_import { "id": "<suite id>", "datasetId": "<dataset id>" }

Cases can also be written one at a time (test_case_create), changed, or set aside with a reason until a day (test_case_update).

5. Run the suite on the draft

A run starts one TEST run per ready case. version is "draft" (the saved draft, the default), a version number deployed to Test, or "deployed:<environment id>". compareRunId names an earlier run to compare with. countsForDeploy marks a run of a version as one the deploy check may count; a draft never counts. The run's id comes back at once.

curl
curl -s -X POST "$MIJI_API_URL"/replay-batches -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json" -d '{
  "scenarioSetId": "<suite id>", "version": "draft", "compareRunId": "<previous run id>"
}'
# → { "batchId": "<run id>", "status": "RUNNING", "cases": 2, "version": "draft" }

curl -s "$MIJI_API_URL"/replay-batches/<run id> -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json"   # progress: status and counts
MCP
test_run_start { "suiteId": "<suite id>", "version": "draft", "compareRunId": "<previous run id>" }
test_run_get { "id": "<run id>" }

6. Read the results

The results give the totals by outcome (passed, failed, errored, needs_person, set_aside), the matrix of expected against actual results, failures by expectation, what changed against the compared run, the facts (calls outside Miji, which must be 0), and a page of cases. Narrow the cases by outcome — not_passed means failed, errored or needing a person — or by a matrix cell, a failed expectation, a tag, or changed.

curl
curl -s "$MIJI_API_URL/replay-batches/<run id>/results?outcome=not_passed&page=1&pageSize=50" -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json"
# → { "totals": { "cases": 2, "passed": 1, "failed": 1, ... }, "cases": { "items": [...], "page": 1, "total": 1 }, ... }

curl -s "$MIJI_API_URL"/replay-batches/<run id>/cases/<case id> -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json"   # one case: verdicts, paths, output diff
MCP
test_run_results { "id": "<run id>", "outcome": "not_passed" }
test_run_case { "id": "<run id>", "caseId": "<case id>" }

Through MCP, test_run_results asks for 50 cases a page and never returns more than 64 KB: lists that grow with a run are counted, and a page too heavy for the limit shows its cases in brief. Page with page until cases.next is null.

When a case did what it now should, accept it as expected with test_expectation_change_request. On a suite a deploy check requires, the change waits for a second person.

7. Compare with the previous run

A comparison of two runs of one suite says what differs between them (the version, the configuration, the rules and tables read, the stubs, the models), how many cases now fail or now pass, and why each moved case moved. The first id is the earlier run.

curl
curl -s "$MIJI_API_URL/replay-batches/<previous run id>/compare/<run id>?show=outcome" -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json"
MCP
test_run_compare { "id": "<previous run id>", "other": "<run id>" }

8. Check a deploy

A workflow's deploy check names the suites that must pass before a version goes to Production, in one of three modes: Off, Warn or Block. Reading it says, for a version, whether it would pass, and each required suite's state on that exact version: passed, failed, running, stale or no_run. Only runs started with countsForDeploy count, and the draft never passes.

curl
curl -s "$MIJI_API_URL/workflows/<workflow id>/deploy-check?version=8" -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json"
# → { "version": 8, "mode": "block", "passed": true, "suites": [{ "name": "Credit decisions", "state": "passed", ... }] }
MCP
workflow_deploy_check { "id": "<workflow id>", "version": 8 }

A deploy to Production asks the same check. Under Block a check that did not pass refuses the deploy with DEPLOY_CHECK_FAILED, unless override.reason is given by a role the check lets override; through an API key, the grant must also include testing.deploy_override. The override is written to the audit ledger with its reason.

One TEST run, with no suite

To try a single input on the draft, trigger the workflow as a test. It runs on the Test environment's deployment, sends nothing outside Miji, and stops at a read it has no answer for with NO_ANSWER_FOR_STEP.

curl
curl -s -X POST "$MIJI_API_URL"/workflows/<workflow id>/trigger -H "Authorization: Bearer $MIJI_API_KEY" -H "Content-Type: application/json" -d '{ "mode": "TEST", "inputData": { "applicantId": "app-7" } }'
MCP
workflow_trigger { "id": "<workflow id>", "asTest": true, "inputData": { "applicantId": "app-7" } }

Every MCP tool for testing

ToolRoutePermission
test_dataset_createPOST /test-datasetstesting.manage
test_dataset_from_runsPOST /test-datasets/from-runstesting.manage
test_suite_list, test_suite_getGET /scenario-sets, GET /scenario-sets/:idexperiments.run
test_suite_createPOST /scenario-setsexperiments.run and testing.manage
test_suite_updatePATCH /scenario-sets/:idtesting.manage
test_suite_mapping_savePUT /scenario-sets/:id/mappingtesting.manage
test_suite_importPOST /scenario-sets/:id/import (or /import/preview)testing.manage
test_case_list, test_case_getGET /scenario-sets/:id/cases, …/cases/:caseIdtesting.view
test_case_create, test_case_updatePOST …/cases, PATCH …/cases/:caseIdtesting.manage
test_run_start, test_run_cancelPOST /replay-batches, POST /replay-batches/:id/canceltesting.run
test_run_get, test_run_results, test_run_caseGET /replay-batches/:id, …/results, …/cases/:caseIdtesting.view
test_run_compareGET /replay-batches/:a/compare/:btesting.view
test_expectation_change_requestPOST /scenario-sets/:id/expectation-changestesting.manage
test_stub_list, test_stub_upsert, test_stub_deleteGET, POST, PUT, DELETE /test-stubstesting.view, testing.manage
workflow_deploy_checkGET /workflows/:id/deploy-checktesting.view

The run routes also need experiments.run and the Operator role. The full request and response schemas are in the platform's OpenAPI document for testing.

See a suite run on a real decision workflow.

Book a demo

Frequently asked questions

Does a test run call my credit bureau or send emails?

No. A TEST run sends nothing outside Miji: reads are answered from the case, the suite or the stub library, and writes and messages are held with what they would have sent. The results' facts count calls outside Miji, and the count is 0.

Can an AI assistant run my tests?

Yes, through Miji's MCP server, with an API key whose grant includes the testing permissions. It cannot deploy past a Block deploy check unless the key's grant includes testing.deploy_override and its role is one the check allows.

How large can a suite be?

Up to 2,000 cases per suite, and up to 10,000 rows per dataset.

White paper

Enterprise Decision Automation White Paper

Which decisions to automate, where AI belongs and where it does not, and how to change a policy without losing the record.

Download the white paper