Commit Graph

797 Commits (d90cac990c8f9864d2bcb0f3605a057fad2003cf)

Author SHA1 Message Date
Tyler Yust d90cac990c
fix: cron scheduler reliability, store hardening, and UX improvements (#10776)
* refactor: update cron job wake mode and run mode handling

- Changed default wake mode from 'next-heartbeat' to 'now' in CronJobEditor and related CLI commands.
- Updated cron-tool tests to reflect changes in run mode, introducing 'due' and 'force' options.
- Enhanced cron-tool logic to handle new run modes and ensure compatibility with existing job structures.
- Added new tests for delivery plan consistency and job execution behavior under various conditions.
- Improved normalization functions to handle wake mode and session target casing.

This refactor aims to streamline cron job configurations and enhance the overall user experience with clearer defaults and improved functionality.

* test: enhance cron job functionality and UI

- Added tests to ensure the isolated agent correctly announces the final payload text when delivering messages via Telegram.
- Implemented a new function to pick the last deliverable payload from a list of delivery payloads.
- Enhanced the cron service to maintain legacy "every" jobs while minute cron jobs recompute schedules.
- Updated the cron store migration tests to verify the addition of anchorMs to legacy every schedules.
- Improved the UI for displaying cron job details, including job state and delivery information, with new styles and layout adjustments.

These changes aim to improve the reliability and user experience of the cron job system.

* test: enhance sessions thinking level handling

- Added tests to verify that the correct thinking levels are applied during session spawning.
- Updated the sessions-spawn-tool to include a new parameter for overriding thinking levels.
- Enhanced the UI to support additional thinking levels, including "xhigh" and "full", and improved the handling of current options in dropdowns.

These changes aim to improve the flexibility and accuracy of thinking level configurations in session management.

* feat: enhance session management and cron job functionality

- Introduced passthrough arguments in the test-parallel script to allow for flexible command-line options.
- Updated session handling to hide cron run alias session keys from the sessions list, improving clarity.
- Enhanced the cron service to accurately record job start times and durations, ensuring better tracking of job execution.
- Added tests to verify the correct behavior of the cron service under various conditions, including zero-delay timers.

These changes aim to improve the usability and reliability of session and cron job management.

* feat: implement job running state checks in cron service

- Added functionality to prevent manual job runs if a job is already in progress, enhancing job management.
- Updated the `isJobDue` function to include checks for running jobs, ensuring accurate scheduling.
- Enhanced the `run` function to return a specific reason when a job is already running.
- Introduced a new test case to verify the behavior of forced manual runs during active job execution.

These changes aim to improve the reliability and clarity of cron job execution and management.

* feat: add session ID and key to CronRunLogEntry model

- Introduced `sessionid` and `sessionkey` properties to the `CronRunLogEntry` struct for enhanced tracking of session-related information.
- Updated the initializer and Codable conformance to accommodate the new properties, ensuring proper serialization and deserialization.

These changes aim to improve the granularity of logging and session management within the cron job system.

* fix: improve session display name resolution

- Updated the `resolveSessionDisplayName` function to ensure that both label and displayName are trimmed and default to an empty string if not present.
- Enhanced the logic to prevent returning the key if it matches the label or displayName, improving clarity in session naming.

These changes aim to enhance the accuracy and usability of session display names in the UI.

* perf: skip cron store persist when idle timer tick produces no changes

recomputeNextRuns now returns a boolean indicating whether any job
state was mutated. The idle path in onTimer only persists when the
return value is true, eliminating unnecessary file writes every 60s
for far-future or idle schedules.

* fix: prep for merge - explicit delivery mode migration, docs + changelog (#10776) (thanks @tyler6204)
2026-02-06 18:03:03 -08:00
Tak Hoffman 8a352c8f9d
Web UI: add token usage dashboard (#10072)
* feat(ui): Token Usage dashboard with session analytics

Adds a comprehensive Token Usage view to the dashboard:

Backend:
- Extended session-cost-usage.ts with per-session daily breakdown
- Added date range filtering (startMs/endMs) to API endpoints
- New sessions.usage, sessions.usage.timeseries, sessions.usage.logs endpoints
- Cost breakdown by token type (input/output/cache read/write)

Frontend:
- Two-column layout: Daily chart + breakdown | Sessions list
- Interactive daily bar chart with click-to-filter and shift-click range select
- Session detail panel with usage timeline, conversation logs, context weight
- Filter chips for active day/session selections
- Toggle between tokens/cost view modes (default: cost)
- Responsive design for smaller screens

UX improvements:
- 21-day default date range
- Debounced date input (400ms)
- Session list shows filtered totals when days selected
- Context weight breakdown shows skills, tools, files contribution

* fix(ui): restore gatewayUrl validation and syncUrlWithSessionKey signature

- Restore normalizeGatewayUrl() to validate ws:/wss: protocol
- Restore isTopLevelWindow() guard for iframe security
- Revert syncUrlWithSessionKey signature (host param was unused)

* feat(ui): Token Usage dashboard with session analytics

Adds a comprehensive Token Usage view to the dashboard:

Backend:
- Extended session-cost-usage.ts with per-session daily breakdown
- Added date range filtering (startMs/endMs) to API endpoints
- New sessions.usage, sessions.usage.timeseries, sessions.usage.logs endpoints
- Cost breakdown by token type (input/output/cache read/write)

Frontend:
- Two-column layout: Daily chart + breakdown | Sessions list
- Interactive daily bar chart with click-to-filter and shift-click range select
- Session detail panel with usage timeline, conversation logs, context weight
- Filter chips for active day/session selections
- Toggle between tokens/cost view modes (default: cost)
- Responsive design for smaller screens

UX improvements:
- 21-day default date range
- Debounced date input (400ms)
- Session list shows filtered totals when days selected
- Context weight breakdown shows skills, tools, files contribution

* fix: usage dashboard data + cost handling (#8462) (thanks @mcinteerj)

* Usage: enrich metrics dashboard

* Usage: add latency + model trends

* Gateway: improve usage log parsing

* UI: add usage query helpers

* UI: client-side usage filter + debounce

* Build: harden write-cli-compat timing

* UI: add conversation log filters

* UI: fix usage dashboard lint + state

* Web UI: default usage dates to local day

* Protocol: sync session usage params (#8462) (thanks @mcinteerj, @TakHoffman)

---------

Co-authored-by: Jake McInteer <mcinteerj@gmail.com>
2026-02-05 22:35:46 -06:00
Coy Geek 717129f7f9
fix: silence unused hook token url param (#9436)
* fix: Gateway authentication token exposed in URL query parameters

* fix: silence unused hook token url param

* fix: remove gateway auth tokens from URLs (#9436) (thanks @coygeek)

* test: fix Windows path separators in audit test (#9436)

---------

Co-authored-by: George Pickett <gpickett00@gmail.com>
2026-02-05 18:08:29 -08:00
George Pickett a459e237e8 fix(gateway): require auth for canvas host and a2ui assets (#9518) (thanks @coygeek) 2026-02-05 16:37:58 -08:00
Coy Geek 47538bca4d fix: Gateway canvas host bypasses auth and serves files unauthenticated 2026-02-05 16:37:58 -08:00
Abdel Sy Fane 0c7fa2b0d5
security: redact credentials from config.get gateway responses (#9858)
* security: add skill/plugin code safety scanner module

* security: integrate skill scanner into security audit

* security: add pre-install code safety scan for plugins

* style: fix curly brace lint errors in skill-scanner.ts

* docs: add changelog entry for skill code safety scanner

* security: redact credentials from config.get gateway responses

The config.get gateway method returned the full config snapshot
including channel credentials (Discord tokens, Slack botToken/appToken,
Telegram botToken, Feishu appSecret, etc.), model provider API keys,
and gateway auth tokens in plaintext.

Any WebSocket client—including the unauthenticated Control UI when
dangerouslyDisableDeviceAuth is set—could read every secret.

This adds redactConfigSnapshot() which:
- Deep-walks the config object and masks any field whose key matches
  token, password, secret, or apiKey patterns
- Uses the existing redactSensitiveText() to scrub the raw JSON5 source
- Preserves the hash for change detection
- Includes 15 test cases covering all channel types

* security: make gateway config writes return redacted values

* test: disable control UI by default in gateway server tests

* fix: redact credentials in gateway config APIs (#9858) (thanks @abdelsfane)

---------

Co-authored-by: George Pickett <gpickett00@gmail.com>
2026-02-05 16:34:48 -08:00
Abdel Sy Fane bc88e58fcf
security: add skill/plugin code safety scanner (#9806)
* security: add skill/plugin code safety scanner module

* security: integrate skill scanner into security audit

* security: add pre-install code safety scan for plugins

* style: fix curly brace lint errors in skill-scanner.ts

* docs: add changelog entry for skill code safety scanner

* style: append ellipsis to truncated evidence strings

* fix(security): harden plugin code safety scanning

* fix: scan skills on install and report code-safety details

* fix: dedupe audit-extra import

* fix(security): make code safety scan failures observable

* fix(test): stabilize smoke + gateway timeouts (#9806) (thanks @abdelsfane)

---------

Co-authored-by: Darshil <ddhameliya@mail.sfsu.edu>
Co-authored-by: Darshil <81693876+dvrshil@users.noreply.github.com>
Co-authored-by: George Pickett <gpickett00@gmail.com>
2026-02-05 16:06:11 -08:00
Gustavo Madeira Santana 4629054403
chore: apply local workspace updates (#9911)
* chore: apply local workspace updates

* fix: resolve prep findings after rebase (#9898) (thanks @gumadeiras)

* refactor: centralize model allowlist normalization (#9898) (thanks @gumadeiras)

* fix: guard model allowlist initialization (#9911)

* docs: update changelog scope for #9911

* docs: remove model names from changelog entry (#9911)

* fix: satisfy type-aware lint in model allowlist (#9911)
2026-02-05 16:54:44 -05:00
Gustavo Madeira Santana a13ff55bd9
Security: Prevent gateway credential exfiltration via URL override (#9179)
* Gateway: require explicit auth for url overrides

* Gateway: scope credential blocking to non-local URLs only

Address review feedback: the previous fix blocked credential fallback for
ALL URL overrides, which was overly strict and could break workflows that
use --url to switch between loopback/tailnet without passing credentials.

Now credential fallback is only blocked for non-local URLs (public IPs,
external hostnames). Local addresses (127.0.0.1, localhost, private IPs
like 192.168.x.x, 10.x.x.x, tailnet 100.x.x.x) still get credential
fallback as before.

This maintains the security fix (preventing credential exfiltration to
attacker-controlled URLs) while preserving backward compatibility for
legitimate local URL overrides.

* Security: require explicit credentials for gateway url overrides (#8113) (thanks @victormier)

* Gateway: reuse explicit auth helper for url overrides (#8113) (thanks @victormier)

* Tests: format gateway chat test (#8113) (thanks @victormier)

* Tests: require explicit auth for gateway url overrides (#8113) (thanks @victormier)

---------

Co-authored-by: Victor Mier <victormier@gmail.com>
2026-02-04 18:59:44 -05:00
Gustavo Madeira Santana 38e6da1fe0
TUI/Gateway: fix pi streaming + tool routing + model display + msg updating (#8432)
* TUI/Gateway: fix pi streaming + tool routing

* Tests: clarify verbose tool output expectation

* fix: avoid seq gaps for targeted tool events (#8432) (thanks @gumadeiras)
2026-02-04 17:12:16 -05:00
mudrii 5d82c82313
feat: per-channel responsePrefix override (#9001)
* feat: per-channel responsePrefix override

Add responsePrefix field to all channel config types and Zod schemas,
enabling per-channel and per-account outbound response prefix overrides.

Resolution cascade (most specific wins):
  L1: channels.<ch>.accounts.<id>.responsePrefix
  L2: channels.<ch>.responsePrefix
  L3: (reserved for channels.defaults)
  L4: messages.responsePrefix (existing global)

Semantics:
  - undefined -> inherit from parent level
  - empty string -> explicitly no prefix (stops cascade)
  - "auto" -> derive [identity.name] from routed agent

Changes:
  - Core logic: resolveResponsePrefix() in identity.ts accepts
    optional channel/accountId and walks the cascade
  - resolveEffectiveMessagesConfig() passes channel context through
  - Types: responsePrefix added to WhatsApp, Telegram, Discord, Slack,
    Signal, iMessage, Google Chat, MS Teams, Feishu, BlueBubbles configs
  - Zod schemas: responsePrefix added for config validation
  - All channel handlers wired: telegram, discord, slack, signal,
    imessage, line, heartbeat runner, route-reply, native commands
  - 23 new tests covering backward compat, channel/account levels,
    full cascade, auto keyword, empty string stops, unknown fallthrough

Fully backward compatible - no existing config is affected.
Fixes #8857

* fix: address CI lint + review feedback

- Replace Record<string, any> with proper typed helpers (no-explicit-any)
- Add curly braces to single-line if returns (eslint curly)
- Fix JSDoc: 'Per-channel' → 'channel/account' on shared config types
- Extract getChannelConfig() helper for type-safe dynamic key access

* fix: finish responsePrefix overrides (#9001) (thanks @mudrii)

* fix: normalize prefix wiring and types (#9001) (thanks @mudrii)

---------

Co-authored-by: Gustavo Madeira Santana <gumadeiras@gmail.com>
2026-02-04 16:16:34 -05:00
Tyler Yust f8d2534062 fix(cron): fix test failures and regenerate protocol files
- Add forceReload option to ensureLoaded to avoid stat I/O in normal
  paths while still detecting cross-service writes in the timer path
- Post isolated job summary back to main session (restores the old
  isolation.postToMainPrefix behavior via delivery model)
- Update legacy migration tests to check delivery.channel instead of
  payload.channel (normalization now moves delivery fields to top-level)
- Remove legacy deliver/channel/to/bestEffortDeliver from payload schema
- Update protocol conformance test for delivery modes
- Regenerate GatewayModels.swift (isolation -> delivery)
2026-02-04 01:03:59 -08:00
Tyler Yust 3f82daefd8 feat(cron): enhance delivery modes and job configuration
- Updated isolated cron jobs to support new delivery modes: `announce` and `none`, improving output management.
- Refactored job configuration to remove legacy fields and streamline delivery settings.
- Enhanced the `CronJobEditor` UI to reflect changes in delivery options, including a new segmented control for delivery mode selection.
- Updated documentation to clarify the new delivery configurations and their implications for job execution.
- Improved tests to validate the new delivery behavior and ensure backward compatibility with legacy settings.

This update provides users with greater flexibility in managing how isolated jobs deliver their outputs, enhancing overall usability and clarity in job configurations.
2026-02-04 01:03:59 -08:00
Tyler Yust 511c656cbc feat(cron): introduce delivery modes for isolated jobs
- Added support for new delivery modes in cron jobs: `announce`, `deliver`, and `none`.
- Updated documentation to reflect changes in delivery options and usage examples.
- Enhanced the cron job schema to include delivery configuration.
- Refactored related CLI commands and UI components to accommodate the new delivery settings.
- Improved handling of legacy delivery fields for backward compatibility.

This update allows users to choose how output from isolated jobs is delivered, enhancing flexibility in job management.
2026-02-04 01:03:59 -08:00
Tyler Yust 3a03e38378 fix(cron): fix timeout, add timestamp validation, enable file sync
Fixes #7667

Task 1: Fix cron operation timeouts
- Increase default gateway tool timeout from 10s to 30s
- Increase cron-specific tool timeout to 60s
- Increase CLI default timeout from 10s to 30s
- Prevents timeouts when gateway is busy with long-running jobs

Task 2: Add timestamp validation
- New validateScheduleTimestamp() function in validate-timestamp.ts
- Rejects atMs timestamps more than 1 minute in the past
- Rejects atMs timestamps more than 10 years in the future
- Applied to both cron.add and cron.update operations
- Provides helpful error messages with current time and offset

Task 3: Enable file sync for manual edits
- Track file modification time (storeFileMtimeMs) in CronServiceState
- Check file mtime in ensureLoaded() and reload if changed
- Recompute next runs after reload to maintain accuracy
- Update mtime after persist() to prevent reload loop
- Dashboard now picks up manual edits to ~/.openclaw/cron/jobs.json
2026-02-04 01:03:59 -08:00
Peter Steinberger bbe9cb3022 fix(update): honor update.channel for update.run 2026-02-03 17:57:55 -08:00
Armin Ronacher efe2a464af fix(approvals): gate /approve by gateway scopes 2026-02-03 16:18:49 -08:00
Peter Steinberger 66d8117d44 fix: harden control ui framing + ws origin 2026-02-03 16:00:57 -08:00
Gustavo Madeira Santana 5935c4d23d fix(ui): fix web UI after tsdown migration and typing changes 2026-02-03 13:56:20 -05:00
Gustavo Madeira Santana 2a68bcbeb3 feat(ui): add Agents dashboard 2026-02-02 21:31:17 -05:00
Peter Steinberger fe81b1d712 fix(gateway): require shared auth before device bypass 2026-02-02 16:56:38 -08:00
Mariano Belinky 4ab814fd50 Revert "iOS: wire node services and tests"
This reverts commit 7b0a0f3dac.
2026-02-02 17:36:49 +00:00
Mariano Belinky 532b9653be iOS: wire node commands and incremental TTS 2026-02-02 16:42:18 +00:00
Mariano Belinky 1a48bce294 iOS: add PTT once/cancel 2026-02-02 16:42:18 +00:00
Mariano Belinky 9f101d3a9a iOS: add push-to-talk node commands 2026-02-02 16:42:18 +00:00
Mariano Belinky a884955cd6 iOS: add write commands for contacts/calendar/reminders 2026-02-02 16:42:18 +00:00
cpojer 6b0d6e2540
chore: We have a sleep at home. The sleep at home: 2026-02-02 21:44:02 +09:00
cpojer 935a0e5708
chore: Enable `typescript/no-explicit-any` rule. 2026-02-02 16:18:09 +09:00
Tyler Yust 476f367cf1 Gateway: avoid writing host config in tools invoke test 2026-02-01 17:19:23 -08:00
cpojer f06dd8df06
chore: Enable "experimentalSortImports" in Oxfmt and reformat all imorts. 2026-02-01 10:03:47 +09:00
Tak Hoffman 9c29853014 Gateway: inject timestamps into agent/chat.send (#3705) (thanks @conroywhitney, @CashWilliams) 2026-01-31 09:47:27 -06:00
Conroy Whitney 8a5b139a9f revert: drop "Current Date:" label, keep [Wed YYYY-MM-DD HH:MM TZ]
Small model testing showed the label did not meaningfully help:
- Sub-3B models fail regardless of format
- 8B models untested with label specifically
- Frontier models never needed it

The bracket convention [Wed 2026-01-28 22:30 EST] matches existing
channel envelope format and is widely present in training data.
Saves ~2-3 tokens per message vs the labeled version.
2026-01-31 09:47:27 -06:00
Conroy Whitney b6c8c1e89d feat: add "Current Date:" label to timestamp prefix
Changes [Wed 2026-01-28 20:30 EST] to [Current Date: Wed 2026-01-28 20:30 EST].

Tested with qwen3-1.7B: even with DOW in the timestamp, the model
ignored it and tried to compute the day using Zeller's Congruence.
The "Current Date:" semantic label is widely present in training data
and gives small models the best chance of recognizing the timestamp
as authoritative context rather than metadata to parse.

Cost: ~18 tokens per message. Prevents hallucination spirals that
burn hundreds or thousands of tokens on date derivation.
2026-01-31 09:47:27 -06:00
Conroy Whitney a6c68e8690 feat: add 3-letter DOW prefix to injected timestamps
Changes [2026-01-28 20:30 EST] to [Wed 2026-01-28 20:30 EST].
Costs ~1 extra token but provides day-of-week for smaller models
that can't derive DOW from a date. Frontier models already handle
it, but this is cheap insurance for 7B-class models.
2026-01-31 09:47:27 -06:00
Conroy Whitney 76391bba3f refactor: use compact formatZonedTimestamp for injection
Replace verbose formatUserTime (Wednesday, January 28th, 2026 — 8:30 PM)
with the same formatZonedTimestamp used by channel envelopes (2026-01-28
20:30 EST). This:

- Saves ~4 tokens per message (~7 vs ~11)
- Uses globally unambiguous YYYY-MM-DD 24h format
- Removes 12/24h config option (always 24h, agent-facing)
- Anchors envelope detection to the actual format function — if channels
  change their timestamp format, our injection + detection change too
- Adds test that compares injection output to formatZonedTimestamp directly

Exported formatZonedTimestamp from auto-reply/envelope.ts for reuse.
2026-01-31 09:47:27 -06:00
Conroy Whitney 08886eaaa3 test: add DST boundary test for timestamp injection
Verifies that America/New_York correctly resolves to midnight for
both EST (winter, UTC-5) and EDT (summer, UTC-4) using the same
IANA timezone. Intl.DateTimeFormat handles the DST transition.
2026-01-31 09:47:27 -06:00
Conroy Whitney bbf2205640 feat(gateway): inject timestamps into chat.send (webchat/TUI)
The chat.send handler (used by webchat and TUI) is a separate path
from the agent handler. Inject timestamp into BodyForAgent (what the
model sees) while keeping Body raw for UI display.

This completes timestamp coverage for all non-channel paths:
- agent handler: spawned subagents, sessions_send, heartbeats
- chat.send: webchat, TUI
2026-01-31 09:47:27 -06:00
Conroy Whitney 582a4e261a feat(gateway): inject timestamps into agent handler messages
Messages arriving through the gateway agent method (TUI, web, spawned
subagents, sessions_send, heartbeats) now get a timestamp prefix
automatically. This gives all agent contexts date/time awareness
without modifying the system prompt (which is cached for stability).

Channel messages (Discord, Telegram, etc.) already have timestamps
via envelope formatting in a separate code path and never reach
the agent handler, so there is no double-stamping risk.

Cron jobs also inject their own 'Current time:' prefix and are
detected and skipped.

Extracted as a pure function (injectTimestamp) with 12 unit tests
covering: timezone handling, 12/24h format, midnight boundaries,
envelope detection, cron detection, and empty messages.

Integration test verifies the agent handler wires it in correctly.

Closes #3658
Refs: #1897, #1928, #2108
2026-01-31 09:47:27 -06:00
cpojer 59cfff02f6
chore: Emit TypeScript declaration files so that we can type-check the extensions folder soon. 2026-01-31 21:57:21 +09:00
Peter Steinberger a42e1c82d9 fix: restore tsc build and plugin install tests 2026-01-31 07:54:15 +00:00
cpojer 5ceff756e1
chore: Enable "curly" rule to avoid single-statement if confusion/errors. 2026-01-31 16:19:20 +09:00
cpojer b7e401b6b6
chore: Enable no-useless-concat (removing the lint rule from the disabled list automatically enables it because it is in the "suspicious" group). 2026-01-31 16:08:56 +09:00
cpojer 9c4cbaab7b
chore: Enable eslint/no-unused-vars. 2026-01-31 16:06:39 +09:00
cpojer 15792b153f
chore: Enable more lint rules, disable some that trigger a lot. Will clean up later. 2026-01-31 16:04:04 +09:00
Peter Steinberger 08ed62852a chore: update deps and pi model discovery 2026-01-31 06:45:57 +01:00
Mario Zechner c0a6e675a3 Agents: update pi dependencies to 0.50.7 2026-01-31 04:20:12 +01:00
Josh Palmer 6522de6ce0 Tests: use agents_list in tools invoke gateway test
What: swap sessions_list for agents_list in /tools/invoke tests
Why: avoid nested gateway calls that can hang under CI; still validates tool invocation + allowlist
Tests: not run (CI should cover)
2026-01-30 18:17:12 +01:00
Peter Steinberger 9a7160786a refactor: rename to openclaw 2026-01-30 03:16:21 +01:00
Josh Palmer 4b5514a259 Tests: default-disable plugins in VITEST 2026-01-29 17:14:14 +01:00
Peter Steinberger 6d16a658e5 refactor: rename clawdbot to moltbot with legacy compat 2026-01-27 12:21:02 +00:00