Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ Every plugin (`rf-core`, `rf-web`, `rf-api`, `rf-mobile`, `rf-desktop`) and the

## Unreleased

### rf-core

- **New skill `rf-bdd`** for behaviour-driven specifications in native `.robot` files: features and rules as suites, documentation and tags, one step vocabulary, scenario state, outlines as templates, pasted Gherkin or acceptance criteria translated by meaning, and reviews with check IDs. It follows the project's conventions and ships a Robocop configuration for sentence-style steps (`assets/robocop-bdd.toml`) and two runnable example projects. No `.feature` files and no Gherkin parser.
- `rf-language` names `rf-bdd` for BDD specifications. Its casing gotcha now says to follow the project's Robocop configuration instead of requiring Title Case BDD steps.
- The context hook routes Given/When/Then and Gherkin work to `rf-bdd`, and `rf-test-architect` routes BDD specifications to it.

## 5.0.0 — 2026-11-18

One version for everything: every plugin and the installer are now `5.0.0`, released together with one tag, `v5.0.0`.
Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ There are two ways to install, and both work at **user scope** (every project on

| Plugin | What it adds |
|---|---|
| `rf-core` | Robot Framework syntax (`rf-language`), environment setup (`rf-setup`), Python keyword libraries (`rf-python-library`); the `rf-test-architect`, `rf-debug-expert` and `rf-migration-guide` subagents; all hooks. Every other plugin needs it. |
| `rf-core` | Robot Framework syntax (`rf-language`), BDD specifications (`rf-bdd`), environment setup (`rf-setup`), Python keyword libraries (`rf-python-library`); the `rf-test-architect`, `rf-debug-expert` and `rf-migration-guide` subagents; all hooks. Every other plugin needs it. |
| `rf-web` | Browser Library (`rf-browser`) and SeleniumLibrary (`rf-selenium`) |
| `rf-api` | RequestsLibrary (`rf-requests`) and RESTinstance (`rf-restinstance`) |
| `rf-mobile` | AppiumLibrary (`rf-appium`) |
Expand Down Expand Up @@ -272,7 +272,7 @@ The installer, hooks and scripts send nothing anywhere. The installer only touch

## What you get

### 9 Skills
### 10 Skills

| Skill | Type | Command | Description |
|-------|------|---------|-------------|
Expand All @@ -283,6 +283,7 @@ The installer, hooks and scripts send nothing anywhere. The installer only touch
| RESTinstance | library-reference | `/rf-api:rf-restinstance` | REST API tests with RESTinstance (`Library REST`): JSON Schema / OpenAPI-driven assertions |
| PlatynUI (preview) | library-reference | `/rf-desktop:rf-platynui` | Native desktop UI testing (Windows UIA, Linux AT-SPI2) via PlatynUI.BareMetal |
| Robot Framework language | language-guide + script | `/rf-core:rf-language` | Write and review tests (keyword-driven, data-driven, BDD), suites, `__init__.robot`, tags, user keywords, `.resource` and variable files in modern, version-gated syntax; `rf_conventions.py` script detects the project's conventions (requires robotframework). Not for Python keyword libraries |
| BDD specifications | language-guide | `/rf-core:rf-bdd` | Write and review behaviour-driven specifications in native `.robot` files: features and rules as suites, documentation and tags, one step vocabulary, scenario state, outlines as templates, pasted Gherkin or acceptance criteria translated by meaning, reviews with check IDs; follows the project's conventions. Works together with `rf-language` |
| Python keyword libraries | library-guide + script | `/rf-core:rf-python-library` | Write and fix keyword libraries and listeners in Python: `@library`/`@keyword`, scope by state lifetime (TEST/SUITE/GLOBAL), type-hint conversion and custom converters, failures and logging, dynamic/hybrid APIs, listener API v3, libdoc; `check_library.py` script finds "contains no keywords", leaked imports, lost signatures and state kept in `TEST` scope before tests run (requires robotframework, Python 3.10+) |
| Setup | setup-guide | `/rf-core:rf-setup` | Install Robot Framework and libraries with uv, venv + pip or Poetry; fix environment errors (`ModuleNotFoundError`, wrong interpreter); project layout, CI |

Expand Down
1 change: 1 addition & 0 deletions eval/fixtures/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Fixture directories under `eval/fixtures/` should be kept clean (no
| `sut-rf71` | Project pinned to `robotframework==7.1.1` (`uv.lock`, `.python-version`); three cart tests repeating the same four steps. Graded with `uvx --from robotframework==7.1.1 robot` (adversarial typed-argument task). | `robotframework==7.1.1` | `uv` with network on first use (skipped otherwise). |
| `sut-pylib` | rf-python-library tasks: an empty `libraries/` folder on the python-path (`robot.toml` `python-path = ["libraries"]`) where the agent writes `Inventory.py`, `Modes.py` or `FlakySkip.py`; `tests/example.robot` is a smoke test. Hidden grader suites live in `eval/graders/python-library/`, not here. | `robotframework>=7.0` | None. |
| `sut-legacy-style` | Adversarial legacy-syntax task (`adv-legacy-syntax-01`): `resources/legacy.resource` uses `[Return]`, `Run Keyword If`, `Set Suite/Test Variable`, `Create List`; `tests/cart.robot` uses `Force Tags` and passes. The agent must add `resources/orders.resource` in modern syntax despite the "same style" prompt. | `robotframework>=7.4` (`robotframework-robocop` 9.x as dev dependency) | None. |
| `sut-bdd-library` | rf-bdd tasks. One fixture with three variant directories, selected by the task's `fixture` (`sut-bdd-library/<variant>`; the harness copies one directory, so each variant is the base plus its overlay files, kept identical by `tests/eval/test_bdd_grader.py`). `base/`: in-memory lending service `sut/LibraryService.py` (library scope TEST, calendar fixed at 2026-11-12, so loans are due 2026-12-03), `robot.toml` (`python-path = ["resources", "sut"]`), empty `tests/` and `resources/`, technical smoke suite `smoke/library_service.robot`. `steps/`: base + `tests/borrowing.robot` and `resources/borrowing_steps.resource` (2 BDD tests; extend-vocabulary task). `review/`: base + `tests/loans.robot` and `tests/loan_steps.resource` with 7 seeded review issues (L1, L2, W1, W2/C5, W3, L6, W12/W6/W7; review task). Smoke: `robotcode robot smoke` (base), `robotcode robot tests` (steps, review). | `robotframework>=7.3` | None. |
| `sut-trigger` | Neutral working directory for **trigger evals** (not task evals): `pyproject.toml` declaring only `robotframework`, `tests/smoke.robot` (BuiltIn only), `resources/common.resource`. Imports no test library, so the query alone decides which skill loads. A fresh copy is made per trigger session. | `robotframework>=7.0` | None. |

## Prerequisites
Expand Down
9 changes: 9 additions & 0 deletions eval/fixtures/sut-bdd-library/base/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
output.xml
log.html
report.html
results/
.venv/
__pycache__/
*.pyc
.pytest_cache/
.robotcode_cache/
21 changes: 21 additions & 0 deletions eval/fixtures/sut-bdd-library/base/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# library-lending

Acceptance tests for the city library's lending service. The service runs in
memory (`sut/LibraryService.py`, a Robot Framework library with one fresh
instance per test); its business rules are in the module docstring. The
library's calendar starts on 2026-11-12, so due dates are fixed (`Set Today`
moves it).

- `tests/`: acceptance tests (none yet).
- `resources/`: shared keywords for the tests.
- `smoke/library_service.robot`: a technical check that the service library
loads and lends a book.

`robot.toml` puts `resources/` and `sut/` on the python-path, so both are
imported by name. Run from this directory with robotcode, which reads
`robot.toml`:

```bash
robotcode robot # the acceptance tests in tests/
robotcode robot smoke
```
15 changes: 15 additions & 0 deletions eval/fixtures/sut-bdd-library/base/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[project]
name = "library-lending"
version = "0.0.1"
description = "Acceptance tests for the city library's lending service."
requires-python = ">=3.10"
dependencies = [
"robotframework>=7.3",
]

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
packages = []
Empty file.
3 changes: 3 additions & 0 deletions eval/fixtures/sut-bdd-library/base/robot.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
paths = ["tests"]
output-dir = "results"
python-path = ["resources", "sut"]
12 changes: 12 additions & 0 deletions eval/fixtures/sut-bdd-library/base/smoke/library_service.robot
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
*** Settings ***
Documentation Technical smoke check: the service library loads and lends a book.
Library LibraryService


*** Test Cases ***
Service lends an available copy
Register Member Smoke Reader
Add Book Smoke Test Title 1
${loan} Borrow Smoke Reader Smoke Test Title
Should Be Equal ${loan}[status] lent
Should Be Equal As Strings ${loan}[due_date] 2026-12-03
132 changes: 132 additions & 0 deletions eval/fixtures/sut-bdd-library/base/sut/LibraryService.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
"""Lending service of the city library (in-memory backend).

Rules:
- A member borrows a book if a copy is available, the member is not blocked,
has no unpaid fines and is below the loan limit (standard: 3, premium: 10).
- Loans are due 21 days after the lending day.
- Returning late costs 0.50 EUR per day; the fine is added to the member's account.
- A member may reserve a book that has no copy available; reservations queue in order.
- A reserved returned copy is held for the first member in the queue.
"""

from datetime import date, timedelta
from decimal import Decimal

from robot.api.deco import keyword, library

LOAN_LIMITS = {"standard": 3, "premium": 10}
LOAN_DAYS = 21
FINE_PER_DAY = Decimal("0.50")


@library(scope="TEST")
class LibraryService:
"""Fake library backend. A fresh instance is created for every test."""

def __init__(self) -> None:
self._today = date(2026, 11, 12)
self._copies: dict[str, int] = {}
self._members: dict[str, dict] = {}
self._loans: dict[str, dict[str, date]] = {}
self._reservations: dict[str, list[str]] = {}
self._held_for: dict[str, list[str]] = {}
self._last_receipt = ""

@keyword
def set_today(self, day: date) -> None:
"""Sets the library's current date (ISO format, e.g. 2026-11-12)."""
self._today = day

@keyword
def add_book(self, title: str, copies: int = 1) -> None:
"""Adds ``copies`` copies of ``title`` to the shelf."""
self._copies[title] = self._copies.get(title, 0) + copies

@keyword
def register_member(self, name: str, membership: str = "standard") -> None:
"""Registers a member with a ``standard`` or ``premium`` membership."""
if membership not in LOAN_LIMITS:
raise ValueError(f"Unknown membership '{membership}'")
self._members[name] = {"membership": membership, "fines": Decimal("0"), "blocked": False}
self._loans[name] = {}

@keyword
def block_member(self, name: str) -> None:
"""Blocks a member; blocked members cannot borrow."""
self._members[name]["blocked"] = True

@keyword
def add_fine(self, name: str, amount: Decimal) -> None:
"""Adds an unpaid fine of ``amount`` EUR to a member's account."""
self._members[name]["fines"] += Decimal(amount)

@keyword
def borrow(self, name: str, title: str) -> dict:
"""Requests a loan. Returns ``{status, reason, due_date}``."""
member = self._members[name]
if member["blocked"]:
return self._refused("Member is blocked")
if member["fines"] > 0:
return self._refused("Unpaid fines")
if len(self._loans[name]) >= LOAN_LIMITS[member["membership"]]:
return self._refused("Loan limit reached")
held = self._held_for.get(title, [])
if held and held[0] != name:
return self._refused("Copy is reserved for another member")
if held:
held.pop(0)
elif self._copies.get(title, 0) <= 0:
return self._refused("No copy available")
else:
self._copies[title] -= 1
due = self._today + timedelta(days=LOAN_DAYS)
self._loans[name][title] = due
self._last_receipt = f"Title: {title}\nDue: {due.isoformat()}"
return {"status": "lent", "reason": None, "due_date": due}

@keyword
def return_book(self, name: str, title: str) -> Decimal:
"""Returns a loan on today's date. Returns the fine charged (EUR)."""
due = self._loans[name].pop(title)
days_late = max((self._today - due).days, 0)
fine = FINE_PER_DAY * days_late
self._members[name]["fines"] += fine
queue = self._reservations.get(title, [])
if queue:
self._held_for.setdefault(title, []).append(queue.pop(0))
else:
self._copies[title] = self._copies.get(title, 0) + 1
return fine

@keyword
def reserve(self, name: str, title: str) -> int:
"""Reserves ``title``; returns the member's position in the queue (1-based)."""
if self._copies.get(title, 0) > 0:
raise ValueError(f"'{title}' is available; reserve only unavailable books")
queue = self._reservations.setdefault(title, [])
queue.append(name)
return len(queue)

@keyword
def get_loans(self, name: str) -> list[str]:
"""Returns the titles ``name`` currently has on loan, sorted."""
return sorted(self._loans[name])

@keyword
def get_fines(self, name: str) -> Decimal:
"""Returns the unpaid fines of ``name`` in EUR."""
return self._members[name]["fines"]

@keyword
def get_available_copies(self, title: str) -> int:
"""Returns how many copies of ``title`` are on the shelf."""
return self._copies.get(title, 0)

@keyword
def get_last_receipt(self) -> str:
"""Returns the text of the last loan receipt."""
return self._last_receipt

@staticmethod
def _refused(reason: str) -> dict:
return {"status": "refused", "reason": reason, "due_date": None}
Empty file.
9 changes: 9 additions & 0 deletions eval/fixtures/sut-bdd-library/review/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
output.xml
log.html
report.html
results/
.venv/
__pycache__/
*.pyc
.pytest_cache/
.robotcode_cache/
22 changes: 22 additions & 0 deletions eval/fixtures/sut-bdd-library/review/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# library-lending

Acceptance tests for the city library's lending service. The service runs in
memory (`sut/LibraryService.py`, a Robot Framework library with one fresh
instance per test); its business rules are in the module docstring. The
library's calendar starts on 2026-11-12, so due dates are fixed (`Set Today`
moves it).

- `tests/loans.robot`: loan tests.
- `tests/loan_steps.resource`: the keywords the loan tests use.
- `resources/`: shared keywords (none yet).
- `smoke/library_service.robot`: a technical check that the service library
loads and lends a book.

`robot.toml` puts `resources/` and `sut/` on the python-path, so both are
imported by name. Run from this directory with robotcode, which reads
`robot.toml`:

```bash
robotcode robot # the tests in tests/
robotcode robot smoke
```
15 changes: 15 additions & 0 deletions eval/fixtures/sut-bdd-library/review/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[project]
name = "library-lending"
version = "0.0.1"
description = "Acceptance tests for the city library's lending service."
requires-python = ">=3.10"
dependencies = [
"robotframework>=7.3",
]

[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"

[tool.setuptools]
packages = []
Empty file.
3 changes: 3 additions & 0 deletions eval/fixtures/sut-bdd-library/review/robot.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
paths = ["tests"]
output-dir = "results"
python-path = ["resources", "sut"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
*** Settings ***
Documentation Technical smoke check: the service library loads and lends a book.
Library LibraryService


*** Test Cases ***
Service lends an available copy
Register Member Smoke Reader
Add Book Smoke Test Title 1
${loan} Borrow Smoke Reader Smoke Test Title
Should Be Equal ${loan}[status] lent
Should Be Equal As Strings ${loan}[due_date] 2026-12-03
Loading
Loading