Metadata-Version: 2.4
Name: cit-course-tools
Version: 0.3.2
Summary: Terminal workbench, model simulator, evidence, and lab-testing tools for CIT classes
Author: Jeffrey Myers II, M.S.
Keywords: CIT,course tools,education,AI harness,simulation,testing
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Education
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Education
Classifier: Topic :: System :: Systems Administration
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: jsonschema<5,>=4.23
Requires-Dist: packaging>=24
Requires-Dist: PyYAML<7,>=6.0.2
Requires-Dist: textual<9,>=8.2
Provides-Extra: browser
Requires-Dist: playwright<2,>=1.62; extra == "browser"
Provides-Extra: terminal
Requires-Dist: pywinpty<4,>=3.0.5; sys_platform == "win32" and extra == "terminal"

# CIT Course Tools

`cit-course-tools` gives students one small terminal interface for preparing a
course workspace, exploring deterministic model scenarios, testing cumulative
Python harnesses, and producing consistent lab evidence. Its Textual TUI offers
the guided everyday workflow, while seven explicit CLI commands remain available
for transparency, scripting, and troubleshooting. The package is
course-agnostic: instructor-provided YAML defines what each lab teaches and
tests.

The goal is a simple student experience with rigorous behavior underneath.
The package manages ports, simulator sessions, graph traversal, evidence,
fractional scoring, and current-result reports. It does not download course
scenario files or AI models, and it does not install or manage model servers.

## VS Code recording integration (0.3.2)

The companion `cit-course-vscode` extension records scoped project and selected-lab text/code edits, including evaluations,
locally and provides a chronological single-pane replay with dockable recording controls. It does not upload
recordings or detect which AI provider produced text. The Python integration
includes only the selected lab's recording segments and `editing-history/summary.md` in named submission
ZIPs, separate from scored lab evidence. Missing history is explicitly marked;
malformed history fails packaging without replacing an existing ZIP.

Version 0.3.2 uses `labs/labNN/logs/recording.jsonl`, exported as
`editing-history/recording.jsonl` with lab identity in its metadata. Old prefixed
logs remain readable; identical migration prefixes are deduplicated, while
conflicting histories stop packaging. Generated logs/results, archives, hidden
files and dependencies are excluded from recording scope. Editor-event origin
is unknown; disk changes and paused work are unobserved snapshots, not typing.
**Compatibility:** this update requires a companion extension supporting
recording-layout 3. Extension 0.1.14 uses layout 2 and will reject the new editor
handshake. Upgrade both companions together. If the matching extension update
is not available yet, keep the existing Python 0.3.1 / extension 0.1.14 pair
until it is; the explicit Python CLI can also be used independently.

The extension delegates initialization and packaging to this Python package.
A versioned editor handshake shares the TUI's active-lab preference. New logs
support cursor/selection positions and pause/resume boundaries; older unassigned
logs are preserved but excluded from lab ZIPs with a warning. `package --json`
returns the archive path and warnings for the dockable panel.

Install `python -m pip install --upgrade "cit-course-tools[terminal]==0.3.2"`
for the stable VS Code integration, including Windows embedded-terminal support.
Use the matching CIT Course Tools extension published by `jeffm-courseware`.
Check its recording-layout compatibility before deployment.
Recording source edits means submissions contain implementation history; disclose
that requirement before assigning recorded work. No grading deductions are automatic.

## Install

Python 3.10 or newer is required.

```console
python -m pip install cit-course-tools
```

To update later in the semester:

```console
python -m pip install --upgrade cit-course-tools
```

## Start a course workspace

Navigate to the directory you want to use and initialize it once:

```console
cit-course init .
```

The command creates the submitted orientation lab `labs/lab00` and the regular
`labs/lab01` through `labs/lab12`. Each lab directory holds its YAML,
evaluation, runtime logs, current test results, and—when generated—its
`labNN-student-name-submission.zip`. The command preserves student work.

Initialization includes a compatibility migration for older workspaces.
During `init`, a non-empty legacy `labs/labNN/submission/`
directory is first copied into a verified `labNN-submission.zip`; its evaluation
is preserved under `labs/labNN/evaluation.md`; only then is the old directory
removed. An empty legacy directory is removed directly.

Cleanup is best-effort: if Windows denies removal or another cleanup error
occurs, initialization prints a warning with the affected path and continues.
Verified archives/destination files and any remaining old files are retained;
the tool does not change permissions. Close applications using the old folder
and rerun `cit-course init .` to retry. An existing legacy ZIP is reused only
when every remaining source file matches its archived contents. Conflicting
files, unreadable source data, failed verification, and required write failures
still stop initialization so they cannot be mistaken for a successful migration.

Lab 00's package-managed `interaction-guide.yaml` is the canonical CLI
orientation and test suite; `interaction-guide-tui.yaml` drives the TUI
Workbench tour. Download each later lab's YAML and starter files from the course
site and place them where its instructions specify.

The scaffold creates `project/SIM_AGENTS.md`, a stable, course-agnostic Markdown
instruction fixture for simulator tests. Your instructor supplies
`project_harness.py` with the first lab that requires it. Tests then select that
Python file automatically; it must export `run_harness(context)`.

After initialization, start with:

```console
cit-course specs
```

Review the resulting `specs/hardware_specs.txt` before sharing it.

## Open the student TUI

From anywhere inside the initialized workspace, run:

```console
cit-course
```

The base TUI provides a dashboard, lab selection, the always-available
Interaction Guide, single-terminal Guided Interaction, isolated lab tests,
evidence and specs views, and validated submission packaging. Model Simulator
activity remains separate from User Simulator prompts and responses.

Use `cit-course --help` to see the explicit CLI. The TUI calls the same
workspace, simulator, test, evidence, specs, and packaging services; it is not a
parallel implementation.

## Learn the simulator

The recommended path is to run `cit-course`, select **Lab 00 — CIT Course
Tools**, and start Guided Interaction. The Workbench selects the TUI-specific
guide while Tests retains the canonical Lab 00 suite. The TUI runs the separate Model and User Simulator
services together, displays their turns in one chronological conversation, and
keeps Model Simulator activity available as a separate alternate view.

The transparent two-terminal CLI path remains available. Start the Model
Simulator with `labs/lab00/interaction-guide.yaml`; the bare `serve` command
selects it automatically:

```console
cit-course serve
```

Leave it running, then start the User Simulator in a second terminal:

```console
cit-course run
```

The guide introduces the package and its simulator workflow without requiring
a download. Explore its paths because the submitted Lab 00 evaluation asks
questions answered by the guide.

When finished, run the Lab 00 suite and generate its submission files:

```console
cit-course test
```

After completing `labs/lab00/evaluation.md`, create its upload-ready ZIP:

```console
cit-course package lab00
```

## Explore a lab scenario

To explore an instructor-supplied lab instead, start its YAML file explicitly:

```console
cit-course serve labs/lab01/lab01.yaml
```

Then run the User Simulator in a second terminal anywhere inside the same
initialized workspace:

```console
cit-course run
```

The package discovers the active lab automatically. The `serve` terminal shows
model-service activity; the `run` terminal stays focused on predefined user
prompts and Model Simulator responses. The Model Simulator uses port 8081 and
the User Simulator uses port 8082 by default.

## Run a lab test suite

Bare `cit-course test` selects Lab 00. Instructor-supplied labs use an explicit
YAML path. When the YAML requires a harness, the default project harness is
selected automatically:

```console
cit-course test labs/lab01/lab01.yaml
```

The Python file must export one function:

```python
def run_harness(context):
    """Run one complete simulator-test or live-model interaction session."""
```

The harness should write persistent runtime artifacts beneath
`context.artifacts_directory`. The context also identifies the active Markdown
instruction file as `context.instructions_path`; a harness can load it and send
its complete text as a system message. When a YAML step declares
`conversation_log: true`, the runner checks records newly appended to the lab's
working `conversation.jsonl`. The JSONL records identify requests, responses,
timestamps, the model, status, and errors without recording credentials.

An alternate Python filename can be selected explicitly with `--harness`.
The full suite runs by default. During implementation, rerun only failed,
errored, and incomplete steps with:

```console
cit-course test labs/lab03/lab03.yaml \
  --continue
```

For an alternate harness file:

```console
cit-course test labs/lab03/lab03.yaml --harness project/my_harness.py
```

Continued and single-case runs are labeled development results: carried passes
are earlier observations, not proof that changed code still passes. Before
packaging, run the full suite after your last source change. A full run with
graded failures can be submitted for partial credit; infrastructure-incomplete
or interrupted runs cannot. Malformed historical JSONL is preserved in
`.cit-course/recovered-logs/`, allowing a corrected logger to start cleanly.

Each selectable prompt-response step is worth no more than one point. If three
of five required checks pass, that step earns `0.60 / 1.00`. Infrastructure
errors are reported clearly and excluded from grading.

### Browser labs

Labs with a browser client can add package-managed headless checks to the same
YAML suite. Install the optional support once before the first browser lab:

```console
python -m pip install --upgrade "cit-course-tools[browser]"
cit-course browser install
cit-course browser status
```

The package starts an isolated loopback-only web server and deterministic mock
harness for each browser case. Students do not need to run a model, SearXNG, or
their harness API during these checks. YAML can use only documented browser
actions and assertions; it cannot execute arbitrary JavaScript or shell
commands.

Each browser case is worth at most one point, with its assertions sharing that
point equally. Missing Playwright or Chromium is an ungraded infrastructure
error. A failed case also records a screenshot and Playwright trace under the
lab's `results/browser-failures/` directory and links them from the report.

Every run replaces the current generated report and evidence without touching
the evaluation or another lab:

```text
labs/lab03/
├── evaluation.md
├── logs/
│   └── conversation.jsonl  # when required by the YAML
└── results/
    ├── evidence.jsonl
    └── test-summary.md
```

`evaluation.md` is created during initialization and never overwritten.
Students complete it in their own words. The evidence trace and test summary
are regenerated by the package.

## Package a lab for submission

When the evaluation and current test results are ready, run:

```console
cit-course package lab03 --student-name "Jane Doe"
```

The command gathers only the required, allow-listed files and atomically
creates or replaces:

```text
labs/lab03/lab03-jane-doe-submission.zip
```

The student's name is required. Omitting `--student-name` in an interactive
terminal prompts for it; scripts must supply the flag. The TUI asks you to
confirm your name before creating the ZIP. Use your name as shown in Blackboard.
The full name is stored in the manifest; the filename uses a shortened,
filename-safe form. This is a student-provided label, not identity verification.
No student ID, account username, or email is collected automatically.

Repackaging the same lab with the same name replaces its ZIP. If you correct
your name, older differently named ZIPs are preserved; submit the new file
shown by the tool. The name is not added to model requests or grading evidence.

The ZIP includes a generated `submission-manifest.json` with the student name, test identity and
file hashes. It never includes the YAML, harness, unrelated logs, an older ZIP,
or the rest of the workspace, and it does not upload anything.

The command checks the YAML, harness and other project source hashes, plus
generated report/evidence/log hashes, against the completed full run. You may
edit `evaluation.md` afterward. Extra logs or metrics are collected only when
the YAML names them in `test_suite.submission_artifacts`; paths are relative to
`labs/labNN/logs/` and must be produced or updated during that run.

## Connect the project harness to a live model

The same cumulative `run_harness(context)` implementation can run as an
ordinary terminal application against any student-supplied OpenAI-compatible
model endpoint. Start that service independently, note its root URL, and then
run:

```console
python project/project_harness.py
```

The standalone adapter discovers the served model through `/v1/models`, reads
user prompts from the terminal, and supplies empty simulator-specific headers.
It uses `project/AGENTS.md` when that optional student-authored file exists and
otherwise falls back to `project/SIM_AGENTS.md`. Enter `q` at the User Prompt
to stop. An alternate server root, advertised model ID, or instruction file can
be selected explicitly:

```console
python project/project_harness.py --model-url http://127.0.0.1:9000
python project/project_harness.py --model my-model-alias
python project/project_harness.py --instructions project/AGENTS.md
```

Run `python project/project_harness.py --help` for all standalone options. The
student-authored core remains the same: tests import `run_harness(context)` and
supply deterministic User Simulator prompts, while direct execution supplies
terminal prompts and the supplied live endpoint.

The TUI Workbench also provides **Connect Live Model** for direct free-text
experimentation. Enter the root URL of an already-running OpenAI-compatible
service and optionally its model ID. Course Tools discovers the model through
`/v1/models` and sends chat requests, but never starts, stops, downloads, or
configures the external service.

## Hardware report

Create or refresh the optional, privacy-conscious hardware report from anywhere
inside the workspace:

```console
cit-course specs
```

It is written to `specs/hardware_specs.txt`. Review it before sharing it.

## Commands

```text
cit-course init     Create or repair the course workspace
cit-course specs    Generate the optional hardware report
cit-course serve    Start the Model Simulator with the guide or a lab YAML file
cit-course run      Start the User Simulator for the active lab
cit-course test     Run Lab 00 or an explicit lab suite and replace its report
cit-course package  Create one lab's upload-ready ZIP from required current files
```

Run `cit-course --help` or `cit-course COMMAND --help` for options.

## Instructor and developer references

### Recovering from operation failures

Warnings identify optional actions that could not complete; required operations
report errors and do not claim success. Messages include the action, available
file/connection details, and a suggested next step. The TUI displays the same
recovery guidance without closing the entire application for a handled failure.

- Close applications holding a report or submission ZIP before retrying a denied
  write. Existing reports/ZIPs are replaced atomically, not truncated first.
- If a simulator port is occupied, stop your existing simulator or choose another
  port with `--port`. Do not terminate unfamiliar processes or disable security tools.
- Invalid YAML, missing inputs, and invalid port/timeout values must be corrected
  before the operation can produce a valid result.
- Test infrastructure errors are marked separately from student check failures.
  A run with infrastructure errors is not ready for submission.
- Optional browser diagnostic captures and cleanup failures are warnings in the
  test summary; they do not change assertion scores. Only successfully captured
  screenshot/trace files are listed as current failure artifacts.
- `init --with-specs` may succeed with a warning if the optional hardware report
  cannot be saved. Retry the report separately with `cit-course specs`.

### Architecture and publishing

- `TECHNICAL_SUMMARY.md` provides a concise comprehensive architecture and
  workflow summary.
- `SCENARIO_AUTHORING.md` documents the YAML authoring and scoring contract.
- `TECHNICAL_SPECIFICATION.md` documents components, isolation, evidence, and
  runtime behavior.
- `PUBLISHING.md` contains the release checklist.

Run the complete regression suite with:

```console
python -m unittest discover -s tests -v
```
