Metadata-Version: 2.4
Name: pentester_test
Version: 0.0.1
Summary: Pentester CLI
Author-email: Tekdatum <tools@tekdatum.com>
License-Expression: Apache-2.0
Classifier: Programming Language :: Python :: 3
Classifier: Operating System :: OS Independent
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: pydantic-settings>=2.0
Requires-Dist: tqdm>=4.0
Requires-Dist: click>=8.0
Requires-Dist: pyrit>=0.12.1
Requires-Dist: requests>=2.32
Requires-Dist: uncurl>=0.0.11
Requires-Dist: Mako>=1.3
Requires-Dist: fpdf2>=2.7
Requires-Dist: pandas>=2.0
Requires-Dist: python-dotenv>=1.2
Requires-Dist: promptfoo>=0.1.3
Requires-Dist: PyYAML>=6.0.3
Requires-Dist: types-PyYAML>=6.0.12.20250915
Requires-Dist: inspect_ai>=0.3.189
Requires-Dist: inspect_evals>=0.5.0
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# Pentester CLI

Pentester CLI is a command-line tool for running automated prompt-injection security audits against AI systems. It fires a configurable curl command at a target, injects attack prompts, evaluates whether the target was bypassed, and generates reports in multiple formats.

---

## Installation

Install from PyPI:

```
pip install pentester
```

Or install directly from the repository:

```
pip install git+https://github.com/tekdatum/pen-tester-cli.git
```

To include development tools (linter, type checker, test suite):

```
pip install "pentester[dev]"
```

---

## Usage

```
pentester [OPTIONS]
```

All options are optional. Defaults load from environment variables or a `.env` / `.env.local` file. Reports are written to `./output/` by default (override with `--output-dir-path`).

| Option | Description | Default |
|---|---|---|
| `--curl-command` | curl command used to probe the target; must include `$PROMPT` | `None` |
| `--json-dot-target` | Dot-notation path to the response field that indicates bypass (e.g. `body.valid`) | `None` |
| `--output-dir-path` | Directory where report files are written | `./output/` |
| `--generator-keys` | Comma-separated list of report formats: `pdf`, `csv`, `html`, `markdown` | all four |
| `--target-type` | Category of the target: `LLM` or `SEMANTIC_FENCE` | `SEMANTIC_FENCE` |
| `--auditors` | Comma-separated list of auditors to run: `garak`, `pyrit`, `inspect_ai`, `promptfoo` | all |

---

## How To

### 1. Basic usage

Run a scan by providing a curl command pointing at your target:

The tool follows standard curl syntax. The curl command **must** include the `$PROMPT` placeholder, which is replaced with each attack prompt at scan time.

The response can be parsed using dot-notation to extract the field that indicates whether an attack bypassed the target. Both `body.*` and `headers.*` paths are supported.


```
pentester --json-dot-target "body.valid" --curl-command "curl -X POST 'https://api.example.com/chat' -H 'Content-Type: application/json' --data-raw '{\"text\": \"$PROMPT\"}'"
```

### 2. Specify report formats

Use `--generator-keys` with a comma-separated list of formats:

```
pentester --generator-keys html,pdf --json-dot-target "body.valid" --curl-command "curl -X POST 'https://api.example.com/chat' -H 'Content-Type: application/json' --data-raw '{\"text\": \"$PROMPT\"}'"
```

### 3. Specify target type

The tool supports LLMs and Semantic Fences (`LLM` | `SEMANTIC_FENCE`). For semantic fences, the tool cannot rely on an auditor's built-in judge, so `--curl-command` and `--json-dot-target` must be provided to fetch and parse the response directly.

```
pentester \
  --target-type LLM \
  --curl-command "curl -X POST 'https://api.example.com/chat' -H 'Content-Type: application/json' --data-raw '{\"text\": \"$PROMPT\"}'"
```

### 4. Specify auditors

Use `--auditors` with a comma-separated list of auditor names to run only a subset:

```
pentester --auditors garak,pyrit --json-dot-target "body.valid" --curl-command "curl -X POST 'https://api.example.com/chat' -H 'Content-Type: application/json' --data-raw '{\"text\": \"$PROMPT\"}'"
```

### 5. Limit the number of attacks

Use `PENTESTER_MAX_ATTACKS` to cap the number of attack prompts each auditor will run. This is useful for quick smoke tests or when you want to control cost/time without configuring every auditor individually.

```
PENTESTER_MAX_ATTACKS=50 pentester --json-dot-target "body.valid" --curl-command "curl -X POST 'https://api.example.com/chat' -H 'Content-Type: application/json' --data-raw '{\"text\": $PROMPT}'"
```

Or in a `.env` file:

```
PENTESTER_MAX_ATTACKS=50
```

Advanced users can override the limit per auditor — the per-auditor setting always takes priority:

```
PENTESTER_MAX_ATTACKS=50
PENTESTER_GARAK__MAX_ATTACKS=200
```

In this example Garak runs up to 200 attacks while every other auditor is capped at 50.

### 6. Use the Orchestrator in your own code

You can drive scans programmatically by constructing a `PentesterSettings` object and passing it to `Orchestrator`:

```python
from pentester.config.settings import PentesterSettings
from pentester.enums.target_type import TargetType
from pentester.orchestrator import Orchestrator

settings = PentesterSettings()
settings.target_type = TargetType.SEMANTIC_FENCE
settings.scanner.curl_command = "curl -X POST 'https://api.example.com/chat' -H 'Content-Type: application/json' --data-raw '{\"text\": \"$PROMPT\"}'"
settings.scanner.json_dot_target = "body.valid"
settings.reporting.output_dir_path = "./my-reports"
settings.reporting.generator_keys = "html,pdf"

Orchestrator(settings).execute()
```

---
