Metadata-Version: 2.4
Name: speakfill
Version: 1.0.1
Summary: Voice-dictated form filling — build a form, then fill it out by talking, transcribed locally by faster-whisper.
Keywords: speech-to-text,dictation,forms,whisper,voice
Author-email: mohammad.pezeshki1994@gmail.com
License-Expression: MIT
Classifier: Environment :: Web Environment
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: End Users/Desktop
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Dist: faster-whisper>=1.1.0
Requires-Dist: fastapi>=0.139.0
Requires-Dist: numpy>=2.0.0
Requires-Dist: openai>=2.45.0
Requires-Dist: pydantic>=2.13.4
Requires-Dist: pydantic-settings>=2.7.0
Requires-Dist: python-dotenv>=1.2.2
Requires-Dist: sqlalchemy>=2.0.51
Requires-Dist: uvicorn>=0.49.0
Requires-Dist: websockets>=16.0
Requires-Dist: accelerate>=1.14.0 ; extra == 'local-llm'
Requires-Dist: torch>=2.12.1 ; extra == 'local-llm'
Requires-Dist: transformers>=5.13.0 ; extra == 'local-llm'
Requires-Dist: psycopg[binary]>=3.3.4 ; extra == 'postgres'
Requires-Python: >=3.12
Project-URL: Homepage, https://github.com/mohpezeshki/speak-fill
Project-URL: Repository, https://github.com/mohpezeshki/speak-fill
Project-URL: Issues, https://github.com/mohpezeshki/speak-fill/issues
Provides-Extra: local-llm
Provides-Extra: postgres
Description-Content-Type: text/markdown

# Speak Fill

Voice-dictated form filling — build a form, then fill it out by talking. Speech is transcribed locally and in real time by [faster-whisper](https://github.com/SYSTRAN/faster-whisper) (no cloud STT service), and an LLM maps what you said onto the right form fields.

It's a local, single-user tool: no accounts, no login, no cloud dependency beyond an optional LLM call.

## Install

```bash
pip install speakfill
```

## Set your API key

Whole-form dictation (see "How it works" below) uses OpenAI (`gpt-4o-mini`) to map your transcript onto form fields, so it needs `OPENAI_API_KEY`. Set it one of three ways — checked in this order, first match wins:

1. **A real environment variable** — simplest for one-off use:
   ```bash
   export OPENAI_API_KEY=sk-...
   speakfill
   ```
2. **A `.env` file in your project directory** (or any parent of it — same lookup `git` uses):
   ```bash
   echo "OPENAI_API_KEY=sk-..." > .env
   speakfill
   ```
3. **A `.env` file at `~/.speakfill/.env`** — set it once, then run `speakfill` from anywhere without repeating it:
   ```bash
   mkdir -p ~/.speakfill
   echo "OPENAI_API_KEY=sk-..." > ~/.speakfill/.env
   speakfill    # works from any directory now
   ```

No key at all? Per-field dictation still works fully (it never calls an LLM), and speech-to-text itself never needs an API key either way — only whole-form dictation's field-mapping step does. Or skip OpenAI entirely with `LLM_PROVIDER=local` (see Extras below).

## Run it

```bash
speakfill
```

```
Speak Fill starting at http://0.0.0.0:8000
INFO:     Uvicorn running on http://0.0.0.0:8000
```

Open **http://localhost:8000**. One process serves both the API and the UI — no Docker, no separate frontend server. Data is stored locally in SQLite at `~/.speakfill/speakfill.db`, created automatically on first run.

## Tutorial: your first form

1. Open http://localhost:8000 — you land on **Build**, a drag-and-drop form builder.
2. Add a few fields (e.g. a text input for "Name", a number input for "Age", a dropdown for "Department"). Save the form.
3. Switch to **My Forms**, find the form you just saved, and open it to fill it out.
4. Fill it by voice, in one of two modes:
   - **Per-field dictation** — click the mic on a single field, speak, the transcript becomes that field's value directly. No API key needed.
   - **Whole-form dictation** — click the mic on the whole-form recorder, speak freely covering everything the form asks for (e.g. *"My name is Alex, I'm 30, and I work in Engineering"*), stop recording, and the LLM maps what you said onto the right fields in one shot. Needs `OPENAI_API_KEY` (or `LLM_PROVIDER=local`).
5. Submit. Switch to **Submissions** to see everything you've filled out so far.

## Extras

- `pip install speakfill[postgres]` — use Postgres instead of the default SQLite (`DATABASE_URL=postgresql+psycopg://...`)
- `pip install speakfill[local-llm]` — run whole-form field-mapping locally instead of via OpenAI (`LLM_PROVIDER=local`, `LLM_LOCAL_MODEL=Qwen/Qwen2.5-7B-Instruct` by default); pulls in `torch`/`transformers`, a multi-GB install, so it's opt-in

## Using it inside your own project

`speakfill` isn't only a standalone app — it's a normal importable package, so you can pull pieces of it into an existing FastAPI project instead of running `speakfill` as its own process.

**Mount the whole app under a prefix**, API + bundled UI together:

```python
from fastapi import FastAPI
from speakfill import app as speakfill_app

app = FastAPI()
app.mount("/speakfill", speakfill_app)
```

Everything comes with it at `/speakfill/...` — forms API, dictation WebSocket, and the UI (`Base.metadata.create_all()` and the faster-whisper warm-up run automatically, same as standalone). One thing to know: `speakfill_app` has `CORSMiddleware(allow_origins=["*"])` baked in — fine for most embeds, but if you need different CORS behavior, use the next pattern instead.

**Or cherry-pick just what you need** — e.g. only the forms API, none of our CORS/UI/other routes:

```python
from fastapi import FastAPI
from speakfill.api.forms.routes import app as forms_router
from speakfill.database.session import Base, engine

Base.metadata.create_all(bind=engine)  # creates the template_forms/filled_forms tables

app = FastAPI()  # your own app, your own CORS/middleware/other routes
app.include_router(forms_router)
```

Speech-to-text and field-mapping work as plain library calls too, independent of any web framework or the forms API above — `speakfill.stt.whisper_engine.transcribe_pcm16(audio_bytes)` and `speakfill.llm.llm_client.LLMClient().fill_form(form_schema, transcript)` — if you just want the transcription/mapping logic with no HTTP layer at all.

Note that every piece — the DB engine, faster-whisper model, LLM client — reads its config from the same `Settings` object (`speakfill.config.settings`), so it follows whatever `OPENAI_API_KEY`/`DATABASE_URL`/etc. your host process already has set, the same way as standalone.

### Reading/writing data without any web layer at all

For the common case of "I just want the data as Python objects" — no FastAPI, no HTTP, no `Session` to manage — `speakfill` re-exports a handful of plain functions that open/close their own database session per call:

```python
import speakfill

speakfill.list_forms()                # -> [{"id", "title", "json_data" (its fields), "created_at", "updated_at"}, ...]
speakfill.get_form(form_id)            # -> dict, or None if it doesn't exist
speakfill.list_filled_forms()          # same shape, for submissions
speakfill.get_filled_form(filled_id)

speakfill.transcribe(audio_bytes)      # 16kHz mono PCM16 -> str (one-shot; for live/streaming
                                        # dictation, use speakfill.stt.session.WhisperSession instead)
speakfill.map_transcript(form_schema, transcript)  # -> {field_key: value}, same call Mode B uses
```

`form_schema` is the `json_data["fields"]` list from a form returned by `get_form`/`list_forms` — the same field definitions (`element`/`field_name`/`label`/`options`/...) `react-form-builder2` produces, unchanged, so it doubles as documentation of what a "form" actually contains if you're building something on top of it.

## Customizing the UI

The bundled frontend is a normal static build (`index.html` + assets) — point `SPEAKFILL_STATIC_DIR` at a different one to serve that instead:

```bash
SPEAKFILL_STATIC_DIR=/path/to/your/build speakfill
```

Setting it to a path that doesn't exist fails at startup rather than silently falling back, so a typo doesn't quietly serve the wrong UI. This is the mechanism for both "reskin it" (fork the frontend, restyle `theme.css`'s CSS custom properties, rebuild) and "add new form field types" below — build your version, point `speakfill` at it, done. The API itself needs no changes either way (it's a plain REST + WebSocket API, wide-open CORS by default — see `ARCHITECTURE.md`).

### Adding custom form field types

New field types (like the built-in `MicTextInput`/`MicTextArea`) plug into the drag-and-drop builder (`react-form-builder2`) two ways, both in the frontend source (`frontend/src/form-elements/`):

1. **Register the component** so saved forms know how to render it:
   ```ts
   import { Registry } from 'react-form-builder2';
   import { MyField } from './MyField';

   Registry.register('MyField', MyField);
   ```
2. **Add it to the builder's toolbar** (`frontend/src/FormBuilderPage.tsx`) so it's actually draggable onto the canvas:
   ```ts
   {
     key: 'MyField',
     element: 'CustomElement',
     component: MyField,
     type: 'custom',
     forwardRef: true,
     field_name: 'my_field_',
     name: 'My Field',       // shown in the builder's palette
     icon: 'fa fa-star',
   }
   ```

Your component receives the same props any `react-form-builder2` `CustomElement` gets: `name`, `defaultValue`, `disabled`, and the raw field `data`. See `frontend/src/form-elements/MicFields.tsx` for a complete working example (it also shows how a field hooks into the dictation system, if that's relevant to what you're building — most custom fields won't need any of that).

Then rebuild (`./scripts/build_frontend.sh` from the repo, or your own `npm run build`) and point `SPEAKFILL_STATIC_DIR` at the output.

## Other settings

All optional, set the same way as `OPENAI_API_KEY` above:

| Variable | Default | What it does |
|---|---|---|
| `LLM_PROVIDER` | `openai` | `openai` or `local` |
| `LLM_OPENAI_MODEL` | `gpt-4o-mini` | which OpenAI model does field-mapping |
| `STT_MODEL_SIZE` | `small` | faster-whisper model size, e.g. `medium`, `large-v3` |
| `STT_DEVICE` | `auto` | `auto` picks CUDA if available, else CPU |
| `DATABASE_URL` | SQLite at `~/.speakfill/speakfill.db` | set to a Postgres URL to use that instead |
| `SPEAKFILL_HOST` / `SPEAKFILL_PORT` | `0.0.0.0` / `8000` | where the server binds |
| `SPEAKFILL_STATIC_DIR` | the bundled frontend | serve a different built frontend instead (see Customizing the UI) |

## Links

Full documentation, architecture notes, and source: https://github.com/mohpezeshki/speak-fill
