Metadata-Version: 2.4
Name: kwic
Version: 0.2.1
Summary: Key Word In Context
Author: Eva
Author-email: Eva <eva.evangelisti03@gmail.com>
License-Expression: MIT
License-File: LICENSE
Requires-Dist: spacy>=3.8.15
Requires-Dist: cupy-cuda12x>=13 ; extra == 'cuda12x'
Requires-Dist: cupy-cuda13x>=14 ; extra == 'cuda13x'
Requires-Dist: lemminflect>=0.2.3 ; extra == 'lemminflect'
Requires-Dist: stanza>=1.14.0 ; extra == 'stanza'
Requires-Python: >=3.12
Project-URL: Repository, https://github.com/evaevangelisti/kwic
Provides-Extra: cuda12x
Provides-Extra: cuda13x
Provides-Extra: lemminflect
Provides-Extra: stanza
Description-Content-Type: text/markdown

# Key Word In Context

Locates the lemmas of a query in the context that attests them.

<!-- installation -->

## Installation

### Requirements

- Python 3.12 or later

### From PyPI

```sh
pip install kwic
```

A search reads with spaCy, whose pipelines are published apart from the library:

```sh
python -m spacy download en_core_web_sm
```

The other engines are extras:

```sh
pip install "kwic[stanza]"
pip install "kwic[lemminflect]"
```

Reading on the graphics card needs CuPy, whose wheel is named after the CUDA release `nvidia-smi` reports:

```sh
pip install "kwic[cuda12x]"
pip install "kwic[cuda13x]"
```

<!-- usage -->

## Usage

A search takes one context and the lemmas to look for.

```python
from kwic import Locator, POS, Query

locator = Locator()

locator.find("She found the keys she had lost.", [Query("find", POS.VERB)])
# (Match(lemma='find', pos=POS.VERB, form='found', word_index=1, offsets=(4, 9)),)
```

### Query

| Field | Default | |
| --- | --- | --- |
| `lemma` | | Dictionary form to look for, in whatever case |
| `pos` | `None` | Tag a one-word occurrence must carry; a longer lemma is not narrowed |
| `forms` | `()` | How else the lemma is written, taken where the engine read another |

The engine cuts the lemma into words, and a space and a hyphen are one to it.

```python
locator.find("It was hunky-dory.", [Query("hunky dory")])
# (Match(lemma='hunky dory', pos=POS.ADJ, form='hunky-dory', word_index=2, offsets=(7, 17)),)
```

A phrasal verb written apart runs from the verb to the particle.

```python
locator.find("She gave the money up.", [Query("give up")])
# (Match(lemma='give up', pos=POS.VERB, form='gave the money up', word_index=1, offsets=(4, 21)),)
```

### Match

| Field | |
| --- | --- |
| `lemma` | The lemma you asked for, as you wrote it |
| `pos` | The tag it carries |
| `form` | How it is written |
| `word_index` | Where it opens among the words, from zero |
| `offsets` | Where it falls in the text, half-open and in code points |

### Contexts

A context is a text, or the words it was split into. The second has no range.

```python
locator.find(("She", "found", "the", "keys"), [Query("find")])
# (Match(lemma='find', pos=POS.VERB, form='found', word_index=1, offsets=None),)
```

### Many contexts

`find_all` takes context and lemmas in pairs, and reads a batch at a time.

```python
for occurrences in locator.find_all(searches):
    ...
```

### Engines

An engine reads the context; the search reads the engine.

```python
from kwic.engines.stanza import StanzaEngine

locator = Locator(StanzaEngine())
```

| Engine | Reads with | Install |
| --- | --- | --- |
| `SpacyEngine` | a spaCy pipeline, its English lemmatiser rules over the tag | `spacy` |
| `StanzaEngine` | Stanza, a dictionary with a neural model behind it | `kwic[stanza]` |
| `LemmInflectEngine` | spaCy for the tags, LemmInflect for the lemmas | `kwic[lemminflect]` |

An extra is imported from the module wrapping it, so a package without it still loads.

`SpacyEngine` takes the pipeline to load, `StanzaEngine` the language. Both parse unless told otherwise:

| Parser | Buys | Costs |
| --- | --- | --- |
| spaCy | phrasal verbs apart, and several universal tags | a tenth of a reading |
| Stanza | phrasal verbs apart | half its speed |

```python
Locator(SpacyEngine(parse=False))
```

Either engine reads on the graphics card when told to, which needs the CuPy extra above:

```python
Locator(SpacyEngine("en_core_web_trf", gpu=True))
```

| Asked for | Gets | Wants |
| --- | --- | --- |
| `gpu=True` | a card, or an error where none answers | one process, a large batch |
| `gpu=False` | the processor | a small batch, padding being what it costs |

<!-- accuracy -->

## Accuracy

English-EWT test section: 2,077 sentences, 417 lemmas, 3,619 occurrences.

| Engine | Precision | Recall | F1 |
| --- | --- | --- | --- |
| `StanzaEngine` | 0.975 | 0.963 | 0.969 |
| `SpacyEngine`, `en_core_web_trf` | 0.981 | 0.905 | 0.942 |
| `SpacyEngine`, `en_core_web_lg` | 0.978 | 0.891 | 0.933 |
| `SpacyEngine`, `en_core_web_sm` | 0.978 | 0.882 | 0.928 |
| `LemmInflectEngine` | 0.976 | 0.872 | 0.921 |
