Multilingual NLP Toolkit

A collection of focused, dependency-light NLP components for morphologically rich languages, built and battle-tested in production experiments.

Module	What it does	Dependencies
mnlp.turkish	Bidirectional rule-based Turkish morphology: analysis and generation with full vowel harmony, consonant mutation, and suffix stacking	none (stdlib)
mnlp.grammar	Compound tense / mood / voice detection for German and English (auxiliary-chain analysis on dependency parses)	spaCy
mnlp.alignment	Thin, robust Python wrapper around fast_align	fast_align binary
mnlp.modulation	FiLM (Perez et al., 2018) conditioning modules with soft selection and Gaussian variance propagation	PyTorch

Highlights

Turkish morphology — analysis and generation

Turkish is agglutinative: a single word like geliyormuşsunuz ("apparently you (pl.) are coming") stacks aspect, evidentiality, person, and number onto the stem gel-. This module handles both directions with pure rules — no model downloads, no lookup server:

from mnlp.turkish import TurkishMorphology

tm = TurkishMorphology()

tm.analyze("geliyormuşsunuz")
# {'stem': 'gel', 'features': {'ASPEKT': 'progressive',
#  'EVIDENTIALITY': 'indirect', 'PERSON': '2', 'NUMERUS': 'plural', ...}}

tm.generate("ev", {"NUMERUS": "plural", "POSSESSION": "1pl",
                   "KASUS": "ablative"}, pos="NOUN")
# 'evlerimizden'  ("from our houses")

tm.gloss("geliyormuşsunuz")
# 'GEL-PROG-EVID-P.2PL'  (Leipzig-style glossing)

Implemented phenomena: 2/4/8-way vowel harmony, voicing assimilation (kitap → kitabı), buffer consonants (y/n/s), stem mutations (git → gid-), fused suffixes (negation + progressive: gelmiyorum), irregular negative aorist (gelmez), and possessive-aware case selection (evi vs. evini).

Compound tense detection (German / English)

Periphrastic tenses ("wird gemacht worden sein", "would have been doing") can't be read off a single token. This module walks auxiliary chains on spaCy dependency parses and returns normalized tense / aspect / mood / voice / polarity features:

from mnlp.grammar import analyze_sentence_tmv

analyze_sentence_tmv("Das Haus ist gebaut worden.", lang="de")
# [('bauen', {'TEMPUS': 'past', 'ASPEKT': 'perfective', 'VOICE': 'passive'})]

FiLM conditioning

Feature-wise Linear Modulation for conditioning representations on named categorical features — with hard indices or soft probability vectors, and optional variance propagation for Gaussian embeddings:

from mnlp.modulation import FiLMStack, FiLMConfig

stack = FiLMStack({"tense": 3, "number": 2}, FiLMConfig(d_model=64))
h, _ = stack(h, {"tense": tense_idx, "number": number_idx})

Installation

pip install -e .            # core (Turkish morphology, alignment wrapper)
pip install -e ".[grammar]" # + spaCy tense detection
pip install -e ".[ml]"      # + PyTorch FiLM modules
pip install -e ".[all]"

For mnlp.grammar, install the spaCy models you need:

python -m spacy download de_core_news_sm
python -m spacy download en_core_web_sm

For mnlp.alignment, build fast_align and point FAST_ALIGN_PATH at the binary.

Tests

pytest

Tests for optional components (spaCy, PyTorch, fast_align binary) are skipped automatically when the dependency is not available.

License

MIT — see LICENSE.