Nendo Plugin TextGen¤

A text generation plugin using local LLMs or other text generation methods. Builds on top of transformers by Hugging Face.

Features¤

Generate text from a prompt using a local LLM
Access all LLMs from the Hugging Face Model Hub

Requirements¤

Since we depend on transformers, please make sure that you fulfill their requirements. You also need Pytorch installed on your system, please refer to the pytorch installation instructions.

Installation¤

Install Nendo
pip install nendo-plugin-textgen

If you have a CUDA GPU you can install the following library for an additional speedup:

pip install flash-attn --no-build-isolation

Then set ATTN_IMPLEMENTATION=flash_attention_2 in your environment variables.

Usage¤

Take a look at a basic usage example below. For more detailed information, please refer to the documentation.

>>> from nendo import Nendo
>>> nd = Nendo(plugins=["nendo_plugin_textgen"])

>>> nd.plugins.textgen(prompt=["Tell me about your favorite song."])[0]

Contributing¤

Visit our docs to learn all about how to contribute to Nendo: Contributing

License¤

Nendo: MIT License