pgllm: Use LLMs in Postgres#

Github Documentation Status Linkedin Github Sponsors Build License

pgllm brings LLMs to Postgres, by embedding the Python llm library.

API#

  • llm_generate(input text, model text[, params jsonb]) → text

  • llm_embed(input text/bytea, model text[, params jsonb]) → float8[]

Features#

  • Text Generation: Generate text based on input prompts using specified LLM models.

  • Text/Binary Embedding: Convert text into numerical embeddings for use in machine learning models or similarity searches.

  • Customizable Parameters: Pass additional parameters to the LLM models as JSONB for more flexible text generation and embedding.

  • pgvector integration

  • Support for LLM plugins

Installing Models#

You have to be sure that the python3 you’re using is the same one that you pointed to during the Installation.

Some dummy models

python3 -m llm install llm-markov
python3 -m llm install llm-embed-hazo

Some more sophisticated models that will download artifacts in the background, the first time they’ll be used

python3 -m llm install llm-embed-jina
python3 -m llm install llm-embed-onnx

Embeddings#

select llm_embed('hello world', 'hazo');
             llm_embed             
-----------------------------------
 {5,5,0,0,0,0,0,0,0,0,0,0,0,0,0,0}
(1 row)

Generation#

select llm_generate('hello world', 'markov');
          llm_generate          
--------------------------------
 world hello world world hello ....
(1 row)

Model Parameteres#

Can be passed as a jsonb argument.

select llm_generate('hello world', 'markov', '{"length": 20, "delay": 0.2}');
                                                       llm_generate                                                       
--------------------------------------------------------------------------------------------------------------------------
 world world hello world hello world hello world world hello world world world world world world world world world hello 
(1 row)

Installation#

See build.yml