embedPy

Allows the kdb+ interpreter to manipulate Python objects and call Python functions. Part of the Fusion for kdb+_ interface collection.

Please direct any questions to ai@kx.com.

Please report issues in this repository.

Requirements

• kdb+ ≥ 3.5 64-bit/32-bit(Linux/Arm)
• Python ≥ 3.8.0 (macOS/Linux/Arm) ≥ 3.8.0 windows

Overview

You can either

• install embedPy to run on your local machine; or
• download or build a Docker image in which to run embedPy

There are three ways to install embedPy on your local machine:

• Download and install a release

• Clone and build from source, on your local machine or in a Docker image

• Install with Conda - recommended for use with

- Anaconda Python - mlnotebooks - JupyterQ

32-bit Linux/Arm builds require users to build from source, there is not currenly a conda build or provided pre-compiled binary.

Anaconda Python

If you are using Anaconda Python, we recommend installing with Conda. If, instead, you take option (1) or (2) above, and are using Linux or macOS, set your LDLIBRARYPATH (Linux) or DYLDLIBRARYPATH (macOS) to your Python distributions library directory to avoid conflicts between libraries which both q and Python use (e.g. libz, libssl). You can find this directory's location in Python.

>>> import sysconfig
>>> sysconfig.getconfigvar('LIBDIR')

PyQ

If you are currently using PyQ, both interfaces use a file p.k in $QHOME/{l64,m64} which results in a conflict when both are installed.

You may want to run initially from another directory, without installing. Skip the install step above, and run q in the directory where you unzipped the release.

Test script

The test script test.q requires the packages listed in tests/requirements.txt, although embedPy does not itself require them. They can be installed using pip or conda.

pip install -r tests/requirements.txt

or

conda install --file tests/requirements.txt

If the tests all pass, no message is displayed.

Install on local machine

Download and install a release

• Download a release archive from the releases page, and unzip it.

• In the unzipped directory, run the tests.

q test.q

• Install: put p.q and p.k in QHOME and the library file (p.so for macOS/Linux or p.dll for Windows) in $QHOME/{l64,m64,w64}.

Clone and build from source

• Clone this repository from GitHub.

• To run embedPy without Internet access, download the kdb+ C API header file and place it in the build directory.

• Build the interface and run the tests.

make p.so && q test.q

• Install: put p.q and p.k in $QHOME and p.so in $QHOME/{l64,l32,m64}.

Note

>

For ease of install on 32-bit Arm and Linux we suggest a new user use a miniconda version of Python specific to the architecture being used, for example rpi for Raspberry Pi. This is not an explicit requirement but makes install of embedPy and Python packages more seamless.

Install with Conda

This requires either macOS or Linux.

• Download and install either the full Anaconda distribution or Miniconda for Python3

• Use the conda command to install packages as follows:

$ conda install -c kx embedPy

Run on local machine

Start q with embedPy

q p.q

Or from q, load p.q.

q)\l p.q

Documentation is in the :openfilefolder: docs folder.

Run a Docker image

If you have Docker installed, instead of installing embedPy on your machine, you can run:

$ docker run -it --name myembedpy kxsys/embedpy
kdb+ on demand - Personal Edition
[snipped]
I agree to the terms of the license agreement for kdb+ on demand Personal Edition (N/y): y
If applicable please provide your company name (press enter for none): ACME Limited Please provide your name: Bob Smith Please provide your email (requires validation): bob@example.com KDB+ 3.5 2018.04.25 Copyright (C) 1993-2018 Kx Systems l64/ 4()core 7905MB kx 0123456789ab 172.17.0.2 EXPIRE 2018.12.04 bob@example.com KOD #0000000
q)

See docker/README.md for more details.

Configuration

• EMBEDPY_EXECUTABLE export this environment variable to define the Python executable for embedPy to use.

- If not set, embedPy attempts to use python3 and then python from PATH.

• EMBEDPYPYTHONLIB_PATH export this environment variable to define the libpython.so for embedPy to use.

- Used in cases of 'libpython error on load of EmbedPy when filepath is incorrect. See also FAQ.

Example:

$export EMBEDPY_EXECUTABLE="/usr/bin/python"
$export EMBEDPY_PYTH
$q p.q

Back-incompatible changes

V1.0 -> V 1.1

.p.key and .p.value removed

V0.2-beta -> V1.0

• Attribute access from embedPy object

q)objATTRNAME   / old
    q)obj:ATTRNAME  / new

• embedPy objects can be called directly without explicitly specifying the call return type; the default return type is an embedPy object

V0.1-beta -> V0.2beta in V0.2-beta

V0.2-beta features a number of changes back-incompatible with the previous release, V0.1-beta.

Most notably, the default type used in many operations is now the embedPy type, rather than the foreign type.