>[!NOTE] >* >The usage of this shim is now considered deprecated. The underlying libraries should be used directly. >noj is a new librray to combine several of these librraies, without remapping the namespaces. >It contains as well updated versions of several of the tutorials here. >*

• v0.3:
• latest snapshot:
• latest snapshot:

scicloj.ml - A idiomatic Clojure machine learning library.

All documenttaion stays valid when using libraries directly or via noj, except for the namespaces in use.)

Main features:

• Harmonized and idiomatic use of various classification, regression and unsupervised models
• Supports creation of machine learning pipelines as-data
• Includes easy-to-use, sophisticated cross-validations of pipelines
• Includes most important data transformation for data preprocessing
• Experiment tracking can be added by the user via a callback mechanism
• Open architecture to allow to plugin any potential ML model, even in non-JVM languages, including deep learning
• Based on well established Clojure/Java Data Science libraries

- tech.ml.dataset for very efficient underlying data storage - Smile for ML models - metamorph.ml as foundation of higher level ML functions (former: tech.ml )

Quickstart

Dependencies:

clojure
{:deps
 {scicloj/scicloj.ml {:mvn/version "0.3"}}}

Code:

(require '[scicloj.ml.core :as ml]
         '[scicloj.ml.metamorph :as mm]
         '[scicloj.ml.dataset :as ds])
;; read train and test datasets (def titanic-train   (ds/dataset "https://github.com/scicloj/metamorph-examples/raw/main/data/titanic/train.csv" {:key-fn keyword :parser-fn :string}))
(def titanic-test   (-> "https://github.com/scicloj/metamorph-examples/raw/main/data/titanic/test.csv"       (ds/dataset {:key-fn keyword :parser-fn :string})       (ds/add-column :Survived [""] :cycle)))
;; construct pipeline function including Logistic Regression model (def pipe-fn   (ml/pipeline    (mm/select-columns [:Survived :Pclass ])    (mm/add-column :Survived (fn [ds] (map #(case % "1" "yes" "0" "no" nil "") (:Survived ds))))    (mm/categorical->number [:Survived :Pclass])    (mm/set-inference-target :Survived)    {:metamorph/id :model}    (mm/model {:model-type :smile.classification/logistic-regression})))
;;  execute pipeline with train data including model in mode :fit (def trained-ctx   (pipe-fn {:metamorph/data titanic-train             :metamorph/mode :fit}))
;; execute pipeline in mode :transform with test data which will do a prediction  (def test-ctx   (pipe-fn    (assoc trained-ctx           :metamorph/data titanic-test           :metamorph/mode :transform)))
;; extract prediction from pipeline function result (-> test-ctx :metamorph/data     (ds/column-values->categorical :Survived))      ;; => #tech.v3.dataset.column<string>[418] ;;    :Survived ;;    [no, no, yes, no, no, no, no, yes, no, no, no, no, no, yes, no, yes, yes, no, no, no...]

Community

For support use Clojurians on Zulip:

Scicloj.ml on Zulip

or on Clojurians Slack:

Scicloj.ml on Slack

Documentation

Full documentation is here as userguides

API documentation: https://scicloj.github.io/scicloj.ml

Reference to projects scicloj.ml is using/based on:

This library itself is a shim, not containing any functions. The code is present in the following repositories, and the functions get re-exported in scicloj.ml in a small number of namespaces for user convenience.

• https://github.com/techascent/tech.ml
• https://github.com/scicloj/tablecloth
• https://github.com/scicloj/metamorph
• https://github.com/scicloj/metamorph.ml
• https://github.com/techascent/tech.ml.dataset
• https://github.com/scicloj/scicloj.ml.smile
• https://github.com/scicloj/scicloj.ml.xgboost
• https://github.com/haifengl/smile

Scicloj.ml organises the existing code in 3 namespaces, as following:

namespace scicloj.ml.core

Functions are re-exported from:

scicloj.metamorph.ml.

• scicloj.metamorph.core

namespace scicloj.ml.dataset

All functions in this ns take a dataset as first argument. The functions are re-exported from:

• tabecloth.api
• tech.v3.dataset.modelling
• tech.v3.dataset.column-filters

namespace scicloj.ml.metamorph

All functions in this ns take a metamorph context as first argument, so can directly be used in metamorph pipelines. The functions are re-exported from:

• tablecloth.pipeline
• tech.v3.libs.smile.metamorph
• scicloj.metamorph.ml
• tech.v3.dataset.metamorph

In case you are already familar with any of the original namespaces, they can of course be used directly as well:

(require '[tablecloth.api :as tc])
(tc/add-column ...)

Plugins

scicloj.ml can be easely extended by plugins, which contribute models or other algorithms. By now the following plugins exist:

• Builtin: scicloj.ml.smile
• Builtin: scicloj.ml.xgboost
• All sklearn models: sklearn.clj
• top2vec model: scicloj.ml.top2vec
• crf A NER model from standfortNLP
• clj-djl Use fasttext model from djl