Apache Flink® is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. Flink has been designed to run in all common cluster environments, perform computations at in-memory speed and at any scale.

Getting Started with Apache Flink®

Apache Flink® Documentation

Apache Flink® Wiki

Apache Flink® Training Course | Apache Flink®

Apache Flink® Community

Streaming ETL with Apache Flink and Amazon Kinesis Data Analytics | AWS

Certified Apache Flink Training Course | DataFlair

Apache Flink vs Apache Spark | DataFlair

Cloudera Streaming Analytics: Using Apache Flink and SQL Stream Builder on CDP

Apache Flink | A Real Time & Hands-On course on Flink | Udemy

Getting Started with Apache Flink | Udemy

Exploring the Apache Flink API for Processing Streaming Data | Pluralsight

Processing Streaming Data Using Apache Flink | Pluralsight

Apache Flink: Data Processing Technology | Pluralsight

Apache Spark Quick Start

What is Apache Spark? | IBM

Introduction to Apache Spark and Analytics | AWS

Apache Spark 3.0: For Analytics & Machine Learning | NVIDIA

.NET for Apache Spark™ | Big data analytics

Apache Spark Basics | MATLAB & Simulink

MATLAB Hadoop and Spark | MATLAB & Simulink

Top Apache Spark Courses Online | Coursera

Top Apache Spark Courses Online | Udemy

Apache Spark In-Depth (Spark with Scala) | Udemy

Learn Apache Spark with Online Courses | edX

Apache Spark Essential Training Online Class | LinkedIn Learning

Cloudera Developer Training for Apache Spark™ and Hadoop | Cloudera

Databricks Certified Associate Developer for Apache Spark 3.0 certification | Databricks

Apache Spark Training Courses | NobleProg

Apache Flink Tools, Libraries, and Frameworks

Back to the Top

FlinkX is a data synchronization tool based on Apache Flink® that supports data synchronization/integration and streaming SQL computation.

Flink Siddhi is a light-weight library to run Siddhi CEP within Apache Flink® streaming application. Siddhi CEP is a lightweight and easy-to-use Open Source Complex Event Processing Engine (CEP) released as a Java Library under Apache Software License v2.0.

Apache Beam is an open source, unified model and set of language-specific SDKs for defining and executing data processing workflows, and also data ingestion and integration flows, supporting Enterprise Integration Patterns (EIPs) and Domain Specific Languages (DSLs).

Apache Bahir is a tool that provides extensions to distributed analytics platforms such as Apache Spark™ and Apache Flink®.

Zeppelin is a web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala, Python, R and more.

Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data for lightning fast search, fine‑tuned relevancy, and powerful analytics that scale with ease.

Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language.

DataFusion is an extensible query execution framework, written in Rust, that uses Apache Arrow as its in-memory format. DataFusion supports both an SQL and a DataFrame API for building logical query plans as well as a query optimizer and execution engine capable of parallel execution against partitioned data sources (CSV and Parquet) using threads.

Fletcher is a framework that helps to integrate FPGA accelerators with tools and frameworks that use Apache Arrow in their back-ends.

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of streaming event data.

Apache Mesos is a cluster manager that provides efficient resource isolation and sharing across distributed applications, or frameworks. It can run Hadoop, Jenkins, Spark, Aurora, and other frameworks on a dynamically shared pool of nodes.

Apache Kafka® is a distributed data store optimized for ingesting and processing streaming data in real-time. Streaming data is data that is continuously generated by thousands of data sources, which typically send the data records in simultaneously.

Apache Spark™ is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R, and an optimized engine that supports general computation graphs for data analysis. It also supports a rich set of higher-level tools including Spark SQL for SQL and DataFrames, MLlib for machine learning, GraphX for graph processing, and Structured Streaming for stream processing.

Spark SQL is a Spark module for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. Internally, Spark SQL uses this extra information to perform extra optimizations.

Spark Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. It can express your streaming computation the same way you would express a batch computation on static data from various sources including Apache Kafka, Apache Flume, and Amazon Kinesis.

Apache HBase™ is an open-source, NoSQL, distributed big data store. It enables random, strictly consistent, real-time access to petabytes of data. HBase is very effective for handling large, sparse datasets. HBase serves as a direct input and output to the Apache MapReduce framework for Hadoop, and works with Apache Phoenix to enable SQL-like queries over HBase tables.

Hadoop Distributed File System (HDFS) is a distributed file system that handles large data sets running on commodity hardware. It is used to scale a single Apache Hadoop cluster to hundreds (and even thousands) of nodes. HDFS is one of the major components of Apache Hadoop, the others being MapReduce and YARN.

Apache PredictionIO is an open source machine learning framework for developers, data scientists, and end users. It supports event collection, deployment of algorithms, evaluation, querying predictive results via REST APIs. It is based on scalable open source services like Hadoop, HBase (and other DBs), Elasticsearch, Spark and implements what is called a Lambda Architecture.

Apache Samza is a distributed stream processing framework that allows you to build stateful applications that process data in real-time from multiple sources including Apache Kafka. Battle-tested at scale, it supports flexible deployment options to run on YARN or as a standalone library.

Apache Airflow is an open-source workflow management platform created by the community to programmatically author, schedule and monitor workflows. Airflow has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Airflow is ready to scale to infinity.

Apache Arrow is a language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware like CPUs and GPUs. Languages that have Arrow libraries (under development) include C, C++, Go, Java, JavaScript, Python, Ruby and Rust.

Confluent Platform is a full-scale data streaming platform that enables you to easily access, store, and manage data as continuous, real-time streams. Built by the original creators of Apache Kafka®, Confluent expands the benefits of Kafka with enterprise-grade features while removing the burden of Kafka management or monitoring.

Kafka Connec is an open source Apache Kafka framework for connecting Kafka with external systems such as databases, key-value stores, search indexes, and file systems.

IBM Streams is a stream processing framework with Kafka source and sink to consume and produce Kafka messages.

KaBoom is a high-performance HDFS data loader.

Azkarra Streams is a lightweight java framework to make it easy to build and manage streaming microservices based on Kafka Streams.

uReplicator is a tool that provides the ability to replicate across Kafka clusters in other data centers.

Mirus is a tool for distributed, high-volume replication between Apache Kafka clusters based on Kafka Connect.

Kafka Manager is a tool for managing Apache Kafka.

Kafkat is a simplified command-line administration for Kafka brokers.

Kafka Web Console is a tool that displays information about your Kafka cluster including which nodes are up and what topics they host data for.

Kafka Offset Monitor is a tool that displays the state of all consumers and how far behind the head of the stream they are.

Capillary is a tool that displays the state and deltas of Kafka-based Apache Storm topologies.

Doctor Kafka is a service for cluster auto healing and workload balancing.

Cruise Control is a tool that fully automate the dynamic workload rebalance and self-healing of a Kafka cluster.

Burrow is a monitoring tool that provides consumer lag checking as a service without the need for specifying thresholds.

Chaperone is an audit system that monitors the completeness and latency of data stream.

Sematext is an integration tool for Kafka monitoring that collects and charts 200+ Kafka metrics.

Cloudera is the big data software platform of choice across numerous industries, providing customers with components like Hadoop, Spark, and Hive.

Splunk is a software platform that is used for searching, monitoring, and examining machine-generated Big Data through a web interface.

Apache Cassandra™ is an open source NoSQL distributed database trusted by thousands of companies for scalability and high availability without compromising performance. Cassandra provides linear scalability and proven fault-tolerance on commodity hardware or cloud infrastructure make it the perfect platform for mission-critical data.

Aiven for Apache Cassandra is a fully managed NoSQL database, deployable in the cloud of your choice. Snap it into your existing workflows with the click of a button, automate away the mundane tasks, and focus on building your core apps.

Amazon Keyspaces (for Apache Cassandra) is a scalable, highly available, and managed Apache Cassandra–compatible database service.

Azure Managed Instance for Apache Cassandra is a service offering moderate management, elasticity, and instance-based pricing for Cassandra data. Go beyond traditional lift and shift by expanding your Cassandra workloads to the cloud and keep control over what matters to you.

DataStax Astra is a cloud-native, serverless database as-a-service built on Apache Cassandra™, complete with a free-tier and CQL, REST, schemaless JSON Document and GraphQL APIs in addition to language drivers for faster development. It also features an improved secondary index implementation called storage attached indexing (SAI) where you can search/filter on non-primary key columns. Astra is available on AWS, Azure and Google Cloud.

Elassandra is an Apache Cassandra distribution including an Elasticsearch search engine. Elassandra is a multi-master multi-cloud database and search engine with support for replicating across multiple datacenters in active/active mode.

Instaclustr Hosted & Managed Apache Cassandra as a Service is a fully managed and SOC 2 certified hosted & managed service for Apache Cassandra® on AWS, Azure, GCP and IBM Cloud.

Adelphi is an automation tool for testing open-source Cassandra using cassandra-diff, nosqlbench, and fqltool.

Ansible Cassandra Collection is a collection tools that provides all Ansible modules allowing to interact with Apache Cassandra. Link to GitHub repo.

Apache Ignite® is a distributed database for high-performance computing with in-memory speed. Ignite's main goal is to provide performance and scalability by partitioning and distributing data within a cluster. The cluster provides very fast data processing.

Azure-Samples/Cassandra Proxy is a proxy handles client connections and forwards them to two Cassandra clusters simultaneously.

Cassandra.link is a curated site with tools, along with cassandra.tools.

Cassandra Lucene Index is a plugin for Apache Cassandra that extends its index functionality to provide near real time search such as ElasticSearch or Solr, including full text search capabilities and free multivariable, geospatial and bitemporal search.

Cassandra Migration is a simple and lightweight Apache Cassandra database schema migration tool.

Cassandra Prometheus Exporter is a standalone application which exports Cassandra metrics through a prometheus friendly endpoint.

DataStax Bulk Loader is an easy-to-use command line utility for loading and unloading JSON or CSV files to/from the database, counting rows in tables and identifying large partitions.

DataStax Metrics Collector for Cassandra is a tool Based on Collectd, that aggregates OS and Cassandra metrics along with diagnostic events to facilitate problem resolution and remediation

DOSA is a storage framework that provides a declarative object storage abstraction for applications in Golang and (soon) Java.

Dropwizard-Cassandra is a dropwizard-cassandra library provides useful functionality for Dropwizard apps that communicate with Cassandra clusters.

EmoDB is a RESTful HTTP data store built on top of Cassandra that stores schemaless JSON objects and offers a databus that allows subscribers to watch for changes to those events. It’s designed to span multiple data centers and features massive non-blocking writes and no synchronous cross data center communication.

FiloDB is a distributed, Prometheus-compatible, real-time, in-memory, massively scalable, multi-schema time series/event/operational database.

Gocql is a software package that implements a fast and robust Cassandra client for the Go programming language.

Grafana Cassandra Source is a Apache Cassandra Datasource for Grafana. This datasource is to visualise time-series data stored in Cassandra/DSE.

Hackolade is a visual data modeling tool for Cassandra.

Hazelcast Cassandra is a sample implementation of Hazelcast MapStore with DSE Cassandra using DSE Object Mapper.

Instaclustr Esop is the Swiss knife for backup and restore of your node to GCP, Azure, S3, Ceph etc. Supports backup and restoration of commit logs too. Esop is embedded in Instaclustr Icarus sidecar so you may backup and restore your cluster remotely and on-the-fly without any disruption.

Instaclustr Exporter is a Java agent that exports Cassandra metrics to Prometheus.

Instaclustr Go Client for Instaclustr Icarus is a Go client for Instaclustr Icarus sidecar.

Instaclustr Kerberos plugin is a GSSAPI authentication provider for Apache Cassandra.

Instaclustr Java Driver for Kerberos is a GSSAPI authentication provider for the Cassandra Java driver.

Instaclustr LDAP Authenticator is a LDAP Authenticator for Apache Cassandra.

Instaclustr Minotaur is a Command line tool for consistent rebuilding of a Cassandra cluster.

Instaclustr SSTable Generator is a CLI tool for programmatic generation of Cassandra SSTables.

Instaclustr SSTable Tools is a command line tool that helps admins get summaries, metadata, partition info, and cell info for SSTables.

Instaclustr TTL Remover is a Command line tool for rewriting SSTables to remove TTLs.

JanusGraph is a highly scalable graph database optimized for storing and querying large graphs with billions of vertices and edges distributed across a multi-machine cluster.

KairosDB is a fast distributed scalable time series database written on top of Cassandra.

Kong is a cloud-native, fast, scalable, and distributed Microservice Abstraction Layer.

The Last Pickle Cassandra stress tool is a workload-centric stress tool for Apache Cassandra. Designed for simplicity, no math degree required. (DataStax).

The Last Pickle Medusa is an Apache Cassandra Backup and Restore Tool (DataStax).

The Last Pickle Reaper is an automated repair tool for Apache Cassandra (DataStax).

Netflix Data Explorer is a tool that allows users to explore data stored in several popular datastores.

NoSQLBench is a pluggable benchmarking suite for Cassandra and other distributed systems.

OpenNMS is the world’s first enterprise grade network management application platform developed under the open source model.

Phantom is an underlying engine of all other drivers. Phantom, Quill, and the Spark connector all use it underneath the hood to connect and execute queries.

Quill is a tool that provides a Quoted Domain Specific Language (QDSL) to express queries in Scala and execute them in a target language.

Rebar is a ulti-tenant SaaS boilerplate + examples for universal web application with React, Material-UI, Relay, GraphQL, JWT, Node.js, C* DB - Cassandra/Elassandra/Scylla.

Stargate is an pen source data gateway providing CQL, Schemaless JSON Document, REST, and GraphQL APIs for Apache Cassandra.

Stratio Cassandra Lucene Index is a plugin for Apache Cassandra that extends its index functionality to provide near real time search such as ElasticSearch or Solr, including full text search capabilities and free multivariable, geospatial and bitemporal search.

Strongbox is an OpenSource artifact repository manager written in Java.

Temporal is a microservice orchestration platform which enables developers to build scalable applications without sacrificing productivity or reliability.

Trellis LDP is an enterprise-ready linked data server built on existing Web standards that is modular, extensible and fast.

Wasabi is an A/B Testing Service is a real-time, enterprise-grade, 100% API driven project.

D2iQ Cassandra Kudo Operator is the KUDO Cassandra Operator makes it easy to deploy and manage Apache Cassandra on Kubernetes.

DataStax Cassandra operator is a tool that manages DataStax Kubernetes Operator for Apache Cassandra.

Instaclustr Cassandra operator is a tool that manages Cassandra clusters deployed to Kubernetes and automates tasks related to operating a Cassandra cluster.

K8ssandra is a tool that provides a production-ready platform for running Apache Cassandra on Kubernetes, including automation for operational tasks such as installation via helm, repairs, backups, and monitoring. K8ssandra includes the DataStax Cassandra operator.

Orange Cassandra operator is a Kubernetes operator to automate provisioning, management, autoscaling and operations of Apache Cassandra clusters deployed to K8s.

Rook is an open source cloud-native storage orchestrator for Kubernetes, providing the platform, framework, and support for a diverse set of storage solutions to natively integrate with cloud-native environments.

Sky Cassandra Operator is a Kubernetes operator that manages Cassandra clusters inside Kubernetes.

Apache Cassandra cassandra-sidecar is a sidecar for the highly scalable Apache Cassandra database, built as part of the Apache Cassandra project.

DataStax Management API for Apache Cassandra is a RESTful / Secure Management Sidecar for Apache Cassandra.

Apache Camel is an Open Source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data.

Django Cassandra Engine is a Cassandra backend for Django Framework that allows you to use Cqlengine directly in your project.

Express Cassandra is a Cassandra ORM/ODM/OGM for NodeJS with Elassandra & JanusGraph Support.

Marmaray is a generic Hadoop data ingestion and dispersal framework and library. It is a plug-in based framework built on top of the Hadoop ecosystem where support can be added to ingest data from any source and disperse to any sink leveraging the power of Apache Spark.

Micronaut Cassandra isa tool that adds support for the DataStax Cassandra Driver to a Micronaut application.

Quarkus extension for Apache Cassandra is an Apache Cassandra® extension for Quarkus. Quarkus is A Kubernetes Native Java stack tailored for OpenJDK HotSpot and GraalVM, crafted from the best of breed Java libraries and standards.

Stream Framework is a Python library which allows you to build activity streams & newsfeeds using Cassandra and/or Redis.

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.

Cassandra Storage Plugin is Apache Drill’s Cassandra storage plugin that allows you to execute SQL queries against Cassandra tables.

Flink Sink Connector is a connector thatprovides sinks that writes data into a Apache Cassandra database.

Confluent Connect Cassandra is the Confluent Cassandra Sink Connector is used to move messages from Kafka into Apache Cassandra.

DataStax Sink Connector is the DataStax Apache Kafka Connector automatically takes records from Kafka topics and writes them to a DataStax Enterprise or Apache Cassandra™ database. This sink connector is deployed in the Kafka Connect framework and removes the need to build a custom solution to move data between these two systems.

Lenses Sink Connector is the Cassandra Sink allows you to write events from Kafka to Cassandra. The connector converts the value from the Kafka Connect SinkRecords to JSON and uses Cassandra’s JSON insert functionality to insert the rows. The task expects pre-created tables in Cassandra.

Pulsar Sink Connector Cassandra Connector is the Pulsar Cassandra Sink connector is used to write messages to a Cassandra Cluster.

DataStax Spark Cassandra Connector is a library that lets you expose Cassandra tables as Spark RDDs and Datasets/DataFrames, write Spark RDDs and Datasets/DataFrames to Cassandra tables, and execute arbitrary CQL queries in your Spark applications.

Presto is an Cassandra connector allows querying data stored in Cassandra.

Docker community Cassandra images is a collection of Docker images for Apache Cassandra maintained by the Docker community.

DataStax Desktop is a cross-platform (Windows, MacOSX, Linux) application that allows developers to quickly explore Apache Cassandra™ with a few clicks on their laptop, complete with tutorials and walkthroughs.

Tlp-cluster is a tool for launching Cassandra clusters in AWS (DataStax).

MLib is Spark’s machine learning (ML) library. Its goal is to make practical machine learning scalable and easy. It consists of common learning algorithms and utilities, including classification, regression, clustering, collaborative filtering, dimensionality reduction, as well as lower-level optimization primitives and higher-level pipeline APIs.

Graphx is the new Spark API for graphs and graph-parallel computation. At a high-level, GraphX extends the Spark RDD by introducing the Resilient Distributed Property Graph: a directed multigraph with properties attached to each vertex and edge.

PySpark is an interface for Apache Spark in Python. It not only allows you to write Spark applications using Python APIs, but also provides the PySpark shell for interactively analyzing your data in a distributed environment.

Apache Spark Connector for SQL Server and Azure SQL is a high-performance connector that enables you to use transactional data in big data analytics and persists results for ad-hoc queries or reporting. The connector allows you to use any SQL database, on-premises or in the cloud, as an input data source or output data sink for Spark jobs.

Azure Databricks is a fast and collaborative Apache Spark-based big data analytics service designed for data science and data engineering. Azure Databricks, sets up your Apache Spark environment in minutes, autoscale, and collaborate on shared projects in an interactive workspace. Azure Databricks supports Python, Scala, R, Java, and SQL, as well as data science frameworks and libraries including TensorFlow, PyTorch, and scikit-learn.

Koalas is a project that makes data scientists more productive when interacting with big data, by implementing the pandas DataFrame API on top of Apache Spark.

MLflowis a platform to streamline machine learning development, including tracking experiments, packaging code into reproducible runs, and sharing and deploying models. It offers a set of lightweight APIs that can be used with any existing machine learning application or library (TensorFlow, PyTorch, XGBoost, etc), wherever you currently run ML code (notebooks, standalone applications or the cloud). MLflow has four main components:

- The Tracking component that allows you to record machine model training sessions (called runs) and run queries using Java, Python, R, and REST APIs. - The Projects component packages code that is used in data science projects to ensure it can easily be reused and experiments can be reproduced. - The Models component that provides a standard unit for packaging and reusing machine learning models. - The Model Registry component that lets you centrally manage models and their lifecycle.

Cluster Manager for Apache Kafka(CMAK) is a tool for managing Apache Kafka clusters.

BigDL is a distributed deep learning library for Apache Spark. With BigDL, users can write their deep learning applications as standard Spark programs, which can directly run on top of existing Spark or Hadoop clusters.

Jupyter Notebook is an open-source web application that allows you to create and share documents that contain live code, equations, visualizations and narrative text. Jupyter is used widely in industries that do data cleaning and transformation, numerical simulation, statistical modeling, data visualization, data science, and machine learning.

Dask is an open source tool that provides advanced parallelism for analytics, enabling performance at scale for the tools you love. It is developed in coordination with other community projects like NumPy, pandas, and scikit-learn.

Dask DataFrame is a large parallel DataFrame composed of many smaller Pandas DataFrames, split along the index. These Pandas DataFrames may live on disk for larger-than-memory computing on a single machine, or on many different machines in a cluster. One Dask DataFrame operation triggers many operations on the constituent Pandas DataFrames.

Neo4j is the only enterprise-strength graph database that combines native graph storage, advanced security, scalable speed-optimized architecture, and ACID compliance to ensure predictability and integrity of relationship-based queries.

ElasticSearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Elasticsearch is developed in Java.

Logstash is a tool for managing events and logs. When used generically, the term encompasses a larger system of log collection, processing, storage and searching activities.

Kibana is an open source data visualization plugin for Elasticsearch. It provides visualization capabilities on top of the content indexed on an Elasticsearch cluster. Users can create bar, line and scatter plots, or pie charts and maps on top of large volumes of data.

Trino is a Distributed SQL query engine for big data. It is able to tremendously speed up ETL processes, allow them all to use standard SQL statement, and work with numerous data sources and targets all in the same system.

Extract, transform, and load (ETL) is a data pipeline used to collect data from various sources, transform the data according to business rules, and load it into a destination data store.

Redis(REmote DIctionary Server) is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. It provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams.

Apache OpenNLP is an open-source library for a machine learning based toolkit used in the processing of natural language text. It features an API for use cases like Named Entity Recognition, [Sentence Detection](), POS(Part-Of-Speech) tagging, Tokenization) Feature extraction, Chunking), Parsing, and Coreference resolution.

Open Neural Network Exchange(ONNX) is an open ecosystem that empowers AI developers to choose the right tools as their project evolves. ONNX provides an open source format for AI models, both deep learning and traditional ML. It defines an extensible computation graph model, as well as definitions of built-in operators and standard data types.

Apache MXNet is a deep learning framework designed for both efficiency and flexibility. It allows you to mix symbolic and imperative programming to maximize efficiency and productivity. At its core, MXNet contains a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly. A graph optimization layer on top of that makes symbolic execution fast and memory efficient. MXNet is portable and lightweight, scaling effectively to multiple GPUs and multiple machines. Support for Python, R, Julia, Scala, Go, Javascript and more.

AutoGluon is toolkit for Deep learning that automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications. With just a few lines of code, you can train and deploy high-accuracy deep learning models on tabular, image, and text data.

Anaconda is a very popular Data Science platform for machine learning and deep learning that enables users to develop models, train them, and deploy them.

PlaidML is an advanced and portable tensor compiler for enabling deep learning on laptops, embedded devices, or other devices where the available computing hardware is not well supported or the available software stack contains unpalatable license restrictions.

OpenCV is a highly optimized library with focus on real-time computer vision applications. The C++, Python, and Java interfaces support Linux, MacOS, Windows, iOS, and Android.

Scikit-Learn is a Python module for machine learning built on top of SciPy, NumPy, and matplotlib, making it easier to apply robust and simple implementations of many popular machine learning algorithms.

Weka is an open source machine learning software that can be accessed through a graphical user interface, standard terminal applications, or a Java API. It is widely used for teaching, research, and industrial applications, contains a plethora of built-in tools for standard machine learning tasks, and additionally gives transparent access to well-known toolboxes such as scikit-learn, R, and Deeplearning4j.

Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by Berkeley AI Research (BAIR)/The Berkeley Vision and Learning Center (BVLC) and community contributors.

Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently including tight integration with NumPy.

Kubernetes

Back to the Top

Building Highly-Availability(HA) Clusters with kubeadm. Source: Kubernetes.io

Kubernetes Learning Resources

Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications.

Getting Kubernetes Certifications

Getting started with Kubernetes on AWS

Kubernetes on Microsoft Azure

Intro to Azure Kubernetes Service

Azure Red Hat OpenShift

Getting started with Google Cloud

Getting started with Kubernetes on Red Hat

Getting started with Kubernetes on IBM

Red Hat OpenShift on IBM Cloud

Enable OpenShift Virtualization on Red Hat OpenShift

YAML basics in Kubernetes

Elastic Cloud on Kubernetes

Docker and Kubernetes

Running Apache Spark on Kubernetes

Kubernetes Across VMware vRealize Automation

VMware Tanzu Kubernetes Grid

All the Ways VMware Tanzu Works with AWS

VMware Tanzu Education

Using Ansible in a Cloud-Native Kubernetes Environment

Managing Kubernetes (K8s) objects with Ansible

Setting up a Kubernetes cluster using Vagrant and Ansible

Running MongoDB with Kubernetes

Kubernetes Fluentd

Understanding the new GitLab Kubernetes Agent

Intro Local Process with Kubernetes for Visual Studio 2019

Kubernetes Contributors

KubeAcademy from VMware

Kubernetes Tutorials from Pulumi

Kubernetes Playground by Katacoda

Scalable Microservices with Kubernetes course from Udacity

Kubernetes Tools, Frameworks, and Projects

Open Container Initiative is an open governance structure for the express purpose of creating open industry standards around container formats and runtimes.

Buildah is a command line tool to build Open Container Initiative (OCI) images. It can be used with Docker, Podman, Kubernetes.

Podman is a daemonless, open source, Linux native tool designed to make it easy to find, run, build, share and deploy applications using Open Containers Initiative (OCI) Containers and Container Images. Podman provides a command line interface (CLI) familiar to anyone who has used the Docker Container Engine.

Containerd is a daemon that manages the complete container lifecycle of its host system, from image transfer and storage to container execution and supervision to low-level storage to network attachments and beyond. It is available for Linux and Windows.

Google Kubernetes Engine (GKE) is a managed, production-ready environment for running containerized applications.

Azure Kubernetes Service (AKS) is serverless Kubernetes, with a integrated continuous integration and continuous delivery (CI/CD) experience, and enterprise-grade security and governance. Unite your development and operations teams on a single platform to rapidly build, deliver, and scale applications with confidence.

Amazon EKS is a tool that runs Kubernetes control plane instances across multiple Availability Zones to ensure high availability.

AWS Controllers for Kubernetes (ACK) is a new tool that lets you directly manage AWS services from Kubernetes. ACK makes it simple to build scalable and highly-available Kubernetes applications that utilize AWS services.

Container Engine for Kubernetes (OKE) is an Oracle-managed container orchestration service that can reduce the time and cost to build modern cloud native applications. Unlike most other vendors, Oracle Cloud Infrastructure provides Container Engine for Kubernetes as a free service that runs on higher-performance, lower-cost compute.

Anthos is a modern application management platform that provides a consistent development and operations experience for cloud and on-premises environments.

Red Hat Openshift is a fully managed Kubernetes platform that provides a foundation for on-premises, hybrid, and multicloud deployments.

OKD is a community distribution of Kubernetes optimized for continuous application development and multi-tenant deployment. OKD adds developer and operations-centric tools on top of Kubernetes to enable rapid application development, easy deployment and scaling, and long-term lifecycle maintenance for small and large teams.

Odo is a fast, iterative, and straightforward CLI tool for developers who write, build, and deploy applications on Kubernetes and OpenShift.

Kata Operator is an operator to perform lifecycle management (install/upgrade/uninstall) of Kata Runtime on Openshift as well as Kubernetes cluster.

Thanos is a set of components that can be composed into a highly available metric system with unlimited storage capacity, which can be added seamlessly on top of existing Prometheus deployments.

OpenShift Hive is an operator which runs as a service on top of Kubernetes/OpenShift. The Hive service can be used to provision and perform initial configuration of OpenShift 4 clusters.

Rook is a tool that turns distributed storage systems into self-managing, self-scaling, self-healing storage services. It automates the tasks of a storage administrator: deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management.

VMware Tanzu is a centralized management platform for consistently operating and securing your Kubernetes infrastructure and modern applications across multiple teams and private/public clouds.

Kubespray is a tool that combines Kubernetes and Ansible to easily install Kubernetes clusters that can be deployed on AWS, GCE, Azure, OpenStack, vSphere, Packet (bare metal), Oracle Cloud Infrastructure (Experimental), or Baremetal.

KubeInit provides Ansible playbooks and roles for the deployment and configuration of multiple Kubernetes distributions.

Rancher is a complete software stack for teams adopting containers. It addresses the operational and security challenges of managing multiple Kubernetes clusters, while providing DevOps teams with integrated tools for running containerized workloads.

K3s is a highly available, certified Kubernetes distribution designed for production workloads in unattended, resource-constrained, remote locations or inside IoT appliances.

Helm is a Kubernetes Package Manager tool that makes it easier to install and manage Kubernetes applications.

Knative is a Kubernetes-based platform to build, deploy, and manage modern serverless workloads. Knative takes care of the operational overhead details of networking, autoscaling (even to zero), and revision tracking.

KubeFlow is a tool dedicated to making deployments of machine learning (ML) workflows on Kubernetes simple, portable and scalable.

Etcd is a distributed key-value store that provides a reliable way to store data that needs to be accessed by a distributed system or cluster of machines. Etcd is used as the backend for service discovery and stores cluster state and configuration for Kubernetes.

OpenEBS is a Kubernetes-based tool to create stateful applications using Container Attached Storage.

Container Storage Interface (CSI) is an API that lets container orchestration platforms like Kubernetes seamlessly communicate with stored data via a plug-in.

MicroK8s is a tool that delivers the full Kubernetes experience. In a Fully containerized deployment with compressed over-the-air updates for ultra-reliable operations. It is supported on Linux, Windows, and MacOS.

Charmed Kubernetes is a well integrated, turn-key, conformant Kubernetes platform, optimized for your multi-cloud environments developed by Canonical.

Grafana Kubernetes App is a toll that allows you to monitor your Kubernetes cluster's performance. It includes 4 dashboards, Cluster, Node, Pod/Container and Deployment. It allows for the automatic deployment of the required Prometheus exporters and a default scrape config to use with your in cluster Prometheus deployment.

KubeEdge is an open source system for extending native containerized application orchestration capabilities to hosts at Edge.It is built upon kubernetes and provides fundamental infrastructure support for network, app. deployment and metadata synchronization between cloud and edge.

Lens is the most powerful IDE for people who need to deal with Kubernetes clusters on a daily basis. It has support for MacOS, Windows and Linux operating systems.

kind is a tool for running local Kubernetes clusters using Docker container “nodes”. It was primarily designed for testing Kubernetes itself, but may be used for local development or CI.

Flux CD is a tool that automatically ensures that the state of your Kubernetes cluster matches the configuration you've supplied in Git. It uses an operator in the cluster to trigger deployments inside Kubernetes, which means that you don't need a separate continuous delivery tool.

Platform9 Managed Kubernetes (PMK) is a Kubernetes as a service that ensures fully automated Day-2 operations with 99.9% SLA on any environment, whether in data-centers, public clouds, or at the edge.

Docker

Back to the Top

Container Architecture. Source: Containerd.io

Docker Learning Resources

Docker Training Program

Docker Certified Associate (DCA) certification

Docker Documentation | Docker Documentation

The Docker Workshop

Docker Courses on Udemy

Docker Courses on Coursera

Docker Courses on edX

Docker Courses on Linkedin Learning

Docker Tools

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly working in collaboration with cloud, Linux, and Windows vendors, including Microsoft.

Docker Enterprise is a subscription including software, supported and certified container platform for CentOS, Red Hat Enterprise Linux (RHEL), Ubuntu, SUSE Linux Enterprise Server (SLES), Oracle Linux, and Windows Server 2016, as well as for cloud providers AWS and Azure. In November 2019 Docker's Enterprise Platform business was acquired by Mirantis.

Docker Desktop is an application for MacOS and Windows machines for the building and sharing of containerized applications and microservices. Docker Desktop delivers the speed, choice and security you need for designing and delivering containerized applications on your desktop. Docker Desktop includes Docker App, developer tools, Kubernetes and version synchronization to production Docker Engines.

Docker Hub is the world's largest library and community for container images Browse over 100,000 container images from software vendors, open-source projects, and the community.

Docker Compose is a tool that was developed to help define and share multi-container applications. With Docker Compose, you can create a YAML file to define the services

README truncated. View on GitHub

🔗 More in this category