#Spark-streaming
Showing 20 of 20 repositories tagged #spark-streaming, ranked by stars
A Flexible and Powerful Parameter Server for large-scale machine learning
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
:star2: :sparkles: Analyze and visualize Twitter Sentiment on a world map using Spark MLlib
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
Custom PySpark Connectors
This project walks through how you can create recommendations using Apache Spark machine learning. There are a number of jupyter notebooks that you can run on IBM Data Science Experience, and there a live demo of a movie recommendation web application you can interact with. The demo also uses IBM Message Hub (kafka) to push application events to topic where they are consumed by a spark streaming job running on IBM BigInsights (hadoop).
A data engineering project (Twitter monitor app)
For a series of posts on Amazon MSK, Amazon EKS, and Amazon EMR
A data and analytics engineering platform designed for real-time sports betting analytics.
Toolbox for building Generative AI applications on top of Apache Spark.
code, labs and lectures for the course
A Docker Compose template that builds a interactive development environment for PySpark with Jupyter Lab, MinIO as object storage, Hive Metastore, Trino and Kafka
Apache Spark Guide
The demo of using Kafka, Spark, Hive, Cassandra, etc by using Docker. It produces the production ready environment for any kinds of big data project relates to Hadoop ecosystem
Solution for IBM Data Engineering Professional Certificate
velib-v2: An ETL pipeline that employs batch and streaming jobs using Spark, Kafka, Airflow, and other tools, all orchestrated with Docker Compose.
A structured streaming was applied to the robot data from ROS-Gazebo simulation environment using Apache Spark. Data is collected in Kafka, analyzed by Apache Spark and stored in Cassandra.
Spark-Kafka Realtime recommender Engine.
This project builds a scalable log analytics pipeline use Lambda architecture for real-time and batch processing of NASA server logs.
Проекты курса Инженер данных на платформе Yandex Practicum