#Hdfs
Showing 23 of 23 repositories tagged #hdfs, ranked by stars
SeaweedFS is a distributed storage system for object storage (S3), file systems, and Iceberg tables, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.
Ceph is a distributed object, block, and file storage platform
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
Utils for streaming large files (S3, HDFS, gzip, bz2...)
The Universal Storage Engine
A native go client for HDFS
A pure python HDFS client
80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.
CloudEon uses Kubernetes to install and deploy open-source big data components, enabling the containerized operation of an open-source big data platform. This allows you to reduce your focus on underlying resource management and maintenance.
A distributed storage benchmark for file systems, object stores & block devices with support for GPUs
Quick start: pip install jsoniq โ๏ธ RumbleDB 2.1.0 "Cedrus Libani" ๐ณ for Apache Spark | Run queries on your large-scale, messy datasets (JSON, text, CSV, Parquet, Delta...) | Data Lakehouse with Updates, Scripting, Declarative Machine Learning and more
Declarative text based tool for data analysts and engineers to extract, load, transform and orchestrate their data pipelines.
Data Engineering Project with Hadoop HDFS and Kafka
25+ DevOps CLI Tools - Anonymizer, SQL ReCaser (MySQL, PostgreSQL, AWS Redshift, Snowflake, Apache Drill, Hive, Impala, Cassandra CQL, Microsoft SQL Server, Oracle, Couchbase N1QL, Dockerfiles), Hadoop HDFS & Hive tools, Solr/SolrCloud CLI, Nginx stats & HTTP(S) URL watchers for load-balanced web farms, Linux tools etc.
Big Data for Data Engineers Coursera Specialization from Yandex
Python HDFS client
Analysis scripts for log data sets used in anomaly detection.
The goal of this project is to build a docker cluster that gives access to Hadoop, HDFS, Hive, PySpark, Sqoop, Airflow, Kafka, Flume, Postgres, Cassandra, Hue, Zeppelin, Kadmin, Kafka Control Center and pgAdmin. This cluster is solely intended for usage in a development environment. Do not use it to run any production workloads.
Examples of Flink on Azure
Kubernetes ๅ็็ๆฐๆฎไบคไปๅนณๅฐ
Gaffer Docker images and associated Helm charts for deploying on Kubernetes
This is projects of Cloud Computing Course
JupyterLab Notebook for Mesosphere DC/OS