Distributed immutable data store with strong encryption and authentication
ATE ===
Navigation
What is ATE?
...is it a NoSQL database? ...is it a distributed redo log? ...is it a event BUS? ...is it a API framework? ...is it a distributed queue? ...is it a distributed cache? ...is it a secure encrypted vault? ...is it a quantum resistant communication framework? ...is it a WORM archive solution? ATE is all these things and none of them; it is unique way to work with distributed data that can easily implement all of the above use cases - take a look at the examples for how you can achieve them.
Why the name?
The origin of the word "mutate" is the latin word '-ate': https://www.dictionary.com/browse/mutate
Summary
ATE is a distributed immutable data store and built in memory based materialized view with strong encryption and authentication.
What does that mean?
This library is a way of working with data in modern distributed computing.
- ...data is persisted to a distributed commit log.
- ...partitions are divided into chains that shard data into physical domains.
- ...streaming of data to the application occurs on demand as the app needs it.
- ...each chain is a crypto-graph with unique asymmetric keys at differentiating nodes.
- ...the root of the chain-of-trust validates the crypto-graph through various plugins.
- ...strong authentication and authorized is by design built into the data model.
- ...encryption is highly resistant to quantum attacks and uses fine-grained tenant keys.
- ...all this is integrated into a shared-nothing highly portable executable.
Examples
- Typical Client
- Hello World
- Server/Client
- Publish/Subscribe
- Service/API
- Exactly-Once Queue
- Crypto Currency
- Banking Example
Projects
Typical Deployment Pattern
.-------------. .- - - - - - -.
| Server | Server
| | .. .. .. | | .. .. ..
| >atedb solo |
'------|----\-' '- - - - - - -'
| \
ws://yourserver.com/db
| \
.------|------. \
|Native Client| .-----Browser-----.
| | |.---------------.|
| >program | || >wasm32-wasi ||
| \ate.so | || \ate.wasm ||
'-------------' |'---------------'|
'-----------------'
The easiest way to get up and running is to just build your app and point the database URL at ws://wasmer.sh/db. You will need to register an account and verify your identity however after this you can use the free databases and/or paid option.
Alternatively, if you wish to host your own ATE servers in infrastructure that you manage and run then follow these high-level steps.
- Server runs the 'atedb' process on some network reachable location
- Create several records for each IP address under the same A-record in your DNS
- Either create your own authentication server as well using the auth-server binary
or just use the authentication servers hosted at Wasmer by pointing to
ws://wasmer.sh/auth.Quick Start
Cargo.toml
[dependencies]
tokio = { version = "*", features = ["full", "signal", "process"] }
serde = { version = "*", features = ["derive"] }
ate = { version = "*" }
wasmer-auth = { version = "*" }main.rs
use serde::{Serialize, Deserialize};
use wasmer_auth::prelude::*;
#[derive(Debug, Serialize, Deserialize, Clone)] struct MyData { pi: String, }
#[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let dio = DioBuilder::default() .withsessionprompt().await? .build("mychain") .await?;
dio.store(MyData { pi: "3.14159265359".to_string(), })?; dio.commit().await?;
Ok(()) }
Changelog
1.2.2 -= Code Refactoring =-
+ The crypto library has been split out from the main ATE library to reduce dependencies
when using just cryptographic routines and to reduce build times.
1.2.1 -= Lazy Loading =-
+ Subscribing to chains can now load the data in the chain on demand as its needed
which reduces the startup time considerably.
+ Temporal clients will default to lazy loading
+ Data loaded via the lazy loading mechanism will now be cached client side
1.1.1 -= Performance and Bug Fixes =- + Fixed an issue with the web sockets that caused sporadic disconnects + Improved the performance of web socket messages by reusing IV's + Reduced the message overhead with a new message encoding format
1.1.0 -= Comms Upgrade =- + Streaming websockets are now more stable as they use length headers to delimit messages. + Fixed a bug where disconnecting clients would drop neighbors on the same server. + Various changes to the interfaces for stability reasons (this upgrade is not backwards compatible with version 1.0.6)
1.0.6 -= Bug Fixes =- + Modified the interface slightly but most users should not be impacted + Fixed a bug around validators rejecting events during the subscribe process that re-reads them from disk - these validators should not be running + Added the ability to list all root objects + Added the ability to delete all root objects (and hence wipe a chain) + Fixed a serious deadlock situation when commiting transactions that was causing timeouts
1.0.2 -= WASM BUS =- + Integrated with the WASM bus (wasmer-bus) which allows for ATE to use the web sockets while running in a controlled sandbox.
1.0.0 -= Major Release =- + See README.md
<=0.8.0 See commit history
High Level Design
.--[ atedb ]---. .--[ atedb ]---. .-[auth-server]-. | | | | | | |>local redo-log| |>local redo-log| |>local redo-log| |.-------------.| |.-------------.| |.-------------.| || Chain 1 || || || || user || || || || Chain 2 || || account || |-------------| |------|------| |-----|-------| | | subscribe login | \||_ | | | | >local redo-log | >Crypto-Graph Materiaized View< (in memory) | .----------------------------------. session | | root(hash) | .-----------. | | | | | -token | | | dao----dao(aes) |---| -claims | | | \ | | -keys | | | dao | ----------- | | |
Feature Flags
- 'client' - Client functionality that allows one to connect to ATE datachains
- 'server' - Server functionality required to create and run ATE in distributed
- 'client_web' - Client functionality designed for running within a browser sandbox
WebAssembly
When compiling for WASM use the following command:
cargo build --target wasm32-wasi --no-default-features --features client_webLower Level Example
Cargo.toml
[dependencies]
tokio = { version = "*", features = ["full", "signal", "process"] }
serde = { version = "*", features = ["derive"] }
ate = { version = "*" }main.rs
use serde::{Serialize, Deserialize};
use ate::prelude::*;
#[derive(Clone, Serialize, Deserialize)] struct World { commandment: String }
#[tokio::main] async fn main() -> Result<(), AteError> { // The default configuration will store the redo log locally in the temporary folder let conf = AteConfig::default(); let builder = ChainBuilder::new(&conf).await.build();
// We create a chain with a specific key (this is used for the file name it creates) let chain = builder.open(&ChainKey::from("universe")).await?; // We interact with the data stored in the chain-of-trust using a DIO let session = AteSession::default(); let mut dio = chain.dio(&session).await; // In this example we store some data in the "World" object let key = dio.store(World { commandment: "Hello".to_string(), })?.key().clone(); dio.commit().await?; // Now we retreive the data and print it to console println!("{} world!", dio.load::<World>(&key).await?.commandment);
// All errors in ATE will convert into the AteError Ok(()) }
Contribution
If you would like to help setup a community to continue to develop this project then please contact me at johnathan.sharratt@gmail.com
