Guidance for building distributed systems and Microservices in Go

Articles

• Building Event-Driven Distributed Systems in Go with gRPC, NATS JetStream and CockroachDB

Technologies Used:

• Go
• NATS, NATS JetStream, NATS Micro
• gRPC
• CockroachDB
• Go kit
• Zipkin (for distributed tracing)

Compile Proto files

protoc eventstore/*.proto \
		--go_out=. \
		--go-grpc_out=. \
		--goopt=paths=sourcerelative \
		--go-grpcopt=paths=sourcerelative \
		--proto_path=.

Set up CockroachDB

Set up CockroachDB Cluster with three nodes

cockroach start \
--insecure \
--store=orders-1 \
--listen-addr=localhost:26257 \
--http-addr=localhost:8080 \
--join=localhost:26257,localhost:26258,localhost:26259 \
--background

cockroach start \
--insecure \
--store=orders-2 \
--listen-addr=localhost:26258 \
--http-addr=localhost:8081 \
--join=localhost:26257,localhost:26258,localhost:26259 \
--background

cockroach start \
--insecure \
--store=orders-3 \
--listen-addr=localhost:26259 \
--http-addr=localhost:8082 \
--join=localhost:26257,localhost:26258,localhost:26259 \
--background

cockroach init command to perform a one-time initialization of the cluster

cockroach init --insecure --host=localhost:26257

Start a SQL Shell for CockroachDB:

cockroach sql --insecure --host=localhost:26257

Create user

cockroach user set shijuvar --insecure

Create Databases

cockroach sql --insecure -e 'CREATE DATABASE eventstoredb'

cockroach sql --insecure -e 'CREATE DATABASE ordersdb'

Grant privileges to the shijuvar user

cockroach sql --insecure -e 'GRANT ALL ON DATABASE ordersdb TO shijuvar'

cockroach sql --insecure -e 'GRANT ALL ON DATABASE eventstoredb TO shijuvar'

Run NATS JetStream Server

nats-server -js

Run JetStream with config file

nats-server -c js.conf

// js.conf file

jetstream {     // jetstream data will be in /data/jetstream-server/jetstream     store_dir: "/data/jetstream-server"

// 1GB     maxmemorystore: 1073741824
// 10GB     maxfilestore: 10737418240 }
http_port: 8222

Prerequisites for running the eventstream demo

• Run the NATS JetStream Server
• Run CockroachDB
• Run bootstrapper app (eventstream/bootstrapper) as a one time activity for creating stream in JetStream and create two databases in CockroachDB
• In order to use distributed tracing with gokit-ordersvc (order service with Go kit), run Zipkin distributed tracing system

Basic Workflow in the example (eventstream directory):

• A client app post an Order to an HTTP API (ordersvc)
• An HTTP API (ordersvc) receives the order, then executes a command onto Event Store, which is an immutable log of events of domain events, to create an event via its gRPC API (eventstoresvc).
• The Event Store API executes the command and then publishes an event "ORDERS.created" to NATS JetStream server to let other services know that a domain event is created.
• The Payment worker (paymentworker) subscribes the event "ORDERS.created", then make the payment, and then create another event "ORDERS.paymentdebited" via Event Store API.
• The Event Store API executes a command onto Event Store to create an event "ORDERS.paymentdebited" and publishes an event to NATS JetStream server to let other services know that the payment has been debited.
• The Query synchronising worker (querymodelworker) subscribes the event "ORDERS.created" that synchronise the query data model to provide state of the aggregates for query views.
• The review worker (reviewworker) subscribes the event "ORDERS.paymentdebited" and finally approves the order, and then create another event "ORDERS.approved" via Event Store API.
• A Saga coordinator manages the distributed transactions and makes void transactions on failures (to be implemented)

Training and Consulting in India