Run Multi-node oxla cluster with host mounted network storage

Over the following documentation article, we’ll set you up with an Oxla cluster that has network mounted. This setup might be useful in case of network storage already existing in your environment. To maximize performance, we highly encourage you to setup Oxla with dedicated blob storage, either cloud provided or self-hosted e.g. MinIO or ceph.

Prerequisites

• Single node per machine
• Docker installed
• x86 64bit CPU (Intel or AMD)
• PostgresSQL client installed
• N machines with ssh access to them (N: number of nodes that you want to deploy)
• Networking configured so that all nodes can connect to each other (by default OXLA requires 5769 and 5770 ports for inter node communication and exposes sql interface on 5432 port)
• Network storage mounted and available on all machines that will be hosting Oxla

Installation on Each Node

Firstly create multi_node.yml file on each node and replace placeholder values with appropriate ones:

• /path/to/network/storage/mount: path to the directory that shall be used for Oxla home on network storage mounted in host os
• NETWORK__HOST_NAME: Oxla node hostname (must be unique for every node)
• LEADER_ELECTION__LEADER_NAME: hostname of the node that will be the leader of the cluster
• OXLA_NODES: ; separated list of the addresses of of all the nodes in the cluster

version: '3.5'

volumes:
  oxla_data:

services:
  oxla_node:
    image: public.ecr.aws/oxla/release:latest
    security_opt:
      - seccomp:unconfined
    ulimits:
      nofile:
        soft: 40000
        hard: 40000
    ports:
      - 5432:5432
    volumes:
      - /path/to/network/storage/mount:/oxla/data
    environment:
      - NETWORK__HOST_NAME=oxla_node_1
      - LEADER_ELECTION__LEADER_NAME=oxla_node_1
      - OXLA_NODES=192.168.0.1;192.168.0.2;192.168.0.3

Now you can execute the following command to create and start the docker container:

docker compose -f multi_node.yml up

Connecting to Oxla

You can now connect to the cluster using the following command:

psql -h IP_ADDRESS oxla oxla

Important Notes

• Ensure that all the machines you are using to deploy Oxla on, are connected in the same network
• Replace IP_ADDRESS with one of the IP addresses that you set up in your YAML file
• Ensure that you repeat all of the steps above on all of the nodes