Connecting Oxla to MinIO

Overview

This guide provides step-by-step instructions on how to connect MinIO Single-Node Single-Drive setup to Oxla using a Docker container.

Configuration

Starting MinIO Service

If MinIO is not installed on your machine, you can download and install it by referring to the instructions provided in the MinIO Single-Node Single-Drive documentation. Once that is done, you can start the MinIO service by running this command:

sudo systemctl start minio.service

Accessing MinIO console

To access the MinIO console, navigate to the localhost:9001 URL in your web browser. Once there, use your MinIO credentials to log in.

Creating Access Keys

After logging into the MinIO console, navigate to the Access Keys > Create access key tab. Here, you have the option to either let MinIO auto-generate access and secret keys or specify custom values yourself. Once you’ve entered all the necessary details, proceed by clicking the Create button to save the changes. It’s crucial to save the generated credentials for later use.

Creating a Bucket

In the MinIO console, click Buckets > Create Bucket tab. Next, provide a unique name for your bucket, such as “oxla-bucket” and then click the Create button to finalize the creation of your new bucket.

Make sure that the object storage bucket directory is either completely empty or contains only files in the Oxla format

Running Oxla Docker Container (Single-Node)

Open your terminal
Pull and run the Oxla Docker container using the following command:

        docker run \
            -e STORAGE__S3__HTTP=http \
            -e STORAGE__S3__ENDPOINT=http://127.0.0.1:9000 \
            -e OXLA_HOME=s3://{MINIO_BUCKET_NAME} \
            -e AWS_ACCESS_KEY_ID={MINIO_ACCESS_KEY} \
            -e AWS_SECRET_ACCESS_KEY={MINIO_SECRET_KEY} \
            --network host \
            public.ecr.aws/oxla/release:latest

Replace the following placeholders with your actual values:

MINIO_BUCKET_NAME: desired name for your bucket (e.g. “oxla-bucket”)
MINIO_ACCESS_KEY: your MinIO access key
MINIO_SECRET_KEY: your MinIO secret key

Configuring Oxla Multi-Node Cluster

Oxla supports multi-node deployment to enable clustering across multiple machines, providing scalability. This section shows you how to set up a multi-node Oxla cluster using Docker Compose.

Creating Docker Compose file

Create a configuration file named oxla_multi_node.yml using your preferred text editor:

vim oxla_multi_node.yaml

This command opens the vim editor, allowing you to create and edit the YAML file. Copy and paste the following configuration into the file. Replace the placeholders values with your actual settings.

services:
  oxla_node_1:
    image: public.ecr.aws/oxla/release:latest
    ports:
      - 5432:5432
    environment:
      - HOST_NAME=oxla_node_1
      - OXLA_NODES=${COMPOSE_PROJECT_NAME}-oxla_node_1-1;${COMPOSE_PROJECT_NAME}-oxla_node_2-1;${COMPOSE_PROJECT_NAME}-oxla_node_3-1
      - OXLA_HOME=s3://oxla-bucket/
      - AWS_ACCESS_KEY_ID={MINIO_ACCESS_KEY}
      - AWS_SECRET_ACCESS_KEY={MINIO_SECRET_KEY}
      - LOG_LEVEL=Debug
      - STORAGE__S3__ENDPOINT=http://172.17.0.1:9000
      - LEADER_ELECTION__LEADER_NAME=oxla_node_1

  oxla_node_2:
    image: public.ecr.aws/oxla/release:latest
    ports:
      - 5500:5432
    environment:
      - HOST_NAME=oxla_node_2
      - OXLA_NODES=${COMPOSE_PROJECT_NAME}-oxla_node_1-1;${COMPOSE_PROJECT_NAME}-oxla_node_2-1;${COMPOSE_PROJECT_NAME}-oxla_node_3-1
      - OXLA_HOME=s3://oxla-bucket/
      - AWS_ACCESS_KEY_ID={MINIO_ACCESS_KEY}
      - AWS_SECRET_ACCESS_KEY={MINIO_SECRET_KEY}
      - LOG_LEVEL=Debug
      - STORAGE__S3__ENDPOINT=http://172.17.0.1:9000
      - LEADER_ELECTION__LEADER_NAME=oxla_node_1

  oxla_node_3:
    image: public.ecr.aws/oxla/release:latest
    ports:
      - 5600:5432
    environment:
      - HOST_NAME=oxla_node_3
      - OXLA_NODES=${COMPOSE_PROJECT_NAME}-oxla_node_1-1;${COMPOSE_PROJECT_NAME}-oxla_node_2-1;${COMPOSE_PROJECT_NAME}-oxla_node_3-1
      - OXLA_HOME=s3://oxla-bucket/
      - AWS_ACCESS_KEY_ID={MINIO_ACCESS_KEY}
      - AWS_SECRET_ACCESS_KEY={MINIO_SECRET_KEY}
      - LOG_LEVEL=Debug
      - STORAGE__S3__ENDPOINT=http://172.17.0.1:9000
      - LEADER_ELECTION__LEADER_NAME=oxla_node_1

Save and close the file by typing :wq and pressing Enter to exit the editor.

Make sure that the ports for inter-node communication and port 5432 for Oxla access are open

Replace the placeholders with your actual values:

{MINIO_BUCKET_NAME}: your desired bucket name (e.g. “oxla-bucket”)
{MINIO_ACCESS_KEY}: your MinIO access key
{MINIO_SECRET_KEY}: your MinIO secret key
STORAGE__S3__ENDPOINT: your bridge network created by Docker (e.g. 172.17.0.1).
Use ip addr show docker0 to find the docker0 network interface

Setting Environment Variables

Oxla uses several environment variables to configure its connection to MinIO and to manage the cluster behavior. Below is a list of the most important variables you will need to set when running Oxla’s multi-node configuration.

Variable	Description
`HOST_NAME`	Unique hostname for Oxla node (required for multi-node setups)
`OXLA_NODES`	Semicolon-separated (;) list of IP addresses of all nodes in the cluster (multi-node only)
`OXLA_HOME`	Root directory for Oxla’s data storage in MinIO S3 bucket
`AWS_ACCESS_KEY_ID`	Access key ID for authenticating with MinIO storage
`AWS_SECRET_ACCESS_KEY`	Secret access key for authenticating with MinIO storage
`STORAGE__S3__ENDPOINT`	MinIO server endpoint URL including protocol and port
`LEADER_ELECTION__LEADER_NAME`	Hostname of the leader node in a multi-node cluster

Starting Multi-Node Cluster

Use Docker Compose to deploy your configuration and execute the following command:

docker compose -f oxla_3_node_config.yaml up

This command initializes and starts the Oxla nodes defined in your configuration file. Docker Compose reads the YAML file, creates three separate Oxla nodes (oxla_node_1, oxla_node_2 and oxla_node_3) and configures them according to the defined environment variables and port mappings. Each node runs in its own container and Docker Compose manages their interconnections, ensuring they can communicate as a cluster.

Example

Let’s assume that you want to create a “film” table in your Oxla database that stores metadata about films in a MinIO bucket. To query this table, you can use the psql application, but first you’ll need to go through Oxla’s access control:

psql -h localhost -U oxla oxla

The default username and password are both set to “oxla”

You can connect to different nodes by specifying the corresponding port number as configured in the Docker Compose file. For this example, use the following ports to connect to their respective nodes:

Port 5432 connects to oxla_node_1
Port 5500 connects to oxla_node_2
Port 5600 connects to oxla_node_3

For the needs of this section, we’re going to use a simplified version of the film table from the Pagila database, containing only the title, length and rating columns. The complete schema for the film table can be found on the Pagila database website.

DROP TABLE IF EXISTS film;
CREATE TABLE film (
  title text NOT NULL,
  rating text,
  length int
);
INSERT INTO film(title, length, rating) VALUES
  ('ATTRACTION NEWTON', 83, 'PG-13'),
  ('CHRISTMAS MOONSHINE', 150, 'NC-17'),
  ('DANGEROUS UPTOWN', 121, 'PG'),
  ('KILL BROTHERHOOD', 54, 'G'),
  ('HALLOWEEN NUTS', 47, 'PG-13'),
  ('HOURS RAGE', 122, 'NC-17'),
  ('PIANIST OUTFIELD', 136, 'NC-17'),
  ('PICKUP DRIVING', 77, 'G'),
  ('INDEPENDENCE HOTEL', 157, 'NC-17'),
  ('PRIVATE DROP', 106, 'PG'),
  ('SAINTS BRIDE', 125, 'G'),
  ('FOREVER CANDIDATE', 131, 'NC-17'),
  ('MILLION ACE', 142, 'PG-13'),
  ('SLEEPY JAPANESE', 137, 'PG'),
  ('WRATH MILE', 176, 'NC-17'),
  ('YOUTH KICK', 179, 'NC-17'),
  ('CLOCKWORK PARADISE', 143, 'PG-13');

SELECT * FROM film;

Running the query above will return all rows from your film table that are stored in your MinIO bucket:

        title        | rating | length 
---------------------+--------+--------
 ATTRACTION NEWTON   | PG-13  |     83
 CHRISTMAS MOONSHINE | NC-17  |    150
 DANGEROUS UPTOWN    | PG     |    121
 KILL BROTHERHOOD    | G      |     54
 HALLOWEEN NUTS      | PG-13  |     47
 HOURS RAGE          | NC-17  |    122
 PIANIST OUTFIELD    | NC-17  |    136
 PICKUP DRIVING      | G      |     77
 INDEPENDENCE HOTEL  | NC-17  |    157
 PRIVATE DROP        | PG     |    106
 SAINTS BRIDE        | G      |    125
 FOREVER CANDIDATE   | NC-17  |    131
 MILLION ACE         | PG-13  |    142
 SLEEPY JAPANESE     | PG     |    137
 WRATH MILE          | NC-17  |    176
 YOUTH KICK          | NC-17  |    179
 CLOCKWORK PARADISE  | PG-13  |    143
(17 rows)

Additional Resources

For more information on Oxla’s environment variables check out the Configuration file section.

Home

Introduction

Development Guide

Resources

Overview

Configuration

Starting MinIO Service

Accessing MinIO console

Creating Access Keys

Creating a Bucket

Running Oxla Docker Container (Single-Node)

Configuring Oxla Multi-Node Cluster

Creating Docker Compose file

Setting Environment Variables

Starting Multi-Node Cluster

Example

Additional Resources

Home

Introduction

Development Guide

Resources

​Overview

​Configuration

​Starting MinIO Service

​Accessing MinIO console

​Creating Access Keys

​Creating a Bucket

​Running Oxla Docker Container (Single-Node)

​Configuring Oxla Multi-Node Cluster

​Creating Docker Compose file

​Setting Environment Variables

​Starting Multi-Node Cluster

​Example

​Additional Resources

Overview

Configuration

Starting MinIO Service

Accessing MinIO console

Creating Access Keys

Creating a Bucket

Running Oxla Docker Container (Single-Node)

Configuring Oxla Multi-Node Cluster

Creating Docker Compose file

Setting Environment Variables

Starting Multi-Node Cluster

Example

Additional Resources