ReefGuide Documentation

Documentation for ReefGuide, a cloud-based platform for reef management and analysis, deployed on AWS infrastructure.

Getting Started

Deploying with CDK

Guide to deploying ReefGuide infrastructure ‘from-scratch’ using AWS CDK, including prerequisites, configuration setup, and deployment operations.

Component overview

Provides a high level description of the pieces that form ReefGuide.

Development

Locally developing the capacity manager

Guide for developing and debugging the capacity manager locally against AWS infrastructure, including environment setup and configuration generation.

Use cases: Developing capacity manager features, debugging job system issues, testing worker scaling

Unit tests and linting

How to run the unit tests, format and lint code, locally.

Operations & Management

Performing CDK operations on ReefGuide

Once you have a deployed CDK ReefGuide stack, how do you maintain it i.e. run CDK deploy, diff, destroy. This guide explains the high level steps to deploy an update to ReefGuide.

Managing Users

How to manage user registration, roles and bulk changes.

Use cases: Helping users register, pre-authorising users, managing users in bulk

Managing EFS Data

Instructions for reading, writing, and managing files in the shared job system storage using AWS Elastic File System (EFS).

Use cases: Updating data files, managing worker node data, file system operations

Clearing the job cache

Instructions on how to invalidate jobs from the cache when data becomes stale.

Use cases: Invalidating individual job types, invalidating all job types, invalidate cache

Database Migration

Guide for migrating databases using the Prisma ORM, including credential management and verification procedures.

When to use: Schema updates, production database changes

Database Backup and Restore

Guide for dumping and restoring data from the PostgreSQL application database.

When to use: Taking backups, restoring from backups, disaster recovery

Setting up the ReefGuide CLI

Configure the administrative command-line interface for managing ReefGuide instances, including user management and system operations.

Features: User auditing, cache management, administrative tasks

Troubleshooting & Debugging

Debugging ReefGuide

Troubleshooting guide for AWS ECS services, covering web API, capacity manager, and worker nodes with common issue resolution.

Covers: ECS task debugging, log analysis, startup failures, EFS mount issues

Data Specification Reload

Guide for updating the system’s data specifications and clearing cached regional data to reflect new parameter configurations.

When needed: After data updates, parameter changes, cache invalidation

Monitoring

Monitoring

Overview of the monitoring stack including uptime monitoring with Uptime Kuma and error tracking with BugSink.

Components: Uptime monitoring, error tracking, status pages, alerting

System Architecture

ReefGuide is built using:

  • Infrastructure: AWS ECS Fargate, RDS PostgreSQL, EFS, CloudFront
  • Backend: Node.js/Express API with Prisma ORM
  • Frontend: Angular application
  • Workers: Julia-based computation nodes
  • Deployment: AWS CDK for Infrastructure as Code
  • Monitoring: Open-source stack (Uptime Kuma + BugSink)

ReefGuide is part of a larger ecosystem of repositories: