# DOKUMEN SISTEM KERJA OPERASIONAL & SOP TIM TEKNIS
## ARSITEKTUR BACKUP GFS MARIABACKUP STREAM ZSTD AUTO-VAULT (ISO 27001 & ISO 22301)

---

### 1. METADATA & TOPOLOGI SISTEM (3-TIER ARCHITECTURE)

| Peran Server | IP Address / Host | OS / Spesifikasi | Fungsi Utama |
| :--- | :--- | :--- | :--- |
| **Server 1: Produksi DB** | `192.168.5.14` | MariaDB 10.3.39 (Debian/Linux) | Menyajikan data fisik 90 database ERP Everest (446 GB mentah). |
| **Server 2: Backup Engine** | `192.168.5.10` (`root@erp`) | AlmaLinux / 24 Cores / 78 GB RAM | Mengolah pemadatan kompresi ZSTD 16-thread & eksekusi SSH stream. |
| **Server 3: Storage Vault** | `192.168.5.15` (`opc windows`) | Windows Server / Shared CIFS (4.1 TB) | Menyimpan arsip jangka panjang (`gfs_database_backup`) & checksum. |

---

### 2. ALUR SISTEM KERJA SKRIP BASH (`backup_mariabackup_gfs.sh`)

```mermaid
graph TD
    Start([Mulai Eksekusi Cronjob 02:00 WIB]) --> Step1[1. Baca Konfigurasi /etc/backup_config/backup.ini]
    Step1 --> Step2{2. Penguncian flock /var/run/backup.lock}
    Step2 -->|Proses Lain Berjalan| Abort1[Gagal: Exit 1 Overlap Error]
    Step2 -->|Lock Berhasil| Step3{3. Cek Auto-Mount CIFS //192.168.5.15/}
    Step3 -->|Belum Mounted| MountCIFS[Mount CIFS ke /mnt/storage_vault]
    Step3 -->|Sudah Mounted| Step4{4. Pre-flight Test SSH 192.168.5.14}
    MountCIFS --> Step4
    Step4 -->|SSH Timeout/Down| Abort2[Gagal: Exit 1 SSH Error]
    Step4 -->|SSH OK| Step5{5. Evaluasi Skenario Backup}
    Step5 -->|Minggu / Full Belum Ada| FullBackup[Full Backup: Stream SSH mariabackup 10.3 -> ZSTD 16-Thread]
    Step5 -->|Senin - Sabtu| IncBackup[Incremental Backup: Stream mariabackup --incremental-lsn]
    FullBackup --> Step6[6. Ekstraksi Checkpoint LSN Presisi]
    IncBackup --> Step6
    Step6 --> Step7[7. Hitung Checksum SHA256 & Transfer ke Windows Vault 4.1 TB]
    Step7 --> Step8{8. Verifikasi Berkas Terkonfirmasi di Vault}
    Step8 -->|Sukses| CleanTransit[Hapus Berkas Transit Lokal di Server 2]
    CleanTransit --> End([Selesai: Log Exit 0])
```

---

### 3. DETAIL RUJUKAN ELEMEN KODE SCRIPT (`backup_mariabackup_gfs.sh`)

#### A. Keamanan Kredensial & Terisolasi (`/etc/backup_config/backup.ini`)
* **Fungsi:** Mencegah pencatatan password *plain-text* dalam repositori atau proses sistem (`ps aux`).
* **Implementasi:** Parameter `db_host`, `db_user`, `db_pass`, `remote_ip`, `remote_pass` dibaca otomatis menggunakan `grep` & `cut` dari berkas terkunci (`chmod 600`).

#### B. Penguncian Overlap Proses (*Concurrency Control*)
* **Fungsi:** Menjamin tidak ada 2 proses backup yang berjalan bersamaan saat eksekusi backup lama memakan waktu.
* **Implementasi:** `flock -n 200` mengunci berkas sistem `/var/run/backup_mariabackup.lock`.

#### C. Penanganan Kegagalan Koneksi (*Fail-Fast Pattern*)
* **Fungsi:** Mencegah pembuatan file cadangan korupsi jika koneksi SSH ke Server Produksi terputus.
* **Implementasi:** `ssh -q -o ConnectTimeout=5 root@"$DB_HOST" exit` dipanggil sebelum memicu pipa stream.

#### D. Penentuan Skenario Backup Pintar (GFS - Grandfather-Father-Son)
* **Full Backup:** Berjalan otomatis setiap hari Minggu (`DAY_OF_WEEK = 7`) atau saat stempel LSN belum ada.
* **Incremental Backup:** Berjalan hari Senin–Sabtu, hanya menarik data delta perubahan sejak LSN sebelumnya.

#### E. Ekstraksi Checkpoint LSN Presisi
* **Fungsi:** Menghilangkan selisih 9 byte checkpoint InnoDB pada pemulihan multi-incremental.
* **Implementasi:** Memprioritaskan penangkapan `checkpoint LSN` dari log remote MariaDB.

#### F. Integrasi Storage Vault 4.1 TB & SHA256 Checksum
* **Fungsi:** Menyimpan arsip di Server Storage Windows (`192.168.5.15`) secara langsung.
* **Implementasi:** Memproduksi berkas `.sha256`, menyalin ke `/mnt/storage_vault/`, memverifikasi keberadaan berkas di Windows Storage, dan membersihkan salinan lokal di Server Backup Engine.

#### G. Prinsip Arsitektur Enterprise: Incremental Harian vs Binary Log (BINLOG) Per Jam
* **MariaBackup GFS (Harian 02:00 WIB):** Berfungsi sebagai *Physical Baseline* agar RTO restore tetap super cepat (hanya menempelkan 1–6 file incremental harian). Tidak dijalankan per jam agar tidak menciptakan rantai 144 file incremental/minggu yang memperlambat restore.
* **Binary Log / BINLOG (Per Jam / Real-Time):** Digunakan untuk merekam setiap transaksi SQL detik demi detik. Saat bencana total, tim teknis melakukan restore fisik MariaBackup Harian, lalu memutar BINLOG jam terakhir via `mysqlbinlog` untuk mencapai **RPO = 0 (Zero Data Loss)**.

---

---

### 4. PANDUAN DISASTER RECOVERY & PEMULIHAN BENCANA TOTAL (SERVER PENGGANTI BARE-METAL)

#### 🚨 ALUR LENGKAP BENCANA TOTAL (SERVER PRODUKSI LAMA MATI TOTAL ➔ RESTORE KE SERVER PENGGANTI BARU)

---

#### **LANGKAH 0: SETUP AWAL SERVER PENGGANTI BARU (FRESH OS PREPARATION)**

Pada Server DB Pengganti yang baru di-install OS (Debian/Ubuntu/Rocky):

```bash
# 1. Install MariaDB Server & Utilitas Pendukung
apt-get update && apt-get install -y mariadb-server mariadb-backup zstd cifs-utils curl

# 2. Buat Mount Point & Mount Storage Vault Windows (192.168.5.15)
mkdir -p /mnt/storage_vault
mount -t cifs //192.168.5.15/gfs_database_backup /mnt/storage_vault \
  -o username=sos,password=NetworkMGK,vers=3.0
```

---

#### **LANGKAH 1: SALIN BERKAS ARSIP DARI VAULT KE DISK LOKAL SERVER PENGGANTI**

Salin berkas Full Backup dan seluruh Incremental yang dibutuhkan dari Storage Vault ke folder lokal Server Pengganti:

```bash
mkdir -p /backup_restore/vault_download

# A. Salin Full Backup & SHA256
cp /mnt/storage_vault/full/full_2026-07-26.mbstream.zst /backup_restore/vault_download/
cp /mnt/storage_vault/full/full_2026-07-26.mbstream.zst.sha256 /backup_restore/vault_download/

# B. Salin Incremental 1 & SHA256
cp /mnt/storage_vault/inc/inc_2026-07-27_040508.mbstream.zst /backup_restore/vault_download/
cp /mnt/storage_vault/inc/inc_2026-07-27_040508.mbstream.zst.sha256 /backup_restore/vault_download/

# C. Salin Incremental 2 & SHA256 (Incremental Terakhir)
cp /mnt/storage_vault/inc/inc_2026-07-28_213032.mbstream.zst /backup_restore/vault_download/
cp /mnt/storage_vault/inc/inc_2026-07-28_213032.mbstream.zst.sha256 /backup_restore/vault_download/
```

---

#### **LANGKAH 2: VERIFIKASI HASH SHA256 DI SERVER PENGGANTI**

```bash
cd /backup_restore/vault_download
sha256sum -c *.sha256
```

---

#### **LANGKAH 3: DEKOMPRESI ZSTD & EKSTRAKSI MBSTREAM DI LOKAL**

```bash
mkdir -p /backup_restore/extracted/full
mkdir -p /backup_restore/extracted/inc1
mkdir -p /backup_restore/extracted/inc2

# Ekstrak Full Backup
zstd -d -c /backup_restore/vault_download/full_2026-07-26.mbstream.zst | mbstream -x -C /backup_restore/extracted/full

# Ekstrak Incremental 1
zstd -d -c /backup_restore/vault_download/inc_2026-07-27_040508.mbstream.zst | mbstream -x -C /backup_restore/extracted/inc1

# Ekstrak Incremental 2 (Terakhir)
zstd -d -c /backup_restore/vault_download/inc_2026-07-28_213032.mbstream.zst | mbstream -x -C /backup_restore/extracted/inc2
```

---

#### **LANGKAH 4: PROSES MARIABACKUP PREPARE (PENGGABUNGAN LOG)**

> ⚠️ **ATURAN EMAS:** Full & Inc 1 gunakan `--apply-log-only`. Inc 2 (terakhir) **LEPAS** `--apply-log-only`.

```bash
# Step 4A: Prepare Full Backup
mariabackup --prepare --target-dir=/backup_restore/extracted/full --apply-log-only

# Step 4B: Tempel Incremental 1
mariabackup --prepare --target-dir=/backup_restore/extracted/full --incremental-dir=/backup_restore/extracted/inc1 --apply-log-only

# Step 4C: Tempel Incremental 2 (Lepas --apply-log-only pada Incremental Terakhir)
mariabackup --prepare --target-dir=/backup_restore/extracted/full --incremental-dir=/backup_restore/extracted/inc2
```

---

#### **LANGKAH 5: COPY-BACK KE DIREKTORI MARIADB & START SERVICE**

```bash
# 1. Hentikan Service MariaDB Bawaan
systemctl stop mariadb

# 2. Bersihkan Direktori Data MySQL Bawaan
rm -rf /var/lib/mysql/*

# 3. Copy-Back Data Hasil Prepare
mariabackup --copy-back --target-dir=/backup_restore/extracted/full

# 4. Perbaiki Hak Akses Kepemilikan (Wajib agar MariaDB mau start)
chown -R mysql:mysql /var/lib/mysql
chmod 755 /var/lib/mysql

# 5. Start Service MariaDB
systemctl start mariadb
systemctl status mariadb
```

---

#### **LANGKAH 6: VERIFIKASI SQL & PEMBERSIHAN TRANSIT**

```bash
# 1. Uji Database & Tabel
mysqladmin ping
mysql -u root -e "SHOW DATABASES; USE run_san_ibb; SHOW TABLES;"

# 2. Bersihkan Berkas Transit Download & Extract
rm -rf /backup_restore
```

---

### 5. CHECKLIST PEMELIHARAAN CRONJOB TIM TEKNIS

* **Jadwal Cronjob (Server Backup `root@erp`):**
  ```bash
  0 2 * * * /home/sos/backup_mariabackup_gfs.sh >> /backupDB/logs/cron_execution.log 2>&1
  ```
* **Pengecekan Log Harian:**
  ```bash
  cat /backupDB/logs/backup_$(date +%Y-%m-%d).log
  ```
* **Verifikasi Kapasitas Storage Windows:**
  ```bash
  df -h /mnt/storage_vault
  ```