Installazione

HOW-TO — CEPH RGW S3 SU PROXMOX VE 9.2 / CEPH 20.2.x TENTACLE
================================================================

Cluster:
- CephS3-1
- CephS3-2
- CephS3-3

Configurazione RGW:
- Realm: eagle
- Zonegroup: eagle-zg
- Zone: eagle-zone
- Porta S3 HTTP: 7480

Prerequisiti:
- Cluster Proxmox già operativo
- Ceph già installato
- 3 MON, uno per nodo
- 3 MGR, uno per nodo
- OSD già creati
- ceph -s = HEALTH_OK

📦 1. INSTALLARE RADOSGW
========================

ESEGUIRE SU: CephS3-1

apt update
apt install -y radosgw

ESEGUIRE SU: CephS3-2

apt update
apt install -y radosgw

ESEGUIRE SU: CephS3-3

apt update
apt install -y radosgw

VERIFICA SU: TUTTI I NODI

apt policy radosgw

🔑 2. CREARE LE IDENTITÀ CEPH DEI TRE RGW
=========================================

ESEGUIRE SOLO SU: CephS3-1

ceph auth get-or-create client.CephS3-1 \
mon 'allow rw' \
osd 'allow rwx'

ceph auth get-or-create client.CephS3-2 \
mon 'allow rw' \
osd 'allow rwx'

ceph auth get-or-create client.CephS3-3 \
mon 'allow rw' \
osd 'allow rwx'

VERIFICA SU: CephS3-1

ceph auth get client.CephS3-1
ceph auth get client.CephS3-2
ceph auth get client.CephS3-3

Risultato atteso per ogni client:

caps mon = "allow rw"
caps osd = "allow rwx"

📁 3. CREARE DIRECTORY E KEYRING LOCALI
=======================================

ESEGUIRE SU: CephS3-1

install -d -o ceph -g ceph \
/var/lib/ceph/radosgw/ceph-CephS3-1

ceph auth get client.CephS3-1 \
-o /var/lib/ceph/radosgw/ceph-CephS3-1/keyring

chown ceph:ceph \
/var/lib/ceph/radosgw/ceph-CephS3-1/keyring

chmod 600 \
/var/lib/ceph/radosgw/ceph-CephS3-1/keyring

ESEGUIRE SU: CephS3-2

install -d -o ceph -g ceph \
/var/lib/ceph/radosgw/ceph-CephS3-2

ceph auth get client.CephS3-2 \
-o /var/lib/ceph/radosgw/ceph-CephS3-2/keyring

chown ceph:ceph \
/var/lib/ceph/radosgw/ceph-CephS3-2/keyring

chmod 600 \
/var/lib/ceph/radosgw/ceph-CephS3-2/keyring

ESEGUIRE SU: CephS3-3

install -d -o ceph -g ceph \
/var/lib/ceph/radosgw/ceph-CephS3-3

ceph auth get client.CephS3-3 \
-o /var/lib/ceph/radosgw/ceph-CephS3-3/keyring

chown ceph:ceph \
/var/lib/ceph/radosgw/ceph-CephS3-3/keyring

chmod 600 \
/var/lib/ceph/radosgw/ceph-CephS3-3/keyring

⚙️ 4. CONFIGURARE IL PERCORSO DEI KEYRING
==========================================

NOTA:
Il file /etc/pve/ceph.conf è condiviso dal cluster Proxmox.

ESEGUIRE SOLO SU: CephS3-1

nano /etc/pve/ceph.conf

Aggiungere in fondo:

[client.CephS3-1]
keyring = /var/lib/ceph/radosgw/ceph-CephS3-1/keyring

[client.CephS3-2]
keyring = /var/lib/ceph/radosgw/ceph-CephS3-2/keyring

[client.CephS3-3]
keyring = /var/lib/ceph/radosgw/ceph-CephS3-3/keyring

VERIFICA SU: CephS3-1

radosgw --name client.CephS3-1 --show-config-value keyring

Risultato atteso:

/var/lib/ceph/radosgw/ceph-CephS3-1/keyring

VERIFICA SU: CephS3-2

radosgw --name client.CephS3-2 --show-config-value keyring

VERIFICA SU: CephS3-3

radosgw --name client.CephS3-3 --show-config-value keyring

🌍 5. CREARE REALM, ZONEGROUP E ZONE
====================================

ESEGUIRE SOLO SU: CephS3-1

radosgw-admin realm create \
--rgw-realm=eagle \
--default

radosgw-admin zonegroup create \
--rgw-zonegroup=eagle-zg \
--rgw-realm=eagle \
--master \
--default

radosgw-admin zone create \
--rgw-zonegroup=eagle-zg \
--rgw-zone=eagle-zone \
--master \
--default

radosgw-admin period update \
--rgw-realm=eagle \
--commit

VERIFICA SU: CephS3-1

radosgw-admin realm list
radosgw-admin zonegroup list
radosgw-admin zone list

Risultato atteso:

Realm:
eagle

Zonegroup:
eagle-zg

Zone:
eagle-zone

🌐 6. ASSOCIARE I TRE RGW ALLA STESSA ZONE
==========================================

ESEGUIRE SOLO SU: CephS3-1

ceph config set client.CephS3-1 rgw_realm eagle
ceph config set client.CephS3-1 rgw_zonegroup eagle-zg
ceph config set client.CephS3-1 rgw_zone eagle-zone
ceph config set client.CephS3-1 rgw_frontends "beast port=7480"

ceph config set client.CephS3-2 rgw_realm eagle
ceph config set client.CephS3-2 rgw_zonegroup eagle-zg
ceph config set client.CephS3-2 rgw_zone eagle-zone
ceph config set client.CephS3-2 rgw_frontends "beast port=7480"

ceph config set client.CephS3-3 rgw_realm eagle
ceph config set client.CephS3-3 rgw_zonegroup eagle-zg
ceph config set client.CephS3-3 rgw_zone eagle-zone
ceph config set client.CephS3-3 rgw_frontends "beast port=7480"

🔎 7. VERIFICARE IL FRONTEND RGW
================================

ESEGUIRE SU: CephS3-1

radosgw \
--name client.CephS3-1 \
--show-config-value rgw_frontends

ESEGUIRE SU: CephS3-2

radosgw \
--name client.CephS3-2 \
--show-config-value rgw_frontends

ESEGUIRE SU: CephS3-3

radosgw \
--name client.CephS3-3 \
--show-config-value rgw_frontends

Risultato atteso su tutti:

beast port=7480

🚀 8. ABILITARE E AVVIARE RGW
=============================

ESEGUIRE SU: CephS3-1

systemctl enable ceph-radosgw@CephS3-1.service
systemctl restart ceph-radosgw@CephS3-1.service

ESEGUIRE SU: CephS3-2

systemctl enable ceph-radosgw@CephS3-2.service
systemctl restart ceph-radosgw@CephS3-2.service

ESEGUIRE SU: CephS3-3

systemctl enable ceph-radosgw@CephS3-3.service
systemctl restart ceph-radosgw@CephS3-3.service

✅ 9. VERIFICARE I SERVIZI
==========================

ESEGUIRE SU: CephS3-1

systemctl status ceph-radosgw@CephS3-1.service --no-pager

ESEGUIRE SU: CephS3-2

systemctl status ceph-radosgw@CephS3-2.service --no-pager

ESEGUIRE SU: CephS3-3

systemctl status ceph-radosgw@CephS3-3.service --no-pager

Risultato atteso:

Active: active (running)

🔌 10. VERIFICARE LA PORTA S3
=============================

ESEGUIRE SU: TUTTI I NODI

ss -lntp | grep 7480

Risultato atteso:

LISTEN ... 0.0.0.0:7480 ... radosgw
LISTEN ... [::]:7480 ... radosgw

Endpoint disponibili:

http://IP-CephS3-1:7480
http://IP-CephS3-2:7480
http://IP-CephS3-3:7480

🧪 11. TEST HTTP DEL GATEWAY
============================

ESEGUIRE SU: QUALSIASI NODO

curl -v http://127.0.0.1:7480/

Oppure verso un altro nodo:

curl -v http://IP_DEL_NODO:7480/

Una risposta XML S3, anche con errore di autenticazione,
indica che RGW sta rispondendo.

💾 12. VERIFICARE I POOL RGW
============================

ESEGUIRE SU: QUALSIASI NODO

ceph osd pool ls | grep rgw

ceph -s

Risultato atteso:

health: HEALTH_OK

👤 13. CREARE IL PRIMO UTENTE S3
================================

ESEGUIRE SOLO SU: CephS3-1

radosgw-admin user create \
--uid=s3admin \
--display-name="S3 Admin"

Conservare:

access_key
secret_key

Per rivedere i dati:

radosgw-admin user info \
--uid=s3admin

🧪 14. TEST S3 CON AWS CLI
==========================

ESEGUIRE SU: UN CLIENT CON AWS CLI INSTALLATO

export AWS_ACCESS_KEY_ID='ACCESS_KEY'
export AWS_SECRET_ACCESS_KEY='SECRET_KEY'
export AWS_DEFAULT_REGION='us-east-1'

Elenco bucket:

aws \
--endpoint-url http://IP-CephS3-1:7480 \
s3 ls

Creazione bucket:

aws \
--endpoint-url http://IP-CephS3-1:7480 \
s3 mb s3://test-bucket

Creazione file:

echo "Test Ceph S3" > test.txt

Upload:

aws \
--endpoint-url http://IP-CephS3-1:7480 \
s3 cp test.txt s3://test-bucket/

Elenco contenuto bucket:

aws \
--endpoint-url http://IP-CephS3-1:7480 \
s3 ls s3://test-bucket/

Download:

aws \
--endpoint-url http://IP-CephS3-1:7480 \
s3 cp s3://test-bucket/test.txt /tmp/test-s3.txt

Verifica:

cat /tmp/test-s3.txt

🏗️ 15. RISULTATO FINALE
========================

CephS3-1
MON
MGR
OSD
RGW :7480

CephS3-2
MON
MGR
OSD
RGW :7480

CephS3-3
MON
MGR
OSD
RGW :7480

Configurazione RGW condivisa:

Realm: eagle
Zonegroup: eagle-zg
Zone: eagle-zone

Endpoint S3:

http://CephS3-1:7480
http://CephS3-2:7480
http://CephS3-3:7480

⭐ STEP SUCCESSIVO CONSIGLIATO
=============================

Creare un endpoint unico e altamente disponibile: