diff --git a/add-on/blackbox-exporter.md b/add-on/blackbox-exporter.md new file mode 100644 index 0000000..ff3b867 --- /dev/null +++ b/add-on/blackbox-exporter.md @@ -0,0 +1,537 @@ +BLACKBOX EXPORTER - GUIDA OPERATIVA KUBERNETES +============================================== + +Stack di partenza +----------------- +Nel cluster e' gia' installato kube-prometheus-stack nel namespace monitoring: + +```text +helm list -A | grep prometheus + +kube-prometheus-stack monitoring 4 2026-08-08 23:55:14.193903052 +0000 UTC deployed kube-prometheus-stack-88.1.5 +``` + +Obiettivo +--------- +Installare prometheus-blackbox-exporter per eseguire test HTTP/HTTPS su servizi Kubernetes interni, Ingress o URL esterni, esportando le metriche verso Prometheus tramite Prometheus Operator. + +Metriche principali: + +```text +probe_success +probe_duration_seconds +probe_http_status_code +probe_http_ssl +probe_dns_lookup_time_seconds +``` + +Legenda +------- +- COMANDO: istruzione da eseguire nel terminale. +- FILE: file da creare o aggiornare. +- NOTA: informazione da verificare prima di procedere. +- OUTPUT ATTESO: risultato indicativo del comando. + +Prerequisiti +------------ +- Accesso kubectl al cluster. +- Helm installato. +- Namespace monitoring presente o creabile. +- kube-prometheus-stack installato con Prometheus Operator. +- CRD Probe disponibile: probes.monitoring.coreos.com. + + +1. VERIFICA PROMETHEUS E PROMETHEUS OPERATOR +============================================ + +COMANDO - verifica release Helm Prometheus +```bash +helm list -A | grep prometheus +``` + +COMANDO - verifica pod Prometheus +```bash +kubectl get pods -A | grep prometheus +``` + +COMANDO - verifica CRD Probe +```bash +kubectl get crd probes.monitoring.coreos.com +``` + +COMANDO - verifica label delle risorse Prometheus +```bash +kubectl get prometheus -A --show-labels +``` + +COMANDO - controlla selector della risorsa Prometheus +```bash +kubectl get prometheus -n monitoring -o yaml +``` + +NOTA - label dei manifest +Dal comando `kubectl get prometheus -A --show-labels` risulta che Prometheus seleziona le risorse con la label `release=kube-prometheus-stack`. +Negli esempi sotto viene quindi usata questa label: + +```yaml +release: kube-prometheus-stack +``` + +Se in futuro il tuo Prometheus usa un selector diverso, sostituisci questa label nei manifest Probe e PrometheusRule. +Controlla soprattutto questi campi nella risorsa Prometheus: + +```yaml +spec: + probeSelector: + probeNamespaceSelector: + ruleSelector: + ruleNamespaceSelector: +``` + + +2. AGGIUNTA REPOSITORY HELM +=========================== + +COMANDO - aggiungi repository prometheus-community +```bash +helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +``` + +COMANDO - aggiorna indice chart Helm +```bash +helm repo update +``` + + +3. CREA FILE VALUES HELM +======================== + +FILE - blackbox-values.yaml + +Descrizione: +- configura i moduli HTTP usati da blackbox-exporter; +- espone blackbox-exporter come Service ClusterIP sulla porta 9115; +- lascia disabilitato ServiceMonitor perche' lo scraping dei target viene configurato tramite Probe. + +COMANDO - crea blackbox-values.yaml +```bash +cat > blackbox-values.yaml <<'EOF' +fullnameOverride: blackbox-exporter + +config: + modules: + http_2xx: + prober: http + timeout: 10s + http: + method: GET + preferred_ip_protocol: ip4 + valid_http_versions: + - HTTP/1.1 + - HTTP/2.0 + valid_status_codes: + - 200 + - 204 + - 301 + - 302 + + http_post_2xx: + prober: http + timeout: 10s + http: + method: POST + preferred_ip_protocol: ip4 + valid_status_codes: + - 200 + - 201 + - 202 + - 204 + + http_k8s_health: + prober: http + timeout: 10s + http: + method: GET + preferred_ip_protocol: ip4 + fail_if_ssl: false + fail_if_not_ssl: false + valid_status_codes: + - 200 + +service: + type: ClusterIP + port: 9115 + +serviceMonitor: + enabled: false + +resources: + requests: + cpu: 50m + memory: 64Mi + limits: + cpu: 200m + memory: 256Mi +EOF +``` + + +4. INSTALLA BLACKBOX EXPORTER +============================= + +COMANDO - installa o aggiorna blackbox-exporter +```bash +helm upgrade --install blackbox-exporter prometheus-community/prometheus-blackbox-exporter \ + --namespace monitoring \ + --create-namespace \ + -f blackbox-values.yaml +``` + +COMANDO - verifica pod blackbox-exporter +```bash +kubectl get pods -n monitoring | grep blackbox +``` + +COMANDO - verifica Service blackbox-exporter +```bash +kubectl get svc -n monitoring | grep blackbox +``` + +OUTPUT ATTESO +```text +blackbox-exporter ClusterIP 9115/TCP +``` + + +5. CREA PROBE HTTP PER SERVIZI KUBERNETES INTERNI +================================================= + +FILE - blackbox-probes.yaml + +Descrizione: +- testa endpoint HTTP interni tramite DNS Kubernetes; +- usa il modulo http_k8s_health definito in blackbox-values.yaml; +- invia le metriche a Prometheus tramite la risorsa Probe del Prometheus Operator. + +Da personalizzare: +- my-service +- my-namespace +- porta del servizio +- path HTTP, ad esempio /health, /ready o / +- label release se diversa da prometheus + +COMANDO - crea blackbox-probes.yaml +```bash +cat > blackbox-probes.yaml <<'EOF' +apiVersion: monitoring.coreos.com/v1 +kind: Probe +metadata: + name: http-services-probe + namespace: monitoring + labels: + release: kube-prometheus-stack +spec: + jobName: http-services-probe + interval: 30s + scrapeTimeout: 15s + module: http_k8s_health + prober: + url: blackbox-exporter.monitoring.svc.cluster.local:9115 + scheme: http + path: /probe + targets: + staticConfig: + static: + - http://my-service.my-namespace.svc.cluster.local:8080/health + - http://another-service.default.svc.cluster.local:80/ +EOF +``` + +COMANDO - applica Probe servizi interni +```bash +kubectl apply -f blackbox-probes.yaml +``` + +COMANDO - verifica Probe +```bash +kubectl get probe -n monitoring +kubectl describe probe http-services-probe -n monitoring +``` + +NOTA - campi importanti della Probe +```yaml +spec: + module: http_k8s_health + prober: + url: blackbox-exporter.monitoring.svc.cluster.local:9115 + targets: + staticConfig: + static: + - http://my-service.my-namespace.svc.cluster.local:8080/health +``` + + +6. CREA PROBE HTTP/HTTPS PER INGRESS O URL ESTERNI +================================================== + +FILE - blackbox-ingress-probes.yaml + +Descrizione: +- testa URL esposti tramite Ingress, Gateway, reverse proxy o endpoint pubblici; +- usa il modulo generico http_2xx. + +COMANDO - crea blackbox-ingress-probes.yaml +```bash +cat > blackbox-ingress-probes.yaml <<'EOF' +apiVersion: monitoring.coreos.com/v1 +kind: Probe +metadata: + name: ingress-http-probe + namespace: monitoring + labels: + release: kube-prometheus-stack +spec: + jobName: ingress-http-probe + interval: 30s + scrapeTimeout: 15s + module: http_2xx + prober: + url: blackbox-exporter.monitoring.svc.cluster.local:9115 + scheme: http + path: /probe + targets: + staticConfig: + static: + - https://app.example.com/health + - https://api.example.com/ready +EOF +``` + +COMANDO - applica Probe endpoint esterni +```bash +kubectl apply -f blackbox-ingress-probes.yaml +``` + + +7. VERIFICA METRICHE IN PROMETHEUS +================================== + +COMANDO - apri Prometheus in locale con port-forward +```bash +kubectl port-forward -n monitoring svc/kube-prometheus-stack-prometheus 9090:9090 +``` + +Poi apri nel browser: + +```text +http://localhost:9090 +``` + +QUERY PROMQL - tutte le probe +```promql +probe_success +``` + +QUERY PROMQL - probe servizi interni +```promql +probe_success{job="http-services-probe"} +``` + +QUERY PROMQL - status code HTTP +```promql +probe_http_status_code{job="http-services-probe"} +``` + +QUERY PROMQL - durata probe +```promql +probe_duration_seconds{job="http-services-probe"} +``` + +QUERY PROMQL - target falliti +```promql +probe_success{job="http-services-probe"} == 0 +``` + +Interpretazione: +- probe_success = 1: test riuscito. +- probe_success = 0: test fallito. +- probe_http_status_code: status HTTP ricevuto. +- probe_duration_seconds: durata della probe. + + +8. CREA ALERT PROMETHEUSRULE +============================ + +FILE - blackbox-http-alerts.yaml + +Descrizione: +- genera alert quando un target HTTP non risponde correttamente; +- genera alert quando un target risponde troppo lentamente. + +COMANDO - crea blackbox-http-alerts.yaml +```bash +cat > blackbox-http-alerts.yaml <<'EOF' +apiVersion: monitoring.coreos.com/v1 +kind: PrometheusRule +metadata: + name: blackbox-http-alerts + namespace: monitoring + labels: + release: kube-prometheus-stack +spec: + groups: + - name: blackbox-http + rules: + - alert: BlackboxHttpProbeFailed + expr: probe_success == 0 + for: 2m + labels: + severity: warning + annotations: + summary: "HTTP probe fallita" + description: "Il target {{ $labels.instance }} non risponde correttamente da almeno 2 minuti." + + - alert: BlackboxHttpSlowResponse + expr: probe_duration_seconds > 2 + for: 5m + labels: + severity: warning + annotations: + summary: "HTTP probe lenta" + description: "Il target {{ $labels.instance }} risponde in piu' di 2 secondi da almeno 5 minuti." +EOF +``` + +COMANDO - applica alert +```bash +kubectl apply -f blackbox-http-alerts.yaml +``` + +COMANDO - verifica PrometheusRule +```bash +kubectl get prometheusrule -n monitoring | grep blackbox +kubectl describe prometheusrule blackbox-http-alerts -n monitoring +``` + + +9. TEST MANUALE BLACKBOX EXPORTER +================================= + +COMANDO - port-forward blackbox-exporter +```bash +kubectl port-forward -n monitoring svc/blackbox-exporter 9115:9115 +``` + +COMANDO - esegui probe manuale da un altro terminale +```bash +curl "http://localhost:9115/probe?target=http://my-service.my-namespace.svc.cluster.local:8080/health&module=http_k8s_health" +``` + +OUTPUT ATTESO - metriche indicative +```text +probe_success 1 +probe_http_status_code 200 +``` + + +10. TROUBLESHOOTING +=================== + +CASO - Prometheus non vede la Probe + +COMANDO +```bash +kubectl get probe -A +kubectl describe probe -n monitoring http-services-probe +kubectl get prometheus -n monitoring -o yaml +``` + +Controlla: +- spec.probeSelector +- spec.probeNamespaceSelector +- metadata.labels della Probe + +CASO - blackbox-exporter non parte + +COMANDO +```bash +kubectl get pods -n monitoring | grep blackbox +kubectl logs -n monitoring deploy/blackbox-exporter +``` + +CASO - servizio target non raggiungibile + +COMANDO +```bash +kubectl get svc -n my-namespace +kubectl get endpoints -n my-namespace my-service +kubectl describe svc -n my-namespace my-service +``` + +CASO - DNS Kubernetes non risolve + +COMANDO +```bash +kubectl run dns-test --rm -it --image=busybox:1.36 --restart=Never -- nslookup my-service.my-namespace.svc.cluster.local +``` + +CASO - endpoint HTTPS fallisce per certificati, redirect o header + +Azioni consigliate: +- usare il modulo http_2xx per endpoint esterni generici; +- verificare se l'endpoint richiede SNI, autenticazione, header custom o path diverso; +- aggiungere un modulo dedicato in blackbox-values.yaml se serve una configurazione HTTP specifica. + + +11. ORDINE DI ESECUZIONE CONSIGLIATO +==================================== + +COMANDO - sequenza completa +```bash +helm repo add prometheus-community https://prometheus-community.github.io/helm-charts +helm repo update + +helm upgrade --install blackbox-exporter prometheus-community/prometheus-blackbox-exporter \ + --namespace monitoring \ + --create-namespace \ + -f blackbox-values.yaml + +kubectl apply -f blackbox-probes.yaml +kubectl apply -f blackbox-ingress-probes.yaml +kubectl apply -f blackbox-http-alerts.yaml +``` + +FILE CONSIGLIATI DA VERSIONARE +```text +blackbox-values.yaml +blackbox-probes.yaml +blackbox-ingress-probes.yaml +blackbox-http-alerts.yaml +``` + + +12. NOTE PER DEV, QA E PROD +=========================== + +Per separare gli ambienti e semplificare dashboard e alert, usa Probe distinte: + +```text +dev-http-services-probe +qa-http-services-probe +prod-http-services-probe +``` + +QUERY PROMQL - ambiente dev +```promql +probe_success{job="dev-http-services-probe"} +``` + +QUERY PROMQL - target prod falliti +```promql +probe_success{job="prod-http-services-probe"} == 0 +``` + +Per endpoint critici in produzione, valuta: +- interval piu' basso, ad esempio 15s o 30s; +- durata for degli alert tra 1m e 5m in base alla criticita'; +- dashboard Grafana con stato, status code e latenza per target. \ No newline at end of file diff --git a/st2idc/lnd-script/mainload.sh b/st2idc/lnd-script/mainload.sh index e69de29..5a2c8af 100644 --- a/st2idc/lnd-script/mainload.sh +++ b/st2idc/lnd-script/mainload.sh @@ -0,0 +1,39 @@ +#!/bin/bash +. ./.profile +cd /project/inbound_figc +rm -f *.csv +rm -f *.csv_int* + +./getftp.sh +./loadpgare.sh csexc17f.csv ana +./loadpgare.sh cseri17f.csv imp +./loadpgare.sh cserc17f.csv cal +./loadpgare.sh csezc17f.csv cla +#./loadpgare.sh cseyc17f.csv + +/usr/local/bin/kubectl cp cserc17f.csv_int_utf8_nobom.csv db/postgresql-1-postgresql-0:/etc/cserc17f_2load_utf8_nobom.csv +/usr/local/bin/kubectl cp cseri17f.csv_int_utf8_nobom.csv db/postgresql-1-postgresql-0:/etc/cseri17f_2load_utf8_nobom.csv +/usr/local/bin/kubectl cp csexc17f.csv_int_utf8_nobom.csv db/postgresql-1-postgresql-0:/etc/csexc17f_2load_utf8_nobom.csv +/usr/local/bin/kubectl cp csezc17f.csv_int_utf8_nobom.csv db/postgresql-1-postgresql-0:/etc/csezc17f_2load_utf8_nobom.csv +#k cp csezc17f.csv_int_utf8_nobom.csv db/postgresql-1-postgresql-0:/etc/csezc17f_2load_utf8_nobom.csv + +name=$(date '+%Y-%m-%d') +tar -zcvf "$name.tar.gz" *.csv +mv *.gz ./arch + +/usr/local/bin/kubectl cp pgareload.sql db/postgresql-1-postgresql-0:/etc/pgareload.sql +/usr/local/bin/kubectl exec -it postgresql-1-postgresql-0 -c postgresql-server -n db -- psql -U postgres -d pgare -f /etc/pgareload.sql + +/usr/local/bin/kubectl cp loadclub.sql db/postgresql-1-postgresql-0:/etc/loadclub.sql +/usr/local/bin/kubectl exec -it postgresql-1-postgresql-0 -c postgresql-server -n db -- psql -U postgres -d segdigi -f /etc/loadclub.sql + +sleep 120 + +/usr/local/bin/kubectl cp partite_ftp.sql db/postgresql-1-postgresql-0:/etc/partite_ftp.sql +/usr/local/bin/kubectl exec -it postgresql-1-postgresql-0 -c postgresql-server -n db -- psql -U postgres -d gare -f /etc/partite_ftp.sql + +/usr/local/bin/kubectl cp campionati_ftp.sql db/postgresql-1-postgresql-0:/etc/campionati_ftp.sql +kubectl exec -it postgresql-1-postgresql-0 -c postgresql-server -n db -- psql -U postgres -d gare -f /etc/campionati_ftp.sql + +/usr/local/bin/kubectl cp companies_ftp.sql db/postgresql-1-postgresql-0:/etc/companies_ftp.sql +/usr/local/bin/kubectl exec -it postgresql-1-postgresql-0 -c postgresql-server -n db -- psql -U postgres -d companies -f /etc/companies_ftp.sql \ No newline at end of file diff --git a/add-on/migrazione st2idc applicazioni. sporteams.txt b/st2idc/migrazione st2idc applicazioni. sporteams.txt similarity index 100% rename from add-on/migrazione st2idc applicazioni. sporteams.txt rename to st2idc/migrazione st2idc applicazioni. sporteams.txt diff --git a/add-on/migrazione st2idc applicazioni.txt b/st2idc/migrazione st2idc applicazioni.txt similarity index 100% rename from add-on/migrazione st2idc applicazioni.txt rename to st2idc/migrazione st2idc applicazioni.txt diff --git a/add-on/migrazione st2idc database.txt b/st2idc/migrazione st2idc database.txt similarity index 100% rename from add-on/migrazione st2idc database.txt rename to st2idc/migrazione st2idc database.txt