add rabbitmq ansible role

ansible/roles/rabbitmq/README.md

@@ -0,0 +1,106 @@
+# Installation et configuration de RabbitMQ
+
+## Documentation
+
+### RabbitMQ :
+* [RabbitMQ Production Checklist](https://www.rabbitmq.com/production-checklist.html)
+* [RabbitMQ Prometheus exporter](https://www.rabbitmq.com/prometheus.html)
+* [RabbitMQ Config file exemple complet](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbit/docs/rabbitmq.conf.example)
+* [RabbitMQ Authorisation and access control](https://rabbitmq.com/access-control.html)
+* [RabbitMQctl](https://www.rabbitmq.com/rabbitmqctl.8.html)
+### Modules ansible-galaxy :
+* [Ansible Galaxy : Rabbitmq](https://galaxy.ansible.com/ui/repo/published/community/rabbitmq/)
+
+
+## Configuration
+
+Les modifications de configuration sont à faire dans le fichier [templates/rabbitmq.conf.j2](templates/rabbitmq.conf.j2)
+
+## Variables
+* rabbitmq_cluster_name : Nom du cluster rabbitq. (Default: default)
+* rabbitmq_cluster_nodes : Liste des noeuds appartenant au cluster.
+* rabbitmq_admin_username : Nom de l'utilisateur admin. (Default : admin)
+* rabbitmq_admin_password : Mot de passe de l'utilisateur admin.
+* rabbitmq_plugins: Liste des plugins Rabbitmq à installer. (Default : rabbitmq_management,rabbitmq_shovel,rabbitmq_prometheus)
+* rabbitmq_vhosts : Liste des vhosts. (Default : "/")
+* rabbitmq_app_users : Liste des utilisateurs applicatifs à créer. Par défaut les utilisateurs ont tous les privilèges sur le vhost.
+```
+rabbitmq_app_users:
+    - username: "consult"
+      password: "{{ lookup('community.hashi_vault.hashi_vault', 'ansible/data/rabbitmq/{{ env }}/users/consult:password') }}"
+      vhost: "consult"
+```
+## Fonctionnalités
+
+* Installe les dépendances du rôle, rabbitmq et erlang.
+* Supprime l'utilisateur guest créé par défaut et créer un utilisateur admin.
+* Active les plugins de management (interface web), prometheus (exporter intégré) et shovel.
+* Déploie les utilisateurs et les vhosts applicatifs.
+
+## Tags
+
+* install : installe rabbitmq et ses dépendances.
+* config : supprime l'utilisateur guest, créer l'utilisateur admin, les vhosts et les utilisateurs applicatifs.
+* users: deploie les utilisateurs et les vhosts.
+* vhosts: deploie les vhosts.
+* exchanges: deploie les exchanges.
+
+## Premier lancement pour création d'un cluster
+
+1. Lancer le playbook avec le tag install :
+```
+ansible-playbook -l rabbitmq_cluster playbooks/rabbitmq.yml -t install
+```
+2. Se rendre sur les machines 2 et 3 et renseigner les commandes suivantes pour créer le cluster :
+```
+rabbitmqctl stop_app
+rabbitmqctl --longnames join_cluster node1.example.net
+rabbitmqctl start_app
+```
+3. Lancer le playbook avec le tag config :
+```
+ansible-playbook -l rabbitmq_cluster playbooks/rabbitmq.yml -t config
+```
+
+## Modification de configuration
+
+* Création de toutes les ressources : users, vhost, exchange, queues et bindings :
+```
+ansible-playbook playbooks/rabbitmq.yml -t config -l rabbitmq_cluster
+```
+
+* Déploiement des utilisateurs applicatifs et des vhosts :
+```
+ansible-playbook playbooks/rabbitmq.yml -t users -l rabbitmq_cluster
+```
+
+## Tests de performance
+
+[RabbitMQ perf-tests](https://github.com/rabbitmq/rabbitmq-perf-test)
+
+### Pré-requis
+* Installer Java
+* Créer un utilisateur avec tous les droits sur un vhost dédiés.
+```
+rabbitmqctl add_vhost testsla
+rabbitmqctl add_user test_sla sebisdown -p testsla
+rabbitmqctl set_permissions -p testsla test_sla ".*" ".*" ".*"
+```
+
+### Exemple de test
+* Test sur une quorum-queue nommée 'qq', avec des messages de 4Ko publiés par 5 process et consommés par 15 process. Avec des taux variables : 200 msg/process/seconde pendant 240 secondes puis 400 msg/process/seconde pendant 120 secondes puis 300 msg/process/seconde pendant 120 secondes, en boucle.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq-vip.example.com:5674/testsla --quorum-queue --queue qq --size 4000  --variable-rate 200:240 --variable-rate 400:120 --variable-rate 300:120 --producers 5 --consumers 15
+```
+* Test illimité avec un seul publieur et un consommateur.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq-vip.example.com:5674/testsla
+```
+* Test illimité sur une quorum-queue avec un seul publieur et un consommateur.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq-vip.example.com:5674/testsla --quorum-queue --queue qq
+```
+* Test illimité sur une quorum-queue avec un taux de 100 msg/secondes pour un seul publieur et un seul consommateur.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq-vip.example.com:5674/testsla --quorum-queue --queue qq --rate 100
+```

ansible/roles/rabbitmq/defaults/main.yml

@@ -0,0 +1,8 @@
+---
+
+rabbitmq_cluster_name: "default"
+rabbitmq_admin_username: "admin"
+rabbitmq_app_users: ""
+rabbitmq_vhosts: ""
+rabbitmq_plugins: "rabbitmq_management,rabbitmq_shovel,rabbitmq_prometheus,rabbitmq_shovel_management"
+rabbitmq_collect_statistics_interval: 30000

ansible/roles/rabbitmq/files/limits.conf

@@ -0,0 +1,4 @@
+{{ ansible_managed }}
+
+[Service]
+LimitNOFILE=65536

ansible/roles/rabbitmq/files/logrotate

@@ -0,0 +1,9 @@
+{{ ansible_managed }}
+
+/var/log/rabbitmq/*.log {
+        daily
+        missingok
+        rotate 7
+        compress
+        notifempty
+}

ansible/roles/rabbitmq/handlers/main.yml

@@ -0,0 +1,10 @@
+---
+
+- name: Daemon_reload
+  ansible.builtin.systemd_service:
+    daemon_reload: true
+
+- name: Restart Rabbitmq
+  ansible.builtin.systemd_service:
+    name: rabbitmq-server.service
+    state: restarted

ansible/roles/rabbitmq/tasks/config.yml

@@ -0,0 +1,117 @@
+---
+
+- name: Deploy rabbitmq config file
+  ansible.builtin.template:
+    src: rabbitmq.conf.j2
+    dest: /etc/rabbitmq/rabbitmq.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: config
+
+- name: Deploy rabbitmq env file
+  ansible.builtin.template:
+    src: rabbitmq-env.conf.j2
+    dest: /etc/rabbitmq/rabbitmq-env.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: config
+
+- name: Enabling Rabbitmq plugins
+  community.rabbitmq.rabbitmq_plugin:
+    name: "{{ rabbitmq_plugins }}"
+    state: enabled
+  tags: config
+
+- name: Delete guest user
+  community.rabbitmq.rabbitmq_user:
+    user: guest
+    state: absent
+  tags: config
+
+- name: Create vhosts
+  community.rabbitmq.rabbitmq_vhost:
+    name: "{{ item }}"
+    state: present
+  with_items: "{{ rabbitmq_vhosts }}"
+  tags: config,users,vhosts
+
+- name: Create admin user
+  community.rabbitmq.rabbitmq_user:
+    user: "{{ rabbitmq_admin_username }}"
+    password: "{{ rabbitmq_admin_password }}"
+    vhost: "{{ item }}"
+    configure_priv: .*
+    read_priv: .*
+    write_priv: .*
+    state: present
+    tags: administrator
+  tags: config
+  with_items: "{{ rabbitmq_vhosts }}"
+
+- name: Create checkmk user
+  community.rabbitmq.rabbitmq_user:
+    user: "{{ rabbitmq_checkmk_username }}"
+    password: "{{ rabbitmq_checkmk_password }}"
+    vhost: /
+    configure_priv: ""
+    read_priv: .*
+    write_priv: ""
+    state: present
+    tags: monitoring
+  tags: config
+
+- name: Create exchanges
+  community.rabbitmq.rabbitmq_exchange:
+    name: "{{ item.name }}"
+    type: "{{ item.type }}"
+    vhost: "{{ item.vhost }}"
+    state: present
+    login_user: "{{ rabbitmq_admin_username }}"
+    login_password: "{{ rabbitmq_admin_password }}"
+  with_items: "{{ rabbitmq_exchanges }}"
+  tags: config,exchanges
+
+- name: Create queues
+  community.rabbitmq.rabbitmq_queue:
+    login_user: "{{ rabbitmq_admin_username }}"
+    login_password: "{{ rabbitmq_admin_password }}"
+    state: "{{ item.state | default('present')}}"
+    vhost: "{{ item.vhost }}"
+    name: "{{ item.name }}"
+    durable: "{{ item.durable | default(true) }}"
+    dead_letter_exchange: "{{ item.dead_letter_exchange | default() }}"
+    dead_letter_routing_key: "{{ item.dead_letter_routing_key | default() }}"
+    arguments: "{{ item.arguments | default({}) }}"
+  with_items: "{{ rabbitmq_queues }}"
+  tags: config,queues
+
+- name: Create bindings
+  community.rabbitmq.rabbitmq_binding:
+    login_user: "{{ rabbitmq_admin_username }}"
+    login_password: "{{ rabbitmq_admin_password }}"
+    state: "{{ item.state |default('present') }}"
+    vhost: "{{ item.vhost }}"
+    name: "{{ item.name }}"
+    destination: "{{ item.destination }}"
+    destination_type: "{{ item.destination_type }}"
+    routing_key: "{{ item.routing_key }}"
+    arguments: "{{ item.arguments | default({}) }}"
+  with_items: "{{ rabbitmq_bindings }}"
+  tags: config,bindings
+
+- name: Create app users
+  community.rabbitmq.rabbitmq_user:
+    user: "{{ item.username }}"
+    password: "{{ item.password }}"
+    vhost: "{{ item.vhost }}"
+    read_priv: "{{ item.read_priv | default('.*') }}"
+    write_priv: "{{ item.write_priv | default('.*') }}"
+    configure_priv: "{{ item.configure_priv | default('.*') }}"
+    state: present
+    tags: monitoring,management
+  with_items: "{{ rabbitmq_app_users }}"
+  tags: config,users

ansible/roles/rabbitmq/tasks/install.yml

@@ -0,0 +1,156 @@
+---
+
+- name: Install requirements
+  ansible.builtin.apt:
+    name: "{{ item }}"
+    state: present
+  with_items:
+    - gnupg
+    - curl
+    - apt-transport-https
+    - debian-keyring
+  tags: install,conf
+
+- name: Import cloudsmith key (Rabbitmq-server)
+  ansible.builtin.get_url:
+    url: "https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/gpg.9F4587F226208342.key"
+    dest: "/usr/share/keyrings/rabbitmq.9F4587F226208342.asc"
+    mode: 0644
+    force: true
+  tags: install
+
+- name: Import cloudsmith key (Erlang)
+  ansible.builtin.get_url:
+    url: "https://github.com/rabbitmq/signing-keys/releases/download/3.0/cloudsmith.rabbitmq-erlang.E495BB49CC4BBE5B.key"
+    dest: "/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg"
+    mode: 0644
+    force: true
+  tags: install
+
+- name: Import cloudsmith key (Erlang) - 2
+  ansible.builtin.get_url:
+    url: "https://github.com/rabbitmq/signing-keys/releases/download/3.0/cloudsmith.rabbitmq-server.9F4587F226208342.key"
+    dest: "/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg"
+    mode: 0644
+    force: true
+  tags: install
+
+- name: Add rabbitmq repository
+  ansible.builtin.apt_repository:
+    filename: rabbitmq
+    repo: "{{ item }}"
+  with_items:
+    - 'deb [arch=amd64 signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.rabbitmq.com/rabbitmq/rabbitmq-erlang/deb/debian bookworm main'
+    - 'deb-src [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa1.rabbitmq.com/rabbitmq/rabbitmq-erlang/deb/debian bookworm main'
+    - 'deb [arch=amd64 signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa2.rabbitmq.com/rabbitmq/rabbitmq-erlang/deb/debian bookworm main'
+    - 'deb-src [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.gpg] https://ppa2.rabbitmq.com/rabbitmq/rabbitmq-erlang/deb/debian bookworm main'
+    - 'deb [arch=amd64 signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.rabbitmq.com/rabbitmq/rabbitmq-server/deb/debian bookworm main'
+    - 'deb-src [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa1.rabbitmq.com/rabbitmq/rabbitmq-server/deb/debian bookworm main'
+    - 'deb [arch=amd64 signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa2.rabbitmq.com/rabbitmq/rabbitmq-server/deb/debian bookworm main'
+    - 'deb-src [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.gpg] https://ppa2.rabbitmq.com/rabbitmq/rabbitmq-server/deb/debian bookworm main'
+  tags: install
+
+- name: Install Erlang packages
+  ansible.builtin.apt:
+    name: "{{ item }}"
+    state: present
+  with_items:
+    - erlang-base
+    - erlang-asn1
+    - erlang-crypto
+    - erlang-eldap
+    - erlang-ftp
+    - erlang-inets
+    - erlang-mnesia
+    - erlang-os-mon
+    - erlang-parsetools
+    - erlang-public-key
+    - erlang-runtime-tools
+    - erlang-snmp
+    - erlang-ssl
+    - erlang-syntax-tools
+    - erlang-tftp
+    - erlang-tools
+    - erlang-xmerl
+  tags: install
+
+- name: Install rabbitmq-server
+  ansible.builtin.apt:
+    name: "rabbitmq-server"
+    state: present
+  tags: install
+
+- name: Hold rabbitmq and Erlang packages
+  ansible.builtin.dpkg_selections:
+    name: "{{ item }}"
+    selection: hold
+  with_items:
+    - rabbitmq-server
+    - erlang-base
+  tags: install
+
+- name: Create systemd rabbit override directory
+  ansible.builtin.file:
+    path: "/etc/systemd/system/rabbitmq-server.service.d"
+    state: directory
+    owner: root
+    group: root
+    mode: 0755
+  tags: install
+
+- name: Set LimitNOFILE
+  ansible.builtin.copy:
+    src: limits.conf
+    dest: "/etc/systemd/system/rabbitmq-server.service.d/limits.conf"
+    mode: 0644
+    owner: root
+    group: root
+  tags: install
+  notify:
+    - Daemon_reload
+    - Restart Rabbitmq
+
+- name: Set Erlang cookie
+  ansible.builtin.template:
+    src: erlang.cookie
+    dest: /var/lib/rabbitmq/.erlang.cookie
+    owner: rabbitmq
+    group: rabbitmq
+    mode: 0400
+  tags: install
+  notify: Restart Rabbitmq
+
+- name: Ensure rabbitmq-server service is enabled
+  ansible.builtin.systemd_service:
+    name: rabbitmq-server.service
+    enabled: true
+  tags: install
+
+- name: Deploy rabbitmq config file
+  ansible.builtin.template:
+    src: rabbitmq.conf.j2
+    dest: /etc/rabbitmq/rabbitmq.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: install
+
+- name: Deploy rabbitmq env file
+  ansible.builtin.template:
+    src: rabbitmq-env.conf.j2
+    dest: /etc/rabbitmq/rabbitmq-env.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: install
+
+- name: Deploy logrotate conf
+  ansible.builtin.copy:
+    src: logrotate
+    dest: /etc/logrotate.d/rabbitmq-server
+    mode: 0644
+    user: root
+    group: root
+  tags: install

ansible/roles/rabbitmq/tasks/main.yml

@@ -0,0 +1,9 @@
+---
+
+- name: Import install tasks
+  ansible.builtin.include_tasks: install.yml
+  tags: install
+
+- name: Import config tasks
+  ansible.builtin.include_tasks: config.yml
+  tags: config,users,queues,vhosts,exchanges,bindings

ansible/roles/rabbitmq/templates/erlang.cookie

@@ -0,0 +1 @@
+{{ lookup('community.hashi_vault.hashi_vault','ansible/data/rabbitmq/{{ env }}/erlang-cookie:value') }}

ansible/roles/rabbitmq/templates/rabbitmq-env.conf.j2

@@ -0,0 +1,3 @@
+#{{ ansible_managed }}
+
+RABBITMQ_USE_LONGNAME=true

ansible/roles/rabbitmq/templates/rabbitmq.conf.j2

@@ -0,0 +1,31 @@
+#{{ ansible_managed }}
+
+# Limit to 70% of RAM
+vm_memory_high_watermark.relative = 0.7
+
+# Tuning raft WAL log file https://rabbitmq.com/quorum-queues.html#segment-entry-count
+raft.segment_max_entries = 32768
+raft.wal_max_size_bytes = 32000000
+
+####################
+#
+# Cluster settings
+#
+####################
+
+cluster_formation.peer_discovery_backend = classic_config
+cluster_name = {{ rabbitmq_cluster_name }}
+{% for item in rabbitmq_cluster_nodes %}
+cluster_formation.classic_config.nodes.{{ loop.index }} = rabbit{{ loop.index }}@{{ item.hostname }}
+{% endfor %}
+
+####################
+#
+# Prometheus Plugins
+#
+####################
+
+## Statistics collection interval (in milliseconds). Increasing
+## this will reduce the load on management database.
+##
+collect_statistics_interval = {{ rabbitmq_collect_statistics_interval }}