Init repo with working rabbitmq role

2023-11-14 11:07:18 +01:00 · 2023-11-14 11:07:18 +01:00 · c43815c9b6
commit c43815c9b6
12 changed files with 383 additions and 0 deletions
--- a/README.md
+++ b/README.md
@ -0,0 +1,99 @@
+# Installation et configuration de RabbitMQ
+
+## Documentation
+
+### RabbitMQ :
+* [RabbitMQ Production Checklist](https://www.rabbitmq.com/production-checklist.html)
+* [RabbitMQ Prometheus exporter](https://www.rabbitmq.com/prometheus.html)
+* [RabbitMQ Config file exemple complet](https://github.com/rabbitmq/rabbitmq-server/blob/v3.12.x/deps/rabbit/docs/rabbitmq.conf.example)
+* [RabbitMQ Authorisation and access control](https://rabbitmq.com/access-control.html)
+* [RabbitMQctl](https://www.rabbitmq.com/rabbitmqctl.8.html)
+### Modules ansible-galaxy :
+* [Ansible Galaxy : Rabbitmq](https://galaxy.ansible.com/ui/repo/published/community/rabbitmq/)
+
+
+## Configuration
+
+Les modifications de configuration sont à faire dans le fichier [templates/rabbitmq.conf.j2](templates/rabbitmq.conf.j2)
+
+## Variables
+* rabbitmq_cluster_name : Nom du cluster rabbitq. (Default: default)
+* rabbitmq_cluster_nodes : Liste des noeuds appartenant au cluster.
+* rabbitmq_admin_username : Nom de l'utilisateur admin. (Default : admin)
+* rabbitmq_admin_password : Mot de passe de l'utilisateur admin.
+* rabbitmq_plugins: Liste des plugins Rabbitmq à installer. (Default : rabbitmq_management,rabbitmq_shovel,rabbitmq_prometheus)
+* rabbitmq_vhosts : Liste des vhosts. (Default : "/")
+* rabbitmq_app_users : Liste des utilisateurs applicatifs à créer. Par défaut les utilisateurs ont tous les privilèges sur le vhost.
+```
+rabbitmq_app_users:
+    - username: "test"
+      password: "changeme"
+      vhost: "test"
+```
+## Fonctionnalités
+
+* Installe les dépendances du rôle, rabbitmq et erlang.
+* Supprime l'utilisateur guest créé par défaut et créer un utilisateur admin.
+* Active les plugins de management (interface web), prometheus (exporter intégré) et shovel.
+* Déploie les utilisateurs et les vhosts applicatifs.
+
+## Tags
+
+* install : installe rabbitmq et ses dépendances.
+* config : supprime l'utilisateur guest, créer l'utilisateur admin, les vhosts et les utilisateurs applicatifs.
+* users: deploie les utilisateurs et les vhosts.
+
+## Premier lancement pour création d'un cluster
+
+1. Lancer le playbook avec le tag install :
+```
+ansible-playbook -l rabbitmq playbooks/rabbitmq.yml -t install
+```
+2. Se rendre sur les machines 2 et 3 et renseigner les commandes suivantes pour créer le cluster :
+```
+rabbitmqctl stop_app
+rabbitmqctl --longnames join_cluster rabbit@rabbit-1
+rabbitmqctl start_app
+```
+3. Lancer le playbook avec le tag config :
+```
+ansible-playbook -l rabbitmq playbooks/rabbitmq.yml -t config
+```
+
+## Modification de configuration
+
+* Déploiement des utilisateurs applicatifs et des vhosts :
+```
+ansible-playbook playbooks/rabbitmq.yml -t users -l rabbitmq
+```
+
+## Tests de performance
+
+[RabbitMQ perf-tests](https://github.com/rabbitmq/rabbitmq-perf-test)
+
+### Pré-requis
+* Installer Java
+* Créer un utilisateur avec tous les droits sur un vhost dédiés.
+```
+rabbitmqctl add_vhost testsla
+rabbitmqctl add_user test_sla sebisdown -p testsla
+rabbitmqctl set_permissions -p testsla test_sla ".*" ".*" ".*"
+```
+
+### Exemple de test
+* Test sur une quorum-queue nommée 'qq', avec des messages de 4Ko publiés par 5 process et consommés par 15 process. Avec des taux variables : 200 msg/process/seconde pendant 240 secondes puis 400 msg/process/seconde pendant 120 secondes puis 300 msg/process/seconde pendant 120 secondes, en boucle.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq/testsla --quorum-queue --queue qq --size 4000  --variable-rate 200:240 --variable-rate 400:120 --variable-rate 300:120 --producers 5 --consumers 15
+```
+* Test illimité avec un seul publieur et un consommateur.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq/testsla
+```
+* Test illimité sur une quorum-queue avec un seul publieur et un consommateur.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq/testsla --quorum-queue --queue qq
+```
+* Test illimité sur une quorum-queue avec un taux de 100 msg/secondes pour un seul publieur et un seul consommateur.
+```
+java -jar perf-test-2.20.0.jar -h amqp://test_sla:sebisdown@rabbitmq/testsla --quorum-queue --queue qq --rate 100
+```
--- a/group_vars/rabbitmq_stg
+++ b/group_vars/rabbitmq_stg
@ -0,0 +1,15 @@
+---
+
+rabbitmq_admin_username: "admin"
+rabbitmq_admin_password: "changeme"
+
+rabbitmq_cluster_name: "rabbitmq_stg"
+rabbitmq_cluster_nodes: {}
+
+rabbitmq_vhosts:
+    - test
+
+rabbitmq_app_users:
+    - username: "seb"
+      password: "changeme"
+      vhost: "test"
--- a/playbooks/rabbitmq.yml
+++ b/playbooks/rabbitmq.yml
@ -0,0 +1,9 @@
+---
+
+- name: install and configure rabbitmq
+  hosts: rabbitmq
+  serial: 1
+
+  roles:
+    - rabbitmq
+
--- a/roles/rabbitmq/defaults/main.yml
+++ b/roles/rabbitmq/defaults/main.yml
@ -0,0 +1,8 @@
+---
+
+rabbitmq_cluster_name: "default"
+rabbitmq_admin_username: "admin"
+rabbitmq_app_users: ""
+rabbitmq_vhosts: ""
+rabbitmq_plugins: "rabbitmq_management,rabbitmq_shovel,rabbitmq_prometheus"
+rabbitmq_collect_statistics_interval: 30000
--- a/roles/rabbitmq/files/limits.conf
+++ b/roles/rabbitmq/files/limits.conf
@ -0,0 +1,2 @@
+[Service]
+LimitNOFILE=65536
--- a/roles/rabbitmq/handlers/main.yml
+++ b/roles/rabbitmq/handlers/main.yml
@ -0,0 +1,10 @@
+---
+
+- name: Daemon_reload
+  ansible.builtin.systemd_service:
+    daemon_reload: true
+
+- name: Restart Rabbitmq
+  ansible.builtin.systemd_service:
+    name: rabbitmq-server.service
+    state: restarted
--- a/roles/rabbitmq/tasks/config.yml
+++ b/roles/rabbitmq/tasks/config.yml
@ -0,0 +1,65 @@
+---
+
+- name: Deploy rabbitmq config file
+  ansible.builtin.template:
+    src: rabbitmq.conf.j2
+    dest: /etc/rabbitmq/rabbitmq.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: config
+
+- name: Deploy rabbitmq env file
+  ansible.builtin.template:
+    src: rabbitmq-env.conf.j2
+    dest: /etc/rabbitmq/rabbitmq-env.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: config
+
+- name: Enabling Rabbitmq plugins
+  community.rabbitmq.rabbitmq_plugin:
+    name: "{{ rabbitmq_plugins }}"
+    state: enabled
+  tags: config
+
+- name: Delete guest user
+  community.rabbitmq.rabbitmq_user:
+    user: guest
+    state: absent
+  tags: config
+
+- name: Create admin user
+  community.rabbitmq.rabbitmq_user:
+    user: "{{ rabbitmq_admin_username }}"
+    password: "{{ rabbitmq_admin_password }}"
+    vhost: /
+    configure_priv: .*
+    read_priv: .*
+    write_priv: .*
+    state: present
+    tags: administrator
+  tags: config
+
+- name: Create vhosts
+  community.rabbitmq.rabbitmq_vhost:
+    name: "{{ item }}"
+    state: present
+  with_items: "{{ rabbitmq_vhosts }}"
+  tags: config,users
+
+- name: Create app users
+  community.rabbitmq.rabbitmq_user:
+    user: "{{ item.username }}"
+    password: "{{ item.password }}"
+    vhost: "{{ item.vhost }}"
+    read_priv: .*
+    write_priv: .*
+    configure_priv: .*
+    state: present
+    tags: monitoring,management
+  with_items: "{{ rabbitmq_app_users }}"
+  tags: config,users
--- a/roles/rabbitmq/tasks/install.yml
+++ b/roles/rabbitmq/tasks/install.yml
@ -0,0 +1,135 @@
+---
+
+- name: Install requirements
+  ansible.builtin.apt:
+    name: "{{ item }}"
+    state: present
+  with_items:
+    - gnupg
+    - curl
+    - apt-transport-https
+    - debian-keyring
+  tags: install,conf
+
+- name: Import cloudsmith key (Rabbitmq-server)
+  ansible.builtin.get_url:
+    url: "https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/gpg.9F4587F226208342.key"
+    dest: "/usr/share/keyrings/rabbitmq.9F4587F226208342.asc"
+    mode: 0644
+    force: true
+  tags: install
+
+- name: Import cloudsmith key (Erlang)
+  ansible.builtin.get_url:
+    url: "https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/gpg.E495BB49CC4BBE5B.key"
+    dest: "/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.asc"
+    mode: 0644
+    force: true
+  tags: install
+
+- name: Add cloudsmith rabbitmq repository
+  ansible.builtin.apt_repository:
+    filename: rabbitmq
+    repo: "{{ item }}"
+  with_items:
+    - 'deb [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.asc] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/deb/debian bookworm main'
+    - 'deb-src [signed-by=/usr/share/keyrings/rabbitmq.9F4587F226208342.asc] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-server/deb/debian bookworm main'
+    - 'deb [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.asc] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/deb/debian bookworm main'
+    - 'deb-src [signed-by=/usr/share/keyrings/rabbitmq.E495BB49CC4BBE5B.asc] https://dl.cloudsmith.io/public/rabbitmq/rabbitmq-erlang/deb/debian bookworm main'
+  tags: install
+
+- name: Install Erlang packages
+  ansible.builtin.apt:
+    name: "{{ item }}"
+    state: present
+  with_items:
+    - erlang-base
+    - erlang-asn1
+    - erlang-crypto
+    - erlang-eldap
+    - erlang-ftp
+    - erlang-inets
+    - erlang-mnesia
+    - erlang-os-mon
+    - erlang-parsetools
+    - erlang-public-key
+    - erlang-runtime-tools
+    - erlang-snmp
+    - erlang-ssl
+    - erlang-syntax-tools
+    - erlang-tftp
+    - erlang-tools
+    - erlang-xmerl
+  tags: install
+
+- name: Install rabbitmq-server
+  ansible.builtin.apt:
+    name: "rabbitmq-server"
+    state: present
+  tags: install
+
+- name: Hold rabbitmq and Erlang packages
+  ansible.builtin.dpkg_selections:
+    name: "{{ item }}"
+    selection: hold
+  with_items:
+    - rabbitmq-server
+    - erlang-base
+  tags: install
+
+- name: Create systemd rabbit override directory
+  ansible.builtin.file:
+    path: "/etc/systemd/system/rabbitmq-server.service.d"
+    state: directory
+    owner: root
+    group: root
+    mode: 0755
+  tags: install
+
+- name: Set LimitNOFILE
+  ansible.builtin.copy:
+    src: limits.conf
+    dest: "/etc/systemd/system/rabbitmq-server.service.d/limits.conf"
+    mode: 0644
+    owner: root
+    group: root
+  tags: install
+  notify:
+    - Daemon_reload
+    - Restart Rabbitmq
+
+- name: Set Erlang cookie
+  ansible.builtin.template:
+    src: erlang.cookie
+    dest: /var/lib/rabbitmq/.erlang.cookie
+    owner: rabbitmq
+    group: rabbitmq
+    mode: 0400
+  tags: install
+  notify: Restart Rabbitmq
+
+- name: Ensure rabbitmq-server service is enabled
+  ansible.builtin.systemd_service:
+    name: rabbitmq-server.service
+    enabled: true
+  tags: install
+
+- name: Deploy rabbitmq config file
+  ansible.builtin.template:
+    src: rabbitmq.conf.j2
+    dest: /etc/rabbitmq/rabbitmq.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: install
+
+- name: Deploy rabbitmq env file
+  ansible.builtin.template:
+    src: rabbitmq-env.conf.j2
+    dest: /etc/rabbitmq/rabbitmq-env.conf
+    mode: 0644
+    owner: rabbitmq
+    group: rabbitmq
+  notify: Restart Rabbitmq
+  tags: install
--- a/roles/rabbitmq/tasks/main.yml
+++ b/roles/rabbitmq/tasks/main.yml
@ -0,0 +1,9 @@
+---
+
+- name: Import install tasks
+  ansible.builtin.include_tasks: install.yml
+  tags: install
+
+- name: Import config tasks
+  ansible.builtin.include_tasks: config.yml
+  tags: config,users
--- a/roles/rabbitmq/templates/erlang.cookie
+++ b/roles/rabbitmq/templates/erlang.cookie
@ -0,0 +1 @@
+{{ lookup('community.hashi_vault.hashi_vault','ansible/data/rabbitmq/{{ env }}/erlang-cookie:value') }}
--- a/roles/rabbitmq/templates/rabbitmq-env.conf.j2
+++ b/roles/rabbitmq/templates/rabbitmq-env.conf.j2
@ -0,0 +1,3 @@
+#{{ ansible_managed }}
+
+RABBITMQ_USE_LONGNAME=true
--- a/roles/rabbitmq/templates/rabbitmq.conf.j2
+++ b/roles/rabbitmq/templates/rabbitmq.conf.j2
@ -0,0 +1,27 @@
+#{{ ansible_managed }}
+
+# Limit to 70% of RAM
+vm_memory_high_watermark.relative = 0.7
+
+####################
+#
+# Cluster settings
+#
+####################
+
+cluster_formation.peer_discovery_backend = classic_config
+cluster_name = {{ rabbitmq_cluster_name }}
+{% for item in rabbitmq_cluster_nodes %}
+cluster_formation.classic_config.nodes.{{ loop.index }} = rabbit{{ loop.index }}@{{ item.hostname }}
+{% endfor %}
+
+####################
+#
+# Prometheus Plugins
+#
+####################
+
+## Statistics collection interval (in milliseconds). Increasing
+## this will reduce the load on management database.
+##
+collect_statistics_interval = {{ rabbitmq_collect_statistics_interval }}
				`@ -0,0 +1 @@`
				`{{ lookup('community.hashi_vault.hashi_vault','ansible/data/rabbitmq/{{ env }}/erlang-cookie:value') }}`