Refactor

README.md

@@ -1,6 +1,63 @@
-# Updates-dashboard
+# Updates Dashboard
 
-# Description
+## Summary
 
-* Front : Projet Django de consultation/gestion des données
+This is a tool have a clear view of the which servers are outdated, and keep trace of the updates.
-* Ansible : Playbooks ansible de génération des données
+
+## Technical information
+
+It runs with Django. The information are daily generated by an ansible playbook, which is located in ../ansible in this repo.
+
+## Install
+
+Dependencies in case of Debian 10.
+```
+    apt install python3-venv libmariadb-dev-compat libmariadb-dev mariadb-client python3-dev gcc
+```
+Following procedure to install the apps.
+```
+    useradd -d /var/www/updates-dashboard/ -g www-data -M -s /bin/false www-updash
+    cd /var/www/
+    git clone git@gitlab.infolegale.net:infrastructure/updates-dashboard.git updates-dashboard
+    chown -R www-updash:www-data updates-dashboard
+    touch /var/log/gunicorn.log
+    chown www-updash:www-data /var/log/gunicorn.log
+    cd updates-dashboard
+    python3 -m venv updash-venv
+    source updash-venv/bin/activate
+    (updash-venv) pip install -r requirements.txt
+    cp defaults/settings_local.py updatesdashboard/
+    cp defaults/gunicorn.service /etc/systemd/system/
+    cp defaults/updates-dashboard.conf /etc/nginx/sites-available
+    cd /etc/nginx/sites-enabled
+    ln -s /etc/nginx/sites-avaiable/updates-dashboard.conf .
+```
+* Set `settings_local.py` with correct values
+* Set `gunicorn.service` with correct values
+* Set `updates-dashboard.conf` with correct values
+```shell
+    systemctl daemon-reload
+    systemctl enable gunicorn.service
+    nginx -t
+    systemctl reload nginx
+```
+To initialize the project:
+```shell
+    (updash-venv) ./manage.py makemigrations
+    (updash-venv) ./manage.py makemigrations dashboard
+    (updash-venv) ./manage.py collectstatic
+    (updash-venv) ./manage.py migrate
+    (updash-venv) ./manage.py loaddata dashboard/fixtures/os.yaml
+    (updash-venv) ./manage.py loaddata dashboard/fixtures/teams.yaml
+```
+Vérifier les flux de mise à jour des données. Ansible->Dashboard
+Vérifier le sql mode de la base de données
+```shell
+set @@global.sql_mode='NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
+```
+
+## TODO
+
+* playbook to install via ansible ?
+* playbook to update via ansible ?
+* buttons should be 'previous / next results' instead of 'previous / next day'

ansible/2022-08-17.csv

@@ -1,19 +0,0 @@
-vlearning-db001-tst.infolegale.net;Debian;10;0;5
-app-db001-tst.infolegale.net;Debian;10;0;9
-consults-db001-tst.infolegale.net;Debian;10;0;147
-vador-db001-tst.infolegale.net;Debian;10;0;147
-tlearning-db003-tst.infolegale.net;Ubuntu;18.04;0;308
-transdb-db001-tst.infolegale.net;Debian;10;0;147
-mysql8-db001-tst.infolegale.net;Debian;11;3;119
-mongo-db001-tst.infolegale.net;Ubuntu;20.04;0;28
-orchestrator-mgr001-tst.infolegale.net;Ubuntu;20.04;0;28
-redis5-db001-tst.infolegale.net;Debian;10;0;326
-redis-db001-tst.infolegale.net;Ubuntu;18.04;0;326
-docker-hpv001-tst.infolegale.net;Ubuntu;20.04;0;106
-docker-hpv002-tst.infolegale.net;Ubuntu;20.04;0;398
-docker-hpv004-tst.infolegale.net;Ubuntu;20.04;0;28
-rundeck-sch001-tst.infolegale.net;Debian;10;0;91
-docker-hpv003-tst.infolegale.net;Ubuntu;20.04;0;125
-proxysql-db001-tst.infolegale.net;Ubuntu;20.04;0;475
-docker-hpv005-tst.infolegale.net;Ubuntu;20.04;0;119
-docker-hpv006-tst.infolegale.net;Ubuntu;20.04;0;309

ansible/README.md

@@ -1,22 +0,0 @@
-## Name
-Playbooks ansible utilisés pour récupérer les informations à intégrer dans l'updates-dashboard.
-
-## Description
-Les playbooks vont récuperer et afficher les informations suivantes sur les machines :
-* Main.yml : hostname, distribution, version, update disponible, uptime.
-* Packages.yml : hostname, paquet, version.
-
-La liste des paquets à vérifier se trouve dans le playbook packages.yml.
-Un callback particulier est utilisé pour afficher seulement les informations voulues pour construire le csv.
-
-## Usage
-
-* Ajouter callback_plugins=./hooks/ à ansible.cfg
-```
-export ANSIBLE_STDOUT_CALLBACK="update_dashboard"
-DATE=$(date '+%Y-%m-%d')
-ansible-playbook -i hosts-tst playbooks/main.yml > $DATE.csv
-```
-
-## Authors and acknowledgment
-Show your appreciation to those who have contributed to the project.

ansible/ansible.cfg

@@ -1,12 +0,0 @@
-[defaults]
-vault_password_file     = ~/.vault_password
-roles_path              = roles
-interpreter_python      = auto
-callback_plugins        = ./hooks/
-fact_caching            = jsonfile
-fact_caching_connection = ~/.cache/ansible/
-deprecation_warnings=False
-[hashi_vault_collection]
-url                     = "https://vault.infolegale.net"
-token_file              = .vault-token
-token_validate          = None

ansible/hooks/update_dashboard.py

@@ -1,23 +0,0 @@
-from __future__ import (absolute_import, division, print_function)
-__metaclass__ = type
-
-from ansible.plugins.callback import CallbackBase
-
-
-class CallbackModule(CallbackBase):
-
-    CALLBACK_VERSION = 2.0
-    CALLBACK_TYPE = 'stdout'
-    CALLBACK_NAME = 'update_dashboard'
-
-    def v2_runner_on_ok(self, result):
-        if result._task.action == 'debug':
-            ansible_msg = result._result.get('msg', None)
-            if ansible_msg:
-                print(ansible_msg)
-
-    def v2_runner_on_unreachable(self, result):
-        print("%s;UNREACHABLE" % result._host.get_name())
-
-    def v2_runner_on_failed(self, result, ignore_errors=False):
-        print("%s;FAILED" % result._host.get_name())

ansible/hosts-tst

@@ -1,44 +0,0 @@
-##########
-# VARS
-##########
-[all:vars]
-env='tst'
-symfony_env="recette"
-vmpass='$6$oIJS8mHoKPJVXZIR$fQVPI7.M9/3wD6O.8JS5iYIL6v0tx3BfmiHKIeb/h88yL9R9GYsGJnePK9UEoBZ2F8AfZhwHGRvV3iv9CGChV.'
-relayhost='maildump.infolegale.net'
-scriptsToSupervise=[['borg-backup.sh','129600']]
-graylogapp="graylog-log001-stg.infolegale.net:12201"
-cmk_server_ip="10.12.251.1"
-
-[mysql]
-tlearning-db003-tst.infolegale.net
-vlearning-db001-tst.infolegale.net
-app-db001-tst.infolegale.net
-consults-db001-tst.infolegale.net
-vador-db001-tst.infolegale.net
-transdb-db001-tst.infolegale.net
-mysql8-db001-tst.infolegale.net
-
-[mongo]
-mongo-db001-tst.infolegale.net
-
-[mgr]
-orchestrator-mgr001-tst.infolegale.net
-
-[redis]
-redis-db001-tst.infolegale.net
-redis5-db001-tst.infolegale.net
-
-[docker]
-docker-hpv001-tst.infolegale.net
-docker-hpv002-tst.infolegale.net
-docker-hpv003-tst.infolegale.net
-docker-hpv004-tst.infolegale.net
-docker-hpv005-tst.infolegale.net
-docker-hpv006-tst.infolegale.net
-
-[proxysql]
-proxysql-db001-tst.infolegale.net
-
-[rundeck]
-rundeck-sch001-tst.infolegale.net

ansible/playbooks/auto-updates-result.yml

@@ -1,55 +0,0 @@
----
-
-# This is to get the auto-updates script status from log file
-# It returns in stat_log:
-# - OUT if file older than 40 days
-# - ERR if recent but ERROR in file
-# - OK if recent and OK in file
-# nothing else (main playbook manage it as an UNK status)
-
-- name: check if there is an auto-updates log file
-  stat: path="{{ log_dir }}/scripts/auto-updates.log"
-  register: stat_log
-
-- name: set error status
-  set_fact: auto_updates_status=ERR
-  when: not stat_log.stat.exists
-
-- name: check if log file is less than 1 month old
-  shell: "find {{ log_dir }}/scripts/ -name 'auto-updates.log' -mtime -40 | wc -l"
-  register: date_log
-  when: stat_log.stat.exists
-
-- name: set outdated status
-  set_fact: auto_updates_status=OUT
-  when: date_log.stdout is defined and date_log.stdout != "1"
-
-- name: check ok status if recent log file
-  command: "grep OK {{ log_dir }}/scripts/auto-updates.log"
-  register: ok_log
-  failed_when: False
-  when: date_log.stdout is defined and date_log.stdout == "1"
-
-- name: set ok status
-  set_fact: auto_updates_status=OK
-  when: ok_log.rc is defined and ok_log.rc != 1
-
-- name: check error if old log file
-  command: "grep ERROR {{ log_dir }}/scripts/auto-updates.log"
-  register: error_log
-  failed_when: false
-  when: date_log.stdout is defined and date_log.stdout == "1"
-
-- name: set error status
-  set_fact: auto_updates_status=ERR
-  when: error_log.rc is defined and error_log.rc != 1
-
-- name: check running status if recent log file
-  command: "grep RUNNING {{ log_dir }}/scripts/auto-updates.log"
-  register: running_log
-  failed_when: false
-  when: stat_log.stat.exists
-
-- name: set error status
-  set_fact: auto_updates_status=ERR
-  when: running_log.rc is defined and running_log.rc != 1

ansible/playbooks/main.yml

@@ -1,47 +0,0 @@
----
-
-###
-# This playbook aims to retrieve a 'state' of our servers, ie for each server:
-# - what is the OS, in which version (we focus on debian / ubuntu and centos)
-# - for centos and debian-based distribs, how many available updates are there?
-#
-# Call it with the callback made for it: callback_plugin=update_dashboard in ansible.cfg
-# or after running export ANSIBLE_STDOUT_CALLBACK="update_dashboard"
-# also: callback_plugins = ./hooks/callback:/usr/share/ansible_plugins/callback_plugins
-###
-
-- hosts: "all"
-
-  strategy: free
-
-  tasks:
-
-  #
-  ## All hosts ##
-  #
-
-  - name: get uptime
-    shell: uptime=$(uptime | grep -o "[0-9][0-9]* days" | awk '{ print $1 }'); if [ "$uptime" ]; then echo $uptime; else echo 0; fi
-    register: uptime
-    changed_when: False
-    tags:
-      - skip_ansible_lint
-
-  # get number of updates and debug
-  - name: (debian) update apt cache if older than 1 day
-    apt: update_cache=yes cache_valid_time=86400
-    register: update_cache_result
-    ignore_errors: yes
-
-  - name: (debian) check available updates on debian systems
-    shell: apt-get --dry-run dist-upgrade | grep '^Inst' | wc -l warn=no
-    register: available_updates
-    when: update_cache_result is succeeded
-
-  - name: (Debian) output
-    debug: msg="{{ inventory_hostname }};{{ ansible_distribution }};{{ ansible_distribution_major_version }};{{ available_updates.stdout | default('') }};{{ uptime.stdout }}"
-    when: ansible_distribution == "Debian"
-
-  - name: (Ubuntu) output
-    debug: msg="{{ inventory_hostname }};{{ ansible_distribution }};{{ ansible_distribution_version }};{{ available_updates.stdout | default('') }};{{ uptime.stdout }}"
-    when: ansible_distribution == "Ubuntu"

ansible/playbooks/packages.yml

@@ -1,68 +0,0 @@
----
-
-- hosts: all
-
-  strategy: free
-
-  vars:
-    - packages :
-        - apt
-        - apache2
-        - bash
-        - check-mk-agent
-        - elasticsearch
-        - libc-bin
-        - make
-        - mysql-server
-        - nginx
-        - openssh-server
-        - php5
-        - php7.0
-        - php7.1
-        - php7.2
-        - php7.4
-        - postfix
-        - proxysql
-        - redis
-        - redis-server
-        - sphinxsearch
-        - sudo
-        - systemd
-
-  tasks:
-
-    # Loop on package list to get versions
-
-    - name: Check packages lists
-      include: packages_include.yml package="{{ item }}"
-      with_items:
-        - "{{ packages }}"
-
-
-    # Get the RUNNING kernel
-
-    ## Debian
-
-    - when: ansible_distribution == "Debian" and ansible_distribution_major_version|int >= 7
-      block:
-
-        - name: (Debian) get kernel version
-          shell: uname -v | awk '{ print $4 }'
-          register: kernel_version_debian
-          changed_when: false
-
-        - name: (Debian) Kernel output
-          debug: msg="{{ inventory_hostname }};kernel;{{ kernel_version_debian.stdout }}"
-
-    ## Ubuntu
-
-    - when: ansible_distribution == "Ubuntu" and ansible_distribution_major_version|int >= 16
-      block:
-
-        - name: (Ubuntu) get kernel version
-          shell: uname -r
-          register: kernel_version_ubuntu
-          changed_when: false
-
-        - name: (ubuntu) Kernel output
-          debug: msg="{{ inventory_hostname }};kernel;{{ kernel_version_ubuntu.stdout }}"

ansible/playbooks/packages_include.yml

@@ -1,10 +0,0 @@
----
-
-- name: check package version
-  shell: dpkg -l | awk '$2=="{{ package }}" { print $3 }'
-  register: package_version_debian
-  changed_when: false
-
-- name: output
-  debug: msg="{{ inventory_hostname }};{{ package }};{{ package_version_debian.stdout }}"
-  when: package_version_debian.stdout != ""

front/README.md

@@ -1,63 +0,0 @@
-# Updates Dashboard
-
-## Summary
-
-This is a tool have a clear view of the which servers are outdated, and keep trace of the updates.
-
-## Technical information
-
-It runs with Django. The information are daily generated by an ansible playbook, which is located in ../ansible in this repo.
-
-## Install
-
-Dependencies in case of Debian 10.
-```
-    apt install python3-venv libmariadb-dev-compat libmariadb-dev mariadb-client python3-dev gcc
-```
-Following procedure to install the apps.
-```
-    useradd -d /var/www/updates-dashboard/ -g www-data -M -s /bin/false www-updash
-    cd /var/www/
-    git clone git@gitlab.infolegale.net:infrastructure/updates-dashboard.git updates-dashboard
-    chown -R www-updash:www-data updates-dashboard
-    touch /var/log/gunicorn.log
-    chown www-updash:www-data /var/log/gunicorn.log
-    cd updates-dashboard
-    python3 -m venv updash-venv
-    source updash-venv/bin/activate
-    (updash-venv) pip install -r requirements.txt
-    cp defaults/settings_local.py updatesdashboard/
-    cp defaults/gunicorn.service /etc/systemd/system/
-    cp defaults/updates-dashboard.conf /etc/nginx/sites-available
-    cd /etc/nginx/sites-enabled
-    ln -s /etc/nginx/sites-avaiable/updates-dashboard.conf .
-```
-* Set `settings_local.py` with correct values
-* Set `gunicorn.service` with correct values
-* Set `updates-dashboard.conf` with correct values
-```shell
-    systemctl daemon-reload
-    systemctl enable gunicorn.service
-    nginx -t
-    systemctl reload nginx
-```
-To initialize the project:
-```shell
-    (updash-venv) ./manage.py makemigrations
-    (updash-venv) ./manage.py makemigrations dashboard
-    (updash-venv) ./manage.py collectstatic
-    (updash-venv) ./manage.py migrate
-    (updash-venv) ./manage.py loaddata dashboard/fixtures/os.yaml
-    (updash-venv) ./manage.py loaddata dashboard/fixtures/teams.yaml
-```
-Vérifier les flux de mise à jour des données. Ansible->Dashboard
-Vérifier le sql mode de la base de données
-```shell
-set @@global.sql_mode='NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
-```
-
-## TODO
-
-* playbook to install via ansible ?
-* playbook to update via ansible ?
-* buttons should be 'previous / next results' instead of 'previous / next day'