# Customizações Duplicati # Duplicati Monitor Link: [https://github.com/RafaMunoz/duplicati-monitor](https://github.com/RafaMunoz/duplicati-monitor) git clone [https://github.com/RafaMunoz/duplicati-monitor.git](https://github.com/RafaMunoz/duplicati-monitor.git) # Duplicati Monitor
[](https://github.com/RafaMunoz/duplicati-monitor#duplicati-monitor)
Duplicati Monitor is a docker-packaged monitoring solution for Duplicati. It allows you to control the status of your backups and statistics for each report and notifies you by any means supported by **Apprice**. It is meant to be self-hosted and powered by docker. ## Quick Start
[](https://github.com/RafaMunoz/duplicati-monitor#quick-start)
### Notifications
[](https://github.com/RafaMunoz/duplicati-monitor#notifications)
To send notifications and satisfy the end user, it has been implemented with the Apprise library, which allows you to send a notification to almost all the most popular notification services available today, such as: Telegram, Discord, Slack, Amazon SNS, Gotify, etc. You can check all the services available in the [official documentation](https://github.com/caronc/apprise#productivity-based-notifications). ### Templates
[](https://github.com/RafaMunoz/duplicati-monitor#templates)
You can create your own templates to receive the information you exactly need. By default we use the following two: ``` # Succes backup 💾 🟢 # Error backup 💾 🔴 ```
You can create your own templates in a simple way. You only have to use the symbols `<` and `>` delimit the fields and separating keys with dots `.` to send along with the message that you want. In the [docs/examples\_report](https://github.com/RafaMunoz/duplicati-monitor/blob/main/docs/examples_report) folder of this repository you have two examples of the JSON reports that Duplicati sends and in which you can see the fields that compose it. For example, with the following template, the result shown in the telegram image would be obtained. ``` 💾 🟢 Backup: \n - Examined Files: \n - Duration: \n - Status: ** ```
[![telegram-example-notication](https://github.com/RafaMunoz/duplicati-monitor/raw/main/docs/img/telegram-example-notication.png)](https://github.com/RafaMunoz/duplicati-monitor/blob/main/docs/img/telegram-example-notication.png) ### Environment Variables
[](https://github.com/RafaMunoz/duplicati-monitor#environment-variables)
ENVIRONMENT VARIABLETYPEDEFAULTDESCRIPTION
`URI_NOTIFICATION`required URI in Apprise format where the notifications will be sent.
`TEMPLATE_SUCCESS`opcional💾 🟢 <Extra.backup-name>Message template to be sent when the backup is successful.
`TEMPLATE_ERROR`opcional💾 🔴 <Extra.backup-name>Message template that will be sent when the backup is executed in an erroneous way.
`PORT`opcional8000Listening port on which the service is set up to receive the reports.
### Docker Run
[](https://github.com/RafaMunoz/duplicati-monitor#docker-run)
To start the container you can do it with the following command. ``` docker run -d --name=duplicati-monitor -p 8000:8000 -e URI_NOTIFICATION=tgram:////?format=markdown rafa93m/duplicati-monitor ```
Or you can also use the following docker compose. ``` version: "3" services: duplicati-monitor: container_name: duplicati-monitor image: rafa93m/duplicati-monitor ports: - 8000:8000 environment: URI_NOTIFICATION: "tgram:////?format=markdown" TEMPLATE_SUCCESS: "💾 🟢 Backup: " TEMPLATE_ERROR: "💾 🔴 Backup failed 🔥 🔥" restart: unless-stopped ```
### Setup Duplicati
[](https://github.com/RafaMunoz/duplicati-monitor#setup-duplicati)
Add theses two options for each backup you want to monitor: - **send-http-result-output-format**: json - **send-http-url**: http://**IP\_ADDRESS**:**PORT**/report [![advanced-options](https://github.com/RafaMunoz/duplicati-monitor/raw/main/docs/img/advanced-options.png)](https://github.com/RafaMunoz/duplicati-monitor/blob/main/docs/img/advanced-options.png) # Duplicati Dashboard Link: [https://github.com/fabien-github/duplicati\_dashboard?tab=readme-ov-file#demo](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#demo) git clone [https://github.com/fabien-github/duplicati\_dashboard.git](https://github.com/fabien-github/duplicati_dashboard.git) Duplicati Dashboard is a monitoring solution for [Duplicati](https://www.duplicati.com/). It allows you to monitor your backups status, collects stats for each reports and alerts you by email when a backup fails. It is intended to be self-hosted and works with [docker-compose](https://docs.docker.com/compose/). Everything is already pre-configured and ready to be deployed. - [Duplicati Dashboard](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#duplicati-dashboard) - [Demo](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#demo) - [Quick Start](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#quick-start) - [Running with docker-compose](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#running-with-docker-compose) - [Setup Duplicati](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#setup-duplicati) - [Connect to your dashboard](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#connect-to-your-dashboard) - [Configuration](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#configuration) - [Env file](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#env-file) - [Notes](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#notes) - [Grafana configuration locked](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#grafana-configuration-locked) - [Backup over more than 30 days rotation](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#backup-over-more-than-30-days-rotation) - [Alerting graph](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#alerting-graph) - [Deleting removed backup data](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#deleting-removed-backup-data) - [Docker-compose](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#docker-compose) - [Other informations](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#other-informations) - [License](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#license) # Demo
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#demo)
[![](https://github.com/fabien-github/duplicati_dashboard/raw/main/docs/img/demo.gif)](https://github.com/fabien-github/duplicati_dashboard/blob/main/docs/img/demo.gif) > The right side of the video is not integrated in the dashboard. You can't control your backup with it. # Quick Start
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#quick-start)
### Running with docker-compose
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#running-with-docker-compose)
``` git clone https://github.com/fabien-github/duplicati_dashboard.git cd duplicati_dashboard docker-compose up -d ```
### Setup Duplicati
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#setup-duplicati)
Add theses two options for each backup you want to monitor: - **send-http-result-output-format:** json - **send-http-url:** [http://localhost:8080](http://localhost:8080) [![](https://github.com/fabien-github/duplicati_dashboard/raw/main/docs/img/backup_options_report_config.png)](https://github.com/fabien-github/duplicati_dashboard/blob/main/docs/img/backup_options_report_config.png) > This assumes that your Duplicati instance is on the same host as your Duplicati Dashboard. ### Connect to your dashboard
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#connect-to-your-dashboard)
[http://localhost:3000](http://localhost:3000) Login: `admin` Password: `password` > For email alerting, you need to configure a SMTP relay. See [Configuration > Env file](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#env-file) # Configuration
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#configuration)
## Env file
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#env-file)
The file [`config.env`](https://github.com/fabien-github/duplicati_dashboard/blob/main/config.env) is used to configure some options. It will be shared between the 3 containers. Only use this default configuration for testing purposes. Influxdb variable will create and setup the database only on the first startup.
VariablesDefaultDescription
DOCKER\_INFLUXDB\_INIT\_MODEsetup[Automatically bootstrap the system](https://github.com/docker-library/docs/blob/master/influxdb/README.md#automated-setup)
DOCKER\_INFLUXDB\_INIT\_USERNAMEtelegraf\_userInfluxdb superadmin user
DOCKER\_INFLUXDB\_INIT\_PASSWORDtelegraf\_passwordInfluxdb superadmin password
DOCKER\_INFLUXDB\_INIT\_ORGtelegraf\_orgInfluxdb Organization (used by influxdb / telegraf / grafana)
DOCKER\_INFLUXDB\_INIT\_BUCKETtelegrafInfluxdb bucket to store reports (used by influxdb / telegraf / grafana)
DOCKER\_INFLUXDB\_INIT\_ADMIN\_TOKENtelegraf\_tokenInfluxdb superadmin token (used by influxdb / telegraf / grafana)
DOCKER\_INFLUXDB\_INIT\_RETENTION Influxdb data retention, default will retain forever
INFLUXD\_REPORTING\_DISABLEDfalseDisable [InfluxData telemetry](https://docs.influxdata.com/influxdb/v2.0/reference/config-options/#reporting-disabled)
TELEGRAF\_LISTENER\_PORT8080Port used by [http\_listener\_v2](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/http_listener_v2) input, endpoint for the reports sent by Duplicati
TELEGRAF\_LISTENER\_PATH/Path to listen to
GF\_SECURITY\_ADMIN\_USERadminGrafana superadmin user
GF\_SECURITY\_ADMIN\_PASSWORDpasswordGrafana superadmin password
GF\_SERVER\_ROOT\_URL[http://localhost:3000](http://localhost:3000)Grafana URL, used in some templates like email notifications
GF\_DASHBOARDS\_DEFAULT\_HOME\_DASHBOARD\_PATH/etc/grafana/provisioning/dashboards/duplicati\_dashboard.jsonForce Duplicati dashboard by default on home page
GF\_SMTP\_ENABLEDfalse[Set to true for email notifications](https://grafana.com/docs/grafana/latest/administration/configuration/#smtp)
GF\_SMTP\_HOSTlocalhost:25SMTP relay server. \[host\]:\[port\]
GF\_SMTP\_FROM\_NAMEGrafanaName of the email sender
GF\_SMTP\_USER In case of SMTP auth
GF\_SMTP\_PASSWORD In case of SMTP auth
GF\_SMTP\_FROM\_ADDRESSAddress used when sending out emails
GF\_SMTP\_EHLO\_IDENTITY${HOSTNAME}Name to be used as client identity for EHLO in SMTP dialog (Default will be the container ID)
GF\_SMTP\_STARTTLS\_POLICY “OpportunisticStartTLS”, “MandatoryStartTLS”, “NoStartTLS”
NOTIFIER\_EMAIL\_RECIPIENTRecipients for email notification (separated by a semicolon)
NOTIFIER\_EMAIL\_REMINDER\_ENABLEtrueRe-send an email if alerts are still active
NOTIFIER\_EMAIL\_REMINDER\_FREQUENCY2hDelay between email reminders
# Notes
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#notes)
## Grafana configuration locked
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#grafana-configuration-locked)
The dashboard is locked by the Grafana provisioning system. You can't edit the datasource, the dashboard or the alert notifier from the UI. You will need to copy the dashboard or disable the provisioning configuration. Dashboard path: `./grafana/provisioning/dashboards/duplicati_dashboard.json` The idea is to keep the stack easy to deploy for everyone without investing time to learn Grafana configuration. Feel free to fork the project or directly edit files on your own. More information [here](https://grafana.com/docs/grafana/latest/administration/provisioning/). ## Backup over more than 30 days rotation
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#backup-over-more-than-30-days-rotation)
- Grafana will discover your backups name from the reports but only over the last 90 days. So if your backups are scheduled for more than 90 days, you will need to edit the request of the variable `Backup` in the dashboard configuration: ``` from(bucket: v.defaultBucket) |> range(start: -90d) |> filter(fn: (r) => true) |> toString() |> group(columns: ["backup-name"]) |> distinct(column: "backup-name") |> keep(columns: ["_value"]) ```
- Last reported status / Last reported variations / Alerting graph are based over the past 30 days. You will need to adapt each panel requests if your backups are scheduled over more than 30 days. ``` import "influxdata/influxdb/schema" from(bucket: v.defaultBucket) |> range(start: -30d) ... ```
- In [v2](https://github.com/fabien-github/duplicati_dashboard/releases/tag/v1.0.0), alerts have their own provisioning file and query range can be edited [here](https://github.com/fabien-github/duplicati_dashboard/blob/3e1d558d54c6b3ecbb7ed67627a812c5dd060f31/grafana/provisioning/alerting/duplicati_alert.yml#L30). ## Alerting graph
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#alerting-graph)
The section "Alerting graph" is only used to trigger an alert when a backup fails. This is due to the lack of grafana alert support on other panel type. [\#6983](https://github.com/grafana/grafana/issues/6983) Backups status will be checked every minutes. An alert will be triggered after a pending status of 10min. Same delays are used on the recovery. Warning reports don't trigger an alert. ## Deleting removed backup data
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#deleting-removed-backup-data)
After getting inside the influxdb container: ``` influx delete \ --org telegraf_org \ --bucket telegraf \ --token "telegraf_token" \ --start 1970-01-01T00:00:00Z \ --stop $(date +"%Y-%m-%dT%H:%M:%SZ") \ --predicate '_measurement="duplicati" AND "backup-name"="backup_to_delete"' ```
## Docker-compose
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#docker-compose)
- **Telegraf**: Receive JSON reports from Duplicati. - **Influxdb**: Store reports converted by Telegraf. - **Grafana**: Requests Influxdb to generate dashboard and alerts. Telegraf endpoint provides a limited [HTTP autentification](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/http_listener_v2). The configuration file is located here : `./telegraf/telegraf.conf` Feel free to add a proxy like traefik or nginx to protect the stack on an unsecure network. (TLS, IP Restrictions, ...) # Other informations
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#other-informations)
- Not sure of the scalabilty, requests to the database are not efficients. Timeseries databases are not really adapted for this kind of data. This is mainly due to the nested and uneven json format from the reports and the variation time between reports. - Features are limited directly by the stack itself. For example, it's nearly impossible to add a managment system for the backups. - This project has no link with the development of Duplicati and his team. # License
[](https://github.com/fabien-github/duplicati_dashboard?tab=readme-ov-file#license)
Distributed under the GNU General Public License v3.0 License. See [LICENSE](https://github.com/fabien-github/duplicati_dashboard/blob/main/LICENSE) for more information. # Duplicati Dashboard2 Link: [https://github.com/wchorski/duplicati-dashboard](https://github.com/wchorski/duplicati-dashboard) git clone [https://github.com/wchorski/duplicati-dashboard.git](https://github.com/wchorski/duplicati-dashboard.git) A NodeJS based server that collects JSON data from Duplicati backup logs 📦 Tech Frontend: NextJS API: NextJS Database: InfluxDB \[!warning\] Backups sharing the same name will cause issues. The backup's name must be unique across all Duplicati instances, it will be used as an ID for logging. MUST BE URL FRIENDLY example: "Laptop--Home\_Folder\_Backup", "Desktop--Home\_Folder\_Backup" is a good naming convention. Usecase Initally this was just some middle ware that serves as an endpoint for JSON friendly monitoring apps, but I also built a simple UI so it could be used as a standalone app. Duplicati Setup you can either add these settings for the globally or per backup in the Advanced options send-http-result-output-format = json send-http-url = "http://APPSDOMAIN/api/backups" API Here is a breakdown of what endpoints and search parameters that can be passed through. URL Breakdown http://APPDOMAIN/backups/BACKUP\_ID?start=-5h&first=true BACKUP\_ID => the id (or name) of the backup saved stuff after the "?" search query sets range of time of pulled data start => how far back to you want to start pulling data i.e. -40d 40 days ago \[the default\] -5h 5 hours ago -60m 60 miniutes ago 1999-12-31T00:00:00 starting on December 31st, 1999 1694117521 starting on this UNIX time (seconds) stop => the end of the range now() => my current time \[the default\] all the other examples above as long as the date is after the start both query parameters can be omitted last => set to true if you'd like the last recorded point in the table can also use the http://localhost:3000/api/backups/last/BACKUP\_ID endpoint for cleaner GET (this endpoint also takes start and stop query parameters) first => same as last but returns the first point of recoreded data within the range \[!note\] make sure relative dates have a negative i.e. -5h as your are looking back in time. Positive time values will cause errors Examples query url all backup stats in database http://APPDOMAIN/backups single backup stats http://APPDOMAIN/backups/BACKUP\_ID last recorded backup stat http://APPDOMAIN/backups/last/BACKUP\_ID same as above http://APPDOMAIN/backups/BACKUP\_ID?last=true last recorded backup stat in the last 5 hours http://APPDOMAIN/backups/last/BACKUP\_ID?start=-5h ⚙️ Development git clone https://github.com/wchorski/duplicati-dashboard.git && cd duplicati-dashboard cp .env.template .env.local set up InfluxDB instance get InfluxDB API Token for .env.local yarn install yarn dev 🏭 Production git clone https://github.com/wchorski/duplicati-dashboard.git && cd duplicati-dashboard cp .env.template .env the INFLUX\_TOKEN in .env should be a long ~88 character string docker compose up -d Home Assistant rest: \- authentication: basic username: "admin" password: "password" scan\_interval: 86400 resource: http://APPDOMAIN.lan/api/backups/last/DUPLICATI\_ID sensor: \- name: "duplicati-DUPLICATI\_ID-status" value\_template: "{{ value\_json.status }}" \- name: "duplicati-DUPLICATI\_ID-time" value\_template: > {% set thistime = value\_json.time %} {{ as\_timestamp(thistime) | timestamp\_custom("%Y %M, %d %H:%M") }} \#Todo create dynamic nav based on unique duplicati\_ids from database add FAQ as a page inside the app mobile friendly (almost there) graph trends in app Home Assistant Template sensor Don't be lazy and figure out Types in TableClient.tsx component get real data for screenshots why is bg tile image weird when scrolling on mobile? human readable bytes formatter (gb tb) human readable duration formatter