rubackup - ruby based backup program for Linux
This page explains how to write configuration files for rubackup in general. It does not provide a list of all possible options. Please refer to the module specific pages for these details.
The configuration is stored in yaml files, hence you really have to understand the YAML format first. Basically this is an easy way to represent structured data in text file in a way which is similar to JSON. Data in a yaml file correspond to hashes and lists once they have been loaded in ruby, hence they are easy to read and use without any extra processing required.
The configuration can be stored either in a single monolithic yaml file, or it can be split in multiple yaml files. This can be useful when the configuration is quite large as one separate configuration file can be created for each project. It also allows to share the configuration between multiple systems when part of the configuration are common on multiple systems. It is recommended to use different yaml files for the global section, the scheduling, and the various backup entries. When you use multiple yaml files this does not affect the way the configuration works. All files will be loaded and merged in memory and behave just like a single file. Hence you can put any type of section in any yaml file as long as they are valid yaml files.
By default rubackup is going to read the following files:
/etc/rubackup.d/*.yaml. This can be overridden using command line:
rubackup.rb --config="/etc/somewhere/one-file-configuration.yaml" rubackup.rb --config="/etc/somewhere/multiple-files/*.yaml"
There are multiple sections: the global section is mandatory and applies to everything. It defines for example which day should be used to make weekly backups. There are also resource sections. They define resources such as a list of S3 buckets or notification media which can be used to send an email at the end of the run. There are also backup entries which define what to do for a particular backup.
Here is a minimal configuration example which shows how to manage two
type of backups which correspond to the document root and mysql database
of a website. This configuration file can be put in the default
directory in a file such as
/etc/rubackup.d/rubackup.yaml so it gets
picked up without having to pass any command line option to refer to it.
--- global: day_of_week: Mon day_of_month: 1 schedules: myschedule_long_term: daily: 7 weekly: 4 monthly: 12 myschedule_short_term: daily: 7 weekly: 4 monthly: 3 entries: website1-tar: enabled: true backup_type: ModuleBackupTarball backup_opts: includes: - /var/www/website1 - /etc/httpd/conf.d/website1.conf excludes: - '.git' - '.svn' command_opts: - '--sparse' compress_prog: gzip compress_opts: - '-6' backup_schedule: myschedule_short_term bakfile_dir: /backup/backup-web bakfile_basename: backup-website1-www bakfile_owner: root bakfile_group: root bakfile_mode: 0600 website1-sql: enabled: true backup_type: ModuleBackupMysqldp backup_opts: dbhost: 127.0.0.1 dbuser: website1 dbname: website1 dbpass: 'SqlPassForWebSite1' compress_prog: xz compress_opts: - '-4' - '--memlimit-compress=80MiB' backup_schedule: myschedule_long_term bakfile_dir: /backup/backup-web bakfile_basename: backup-website1-sql bakfile_owner: root bakfile_group: root bakfile_mode: 0600
This minimal example shows there are three top level mandatory sections:
There are not many global options. The ones used here say monthly backups are created every Monday and monthly backups are created the first of each month.
You can have a single schedule which you reuse for all entries or you
can have multiple schedules. You must give a name to each schedules, in
this case they are
For example in the
myschedule_long_term schedule backups will be
created everyday and then it will keep the last 7 days, 4 weeks (the
last 4 mondays) and the last 12 months (first of the month for the last
12 months). Other backups will be deleted.
The entries are the biggest section as it lists all the backups to
manage and all their specific options. Entries have options in common.
For example all entries must have a
backup_schedule as rubackup needs
to know what the creation / retention policy is for all backups. The
enabled option can be set to false to temporarily disable a backup
without having to delete its configuration. Most backups (all backups
that generate a local file) require
bakfile_dir to specify where the
backup will created,
bakfile_basename to specify the name to give to
the backup files (the date and extensions will be added automatically to
the basename), and the ownerships and permissions. Each backup entry has
backup_opts section where the options depend on the module type
You can see the backup module which produces tar archives requires a
list of files and directories to be included and excluded in the
includes option must be a list even when there is just
one item, hence the lines must start with a dash. The
is optional and is also a list. The
command_opts option is also not
mandatory and it can be used to provide extra arguments that must be
passed to the
tar command line.
The mysql backup module requires different options which are the
database connection details. The
dbhost option is mandatory as
127.0.0.1 will be used by default if it is not specified. The password
is also optional as the mysqldump command won’t need a password if it is
These two modules support a
compress_prog option which you can use to
set the compression program to run in the pipeline (can be lzop, lz4,
gzip, bzip2, xz). The
xz program will be used by default if it is not
specified. They also both support
compress_opts which is not mandatory
and which can be used to provide extra command arguments to the
compression program. This can be used to change the compression
The previous configuration would produce the following type of backup
files. The date in the
YYYYMMDD format will automatically be added. It
is important as this date in the file name is what rubackup uses to
determine when backups need to be deleted. Also a checksum file is
automatically created and deleted when backups expire.
/backup/backup-web/backup-website1-www-20141201.tar.gz /backup/backup-web/backup-website1-www-20141201.tar.gz.sha256 /backup/backup-web/backup-website1-www-20141202.tar.gz /backup/backup-web/backup-website1-www-20141202.tar.gz.sha256 /backup/backup-web/backup-website1-sql-20141201.sql.xz /backup/backup-web/backup-website1-sql-20141201.sql.xz.sha256 /backup/backup-web/backup-website1-sql-20141202.sql.xz /backup/backup-web/backup-website1-sql-20141202.sql.xz.sha256
Here is an example which shows all the options currently supported in
global section of the configuration:
--- global: day_of_week: Sun day_of_month: 1 notify_type: ModuleNotifyAwsSes notify_opts: ses_media: my_ses_media path_extra: - /sbin - /usr/sbin - /usr/local/sbin aws_ses_medias: my_ses_media: awsregion: us-west-2 mailsrc: [email protected] maildst: [email protected] accesskey: access_key_ses aws_access_keys: access_key_ses: public: 'Your_AWS_Access_Key_Here_PUBLIC' secret: 'Your_AWS_Access_Key_Here_SECRET'
path_extra option allows you to provide a list of directories
where commands required to perform backups are located. You can use it
when directories are not in your PATH environment variable. The
notify_opts options let you use a notification
module which also has its own configuration and may refer to resources
such as AWS Access Keys.
Here is an example which shows how to use multiple configuration files
with rubackup. This is more flexible and useful in environments with
many modules and backup entries especially when part of the
configuration is identical on multiple systems. By default rubackup will
attempt to load all yaml files in
/etc/rubackup.d/ hence it is
recommended to use this default configuration directory.
The first file is
/etc/rubackup.d/010-global.yaml and it contains the
--- global: day_of_week: Sat day_of_month: 1 path_extra: - /sbin - /usr/sbin
Here is another file:
/etc/rubackup.d/020-schedules.yaml to define the
--- schedules: my_sched_tarballs: daily: 7 weekly: 4 monthly: 2 my_sched_databases: daily: 30 weekly: 52 monthly: 36
Now we can have one file such as
two backup entries related to a particular project.
--- entries: website1-tar: enabled: true backup_type: ModuleBackupTarball backup_opts: includes: - /var/www/website1 - /etc/httpd/conf.d/website1.conf backup_schedule: my_sched_tarballs bakfile_dir: /backup/backup-web bakfile_basename: backup-website1-www bakfile_owner: root bakfile_group: root bakfile_mode: 0600 website1-sql: enabled: true backup_type: ModuleBackupMysqldp backup_opts: dbhost: 127.0.0.1 dbuser: website1 dbname: website1 dbpass: 'SqlPassForWebSite1' backup_schedule: my_sched_databases bakfile_dir: /backup/backup-web bakfile_basename: backup-website1-sql bakfile_owner: root bakfile_group: root bakfile_mode: 0600
And we can have another file such as
for another project.
--- entries: website2-tar: enabled: true backup_type: ModuleBackupTarball backup_opts: includes: - /var/www/website2 - /etc/httpd/conf.d/website2.conf backup_schedule: my_sched_tarballs bakfile_dir: /backup/backup-web bakfile_basename: backup-website2-www bakfile_owner: root bakfile_group: root bakfile_mode: 0600 website2-sql: enabled: true backup_type: ModuleBackupMysqldp backup_opts: dbhost: 127.0.0.1 dbuser: website2 dbname: website2 dbpass: 'SqlPassForWebSite2' backup_schedule: my_sched_databases bakfile_dir: /backup/backup-web bakfile_basename: backup-website2-sql bakfile_owner: root bakfile_group: root bakfile_mode: 0600