Child pages
  • Metadata for Backups
Skip to end of metadata
Go to start of metadata

Overview

The /usr/local/cpanel/scripts/backups_create_metadata script allows a user to create metadata files for all backups or retrieve information about a specific backup or user.

Important:

If you configure incremental backups with additional destinations that do not support incremental backups, the following will happen:

  • The system will transfer system backups only to those destinations that do not support incremental backups.
  • The system will not transfer accounts to destinations that do not support incremental backups.

If you configure backups for compressed or uncompressed backups, any transport will work as expected for accounts and system files.

The backups_create_metadata script

The following table lists the three available parameters for this script:

ParameterDescriptionExample
--all

Creates metadata files for all backups.

Note:

This parameter does not work with any other parameter.

/scripts/backups_create_metadata --all=/backup
--backup

Creates metadata files for any single backup.

/scripts/backups_create_metadata --backup=/backup/monthly/2017-03-01
--user=[user]

This parameter is optional.

Creates metadata files for a specific user within the desired backup.

Note:

You must use this parameter with the --backup parameter to create a metadata file for a user within a backup.

/scripts/backups_create_metadata --backup=/backup/monthly/2017-03-01 --user=abc1

The /backup directory structure

The following code block diagram resembles what the layout and structure of the /backup directory might look like:

/backup
├── 2017-03-08
│   ├── accounts
│   │   ├── abc1-=-meta
│   │   ├── abc1
│   └── system
│       ├── dirs
│       └── files
├── monthly
│   └── 2017-03-15
│       ├── accounts
│       |   ├── abc1-=-meta
│       |   ├── abc1.tar
│       └── system
└── weekly
    ├── 2017-03-05
        ├── accounts
        |   ├── abc1-=-meta
        |   ├── abc1.tar.gz
        └── system

The following table explains the diagram above in further detail:

ValueDescription
2017-03-08

A daily incremental backup, named by the time stamp. Monthly and weekly backup directories follow the same structure.

Note:

For this example, 2017-03-08 represents an incremental daily backup for the accounts and system backup directories.

abc1The user in this example.
/backup/2017-03-08/accounts/abc1The directory where all of the user's files reside.
2017-03-15        
An uncompressed monthly backup.
/backup/monthly/2017-03-15/accounts/abc1.tarThe monthly uncompressed user's backup file.
abc1.tarA tarball, or a single archive file that contains all of user's (abc1) backup files.
2017-03-05        
A compressed weekly backup.
/backup/weekly/2017-03-05/accounts/abc1.tar.gzThe weekly compressed user's backup file.
abc1.tar.gzA compressed tarball, similar to the one in the monthly backup.
abc1-=-meta

The companion file for a compressed, uncompressed, or incremental backup directory. For example, abc1.tar.gz for a compressed file; abc1.tar for an uncompressed file; abc1 for an incremental backup directory. This file contains the same formatting regardless of the backup file-type.

Note:

The metadata file contains "high level" information about the backup itself. The metadata file also contains a list of all of the files in the specified backup.

Metadata values

The following table represents the primary metadata values for backups, known as "high level" data:

Note:

The format of this "high level" data is in the Comma Separated Value (CSV) format. Each value name is separated by a comma from its actual value.

ValueDescriptionExample
server

The hostname of the server.

example1.dev.cpanel.net
UID

The user ID of the user on the server.

Note:

This is always a number.

2183
GID

The group ID of the user's files.

Note:

This is always a number.

2186
usernameThe username of the user for the backed up files.abc1
dateThe date the backup took place.2017-02-28
epoch

The date in Unix epoch time.

Note:

This is always a large number.

1488261600
archive_size

The size of an archive in bytes. For example, the size of the abc1.tar.gz file.

653824
uncompressed_sizeThe size of an uncompressed archive in bytes.531748
pkgacct_version

The version of the package account program.

Note:

cPanel & WHM uses the pkgacct program to create the archives.

10
archive_versionThe program version used to prepare the archive.3
file_countThe number of files in the archive.108
------------------

The dashes represent the end of the "high level" metadata for an archive.

Note:

Everything that follows the list of dashes consists of the files in the archive.

 

List of files in the archive

For the example above, the following code block represents the list of files found in the archive:

720,"2017-01-30 09:16",abc1/cp/abc1,/backup/2017-02-28/accounts/abc1.tar,0,2186 
4221,"2017-03-01 09:11",abc1/dnszones/abc1.tld.db,/backup/2017-02-28/accounts/abc1.tar,0,0
1731,"2017-03-01 09:11",abc1/dnszones/add1abc1.tld.db,/backup/2017-02-28/accounts/abc1.tar,0,0
1675,"2017-01-16 07:44",abc1/domainkeys/private/abc1.tld,/backup/2017-02-28/accounts/abc1.tar,0,12 

Metadata file format example

The following code block represents an example of a compressed or uncompressed backup file format:

server,example1.dev.cpanel.net
uid,2183
gid,2186
username,abc1
date,2017-02-28
epoch,1488261600
archive_size,653824
uncompressed_size,531748
pkgacct_version,10
archive_version,3
file_count,108
-------------------------------------------------
720,"2017-01-30 09:16",abc1/cp/abc1,/backup/2017-02-28/accounts/abc1.tar,0,2186
4221,"2017-03-01 09:11",abc1/dnszones/abc1.tld.db,/backup/2017-02-28/accounts/abc1.tar,0,0
1731,"2017-03-01 09:11",abc1/dnszones/add1abc1.tld.db,/backup/2017-02-28/accounts/abc1.tar,0,0
1675,"2017-01-16 07:44",abc1/domainkeys/private/abc1.tld,/backup/2017-02-28/accounts/abc1.tar,0,12
....

File formats

Backups use two file formats, based on the type of backup. One format handles compressed and uncompressed backups, while the second handles incremental backups. 

Compressed and uncompressed backups

The following list displays the file formats for compressed and uncompressed backups:

File Format Name
File Size (in bytes)
File Modified Date
File name internal to the archive
Full path to the archive file
File UID
File GID

Incremental backups

An incremental backup is a type of backup that only copies files that have changed since the previous backup. This option will only save backup information that has changed since your last backup.  Incremental backups use the same "high level" data as uncompressed and compressed backups. However, instead of the internal archive file name, it displays the word "FILE", "DIR", or "SYMLINK". 

Notes:

  • The actual file resides on the disk.
  • The system lists the full path to the file, and the archive represents the whole file.
  • If additional destinations is enabled, with backups set to incremental, the system will only transfer the system backup.
The following list displays the file formats for incremental backups:

File Format Name

File Size (in bytes)

File Modified Date
Full path to the archive file
the word "FILE", "DIR", or "SYMLINK"
File UID
File GID

The following code block represents an example of an incremental backup file format:

server,example1.dev.cpanel.net
uid,2183
gid,2186
username,abc1
date,2017-03-22
epoch,1490158800
archive_size,836
uncompressed_size,533260
pkgacct_version,10
archive_version,3
file_count,110
-------------------------------------------------
39,"2017-03-22 11:54",/backup/2017-03-22/accounts/abc1/version,FILE,0,0
11,"2017-03-22 11:54",/backup/2017-03-22/accounts/abc1/homedir_paths,FILE,0,0
0,"2017-03-22 11:54",/backup/2017-03-22/accounts/abc1/has_sslstorage,FILE,0,0
8,"2017-03-22 11:54",/backup/2017-03-22/accounts/abc1/ssldomain,FILE,0,0

Additional documentation

  • No labels