Page tree
Skip to end of metadata
Go to start of metadata

Overview

System administrators can manage whether to create metadata for backups. Every time that cPanel & WHM creates a backup, it creates metadata for that backup. You can also manually generate backup metadata with the  backups_create_metadata script. There are several settings that disable metadata creation. You may want to disable metadata creation for testing, before an upgrade, or if your server experiences performance issues.

Note:

When you disable metadata creation, the system disables the File and Directory Restoration interfaces in cPanel & WHM version 72 and later. WHM's File and Directory Restoration interface (Home >> WHM >> Backups >> File and Directory Restoration) will indicate whether it is disabled. cPanel will hide the File and Directory Restoration interface (Home >> cPanel >> Files >> File and Directory Restoration) from the Files section of the Home interface.

In version 70, the system disables the File Restoration interfaces in cPanel & WHM, with the same behaviors noted above for the respective interfaces.

Enable metadata creation

Metadata for backups version 3.1

Every time that cPanel & WHM creates a backup, it creates metadata for that backup. The system stores that metadata as entries in a username.db database (where username is the cPanel account's username). Then, the system saves the database to the .meta directory under your configured backup directory. The metadata databases store the indexed information of their related backups. These smaller database entries provide a faster information-retrieval method than their source backup.

You can use the backups_create_metadata script to manually generate metadata for a backup.

Note:

The script will generate metadata for all directories under the /home/username directory, except for the /mail and /.cpanel directories.

The backup_paths table

The backup_paths table lists the backup files' paths.

FieldTypeDescriptionPossible valuesExample
backup_pathstringThe backup files's filepath, relative to the configured backup directory.A string value./backup/2018-04-12/accounts
backup_idintegerThe backup file's identification number.

A positive integer.

This value references the backup_id value in the backups table.

1

The backups table

The backups table lists the backup files on the disk.

FieldTypeDescriptionPossible valuesExample
backup_idintegerThe backup file's identification number.An automatically-incrementing positive integer.1
timestamptimestampThe backup files's creation date in Universal Time Coordinated (UTC) format.An integer.1523642274
does_existBooleanWhether the backup file exists.
  • 1 — Exists.
  • 0 — Does not exist.

1

The file_changes table

The file_changes table lists backup file changes when any of the following actions occur:

  • The first time the system backs up the file.
  • The user modifies the file.
  • The user removes the file.
FieldTypeDescriptionPossible valuesExample
seen_files_idintegerThe filepath's identification number.

A positive integer.

This value references the file_id value in the seen_files table.

1
backup_idintegerThe backup file's identification number.

A positive integer.

This value references thebackup_id value in the backups table.

1
sizeintegerThe backup file's size, in bytes.
  • A file size value displays the file's size.
  • A directory or a symlink displays a 0 file size.
660
mtimeintegerThe date when the user last modified the file, in UTC format.

An integer.

1523642274
operationintegerThe change type.
  • 0 — The user created the backup file.
  • 1 — The user changed the file.
  • 2 — The user removed the file.
0
typeintegerThe backup file's type.
  • 0 — A file.
  • 1 — A directory.
  • 2 — A symlink.
0

The metadata table

This table stores metadata keys and values.

FieldTypeDescriptionPossible valuesExample
keystringThe metadata code type.

schema_version — The metadata schema's version.

schema_version
valuestringThe metadata code's version number.

A string value.

3.1

The seen_files table

This table stores filenames.

FieldTypeDescriptionPossible valuesExample
file_idintegerThe file's identification number.An automatically-incrementing positive integer.1
pathstringThe file's filepath, relative to the /home/username directory.A string value./public_html/

Note:

  • The system enables metadata creation automatically in cPanel & WHM version 66 and 68. You cannot disable metadata creation in these versions.
  • If you upgrade from version 66 or 68 to later versions of cPanel & WHM, your backup configuration may affect whether the File Restoration interface (version 70) or the File and Directory Restoration interface (version 72 and later) is disabled or visible. For more information, see the cPanel & WHM version 70 or cPanel & WHM version 72 or later tabs in the Disable metadata creation section of this document.

Every time cPanel & WHM creates a backup, it also generates a backup metadata file that maps the contents of that backup's directory. The metadata lists files and directories in the backup and stores the information in CSV format. You can also manually generate a backup metadata file with the backups_create_metadata script.

Metadata files store indexed information of their related backup. These smaller database entries provide a faster information-retrieval method than their source backup.

Backup metadata format

A backup metadata file, or metafile, has two sections. The first section contains the metafile's meta attributes and appears above a dotted line. The second section provides details of the backup archive's files and appears below the dotted line.

Compressed and uncompressed backups metafile example

A metafile for compressed and uncompressed backups will resemble the following example:

server,example.cpanel.net
uid,2183
gid,2186
username,abc1
metaversion,1
date,2017-02-28
epoch,1488261600
archive_size,653824
uncompressed_size,531748
pkgacct_version,10
archive_version,3
file_count,108
-------------------------------------------------
720,"2017-01-30 09:16",YWJjMS9jcC9hYmMx,/backup/2017-02-28/accounts/abc1.tar,0,2186
4221,"2017-03-01 09:11",YWJjMS9kbnN6b25lcy9hYmMxLnRsZC5kYg==,/backup/2017-02-28/accounts/abc1.tar,0,0
1731,"2017-03-01 09:11",YWJjMS9kbnN6b25lcy9hZGQxYWJjMS50bGQuZGI=,/backup/2017-02-28/accounts/abc1.tar,0,0
1675,"2017-01-16 07:44",YWJjMS9kb21haW5rZXlzL3ByaXZhdGUvYWJjMS50bGQ=,/backup/2017-02-28/accounts/abc1.tar,0,12
....

Incremental backups metafile example

A metafile for incremental backups will resemble the following example:

server,example.cpanel.net
uid,2183
gid,2186
username,abc1
metaversion,1
date,2017-03-22
epoch,1490158800
archive_size,836
uncompressed_size,533260
pkgacct_version,10
archive_version,3
file_count,110
-------------------------------------------------
39,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvdmVyc2lvbg==,FILE,0,0
11,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvaG9tZWRpcl9wYXRocw==,FILE,0,0
0,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvaGFzX3NzbHN0b3JhZ2U=,FILE,0,0
8,"2017-03-22 11:54",L2JhY2t1cC8yMDE3LTAzLTIyL2FjY291bnRzL2FiYzEvc3NsZG9tYWlu,FILE,0,0

Meta attributes

The meta attributes section displays backup archive values, in CSV format, and uses a attribute name, attribute value convention. 

AttributeDescriptionExample
server

The server's hostname.

example.cpanel.net
uid

The system user's identification number on the server.

2183
gid

The user's file's group identification number.

2186
usernameThe system user that owns the backed-up files.abc1
metaversionThe metafile's format version.1
dateThe date when the system backed up the file.2017-02-28
epoch

The date in Unix epoch time.

1488261600
archive_size

The archive's size, in bytes. For example, the size of the abc1.tar.gz file.

653824
uncompressed_sizeAn uncompressed archive's size, in bytes.531748
pkgacct_version

The package account script's version.

Note:

cPanel & WHM uses the pkgacct script to create the archives.

10
archive_versionThe program version that the system uses to prepare the backup archive.3
file_countThe number of files in the backup archive.108

Backup archive files' details

This section displays backup's details in CSV format. The format uses two conventions: one for compressed and uncompressed backup files, and one for incremental backups.

Notes:

  • The backup resides on the disk.
  • The file details list the full path to the backup. The backup archive contains the backup's entire data.

Compressed and uncompressed backup file details


PartDescription
File sizeThe backup file's size, in bytes.
Last modified dateThe date when the system last modified the file.
The file's archive nameThe file's name in the archive, stored in Base64 format.
The file's full path in the archiveThe full path to the file in the archive.
UIDThe system user's user identification number on the server.
GIDThe user's file's group identification number.


Incremental backup details


PartDescription
File sizeThe backup's size, in bytes.
Last modified dateThe date when the system last modified the backup.
The file's archive nameThe backup's name in the archive, stored in Base64 format.
The file's full path in the archiveThe full path to the backup in the archive.
The archived file's type

One of these archived file types:

  • FILE — A file.
  • DIR — A directory.
  • SYMLINK — A symlink.
UIDThe system user's identification number.
GIDThe user's file's group identification number.


Notes:

The system can only store the system backup (/backup/.../system) as a compressed or uncompressed backup file. The system cannot store a system backup as an incremental backup file. This condition creates a limitation when you configure a remote destination for your backups in the Additional Destinations section in WHM's Backup Configuration interface (WHM >> Home >> Backup >> Backup Configuration). Because the system can only store system backup files as compressed or uncompressed files, the system copies the system backup file to the destination.

The /backup directory structure

The system stores backups inside the /backup directory in one of three different formats: compressed, uncompressed, and incremental. Each format uses a different filename extension.

Backup formatFilename extensionExample
compressed.tar.gzusername.tar.gz
uncompressed.tarusername.tar
incrementalNoneusername

The system distinguishes each backup by its filepath, even if the system stores the backup information under the same name. The filepath includes the backup directory where the backup resides. The following example shows the layout and structure of a /backup directory.

This directory includes daily, monthly, and weekly uncompressed backups. You can distinguish each backup type by its complete filepath. This filepath is the file's full path shown in the metafile's backup file details.

The system also stores metafiles in the /backup directory and distinguished by their filepath. The system stores metafiles inside the /backup directory as username-=-meta, where username represents the system user's name.

Disable metadata creation

You can manually disable metadata with WHM's Backup Configuration interface (WHM >> Home >> Backup >> Backup Configuration). The following table shows which settings affect the management of metadata manually and the Backup Configuration interface. These settings function independently from one another.

Notes:

  • When you configure any of these settings to disable metadata creation, the following actions occur:
    • WHM disables the File and Directory Restoration interface (Home >> WHM >> Backup >> File and Directory Restoration). The interface displays a warning that indicates the setting that disabled it.
    • cPanel hides the File and Directory Restoration interface (Home >> cPanel >> Files >> File and Directory Restoration) from the cPanel Home interface.
  • You can disable metadata with the WHM API 1 backup_config_set function.


Backup configuration file settingBackup Configuration interface settingTypeDescriptionPossible values
DISABLE_METADATA

None.

You must manually configure this setting with the WHM API 1 backup_config_set function.

string

Whether the Backup system creates metadata when a backup runs.

  • yes — Disables metadata creation.
  • no — Enables metadata creation.
BACKUPACCTSBackup Accountsstring

Whether the system includes cPanel user accounts in the backup.

Note:

System administrators can select which cPanel user accounts they wish to include in the backup through the Backup User Selection interface (WHM >> Home >> Backup >> Backup User Selection).

  • yes or selected checkbox — Include the cPanel user accounts in the backup and enable metadata creation.
  • no or deselected checkbox — Do not include the cPanel user accounts in the backup and disable metadata creation.
BACKUPENABLEBackup StatusstringWhether the WHM System Administrator wants backups enabled.
  • yes or set toggle to Enable — Enables backup and metadata creation.
  • no or set toggle to Disable — Disables backup and metadata creation.
BACKUPMOUNTMount Backup Drive as NeededBoolean

Whether the system mounts the backup directory as a mount point before a backups runs, then unmounts it when the run completes.

  • 1 or selected checkbox — Mount the backup directory and disable metadata creation.
  • 0 or deselected checkbox — Do not mount the backup directory and enable metadata creation.
KEEPLOCALRetain backups in the default backup directory.Boolean

Whether the system retains backups in the default local backup directory.

  • 1 or selected checkbox — Retains the backup and enable metadata creation.
  • 0 or deselected checkbox — Does not retain the backup and disable metadata creation.

You can manually disable metadata with WHM's Backup Configuration interface (WHM >> Home >> Backup >> Backup Configuration). The following table shows which settings affect the management of metadata manually and the Backup Configuration interface. These settings function independently from one another.

Notes:

  • When you configure any of these settings to disable metadata creation, the following actions occur:
    • WHM disables the File Restoration interface (Home >> WHM >> Backups >> File Restoration). The interface will display a warning that indicates the setting that disabled it.
    • cPanel hides the File Restoration interface (Home >> cPanel >> Files >> File Restoration) from the cPanel Home screen.
  • You can disable metadata with the WHM API 1 backup_config_set function.
Backup configuration file settingBackup Configuration interface settingTypeDescriptionPossible values
DISABLE_METADATA

None.

You must manually configure this setting with the WHM API 1 backup_config_set function.

string

Whether the Backup system creates metadata when a backup runs.

  • yes — Disables metadata creation.
  • no — Enables metadata creation.
BACKUPACCTSBackup Accountsstring

Whether the system includes cPanel user accounts in the backup.

Note:

System administrators can select which cPanel user accounts they wish to include in the backup through the Backup User Selection interface (WHM >> Home >> Backup >> Backup User Selection).

  • yes or selected checkbox — Include the cPanel user accounts in the backup and enable metadata creation.
  • no or deselected checkbox — Do not include the cPanel user accounts in the backup and disable metadata creation.
BACKUPENABLEBackup StatusstringWhether the system creates backups.
  • yes or set toggle to Enable — Enables backup and metadata creation.
  • no or set toggle to Disable — Disables backup and metadata creation.
KEEPLOCALRetain backups in the default backup directory.Boolean

Whether the system retains backups in the default local backup directory.

  • 1 or selected checkbox — Retains the backup and enables metadata creation.
  • 0 or deselected checkbox — Does not retain the backup and disables metadata creation.

BACKUPMOUNTMount Backup Drive as NeededBoolean

Whether the system mounts the backup directory as a mount point before a backups runs, then unmounts it when the run completes.

  • 1 or selected checkbox — Mount the backup directory and disable metadata creation.
  • 0 or deselected checkbox — Do not mount the backup directory and enable metadata creation.

Note:

  • The system enables metadata creation automatically in cPanel & WHM version 66 and 68. You cannot disable metadata creation in these versions.
  • If you upgrade from version 66 or 68 to later versions of cPanel & WHM, your backup configuration may affect whether the File Restoration interface (version 70) or the File and Directory Restoration interface (version 72 and later) is disabled or visible. For more information, see the cPanel & WHM version 70 or cPanel & WHM version 72 or later tabs in the Disable metadata creation section of this document.

Additional documentation