Package CedarBackup2 :: Package actions :: Module collect
[hide private]
[frames] | no frames]

Module collect

source code

Implements the standard 'collect' action.


Author: Kenneth J. Pronovici <pronovic@ieee.org>

Functions [hide private]
 
executeCollect(configPath, options, config)
Executes the collect backup action.
source code
 
_collectFile(config, absolutePath, tarfilePath, collectMode, archiveMode, resetDigest, digestPath)
Collects a configured collect file.
source code
 
_collectDirectory(config, absolutePath, collectMode, archiveMode, ignoreFile, linkDepth, dereference, resetDigest, excludePaths, excludePatterns, recursionLevel)
Collects a configured collect directory.
source code
 
_executeBackup(config, backupList, absolutePath, tarfilePath, collectMode, archiveMode, resetDigest, digestPath)
Execute the backup process for the indicated backup list.
source code
 
_loadDigest(digestPath)
Loads the indicated digest path from disk into a dictionary.
source code
 
_writeDigest(config, digest, digestPath)
Writes the digest dictionary to the indicated digest path on disk.
source code
 
_getCollectMode(config, item)
Gets the collect mode that should be used for a collect directory or file.
source code
 
_getArchiveMode(config, item)
Gets the archive mode that should be used for a collect directory or file.
source code
 
_getIgnoreFile(config, item)
Gets the ignore file that should be used for a collect directory or file.
source code
 
_getLinkDepth(item)
Gets the link depth that should be used for a collect directory.
source code
 
_getDereference(item)
Gets the dereference flag that should be used for a collect directory.
source code
 
_getRecursionLevel(item)
Gets the recursion level that should be used for a collect directory.
source code
 
_getDigestPath(config, absolutePath)
Gets the digest path associated with a collect directory or file.
source code
 
_getTarfilePath(config, absolutePath, archiveMode)
Gets the tarfile path (including correct extension) associated with a collect directory.
source code
 
_getExclusions(config, collectDir)
Gets exclusions (file and patterns) associated with a collect directory.
source code
Variables [hide private]
  logger = <logging.Logger object>
  __package__ = 'CedarBackup2.actions'
Function Details [hide private]

executeCollect(configPath, options, config)

source code 

Executes the collect backup action.

Parameters:
  • configPath (String representing a path on disk.) - Path to configuration file on disk.
  • options (Options object.) - Program command-line options.
  • config (Config object.) - Program configuration.
Raises:
  • ValueError - Under many generic error conditions
  • TarError - If there is a problem creating a tar file

Note: When the collect action is complete, we will write a collect indicator to the collect directory, so it's obvious that the collect action has completed. The stage process uses this indicator to decide whether a peer is ready to be staged.

_collectFile(config, absolutePath, tarfilePath, collectMode, archiveMode, resetDigest, digestPath)

source code 

Collects a configured collect file.

The indicated collect file is collected into the indicated tarfile. For files that are collected incrementally, we'll use the indicated digest path and pay attention to the reset digest flag (basically, the reset digest flag ignores any existing digest, but a new digest is always rewritten).

The caller must decide what the collect and archive modes are, since they can be on both the collect configuration and the collect file itself.

Parameters:
  • config - Config object.
  • absolutePath - Absolute path of file to collect.
  • tarfilePath - Path to tarfile that should be created.
  • collectMode - Collect mode to use.
  • archiveMode - Archive mode to use.
  • resetDigest - Reset digest flag.
  • digestPath - Path to digest file on disk, if needed.

_collectDirectory(config, absolutePath, collectMode, archiveMode, ignoreFile, linkDepth, dereference, resetDigest, excludePaths, excludePatterns, recursionLevel)

source code 

Collects a configured collect directory.

The indicated collect directory is collected into the indicated tarfile. For directories that are collected incrementally, we'll use the indicated digest path and pay attention to the reset digest flag (basically, the reset digest flag ignores any existing digest, but a new digest is always rewritten).

The caller must decide what the collect and archive modes are, since they can be on both the collect configuration and the collect directory itself.

Parameters:
  • config - Config object.
  • absolutePath - Absolute path of directory to collect.
  • collectMode - Collect mode to use.
  • archiveMode - Archive mode to use.
  • ignoreFile - Ignore file to use.
  • linkDepth - Link depth value to use.
  • dereference - Dereference flag to use.
  • resetDigest - Reset digest flag.
  • excludePaths - List of absolute paths to exclude.
  • excludePatterns - List of patterns to exclude.
  • recursionLevel - Recursion level (zero for no recursion)

_executeBackup(config, backupList, absolutePath, tarfilePath, collectMode, archiveMode, resetDigest, digestPath)

source code 

Execute the backup process for the indicated backup list.

This function exists mainly to consolidate functionality between the _collectFile and _collectDirectory functions. Those functions build the backup list; this function causes the backup to execute properly and also manages usage of the digest file on disk as explained in their comments.

For collect files, the digest file will always just contain the single file that is being backed up. This might little wasteful in terms of the number of files that we keep around, but it's consistent and easy to understand.

Parameters:
  • config - Config object.
  • backupList - List to execute backup for
  • absolutePath - Absolute path of directory or file to collect.
  • tarfilePath - Path to tarfile that should be created.
  • collectMode - Collect mode to use.
  • archiveMode - Archive mode to use.
  • resetDigest - Reset digest flag.
  • digestPath - Path to digest file on disk, if needed.

_loadDigest(digestPath)

source code 

Loads the indicated digest path from disk into a dictionary.

If we can't load the digest successfully (either because it doesn't exist or for some other reason), then an empty dictionary will be returned - but the condition will be logged.

Parameters:
  • digestPath - Path to the digest file on disk.
Returns:
Dictionary representing contents of digest path.

_writeDigest(config, digest, digestPath)

source code 

Writes the digest dictionary to the indicated digest path on disk.

If we can't write the digest successfully for any reason, we'll log the condition but won't throw an exception.

Parameters:
  • config - Config object.
  • digest - Digest dictionary to write to disk.
  • digestPath - Path to the digest file on disk.

_getCollectMode(config, item)

source code 

Gets the collect mode that should be used for a collect directory or file. If possible, use the one on the file or directory, otherwise take from collect section.

Parameters:
  • config - Config object.
  • item - CollectFile or CollectDir object
Returns:
Collect mode to use.

_getArchiveMode(config, item)

source code 

Gets the archive mode that should be used for a collect directory or file. If possible, use the one on the file or directory, otherwise take from collect section.

Parameters:
  • config - Config object.
  • item - CollectFile or CollectDir object
Returns:
Archive mode to use.

_getIgnoreFile(config, item)

source code 

Gets the ignore file that should be used for a collect directory or file. If possible, use the one on the file or directory, otherwise take from collect section.

Parameters:
  • config - Config object.
  • item - CollectFile or CollectDir object
Returns:
Ignore file to use.

_getLinkDepth(item)

source code 

Gets the link depth that should be used for a collect directory. If possible, use the one on the directory, otherwise set a value of 0 (zero).

Parameters:
  • item - CollectDir object
Returns:
Link depth to use.

_getDereference(item)

source code 

Gets the dereference flag that should be used for a collect directory. If possible, use the one on the directory, otherwise set a value of False.

Parameters:
  • item - CollectDir object
Returns:
Dereference flag to use.

_getRecursionLevel(item)

source code 

Gets the recursion level that should be used for a collect directory. If possible, use the one on the directory, otherwise set a value of 0 (zero).

Parameters:
  • item - CollectDir object
Returns:
Recursion level to use.

_getDigestPath(config, absolutePath)

source code 

Gets the digest path associated with a collect directory or file.

Parameters:
  • config - Config object.
  • absolutePath - Absolute path to generate digest for
Returns:
Absolute path to the digest associated with the collect directory or file.

_getTarfilePath(config, absolutePath, archiveMode)

source code 

Gets the tarfile path (including correct extension) associated with a collect directory.

Parameters:
  • config - Config object.
  • absolutePath - Absolute path to generate tarfile for
  • archiveMode - Archive mode to use for this tarfile.
Returns:
Absolute path to the tarfile associated with the collect directory.

_getExclusions(config, collectDir)

source code 

Gets exclusions (file and patterns) associated with a collect directory.

The returned files value is a list of absolute paths to be excluded from the backup for a given directory. It is derived from the collect configuration absolute exclude paths and the collect directory's absolute and relative exclude paths.

The returned patterns value is a list of patterns to be excluded from the backup for a given directory. It is derived from the list of patterns from the collect configuration and from the collect directory itself.

Parameters:
  • config - Config object.
  • collectDir - Collect directory object.
Returns:
Tuple (files, patterns) indicating what to exclude.