cliapp is a Python framework for Unix-like command line programs, which typically have the following characteristics:
Programs like the above are often used as filters in a pipeline. The scaffoling to set up a command line parser, open each input file, read each line of input, etc, is the same in each program. Only the logic of what to do with each line differs.
cliapp is not restricted to line-based filters, but is a more general framework. It provides ways for its users to override most behavior. For example:
Despite all the flexibility, writing simple line-based filters remains very straightforward. The point is to get the framework to do all the usual things, and avoid repeating code across users of the framework.
class ExampleApp(cliapp.Application):
def add_settings(self):
self.settings.string_list(['pattern', 'e'],
'search for regular expression PATTERN',
metavar='REGEXP')
# We override process_inputs to be able to do something after the last
# input line.
def process_inputs(self, args):
self.matches = 0
cliapp.Application.process_inputs(self, args)
self.output.write('There were %s matches.\\n' % self.matches)
def process_input_line(self, name, line):
for pattern in self.settings['pattern']:
if pattern in line:
self.output.write('%s:%s: %s' % (name, self.lineno, line))
self.matches += 1
logging.debug('Match: %s line %d' % (name, self.lineno))
if __name__ == '__main__':
ExampleApp().run()
Every application should be a class that subclasses cliapp.Application. The subclass should provide specific methods. Read the documentation for the cliapp.Application class to see all methods, but a rough summary is here:
This cascade of overrideable methods is started by the run method, which also sets up logging, loads configuration files, parses the command line, and handles reporting of exceptions. It can also run the rest of the code under the Python profiler, if the appropriate environment variable is set.
Logging support: by default, no log file is written, it must be requested explicitly by the user. The default log level is info.
Sometimes a command line tool needs to support subcommands. For example, version control tools often do this: git commit, git clone, etc. To do this with cliapp, you need to add methods with names like cmd_commit and cmd_clone:
class VersionControlTool(cliapp.Application):
def cmd_commit(self, args):
pass
def cmd_clone(self, args):
pass
If any such methods exist, cliapp automatically supports subcommands. The name of the method, without the cmd_ prefix, forms the name of the subcommand. Any underscores in the method name get converted to dashes in the command line. Case is preserved.
Subcommands may also be added using the add_subcommand method.
All options are global, not specific to the subcommand. All non-option arguments are passed to the method in its only argument.
Subcommands are implemented by the process_args method. If you override that method, you need to support subcommands yourself (perhaps by calling the cliapp implementation).
cliapp provides a way to fill in a manual page template, in troff format, with information about all options. This allows you to write the rest of the manual page without having to remember to update all options. This is a compromise between ease-of-development and manual page quality.
A high quality manual page probably needs to be written from scratch. For example, the description of each option in a manual page should usually be longer than what is suitable for --help output. However, it is tedious to write option descriptions many times.
To use this, use the --generate-manpage=TEMPLATE option, where TEMPLATE is the name of the template file. See example.1 in the cliapp source tree for an example.
If sys.argv[0] is foo, and the environment variable FOO_PROFILE is set, then the execution of the application (the run method) is profiled, using cProfile, and the profile written to the file named in the environment variable.
Base class for application specific exceptions.
Any exceptions that are subclasses of this one get printed as nice errors to the user. Any other exceptions cause a Python stack trace to be written to stderr.
Settings for a cliapp application.
You probably don’t need to create a settings object yourself, since cliapp.Application does it for you.
Settings are read from configuration files, and parsed from the command line. Every setting has a type, name, and help text, and may have a default value as well.
For example:
settings.boolean(['verbose', 'v'], 'show what is going on')
This would create a new setting, verbose, with a shorter alias v. On the command line, the options --verbose and -v would work equally well. There can be any number of aliases.
The help text is shown if the user uses --help or --generate-manpage. You can use the metavar keyword argument to set the name shown in the generated option lists; the default name is whatever optparse decides (i.e., name of option).
Use load_configs to read configuration files, and parse_args to parse command line arguments.
The current value of a setting can be accessed by indexing the settings class:
settings['verbose']
The list of configuration files for the appliation is stored in config_files. Add or remove from the list if you wish. The files need to exist: those that don’t are silently ignored.
Add a setting with a boolean value.
Build OptionParser for parsing command line.
Add a setting with a size in bytes.
The user can use suffixes for kilo/mega/giga/tera/kibi/mibi/gibi/tibi.
Add a setting which chooses from list of acceptable values.
An example would be an option to set debugging level to be one of a set of accepted names: debug, info, warning, etc.
The default value is the first possibility.
Add an integer setting.
Load all config files in self.config_files.
Silently ignore files that do not exist.
Parse the command line.
Return list of non-option arguments. args would usually be sys.argv[1:].
Raise exception if setting has not been set.
Option must have a value, and a default value is OK.
Add a setting with a string value.
Add a setting which have multiple string values.
An example would be an option that can be given multiple times on the command line, e.g., “–exclude=foo –exclude=bar”.
A framework for Unix-like command line programs.
The user should subclass this base class for each application. The subclass does not need code for the mundane, boilerplate parts that are the same in every utility, and can concentrate on the interesting part that is unique to it.
To start the application, call the run method.
The progname argument sets tne name of the program, which is used for various purposes, such as determining the name of the configuration file.
Similarly, version sets the version number of the program.
description and epilog are included in the output of --help. They are formatted to fit the screen. Unlike the default behavior of optparse, empty lines separate paragraphs.
Add application specific settings.
Add a subcommand.
Normally, subcommands are defined by add cmd_foo methods to the application class. However, sometimes it is more convenient to have them elsewhere (e.g., in plugins). This method allows doing that.
The callback function must accept a list of command line non-option arguments.
Log memory profiling information.
Get the memory profiling method from the dump-memory-profile setting, and log the results at DEBUG level. msg is a message the caller provides to identify at what point the profiling happens.
Open an input file for reading.
The default behaviour is to open a file named on the local filesystem. A subclass might override this behavior for URLs, for example.
The optional mode argument speficies the mode in which the file gets opened. It should allow reading. Some files should perhaps be opened in binary mode (‘rb’) instead of the default text mode.
Parse the command line.
Return list of non-option arguments.
Process command line non-option arguments.
The default is to call process_inputs with the argument list, or to invoke the requested subcommand, if subcommands have been defined.
Process a particular input file.
The stdin argument is meant for unit test only.
Process one line of the input file.
Applications that are line-oriented can redefine only this method in a subclass, and should not need to care about the other methods.
Process all arguments as input filenames.
The default implementation calls process_input for each input filename. If no filenames were given, then process_input is called with - as the argument name. This implements the usual Unix command line practice of reading from stdin if no inputs are named.
The attributes fileno, global_lineno, and lineno are set, and count files and lines. The global line number is the line number as if all input files were one.
Run the application.
Run external command.
Return the standard output of the command.
Raise cliapp.AppException if external command returns non-zero exit code. *args and **kwargs are passed onto subprocess.Popen.
Run external command.
Return the exit code, and contents of standard output and error of the command.
Set up logging.