Reference¶
utah¶
utah
utah.cleanup¶
Generic functionality to execute callbacks on exit.
- class utah.cleanup._Cleanup[source]¶
Cleanup allocated resources on exit.
Warning
This is private class not expected to be instanciated please use utah.cleanup.cleanup singleton object to call any of the methods documented below.
- add_command(cmd)[source]¶
Register a command to be run on cleanup.
Parameters: cmd (iterable) – A command as would be passed subprocess.Popen.
- add_function(timeout, function, *args, **kw)[source]¶
Register a function to be run on cleanup.
The function will be run through utah.timeout.timeout().
Parameters: - timeout (int) – Timeout in seconds for the function to return a result.
- function (callable) – A callable that will do some cleanup.
- args (dict) – Positional arguments to the function.
- kw – Keyword arguments to the function.
- add_path(path)[source]¶
Register a path to be cleaned later.
- The way to clean different paths is as follows:
- Links will be unlinked.
- Files will have permissions changed to 664 and unlinked.
- Directories will recursively have permissions changed to 775 (except for links contained within) and then be recursively removed.
Parameters: path (str) – A link, file, or directory to be cleaned later.
- utah.cleanup.cleanup = <utah.cleanup._Cleanup object at 0x7fdc2ca3a610>¶
Singleton object used to cleanup everything
Provide stringification for commands used by timeout and retry.
- utah.commandstr.commandstr(command, *args, **kw)[source]¶
Provide a user-oriented command display.
i.e. commandstr(command, arg1, arg2, kw1=1, kw2=2) should return ‘command(arg1, arg2, kw1=1, kw2=2)’
Provide config variables via utah.config.
When run directly as part of the package build, output machine-independent defaults as json for writing a config file.
Provide exceptions for UTAH.
- exception utah.exceptions.UTAHException(*args, **kw)[source]¶
Provide a foundation class for UTAH exceptions.
Support retry and external arguments, but default to False.
Provide unix group checking functionality.
- utah.group.check_user_group(group='utah')[source]¶
Return whether the user is a member of the given group.
- utah.group.print_group_error_message(script)[source]¶
Print error message to stderr to be used by scripts.
All commands use bsdtar and accept a logmethod to use for logging.
- class utah.iso.ISO(arch=None, image=None, installtype=None, logger=None, series=None)[source]¶
Provide a simplified method of interfacing with images.
- dldisplay(blocks, size, total)[source]¶
Log download information (i.e., as a urllib callback).
Parameters: - blocks (int) – Number of blocks downloaded
- size (int) – Size of blocks downloaded
- total – Total size of download
- downloadiso(arch=None, installtype=None, series=None)[source]¶
Download an ISO given series, type, and arch.
Parameters: - arch (str) – Architecture of image to download
- installtype (str) – Install type of image to download
- series (str) – Series codename of image to download
Returns: Local path of downloaded image
Return type: str
- dump(filename, **kw)[source]¶
Extract file contents from an ISO.
Parameters: filename (str) – Name of the file to be extracted Returns: Contents of the file Return type: str See also
- extract(filename, outdir='', outfile=None, **kw)[source]¶
Extract file from an ISO.
Parameters: - filename (str) – Path to the file in the ISO to be extracted
- outdir (str) – Destination directory
- outfile (str) – Destination filename
Returns: Path to the extracted file
Return type: str
See also
- getarch()[source]¶
Unpack the image’s info file to get the arch.
Returns: Image architecture Return type: str
- getbuildnumber()[source]¶
Unpack the image’s info file to get the build number.
Returns: Build number of the image. Return type: str
- getinstalltype()[source]¶
Inspect the image’s files to get the image type.
If .disk/mini-info exists, it’s mini. If the casper directory exists, it’s desktop. If ubuntu-server.seed exists in the preseeds directory, it’s server. :returns: Image type :rtype: str
- getmd5(path=None)[source]¶
Return the MD5 checksum of a file.
Default file is this image. :param path: Path of file to checksum :type path: str :returns: MD5 checksum of file :rtype: str
- getrealfile(filename)[source]¶
Return a command to safely extract a file from an ISO.
Based on unbsdtar-safelink from ubuntu ISO testing.
Params filename: Path to the file in the ISO to be extracted Returns: Command that can be passed to a subprocess method Return type: list
- getseries()[source]¶
Unpack the image’s info file to get the series.
Returns: Image series Return type: str
utah.parser¶
Command line parser for the utah server script.
- utah.parser.directory_argument(directory)[source]¶
Argument passed through the command line which is a valid directory.
Parameters: directory (string) – A path to a directory passed through the command line. Returns: The same directory passed if it’s valid. Return type: string Raises argparse.ArgumentTypeError: If path doesn’t point to a directory.
utah.preseed¶
- This module provides all the classes needed to:
- Parse a preseed file
- Update some values
- Add new sections
- Write the changes back to a file
The expected way to use it is by passing a file-like object or an iterable that yields one line of the preseed at a time:
>>> from utah.preseed import Preseed
>>> from StringIO import StringIO
>>> preseed_text = StringIO(
... '# Comment\n'
... '\n'
... 'd-i passwd/username string utah\n')
>>> preseed = Preseed(preseed_text)
After that, any of the configuration sections can be accessed by the question name:
>>> section = preseed['passwd/username']
>>> section
<ConfigurationSection: 'd-i passwd/username string utah\n'>
and values can be updated by setting them directly in the section objects:
>>> section.value = 'ubuntu'
>>> section
<ConfigurationSection: 'd-i passwd/username string ubuntu\n'>
In addition to this, if a new section is needed, it can be appended/prepended to the preseed by calling directly the Preseed methods or the ConfigurationSection methods to use the section as a reference, that is, append/prepend after/before the given section.
>>> section.append('d-i passwd/user-password password\n')
>>> section.append('d-i passwd/user-password-again password\n')
Once the desired changes have been applied, the Preseed.dump() method can be used to write the output to a new file:
>>> print preseed.dump()
# Comment
d-i passwd/username string ubuntu
d-i passwd/user-password-again password
d-i passwd/user-password password
- class utah.preseed.Preseed(lines=None)[source]¶
Read/Write preseed files easily.
Parameters: lines (iterable) – File-like object or iterable that yields one line from the preseed at a time. - load(lines)[source]¶
Parse preseed configuration lines.
Parameters: lines (iterable) – Any iterable that yields preseed file configuration lines Returns: Preseed file object with information parsed Return type: Preseed Raises ParsingError: If there’s a problem parsing some configuration lines, this exception will be raised with the line number where the problem was detected in the message. Example: >>> from StringIO import StringIO >>> preseed_text = StringIO( ... '# Comment\n' ... '\n' ... 'd-i passwd/username string utah\n') >>> preseed = Preseed() >>> preseed.load(preseed_text) >>> print preseed.dump() # Comment d-i passwd/username string utah
Note
This method is automatically called at initialization time if the lines parameter is passed to the constructor, so it’s not really expected to be used directly.
See also
- dump()[source]¶
Dump preseed configuration statements.
This method returns the contents of the preseed after the changes applied. The string returned is normally used to write the changes back to a file that can be used as the new preseed to provision a system.
Returns: Formatted preseed configuration lines Return type: string Example: >>> preseed = Preseed('# Comment\n'.splitlines()) >>> preseed.dump() '# Comment\n'
See also
- prepend(new_section, ref_section=None)[source]¶
Prepend a new section to the preseed.
Parameters: - new_section (Section | basestring) –
The new section to be prepended.
Note
If a string is passed instead, a new section will be created from the string with Section.new().
- ref_section (Section) –
A section to be used as a reference, meaning that the new section will be prepended after the reference section.
Note
If no reference section is passed, then the new section will be prepended just to the beginning of the preseed.
Example: >>> preseed = Preseed('d-i passwd/username string utah\n'.splitlines()) >>> preseed.prepend('# Comment') >>> print preseed.dump() # Comment d-i passwd/username string utah
See also
- new_section (Section | basestring) –
- append(new_section, ref_section=None)[source]¶
Append a new section to the preseed.
Parameters: - new_section (Section | basestring) –
The new section to be appended.
Note
If a string is passed instead, a new section will be created from the string with Section.new().
- ref_section (Section) –
A section to be used as a reference, meaning that the new section will be appended after the reference section.
Note
If no reference section is passed, then the new section will be appended just to the end of the preseed.
Example: >>> preseed = Preseed('# Comment\n'.splitlines()) >>> preseed.append('d-i passwd/username string utah\n') >>> print preseed.dump() # Comment d-i passwd/username string utah
See also
- new_section (Section | basestring) –
- section_updated(section, property_name, old_text, new_text)[source]¶
Update question names index.
This is a callback called every time a section property is updated and used to maintain question names index integrity
Parameters: - section (Section) – Section object calling the callback
- property_name (string) – Name of the updated property
- old_text (string | None) – Old property text
- new_text (string) – New property text
Raises DuplicatedQuestionName: If the updated property is qname and the new text is already taken by other section which would break the access to a section by the question name.
- class utah.preseed.Section[source]¶
Any kind of preseed section (blank, comment or configuration).
- classmethod new(lines)[source]¶
Create new section subclass based on the lines in the preseed.
This method is used by the Preseed.load() method to create new sections while parsing a preseed file.
Parameters: lines (list) – Lines to be parsed for this particular section Returns: Section object the properly represents the lines passed Return type: subclass of Section Example: >>> from utah.preseed import Section >>> Section.new('\n'.splitlines()) <BlankSection: '\n'> >>> Section.new('# Comment\n'.splitlines()) <CommentSection: '# Comment\n'> >>> Section.new('d-i passwd/username string utah\n'.splitlines()) <ConfigurationSection: 'd-i passwd/username string utah\n'>
See also
- class utah.preseed.BlankSection(lines_count)[source]¶
A pressed section that represents a group of consecutive blank lines.
Parameters: lines_count (int) – Number of blank lines represented by this section Variables: lines_count – The number of lines as passed to the constructor Example: >>> from utah.preseed import BlankSection >>> section = BlankSection(3) >>> section.lines_count 3
- __add__(other)[source]¶
Add two blank sections.
Parameters: other (BlankSection) – The section to add Returns: A new blank section that contains the amount of lines lines from the two sections being added. Return type: BlankSection Example: >>> blank1 = BlankSection(1) >>> blank2 = BlankSection(2) >>> blank3 = blank1 + blank2 >>> blank3.lines_count 3
See also
- __iadd__(other)[source]¶
Add two blank sections in place.
Parameters: other (BlankSection) – The section to add Returns: The section on the left updated to contain the amount of lines lines from the two sections being added. Return type: BlankSection Example: >>> blank1 = BlankSection(1) >>> blank2 = BlankSection(2) >>> blank3 = blank1 + blank2 >>> blank3.lines_count 3
See also
- class utah.preseed.CommentSection(lines)[source]¶
A preseed section that represents a group consecutive comment lines.
Parameters: lines (iterable) – An iterable that yields one line at a time Variables: lines – The comment lines that were passed to the constructor Example: >>> from utah.preseed import CommentSection >>> comment_str = '# Comment\n' >>> section = CommentSection(comment_str.splitlines()) >>> section.lines ['# Comment']
- __add__(other)[source]¶
Add two comment sections.
Parameters: other (CommentSection) – The section to add Returns: A new comment section that contains the comment lines from the two sections being added. Return type: CommentSection Example: >>> comment1 = CommentSection('# Comment 1\n'.splitlines()) >>> comment2 = CommentSection('# Comment 2\n'.splitlines()) >>> comment3 = comment1 + comment2 >>> comment3.lines ['# Comment 1', '# Comment 2']
See also
- __iadd__(other)[source]¶
Add two comment sections in place.
Parameters: other (CommentSection) – The section to add Returns: The section on the left updated to contain the comment lines from the two sections being added. Return type: CommentSection Example: >>> comment1 = CommentSection('# Comment 1\n'.splitlines()) >>> comment2 = CommentSection('# Comment 2\n'.splitlines()) >>> comment1 += comment2 >>> comment1.lines ['# Comment 1', '# Comment 2']
See also
- class utah.preseed.TextProperty(name)[source]¶
A text property used in ConfigurationSection objects.
See also
- class utah.preseed.TextPropertyValue(parent, obj, text='')[source]¶
A text value used in TextProperty objects.
The value being stored is just a text string, so there’s currently no type even if the configuration sections in a preseed use types for values.
- prepend(other_text)[source]¶
Prepend a string to the stored value.
Parameters: other_text (basestring) – The text to be prepended Returns: The updated value Return type: TextPropertyValue Example: >>> late_command_str = 'd-i preseed/late_command string some_command\n' >>> section = Section.new(late_command_str.splitlines()) >>> section.value <TextPropertyValue: 'some_command'> >>> section.value.prepend('another_command; ') <TextPropertyValue: 'another_command; some_command'>
Note
The change happens in place, so there’s no need to assign any result back to the TextProperty object:
- append(other_text)[source]¶
Append a string to the stored value.
Parameters: other_text (basestring) – The text to be appended Returns: The updated value Return type: TextPropertyValue Example: >>> late_command_str = 'd-i preseed/late_command string some_command\n' >>> section = Section.new(late_command_str.splitlines()) >>> section.value <TextPropertyValue: 'some_command'> >>> section.value.append('; another_command') <TextPropertyValue: 'some_command; another_command'>
Note
The change happens in place, so there’s no need to assign any result back to the TextProperty object:
- class utah.preseed.ConfigurationSection(raw_lines)[source]¶
A preseed configuration statement made of one or multiple lines.
The expected format of a configuration section is as follows:
<owner> <qname> <qtype> <value>
where the whole section might be made of multiple lines. A line is considered not to finish the statement if there’s a backslash character just before the newline character.
If the parsing succeeds, every field is accessible using the same name as above.
Parameters: raw_lines (iterable) – An iterable that yields one line at a time Raises ParsingError: If the configuration lines don’t follow the expected format above, this exception will be raised. Example: >>> from utah.preseed import ConfigurationSection >>> configuration_str = 'd-i passwd/username string utah\n' >>> section = ConfigurationSection(configuration_str.splitlines()) >>> section.owner <TextPropertyValue: 'd-i'> >>> section.qname <TextPropertyValue: 'passwd/username'> >>> section.qtype <TextPropertyValue: 'string'> >>> section.value <TextPropertyValue: 'utah'>
- prepend(new_section)[source]¶
Prepend a new section to this one.
This is a wrapper method that actually calls the Preseed.prepend() method in the preseed using this section as a reference section to set the insertion position.
Parameters: new_section (Section | basestring) – The new section to be prepended.
Note
If a string is passed instead, a new section will be created from the string with Section.new().
Returns: None Return type: None Example: >>> preseed = Preseed() >>> section = Section.new('d-i passwd/username string utah\n' ... .splitlines()) >>> preseed.append(section) >>> section.prepend('d-i passwd/user-password password\n') >>> print preseed.dump() d-i passwd/user-password password d-i passwd/username string utah
See also
- append(new_section)[source]¶
Append a new section to this one.
This is a wrapper method that actually calls the Preseed.append() method in the preseed using this section as a reference section to set the insertion position.
Parameters: new_section (Section | basestring) – The new section to be appended.
Note
If a string is passed instead, a new section will be created from the string with Section.new().
Returns: None Return type: None Example: >>> preseed = Preseed() >>> section = Section.new('d-i passwd/username string utah\n' ... .splitlines()) >>> preseed.append(section) >>> section.append('d-i passwd/user-password password\n') >>> print preseed.dump() d-i passwd/username string utah d-i passwd/user-password password
See also
- property_updated(property_name, old_value, new_value)[source]¶
Propagate property updates to preseed parent.
If a parent preseed is set, for every updated received from a property value, the same update is propagated to the parent preseed object.
Parameters: - property_name (string) – Name of the updated property
- old_value (string | None) – Old property value
- new_value (string) – New property value
- exception utah.preseed.ParsingError[source]¶
An error happened when parsing a preseed section.
This exception is raised when a preseed is being parsed and a new section object is being created to store some contents.
Example: >>> preseed_str = ('not valid\n') >>> preseed = Preseed(preseed_str.splitlines()) Traceback (most recent call last): ... ParsingError: Line 1: Unable to parse configuration lines: not valid Traceback (most recent call last): ... ParsingError: Line 1: Unable to parse configuration lines: not valid
See also
- exception utah.preseed.DuplicatedQuestionName[source]¶
Duplicated question name found in preseed.
This exception is raised when a question name is found more than once in a preseed. This is part of the process used in the Preseed class to guarantee that questions can be accessed based on their name.
Example: >>> preseed_str = ('d-i passwd/username string utah\n' ... 'd-i passwd/username string ubuntu\n') >>> preseed = Preseed(preseed_str.splitlines()) Traceback (most recent call last): ... DuplicatedQuestionName: passwd/username Traceback (most recent call last): ... DuplicatedQuestionName: passwd/username
utah.process¶
Process related utilities.
- class utah.process.ProcessChecker[source]¶
Check all running process looking for a given pattern.
- class utah.process.ProcessRunner(arglist)[source]¶
Run a command in a subprocess and log output properly.
Parameters: arglist (list) – Argument list as would be passsed to subprocess.Popen Attribute output: Command output (both stdout and stderr) Attribute returncode: Command return code Note
Process is launched when object is instantiated, there isn’t any separate run method.
- utah.process.pid_in_use(pid, arg_patterns=None)[source]¶
Test if the given pid is running.
If arg_patterns is provided then the pid’s command line will be compared to make sure it matches an expected value.
Returns: proc object if the pid is active and optionally matches one of the arg_patterns, otherwise None Return type: psutil.Process
- utah.process.run(cmd, cwd, timeout, stream_syslog)[source]¶
Run a command using fork/exec.
subprocess.* methods, don’t provide a good way to do the UNIX classic fork/exec/select to get stdout/stderr while a process is executing. This provides a version for UTAH.
Returns: rc, timed_out, stdout, stderr Return type: tuple(int, bool, string, string)
Provide a retry loop that exits on success or an unretryable exception.
- utah.retry.retry(command, *args, **kw)[source]¶
Retry a command as long as a retriable exception is captured.
A retriable exception is considered to be a UTAHException that has the attribute retry set to True.
Parameters: - command (callable) – Command to be executed.
- args (tuple) – Positional arguments to be passed to the callable.
- kwargs (dict) –
Keyword arguments to be passed to the callable.
Note
There are a few keywords that are consumed by retry and not passed to the callable:
- logmethod: Preferred log method for every retry attempt (sys.stderr by default)
- retry_timeout: Timeout in seconds between each retry attempt
Returns: The value returned by the callable.
See also
Provide routines used to process command line arguments and run tests.
- class utah.run.ReturnCodes[source]¶
Provide standard return codes for run_ scripts.
- static client_error(returncode)[source]¶
Add offset to client error to avoid overlapping.
This is useful to be able to know when a failure happened in the server or in the client at the shell level.
Parameters: returncode (int) – The code returned by the client Returns: The code to be returned by the server Return type: int
- utah.run.configure_logging(debug=False)[source]¶
Configure logging.
Configure root logger using three different handlers with a different log level: - Console: Set according to consoleloglevel from configuration - Log file: Set according to fileloglevel from configuration - Debug log file: Always set to debug
Other loggers are expected to propagate their messages up to the logging hierarchy.
- utah.run.getfiles(args, machine)[source]¶
Download files from machine.
Returns: list of download files Return type: list(str)
- utah.run.install_sigterm_handler()[source]¶
Capture SIGTERM signal to avoid abrupt process termination.
This function registers a handler that raises an exception when SIGTERM is received to ensure that machine object cleanup methods are called. Otherwise, cleanup methods aren’t called when the process is abruptly terminated.
- utah.run.is_utah_done(machine, checktimeout)[source]¶
Use utah-done.py to check if the run is finished.
Parameters: - machine (Machine) – Machine object to check
- checktimeout (int) – Time to wait before raising exception if check fails
Returns: The exit status of utah-done.py if it’s not UNKNOWN.
Raises UTAHException: When the utah client process isn’t finished yet.
- utah.run.master_runlist_argument(url)[source]¶
Get master runlist and validate it against its schema.
This function calls utah.url.url_argument() to download the runlist file if needed and then uses the schema defined in utah.client.runner.Runner to validate it.
This functions is expected to be passed as the type argument of an argparse.ArgumentParser object to make sure the master runlist is reachable and valid before provisioning the system under test.
Parameters: url (basestring) – URL as passed to parser object. Returns: URL or path to the local file Return type: basestring See also
Example: >>> from utah.run import master_runlist_argument >>> import argparse >>> parser = argparse.ArgumentParser() >>> parser.add_argument('url', ... type=master_runlist_argument) _StoreAction(... dest='url', ...) >>> runlist = 'lp:utah/utah/client/examples/pass.run' >>> parser.parse_args([runlist]) Namespace(url='/tmp/utah_...')
- utah.run.run_tests(args, machine)[source]¶
Run a runlist and retrieve results.
Returns: exitstatus and list of logs returned Return type: tuple(int, list(str))
Provide functionality to execute command with timeouts.
- exception utah.timeout.UTAHTimeout[source]¶
Provide a special exception to indicate an operation timed out.
- utah.timeout.get_process_children(pid, logmethod=<built-in method write of file object at 0x7fdc2fca41e0>)[source]¶
Find process children so they can be killed when the timeout expires.
Parameters: pid (int) – Process ID for the parent process Returns: The pid for each children process Return type: list(int)
- utah.timeout.subprocesstimeout(timeout, *args, **kw)[source]¶
Run command through subprocess.Popen for up to timeout seconds.
Command termination is checked using subprocess.Popen.poll.
Parameters: - timeout (int) – Maximum amount of time to wait for the command to complete in seconds.
- args – Positional arguments to be passed to subprocess.Popen.
- kwargs – Keyword arguments to be passed to the subprocess.Popen.
Returns: The subprocess object
Return type: subprocess.Popen
Raises UTAHTimeout: If command execution hasn’t finished before timeout seconds.
- utah.timeout.timeout(timeout, command, *args, **kw)[source]¶
Run a command for up to timeout seconds.
Parameters: - timeout (int) – Maximum amount of time to wait for the command to complete in seconds.
- command (callable) – Command whose execution should complete before timeout expires.
- args – Positional arguments to be passed to the callable.
- kwargs – Keyword arguments to be passed to the callable.
Returns: The value returned by the callable.
Raises UTAHTimeout: If command execution hasn’t finished before timeout seconds.
See also
utah.template¶
Provide easy accessors to the Jinja2 template library
utah.url¶
This module provides the classes/functions needed to:
- Check that a URL is valid and readable
- Use a url type in an argparse.ArgumentParser object
- class utah.url.HeadRequest(url, data=None, headers={}, origin_req_host=None, unverifiable=False)[source]¶
A request that sends HEAD method instead of GET.
See also
- class utah.url.URLChecker(proxies=None, **x509)[source]¶
An opener to checks a URL is valid and readable.
To use it, create an object instance and call the open method passing the url to be checked as argument.
- open_http(url)[source]¶
Check if http URL exists and is readable.
The check is performed by sending an HTTP HEAD request, waiting for the response and checking that the code is 200 OK.
If a redirect response is received, the URL will still be reported as working fine, but the underlying implementation will use an HTTP GET method instead, so it won’t be as efficient as in the standard case.
Parameters: url (basestring) – The HTTP URL to be checked Returns: The url passed as argument when it’s valid and readable. Return type: basestring Raises URLNotFound: When there’s a problem opening the URL or isn’t found. Example: >>> from utah.url import URLChecker >>> opener = URLChecker() >>> opener.open('http://www.ubuntu.com') 'http://www.ubuntu.com'
Note
This method is called by the open method when the URL protocol is http, so it’s not expected to be called directly.
See also
- open_local_file(url)[source]¶
Check if local file exists.
Parameters: url (basestring) – The file URL to be checked
Returns: The path to the file if it was found and readable.
Note
The returned value is a path, not a URL, so it can be used to open the file the same way as any other files.
Return type: basestring
Raises: - URLNotFound – when the path to the file doesn’t exist.
- URLNotReadable – when the user doesn’t have read permissions to open the file.
Example: >>> import tempfile >>> with tempfile.NamedTemporaryFile() as f: ... opener = URLChecker() ... opener.open(f.name) '/tmp/tmp...'
Note
This method is called by the open method when the URL protocol is file, so it’s not expected to be called directly.
See also
- utah.url.url_argument(url)[source]¶
URL argument to be used in an argparse.ArgumentParser object.
Parameters: url (basestring) – URL as passed to the parser object.
Note
The URL passed as argument can be a launchpad URL. In that case, the file pointed by the URL will be downloaded as when using bzr export and the returned value is the path to the downloaded file.
Returns: URL or path to local file Return type: basestring Raises argparse.ArgumentTypeError: when the URL is invalid or unreadable. In any case, the error message will provide information to be displayed by the argparse.ArgumentParser object in the command line. Example: >>> from utah.url import url_argument >>> import argparse >>> parser = argparse.ArgumentParser() >>> parser.add_argument('url', type=url_argument) _StoreAction(... dest='url', ...) >>> parser.parse_args(['http://www.ubuntu.com']) Namespace(url='http://www.ubuntu.com') >>> parser.parse_args(['lp:utah/setup.py']) Namespace(url='/tmp/utah_...')
See also
- exception utah.url.URLNotFound(*args, **kw)[source]¶
Exception raised when a URL isn’t found.
Example: >>> opener = URLChecker() >>> opener.open('http://localhost/invalid_url') Traceback (most recent call last): ... URLNotFound: http://localhost/invalid_url >>> opener.open('file:///tmp/invalid_url') Traceback (most recent call last): ... URLNotFound: /tmp/invalid_url Traceback (most recent call last): ... URLNotFound: /tmp/invalid_url
See also
- exception utah.url.URLNotReadable(*args, **kw)[source]¶
Exception raised when a URL isn’t readable.
Example: >>> import os >>> with tempfile.NamedTemporaryFile() as f: ... os.chmod(f.name, 0000) ... opener = URLChecker() ... opener.open(f.name) Traceback (most recent call last): ... URLNotReadable: /tmp/tmp... Traceback (most recent call last): ... URLNotReadable: /tmp/tmp...
See also
utah.client¶
utah.client
utah.client.common¶
UTAH client common classes and functions.
- class utah.client.common.ReturnCodes[source]¶
Provide consistent return codes for UTAH client.
PASS: All test cases were executed and passed FAIL: All test cases were executed, but at least one of them failed ERROR: At least one error was detected that prevented a test case from being executed. Examples of situations that are considered an error are: - Fetch command failure - Setup command failure REBOOT: The system under test rebooted as required by a test case. To get final result the state file has to be checked. UNKNOWN: Unable to retrieve state file to check if client finished properly. INVALID_USER: The client was launched with a user other than root. EXCEPTION_ERROR: An exception error was encountered. CMD_PARSING_ERROR: Command line arguments parsing error.
- utah.client.common.debug_print(data, force=False)[source]¶
Print debugging information according to CONFIG['DEBUG'] value.
Data will be printed with the DEBUG: string prepended to it.
Parameters: - data (object) – Data to be printed
- force (bool) – Print data regardless of the configuration
Returns: Whether something was printed or not
Return type: bool
- utah.client.common.get_arch()[source]¶
Get the host’s architecture.
Returns: The human readable architecture or 'unknown' Return type: str
- utah.client.common.get_build_number()[source]¶
Get build number.
Returns: Build number as according to media-info or '?' if not found Return type: str See also
- utah.client.common.get_host_info()[source]¶
Get host info, useful for debugging.
Returns: Host uname, media-info and product_uuid together Return type: dict See also
- utah.client.common.get_install_type()[source]¶
Get the contents of the install-type file if available.
Returns: The contents of the install-type file or 'unknown' if not available. Return type: str Note
This is only a best-effort approach.
See also
- utah.client.common.get_media_info()[source]¶
Get the contents of the media-info file if available.
Returns: The contents of the media-info file or 'unknown' if not available. Return type: str Note
This is only a best-effort approach.
- utah.client.common.get_product_uuid()[source]¶
Get the product_uuid of the machine under test.
Returns: The contents of the product_uuid file or None if not available. Note
This is only a best-effort approach.
- utah.client.common.get_release()[source]¶
Get the host’s release name.
Returns: Release name (i.e. quantal, raring, etc.) Return type: str
- utah.client.common.make_result(command, retcode, stdout='', stderr='', start_time='', time_delta='', cmd_type='testcase_test', user='unknown', probes=None)[source]¶
Make a result data structure.
Note
Battery information will only be included if some value is passed as argument.
Parameters: - command (str) – Command that was executed
- retcode (int) – Return code
- stdout (str) – Standard output
- stderr (str) – Standard error
- start_time (str) – Time in which command execution started
- time_delta (str) – Amount of time that the command took to complete
- cmd_type (str) – Command type to be displayed in the report
- user (str) – User name the executed the command
- probes (dict) – probe data obtained while running the command
Returns: Result data
Return type: dict
- utah.client.common.mkdir(path)[source]¶
Create a directory only if needed with proper error handling.
- utah.client.common.parse_control_file(filename, schema)[source]¶
Parse a control file and check against a jsonschema.
Parameters: - filename (str) – Path to the yaml file to be parsed
- schema (dict) – jsonschema to validate data against
Returns: Parsed data
Return type: object
Raises jsonschema.ValidationError: If file contents doesn’t follow the schema definitions
See also
parse_yaml_file(), DefaultValidator
- utah.client.common.parse_yaml_file(filename)[source]¶
Parse yaml file.
Parameters: filename (str) – Path to the file that should be read Returns: Parsed data Return type: object Raises YAMLParsingError: If there’s a problem while parsing the file with the information about where in the file the problem was detected. See also
- utah.client.common.run_cmd(command, cwd=None, timeout=0, cmd_type='testcase_test', run_as=None)[source]¶
Run command and return result using the client’s format.
Parameters: - command (str) – Command as it would be written in a console
- cwd (str) – Current working directory path
- timeout (int) – Maximum amount of time in seconds to wait for the command to complete
- cmd_type (str) – Command type as displayed in the result object
- run_as (int) – Username to use to run the command
Returns: Command execution result
Return type: dict
See also
Provide UTAH client exceptions.
- exception utah.client.exceptions.BadDir[source]¶
Directory error.
Raised when some test directory isn’t found or an error happens when trying to change to it
- exception utah.client.exceptions.BadMasterRunlist[source]¶
Raised when master runlist isn’t in the expected format.
- exception utah.client.exceptions.MissingFile[source]¶
Raised when yaml file with metadata about tests cannot be found.
- exception utah.client.exceptions.UTAHClientError[source]¶
Base class of all exceptions in the client.
- exception utah.client.exceptions.ValidationError[source]¶
Used to provide additional information when schema validation fails.
- exception utah.client.exceptions.YAMLEmptyFile[source]¶
Raised when a file that was supposed to contain yaml data is empty.
- exception utah.client.exceptions.YAMLParsingError[source]¶
Provide more detailed yaml.load exception handling.
Used to provide the filename and the location in which the parsing error happened when calling yaml.load
Get package information in the system under test.
- class utah.client.package_info.DeltaPackages(start, end)[source]¶
Calculate delta between to set of installed packages.
Parameters: - start (InstalledPackages) – Packages installed when test run started
- end (InstalledPackages) – Packages installed when test run ended
Variables: - installed – Packages that were installed in between
- uninstalled – Packages that were uninstalled in between
- updated – Packages that were uninstalled in between
- class utah.client.package_info.InstalledPackages[source]¶
Query apt cache for installed packages and their version.
Provide functionality for test result handling.
- class utah.client.result.Result(filename, append_to_file, name=None, testsuite=None, testcase=None, runlist=None, install_type=None)[source]¶
Result collection class.
Params filename: Filename where the report should be written to or None to print it to sys.stdout Parameters: append_to_file (bool) – Whether results should be appended to the results from a previous execution or not. If that’s the case, then filename is read to get the old results. - add_result(result, extra_info=None)[source]¶
Add a result to the object.
Note: ‘result’ is expected to be a dictionary like this:
{ 'command': '', 'returncode': 0, 'stdout': '', 'stderr': '', 'start_time': '', 'time_delta': '', }
- class utah.client.result.ResultJSON(filename, append_to_file, name=None, testsuite=None, testcase=None, runlist=None, install_type=None)[source]¶
Return results in a JSON format.
- class utah.client.result.ResultYAML(filename, append_to_file, name=None, testsuite=None, testcase=None, runlist=None, install_type=None)[source]¶
Return results in a YAML format.
Provide code to actually run the tests.
- class utah.client.runner.Runner(install_type, runlist=None, result=None, testdir='/var/lib/utah', state_agent=None, resume=False, old_results=None, output=None, repeat_count=0)[source]¶
Provide The main runner class.
Parses a master runlist, builds a list of TestSuites, and runs them.
- process_master_runlist(runlist=None, resume=False)[source]¶
Parse a master runlist and build a list of suites from the data.
Parameters: - runlist – URL pointing to a runlist
- resume (boolean) – Continue previous execution
- process_results()[source]¶
Add stats to results and process them.
Returns: A return code based on the test status. Return type: int
- run(iteration=0)[source]¶
Run the test suites we’ve parsed.
Returns: The result of process_results, which is a return code. Return type: int See also
Provide functionality for saving and restoring run state.
- class utah.client.state_agent.StateAgent(state_file=None)[source]¶
State saving base class.
Accepts a dictionary of state info and prints it to the state file.
- class utah.client.state_agent.StateAgentYAML(state_file=None)[source]¶
YAML based state saver.
Testcase specific code.
- class utah.client.testcase.TestCase(name, suite, command=None)[source]¶
Base class describing a test case.
status is one of ‘NOTRUN’, ‘BUILD’, ‘SETUP’, ‘RUN’, ‘CLEANUP’, or ‘DONE’
- is_done()[source]¶
Determine if the case is done.
This might mean that something has failed. Used by suite to determine if the suite needs to be re-run on resume.
Returns: Whether the test case is finished (done or cleaned up) Return type: bool
- load_state(state)[source]¶
Restore state from the supplied dictionary.
Requires that ‘state’ has the same fieldnames as the TestCase class.
- process_overrides(overrides)[source]¶
Set override values from a TestSuite runlist for this test case.
- run(rerun=False)[source]¶
Run the complete test case.
This includes any build, setup, and cleanup commands.
Returns: Whether to keep running tests (True) or reboot (False) Return type: bool
- save_state_callback(_obj=None)¶
Placeholder for save_state_callbacks.
Testsuite specific code.
- class utah.client.testsuite.DynamicTestSuite(name, runner, autolist)[source]¶
A dynamic test suite.
This class contructs itself based on a tslist.auto file that builds the list of test cases it should execute at runtime.
- class utah.client.testsuite.TestSuite(name, runner, runlist_file='tslist.run', includes=None, excludes=None, skip_runlist=False)[source]¶
Base class describing a test suite.
- is_done()[source]¶
Determine if the suite is done.
This might mean that something has failed. Used by Runner to determine if the suite needs to be re-run on resume.
Returns: Whether the test suite is finished (done or cleaned up) Return type: bool
- load_state(state)[source]¶
Restore our state from the supplied dictionary.
Requires that the fieldnames in the dictionary match the class properties.
- run(rerun=False)[source]¶
Run the complete test suite.
This includes any build, setup, and cleanup commands, as well as all test cases (including build, setup, and cleanup.)
Returns: Whether to keep running tests (True) or reboot (False) Return type: bool
- save_state_callback(_obj=None)¶
Placeholder for save_state_callbacks.
utah.provisioning¶
utah.provisioning
Provide exceptions specific to provisioning.
- exception utah.provisioning.exceptions.UTAHProvisioningException(*args, **kw)[source]¶
Provide a foundation class for UTAH provisioning exceptions.
Provide basic inventory routines.
- class utah.provisioning.inventory.Inventory(uniqueid, lockfile='~/.utah-inventory')[source]¶
Provide a generic class for an arbitrary inventory of machines.
Raise exceptions for most methods, since they should be defined by subclasses. Except for special read-only cases, subclasses should provide request(). Subclasses will generally also need to provide delete() for cleanup purposes. All other methods (i.e. release, destroy) are optional.
- delete()[source]¶
Delete this inventory and remove the lockfile.
Subclasses can call this via super to delete the lockfile, or implement lockfile deletion in their delete() method.
- destroy()[source]¶
Mark a machine as destroyed and unusable for future tests.
Current implementations do not call machine.destroy(), requiring that do be done separately.
Returns: False since this is unimplemented but not exception-worthy Return type: bool
- class utah.provisioning.inventory.SQLiteInventory(db='~/.utah-sqlite-inventory', *args, **kw)[source]¶
Provide basic SQLite database access with a cursor.
- execute(sql, parameters=None)[source]¶
Execute SQL statement and return cursor.
This method is expected to be used as a wrapper around all connection.execute calls so that all the SQL statements are logged in case this information is needed to troubleshoot problems.
Parameters: - sql (str) – A sql statement to be executed
- parameters (list (question mark placeholders) | dict (colon named placeholders)) – Parameters to use to replace the placeholder values in the sql statement.
Returns: Cursor used to execute statement
Return type: object
- exception utah.provisioning.inventory.UTAHProvisioningInventoryException(*args, **kw)[source]¶
Provide a class for UTAH provisioning inventory exceptions.
Provide common functions for provisioning and interacting with machines. Functions here should apply to multiple machine types (VM, bare metal, etc.)
- class utah.provisioning.provisioning.CustomInstallMixin[source]¶
Provide routines for automating an install from an image.
- class utah.provisioning.provisioning.Machine(boot=None, clean=True, debug=False, image=None, initrd=None, inventory=None, kernel=None, machineid=None, machineuuid=None, name=None, new=False, preseed='/etc/utah/default-preseed.cfg', rewrite='all', template=None, xml='/etc/utah/default-vm.xml')[source]¶
Provide a generic class to provision an arbitrary machine.
Raise exceptions for most methods, since subclasses should provide them. run, uploadfiles, and downloadfiles are the most important methods for interacting with a machine.
Some method of satisfying activecheck is generally required to run commands on a machine (i.e., implementing _start or reimplementing activecheck.)
Some method of satisfying provisioncheck is generally required to install on a machine (i.e., implementing some combination of _create, _load, and _provision, or reimplementing provisioncheck.) Installation may be separated from the Machine classes in the future.
- activecheck()[source]¶
Ensure the machine is active and capable of accepting commands.
Check if the machine is running and able to accept commands, start it if necessary, and raise an exception if it cannot be started.
- cleancommand(cmd)[source]¶
Register a command to be run on cleanup.
Parameters: cmd (iterable) – A command as would be passed subprocess.Popen. See also
- cleanfile(path)[source]¶
Register a path to be cleaned later.
Parameters: path (str) – A link, file, or directory to be cleaned later. See also
- cleanfunction(function, *args, **kw)[source]¶
Register a function to be run on cleanup.
Parameters: - function (callable) – A callable that will do some cleanup.
- args (dict) – Positional arguments to the function.
- kw – Keyword arguments to the function.
See also
- destroy()[source]¶
Free up the machine and associated resources.
Release resources consumed by the machine, set provisioned to False, and return True on success. For a VM, this should destroy the VM files on the disk, and remove it from libvirt if it is registered there. For a physical machine, this should free it up to be used by another process in the future, and destroy sensitive data if any exists. Destroying the install on a physical machine is optional, but the machine must be left in a state where a new install can start.
- dldisplay(blocks, size, total)[source]¶
Log download information (i.e., as a urllib callback).
Parameters: - blocks (int) – Number of blocks downloaded
- size (int) – Size of blocks downloaded
- total – Total size of download
- downloadfiles(_files, _target='/tmp')[source]¶
Download a list of files from the machine to a local target.
Support for files as a string specifying a single file is recommended but not required. target should be a valid target path for cp, i.e. a filename for a single file, a directory name for multiple files. Recursive directory download is not currently supported by all implementations.
- installclient()[source]¶
Install the required packages on the machine.
Install the python-jsonschema, utah-common, and utah-client packages.
Raises UTAHProvisioningException: When packages won’t install
- pingcheck()[source]¶
Check network connectivity using ping.
Raises UTAHProvisioningException: When ping fails. See also
- pingpoll(timeout=None, checktimeout=180, logmethod=None)[source]¶
Run pingcheck over and over until timeout expires.
Parameters: - timeout (int or None) – How long to try pinging the machine before giving up
- checktimeout (int) – How long to wait between pings
- logmethod (function) – Function to use for logging
- provisioncheck(provision_data=None)[source]¶
Ensure the machine is provisioned and installed.
Check if the machine is provisioned, provision it if necessary, run an install if needed, and raise an exception if it cannot be provisioned, or if it exceeds the timeout value set in config for installation.
Raises UTAHProvisioningException: When the machine fails to install within the timeout value.
- rsyslog[source]¶
Use the default rsyslog method if no override is in place.
Returns: rsyslog method for this machine Return type: object
- run(_command, _quiet=None, _root=False, _timeout=None)[source]¶
Run a command on the machine.
Parameters: - quiet (bool) – If True, suppress output from helper programs like ssh
- root (bool) – If True, execute command with elevated privileges
- timeout (int or None) – Number of seconds to wait before timing out
Returns: A tuple of the form (returncode, stdout, stder)
Return type: tuple
rsyslog processing and monitoring.
- class utah.provisioning.rsyslog.RSyslog(hostname, logpath, usefile=None)[source]¶
Listen to rsyslog messages and process them.
Parameters: - hostname (str) – Host where the syslog is coming from.
- logpath (str) – Base directory where log files are created.
- usefile (str | None) – allows class to tail a file rather than act as an rsyslogd server
- wait_for_booted(uuid)[source]¶
Monitor rsyslog during boot up.
Works the same as the wait_for_install() method but takes in steps that determine a system has booted after the install has completed.
Parameters: uuid (string) – The UUID generated by provisioning code.
- wait_for_install(booted_callback=None)[source]¶
Monitor rsyslog messages during the installation.
Works through each step in the steps array to find messages in the syslog indicating what part of the install we are in.
Parameters: booted_callback (callable | None) – function to be called once the system has been booted
SSH based machine class for a provisioned system and SSHMixin for every machine class that needs SSH support.
- class utah.provisioning.ssh.ProvisionedMachine(name)[source]¶
A machine that is provisioned and can be accessed through ssh.
- class utah.provisioning.ssh.SSHMixin(*args, **kwargs)[source]¶
Provide methods for machines accessed via ssh.
- downloadfiles(files, target='/tmp')[source]¶
Copy a file or list of files from the machine to a local target.
Parameters: - files (list or str) – File or list of files to download
- target (str) – Local path to download files
- downloadfilesrecursive(files, target='/tmp')[source]¶
Recursively copy files to the target directory on the machine.
Parameters: - files (list or str) – File or list of files to download
- target (str) – Local path to download files
- initialize()[source]¶
SSH mixin initialization.
Use this method when it isn’t appropriate to follow the MRO as in __init__
- run(command, quiet=False, root=False, command_timeout=None)[source]¶
Run a command through SSH.
Parameters: - command (str | list(str)) – Command to execute on the remote machine
- quiet (bool) – Use debug level to log failure return command message when this flag is set. Otherwise, use warning.
- root (bool) – Run command as root user when this flag is set. Otherwise, use the user specified in the configuration file.
- command_timeout (int) –
Amount of time in seconds to wait for the command to complete.
See also
Provide functions for virtual machine provisioning.
- class utah.provisioning.vm.CustomVM(directory=None, diskbus='virtio', disksizes=[8], emulator=None, machineid=None, macs=[], name=None, prefix='utah', *args, **kw)[source]¶
Install a VM from an image using libvirt direct kernel booting.
- class utah.provisioning.vm.LibvirtVM(*args, **kw)[source]¶
Provide a class to utilize VMs using libvirt.
Capable of utilizing existing VMs Creation currently handled by sublcasses
- class utah.provisioning.vm.TinySQLiteInventory(*args, **kw)[source]¶
Provide basic SQLite inventory for VMs.
Only implements request, release, and destroy. No authentication or conflict checking currently exists. Only suitable for VMs at present.
- exception utah.provisioning.vm.UTAHVMProvisioningException(*args, **kw)[source]¶
Provide a foundation class for UTAH VM provisioning exceptions.
- utah.provisioning.vm.get_vm(**kw)[source]¶
Return a Machine object for a VM with the passed in arguments.
Parameters: kw (dict) – All parameters are passed to the Machine constructor Returns: Appropriately constructed Machine object Return type: object
utah.provisioning.baremetal¶
utah.provisioning.baremetal
Provide inventory functions specific to bare metal deployments.
- class utah.provisioning.baremetal.inventory.ManualBaremetalSQLiteInventory(db='~/.utah-baremetal-inventory', lockfile='~/.utah-baremetal-lock', *args, **kw)[source]¶
Keep an inventory of manually entered machines.
All columns other than machineid, name, and state are assumed to be arguments for system creation (i.e., with cobbler).
- release(machine=None, name=None)[source]¶
Release a machine so it can be used by other processes.
Parameters: - machine (obj or None) – Machine object to release
- name (str or None) – name of machine to release
- request(machinetype, name=None, *args, **kw)[source]¶
Return a Machine object meeting the given criteria.
Parameters: - machinetype (class) – What machine type to use
- name – Name of machine to use
Type: str or None
All other parameters are passed to Machine constructor.
Returns: Machine object Return type: obj
Support bare metal provisioning through cobbler.
- class utah.provisioning.baremetal.cobbler.CobblerMachine(machineinfo=None, name=None, *args, **kw)[source]¶
Provision and manage a machine via cobbler.
Support provisioning of bamboo-feeder-based systems.
- class utah.provisioning.baremetal.bamboofeeder.BambooFeederMachine(machineinfo=None, name=None, preboot=None, *args, **kw)[source]¶
Provision and manage an ARM board in a bamboo-feeder setup.
Provide exceptions specific to bare metal provisioning.