Base Record#
A record is an object that represents a single computation. Records can be simple, such as a singlepoint record or complex with lots of subrecords of different types (such as a NEB record.
However, all records share a set of common methods and properties.
Records can be submitted, retrieved, and modified using the QCPortal client.
Tip
All records are Pydantic models, and so incorporates all the features from that library.
Metadata#
ID, status, created_on, modified_on, comments, owner_user, owner_group, extras
Computation History#
compute_history, manager_name, provenance, stdout, stderr, error, native_files
Task or Service#
task, service, is_service
See: Tasks and Services
Base Record API#
- pydantic model BaseRecord[source]#
Bases:
BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
- Config:
allow_mutation: bool = True
extra: Extra = Extra.forbid
validate_assignment: bool = True
- Fields:
- Validators:
_validate_extras
»extras
- field id: int [Required]#
- field record_type: str [Required]#
- field is_service: bool [Required]#
- field properties: Dict[str, Any] | None = None#
- field extras: Dict[str, Any] = {}#
- Validated by:
_validate_extras
- field status: RecordStatusEnum [Required]#
- field manager_name: str | None = None#
- field created_on: datetime [Required]#
- field modified_on: datetime [Required]#
- field owner_user: str | None = None#
- field owner_group: str | None = None#
- field compute_history_: List[ComputeHistory] | None = None (alias 'compute_history')#
- field task_: RecordTask | None = None (alias 'task')#
- field service_: RecordService | None = None (alias 'service')#
- field comments_: List[RecordComment] | None = None (alias 'comments')#
- field native_files_: Dict[str, NativeFile] | None = None (alias 'native_files')#
- classmethod get_subclass(record_type)[source]#
Obtain a subclass of this class given its record_type
- Parameters:
record_type (str)
- Return type:
Type[BaseRecord]
- classmethod fetch_children_multi(records, include=None, force_fetch=False)[source]#
Fetches all children of the given records
This tries to work efficiently, fetching larger batches of children that can span multiple records
- Parameters:
records (Iterable[BaseRecord | None])
include (Iterable[str] | None)
force_fetch (bool)
- fetch_children(include=None, force_fetch=False)[source]#
Fetches all children of this record recursively
- Parameters:
include (Iterable[str] | None)
force_fetch (bool)
- sync_to_cache(detach=False)[source]#
Syncs this record to the cache
If detach is True, then the record will be removed from the cache
- Parameters:
detach (bool)
- propagate_client(client)[source]#
Propagates a client and related information to this record to any fields within this record that need it
This is expected to be called from derived class propagate_client functions as well
- property offline: bool#
- property children_status: Dict[RecordStatusEnum, int]#
Returns a dictionary of the status of all children of this record
- property children_errors: List[BaseRecord]#
Returns errored child records
- property compute_history: List[ComputeHistory]#
- property task: RecordTask | None#
- property service: RecordService | None#
- property comments: List[RecordComment] | None#
- property native_files: Dict[str, NativeFile] | None#
- property stdout: str | None#
- property stderr: str | None#
- property error: Dict[str, Any] | None#
- property provenance: Provenance | None#
- pydantic model BaseDataset[source]#
Bases:
BaseModel
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
- Config:
allow_mutation: bool = True
extra: Extra = Extra.forbid
validate_assignment: bool = True
- Fields:
- field id: int [Required]#
- field dataset_type: str [Required]#
- field name: str [Required]#
- field description: str [Required]#
- field tagline: str [Required]#
- field tags: List[str] [Required]#
- field group: str [Required]#
- field visibility: bool [Required]#
- field provenance: Dict[str, Any] [Required]#
- field default_tag: str [Required]#
- field default_priority: PriorityEnum [Required]#
- field owner_user: str | None = None#
- field owner_group: str | None = None#
- field metadata: Dict[str, Any] [Required]#
- field extras: Dict[str, Any] [Required]#
- field auto_fetch_missing: bool = True#
- propagate_client(client)[source]#
Propagates a client to this record to any fields within this record that need it
This may also be called from derived class propagate_client functions as well
- submit(entry_names=None, specification_names=None, tag=None, priority=None, find_existing=True)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
tag (str | None)
priority (PriorityEnum)
find_existing (bool)
- property is_view: bool#
- status_table()[source]#
Returns the status of the dataset’s computations as a table (in a string)
- Return type:
str
- detailed_status()[source]#
- Return type:
List[Tuple[str, str, RecordStatusEnum]]
- property offline: bool#
- property record_count: int#
- property computed_properties#
- set_default_priority(new_default_priority)[source]#
- Parameters:
new_default_priority (PriorityEnum)
- fetch_specification_names()[source]#
Fetch all entry names from the remote server
These are fetched and then stored internally, and not returned.
- Return type:
None
- fetch_specifications(specification_names=None, force_refetch=False)[source]#
Fetch specifications from the remote server, storing them internally
- Parameters:
specification_names (str | Iterable[str] | None) – Names of specifications to fetch. If None, fetch all specifications
force_refetch (bool) – If true, fetch data from the server even if it already exists locally
- Return type:
None
- property specification_names: List[str]#
- property specifications: Mapping[str, Any]#
- delete_specification(name, delete_records=False)[source]#
- Parameters:
name (str)
delete_records (bool)
- Return type:
- fetch_entry_names()[source]#
Fetch all entry names from the remote server
These are fetched and then stored internally, and not returned.
- Return type:
None
- fetch_entries(entry_names=None, force_refetch=False)[source]#
Fetches entry information from the remote server, storing it internally
By default, already-fetched entries will not be fetched again, unless force_refetch is True.
- Parameters:
entry_names (str | Iterable[str] | None) – Names of entries to fetch. If None, fetch all entries
force_refetch (bool) – If true, fetch data from the server even if it already exists locally
- Return type:
None
- get_entry(entry_name, force_refetch=False)[source]#
Obtain entry information
The entry will be automatically fetched from the remote server if needed.
- Parameters:
entry_name (str)
force_refetch (bool)
- Return type:
Any | None
- iterate_entries(entry_names=None, force_refetch=False)[source]#
Iterate over all entries
This is used as a generator, and automatically fetches entries as needed
- Parameters:
entry_names (str | Iterable[str] | None) – Names of entries to iterate over. If None, iterate over all entries
force_refetch (bool) – If true, fetch data from the server even if it already exists locally
- property entry_names: List[str]#
- delete_entries(names, delete_records=False)[source]#
- Parameters:
names (str | Iterable[str])
delete_records (bool)
- Return type:
- fetch_records(entry_names=None, specification_names=None, status=None, include=None, fetch_updated=True, force_refetch=False)[source]#
Fetches record information from the remote server, storing it internally
By default, this function will only fetch records that have not been fetch previously. If force_refetch is True, then this will always fetch the records.
- Parameters:
entry_names (str | Iterable[str] | None) – Names of the entries whose records to fetch. If None, fetch all entries
specification_names (str | Iterable[str] | None) – Names of the specifications whose records to fetch. If None, fetch all specifications
status (RecordStatusEnum | Iterable[RecordStatusEnum] | None) – Fetch only records with these statuses
include (Iterable[str] | None) – Additional fields to include in the returned record
fetch_updated (bool) – Fetch any records that exist locally but have been updated on the server
force_refetch (bool) – If true, fetch data from the server even if it already exists locally
- get_record(entry_name, specification_name, include=None, fetch_updated=True, force_refetch=False)[source]#
Obtain a calculation record related to this dataset
The record will be automatically fetched from the remote server if needed. If a record does not exist for this entry and specification, None is returned
- Parameters:
entry_name (str)
specification_name (str)
include (Iterable[str] | None)
fetch_updated (bool)
force_refetch (bool)
- Return type:
BaseRecord | None
- iterate_records(entry_names=None, specification_names=None, status=None, include=None, fetch_updated=True, force_refetch=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
status (RecordStatusEnum | Iterable[RecordStatusEnum] | None)
include (Iterable[str] | None)
fetch_updated (bool)
force_refetch (bool)
- remove_records(entry_names=None, specification_names=None, delete_records=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
delete_records (bool)
- Return type:
- modify_records(entry_names=None, specification_names=None, new_tag=None, new_priority=None, new_comment=None, *, refetch_records=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
new_tag (str | None)
new_priority (PriorityEnum | None)
new_comment (str | None)
refetch_records (bool)
- reset_records(entry_names=None, specification_names=None, *, refetch_records=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
refetch_records (bool)
- cancel_records(entry_names=None, specification_names=None, *, refetch_records=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
refetch_records (bool)
- uncancel_records(entry_names=None, specification_names=None, *, refetch_records=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
refetch_records (bool)
- invalidate_records(entry_names=None, specification_names=None, *, refetch_records=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
refetch_records (bool)
- uninvalidate_records(entry_names=None, specification_names=None, *, refetch_records=False)[source]#
- Parameters:
entry_names (str | Iterable[str] | None)
specification_names (str | Iterable[str] | None)
refetch_records (bool)
- compile_values(value_call, value_names='value', entry_names=None, specification_names=None, unpack=False)[source]#
Compile values from records into a pandas DataFrame.
- Parameters:
value_call (Callable) – Function to call on each record to extract the desired value. Must return a scalar value or a sequence of values if ‘unpack’ is set to True.
value_names (Union[Sequence[str], str]) – Column name(s) for the extracted value(s). If a string is provided and multiple values are returned by ‘value_call’, columns are named by appending an index to this string. If a list of strings is provided, it must match the length of the sequence returned by ‘value_call’. Default is “value”.
entry_names (Optional[Union[str, Iterable[str]]]) – Entry names to filter records. If not provided, considers all entries.
specification_names (Optional[Union[str, Iterable[str]]]) – Specification names to filter records. If not provided, considers all specifications.
unpack (bool) – If True, unpack the sequence of values returned by ‘value_call’ into separate columns. Default is False.
- Returns:
A multi-index DataFrame where each row corresponds to an entry. Each column corresponds has a top level index as a specification, and a second level index as the appropriate value name. Values are extracted from records using ‘value_call’.
- Return type:
pandas.DataFrame
- Raises:
ValueError – If the length of ‘value_names’ does not match the number of values returned by ‘value_call’ when ‘unpack’ is set to True.
Notes
The DataFrame is structured such that the rows are entries and columns are specifications.
2. If ‘unpack’ is True, the function assumes ‘value_call’ returns a sequence of values that need to be distributed across columns in the resulting DataFrame. ‘value_call’ should always return the same number of values for each record if unpack is True.
- get_properties_df(properties_list)[source]#
Retrieve a DataFrame populated with the specified properties from dataset records.
This function uses the provided list of property names to extract corresponding values from each record’s properties. It returns a DataFrame where rows represent each record. Each column corresponds has a top level index as a specification, and a second level index as the appropriate value name. Columns with all NaN values are dropped.
Parameters:#
- properties_list
List of property names to retrieve from the records.
Returns:#
- pandas.DataFrame
A DataFrame populated with the specified properties for each record.
- Parameters:
properties_list (Sequence[str])
- Return type:
DataFrame
- refresh_cache(entry_names=None, specification_names=None)[source]#
Refreshes some information in the cache with information on the server
This can be used to fix some inconsistencies in the cache without deleting and starting over. For example, this can fix instances where the record attached to a given entry & specification has changed (new record id) due to renaming specifications and entries, or via remove_records followed by a submit without duplicate checking.
This will also fetch any updated records
- Parameters:
entry_names (str | Iterable[str] | None) – Names of the entries whose records to fetch. If None, fetch all entries
specification_names (str | Iterable[str] | None) – Names of the specifications whose records to fetch. If None, fetch all specifications
- property contributed_values: Dict[str, ContributedValues]#
- class RecordStatusEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#
Bases:
str
,Enum
The state of a record object. The states which are available are a finite set.
- complete = 'complete'#
- invalid = 'invalid'#
- running = 'running'#
- error = 'error'#
- waiting = 'waiting'#
- cancelled = 'cancelled'#
- deleted = 'deleted'#
- classmethod make_ordered_status(statuses)[source]#
Returns a list of the given statuses but in a defined order
- Parameters:
statuses (Iterable[RecordStatusEnum])
- Return type:
List[RecordStatusEnum]
- encode(encoding='utf-8', errors='strict')#
Encode the string using the codec registered for encoding.
- encoding
The encoding in which to encode the string.
- errors
The error handling scheme to use for encoding errors. The default is ‘strict’ meaning that encoding errors raise a UnicodeEncodeError. Other possible values are ‘ignore’, ‘replace’ and ‘xmlcharrefreplace’ as well as any other name registered with codecs.register_error that can handle UnicodeEncodeErrors.
- replace(old, new, count=-1, /)#
Return a copy with all occurrences of substring old replaced by new.
- count
Maximum number of occurrences to replace. -1 (the default value) means replace all occurrences.
If the optional argument count is given, only the first count occurrences are replaced.
- split(sep=None, maxsplit=-1)#
Return a list of the substrings in the string, using sep as the separator string.
- sep
The separator used to split the string.
When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.
- maxsplit
Maximum number of splits. -1 (the default value) means no limit.
Splitting starts at the front of the string and works to the end.
Note, str.split() is mainly useful for data that has been intentionally delimited. With natural text that includes punctuation, consider using the regular expression module.
- rsplit(sep=None, maxsplit=-1)#
Return a list of the substrings in the string, using sep as the separator string.
- sep
The separator used to split the string.
When set to None (the default value), will split on any whitespace character (including n r t f and spaces) and will discard empty strings from the result.
- maxsplit
Maximum number of splits. -1 (the default value) means no limit.
Splitting starts at the end of the string and works to the front.
- join(iterable, /)#
Concatenate any number of strings.
The string whose method is called is inserted in between each given string. The result is returned as a new string.
Example: ‘.’.join([‘ab’, ‘pq’, ‘rs’]) -> ‘ab.pq.rs’
- capitalize()#
Return a capitalized version of the string.
More specifically, make the first character have upper case and the rest lower case.
- casefold()#
Return a version of the string suitable for caseless comparisons.
- title()#
Return a version of the string where each word is titlecased.
More specifically, words start with uppercased characters and all remaining cased characters have lower case.
- center(width, fillchar=' ', /)#
Return a centered string of length width.
Padding is done using the specified fill character (default is a space).
- count(sub[, start[, end]]) int #
Return the number of non-overlapping occurrences of substring sub in string S[start:end]. Optional arguments start and end are interpreted as in slice notation.
- expandtabs(tabsize=8)#
Return a copy where all tab characters are expanded using spaces.
If tabsize is not given, a tab size of 8 characters is assumed.
- find(sub[, start[, end]]) int #
Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.
Return -1 on failure.
- partition(sep, /)#
Partition the string into three parts using the given separator.
This will search for the separator in the string. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.
If the separator is not found, returns a 3-tuple containing the original string and two empty strings.
- index(sub[, start[, end]]) int #
Return the lowest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.
Raises ValueError when the substring is not found.
- ljust(width, fillchar=' ', /)#
Return a left-justified string of length width.
Padding is done using the specified fill character (default is a space).
- lower()#
Return a copy of the string converted to lowercase.
- lstrip(chars=None, /)#
Return a copy of the string with leading whitespace removed.
If chars is given and not None, remove characters in chars instead.
- rfind(sub[, start[, end]]) int #
Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.
Return -1 on failure.
- rindex(sub[, start[, end]]) int #
Return the highest index in S where substring sub is found, such that sub is contained within S[start:end]. Optional arguments start and end are interpreted as in slice notation.
Raises ValueError when the substring is not found.
- rjust(width, fillchar=' ', /)#
Return a right-justified string of length width.
Padding is done using the specified fill character (default is a space).
- rstrip(chars=None, /)#
Return a copy of the string with trailing whitespace removed.
If chars is given and not None, remove characters in chars instead.
- rpartition(sep, /)#
Partition the string into three parts using the given separator.
This will search for the separator in the string, starting at the end. If the separator is found, returns a 3-tuple containing the part before the separator, the separator itself, and the part after it.
If the separator is not found, returns a 3-tuple containing two empty strings and the original string.
- splitlines(keepends=False)#
Return a list of the lines in the string, breaking at line boundaries.
Line breaks are not included in the resulting list unless keepends is given and true.
- strip(chars=None, /)#
Return a copy of the string with leading and trailing whitespace removed.
If chars is given and not None, remove characters in chars instead.
- swapcase()#
Convert uppercase characters to lowercase and lowercase characters to uppercase.
- translate(table, /)#
Replace each character in the string using the given translation table.
- table
Translation table, which must be a mapping of Unicode ordinals to Unicode ordinals, strings, or None.
The table must implement lookup/indexing via __getitem__, for instance a dictionary or list. If this operation raises LookupError, the character is left untouched. Characters mapped to None are deleted.
- upper()#
Return a copy of the string converted to uppercase.
- startswith(prefix[, start[, end]]) bool #
Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.
- endswith(suffix[, start[, end]]) bool #
Return True if S ends with the specified suffix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. suffix can also be a tuple of strings to try.
- removeprefix(prefix, /)#
Return a str with the given prefix string removed if present.
If the string starts with the prefix string, return string[len(prefix):]. Otherwise, return a copy of the original string.
- removesuffix(suffix, /)#
Return a str with the given suffix string removed if present.
If the string ends with the suffix string and that suffix is not empty, return string[:-len(suffix)]. Otherwise, return a copy of the original string.
- isascii()#
Return True if all characters in the string are ASCII, False otherwise.
ASCII characters have code points in the range U+0000-U+007F. Empty string is ASCII too.
- islower()#
Return True if the string is a lowercase string, False otherwise.
A string is lowercase if all cased characters in the string are lowercase and there is at least one cased character in the string.
- isupper()#
Return True if the string is an uppercase string, False otherwise.
A string is uppercase if all cased characters in the string are uppercase and there is at least one cased character in the string.
- istitle()#
Return True if the string is a title-cased string, False otherwise.
In a title-cased string, upper- and title-case characters may only follow uncased characters and lowercase characters only cased ones.
- isspace()#
Return True if the string is a whitespace string, False otherwise.
A string is whitespace if all characters in the string are whitespace and there is at least one character in the string.
- isdecimal()#
Return True if the string is a decimal string, False otherwise.
A string is a decimal string if all characters in the string are decimal and there is at least one character in the string.
- isdigit()#
Return True if the string is a digit string, False otherwise.
A string is a digit string if all characters in the string are digits and there is at least one character in the string.
- isnumeric()#
Return True if the string is a numeric string, False otherwise.
A string is numeric if all characters in the string are numeric and there is at least one character in the string.
- isalpha()#
Return True if the string is an alphabetic string, False otherwise.
A string is alphabetic if all characters in the string are alphabetic and there is at least one character in the string.
- isalnum()#
Return True if the string is an alpha-numeric string, False otherwise.
A string is alpha-numeric if all characters in the string are alpha-numeric and there is at least one character in the string.
- isidentifier()#
Return True if the string is a valid Python identifier, False otherwise.
Call keyword.iskeyword(s) to test whether string s is a reserved identifier, such as “def” or “class”.
- isprintable()#
Return True if the string is printable, False otherwise.
A string is printable if all of its characters are considered printable in repr() or if it is empty.
- zfill(width, /)#
Pad a numeric string with zeros on the left, to fill a field of the given width.
The string is never truncated.
- format(*args, **kwargs) str #
Return a formatted version of S, using substitutions from args and kwargs. The substitutions are identified by braces (‘{’ and ‘}’).
- format_map(mapping) str #
Return a formatted version of S, using substitutions from mapping. The substitutions are identified by braces (‘{’ and ‘}’).
- static maketrans()#
Return a translation table usable for str.translate().
If there is only one argument, it must be a dictionary mapping Unicode ordinals (integers) or characters to Unicode ordinals, strings or None. Character keys will be then converted to ordinals. If there are two arguments, they must be strings of equal length, and in the resulting dictionary, each character in x will be mapped to the character at the same position in y. If there is a third argument, it must be a string, whose characters will be mapped to None in the result.
- class PriorityEnum(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]#
Bases:
int
,Enum
The priority of a Task. Higher priority will be pulled first.
- high = 2#
- normal = 1#
- low = 0#
- conjugate()#
Returns self, the complex conjugate of any int.
- bit_length()#
Number of bits necessary to represent self in binary.
>>> bin(37) '0b100101' >>> (37).bit_length() 6
- bit_count()#
Number of ones in the binary representation of the absolute value of self.
Also known as the population count.
>>> bin(13) '0b1101' >>> (13).bit_count() 3
- to_bytes(length=1, byteorder='big', *, signed=False)#
Return an array of bytes representing an integer.
- length
Length of bytes object to use. An OverflowError is raised if the integer is not representable with the given number of bytes. Default is length 1.
- byteorder
The byte order used to represent the integer. If byteorder is ‘big’, the most significant byte is at the beginning of the byte array. If byteorder is ‘little’, the most significant byte is at the end of the byte array. To request the native byte order of the host system, use `sys.byteorder’ as the byte order value. Default is to use ‘big’.
- signed
Determines whether two’s complement is used to represent the integer. If signed is False and a negative integer is given, an OverflowError is raised.
- from_bytes(byteorder='big', *, signed=False)#
Return the integer represented by the given array of bytes.
- bytes
Holds the array of bytes to convert. The argument must either support the buffer protocol or be an iterable object producing bytes. Bytes and bytearray are examples of built-in objects that support the buffer protocol.
- byteorder
The byte order used to represent the integer. If byteorder is ‘big’, the most significant byte is at the beginning of the byte array. If byteorder is ‘little’, the most significant byte is at the end of the byte array. To request the native byte order of the host system, use `sys.byteorder’ as the byte order value. Default is to use ‘big’.
- signed
Indicates whether two’s complement is used to represent the integer.
- as_integer_ratio()#
Return integer ratio.
Return a pair of integers, whose ratio is exactly equal to the original int and with a positive denominator.
>>> (10).as_integer_ratio() (10, 1) >>> (-10).as_integer_ratio() (-10, 1) >>> (0).as_integer_ratio() (0, 1)
- real#
the real part of a complex number
- imag#
the imaginary part of a complex number
- numerator#
the numerator of a rational number in lowest terms
- denominator#
the denominator of a rational number in lowest terms
- class RecordQueryIterator(client, query_filters, record_type, include=None)[source]#
Bases:
QueryIteratorBase
[_Record_T
]Iterator for all types of record queries
This iterator transparently handles batching and pagination over the results of a record query, and works with all kinds of records.
Construct an iterator
- Parameters:
client – QCPortal client object used to contact/retrieve data from the server
query_filters (RecordQueryFilters) – The actual query information to send to the server
record_type (Type[_Record_T]) – What type of record we are querying for
include (Optional[Iterable[str]])
- reset()#
Starts retrieval of results from the beginning again