hydra_config.utils
==================
.. py:module:: hydra_config.utils
.. autoapi-nested-parse::
This module contains utility functions for use with Hydra.
Classes
-------
.. autoapisummary::
hydra_config.utils.HydraFlagWrapperMeta
Functions
---------
.. autoapisummary::
hydra_config.utils.run_hydra
hydra_config.utils.store
hydra_config.utils.builds
hydra_config.utils.safe_eval
hydra_config.utils.glob
hydra_config.utils.merge_with_kwargs
hydra_config.utils.instance_wrapper
Module Contents
---------------
.. py:function:: run_hydra(main_fn = lambda *_, **__: None, /, *, parser = argparse.ArgumentParser(), config_path = Path.cwd() / 'configs', config_name = 'base', instantiate = True, **kwargs)
This function is the main entry point for the hydra application.
The benefits of using this setup rather than the compose API is that we can
use the sweeper and launcher APIs, which are not available in the compose API.
An additional ``--hydra-help`` flag is added to the parser to print the hydra help
message when passed.
:Parameters: **main_fn** (*Callable[[Concatenate[[HydraContainerConfig], ...], None]*) -- The
main function to be called after the hydra configuration is parsed. It
should take the config as an argument and kwargs which correspond to the
argument parser returns. We don't return the config directly because hydra
allows multi-run sweeps and it doesn't make sense to return multiple
configs in this case.
Example:
.. code-block:: python
def main(config: HydraContainerConfig, *, verbose: int):
pass
parser = argparse.ArgumentParser()
parser.add_argument("--verbose", type=int, default=0)
run_hydra(main_fn=main, parser=parser)
:keyword parser: The parser to use for the hydra
application. If None, a new parser will be created.
:kwtype parser: argparse.ArgumentParser
:keyword config_path: The path to the config directory. This should be the
absolute path to the directory containing the config files. By default,
this is set to the current working directory.
:kwtype config_path: Path | str
:keyword config_name: The name of the config file to use. This should be the
name of the file without the extension. By default, this is set to
"base".
:kwtype config_name: str
:keyword instantiate: Whether to instantiate the config. If False, create
will be used.
:kwtype instantiate: bool
:keyword kwargs: Additional keyword arguments to pass to the instantiate function.
.. py:function:: store(func_or_cls, /, *, name = '', group = '', _max_recursion = 10, **kwargs)
Store a function or class in Hydra Zen's store with a specific group and name.
:Parameters: * **func_or_cls** (*Callable[..., Any] | Type[Any]*) -- The function or class to store.
* **name** (*str*) -- The name under which to store the function or class. Defaults to
an empty string.
* **group** (*str*) -- The group name to associate with the store entry. Defaults to an
empty string.
* **\*\*kwargs** (*Any*) -- Additional arguments passed to :attr:`hydra_store`.
.. py:function:: builds(func_or_cls, /, *, auto_detect = True, group = '', **kwargs)
Build a Hydra Zen configuration for a given function or class.
:Parameters: * **func_or_cls** (*Callable[..., Any] | Type[Any]*) -- The function or class to build a
configuration for.
* **auto_detect** (*bool*) -- Automatically detect and store parameter types. Defaults to
True.
* **group** (*str*) -- The group name for the configuration. Defaults to an empty string.
* **\*\*kwargs** (*Any*) -- Additional arguments passed to `zen.builds`.
:returns: *Any* -- A dataclass representing the Hydra Zen configuration.
.. py:function:: safe_eval(src, additional_vars = {})
Evaluate a string containing a Python expression in a safe manner.
This function uses
`RestrictedPython `_ to evaluate
the expression, only allowing certain built-in functions and types, and any
additional variables provided. It prevents execution of arbitrary code or access to
unauthorized functions and methods.
A number of built-in functions are supported, as provided by ``utility_builtins``,
``safe_builtins``, and ``limited_builtins`` from
`here `_.
:Parameters: * **src** (*str*) -- The source code to evaluate.
* **additional_vars** (*Dict[str, Any]*) -- A dictionary of additional variables or
functions to include in the evaluation environment.
.. warning::
This can be unsafe if the variables are not properly sanitized, thus
``additional_vars`` should be used with caution.
:returns: *Any* -- The result of the evaluated expression.
.. rubric:: Example
>>> safe_eval("1 + 2")
3
>>> safe_eval("max([1, 2, 3])")
3
>>> safe_eval("math.sqrt(a)", {'a': 16})
4.0
.. py:function:: glob(key, flattened, _root_)
This resolver will glob a key in the config. This is useful for finding all keys
that match a pattern. This is particularly useful for finding all keys that match a
pattern in a nested config. This is effectively select, but allows ``*`` to be used
as a wildcard.
This method works by finding all ``*`` in the key and then iterating over all
subsequent keys that match the globbed pattern.
.. note::
yaml files aren't necessarily built to support globbing (like xml), so
this method is fairly slow and should be used sparingly.
.. note::
List indexing is limited in support. To index an element in a list, you
must use bracket notation. For instance, ``a[0].b`` is supported, but ``a.0.b``
is not.
:Parameters: * **key** (*str*) -- The key to glob. This is a dotlist key, like ``a.b.*``. Multiple
globs can be used, like ``a.*.c.*.d.*``. Globs in keys can be used, as
well, such as ``a.ab*.c``
* **flatten** (*bool*) -- If true, the output will be a dict of the leaf keys and
the accumulated values if there are like leaf keys. If False, the
output will be a nested dict. Defaults to False.
* **_root_** (*DictConfig*) -- The root config.
.. py:function:: merge_with_kwargs(config, *, instantiate = True, **kwargs)
This method will merge the kwargs into the config. This is useful for merging
"late", as in after the config has been resolved (not instantiated). By specifying
the merge to happen at instantiation time rather than at resolution time, it gives
more freedom in defining overrides within the config.
This is intended to be called from a yaml config file like:
.. code-block:: yaml
config_to_merge_late:
_target_: .merge_with_kwargs
_recursive_: False
config: ${...} # this is what the kwargs are merged into
kwarg1: value1
kwarg2: value2
...
.. note::
You may want ``_recursive_=False`` (as above) to avoid instantiating the config
before merging the kwargs. If you want to override a config attribute in the
config object which is instantiated (i.e. is a partial), you won't have access
to the config attribute (only the partial object), so you would want
``_recursive_=False``. Simpler cases can just use ``_recursive_=True``.
:Parameters: **config** (*DictConfig*) -- The config to merge the kwargs into.
:keyword kwargs: The kwargs to merge into the config.
.. py:function:: instance_wrapper(*, instance, key = None, locate = False, eval = False, setitem = False, **kwargs)
Wraps a class instance to allow setting class attributes after initialization.
This utility is useful when not all attributes are available during class
instantiation, allowing attributes to be set post-construction using either
direct assignment, item setting, or attribute modification based on optional flags.
:Parameters: * **instance** (*Any*) -- The class instance to wrap.
* **key** (*Optional[str], optional*) -- If provided, fetches the specified attribute
from the instance to modify. Defaults to None.
* **locate** (*bool, optional*) -- If True, attempts to resolve attribute names
dynamically (e.g., via object lookup). Defaults to False.
* **eval** (*bool, optional*) -- If True, evaluates attribute values using safe_eval
before assignment. Defaults to False.
* **setitem** (*bool, optional*) -- If True, uses item assignment
(e.g., ``instance[key]``) instead of ``setattr``. Defaults to False.
* **\*\*kwargs** -- Key-value pairs of attributes to set on the instance.
:returns: *Any* -- The modified instance.
:raises ValueError: If there is an error while setting an attribute.
.. rubric:: Example
.. code-block:: yaml
obj_to_instantiate:
_target_: .instance_wrapper
instance:
_target_:
_args_: [arg1, arg2]
init_arg1: value1
init_arg2: value2
set_arg1: value1
set_arg2: value2
For partial instantiation:
.. code-block:: yaml
partial_obj_to_instantiate:
_target_: .instance_wrapper
instance:
_target_:
_partial_: True
_args_: [arg1, arg2]
init_arg3: '???' # Set later
set_arg1: value1
set_arg2: value2
.. py:class:: HydraFlagWrapperMeta
Bases: :py:obj:`enum.EnumMeta`
This is a simple metaclass to allow for the use of the | operator to combine
flags. This means you can simply put ``flag1 | flag2`` in the yaml file and it will
be combined into a single flag.
The following forms are supported and any combination thereof:
- ``flag1 | flag2 | flag3 | ...``
- ``flag1|flag2|flag3|...``
- ``flag1``
.. py:method:: __getitem__(item)
This method allows for the use of the | operator to combine flags. This
means you can simply put ``flag1 | flag2`` in the yaml file and it will be
combined into a single flag.