hydra_config.utils ================== .. py:module:: hydra_config.utils .. autoapi-nested-parse:: This module contains utility functions for use with Hydra. Classes ------- .. autoapisummary:: hydra_config.utils.HydraFlagWrapperMeta Functions --------- .. autoapisummary:: hydra_config.utils.run_hydra hydra_config.utils.store hydra_config.utils.builds hydra_config.utils.safe_eval hydra_config.utils.glob hydra_config.utils.merge_with_kwargs hydra_config.utils.instance_wrapper Module Contents --------------- .. py:function:: run_hydra(main_fn = lambda *_, **__: None, /, *, parser = argparse.ArgumentParser(), config_path = Path.cwd() / 'configs', config_name = 'base', instantiate = True, **kwargs) This function is the main entry point for the hydra application. The benefits of using this setup rather than the compose API is that we can use the sweeper and launcher APIs, which are not available in the compose API. An additional ``--hydra-help`` flag is added to the parser to print the hydra help message when passed. :Parameters: **main_fn** (*Callable[[Concatenate[[HydraContainerConfig], ...], None]*) -- The main function to be called after the hydra configuration is parsed. It should take the config as an argument and kwargs which correspond to the argument parser returns. We don't return the config directly because hydra allows multi-run sweeps and it doesn't make sense to return multiple configs in this case. Example: .. code-block:: python def main(config: HydraContainerConfig, *, verbose: int): pass parser = argparse.ArgumentParser() parser.add_argument("--verbose", type=int, default=0) run_hydra(main_fn=main, parser=parser) :keyword parser: The parser to use for the hydra application. If None, a new parser will be created. :kwtype parser: argparse.ArgumentParser :keyword config_path: The path to the config directory. This should be the absolute path to the directory containing the config files. By default, this is set to the current working directory. :kwtype config_path: Path | str :keyword config_name: The name of the config file to use. This should be the name of the file without the extension. By default, this is set to "base". :kwtype config_name: str :keyword instantiate: Whether to instantiate the config. If False, create will be used. :kwtype instantiate: bool :keyword kwargs: Additional keyword arguments to pass to the instantiate function. .. py:function:: store(func_or_cls, /, *, name = '', group = '', _max_recursion = 10, **kwargs) Store a function or class in Hydra Zen's store with a specific group and name. :Parameters: * **func_or_cls** (*Callable[..., Any] | Type[Any]*) -- The function or class to store. * **name** (*str*) -- The name under which to store the function or class. Defaults to an empty string. * **group** (*str*) -- The group name to associate with the store entry. Defaults to an empty string. * **\*\*kwargs** (*Any*) -- Additional arguments passed to :attr:`hydra_store`. .. py:function:: builds(func_or_cls, /, *, auto_detect = True, group = '', **kwargs) Build a Hydra Zen configuration for a given function or class. :Parameters: * **func_or_cls** (*Callable[..., Any] | Type[Any]*) -- The function or class to build a configuration for. * **auto_detect** (*bool*) -- Automatically detect and store parameter types. Defaults to True. * **group** (*str*) -- The group name for the configuration. Defaults to an empty string. * **\*\*kwargs** (*Any*) -- Additional arguments passed to `zen.builds`. :returns: *Any* -- A dataclass representing the Hydra Zen configuration. .. py:function:: safe_eval(src, additional_vars = {}) Evaluate a string containing a Python expression in a safe manner. This function uses `RestrictedPython `_ to evaluate the expression, only allowing certain built-in functions and types, and any additional variables provided. It prevents execution of arbitrary code or access to unauthorized functions and methods. A number of built-in functions are supported, as provided by ``utility_builtins``, ``safe_builtins``, and ``limited_builtins`` from `here `_. :Parameters: * **src** (*str*) -- The source code to evaluate. * **additional_vars** (*Dict[str, Any]*) -- A dictionary of additional variables or functions to include in the evaluation environment. .. warning:: This can be unsafe if the variables are not properly sanitized, thus ``additional_vars`` should be used with caution. :returns: *Any* -- The result of the evaluated expression. .. rubric:: Example >>> safe_eval("1 + 2") 3 >>> safe_eval("max([1, 2, 3])") 3 >>> safe_eval("math.sqrt(a)", {'a': 16}) 4.0 .. py:function:: glob(key, flattened, _root_) This resolver will glob a key in the config. This is useful for finding all keys that match a pattern. This is particularly useful for finding all keys that match a pattern in a nested config. This is effectively select, but allows ``*`` to be used as a wildcard. This method works by finding all ``*`` in the key and then iterating over all subsequent keys that match the globbed pattern. .. note:: yaml files aren't necessarily built to support globbing (like xml), so this method is fairly slow and should be used sparingly. .. note:: List indexing is limited in support. To index an element in a list, you must use bracket notation. For instance, ``a[0].b`` is supported, but ``a.0.b`` is not. :Parameters: * **key** (*str*) -- The key to glob. This is a dotlist key, like ``a.b.*``. Multiple globs can be used, like ``a.*.c.*.d.*``. Globs in keys can be used, as well, such as ``a.ab*.c`` * **flatten** (*bool*) -- If true, the output will be a dict of the leaf keys and the accumulated values if there are like leaf keys. If False, the output will be a nested dict. Defaults to False. * **_root_** (*DictConfig*) -- The root config. .. py:function:: merge_with_kwargs(config, *, instantiate = True, **kwargs) This method will merge the kwargs into the config. This is useful for merging "late", as in after the config has been resolved (not instantiated). By specifying the merge to happen at instantiation time rather than at resolution time, it gives more freedom in defining overrides within the config. This is intended to be called from a yaml config file like: .. code-block:: yaml config_to_merge_late: _target_: .merge_with_kwargs _recursive_: False config: ${...} # this is what the kwargs are merged into kwarg1: value1 kwarg2: value2 ... .. note:: You may want ``_recursive_=False`` (as above) to avoid instantiating the config before merging the kwargs. If you want to override a config attribute in the config object which is instantiated (i.e. is a partial), you won't have access to the config attribute (only the partial object), so you would want ``_recursive_=False``. Simpler cases can just use ``_recursive_=True``. :Parameters: **config** (*DictConfig*) -- The config to merge the kwargs into. :keyword kwargs: The kwargs to merge into the config. .. py:function:: instance_wrapper(*, instance, key = None, locate = False, eval = False, setitem = False, **kwargs) Wraps a class instance to allow setting class attributes after initialization. This utility is useful when not all attributes are available during class instantiation, allowing attributes to be set post-construction using either direct assignment, item setting, or attribute modification based on optional flags. :Parameters: * **instance** (*Any*) -- The class instance to wrap. * **key** (*Optional[str], optional*) -- If provided, fetches the specified attribute from the instance to modify. Defaults to None. * **locate** (*bool, optional*) -- If True, attempts to resolve attribute names dynamically (e.g., via object lookup). Defaults to False. * **eval** (*bool, optional*) -- If True, evaluates attribute values using safe_eval before assignment. Defaults to False. * **setitem** (*bool, optional*) -- If True, uses item assignment (e.g., ``instance[key]``) instead of ``setattr``. Defaults to False. * **\*\*kwargs** -- Key-value pairs of attributes to set on the instance. :returns: *Any* -- The modified instance. :raises ValueError: If there is an error while setting an attribute. .. rubric:: Example .. code-block:: yaml obj_to_instantiate: _target_: .instance_wrapper instance: _target_: _args_: [arg1, arg2] init_arg1: value1 init_arg2: value2 set_arg1: value1 set_arg2: value2 For partial instantiation: .. code-block:: yaml partial_obj_to_instantiate: _target_: .instance_wrapper instance: _target_: _partial_: True _args_: [arg1, arg2] init_arg3: '???' # Set later set_arg1: value1 set_arg2: value2 .. py:class:: HydraFlagWrapperMeta Bases: :py:obj:`enum.EnumMeta` This is a simple metaclass to allow for the use of the | operator to combine flags. This means you can simply put ``flag1 | flag2`` in the yaml file and it will be combined into a single flag. The following forms are supported and any combination thereof: - ``flag1 | flag2 | flag3 | ...`` - ``flag1|flag2|flag3|...`` - ``flag1`` .. py:method:: __getitem__(item) This method allows for the use of the | operator to combine flags. This means you can simply put ``flag1 | flag2`` in the yaml file and it will be combined into a single flag.