satpy.readers package

Submodules

Module contents

Shared objects of the various reader classes.

class satpy.readers.FSFile(file, fs=None)[source]

Bases: PathLike

Implementation of a PathLike file object, that can be opened.

Giving the filenames to Scene with valid transfer protocols will automatically use this class so manual usage of this class is needed mainly for fine-grained control.

This class is made to be used in conjuction with fsspec or s3fs. For example:

from satpy import Scene

import fsspec
filename = 'noaa-goes16/ABI-L1b-RadC/2019/001/17/*_G16_s20190011702186*'

the_files = fsspec.open_files("simplecache::s3://" + filename, s3={'anon': True})

from satpy.readers import FSFile
fs_files = [FSFile(open_file) for open_file in the_files]

scn = Scene(filenames=fs_files, reader='abi_l1b')
scn.load(['true_color_raw'])

Initialise the FSFile instance.

Parameters

  • file (str, Pathlike, or OpenFile) – String, object implementing the os.PathLike protocol, or an fsspec.OpenFile instance. If passed an instance of fsspec.OpenFile, the following argument fs has no effect.

  • fs (fsspec filesystem, optional) – Object implementing the fsspec filesystem protocol.

open(*args, **kwargs)[source]

Open the file.

This is read-only.

satpy.readers.available_readers(as_dict=False, yaml_loader=<class 'yaml.loader.UnsafeLoader'>)[source]

Available readers based on current configuration.

Parameters

  • as_dict (bool) – Optionally return reader information as a dictionary. Default: False.

  • yaml_loader (Optional[Union[yaml.BaseLoader, yaml.FullLoader, yaml.UnsafeLoader]]) – The yaml loader type. Default: yaml.UnsafeLoader.

Returns

List of available reader names. If as_dict is True then a list of dictionaries including additionally reader information is returned.

Return type

Union[list[str], list[dict]]

satpy.readers.configs_for_reader(reader=None)[source]

Generate reader configuration files for one or more readers.

Parameters

reader (Optional[str]) – Yield configs only for this reader

Returns: Generator of lists of configuration files

satpy.readers.find_files_and_readers(start_time=None, end_time=None, base_dir=None, reader=None, sensor=None, filter_parameters=None, reader_kwargs=None, missing_ok=False, fs=None)[source]

Find files matching the provided parameters.

Use start_time and/or end_time to limit found filenames by the times in the filenames (not the internal file metadata). Files are matched if they fall anywhere within the range specified by these parameters.

Searching is NOT recursive.

Files may be either on-disk or on a remote file system. By default, files are searched for locally. Users can search on remote filesystems by passing an instance of an implementation of fsspec.spec.AbstractFileSystem (strictly speaking, any object of a class implementing a glob method works).

If locating files on a local file system, the returned dictionary can be passed directly to the Scene object through the filenames keyword argument. If it points to a remote file system, it is the responsibility of the user to download the files first (directly reading from cloud storage is not currently available in Satpy).

The behaviour of time-based filtering depends on whether or not the filename contains information about the end time of the data or not:

  • if the end time is not present in the filename, the start time of the filename is used and has to fall between (inclusive) the requested start and end times

  • otherwise, the timespan of the filename has to overlap the requested timespan

Example usage for querying a s3 filesystem using the s3fs module:

>>> import s3fs, satpy.readers, datetime
>>> satpy.readers.find_files_and_readers(
...     base_dir="s3://noaa-goes16/ABI-L1b-RadF/2019/321/14/",
...     fs=s3fs.S3FileSystem(anon=True),
...     reader="abi_l1b",
...     start_time=datetime.datetime(2019, 11, 17, 14, 40))
{'abi_l1b': [...]}
Parameters

  • start_time (datetime) – Limit used files by starting time.

  • end_time (datetime) – Limit used files by ending time.

  • base_dir (str) – The directory to search for files containing the data to load. Defaults to the current directory.

  • reader (str or list) – The name of the reader to use for loading the data or a list of names.

  • sensor (str or list) – Limit used files by provided sensors.

  • filter_parameters (dict) – Filename pattern metadata to filter on. start_time and end_time are automatically added to this dictionary. Shortcut for reader_kwargs[‘filter_parameters’].

  • reader_kwargs (dict) – Keyword arguments to pass to specific reader instances to further configure file searching.

  • missing_ok (bool) – If False (default), raise ValueError if no files are found. If True, return empty dictionary if no files are found.

  • fs (fsspec.spec.AbstractFileSystem) – Optional, instance of implementation of fsspec.spec.AbstractFileSystem (strictly speaking, any object of a class implementing .glob is enough). Defaults to searching the local filesystem.

Returns

Dictionary mapping reader name string to list of filenames

Return type

dict

satpy.readers.get_valid_reader_names(reader)[source]

Check for old reader names or readers pending deprecation.

satpy.readers.group_files(files_to_sort, reader=None, time_threshold=10, group_keys=None, reader_kwargs=None, missing='pass')[source]

Group series of files by file pattern information.

By default this will group files by their filename start_time assuming it exists in the pattern. By passing the individual dictionaries returned by this function to the Scene classes’ filenames, a series Scene objects can be easily created.

Parameters

  • files_to_sort (iterable) – File paths to sort in to group

  • reader (str or Collection[str]) – Reader or readers whose file patterns should be used to sort files. If not given, try all readers (slow, adding a list of readers is strongly recommended).

  • time_threshold (int) – Number of seconds used to consider time elements in a group as being equal. For example, if the ‘start_time’ item is used to group files then any time within time_threshold seconds of the first file’s ‘start_time’ will be seen as occurring at the same time.

  • group_keys (list or tuple) – File pattern information to use to group files. Keys are sorted in order and only the first key is used when comparing datetime elements with time_threshold (see above). This means it is recommended that datetime values should only come from the first key in group_keys. Otherwise, there is a good chance that files will not be grouped properly (datetimes being barely unequal). Defaults to a reader’s group_keys configuration (set in YAML), otherwise ('start_time',). When passing multiple readers, passing group_keys is strongly recommended as the behaviour without doing so is undefined.

  • reader_kwargs (dict) – Additional keyword arguments to pass to reader creation.

  • missing (str) – Parameter to control the behavior in the scenario where multiple readers were passed, but at least one group does not have files associated with every reader. Valid values are "pass" (the default), "skip", and "raise". If set to "pass", groups are passed as-is. Some groups may have zero files for some readers. If set to "skip", groups for which one or more readers have zero files are skipped (meaning that some files may not be associated to any group). If set to "raise", raise a FileNotFoundError in case there are any groups for which one or more readers have no files associated.

Returns

List of dictionaries mapping ‘reader’ to a list of filenames. Each of these dictionaries can be passed as filenames to a Scene object.

satpy.readers.load_reader(reader_configs, **reader_kwargs)[source]

Import and setup the reader from reader_info.

satpy.readers.load_readers(filenames=None, reader=None, reader_kwargs=None)[source]

Create specified readers and assign files to them.

Parameters

  • filenames (iterable or dict) – A sequence of files that will be used to load data from. A dict object should map reader names to a list of filenames for that reader.

  • reader (str or list) – The name of the reader to use for loading the data or a list of names.

  • reader_kwargs (dict) – Keyword arguments to pass to specific reader instances. This can either be a single dictionary that will be passed to all reader instances, or a mapping of reader names to dictionaries. If the keys of reader_kwargs match exactly the list of strings in reader or the keys of filenames, each reader instance will get its own keyword arguments accordingly.

Returns: Dictionary mapping reader name to reader instance

satpy.readers.open_file_or_filename(unknown_file_thing)[source]

Try to open the unknown_file_thing, otherwise return the filename.

satpy.readers.read_reader_config(config_files, loader=<class 'yaml.loader.UnsafeLoader'>)[source]

Read the reader config_files and return the extracted reader metadata.