Reference/API#

astropy.io.misc Package#

This package contains miscellaneous utility functions for data input/output with astropy.

Functions#

fnpickle(object, fileorname[, protocol, append])

fnunpickle(fileorname[, number])

astropy.io.misc.ecsv Module#

ECSV Engine Module#

This module provides functionality for reading and writing Enhanced Character Separated Values (ECSV) files using various backends, including: - PyArrow - pandas - Astropy’s ASCII engine

ECSV is a human-readable, YAML-encoded table format used in the Astropy ecosystem for storing tables with metadata, units, and complex data types.

Key Features#

  • Defines data structures for representing ECSV column and header metadata.

  • Implements multiple ECSV reader engines, supporting PyArrow, pandas, and Astropy’s ASCII CSV readers.

  • Handles conversion between ECSV datatypes and numpy/pandas/pyarrow types, including support for JSON-encoded columns, multidimensional arrays, and masked data.

  • Provides robust parsing of ECSV headers and data, including support for compressed files and in-memory file-like objects.

  • Ensures compatibility with legacy ECSV files and provides liberal error handling for unknown datatypes.

  • Integrates with Astropy’s Unified I/O registry for seamless reading and writing of ECSV files.

Main Classes and Functions#

  • ColumnECSV: Represents the attributes of a column as described in the ECSV header.

  • ECSVHeader: Encapsulates the parsed ECSV header, including column definitions and table metadata.

  • ECSVEngine and subclasses: Abstract base class and concrete implementations for different CSV parsing engines.

  • read_ecsv: Reads an ECSV file and returns an Astropy Table object, handling all necessary conversions and metadata.

  • write_ecsv: Writes an Astropy Table to an ECSV file.

  • register_pyarrow_ecsv_table: Registers the PyArrow ECSV reader/writer with Astropy’s I/O registry.

Usage#

This module is intended for internal use within Astropy and for advanced users who need fine-grained control over ECSV parsing and engine selection. For most users, reading and writing ECSV files can be accomplished via the high-level Table.read and Table.write interfaces, e.g.: ` Table.read(filename, format="ecsv", engine="pyarrow.csv") `

Dependencies#

  • numpy

  • astropy.table

  • pyarrow (optional, for PyArrow engine)

  • pandas (optional, for pandas engine)

Functions#

read_ecsv(input_file, *[, encoding, engine, ...])

Read an ECSV (Enhanced Character Separated Values) file and return an Astropy Table.

register_ecsv_table()

Register ECSV reader and writer with Unified I/O as a Table reader.

write_ecsv(tbl, output[, engine])

Thin wrapper around the io.ascii ECSV writer to write ECSV files.

Classes#

ColumnECSV(name, datatype[, subtype, unit, ...])

Class representing attributes of a column in an ECSV header.

ECSVEngine()

Base class for ECSV reader engines.

ECSVEngineIoAscii()

ECSV reader engine using astropy.io.ascii Python CSV reader.

ECSVEnginePandas()

ECSV reader engine using pandas.

ECSVEnginePyArrow()

ECSV reader engine using PyArrow.

ECSVHeader(n_header, n_empty, n_comment, ...)

Class representing the information in an ECSV header.

Class Inheritance Diagram#

Inheritance diagram of astropy.io.misc.ecsv.ColumnECSV, astropy.io.misc.ecsv.ECSVEngine, astropy.io.misc.ecsv.ECSVEngineIoAscii, astropy.io.misc.ecsv.ECSVEnginePandas, astropy.io.misc.ecsv.ECSVEnginePyArrow, astropy.io.misc.ecsv.ECSVHeader

astropy.io.misc.hdf5 Module#

This package contains functions for reading and writing HDF5 tables that are not meant to be used directly, but instead are available as readers/writers in astropy.table. See High-level Unified File I/O for more details.

Functions#

read_table_hdf5(input[, path, ...])

Read a Table object from an HDF5 file.

write_table_hdf5(table, output[, path, ...])

Write a Table object to an HDF5 file.

astropy.io.misc.parquet Module#

This package contains functions for reading and writing Parquet tables that are not meant to be used directly, but instead are available as readers/writers in astropy.table. See High-level Unified File I/O for more details.

astropy.io.misc.pyarrow.csv Module#

This module provides functionality to read CSV files into Astropy Tables using PyArrow.

Functions#

convert_pa_table_to_astropy_table(table_pa)

Convert a PyArrow Table to an Astropy Table.

read_csv(input_file, *[, delimiter, ...])

Read a CSV file into an astropy Table using PyArrow.

astropy.io.misc.yaml Module#

Functions for serializing astropy objects to YAML.

It provides functions dump, load, and load_all which call the corresponding functions in PyYaml but use the AstropyDumper and AstropyLoader classes to define custom YAML tags for the following astropy classes: - astropy.units.Unit - astropy.units.Quantity - astropy.time.Time - astropy.time.TimeDelta - astropy.coordinates.SkyCoord - astropy.coordinates.Angle - astropy.coordinates.Latitude - astropy.coordinates.Longitude - astropy.coordinates.EarthLocation - astropy.table.SerializedColumn

Examples#

>>> from astropy.io.misc import yaml
>>> import astropy.units as u
>>> from astropy.time import Time
>>> from astropy.coordinates import EarthLocation
>>> t = Time(2457389.0, format='mjd',
...          location=EarthLocation(1000, 2000, 3000, unit=u.km))
>>> td = yaml.dump(t)
>>> print(td)
!astropy.time.Time
format: mjd
in_subfmt: '*'
jd1: 4857390.0
jd2: -0.5
location: !astropy.coordinates.earth.EarthLocation
ellipsoid: WGS84
x: !astropy.units.Quantity
    unit: &id001 !astropy.units.Unit {unit: km}
    value: 1000.0
y: !astropy.units.Quantity
    unit: *id001
    value: 2000.0
z: !astropy.units.Quantity
    unit: *id001
    value: 3000.0
out_subfmt: '*'
precision: 3
scale: utc
>>> ty = yaml.load(td)
>>> ty
<Time object: scale='utc' format='mjd' value=2457389.0>
>>> ty.location
<EarthLocation (1000., 2000., 3000.) km>

Functions#

dump(data[, stream])

Serialize a Python object into a YAML stream using the AstropyDumper class.

load(stream)

Parse the first YAML document in a stream using the AstropyLoader and produce the corresponding Python object.

load_all(stream)

Parse the all YAML documents in a stream using the AstropyLoader class and produce the corresponding Python object.

Classes#

AstropyDumper(stream[, default_style, ...])

Custom SafeDumper that represents astropy core objects as well as Python tuple and unicode objects.

AstropyLoader(stream)

Custom SafeLoader that constructs astropy core objects as well as Python tuple and unicode objects.

Class Inheritance Diagram#

Inheritance diagram of astropy.io.misc.yaml.AstropyDumper, astropy.io.misc.yaml.AstropyLoader