|
H5CPP
v1.14.0
Modern C++ templates for HDF5 serial and parallel I/O
|
|
Every type h5cpp's h5::read / h5::write / h5::create / h5::aread / h5::awrite accepts, organised by category. The storage representation column maps each entry to one of the storage_representation_t cases declared at h5cpp/H5Tmeta.hpp:164-170 — that's the value storage_representation_v<T> resolves to at compile time, and what the static-assert stoppers in H5Dwrite.hpp / H5Dread.hpp / H5Awrite.hpp / H5Aread.hpp check before dispatching.
The underlying classification mechanism is the Walter Brown detection idiom: types are recognised by their structural surface (value_type, iterator, size, data, key_type, mapped_type, …), not by name. A custom container with the right shape routes the same way as the matching std:: counterpart — see Supported container shapes in the Cook book for the canonical walkthrough.
All landed at storage_representation_t::scalar. Default-included via h5cpp/all (no extra mapper header required).
| Type | HDF5 representation | Notes |
|---|---|---|
int8_t / int16_t / int32_t / int64_t | H5T_NATIVE_INT8 … H5T_NATIVE_INT64 | signed integer |
uint8_t / uint16_t / uint32_t / uint64_t | H5T_NATIVE_UINT8 … H5T_NATIVE_UINT64 | unsigned integer |
float | H5T_NATIVE_FLOAT | IEEE 754 single precision |
double | H5T_NATIVE_DOUBLE | IEEE 754 double precision |
long double | H5T_NATIVE_LDOUBLE | platform-specific extended precision |
bool | H5T_NATIVE_HBOOL | HDF5 boolean (NB: std::vector<bool> is NOT supported, see NOT supported (intentional stoppers)) |
enum E (registered) | H5T_ENUM over the underlying integer | register with H5CPP_REGISTER_DATATYPE(E, "E", H5Tenum_create(H5T_NATIVE_*), { H5Tenum_insert(...) }) |
Custom datatypes (n-bit packing, opaque blobs, strong-typedef wrappers) register through H5CPP_REGISTER_DATATYPE — see the macro documentation at h5cpp/H5Tall.hpp:514 and the examples/datatypes/ cookbook page.
h5cpp routes both fixed-length (H5T_C_S1 + H5Tset_size(N)) and variable-length (H5T_VARIABLE) HDF5 string forms depending on the C++ type used at the call site.
| Type | storage_representation_t | HDF5 representation |
|---|---|---|
std::string / std::string_view | vlen_text_dataset | H5T_VARIABLE — one char* allocated by HDF5 on read |
char* / const char* | vlen_text_dataset | same variable-length path |
char[N] | fixed_length_string | H5T_C_S1 + H5Tset_size(N), scalar dataspace |
std::array<char,N> | fixed_length_string | same as char[N] — wrapper-equivalent representation |
std::vector<std::string> | vlen_text_dataset | rank-1 dataset, each element variable-length |
std::vector<std::array<char,N>> | fls_dataset | rank-1 dataset of fixed-length strings (no VLEN overhead) |
On read, fixed-length string attributes are detected via H5Aget_type + H5Tis_variable_str — HDF5 has no fixed↔VLEN conversion path, so the dispatch must branch (see h5cpp/H5Aread.hpp and examples/string/ in the cookbook).
| Type | storage_representation_t | How it's registered |
|---|---|---|
POD aggregate registered via H5CPP_REGISTER_STRUCT(MyStruct); | scalar (with H5T_COMPOUND type) | macro at file scope; emits dt_t<MyStruct> specialisation. See examples/compound/. |
Compiler-reflected tier-2 type with [[h5::*]] attributes | scalar | h5cpp-compiler tool emits generated.h; user #includes it. Handles heap-indirection fields (vlen strings, vlen vectors) via the scatter/gather visitor path. See examples/multi-tu/ and examples/reflection/. |
std::tuple<Ts...> | scalar | Packed into an internal C-struct mirror, compound with fields _0, _1, …; field count = tuple arity. |
std::pair<K,V> | scalar | Compound with fields first, second. |
std::complex<float\|double\|long double> | scalar | Native H5T_COMPLEX on HDF5 ≥ 1.14, compound fallback (r, i) otherwise. |
| Type | storage_representation_t | HDF5 representation |
|---|---|---|
T[N] (top-level) | array_element | scalar dataspace + H5T_ARRAY[N] over dt_t<T> |
std::array<T,N> where T is non-char | array_element | same as T[N] — wrapper-equivalent |
T[N][M] and higher | array_element | scalar dataspace + nested H5T_ARRAY |
| Type | storage_representation_t | Write path |
|---|---|---|
std::vector<T> | linear_value_dataset | Direct H5Dwrite (contiguous, has data()) |
std::deque<T> | linear_value_dataset | Staged into contiguous buffer, then H5Dwrite |
std::list<T> | linear_value_dataset | Staged into contiguous buffer, then H5Dwrite |
std::forward_list<T> | linear_value_dataset | Append-style; pair with h5::pt_t extensible dataset |
std::valarray<T> | linear_value_dataset | Via h5cpp/H5Mvalarray.hpp |
| Type | storage_representation_t | HDF5 representation |
|---|---|---|
std::set<T> / std::multiset<T> | linear_value_dataset | Rank-1 of T, written in iteration order |
std::unordered_set<T> / std::unordered_multiset<T> | linear_value_dataset | Same, but iteration order is hash-dependent |
std::map<K,V> / std::multimap<K,V> | key_value_dataset | Rank-1 of compound { K key, V value } |
std::unordered_map<K,V> / std::unordered_multimap<K,V> | key_value_dataset | Same compound layout, hash-iteration order |
Set-like and map-like third-party containers (Abseil flat_hash_*, TSL robin_*, Folly F14*, Boost flat_*, parallel-hashmap) ride the structural fallback and route the same way — see Supported container shapes for the full third-party coverage matrix.
| Type | storage_representation_t | HDF5 representation |
|---|---|---|
std::vector<std::array<T,N>> (T non-char) | array_dataset | Rank-1 of H5T_ARRAY[N] elements — flat rows × N |
std::vector<std::array<char,N>> | fls_dataset | Rank-1 of fixed-length strings — vlen-free path |
std::vector<std::pair<K,V>> | linear_value_dataset | Rank-1 of compound { first, second } |
std::vector<std::tuple<Ts...>> | linear_value_dataset | Rank-1 of packed compound |
| Type | storage_representation_t | HDF5 representation |
|---|---|---|
std::vector<std::string> | vlen_text_dataset | char* relay array + H5T_VARIABLE |
std::vector<std::vector<T>> | ragged_vlen_dataset | hvl_t relay array + H5Tvlen_create |
All dense linalg containers map to linear_value_dataset (or array_dataset for the rank-aware rank-2 / rank-3 cases) and write directly via the underlying contiguous buffer the linalg library exposes. Include the matching mapper header before use.
| Library | Types supported | Mapper header |
|---|---|---|
| Armadillo | arma::Col<T>, arma::Row<T>, arma::Mat<T>, arma::Cube<T> | h5cpp/H5Marma.hpp |
| Eigen | Eigen::Matrix<T,…>, Eigen::Vector<T,…>, Eigen::Array<T,…> (ColMajor) | h5cpp/H5Meigen.hpp |
| Blitz++ | blitz::Array<T,N> | h5cpp/H5Mblitz.hpp |
| Blaze | blaze::DynamicVector<T>, blaze::DynamicMatrix<T> | h5cpp/H5Mblaze.hpp |
| dlib | dlib::matrix<T,…> | h5cpp/H5Mdlib.hpp |
| IT++ | itpp::Vec<T>, itpp::Mat<T> | h5cpp/H5Mitpp.hpp |
| Boost uBLAS | boost::numeric::ublas::vector<T>, boost::numeric::ublas::matrix<T> | h5cpp/H5Mublas.hpp |
| xtensor | xt::xarray<T>, xt::xtensor<T,N> | h5cpp/H5Mxtensor.hpp |
See Linear-Algebra Containers for end-to-end examples per library.
Sparse matrices and vectors land at a custom storage layout: HDF5 group containing four datasets in canonical Compressed Sparse Column form (indptr / indices / data / shape attribute). h5::write(...) returns h5::gr_t (not h5::ds_t); h5::read<T> of a sparse type reads the four datasets and reconstructs the matrix.
| Library | Types supported | Layout |
|---|---|---|
| Armadillo | arma::SpMat<T>, arma::SpCol<T>, arma::SpRow<T> | CSC group |
| Eigen | Eigen::SparseMatrix<T>, Eigen::SparseVector<T> (ColMajor only; RowMajor static_asserts) | CSC group |
Files are interoperable with scipy, Julia HDF5.jl, h5sparse, Loompy, and 10x Genomics. See examples/sparse/ in the cookbook.
std::mdspan is a non-owning view over caller-owned storage. Round-trips through linear_value_dataset (rank-1) or array_element (rank-N) depending on the extents. Gated on __cpp_lib_mdspan >= 202207L — requires libstdc++ ≥ 15 or libc++ ≥ 19; on older toolchains the mdspan example skips gracefully and the type isn't recognised.
Write is straightforward (view supplies data_handle()). Read into mdspan requires the caller to own the storage:
Mapper header: h5cpp/H5Mmdspan.hpp. See examples/mdspan/ in the cookbook.
| Type | storage_representation_t | Notes |
|---|---|---|
h5::reference_t | scalar | Rule-of-five RAII wrapper around H5R_ref_t (HDF5 1.12+) or hdset_reg_ref_t (pre-1.12). One reference per call site; round-trip via h5::reference(fd, path, offset, count). See examples/reference/ and Reference handles in the catalog. |
std::vector<h5::reference_t> | linear_value_dataset | Rank-1 dataset of region/object/attribute references; each entry is RAII-managed independently. |
h5cpp accepts smart-pointer wrappers around contiguous buffers via forwarding overloads — the pointer is unwrapped and the underlying raw T* is routed through the regular h5::write / h5::read paths. Implementation: h5cpp/H5Mmemory_io.hpp.
| Type | Routes to |
|---|---|
std::unique_ptr<T[]> | raw T* with caller-provided h5::count{N} |
std::shared_ptr<T[]> | same |
Caller retains ownership; h5cpp never takes the smart pointer. See examples/smart-ptr/ in the cookbook.
The static-assert stoppers in H5Dwrite.hpp:737 and friends fire for these — the diagnostic includes the resolved storage_representation_t::unsupported and the offending type so the failure is informative.
| Type | Why not |
|---|---|
std::vector<bool> | Bit-packing specialisation; no real contiguous bool*. The standard's allocator-aware proxy reference doesn't survive HDF5's void* interface. |
| Unregistered POD aggregate | No dt_t<T> specialisation. Register via H5CPP_REGISTER_STRUCT(T) or generate via h5cpp-compiler. |
Arbitrary nesting: std::vector<std::list<T>>, std::vector<std::set<T>>, std::vector<std::map<K,V>>, … | HDF5 can represent many of these via nested VLEN / compound, but h5cpp deliberately rejects them until the recursive packer/unpacker path is explicit. |
std::array<std::string, N> / std::array<std::vector<T>, N> | Fixed-rank array of variable-length elements — would require an explicit policy decision on the inner length. |
Containers without T(std::size_t) constructor (iterator-pair-only allocation) | Read-side structural fallback needs T(size) to pre-allocate before the staged copy. |
Compile-time only. storage_representation_v<T> is evaluated during template instantiation; the call site's H5Dwrite / H5Dread overload is selected by SFINAE against the resolved value. There is no runtime polymorphism, no if constexpr chain at the call site, and no virtual dispatch.
The architectural treatment — how storage_representation_t fans out across the per-container mapper headers (H5M*.hpp), where the detection-idiom fallbacks fire, and how third-party containers route without registration — is in Architecture / Type-system architecture notes (Project Reports → Architecture).