torch::data::StatefulDataLoader< Dataset > Class Template Reference

A dataloader for stateful datasets. More...

#include <stateful.h>

Inheritance diagram for torch::data::StatefulDataLoader< Dataset >:

Public Types
using	super = DataLoaderBase< Dataset, typename Dataset::BatchType::value_type, typename Dataset::BatchRequestType >
using	BatchRequestType = BatchRequest
Public Types inherited from torch::data::DataLoaderBase< Dataset, Dataset::BatchType::value_type, Dataset::BatchRequestType >
using	BatchType = Dataset::BatchType::value_type
using	BatchRequestType = Dataset::BatchRequestType

Public Member Functions
	StatefulDataLoader (Dataset dataset, DataLoaderOptions options)
	Constructs the StatefulDataLoader from a dataset and some options.
Public Member Functions inherited from torch::data::DataLoaderBase< Dataset, Dataset::BatchType::value_type, Dataset::BatchRequestType >
	DataLoaderBase (DataLoaderOptions options, std::unique_ptr< Dataset > main_thread_dataset=nullptr)
	Constructs a new DataLoader from a dataset to sample from, options to configure the DataLoader with, and a sampler that specifies the sampling strategy. More...
Iterator< Dataset::BatchType::value_type >	begin ()
	Returns an iterator into the DataLoader. More...
Iterator< Dataset::BatchType::value_type >	end ()
	Returns a special "sentinel" iterator that compares equal with a non-sentinel iterator once the DataLoader is exhausted. More...
void	join ()
	Joins the DataLoader's worker threads and drains internal queues. More...
const FullDataLoaderOptions &	options () const noexcept
	Returns the options with which the DataLoader was configured.

Additional Inherited Members
Protected Member Functions inherited from torch::data::DataLoaderBase< Dataset, Dataset::BatchType::value_type, Dataset::BatchRequestType >
void	prefetch (size_t requested_jobs)
	Schedules requested_jobs many new batches to be fetched. More...
void	prefetch ()
	Schedules the maximum number of jobs (based on the max_jobs option).
optional< BatchType >	next ()
	Returns the next batch of data, or an empty optional if the DataLoader is exhausted. More...
void	worker_thread (Dataset &dataset)
	The function that worker threads run.
void	push_job (T value)
	Convenience method that calls shuttle_.push_job() with the next sequence number. More...
optional< Result >	pop_result ()
	Convenience method that gets the next result from the sequencer.
std::unique_ptr< detail::sequencers::Sequencer< Result > >	new_sequencer ()
	Convenience method that creates a new sequencer based on the enforce_ordering option. More...
Protected Attributes inherited from torch::data::DataLoaderBase< Dataset, Dataset::BatchType::value_type, Dataset::BatchRequestType >
const FullDataLoaderOptions	options_
	The options the DataLoader was configured with.
std::unique_ptr< Dataset >	main_thread_dataset_
	The dataset for the main thread, only has a value if the number of worker threads was configured as zero, meaning the main thread has to do all the work (synchronously). More...
size_t	sequence_number_
	The sequence number for the next batch to be retrieved from the dataset. More...
std::vector< std::thread >	workers_
	The worker threads, running the worker_thread() method.
detail::DataShuttle< Job, Result >	shuttle_
	The DataShuttle which takes care of the life cycle of a job.
std::unique_ptr< detail::sequencers::Sequencer< Result > >	sequencer_
	The Sequencer, which handles optional ordering of batches.
bool	joined_
	True if the DataLoader has joined its worker threads.

Detailed Description

template<typename Dataset>
class torch::data::StatefulDataLoader< Dataset >

A dataloader for stateful datasets.

A dataloader for stateful datatasets differs from one for stateless datasets one in that the dataset is shared among worker threads, and that this dataset is itself responsible for producing batches rather than depending on a sampler. The statefulness here actually refers to the dataset. The StatefulDataLoader simply alters the data loading algorithm to accomodate the stateful, shared nature of the dataset. Note that the dataset must be thread safe if more than one worker thread is used.

A stateful dataloader is created by calling make_data_loader with a stateful dataset.

Definition at line 25 of file stateful.h.

---

The documentation for this class was generated from the following file:

• torch/csrc/api/include/torch/data/dataloader/stateful.h