|
| | DataLoaderBase (DataLoaderOptions options, std::unique_ptr< Dataset > main_thread_dataset=nullptr) |
| | Constructs a new DataLoader from a dataset to sample from, options to configure the DataLoader with, and a sampler that specifies the sampling strategy. More...
|
| |
| Iterator< Batch > | begin () |
| | Returns an iterator into the DataLoader. More...
|
| |
| Iterator< Batch > | end () |
| | Returns a special "sentinel" iterator that compares equal with a non-sentinel iterator once the DataLoader is exhausted. More...
|
| |
| void | join () |
| | Joins the DataLoader's worker threads and drains internal queues. More...
|
| |
|
const FullDataLoaderOptions & | options () const noexcept |
| | Returns the options with which the DataLoader was configured.
|
| |
|
| virtual optional< BatchRequestType > | get_batch_request ()=0 |
| | Subclass hook for getting the next batch request. More...
|
| |
| virtual void | reset () |
| | Resets the internal state of the DataLoader, optionally pre-fetching new jobs. More...
|
| |
| void | prefetch (size_t requested_jobs) |
| | Schedules requested_jobs many new batches to be fetched. More...
|
| |
|
void | prefetch () |
| | Schedules the maximum number of jobs (based on the max_jobs option).
|
| |
| optional< BatchType > | next () |
| | Returns the next batch of data, or an empty optional if the DataLoader is exhausted. More...
|
| |
|
void | worker_thread (Dataset &dataset) |
| | The function that worker threads run.
|
| |
| template<typename T > |
| void | push_job (T value) |
| | Convenience method that calls shuttle_.push_job() with the next sequence number. More...
|
| |
|
optional< Result > | pop_result () |
| | Convenience method that gets the next result from the sequencer.
|
| |
| std::unique_ptr< detail::sequencers::Sequencer< Result > > | new_sequencer () |
| | Convenience method that creates a new sequencer based on the enforce_ordering option. More...
|
| |
template<typename Dataset, typename Batch, typename BatchRequest>
class torch::data::DataLoaderBase< Dataset, Batch, BatchRequest >
Definition at line 27 of file base.h.
template<typename Dataset, typename Batch, typename BatchRequest>
Returns an iterator into the DataLoader.
The lifetime of the iterator is bound to the DataLoader. In C++ standards language, the category of the iterator is OutputIterator. See https://en.cppreference.com/w/cpp/named_req/OutputIterator for what this means. In short: you may increment the iterator and dereference it, but cannot go back, or step forward more than one position at a time. When the DataLoader is exhausted, it will compare equal with the special "sentinel" iterator returned by DataLoader::end(). Most of the time, you should only use range-for loops to loop over the DataLoader, but standard algorithms like std::copy(dataloader.begin(), dataloader.end(), output_iterator) are supported too.
Definition at line 57 of file base.h.
template<typename Dataset, typename Batch, typename BatchRequest>
Joins the DataLoader's worker threads and drains internal queues.
This function may only be invoked from the main thread (in which the DataLoader lives).
Definition at line 77 of file base.h.
template<typename Dataset, typename Batch, typename BatchRequest>
Returns the next batch of data, or an empty optional if the DataLoader is exhausted.
This operation will block until a batch is available if one is still expected.
Definition at line 165 of file base.h.
template<typename Dataset, typename Batch, typename BatchRequest>
Schedules requested_jobs many new batches to be fetched.
The actual number of jobs scheduled may be less if the DataLoader exhausts.
Definition at line 147 of file base.h.
template<typename Dataset, typename Batch, typename BatchRequest>
The dataset for the main thread, only has a value if the number of worker threads was configured as zero, meaning the main thread has to do all the work (synchronously).
NOTE: Really want this to be on the heap when empty, therefore unique_ptr and not optional.
Definition at line 227 of file base.h.
1.8.11
