dset | A tiny utility for safely writing deep Object | GraphQL library

You can access the data and labels of the dataset, for either split, using the data and targets attributes respectively. So, for example, here you can access the training data and labels using mnist_train.data and mnist_train.labels respectively.

Since the targets attribute is a torch.Tensor for this dataset, you can count the number of instances of each target by using torch.bincount. Since there are 10 classes in total, the output will be a tensor of length 10, where the i^th index specifies the number of data points of class i.

Example:

When working with compound datasets, I've discovered it's best to add all row data in a single statement. I tweaked your code and to show how add 3 rows of data (each of different length). Note how I: 1) define the row of data with a tuple; 2) define the list of integers with np.array(); and 3) don't reference the field name ['a'].

I found the solution. After extending the dataset dimensions with H5Dset_extent(dset, dims); inside the else branch, one must execute file_space = H5Dget_space(dset);. The complete working solution is below. The function opens the file and if there is no dataset it writes the first buffer. If a dataset exists, it extends it and then writes the next buffer.

Use a list comprehension to create a list constructed from the desired element of the CValue dicts, then return the list.

Your indexing is wrong. step, i goes like this:

Will this work for you?

I use this extension method to convert IEnumerable to Datatables

just do plt.show() at the end of your code if you are running in a ipython notebook, it should work

Here is a simple example that pulls my suggestions together to show how everything might work in your case. Program flow summary:

1. Open a new file, create dataset 'embeds' with shape=(130, 8, 512, 768), then add 2 sets of data, write the 'last_index' attribute then close the file.
2. Re-Open the file in APPEND mode, accesses dataset 'embeds', add more 2 sets of data (starting at 'last_index'), write the 'last_index' attribute and close the file.
3. The last file open is in READ mode to print dataset attribute and shape parameters.

Notes:

• I use HDFView to visually verify dataset contents. I discovered a problem viewing np.float16, so I used np.float32. This should work with np.float16. I will let you verify that.
• You should also add standard integrity checks and error handling. For example: 1) that 'embeds' dataset and the 'last_index' attribute both exist, 2) check the size of the dataset size to confirm data will fit, and 3) resize if your new data goes beyond the current bounds.

CODE BELOW:

Here's a trivial example of why your final dataset doesn't automatically set the sorted flag.