Hi,
I would like to use the trained model in the assignment to test my own handwriting image.
The assignment given an existing pkl file. How would I create such kind of pkl file from an image?
Thanks,
Xiaofeng
you can use the HDF5 (.h5) file format to store your own handwritten image data and train a machine learning model
thank you for creating new topic!!!
hi @xzhang14
here are the basic steps
1.Collect and Preprocess Images - Assemble your handwritten images and ensure they are a consistent size (e.g., 28x28 pixels, similar to the MNIST dataset) and format (e.g., grayscale)
2.Convert to HDF5 Format - Use a library like h5py in Python to convert your individual images and their corresponding labels into a single HDF5 file. You will create datasets within the file, typically one for the images and another for the labels, organized into groups (e.g., train/images, train/labels, test/images, test/labels).
For this step, you can use tfjs converter, I am sharing the GitHub link for this
[Tfjs converter GitHub]((tfjs-converter/tfjs-converter/README.md at master · tensorflow/tfjs-converter · GitHub)
3..Load Data for Training - When training your model using a framework like TensorFlow, Keras, or PyTorch, you can use the HDF5 file to load data in minibatches (data slicing), which is memory-efficient for large datasets.
4.Train the Model - Feed the image batches to your chosen model architecture (likely a Convolutional Neural Network or CNN, which is ideal for image classification).
5.Save the Trained Model (not necessary but recommended) - The final trained model’s architecture and optimized weights can also be saved in an HDF5 file, which is a common practice for future reuse and deployment.
Hope this helps!!!
Regards
DP
Ok, I thought there is a way just take a picture of a handwriting note paper, then create a pkl file from it. (Because the assignment using a pkl file contains the full handwriting note as an example). From your reply, seems I have to cut each letter into a 28x28 image, then convert all images into certain format and then feed into the model?
you don’t need to cut the paper in 28x28 size 
just after you take the images, make sure the image size are in size of 28x28 (this actually is discussed in module 3 by Lawrence in the course, beautifully explained with flowers 
the first image shows if the images are resize drastically can distort the image quality
Second image, first crops the flower and resize the image resulting in better image quality as well as the size one is looking for
Believe it is not much of a steps, I don’t know if you have taken tensorflow data and deployment specialisation, where we use our own camera to take rock, paper, scissors hand images and then train on the model, it was fun assignment and it surely checked my patience because even after my codes were correct, it wasn’t taking my images and it turned out there was web console issue, one of the learner in this community helped me get through that assignment (long back incidence). You should try.
Hi, I was referring assignment 2, the EMNIST data model to recognize the hand writing notes. There is a file called “hidden_message_images.pkl”, and it contains the hand writing note decoded by the model like “
deqr iaurence
hofe the pytorch course is going weli
do not forget to keep the iabs interesting and engaqing
maybe couid have the students try to decode my messy handwriting
that might be a dit tdo challengihg though
i am impressed you are abie to read this”
I am wondering how did this “hidden_message_images.pkl“ file get generated?
Thanks,
Xiaofeng
1 Like
in the jupyter notebook??
Hello @xzhang14,
pkl is commonly used as an extension for “pickle file” generated with Python’s pickle library. Since you knew it read the file “hidden_message_images.pkl”, you probably had checked out the load_hidden_message_images() function inside the “helper_utils.py” and seen import pickle there.
Pickle file lets you serialize Python object and store it in a file. The object in “hidden_message_images.pkl” is a list of list of list of tensors of shape (1, 28, 28).
To create another pickle file of the same behavior, you need to prepare another python object that’s a list of list of list of tensors of shape (1, 28, 28) which we call it another_object, and then run the code below:
import pickle
with open('./a_new_pkl_file.pkl', 'wb') as f:
pickle.dump(another_object, f)
As a quick try, you may just save a new pickle file for the object in “hidden_message_images.pkl” and then read it back again.
Cheers,
Raymond
To prepare for the Python object I mentioned above, an alternative and I think the easiest way would be to find some EMNIST datasets that contain letters, read them all, and pack the letters in the sequence of the letters in the passage you have.
To make the result by the alternative a bit more natural, unless you don’t care, make sure the letters are upright, because some EMNIST datasets have letters rotated.
Another alternative would be to generate new letter image from handwriting-like Font.
With these images of the two alternatives, you may slightly preprocess them with like small rotation (making it like italic) and resize (people don’t always write each letter in the same size), so that you have more synthesized versions of the letters.
Thanks for the reply. My original thought is this could be a prototype of OCR : people hand you a handwritten document, you use the model read out the handwritten information. Your solutions are far from this real situation.
Xiaofeng
No, that was not meant to be like a OCR, only to make a pickle file. I think I have missed your keyword of “my own handwriting image”.
Thanks for your helpful information anyway.
Xiaofeng