Hi,
I couldn’t think of a good title for this question, and I am not sure if these are more related to Python or PyTorch, but here goes:
-
Why do we use with torch.no_grad(): and not just torch.no_grad():. What does the ‘with’ achieve?
-
Without giving away the answer, at one point we are asked to calculate the value of a variable using the function order variable.sum().item(). What exactly is happening here ? If anything I’d imagine instead it would be variable.sum(item()) or something similar. I’m not seeing how these chain along.
1 Like
Thanks @balaji.ambresh, but for 2) I guess I am asking how does the .sum() know what it is it is ‘summing’ since it is outside the brackets of the function (?)
@Nevermnd
variable.sum(): This part calculates the sum of all elements within the variable tensor. Depending on the tensor’s dimensions, this could sum across all dimensions, or specific dimensions if arguments are provided to the sum() function (e.g., variable.sum(dim=0)).
.item(): This method is then called on the resulting sum. The .item() method is used to extract the single numerical value from a tensor containing only one element. This is particularly useful when you need to convert a single-element tensor back into a standard Python number (e.g., a float or integer) for further calculations or printing.
variable.sum().item() calculates the total sum of all values within a tensor and then extracts that single sum as a standard Python number.
Try this 
import torch
# Create a tensor
my_tensor = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
# Calculate the sum and extract it as a Python number
total_sum = my_tensor.sum().item()
print(f"Original tensor: \n{my_tensor}")
print(f"Sum of tensor elements (as Python number): {total_sum}")
1 Like
Oh okay, thanks, now the order makes sense.
2 Likes
The joys of OOP. 
Just for giggles, let’s try calling the item() method on a tensor with more than one element and see what happens:
# Create a tensor
my_tensor = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
# Calculate the sum and extract it as a Python number
total_sum = my_tensor.sum().item()
print(f"Original tensor: \n{my_tensor}")
print(f"Type of sum tensor: {type(my_tensor.sum())}")
print(f"Sum of tensor elements (as Python number): {total_sum}")
print(f"Try item on original tensor: {my_tensor.item()}")
Which gives:
Original tensor:
tensor([[1., 2.],
[3., 4.]])
Type of sum tensor: <class 'torch.Tensor'>
Sum of tensor elements (as Python number): 10.0
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
Cell In[6], line 11
8 print(f"Type of sum tensor: {type(my_tensor.sum())}")
9 print(f"Sum of tensor elements (as Python number): {total_sum}")
---> 11 print(f"Try item on original tensor: {my_tensor.item()}")
RuntimeError: a Tensor with 4 elements cannot be converted to Scalar
3 Likes
I am a new user, seems I don’t have permission to start new question. I have a question about assignment 2: How can I turn my own handwriting image into a pkl file, so that I can test the model myself? Thanks.
hi @xzhang14
You need to create a new topic always to avoid mixing two topics here, you can always share a previous topic link to your topic if it’s similar or follow up to your topic.
Now to your question, just to be clear are you asking more about how to convert your image file folder into pkl file??
More experiments! One might think that an alternative would be to extract the element of the resultant sum tensor by using indexing. But it turns out that doesn’t work:
# Create a tensor
my_tensor = torch.tensor([[1.0, 2.0], [3.0, 4.0]])
# Calculate the sum and extract it as a Python number
total_sum = my_tensor.sum().item()
print(f"Original tensor: \n{my_tensor}")
print(f"Type of sum tensor: {type(my_tensor.sum())}")
print(f"Shape of sum tensor: {my_tensor.sum().shape}")
print(f"Sum of tensor elements (as Python number): {total_sum}")
total_sum_element = my_tensor.sum()[0]
print(f"Type of total_sum_element: {type(total_sum_element)}")
Which gives:
Original tensor:
tensor([[1., 2.],
[3., 4.]])
Type of sum tensor: <class 'torch.Tensor'>
Shape of sum tensor: torch.Size([])
Sum of tensor elements (as Python number): 10.0
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[7], line 12
9 print(f"Shape of sum tensor: {my_tensor.sum().shape}")
10 print(f"Sum of tensor elements (as Python number): {total_sum}")
---> 12 total_sum_element = my_tensor.sum()[0]
13 print(f"Type of total_sum_element: {type(total_sum_element)}")
IndexError: invalid index of a 0-dim tensor. Use `tensor.item()` in Python or `tensor.item<T>()` in C++ to convert a 0-dim tensor to a number
So it looks like we are stuck with using the item() method. Maybe we could figure out how to use keepdim to get a sum as a resulting 1D tensor and then do the indexing, but it’s not worth the hassle.
1 Like
Thanks for the reply, yes, I am wondering how to apply the trained model in assignments two to read my own handwriting image.
Thanks,
Xiaofeng
1 Like
hi @paulinpaloalto
The .sum().item() was used to the correct predictions to get correct predictions batch wise, so they wanted it to be a python number (a float or integer), in order to further calculate down total predictions and probably that’s the reason they chose this way.
Also torch.sum actually do sum up 1D tensor using the dim=0 as you just mentioned here.
Regards
DP
@xzhang14
can you create a separate new topic, don’t want to mix up or confuse the topic creator’s query. This is an ideal way community learning practiced, just in case future learners when visit this topic they don’t get confused. I will be happy to respond on your created topic and thankful for your wise understanding.
regards
DP
Thanks, just created a new topic:
1 Like
Just created a new topic: C1M2_Assignment: How to test the model with my own handwriting image.
Thanks,
Xiaofeng
1 Like