We Keep Coding

How to do inference with fined-tuned huggingface models?

The bounty expires in 6 days. Answers to this question are eligible for a +100 reputation bounty.
George Liu wants to draw more attention to this question.
I have fine-tuned a Huggingface model using the IMDB dataset, and I was able to use the trainer to make predictions on the test set by doing trainer.predict(test_ds_encoded). However, when doing the same thing with the inference set that has a dummy label feature (all -1s instead of 0s and 1s), the trainer threw an error:
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [0,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [1,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [2,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [3,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [4,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [5,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [6,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [7,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [8,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [9,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [10,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [11,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [12,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [13,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [14,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [15,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [16,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [17,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [18,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [19,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [20,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [21,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [22,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [23,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [24,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [25,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [26,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [27,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [28,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [29,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [30,0,0] Assertion `t >= 0 && t < n_classes` failed.
/usr/local/src/pytorch/aten/src/ATen/native/cuda/Loss.cu:257: nll_loss_forward_reduce_cuda_kernel_2d: block: [0,0,0], thread: [31,0,0] Assertion `t >= 0 && t < n_classes` failed.
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
/tmp/ipykernel_23/4156768683.py in <module>
----> 1 trainer.predict(inference_ds_encoded)
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in predict(self, test_dataset, ignore_keys, metric_key_prefix)
2694 eval_loop = self.prediction_loop if self.args.use_legacy_prediction_loop else self.evaluation_loop
2695 output = eval_loop(
-> 2696 test_dataloader, description="Prediction", ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix
2697 )
2698 total_batch_size = self.args.eval_batch_size * self.args.world_size
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in evaluation_loop(self, dataloader, description, prediction_loss_only, ignore_keys, metric_key_prefix)
2819 )
2820 if logits is not None:
-> 2821 logits = self._pad_across_processes(logits)
2822 logits = self._nested_gather(logits)
2823 if self.preprocess_logits_for_metrics is not None:
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in _pad_across_processes(self, tensor, pad_index)
2953 return tensor
2954 # Gather all sizes
-> 2955 size = torch.tensor(tensor.shape, device=tensor.device)[None]
2956 sizes = self._nested_gather(size).cpu()
2957
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
I then removed the label feature: trainer.predict(inference_ds_encoded.remove_columns('label')), but still got an error:
---------------------------------------------------------------------------
RuntimeError Traceback (most recent call last)
/tmp/ipykernel_23/899960315.py in <module>
----> 1 trainer.predict(inference_ds_encoded.remove_columns('label'))
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in predict(self, test_dataset, ignore_keys, metric_key_prefix)
2694 eval_loop = self.prediction_loop if self.args.use_legacy_prediction_loop else self.evaluation_loop
2695 output = eval_loop(
-> 2696 test_dataloader, description="Prediction", ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix
2697 )
2698 total_batch_size = self.args.eval_batch_size * self.args.world_size
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in evaluation_loop(self, dataloader, description, prediction_loss_only, ignore_keys, metric_key_prefix)
2796
2797 # Prediction step
-> 2798 loss, logits, labels = self.prediction_step(model, inputs, prediction_loss_only, ignore_keys=ignore_keys)
2799 inputs_decode = inputs["input_ids"] if args.include_inputs_for_metrics else None
2800
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in prediction_step(self, model, inputs, prediction_loss_only, ignore_keys)
2999 """
3000 has_labels = all(inputs.get(k) is not None for k in self.label_names)
-> 3001 inputs = self._prepare_inputs(inputs)
3002 if ignore_keys is None:
3003 if hasattr(self.model, "config"):
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in _prepare_inputs(self, inputs)
2261 handling potential state.
2262 """
-> 2263 inputs = self._prepare_input(inputs)
2264 if len(inputs) == 0:
2265 raise ValueError(
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in _prepare_input(self, data)
2243 """
2244 if isinstance(data, Mapping):
-> 2245 return type(data)({k: self._prepare_input(v) for k, v in data.items()})
2246 elif isinstance(data, (tuple, list)):
2247 return type(data)(self._prepare_input(v) for v in data)
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in <dictcomp>(.0)
2243 """
2244 if isinstance(data, Mapping):
-> 2245 return type(data)({k: self._prepare_input(v) for k, v in data.items()})
2246 elif isinstance(data, (tuple, list)):
2247 return type(data)(self._prepare_input(v) for v in data)
/opt/conda/lib/python3.7/site-packages/transformers/trainer.py in _prepare_input(self, data)
2253 # may need special handling to match the dtypes of the model
2254 kwargs.update(dict(dtype=self.args.hf_deepspeed_config.dtype()))
-> 2255 return data.to(**kwargs)
2256 return data
2257
RuntimeError: CUDA error: device-side assert triggered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
I also tried using the trained model object to make predictions, and I got a different error:
text = ["I like the film it's really exciting!", "I hate the movie, it's so boring!!"]
encoding = tokenizer(text)
outputs = model(**encoding)
predictions = outputs.logits.argmax(-1)
Error:
AttributeError Traceback (most recent call last)
/tmp/ipykernel_23/94414684.py in <module>
1 text = ["I like the film it's really exciting!", "I hate the movie, it's so boring!!"]
2 encoding = tokenizer(text)
----> 3 outputs = model(**encoding)
4 predictions = outputs.logits.argmax(-1)
/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1108 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1109 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1110 return forward_call(*input, **kwargs)
1111 # Do not call functions when jit is used
1112 full_backward_hooks, non_full_backward_hooks = [], []
/opt/conda/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py in forward(self, input_ids, attention_mask, head_mask, inputs_embeds, labels, output_attentions, output_hidden_states, return_dict)
752 output_attentions=output_attentions,
753 output_hidden_states=output_hidden_states,
--> 754 return_dict=return_dict,
755 )
756 hidden_state = distilbert_output[0] # (bs, seq_len, dim)
/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
1108 if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
1109 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1110 return forward_call(*input, **kwargs)
1111 # Do not call functions when jit is used
1112 full_backward_hooks, non_full_backward_hooks = [], []
/opt/conda/lib/python3.7/site-packages/transformers/models/distilbert/modeling_distilbert.py in forward(self, input_ids, attention_mask, head_mask, inputs_embeds, output_attentions, output_hidden_states, return_dict)
549 raise ValueError("You cannot specify both input_ids and inputs_embeds at the same time")
550 elif input_ids is not None:
--> 551 input_shape = input_ids.size()
552 elif inputs_embeds is not None:
553 input_shape = inputs_embeds.size()[:-1]
AttributeError: 'list' object has no attribute 'size'
My code can be found on Kaggle here: https://www.kaggle.com/code/georgeliu/imdb-text-classification-with-transformers.

AttributeError: 'float' object has no attribute 'find'

i want to find degree holders.
The following code is causing AttributeError: 'float' object has no attribute 'find' and I do not how to fix it:
edu = Edu_data
# Function to identify degree
def degree(x):
#if x.find('Bachelor') != -1 or x.find("Bachelor's") != -1 or x.find('BS') != -1 or x.find('bs') != -1 or x.find('Bs') != -1 or x.find('Bachelors') != -1 or x.find('Undergraduate') != -1 or x.find('graduated')!= -1 or x.find('BSE')!= -1 or x.find('Enginee') != -1 or x.find('BCS') != -1:
if x.find('Bachelor') != -1 or x.find("Bachelor's") != -1 or x.find('BS') != -1 or x.find('bs') != -1:
return(1)
if x.find('Master') != -1 or x.find("Master's") != -1 or x.find('M.S') != -1 or x.find('MS') != -1 or x.find('MPhil') != -1 or x.find('MBA') != -1 or x.find('MicroMasters') != -1 or x.find('MSc') != -1 or x.find('MSCS') !=-1 or x.find('MSDS')!=-1:
return(2)
if x.find('PhD') != -1 or x.find('P.hd') != -1 or x.find('Ph.D') != -1 or x.find('ph.d') != -1:
return(3)
else:
return(0)
# Create degree column
edu['deg'] = list(map(degree, edu['Last_degree']))
edu
The full traceback for the error:
AttributeError Traceback (most recent call last)
<ipython-input-39-9a79283a9b17> in <module>
16
17 # Create degree column
---> 18 edu['deg'] = list(map(degree, edu['Last_degree']))
19
20 edu
<ipython-input-39-9a79283a9b17> in degree(x)
5 # Function to identify degree
6 def degree(x):
----> 7 if x.find('Bachelor') != -1 or x.find("Bachelor's") != -1 or x.find('BS') != -1 or x.find('bs') != -1 or x.find('Bs') != -1 or x.find('Bachelors') != -1 or x.find('Undergraduate') != -1 or x.find('graduated')!= -1 or x.find('BSE')!= -1 or x.find('Enginee') != -1 or x.find('BCS') != -1:
8 return(1)
9 if x.find('Master') != -1 or x.find("Master's") != -1 or x.find('M.S') != -1 or x.find('MS') != -1 or x.find('MPhil') != -1 or x.find('MBA') != -1 or x.find('MicroMasters') != -1 or x.find('MSc') != -1 or x.find('MSCS') !=-1 or x.find('MSDS')!=-1:
AttributeError: 'float' object has no attribute 'find'
image of dataset

Does the image show all your data. Is it possible that somewhere in the data frame's Last_degree column contains a float?
If yes, change this line:
edu['deg'] = list(map(degree, edu['Last_degree']))
to
edu['deg'] = list(map(degree, edu['Last_degree'].astype(str)))

Python - Script Imrovement Suggestions

I am pretty new to Python. I completed a challenge using a script I created. I am sure it can be improved upon and more efficient. I would appreciate suggestions to improve and simplify this script for future use. The challenge was to read from a file and run the value found in the file through base 16, base 32, & base 64 five times each. Below is the script I created. The last variable (v64) contains the flag.
Thank You!
> import base64
>
> i = 0
> with open('encodedflag.txt') as fp:
> v = fp.read()
>
> while i <= 4:
> decoded_16 = base64.b16decode(v)
> v = decoded_16
> i += 1
>
> v32 = v
>
> j = 0
>
> while j <= 4:
> decode_32 = base64.b32decode(v32)
> v32 = decode_32
> j += 1
>
> v64 = v32
>
> k = 0
>
> while k <= 4:
> decode_64 = base64.b64decode(v64)
> v64 = decode_64
> k += 1
>
> print(v64)

What is causing the Colon Expected Error in Python?

I am trying to get my head around Expected Colon Error in the following code. Any help appreciated.
The code is -
if df.loc['pivot'] > df.loc['Open'] :
df.loc['Signal'] = 1
elif df.loc[['Open'] > df.loc['IB'] and df.loc['Open'] > df.loc['pivot'] and df.loc['Open'] < df.loc['OB']:
df.loc['Signal'] = 0
elif (((df.loc['Open']) <= (((df.loc['2_pips']) - 5)/ 1000)) * (df.loc['pivot'])) and ((df.loc['Open']) >= (((df.loc['1_pips']) + 10)/ 1000 * (df.loc['pivot']))) and ((df.loc['Open']) >= (df.loc['pivot'])) :
df.loc['Signal'] = 1
elif (df.loc['Open'] <= ((df.loc['1_pips'] * df.loc['pivot']) + df.loc['pivot'] )) and (df.loc['Open'] > (((df.loc['1_pips'] - 10) * df.loc['pivot']) + df.loc['pivot'])) :
df.loc['Signal'] = 1
elif ((df.loc['Open'] < OB) and (df.loc['Open'] < df.loc['pivot'])):
df.loc['Signal'] = -1
elif ((df.loc['Open'] <= OB) and (df.loc['Open'] >= IB) and (df.loc['Open'] < df.loc['pivot'])):
df.loc['bs'] = 0
elif (df.loc['Open'] < ((df.Loc['2_pips'] - 5) * df.loc['pivot']) + df.loc['pivot']) and (df.loc['Open'] > ((df.loc['1_pips'] + 10) * pivot) + df.loc['pivot']) and (df.loc['Open'] < df.loc['pivot']):
df.loc['Signal'] = -1
elif (df.loc['Open'] <= (df.loc['1_pips'] * df.loc['pivot']) + df.loc['pivot']) and (df.loc['Open'] > ((df.loc['1_pips'] - 10) * df.loc['pivot']) + df.loc['pivot']):
df.loc['Signal'] = -1
else:
df.loc['Signal'] = 0
Thanks!

On line 4, you have df.loc[['Open'] which I guess is one square bracket too much.
You can also simplify that line as:
elif df.loc['Open'] > df.loc['IB'] and df.loc['pivot'] < df.loc['Open'] < df.loc['OB']:
And generally, you have way too many parentheses. I'm guessing that it's actually because you were trying to find the issue.
In general, when you get a compiler syntax error like an (un)expected token, you have a structural issue just above it. It is parsing the line it's complaining about as the continuation of the previous statement. Unmatched (), [] or {}. To find it, one could have reduced the code with the error and noticed that it was still failing with the same error, meaning that the actual error was in fact from the previous line(s).

Cython: for i from 1 <= i < N

I'm learning Cython and came across this snippit of code:
import numpy as np
cimport numpy as np
def mean(np.ndarray[np.double_t] input):
cdef np.double_t cur
# Py_ssize_t is numpy's index type
cdef Py_ssize_t i
cdef Py_ssize_t N = len(input)
for i from 0 <= i < N:
cur += input[i]
return cur / N
a=np.array([1,2,3,4], dtype=np.double)
Obviously, this returns the mean of a which is 2.5. My question is this:
Is the for loop a Python loop, Cython, or C?

Compile it and see: the C code that Cython produces is nicely annotated.
/* "cyexample.pyx":11
* cdef Py_ssize_t N = len(input)
*
* for i from 0 <= i < N: # <<<<<<<<<<<<<<
* cur += input[i]
*
*/
__pyx_t_1 = __pyx_v_N;
for (__pyx_v_i = 0; __pyx_v_i < __pyx_t_1; __pyx_v_i++) {
/* "cyexample.pyx":12
*
* for i from 0 <= i < N:
* cur += input[i] # <<<<<<<<<<<<<<
*
* return cur / N
*/
__pyx_t_2 = __pyx_v_i;
__pyx_t_3 = -1;
if (__pyx_t_2 < 0) {
__pyx_t_2 += __pyx_bshape_0_input;
if (unlikely(__pyx_t_2 < 0)) __pyx_t_3 = 0;
} else if (unlikely(__pyx_t_2 >= __pyx_bshape_0_input)) __pyx_t_3 = 0;
if (unlikely(__pyx_t_3 != -1)) {
__Pyx_RaiseBufferIndexError(__pyx_t_3);
{__pyx_filename = __pyx_f[0]; __pyx_lineno = 12; __pyx_clineno = __LINE__; goto __pyx_L1_error;}
}
__pyx_v_cur = (__pyx_v_cur + (*__Pyx_BufPtrStrided1d(__pyx_t_5numpy_double_t *, __pyx_bstruct_input.buf, __pyx_t_2, __pyx_bstride_0_input)));
}
And so the loop itself is successfully turned into C. Note that these days Cython can handle range naturally, so the older "from 0 <= i < N" style isn't necessary. The point of introducing the (non-Python) "for/from" syntax was to signify which loops should be C-ified.

for..from seems to be a Pyrex / Cython loop: http://docs.cython.org/src/userguide/language_basics.html#integer-for-loops