Lexicon Matlinae

datasets.Dataset

The class Dataset supplied by Hugging Face streamlines dataset transformations and is convertible to a torch.utils.data.DataLoader or a tf.data.dataset.

def test_method(sample):  
    sample["test"] = [sample["text"], "blabla"]  
    return sample  
  
dataset = Dataset.from_dict({"text": ["Hello world", "Hugging Face is great"]})  
dataset = dataset.map(test_method, batched=True)  
  

The code will generate **ArrowInvalid**: cannot mix list and non-list, non-null values. Without batched=True, the list is attached to each item.