Problems when I use two dataloaders to read the sa

I have an image list, which includes ['1.jpg',......,'1000.jpg'].

I need the first dataloader to load the image list by order.

At some time, I need a second dataloader to load the reverse image list which contains a part of the images until now, e.g. ['55.jpg',...,'1.jpg'], or ['999.jpg', ..., '1.jpg']

However, when I use these two dataloaders in my code, my program will silence sometimes. The GPU memory is not out of memory, but the Volatile GPU-Util keeps to be zero, which seems to be a lock happening.

Here is the code:

# main function, first dataloader, ['1.jpg', ..., '1000.jpg']
seq_dt = SequenceLoader(ori_image_list)
seq_loader = DataLoader(seq_dt, batch_size=1, shuffle=False, sampler=None, num_workers=1, pin_memory=True)
for current_image_idx, img in enumerate(seq_laoder):
    if my_condition:
        flag = my_function(current_image_idx, ori_image_list)

I will generate the second dataloader in my function

def my_function(current_image_idx, ori_image_list):
    backward_image_list = ori_image_list[:current_image_idx]
    backward_image_list.reverse()
    seq_dt = SequenceLoader(backward_image_list)
    seq_loader = DataLoader(seq_dt, batch_size=1, shuffle=False, sampler=None, 
    num_workers=1, pin_memory=True)
    for image_idx, img in enumerate(seq_loader):
        ...
        ...
        my_variable = ...
        ...
        if my_variable<my_threshold:
            return my_flag
    return False

In the main function, I will execute my_function sometimes.
Sometimes, my_function programs cannot return to the main function. If I change the num_workers parameter in the second dataloader to be 0, no fail happens, but the speed is low.

Here is the code of SequenceLoader

import os
import numpy as np
import cv2

import torch
import torchvision
from torch.utils.data import Dataset, DataLoader
from torchvision.transforms import ToTensor

class SequenceLoader(Dataset):
    ''' A dataset for loading images from a sequence '''

    def __init__(self, image_list):

        self.image_list = image_list


    def __getitem__(self, idx):

        # load images in PIL format
        # img = Image.open()
        img = cv2.imread(self.image_list[idx], cv2.IMREAD_COLOR).astype(np.float32)
        img_tensor = torch.from_numpy(img.transpose((2, 0, 1)))
        return img_tensor

    def __len__(self):
        return len(self.image_list)