How to sort files in paste command?

Question

I am trying to paste serious dynamically created files into one files by using the paste command. Because i want to merge all the files in column wise into new file.

Required to sort the files inside the Paste command,

In my directory I have list of files from files_1.csv to files_125.csv when i try to use the below paste command its not pasting files in a order.

paste -d, files_*.csv > new_file.csv

Also i cant use ls inside paste command ls-v will sort files as I expected.

I want to paste the file in order starting from files_1.csv to files_125.csv to new_files.csv.