Batch file tokens ignore empty delimiters

Question

I am trying to load a column from a CSV file into a variable. My CSV file contains empty columns so can have ,, which seems to be ignored by the following:

FOR /F "tokens=3 delims=," %%a in (csvfile.csv) do (
    set date=%%a
    echo %%a
)

So if file csvfile.csv contains

fred,wilma,tony, john, greg, wilber, blake
chris,,steve,deon,bryan,mark,anthony

I would like the result:

tony
steve

But I get:

tony
deon

If I put a white space in the empty column this does work as expected.

How do I set delims so it does not ignore empty columns?

Answer 1

@ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=U:\sourcedir"
SET "filename1=%sourcedir%\q36372429.txt"
SET "oneline="

FOR /f "usebackqdelims=" %%a IN ("%filename1%") DO (
 SET "line=%%a"
 FOR /f "tokens=3delims=," %%h IN (" !line:,= , ! ") DO (
  SET "column=%%h"
  SET "oneline=!oneline! !column:~1,-1!"
  ECHO(!column:~1,-1!
 )
)
ECHO %oneline:~1%

GOTO :EOF

You would need to change the settings of sourcedir and filename1 to suit your circumstances.

I used a file named q36372429.txt containing your data for my testing.

Unfortunately, you haven't shown us a realistic data sample. I'm very suspicious of your use of date as the destination variable.

date is a "magic variable" - maintained by the system as the current date. It can be overridden, but it's not advisable to do so.

It's not really clear what you want out of this procedure. Perhaps you want the entries from column 3 all on line line as you state, or perhaps you want to list only column 3 or perhaps you'd want the output transferred to a new file.

Answer 2

for treats consecutive delimiters as one. In most cases, this is helpful. Sometimes it is not.

So you have to write your lines in a way, that for can handle as you intended.

Read every line of your file as a whole, add a quote at the beginning and at the end and replace every , with ",", resulting in

"chris","","steve","deon","bryan","mark","anthony"

This can happily be parsed with another for. The tilde in %%~b removes the surrounding quotes.

@echo off
setlocal enabledelayedexpansion
FOR /F "tokens=*" %%a in (csvfile.csv) do (
  set line="%%a"
  for /f "tokens=3 delims=," %%b in ("!line:,="^,"!") do echo %%~b
)