I'm working with a Text File in Delphi, and I don't wish to use the method of loading/saving with a string list. I intend to maintain an open filestream where I read and write my data there, keeping massive amounts of data on the hard disk instead of in the memory. I have the simple concept of writing new lines to a text file and reading them, but when it comes to modifying and deleting them, I cannot find any good resources.
Each line in this file contains a name, and equals sign, and the rest is data. For example, SOMEUNIQUENAME=SomeStringValue. I intend to keep a file open for a period of time inside of a thread. This thread performs incoming requests to either get, set, or delete certain fields of data. I use WriteLn and ReadLn in a loop, evaluating EOF. Below is an example of how I read the data:
FFile = TextFile;
...
function TFileWrapper.ReadData(const Name: String): String;
var
S: String; //Temporary line to be parsed
N: String; //Temporary name of field
begin
Result:= '';
Reset(FFile);
while not EOF(FFile) do begin
ReadLn(FFile, S);
N:= UpperCase(Copy(S, 1, Pos('=', S)-1));
if N = UpperCase(Name) then begin
Delete(S, 1, Pos('=', S));
Result:= S;
Break;
end;
end;
end;
...and then I trigger an event which informs sender of result. The requests are inside of a queue, which is sort of a message pump for these requests. The thread simply processes the next request in the queue repeatedly, similar to how typical applications work.
I have procedures ready to be able to write and delete these fields, but I don't know what I have to do to actually perform the action on the file.
procedure TFileWrapper.WriteData(const Name, Value: String);
var
S: String; //Temporary line to be parsed
N: String; //Temporary name of field
begin
Result:= '';
Reset(FFile);
while not EOF(FFile) do begin
ReadLn(FFile, S);
N:= UpperCase(Copy(S, 1, Pos('=', S)-1));
if N = UpperCase(Name) then begin
//How to re-write this line?
Break;
end;
end;
end;
procedure TFileWrapper.DeleteData(const Name: String);
var
S: String; //Temporary line to be parsed
N: String; //Temporary name of field
begin
Result:= '';
Reset(FFile);
while not EOF(FFile) do begin
ReadLn(FFile, S);
N:= UpperCase(Copy(S, 1, Pos('=', S)-1));
if N = UpperCase(Name) then begin
//How to delete this line?
Break;
end;
end;
end;
In the end, I need to avoid loading the entire file into the memory to be able to accomplish this.
I find this an interesting question, so I made a small console app.
I used 3 methods:
- TStringList
- Streamreader/StreamWriter
- Text file
All methods are timed and repeated 100 times with a text file of 10kb in size and a text file 1Mb in size.
Here is the program:
program Project16;
{$APPTYPE CONSOLE}
uses
SysUtils, Classes, StrUtils, Diagnostics, IOUtils;
procedure DeleteLine(StrList: TStringList; SearchPattern: String);
var
Index : Integer;
begin
for Index := 0 to StrList.Count-1 do
begin
if ContainsText(StrList[Index], SearchPattern) then
begin
StrList.Delete(Index);
Break;
end;
end;
end;
procedure DeleteLineWithStringList(Filename : string; SearchPattern : String);
var StrList : TStringList;
begin
StrList := TStringList.Create;
try
StrList.LoadFromFile(Filename);
DeleteLine(StrList, SearchPattern);
// don't overwrite our input file so we can test
StrList.SaveToFile(TPath.ChangeExtension(Filename, '.new'));
finally
StrList.Free;
end;
end;
procedure DeleteLineWithStreamReaderAndWriter(Filename : string; SearchPattern : String);
var
Reader : TStreamReader;
Writer : TStreamWriter;
Line : String;
DoSearch : Boolean;
DoWrite : Boolean;
begin
Reader := TStreamReader.Create(Filename);
Writer := TStreamWriter.Create(TPath.ChangeExtension(Filename, '.new'));
try
DoSearch := True;
DoWrite := True;
while Reader.Peek >= 0 do
begin
Line := Reader.ReadLine;
if DoSearch then
begin
DoSearch := not ContainsText(Line, SearchPattern);
DoWrite := DoSearch;
end;
if DoWrite then
Writer.WriteLine(Line)
else
DoWrite := True;
end;
finally
Reader.Free;
Writer.Free;
end;
end;
procedure DeleteLineWithTextFile(Filename : string; SearchPattern : String);
var
InFile : TextFile;
OutFile : TextFile;
Line : String;
DoSearch : Boolean;
DoWrite : Boolean;
begin
AssignFile(InFile, Filename);
AssignFile(OutFile, TPath.ChangeExtension(Filename, '.new'));
Reset(InFile);
Rewrite(OutFile);
try
DoSearch := True;
DoWrite := True;
while not EOF(InFile) do
begin
Readln(InFile, Line);
if DoSearch then
begin
DoSearch := not ContainsText(Line, SearchPattern);
DoWrite := DoSearch;
end;
if DoWrite then
Writeln(OutFile, Line)
else
DoWrite := True;
end;
finally
CloseFile(InFile);
CloseFile(OutFile);
end;
end;
procedure TimeDeleteLineWithStreamReaderAndWriter(Iterations : Integer);
var
Count : Integer;
Sw : TStopWatch;
begin
Writeln(Format('Delete line with stream reader/writer - file 10kb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithStreamReaderAndWriter('c:\temp\text10kb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
Writeln(Format('Delete line with stream reader/writer - file 1Mb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithStreamReaderAndWriter('c:\temp\text1Mb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
end;
procedure TimeDeleteLineWithStringList(Iterations : Integer);
var
Count : Integer;
Sw : TStopWatch;
begin
Writeln(Format('Delete line with TStringlist - file 10kb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithStringList('c:\temp\text10kb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
Writeln(Format('Delete line with TStringlist - file 1Mb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithStringList('c:\temp\text1Mb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
end;
procedure TimeDeleteLineWithTextFile(Iterations : Integer);
var
Count : Integer;
Sw : TStopWatch;
begin
Writeln(Format('Delete line with text file - file 10kb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithTextFile('c:\temp\text10kb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
Writeln(Format('Delete line with text file - file 1Mb, %d iterations', [Iterations]));
Sw := TStopwatch.StartNew;
for Count := 1 to Iterations do
DeleteLineWithTextFile('c:\temp\text1Mb.txt', 'thislinewillbedeleted=');
Sw.Stop;
Writeln(Format('Elapsed time : %d milliseconds', [Sw.ElapsedMilliseconds]));
end;
begin
try
TimeDeleteLineWithStringList(100);
TimeDeleteLineWithStreamReaderAndWriter(100);
TimeDeleteLineWithTextFile(100);
Writeln('Press ENTER to quit');
Readln;
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
end.
Output:
Delete line with TStringlist - file 10kb, 100 iterations
Elapsed time : 188 milliseconds
Delete line with TStringlist - file 1Mb, 100 iterations
Elapsed time : 5137 milliseconds
Delete line with stream reader/writer - file 10kb, 100 iterations
Elapsed time : 456 milliseconds
Delete line with stream reader/writer - file 1Mb, 100 iterations
Elapsed time : 22382 milliseconds
Delete line with text file - file 10kb, 100 iterations
Elapsed time : 250 milliseconds
Delete line with text file - file 1Mb, 100 iterations
Elapsed time : 9656 milliseconds
Press ENTER to quit
As you can see is TStringList the winner here.
Since you are not able to use TStringList, TextFile is not a bad choice after all...
P.S. : this code omits the part where you have to delete the inputfile and rename the outputfile to the original filename
Without loading the entire file into a container like TStringList, your only option is to:
- Open the file for input
- Open a separate copy for output
- Start a loop
- Read the content line by line from the input file
- Write the content out line by line to the output file until you reach the line you want to change/delete
- Break the loop
- Read the input line from the input file
- Write the changed line (or skip writing the line you want to delete) to the output file
- Start a new loop
- Read the remainder of the input content, line by line
- Write the rest of that input to the output file, line by line
- Break the loop
- Close the files
So to answer your specific questions:
if N = UpperCase(Name) then begin
//How to re-write this line?
Break;
end;
WriteLn the new output to the second (output) file.
if N = UpperCase(Name) then begin
//How to delete this line?
Break;
end;
Just skip the WriteLn that outputs the indicated line to the second (output) file.
Your artificial limitation of "I don't want to use TStringList" simply complicates the task for you, when you can simply:
- Load the original file into
TStringList using LoadFromFile
- Locate the line you want to modify, either by index, iteration, or
IndexOf()
- Modify the line by changing it directly, or deleting it from the
TStringList
- Write the entire content out to the original file using
TStringList.SaveToFile
The only reasons I've found to not use TStringList to perform these kinds of operations have been that the file size exceeds the capacity of a TStringList (never happened) or when dealing with a file that is text but isn't really "line" oriented (for instance, EDI files that are typically one very long single line of text, or XML files that may not contain line feeds and therefore are also one very long single line of text). Even in the case of EDI or XML, though, it's quite frequently to load them into a TStringList, make the conversion to line-based format (inserting line breaks or whatever), and do the retrieval from the stringlist.
Basically, you can't do what you want to do if you treat the files as simple text files. Such files can be read (from the beginning only) or written to (either from the start, thus creating a new file) or from the end (appending to an existing file). They are not random access files.
On the other hand, you might want to consider defining a file of type string: each record in the file would be a string, and you can access this file in a random fashion. The problem then becomes in knowing which record to access for which string.
A third possibility is using INI files which are more structured and sound like a better bet for your purposes. Apart from the section header, they are a series of strings, key=value, and can be accessed on the basis of the key.
