Java: Removing comments from string

问题:

I'd like to do a function which gets a string and in case it has inline comments it removes it. I know it sounds pretty simple but i wanna make sure im doing this right, for example:

private String filterString(String code) {
  // lets say code = "some code //comment inside"

  // return the string "some code" (without the comment)
}

I thought about 2 ways: feel free to advice otherwise

1. Iterating the string and finding double inline brackets and using substring method.
2. regex way.. (im not so sure bout it)

can u tell me what's the best way and show me how it should be done? (please don't advice too advanced solutions)

edited: can this be done somehow with Scanner object? (im using this object anyway)

回答1:

To find the substring before a constant substring using a regular expression replacement is a bit much.

You can do it using indexOf() to check for the position of the comment start and substring() to get the first part, something like:

String code = "some code // comment";
int    offset = code.indexOf("//");

if (-1 != offset) {
    code = code.substring(0, offset);
}

回答2:

If you want a more efficient regex to really match all types of comments, use this one :

replaceAll("(?:/\\*(?:[^*]|(?:\\*+[^*/]))*\\*+/)|(?://.*)",""));

source : http://ostermiller.org/findcomment.html

EDIT:

Another solution, if you're not sure about using regex is to design a small automata like follows :

public static String removeComments(String code){
    final int outsideComment=0;
    final int insideLineComment=1;
    final int insideblockComment=2;
    final int insideblockComment_noNewLineYet=3; // we want to have at least one new line in the result if the block is not inline.

    int currentState=outsideComment;
    String endResult="";
    Scanner s= new Scanner(code);
    s.useDelimiter("");
    while(s.hasNext()){
        String c=s.next();
        switch(currentState){
            case outsideComment: 
                if(c.equals("/") && s.hasNext()){
                    String c2=s.next();
                    if(c2.equals("/"))
                        currentState=insideLineComment;
                    else if(c2.equals("*")){
                        currentState=insideblockComment_noNewLineYet;
                    }
                    else 
                        endResult+=c+c2;
                }
                else
                    endResult+=c;
                break;
            case insideLineComment:
                if(c.equals("\n")){
                    currentState=outsideComment;
                    endResult+="\n";
                }
            break;
            case insideblockComment_noNewLineYet:
                if(c.equals("\n")){
                    endResult+="\n";
                    currentState=insideblockComment;
                }
            case insideblockComment:
                while(c.equals("*") && s.hasNext()){
                    String c2=s.next();
                    if(c2.equals("/")){
                        currentState=outsideComment;
                        break;
                    }

                }

        }
    }
    s.close();
    return endResult;   
}

回答3:

The best way to do this is to use regular expressions. At first to find the /**/ comments and then remove all // commnets. For example:

private String filterString(String code) {
  String partialFiltered = code.replaceAll("/\\*.*\\*/", "");
  String fullFiltered = partialFiltered.replaceAll("//.*(?=\\n)", "")
}

回答4:

Just use the replaceAll method from the String class, combined with a simple regular expression. Here's how to do it:

import java.util.*;
import java.lang.*;

class Main
{
        public static void main (String[] args) throws java.lang.Exception
        {
                String s = "private String filterString(String code) {\n" +
"  // lets say code = \"some code //comment inside\"\n" +
"  // return the string \"some code\" (without the comment)\n}";

                s = s.replaceAll("//.*?\n","\n");
                System.out.println("s=" + s);

        }
}

The key is the line:

s = s.replaceAll("//.*?\n","\n");

The regex //.*?\n matches strings starting with // until the end of the line.

And if you want to see this code in action, go here: http://www.ideone.com/e26Ve

Hope it helps!

回答5:

@Christian Hujer has been correctly pointing out that many or all of the solutions posted fail if the comments occur within a string.

@Loïc Gammaitoni suggests that his automata approach could easily be extended to handle that case. Here is that extension.

enum State { outsideComment, insideLineComment, insideblockComment, insideblockComment_noNewLineYet, insideString };

public static String removeComments(String code) {
  State state = State.outsideComment;
  StringBuilder result = new StringBuilder();
  Scanner s = new Scanner(code);
  s.useDelimiter("");
  while (s.hasNext()) {
    String c = s.next();
    switch (state) {
      case outsideComment:
        if (c.equals("/") && s.hasNext()) {
          String c2 = s.next();
          if (c2.equals("/"))
            state = State.insideLineComment;
          else if (c2.equals("*")) {
            state = State.insideblockComment_noNewLineYet;
          } else {
            result.append(c).append(c2);
          }
        } else {
          result.append(c);
          if (c.equals("\"")) {
            state = State.insideString;
          }
        }
        break;
      case insideString:
        result.append(c);
        if (c.equals("\"")) {
          state = State.outsideComment;
        } else if (c.equals("\\") && s.hasNext()) {
          result.append(s.next());
        }
        break;
      case insideLineComment:
        if (c.equals("\n")) {
          state = State.outsideComment;
          result.append("\n");
        }
        break;
      case insideblockComment_noNewLineYet:
        if (c.equals("\n")) {
          result.append("\n");
          state = State.insideblockComment;
        }
      case insideblockComment:
        while (c.equals("*") && s.hasNext()) {
          String c2 = s.next();
          if (c2.equals("/")) {
            state = State.outsideComment;
            break;
          }
        }
    }
  }
  s.close();
  return result.toString();
}

回答6:

for scanner, use a delimiter,

delimiter example.

import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Scanner;

public class MainClass {
  public static void main(String args[]) throws IOException {
FileWriter fout = new FileWriter("test.txt");
fout.write("2, 3.4,    5,6, 7.4, 9.1, 10.5, done");
fout.close();

FileReader fin = new FileReader("Test.txt");
Scanner src = new Scanner(fin);
// Set delimiters to space and comma.
// ", *" tells Scanner to match a comma and zero or more spaces as
// delimiters.

src.useDelimiter(", *");

// Read and sum numbers.
while (src.hasNext()) {
  if (src.hasNextDouble()) {
    System.out.println(src.nextDouble());
  } else {
    break;
  }
}
fin.close();
  }
}

Use a tokenizer for a normal string

tokenizer:

// start with a String of space-separated words
String tags = "pizza pepperoni food cheese";

// convert each tag to a token
StringTokenizer st = new StringTokenizer(tags," ");

while ( st.hasMoreTokens() )
{
  String token = (String)st.nextToken();
  System.out.println(token);
}

http://www.devdaily.com/blog/post/java/java-faq-stringtokenizer-example

回答7:

I made an open source library (on GitHub) for this purpose , its called CommentRemover you can remove single line and multiple line Java Comments.

It supports remove or NOT remove TODO's.
Also it supports JavaScript , HTML , CSS , Properties , JSP and XML Comments too.

Little code snippet how to use it (There is 2 type usage):

First way InternalPath

 public static void main(String[] args) throws CommentRemoverException {

 // root dir is: /Users/user/Projects/MyProject
 // example for startInternalPath

 CommentRemover commentRemover = new CommentRemover.CommentRemoverBuilder()
        .removeJava(true) // Remove Java file Comments....
        .removeJavaScript(true) // Remove JavaScript file Comments....
        .removeJSP(true) // etc.. goes like that
        .removeTodos(false) //  Do Not Touch Todos (leave them alone)
        .removeSingleLines(true) // Remove single line type comments
        .removeMultiLines(true) // Remove multiple type comments
        .startInternalPath("src.main.app") // Starts from {rootDir}/src/main/app , leave it empty string when you want to start from root dir
        .setExcludePackages(new String[]{"src.main.java.app.pattern"}) // Refers to {rootDir}/src/main/java/app/pattern and skips this directory
        .build();

 CommentProcessor commentProcessor = new CommentProcessor(commentRemover);
                  commentProcessor.start();        
  }

Second way ExternalPath

 public static void main(String[] args) throws CommentRemoverException {

 // example for externalPath

 CommentRemover commentRemover = new CommentRemover.CommentRemoverBuilder()
        .removeJava(true) // Remove Java file Comments....
        .removeJavaScript(true) // Remove JavaScript file Comments....
        .removeJSP(true) // etc..
        .removeTodos(true) // Remove todos
        .removeSingleLines(false) // Do not remove single line type comments
        .removeMultiLines(true) // Remove multiple type comments
        .startExternalPath("/Users/user/Projects/MyOtherProject")// Give it full path for external directories
        .setExcludePackages(new String[]{"src.main.java.model"}) // Refers to /Users/user/Projects/MyOtherProject/src/main/java/model and skips this directory.
        .build();

 CommentProcessor commentProcessor = new CommentProcessor(commentRemover);
                  commentProcessor.start();        
  }