使用工具juniversalchardet.jar 包检测工具:https://code.google.com/p/juniversalchardet/。这是Mozilla开发的一个用于检测文件编码的jar包,用于firefox中的自动编码识别。它支持的被检测的字符编码有:
- Chinese
- ISO-2022-CN
- BIG5
- EUC-TW
- GB18030
- HZ-GB-23121
- Cyrillic
- ISO-8859-5
- KOI8-R
- WINDOWS-1251
- MACCYRILLIC
- IBM866
- IBM855
- Japanese
- ISO-2022-JP
- SHIFT_JIS
- EUC-JP
- Unicode
- UTF-8
- UTF-16BE / UTF-16LE
- UTF-32BE / UTF-32LE / X-ISO-10646-UCS-4-34121 / X-ISO-10646-UCS-4-21431
其示例代码:
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
import org.mozilla.universalchardet.UniversalDetector;
public class ConvertCode{
public static void main(String[] args) throws IOException{
String sourcePage; //输入文件
sourcePage="D:\\CF\\HttpClient\\1d07a51f09ed37fb70aaf911b9c8643e.html";
UniversalDetector detector=new UniversalDetector(null); //用于检测网页编码
byte[] buf=new byte[4096];
FileInputStream fis=new FileInputStream(sourcePage);
int nread;
while((nread=fis.read(buf))>0&&!detector.isDone()){
detector.handleData(buf,0,nread);
}
detector.dataEnd();
String encoding=detector.getDetectedCharset();
if (encoding!=null){
System.out.println("Encoding:"+encoding);
}else{
System.out.println("No encoding detected!");
}
detector.reset();
String cread;
StringBuffer content = new StringBuffer();
InputStreamReader r = new InputStreamReader(new FileInputStream(sourcePage), encoding);
BufferedReader in = new BufferedReader(r);
while ((cread = in.readLine()) != null) {
content.append(cread);
}
System.out.print(content.toString());
}
总的说来juniversalchardet.jar工具包比cpdetector_1.0.5.jar这些工具检测编码准确率高很多,当然在识别只有几个字节的短文本编码时,可能会有错误,不过对于大的文本,则识别正确率很高。
附:
1.按指定编码读文件:
String string = "";
try {
BufferedReader in = new BufferedReader(new InputStreamReader(new FileInputStream(fileName), encoding));
String str = "";
while ((str = in.readLine()) != null) {
string += str + "\n";
}
in.close();
} catch (Exception ex) {
ex.printStackTrace();
}
2.按指定编码写文件:
try {
Writer out = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream(fileName), encoding));
out.write(str);
out.close();
} catch (Exception ex) {
ex.printStackTrace();
}
评论