Java自学者论坛

 找回密码
 立即注册

手机号码,快捷登录

恭喜Java自学者论坛(https://www.javazxz.com)已经为数万Java学习者服务超过8年了!积累会员资料超过10000G+
成为本站VIP会员,下载本站10000G+会员资源,会员资料板块,购买链接:点击进入购买VIP会员

JAVA高级面试进阶训练营视频教程

Java架构师系统进阶VIP课程

分布式高可用全栈开发微服务教程Go语言视频零基础入门到精通Java架构师3期(课件+源码)
Java开发全终端实战租房项目视频教程SpringBoot2.X入门到高级使用教程大数据培训第六期全套视频教程深度学习(CNN RNN GAN)算法原理Java亿级流量电商系统视频教程
互联网架构师视频教程年薪50万Spark2.0从入门到精通年薪50万!人工智能学习路线教程年薪50万大数据入门到精通学习路线年薪50万机器学习入门到精通教程
仿小米商城类app和小程序视频教程深度学习数据分析基础到实战最新黑马javaEE2.1就业课程从 0到JVM实战高手教程MySQL入门到精通教程
查看: 35|回复: 0

poco json 中文字符,抛异常JSON Exception -->iconv 转换 备忘录。

[复制链接]
  • TA的每日心情
    奋斗
    昨天 15:16
  • 签到天数: 187 天

    [LV.7]常住居民III

    1473

    主题

    1531

    帖子

    8万

    积分

    管理员

    Rank: 9Rank: 9Rank: 9

    积分
    84758
    发表于 2021-9-3 15:06:16 | 显示全部楼层 |阅读模式

     

     

    起因

     最近linux服务器通信需要用到json.

     jsoncpp比较出名,但poco 1.5版本以后已经带有json库,所以决定使用poco::json(linux 上已经用到了poco这一套框架).

     网上关于 poco json处理中文比较少.

     有后有网友说:

     Latin1装gbk字符然后转utf8,当时喜出望外.如下:

    "

    Latin1是ISO-8859-1的别名,有些环境下写作Latin-1。ISO-8859-1编码是单字节编码,向下兼容ASCII,其编码范围是0x00-0xFF,0x00-0x7F之间完全和ASCII一致,0x80-0x9F之间是控制字符,0xA0-0xFF之间是文字符号。

    因为ISO-8859-1编码范围使用了单字节内的所有空间,在支持ISO-8859-1的系统中传输和存储其他任何编码的字节流都不会被抛弃。换言之,把其他任何编码的字节流当作ISO-8859-1编码看待都没有问题。这是个很重要的特性,MySQL数据库默认编码是Latin1就是利用了这个特性。ASCII编码是一个7位的容器,ISO-8859-1编码是一个8位的容器。

    "

    Latin1与utf8转换

    测试代码1:

    json="{ \"test\" : \" 欢迎来到abc@123.com \"}";
        //json="{ \"test\" : \" \u6b22\u8fce\u6765\u5230abc@123.com \"}";
        Poco::Latin1Encoding latin1;
        Poco::UTF8Encoding utf8;
        Poco::TextConverter converter(latin1, utf8);
        std::string strUtf8;
        converter.convert(json, strUtf8);
    
        Parser parser;
        Var result;
    
        try
        {
            result = parser.parse(strUtf8);
        }
        catch(JSONException& jsone)
        {
            std::cout << jsone.message() << std::endl;
            //assert(false);
        }
    
        //assert(result.type() == typeid(Object::Ptr));
        //if(result)
    
        Object::Ptr object = result.extract<Object::Ptr>();
        int nObjectSize=object->size();
    
        //assert(object->size() > 0);
        DynamicStruct ds = *object;

    json="{ \"test\" : \" 欢迎来到abc@123.com \"}";


    json="{ \"test\" : \" \u6b22\u8fce\u6765\u5230abc@123.com \"}";

    \u6b22  是unicode编码: 欢->6b22

    windows下是正常的,但在linux 下 却会出现json格式错误的问题.  这真是无比蛋痛....(linux 下poco::json处理还是有问题)

    连 std::cout<<json ; 都不能打印.

      Poco::Latin1Encoding latin1;
        Poco::UTF8Encoding utf8;
        Poco::TextConverter converter(latin1, utf8); ->latin1 转utf8




     Poco::TextConverter converter2( utf8,latin1); ->utf8 转latin1
    都有问题.

     

       使用Poco::Json解析json内容,其中有中文字符,抛异常JSON Exception: Bad character.

    最后只有自己写了个编码转换.

     

    使用iconv实现编码转换

    上代码:

    xxx.h

    #ifndef _MY_CODE_CONVERT_H
    #define  _MY_CODE_CONVERT_H
    
    
    #include <iostream>
    
    
    using namespace std;
    
    ///////////
    //编码转换类
    //
    class CMyCodeConvert
    {
    public:
    
        CMyCodeConvert();
    
        ~CMyCodeConvert();
    
        //
        static std::string Gb2312ToUtf8(std::string strSource);
    
        //
        static std::string Utf8ToGb2312(std::string strSource);
    
    };
    
    
    
    #endif

    xxx.cpp

    #include "MyCodeConverter.h"
    
    #include <stdio.h>
    #include <stdlib.h>
    #include <malloc.h>
    
    #include <iosfwd>
    #include <string.h>
    #include <stdarg.h>
    #include <fstream>
    #include <sstream>
    #include <iomanip>  
    
    #include <cassert>
    #include<time.h>
    
    
    #include <errno.h>
    
    
    #ifdef WIN32   //window
    
    
    #include <Windows.h>
    #include <shlwapi.h>
    #include <direct.h>
    
    #include <tchar.h>
    #pragma comment(lib,"Kernel32.lib")
    #pragma comment(lib,"shlwapi.lib")
    
    #elif __linux    //linux
    #include <cassert>
    #include <sys/types.h>
    #include <iconv.h>
    #include <stdio.h>
    
    #endif
    
    
    #ifdef WIN32   //window
    
    
    static std::string Gb32ToUtf8(const char * lpszGb32Text)
    {
        int nUnicodeBufLen=MultiByteToWideChar(CP_ACP, 0, lpszGb32Text, -1, 0, 0);
        if (nUnicodeBufLen==0)
            return _T("");
    
        WCHAR* pUnicodeBuf=new WCHAR[nUnicodeBufLen];
        if (pUnicodeBuf==0)
            return _T("");
    
        MultiByteToWideChar(CP_ACP, 0, lpszGb32Text, -1, pUnicodeBuf, nUnicodeBufLen);
    
        int nUtf8BufLen=WideCharToMultiByte(CP_UTF8, 0, pUnicodeBuf, -1, 0, 0, NULL, NULL);
        if (nUtf8BufLen==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        char* pUft8Buf=new char[nUtf8BufLen];
        if (pUft8Buf==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        WideCharToMultiByte(CP_UTF8, 0, pUnicodeBuf, -1, pUft8Buf, nUtf8BufLen, NULL, NULL);
    
        std::string strUtf8=pUft8Buf;
    
        delete[] pUnicodeBuf;
        delete[] pUft8Buf;
    
        return strUtf8;
    }
    
    static std::string Utf8ToGb32(const char * lpszUft8Text)
    {
        int nUnicodeBufLen=MultiByteToWideChar(CP_UTF8, 0, lpszUft8Text, -1, 0, 0);
        if (nUnicodeBufLen==0)
            return _T("");
    
        WCHAR* pUnicodeBuf=new WCHAR[nUnicodeBufLen];
        if (pUnicodeBuf==0)
            return _T("");
    
        MultiByteToWideChar(CP_UTF8, 0, lpszUft8Text, -1, pUnicodeBuf, nUnicodeBufLen);
    
        int nGb32BufLen=WideCharToMultiByte(CP_ACP, 0, pUnicodeBuf, -1, 0, 0, NULL, NULL);
        if (nGb32BufLen==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        char* pGb32Buf=new char[nGb32BufLen];
        if (pGb32Buf==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        WideCharToMultiByte(CP_ACP, 0, pUnicodeBuf, -1, pGb32Buf, nGb32BufLen, NULL, NULL);
    
        std::string strGb32=pGb32Buf;
    
        delete[] pUnicodeBuf;
        delete[] pGb32Buf;
    
        return strGb32;
    }
    #elif __linux    //linux
    
    
    
    
    // 编码转换操作类->基于byte
    //
    class CodeConverterBase {
    
        //示例...
        //std::string strxxxxcvx=json;
        //CodeConverterBase cc2 = CodeConverterBase("gb2312","utf-8");
        //char outbuf[3000];
        //cc2.convert((char *)strxxxxcvx.c_str(),strxxxxcvx.size(),outbuf,1024);
        //strUtf8=outbuf;
    
    
    private:
        iconv_t cd;
    public:
    
        // 构造
        CodeConverterBase(const char *from_charset,const char *to_charset) {
            cd = iconv_open(to_charset,from_charset);
        }
    
        // 析构
        ~CodeConverterBase() {
            iconv_close(cd);
        }
    
        // 转换输出
        int convert(char *inbuf,int inlen,char *outbuf,int outlen) {
            char **pin = &inbuf;
            char **pout = &outbuf;
            memset(outbuf,0,outlen);
            return iconv(cd,pin,(size_t *)&inlen,pout,(size_t *)&outlen);
        }
    };
    
    //字符串转换操作 ->基于string
    class CodeConverterString
    {
    public:
        CodeConverterString()
        {
    
        }
    
        ~CodeConverterString()
        {
    
        }
    
    
        static std::string Gb2312ToUtf8(std::string strSource)
        {
            std::string strRet="";
    
            CodeConverterBase cc2 = CodeConverterBase("gb2312","utf-8");
            int nLen=strSource.size()*4+4;
            char * szOutBuf=(char *)malloc(nLen);
    
    
            do 
            {
                if(NULL==szOutBuf)
                    break;
    
                memset(szOutBuf,0,nLen);
                cc2.convert((char *)strSource.c_str(),strSource.size(),szOutBuf,nLen);
                strRet=szOutBuf;
    
            } while (0);
    
            if(szOutBuf)
            {
                free(szOutBuf);
                szOutBuf=NULL;
            }
    
            return strRet;
        }
    
        static std::string Utf8ToGb2312(std::string strSource)
        {
            std::string strRet="";
    
            CodeConverterBase cc2 = CodeConverterBase("utf-8","gb2312");
            int nLen=strSource.size()*4+4;
            char * szOutBuf=(char *)malloc(nLen);
    
    
            do 
            {
                if(NULL==szOutBuf)
                    break;
    
                memset(szOutBuf,0,nLen);
                cc2.convert((char *)strSource.c_str(),strSource.size(),szOutBuf,nLen);
                strRet=szOutBuf;
    
            } while (0);
    
            if(szOutBuf)
            {
                free(szOutBuf);
                szOutBuf=NULL;
            }
    
            return strRet;
        }
    
    
    };
    
    
    
    
    #endif
    
    
    
    
    
    
    
    
    
    
    
    
    
    CMyCodeConvert::CMyCodeConvert()
    {
    
    }
    
    CMyCodeConvert::~CMyCodeConvert()
    {
    
    }
    
    
    std::string CMyCodeConvert::Gb2312ToUtf8(std::string strSource)
    {
    #ifdef WIN32   //window
    
        return Gb32ToUtf8(strSource.c_str());
    
    #elif __linux    //linux
    
        return CodeConverterString::Gb2312ToUtf8(strSource);
    #endif
    }
    
    
    
    //
    std::string CMyCodeConvert::Utf8ToGb2312(std::string strSource)
    {
    #ifdef WIN32   //window
    
        return Utf8ToGb32(strSource.c_str());
    
    #elif __linux    //linux
    
        return CodeConverterString::Utf8ToGb2312(strSource);
    #endif
    }

     

     

    结果多次调用后出现了如下错误: 

    Utf8ToGb2312 ,nError=-1,errno =84
    *** glibc detected *** ./HTTPTimeServer: free(): invalid next size (fast): 0x000000000167dda0 ***
    ======= Backtrace: =========
    /lib/x86_64-linux-gnu/libc.so.6(+0x75be6)[0x7f6ed81cdbe6]
    /lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7f6ed81d298c]
    ./HTTPTimeServer(_ZN19CodeConverterString12Utf8ToGb2312ESs+0x1c2)[0x41a553]
    ./HTTPTimeServer(_ZN14CMyCodeConvert12Utf8ToGb2312ESs+0x48)[0x419e12]
    ./HTTPTimeServer(_Z5suitev+0x17a3)[0x41c13e]
    ./HTTPTimeServer(main+0x1a6)[0x41d157]
    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd)[0x7f6ed8176ead]
    ./HTTPTimeServer[0x419c39]
    ======= Memory map: ========
    00400000-0043e000 r-xp 00000000 08:01 797594                             /home/poco-1.6.0-all/Net/samples/HTTPTimeServer/HTTPTimeServer
    0063e000-0063f000 rw-p 0003e000 08:01 797594                             /home/poco-1.6.0-all/Net/samples/HTTPTimeServer/HTTPTimeServer
    0165d000-016a1000 rw-p 00000000 00:00 0                                  [heap]
    7f6ed0000000-7f6ed0021000 rw-p 00000000 00:00 0 
    7f6ed0021000-7f6ed4000000 ---p 00000000 00:00 0 
    7f6ed74fa000-7f6ed7515000 r-xp 00000000 08:01 276623                     /usr/lib/x86_64-linux-gnu/gconv/GBK.so
    7f6ed7515000-7f6ed7714000 ---p 0001b000 08:01 276623                     /usr/lib/x86_64-linux-gnu/gconv/GBK.so
    7f6ed7714000-7f6ed7715000 r--p 0001a000 08:01 276623                     /usr/lib/x86_64-linux-gnu/gconv/GBK.so
    7f6ed7715000-7f6ed7716000 rw-p 0001b000 08:01 276623                     /usr/lib/x86_64-linux-gnu/gconv/GBK.so
    7f6ed7716000-7f6ed771f000 r-xp 00000000 08:01 270318                     /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0
    7f6ed771f000-7f6ed791e000 ---p 00009000 08:01 270318                     /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0
    7f6ed791e000-7f6ed791f000 rw-p 00008000 08:01 270318                     /usr/lib/x86_64-linux-gnu/libltdl.so.7.3.0
    7f6ed791f000-7f6ed792f000 r-xp 00000000 08:01 400548                     /usr/local/lib/libodbcinst.so.2.0.0
    7f6ed792f000-7f6ed7b2f000 ---p 00010000 08:01 400548                     /usr/local/lib/libodbcinst.so.2.0.0
    7f6ed7b2f000-7f6ed7b30000 rw-p 00010000 08:01 400548                     /usr/local/lib/libodbcinst.so.2.0.0
    7f6ed7b30000-7f6ed7b37000 r-xp 00000000 08:01 399741                     /lib/x86_64-linux-gnu/librt-2.13.so
    7f6ed7b37000-7f6ed7d36000 ---p 00007000 08:01 399741                     /lib/x86_64-linux-gnu/librt-2.13.so
    7f6ed7d36000-7f6ed7d37000 r--p 00006000 08:01 399741                     /lib/x86_64-linux-gnu/librt-2.13.so
    7f6ed7d37000-7f6ed7d38000 rw-p 00007000 08:01 399741                     /lib/x86_64-linux-gnu/librt-2.13.so
    7f6ed7d38000-7f6ed7d3a000 r-xp 00000000 08:01 399733                     /lib/x86_64-linux-gnu/libdl-2.13.so
    7f6ed7d3a000-7f6ed7f3a000 ---p 00002000 08:01 399733                     /lib/x86_64-linux-gnu/libdl-2.13.so
    7f6ed7f3a000-7f6ed7f3b000 r--p 00002000 08:01 399733                     /lib/x86_64-linux-gnu/libdl-2.13.so
    7f6ed7f3b000-7f6ed7f3c000 rw-p 00003000 08:01 399733                     /lib/x86_64-linux-gnu/libdl-2.13.so
    7f6ed7f3c000-7f6ed7f53000 r-xp 00000000 08:01 399742                     /lib/x86_64-linux-gnu/libpthread-2.13.so
    7f6ed7f53000-7f6ed8152000 ---p 00017000 08:01 399742                     /lib/x86_64-linux-gnu/libpthread-2.13.so
    7f6ed8152000-7f6ed8153000 r--p 00016000 08:01 399742                     /lib/x86_64-linux-gnu/libpthread-2.13.so
    7f6ed8153000-7f6ed8154000 rw-p 00017000 08:01 399742                     /lib/x86_64-linux-gnu/libpthread-2.13.so
    7f6ed8154000-7f6ed8158000 rw-p 00000000 00:00 0 
    7f6ed8158000-7f6ed82d9000 r-xp 00000000 08:01 399727                     /lib/x86_64-linux-gnu/libc-2.13.so
    7f6ed82d9000-7f6ed84d9000 ---p 00181000 08:01 399727                     /lib/x86_64-linux-gnu/libc-2.13.so
    7f6ed84d9000-7f6ed84dd000 r--p 00181000 08:01 399727                     /lib/x86_64-linux-gnu/libc-2.13.so
    7f6ed84dd000-7f6ed84de000 rw-p 00185000 08:01 399727                     /lib/x86_64-linux-gnu/libc-2.13.so
    7f6ed84de000-7f6ed84e3000 rw-p 00000000 00:00 0 
    7f6ed84e3000-7f6ed84f8000 r-xp 00000000 08:01 393220                     /lib/x86_64-linux-gnu/libgcc_s.so.1
    7f6ed84f8000-7f6ed86f8000 ---p 00015000 08:01 393220                     /lib/x86_64-linux-gnu/libgcc_s.so.1
    7f6ed86f8000-7f6ed86f9000 rw-p 00015000 08:01 393220                     /lib/x86_64-linux-gnu/libgcc_s.so.1
    7f6ed86f9000-7f6ed877a000 r-xp 00000000 08:01 399721                     /lib/x86_64-linux-gnu/libm-2.13.so
    7f6ed877a000-7f6ed8979000 ---p 00081000 08:01 399721                     /lib/x86_64-linux-gnu/libm-2.13.so
    7f6ed8979000-7f6ed897a000 r--p 00080000 08:01 399721                     /lib/x86_64-linux-gnu/libm-2.13.so
    7f6ed897a000-7f6ed897b000 rw-p 00081000 08:01 399721                     /lib/x86_64-linux-gnu/libm-2.13.so
    7f6ed897b000-7f6ed8a63000 r-xp 00000000 08:01 266157                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
    7f6ed8a63000-7f6ed8c63000 ---p 000e8000 08:01 266157                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
    7f6ed8c63000-7f6ed8c6b000 r--p 000e8000 08:01 266157                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
    7f6ed8c6b000-7f6ed8c6d000 rw-p 000f0000 08:01 266157                     /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17
    7f6ed8c6d000-7f6ed8c82000 rw-p 00000000 00:00 0 
    7f6ed8c82000-7f6ed8fb8000 r-xp 00000000 08:01 400610                     /usr/local/lib/libmyodbc5a.so
    7f6ed8fb8000-7f6ed91b7000 ---p 00336000 08:01 400610                     /usr/local/lib/libmyodbc5a.so
    7f6ed91b7000-7f6ed9298000 rw-p 00335000 08:01 400610                     /usr/local/lib/libmyodbc5a.so
    7f6ed9298000-7f6ed929e000 rw-p 00000000 00:00 0 
    7f6ed929e000-7f6ed947a000 r-xp 00000000 08:01 400674                     /lib/libPocoFoundation.so.30
    7f6ed947a000-7f6ed967a000 ---p 001dc000 08:01 400674                     /lib/libPocoFoundation.so.30
    7f6ed967a000-7f6ed968a000 rw-p 001dc000 08:01 400674                     /lib/libPocoFoundation.so.30
    7f6ed968a000-7f6ed968b000 rw-p 00000000 00:00 0 
    7f6ed968b000-7f6ed9715000 r-xp 00000000 08:01 400698                     /lib/libPocoXML.so.30
    7f6ed9715000-7f6ed9914000 ---p 0008a000 08:01 400698                     /lib/libPocoXML.so.30
    7f6ed9914000-7f6ed991c000 rw-p 00089000 08:01 400698                     /lib/libPocoXML.so.30
    7f6ed991c000-7f6ed9969000 r-xp 00000000 08:01 400678                     /lib/libPocoJSON.so.30
    7f6ed9969000-7f6ed9b68000 ---p 0004d000 08:01 400678                     /lib/libPocoJSON.so.30
    7f6ed9b68000-7f6ed9b6b000 rw-p 0004c000 08:01 400678                     /lib/libPocoJSON.so.30
    7f6ed9b6b000-7f6ed9bdc000 r-xp 00000000 08:01 400694                     /lib/libPocoUtil.so.30
    7f6ed9bdc000-7f6ed9ddc000 ---p 00071000 08:01 400694                     /lib/libPocoUtil.so.30
    7f6ed9ddc000-7f6ed9de0000 rw-p 00071000 08:01 400694                     /lib/libPocoUtil.so.30
    7f6ed9de0000-7f6ed9ef9000 r-xp 00000000 08:01 400686                     /lib/libPocoNet.so.30
    7f6ed9ef9000-7f6eda0f9000 ---p 00119000 08:01 400686                     /lib/libPocoNet.so.30
    7f6eda0f9000-7f6eda105000 rw-p 00119000 08:01 400686                     /lib/libPocoNet.so.30
    7f6eda105000-7f6eda106000 rw-p 00000000 00:00 0 
    7f6eda106000-7f6eda126000 r-xp 00000000 08:01 399739                     /lib/x86_64-linux-gnu/ld-2.13.so
    7f6eda19a000-7f6eda311000 r--p 00000000 08:01 267804                     /usr/lib/locale/locale-archive
    7f6eda311000-7f6eda31b000 rw-p 00000000 00:00 0 
    7f6eda31b000-7f6eda322000 r--s 00000000 08:01 262813                     /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
    7f6eda322000-7f6eda325000 rw-p 00000000 00:00 0 
    7f6eda325000-7f6eda326000 r--p 0001f000 08:01 399739                     /lib/x86_64-linux-gnu/ld-2.13.so
    7f6eda326000-7f6eda327000 rw-p 00020000 08:01 399739                     /lib/x86_64-linux-gnu/ld-2.13.so
    7f6eda327000-7f6eda328000 rw-p 00000000 00:00 0 
    7fff4e4f9000-7fff4e51a000 rw-p 00000000 00:00 0                          [stack]
    7fff4e5ff000-7fff4e600000 r-xp 00000000 00:00 0                          [vdso]
    ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
    Aborted
    root@debian:/home/poco-1.6.0-all/Net/samples/HTTPTimeServer# 1;2c

     

    关于 

     "*** glibc detected *** ./HTTPTimeServer: free(): invalid next size (fast): 0x000000000167dda0 ***" 

    的错误 网上搜索的结果是

     [日记] 当glibc detected *** free(): invalid next size (normal)出现
    [ 2009年5月27日 星期三 ] shanghui    Yes 326 No 319
    是内存泄露的问题。 
    (1)一般是free了没有分配的内存
    (2)还有就是分配了内存忘记释放也有可能会出现这样的问题。 
    (3)最后查出来是数组循环的时候越界了 , 写到了其他的内存里面, 然后一free那个区域就出现了这样的问题。
    
    是有allocate引起的,一个debug的方法是把你怀疑的矩阵(比如释放了就会报内存错误)定义成静态的,这样如果出现越界,会报段错误。依次检查是否你的数组真的越界了。

    但这个转换代码很简单,不应该是这个错.然后就一直loop的查找...抓狂...

     

     

    最后又看了

    size_t iconv(iconv_t cd,
                        char **inbuf, size_t *inbytesleft,
                        char **outbuf, size_t *outbytesleft);

     

    总觉得问题出在这儿.

    打印出执行出错的返回值和错误码:结果...

    iconv 返回失败,但转换成功了

    iconv  返回-1 erron 84 

     

    iconv错误信息要参见:http://blog.163.com/lqy_super/blog/static/19975102120121065399897/

     

    这个引起了我的疑惑.后面有问网友说 size_t  在x86,x64下代表的长度不一样可能引发问题.这个点醒了我,会不会size_t 引发的问题.

    linux下就是查找错误特别麻烦 .

     

    重新改写了 xxx.cpp

    #include "MyCodeConverter.h"
    
    #include <stdio.h>
    #include <stdlib.h>
    #include <malloc.h>
    
    #include <iosfwd>
    #include <string.h>
    #include <stdarg.h>
    #include <fstream>
    #include <sstream>
    #include <iomanip>  
    
    #include <cassert>
    #include<time.h>
    
    
    #include <errno.h>
    
    
    #ifdef WIN32   //window
    
    
    #include <Windows.h>
    #include <shlwapi.h>
    #include <direct.h>
    
    #include <tchar.h>
    #pragma comment(lib,"Kernel32.lib")
    #pragma comment(lib,"shlwapi.lib")
    
    #elif __linux    //linux
    #include <cassert>
    #include <sys/types.h>
    #include <iconv.h>
    #include <stdio.h>
    
    #endif
    
    
    #include "Poco/Foundation.h"
    #include "Poco/Mutex.h"
    
    using Poco::FastMutex;
    FastMutex g_mutexConverter;
    
    #ifdef WIN32   //window
    
    
    static std::string Gb32ToUtf8(const char * lpszGb32Text)
    {
        int nUnicodeBufLen=MultiByteToWideChar(CP_ACP, 0, lpszGb32Text, -1, 0, 0);
        if (nUnicodeBufLen==0)
            return _T("");
    
        WCHAR* pUnicodeBuf=new WCHAR[nUnicodeBufLen];
        if (pUnicodeBuf==0)
            return _T("");
    
        MultiByteToWideChar(CP_ACP, 0, lpszGb32Text, -1, pUnicodeBuf, nUnicodeBufLen);
    
        int nUtf8BufLen=WideCharToMultiByte(CP_UTF8, 0, pUnicodeBuf, -1, 0, 0, NULL, NULL);
        if (nUtf8BufLen==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        char* pUft8Buf=new char[nUtf8BufLen];
        if (pUft8Buf==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        WideCharToMultiByte(CP_UTF8, 0, pUnicodeBuf, -1, pUft8Buf, nUtf8BufLen, NULL, NULL);
    
        std::string strUtf8=pUft8Buf;
    
        delete[] pUnicodeBuf;
        delete[] pUft8Buf;
    
        return strUtf8;
    }
    
    static std::string Utf8ToGb32(const char * lpszUft8Text)
    {
        int nUnicodeBufLen=MultiByteToWideChar(CP_UTF8, 0, lpszUft8Text, -1, 0, 0);
        if (nUnicodeBufLen==0)
            return _T("");
    
        WCHAR* pUnicodeBuf=new WCHAR[nUnicodeBufLen];
        if (pUnicodeBuf==0)
            return _T("");
    
        MultiByteToWideChar(CP_UTF8, 0, lpszUft8Text, -1, pUnicodeBuf, nUnicodeBufLen);
    
        int nGb32BufLen=WideCharToMultiByte(CP_ACP, 0, pUnicodeBuf, -1, 0, 0, NULL, NULL);
        if (nGb32BufLen==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        char* pGb32Buf=new char[nGb32BufLen];
        if (pGb32Buf==0)
        {
            delete[] pUnicodeBuf;
            return _T("");
        }
    
        WideCharToMultiByte(CP_ACP, 0, pUnicodeBuf, -1, pGb32Buf, nGb32BufLen, NULL, NULL);
    
        std::string strGb32=pGb32Buf;
    
        delete[] pUnicodeBuf;
        delete[] pGb32Buf;
    
        return strGb32;
    }
    #elif __linux    //linux
    
    
    
    
    // 编码转换操作类->基于byte
    //
    class CodeConverterBase {
    
        //示例...
        //std::string strxxxxcvx=json;
        //CodeConverterBase cc2 = CodeConverterBase("gb2312","utf-8");
        //char outbuf[3000];
        //cc2.convert((char *)strxxxxcvx.c_str(),strxxxxcvx.size(),outbuf,1024);
        //strUtf8=outbuf;
    
    
    private:
        iconv_t cd;
    public:
    
        // 构造
        CodeConverterBase(const char *from_charset,const char *to_charset) 
        {
            cd = iconv_open(to_charset,from_charset);
        }
    
        // 析构
        ~CodeConverterBase() {
            iconv_close(cd);
        }
    
        // 转换输出
        size_t convert(char *inbuf,size_t inlen,char *outbuf,size_t outlen) 
        {
            char *pin = inbuf;
            char *pout = outbuf;
            memset(outbuf,0,outlen);
            return iconv(cd,&pin,(size_t *)&inlen,&pout,(size_t *)&outlen);
        }
    };
    
    //字符串转换操作 ->基于string
    class CodeConverterString
    {
    public:
        CodeConverterString()
        {
    
        }
    
        ~CodeConverterString()
        {
    
        }
    
    
        static std::string Gb2312ToUtf8(std::string strSource)
        {
            std::string strRet="";
            if(strSource.size()<=0)
                return strRet;
    
            CodeConverterBase cc2 = CodeConverterBase("gbk","utf-8//TRANSLIT//IGNORE");//  //TRANSLIT//IGNORE 转换失败找相似编码.
            
            size_t nInLen=(strSource.size())+1;
            char * szInBuf=(char *)malloc(nInLen);
    
            size_t nOutLen=(strSource.size())*4+4;
            char * szOutBuf=(char *)malloc(nOutLen);
    
            do 
            {
                if(0==szOutBuf ||0==szInBuf)
                    break;
    
                memset(szInBuf, 0, nInLen);
                memcpy(szInBuf, strSource.c_str(), strSource.size());
    
                size_t nError=cc2.convert((char *)szInBuf,nInLen,szOutBuf,nOutLen);
    
                if(nError<0)
                    std::cout<<"Gb2312ToUtf8 ,nError="<<nError<<",errno ="<<errno <<std::endl;
    
                strRet=szOutBuf;
    
            } while (0);
    
            if(szOutBuf)
            {
                free(szOutBuf);
                szOutBuf=NULL;
            }
    
            if(szInBuf)
            {
                free(szInBuf);
                szInBuf=NULL;
            }
    
            return strRet;
        }
    
        static std::string Utf8ToGb2312(std::string strSource)
        {
            std::string strRet="";
            
            if(strSource.size()<=0)
                return strRet;
    
            CodeConverterBase cc2 = CodeConverterBase("utf-8","gbk//TRANSLIT//IGNORE");//  //TRANSLIT//IGNORE 转换失败找相似编码.
    
            size_t nInLen=(strSource.size())+1;
            char * szInBuf=(char *)malloc(nInLen);
    
            size_t nOutLen=(strSource.size())*4+4;
            char * szOutBuf=(char *)malloc(nOutLen);
    
            do 
            {
                if(0==szOutBuf ||0==szInBuf)
                    break;
                
                memset(szInBuf, 0, nInLen);
                memcpy(szInBuf, strSource.c_str(), strSource.size());
    
                size_t nError=cc2.convert((char *)szInBuf,nInLen,szOutBuf,nOutLen);
                
                if(nError<0)
                    std::cout<<"Utf8ToGb2312 ,nError="<<nError<<",errno ="<<errno <<std::endl;
    
                strRet=szOutBuf;
    
            } while (0);
    
            if(szOutBuf)
            {
                free(szOutBuf);
                szOutBuf=NULL;
            }
    
            if(szInBuf)
            {
                free(szInBuf);
                szInBuf=NULL;
            }
    
            return strRet;
        }
    
    
    };
    
    
    
    
    #endif
    
    
    
    
    
    
    
    
    
    
    
    
    
    CMyCodeConvert::CMyCodeConvert()
    {
    
    }
    
    CMyCodeConvert::~CMyCodeConvert()
    {
    
    }
    
    
    std::string CMyCodeConvert::Gb2312ToUtf8(std::string strSource)
    {
    
        FastMutex::ScopedLock lock(g_mutexConverter);
    #ifdef WIN32   //window
    
        return Gb32ToUtf8(strSource.c_str());
    
    #elif __linux    //linux
    
        return CodeConverterString::Gb2312ToUtf8(strSource);
    #endif
    }
    
    
    
    //
    std::string CMyCodeConvert::Utf8ToGb2312(std::string strSource)
    {
    
        FastMutex::ScopedLock lock(g_mutexConverter);
    #ifdef WIN32   //window
    
        return Utf8ToGb32(strSource.c_str());
    
    #elif __linux    //linux
        
    
        return CodeConverterString::Utf8ToGb2312(strSource);
    #endif
    }

      

     

    测试json代码:

    std::string  suite()
    {
    
    
        std::string json = 
            "{"
            "\"id\": 1123,"
            "\"jsonrpc\": \" 123.abc@hello.com,大圣归来.. \","
            "\"total\": 2,"
            "\"result\": "
            "["
            "{"
            "\"id\": null,"
            "\"picture\": \"http://placehold.it/32x32\","
            "\"name\": \"这是中文测试.Crossman\","
            "\"about\": \"《西游记之大圣归来》是根据中国传统神话故事进行拓展和演绎的3D动画电影.xxxx Consectetuer suscipit volutpat eros dolor .\","
            "\"friends\": "
            "["
            "{"
            "},"
            "{"
            "\"id\": 2,"
            "\"name\": \"Bailey Oldridge2\""
            "},"
            "{"
            "\"id\": 3,"
            "\"name\": \"Makayla Campbell3\""
            "}"
            "]"
            "},"
            "{"
            "\"id\": 2,"
            "\"picture\": \"http://placehold.it/32x32\","
            "\"name\": \"2中名 欢迎 \","
            "\"about\": \"2北京你好 dolor .\","
            "\"friends\": "
            "["
            "]"
            "}"
            "]"
            "}";
    
        //json = 
        //    "{"
        //    "\"id\": 1123,"
        //    "\"jsonrpc\": \" 123.abc@hello.com,成都市锦江区. \""
        //    "}";
    
        std::string strUtf8;
    
        
        std::cout<<"+++++++++++++++++++++++++++++++++++++++++++++++++++"<<std::endl;
        std::cout<<"json="<<json<<std::endl;
    
        std::cout<<"json hex:"<<std::endl;
        std::string strxxxx=json;
        for(int nxxx=0;nxxx<strxxxx.size();++nxxx)
        {
            unsigned char chTemp=strxxxx[nxxx];
            std::string strTemp=Poco::format("%hX ",(unsigned short )chTemp);//输出16进制
    
            std::cout<<strTemp;
        }
        std::cout<<std::endl;
    
        
        for(int i=0;i<20;++i)
        {
            std::cout<<"++"<<i<<std::endl;
            strUtf8=CMyCodeConvert::Gb2312ToUtf8(json);//多 次测试 转换..
        }
        
    
    
        std::cout<<std::endl;
    
        std::cout<<"+++++++++++++++++++++++++++++++++++++++++++++++++++"<<std::endl;
        std::cout<<"strUtf8="<<strUtf8<<std::endl;
        
    
        std::cout<<"strUtf8 hex:"<<std::endl;
        strxxxx=strUtf8;
        for(int nxxx=0;nxxx<strxxxx.size();++nxxx)
        {
            unsigned char chTemp=strxxxx[nxxx];
            std::string strTemp=Poco::format("%hX ",(unsigned short )chTemp);//输出16进制
    
            std::cout<<strTemp;
        }
    
        std::cout<<std::endl;
    
        Parser parser;
        Var result;
    
        try
        {
            result = parser.parse(strUtf8);//json转换...
        }
        catch(JSONException& jsone)
        {
            std::cout << jsone.message() << std::endl;
            return "result==NULL";
        }
        
    
        Object::Ptr object = result.extract<Object::Ptr>();
        int nObjectSize=object->size();
    
        //assert(object->size() > 0);
        DynamicStruct ds = *object;
    
        //遍历...
        for(DynamicStruct::Iterator  itBegin=ds.begin();itBegin!=ds.end();itBegin++)
        {
            //std::string strvalue=itBegin;
            //Var var= itBegin->second;
            //std::cout<<"type:"<<var.type()<<std::endl;
    
            std::cout<<"K:"  << itBegin->first << " , value: " <<CMyCodeConvert::Utf8ToGb2312( itBegin->second.toString()) << std::endl;
        }
    
        std::cout<<std::endl<<std::endl<<std::endl;
        std::cout<<"+++++++++++++++++++++++++++++++++++++++++++++++++++"<<std::endl;
    
    
        json="";
    
        std::string strlation1xxxx;
        std::string strjstostring=strUtf8;
        std::string strTemp;
        std::string strKey;
    
    
    
    
        //std::string strKey="id";
    
        //key 是否存在
        strKey="id";
        if(object->has(strKey))
        {
            strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
            std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
        }
    
        strKey="id2232";
        if(object->has(strKey))
        {
            strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
            std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
        }
    
        strKey="jsonrpc";
        if(object->has(strKey))
        {
            strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
            std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
    
            std::string strxxx;
            
            std::cout<<"jsonrpc 编码:"<<std::endl;
            std::string strxxxx=strTemp;
            for(int nxxx=0;nxxx<strxxxx.size();++nxxx)
            {
                unsigned char chTemp=strxxxx[nxxx];
                std::string strTemp=Poco::format("%hX",(unsigned short )chTemp);
                strTemp+=" ";
                //std::cout<<strTemp;
            }
    
            std::cout<<std::endl;
    
            //strjstostring+=strTemp;
    
    
            std::cout<<"object333->get("<<strKey<<")="<<strTemp<<std::endl;
        }
    
        strKey="total";
        if(object->has(strKey))
        {
            strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
            std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
        }
    
    
        strKey="result";
        if(object->has(strKey))
        {
            strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
            std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
    
                
            Var  vResult=object->get("result");
    
            Poco::JSON::Array::Ptr arrayResult = object->getArray("result");
            Poco::JSON::Array::Ptr arr22222 = vResult.extract<Poco::JSON::Array::Ptr>();
    
            Poco::Dynamic::Array dsarrayResult = *arrayResult;
    
            std::cout<<"arr22222->size()"<<arr22222->size()<<std::endl;
            std::cout<<"arrayResult->size()"<<arrayResult->size()<<std::endl;
            std::cout<<"arrayResult->size()"<<dsarrayResult.size()<<std::endl;
    
    
            for(int nIndex=0;nIndex<arrayResult->size();++nIndex)
            {
                Object::Ptr object = arrayResult->getObject(nIndex);
    
    
                strKey="id";
                if(object->has(strKey))
                {
                    strTemp="";
                    if(!object->get(strKey).isEmpty())
                    {
                        strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
                        std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
                    }
    
                }
    
                strKey="picture";
                if(object->has(strKey))
                {
                    strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
                    std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
                }
    
                strKey="name";
                if(object->has(strKey))
                {
                    strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
                    std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
                }
    
                strKey="about";
                if(object->has(strKey))
                {
                    strTemp=CMyCodeConvert::Utf8ToGb2312( object->get(strKey).toString());
    
                    std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
                }
    
                strKey="friends";
                if(object->has(strKey))
                {
                    strTemp=object->get(strKey).toString();
    
                    std::cout<<"object->get("<<strKey<<")="<<strTemp<<std::endl;
    
                    Poco::JSON::Array::Ptr arrayFriends = object->getArray(strKey);
                    for(int nIndex=0;nIndex<arrayFriends->size();++nIndex)
                    {
                        Object::Ptr objectFriends = arrayFriends->getObject(nIndex);
                        DynamicStruct dsTemp = *objectFriends;
    
                        for(DynamicStruct::Iterator  itBegin=dsTemp.begin();itBegin!=dsTemp.end();itBegin++)
                        {
                            std::cout<<"K:"  << itBegin->first << " , value: " << CMyCodeConvert::Utf8ToGb2312(itBegin->second.toString() )<< std::endl;
                        }
    
                    }
                }
    
    
    
                std::cout<<std::endl<<std::endl<<std::endl;
                std::cout<<"+++++++++++++++++++++++++++++++++++++++++++++++++++"<<std::endl;
    
                std::cout<<std::endl<<std::endl<<std::endl;
    
            }
    
    
    
    
        }
    
        return strjstostring;
    }

     

    处理 成功.

     

    至此poco::json 库已经能正常运行且能处理 utf8编码的中文字符。

    教训

    在linux 下的api

    size_t iconv(iconv_t cd,
                        char **inbuf, size_t *inbytesleft,
                        char **outbuf, size_t *outbytesleft);

     

    一定要照着参数原型写.

     

    哎...今天够累的,签到来了1...
    回复

    使用道具 举报

    您需要登录后才可以回帖 登录 | 立即注册

    本版积分规则

    QQ|手机版|小黑屋|Java自学者论坛 ( 声明:本站文章及资料整理自互联网,用于Java自学者交流学习使用,对资料版权不负任何法律责任,若有侵权请及时联系客服屏蔽删除 )

    GMT+8, 2021-12-4 13:27 , Processed in 0.072658 second(s), 29 queries .

    Powered by Discuz! X3.4

    Copyright © 2001-2020, Tencent Cloud.

    快速回复 返回顶部 返回列表