有 Java 编程相关的问题?

你可以在下面搜索框中键入要查询的问题!

java Solr FieldStreamDataSource抛出“不支持的类型:字符串”(非空)

(编辑:ubuntu上的solr-6.6.0)

我试图使用Solr的DataImportHandler为MySQL数据库编制索引,该数据库包含带有RTF文件的blob条目。为此,我使用了以下答案中指定的FieldStreamDataSource

How do I index Rich Format Documents in Blobs

Unsupported type Exception on Importing Documents from Database

当所有其他非Blob字段都被索引时,我一直从FieldStreamDataSource获取以下Java运行时错误。getData()方法,如Solr日志文件中所示:

Caused by: java.lang.RuntimeException: unsupported type : class java.lang.String at org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:77) at org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:47) at org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:187)

它引用了Java class的这些行(请参见箭头“<;=”):

  @Override
  public InputStream getData(String query) {
    Object o = wrapper.getVariableResolver().resolve(dataField);
    if (o == null) {
      throw new DataImportHandlerException(SEVERE, "No field available for name : " + dataField);
    } else if (o instanceof Blob) {                                     // <========= XXX
      Blob blob = (Blob) o;
      try {
        return blob.getBinaryStream();
      } catch (SQLException sqle) {
        LOG.info("Unable to get data from BLOB");
        return null;
      }
    } else if (o instanceof byte[]) {
      byte[] bytes = (byte[]) o;
      return new ByteArrayInputStream(bytes);
    } else {
      throw new RuntimeException("unsupported type : " + o.getClass()); // <========= XXX
    } 

这意味着getData()方法获取的不是Blob类型,而是字符串

我知道在许多其他线程中有人建议,当数据库中的blob值为null时,会出现此错误:

Unsupported type Exception on Importing Documents from Database

Solr DIH Throwing Error Unsupported Type Class Java.Lang.String

然而,这里的情况并非如此,因为我在这个测试数据库中只有5个条目,其中的非条目有一个空值。 此外,我正在检查Solr DIH接口的调试输出,这表明Blob值确实是从DB中检索到的(请参见箭头“<;=”):

"verbose-output": [
        "entity:reports1",
        [
          "document#1",
          [
            "query",
            "SELECT id, institute, exam_date, age, acc, pacs FROM reports",
            "time-taken",
            "0:0:0.8",
            null,
            "----------- row #1-------------",
            "id",
            "1",
            "institute",
            "RADIOLOGY",
            "age",
            "68",
            "acc",
            "165184654",
            "pacs",
            "233215",
            "exame_date",
            "2016-02-05T00:00:00Z",
            null,
            "---------------------------------------------",
            "entity:reports2",
            [
              "query",
              "SELECT report FROM reports WHERE id='1'",
              "time-taken",
              "0:0:0.6",
              null,
              "----------- row #1-------------",
              "report",                   //   <=========   COLUMN NAME RETURNED FROM THE SQL SELECT
              "e1xydGYxXGFkZWZ[...]",    //   <=========   VALUE RETURNED FROM THE SQL SELECT
              null,
              "---------------------------------------------",
              "entity:report",
              [
                "query",
                "report",
                "EXCEPTION",              // <==========   EXCEPTION THROWN
                "java.lang.RuntimeException: unsupported type : class java.lang.String\n\tat org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:77)\n\tat org.apache.solr.handler.dataimport.FieldStreamDataSource.getData(FieldStreamDataSource.java:47)\n\tat org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:187)\n\tat org.apache.solr.handler.dataimport.TikaEntityProcessor.nextRow(TikaEntityProcessor.java:128)\n\tat org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:267)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:475)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:516)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:516)\n\tat org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)\n\tat org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)\n\tat org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:232)\n\tat org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:415)\n\tat org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:474)\n\tat org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:180)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat java.lang.Thread.run(Thread.java:748)\n",
                    "time-taken",
                    "0:0:0.0"
                  ]
                ]
              ],

这是我的数据配置。xml

<dataConfig>

        <dataSource 
            name="db"
            type="JdbcDataSource" 
            driver="com.mysql.jdbc.Driver"
            url="jdbc:mysql://localhost:3306/RIS" 
            user="root" 
            password="********"/>

        <dataSource name="fieldStream" type="FieldStreamDataSource"/>

        <document>
            <entity
                name="reports1"
                query="SELECT id, institute, exam_date, age, acc, pacs FROM reports"
                dataSource="db"
            >
                <field column="id"           name="id"/>
                <field column="institute"    name="institute"/>
                <field column="exam_date"    name="exam_date"/>
                <field column="age"          name="age"/>
                <field column="acc"          name="acc"/>
                <field column="pacs"         name="pacs"/>


                <entity
                    name="reports2"
                    query="SELECT report FROM reports WHERE id='${reports1.id}'"
                    dataSource="db"
                >
                    <entity 
                        name="report" 
                        dataSource="fieldStream"
                        processor="TikaEntityProcessor"
                        url="report"
                        dataField="reports2.REPORT"
                        format="text"
                        onError="continue">
                        <field column="text" name="report"/>
                    </entity>
                </entity>

            </entity>

        </document>
</dataConfig>

因此,在我看来,从DB的blob列检索到的值被传递或计算为字符串,而不是blob类型

非常感谢


共 (1) 个答案

  1. # 1 楼答案

    我发现了(相当琐碎的)问题(尽管我已经尝试了几天)

    将dataField属性更改为小写就可以了,这与this答案的经验相反。对数据配置进行了以下更改(请参见箭头“<;==”)。xml,所有条目都被索引

    <entity
        name="reports2"
        query="SELECT report FROM reports WHERE id='${reports1.id}'"
        dataSource="db">
           <entity 
               name="report" 
               dataSource="fieldStream"
               processor="TikaEntityProcessor"
               url="report"
               dataField="reports2.report"  //    <===== instead of reports2.REPORT
               format="text"
               onError="continue">
    
               <field column="text" name="report"/>
           </entity>
    

    然而,这确实给了我一个不相关的问题,可能值得指出(不确定是否在另一个答案中)

    因为从数据库返回的列(“报告”)与Solr模式字段(“报告”)同名,所以索引自动发生在SQL查询之后,为BLOB中的二进制值编制索引。它忽略了TikaEntityProcessor在以下实体中提取的文本。我通过确保这两个字段(MySQL和Solr)具有不同的名称来修复此问题

    下面的箭头(“<;=”)显示了索引发生的位置(早于计划)

    <entity
        name="reports2"
        query="SELECT report FROM reports WHERE id='${reports1.id}'" // <========  DIH indexed this column (report) from the DB into the Solr field of same name (binary representation)
        dataSource="db">
           <entity 
               name="report" 
               dataSource="fieldStream"
               processor="TikaEntityProcessor"
               url="report"
               dataField="reports2.report"
               format="text"
               onError="continue">
    
               <field column="text" name="report"/>  //<======== instead of waiting for this column (text), the output from Tika (extracted text)
           </entity>
    </entity>