Solr DataImportHandler CachedSqlEntityProcessor ClassCastException

classcastexception, dataimporthandler, solr

Solution

CachedSqlEntityProcessor relies on DIHCacheSupport(`Map<String,Object> getIdCacheData(...)` ) that is not adapted to work with keys of type Integer, which is expected behavior for any cache. (`Object key = context.resolve(cacheForeignKey);`, this key should really be of type String)

SELECT CAST(id as CHAR(32)) AS CID,full_name FROM sm_conferences 

Problem

I am using `Solr 4.6.0` and trying to import my data using `CachedSqlEntityProcessor`, but somehow I end up getting a `ClassCastException`. Schema ``` <fields> <field name="_version_" type="long" indexed="true" stored="true"/> <field name="id" type="int" indexed="true" stored="true" required="true" multiValued="false" /> <field name="conference" type="string" indexed="true" stored="true" /> <field name="year" type="int" indexed="true" stored="true" /> <field name="doi" type="string" indexed="false" stored="true" /> <field name="text" type="text_en_shingling" indexed="true" stored="true" /> </fields> ``` Data config ``` <dataConfig> <dataSource name="mysql" type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="..." batchSize="-1" user="..." password="..." /> <document name="publications"> <entity name="publication" query=" SELECT sm_publications.id AS p_id, sm_publications.year AS p_year, sm_publications.doi AS p_doi, CONCAT(sm_publications.title, ' ', sm_publications.abstract) AS p_text, sm_publications.conference_id FROM sm_publications WHERE '${dataimporter.request.clean}' != 'false' OR modified > '${dataimporter.last_index_time}'"> <field column="p_id" name="id" /> <field column="p_text" name="text" /> <field column="p_year" name="year" /> <field column="p_doi" name="doi" /> <entity name="conference" query="SELECT id AS CID,full_name FROM sm_conferences" processor="CachedSqlEntityProcessor" cacheKey="CID" cacheLookup="publication.conference_id"> <field column="full_name" name="conference" /> </entity> </entity> </document> </dataConfig> ``` Error message ``` Full Import failed:java.lang.RuntimeException: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:270) at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:476) at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:179) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:410) at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:323) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:231) ... 33 more Caused by: org.apache.solr.handler.dataimport.DataImportHandlerException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String at org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:63) at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:246) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:469) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:495) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:408) ... 35 more Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String at java.lang.String.compareTo(Unknown Source) at java.util.TreeMap.getEntry(Unknown Source) at java.util.TreeMap.get(Unknown Source) at org.apache.solr.handler.dataimport.SortedMapBackedCache.iterator(SortedMapBackedCache.java:147) at org.apache.solr.handler.dataimport.DIHCacheSupport.getIdCacheData(DIHCacheSupport.java:179) at org.apache.solr.handler.dataimport.DIHCacheSupport.getCacheData(DIHCacheSupport.java:145) at org.apache.solr.handler.dataimport.EntityProcessorBase.getNext(EntityProcessorBase.java:129) at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:75) at org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243) ... 38 more ``` Table structure

Original source