JDBCConnector crawl DELETE by %TIMESTAMPSEC%, unable to append/update last crawl time - "WARN  [JDBCAdapter] Error setting LastCrawlDate"

So here are my settings in the crawl configuration file:

1. While JDBCSQL is empty, JDBCDeleteSQL does contain a query that returns items to be deleted. The query returns two columns, one being the primary key of documents (field1) and the other being the timestamp in seconds (field2).

<parameter name="JDBCSQL" type="string">
   <value></value>
 </parameter>

<parameter name="JDBCDeleteSQL" type="string">

   <value><![CDATA[exec GetContentsToDelete @FromTimeStampInSeconds = %TIMESTAMPSEC%]]></value>
</parameter>

2. <parameter name="JDBCDeleteTimeStampField" type="string">
    <value>field2</value>
   </parameter>

3. <parameter name="JDBCDeleteTimeType" type="string">
 <value>seconds</value>
   </parameter>

4. <parameter name="JDBCUpdateTimeStampOnDeletion" type="boolean">
 <description>
 <![CDATA[should the delete timestamp file still needs to be updated upon deletion <br>Default: false]]>
 </description>
 <value>true</value>
   </parameter>

5. <parameter name="JDBCPrimaryKeyField" type="string">
 <value>field1</value>
   </parameter>
   <parameter name="JDBCDeletePrimaryKeyField" type="string">
 <value>field1</value>
   </parameter>

6. <parameter name="OperationMode" type="string">
 <description>
 <![CDATA[Must be one of add, or update.
  Update should only be used if you want to do a partial update on already existing documents, i.e. during an incremental crawl. 
  Default: (add)]]>
 </description>
 <valid-values>add;update</valid-values>
 <value>update</value>
   </parameter>

#1- When performing the crawl to DELETE documents from FAST, the %TIMESTAMPSEC% is not getting picked from the JDBCLastCrawlTime.txt file and the query to the DB fails. #2- To bypass this, I replaced the %TIMESTAMPSEC% with a direct value, and the deletion of content works, however the jdbcconnector is unable to update the last crawl time to the JDBCLastCrawlTime.txt file in the WorkFolder.

ERROR Log in #1 crawl:

19:02:11,058 INFO  [JDBCAdapter] Running delete SQL: exec GetContentsToDelete @FromTimeStampInSeconds = %TIMESTAMPSEC%
19:02:11,105 ERROR [JDBCAdapter] Error when crawling the source system: Unable to execute JDBC crawl.
19:02:11,105 ERROR [JDBCAdapter] Caused by: Error while executing SQL: exec GetContentsToDelete @FromTimeStampInSeconds = %TIMESTAMPSEC%
19:02:11,105 ERROR [JDBCAdapter] Caused by: Error while executing SQL prepared statement:exec GetContentsToDelete @FromTimeStampInSeconds = %TIMESTAMPSEC%
19:02:11,105 ERROR [JDBCAdapter] Caused by: Incorrect syntax near '%'.
19:02:11,105 ERROR [JDBCAdapter] Adapter thread will terminate....

ERROR Log in #2 crawl

2014-12-01 16:33:42,314 INFO    [JDBCAdapter] Running postSQL
2014-12-01 16:33:42,314 WARN  [JDBCAdapter] Error setting LastCrawlDate
2014-12-01 16:33:42,314 WARN  [JDBCAdapter] Warning when crawling the source system: java.lang.NullPointerException
2014-12-01 16:33:42,314 INFO    [JDBCAdapter] Finished running postSQL

Any ideas on what could be the problem? Am sure I'm overlooking something very simple. Look forward to any help or suggestions.


  • Edited by Ramkumar-007 Tuesday, December 02, 2014 5:38 AM formatting debug
December 2nd, 2014 4:14am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics