Problem

You may encounter an issue when trying to connect to Sybase from your production environment.

org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 4) (10.244.90.50 executor 1): com.sybase.jdbc4.jdbc.SybSQLException: Incorrect syntax near 'LIMIT'

Cause

The keyword LIMIT is not recognized in Sybase by default, causing SQL queries that include LIMIT to fail when executed against a Sybase database.

Solution

Set LIMIT as a reserved keyword in Sybase.

SET OPTION public.RESERVED_KEYWORDS = 'LIMIT';

For more information, please refer to the Sybase reserved_keywords option documentation.

If you cannot set a Sybase keyword, you can work around the issue by disabling the limit pushdown optimization in Apache Spark. Set the jdbcDSv1LimitPushdown.enabled value to false in your compute cluster's Spark config.

spark.databricks.optimizer.jdbcDSv1LimitPushdown.enabled false

After disabling the limit, restart your compute cluster and retry the connection to Sybase.