What can I do about an ArithmeticException in DSBulk: Cannot convert from BigDecimal to Float?

Question

question

anson asked Mar 30, '21 Erick Ramirez answered Mar 31, '21

What can I do about an ArithmeticException in DSBulk: Cannot convert from BigDecimal to Float?

Hi all,

I am trying to migrate data from sqlite3 to cassandra. The flow is that I am creating csv out of sqlite3 data and uploading that csv to cassandra using dsbulk loader. I have made sure the schema of sqlite3 is compatible with cassandra.

The scenarioi is that in the sqlite3 , i have a column with FLOAT datatype and then converting that sqlite3 files with the specified FLOAT column to csv . The resulted csv has one of the column value (FLOAT) in that as 1599.6000000000001 . On using this csv in dsbulk loader, i am getting this exception,

java.lang.ArithmeticException: Cannot convert 1599.6000000000001 from BigDecimal to Float
    at com.datastax.oss.dsbulk.codecs.api.util.CodecUtils.conversionFailed(CodecUtils.java:602)
    at com.datastax.oss.dsbulk.codecs.api.util.CodecUtils.toFloatValueExact(CodecUtils.java:529)
    at com.datastax.oss.dsbulk.codecs.api.util.CodecUtils.convertNumber(CodecUtils.java:325)
    at com.datastax.oss.dsbulk.codecs.api.util.CodecUtils.narrowNumber(CodecUtils.java:183)
    at com.datastax.oss.dsbulk.codecs.text.string.StringToNumberCodec.narrowNumber(StringToNumberCodec.java:91)
    at com.datastax.oss.dsbulk.codecs.text.string.StringToFloatCodec.externalToInternal(StringToFloatCodec.java:66)
    at com.datastax.oss.dsbulk.codecs.text.string.StringToFloatCodec.externalToInternal(StringToFloatCodec.java:33)
    at com.datastax.oss.dsbulk.codecs.api.ConvertingCodec.encode(ConvertingCodec.java:70)
    at com.datastax.oss.dsbulk.workflow.commons.schema.DefaultRecordMapper.bindColumn(DefaultRecordMapper.java:160)
    at com.datastax.oss.dsbulk.workflow.commons.schema.DefaultRecordMapper.map(DefaultRecordMapper.java:134)

The column of sqlite3 FLOAT is mapped in cassandra as FLOAT ( in the cassandra schema) too.

Why is this happening? Will Cassandra automatically convert the values accordingly? How to avoid this without changing the datatype?

Any help would be appreciated

Thanks

dsbulk

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

DataStax Enterprise is powered by the best distribution of Apache Cassandra ™

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Privacy Policy Terms of Use

Answer 1 · 2021-03-31T00:28:28Z

Erick Ramirez answered Mar 31, '21

The issue occurs because the value 1599.6000000000001 doesn't fit into a float. The Java floating point is only 32 bits and has less precision than a double type.

You can specify an overflow strategy in DSBulk with the --codec.overflowStrategy flag to truncate the data to fit in into the target CQL type:

$ dsbulk [options] --codec.overflowStrategy TRUNCATE

Note that you'll be losing precision when you use this strategy.

For details, see the DSBulk Codec options. Cheers!

Share

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

question