Bringing together the Apache Cassandra experts from the community and DataStax.

Want to learn? Have a question? Want to share your expertise? You are in the right place!

Not sure where to begin? Getting Started

 

question

tecles_192911 avatar image
tecles_192911 asked tecles_192911 commented

Need help with NoSQLBench's CharBufImage() binding function

Hi,

I have used NoSQLBench a bit and it is a great tool. Now, I am trying to use the binding function CharBufImage(), but I must be doing something wrong.

The idea is to insert lines containing long texts in a Cassandra table (I'm using DSE version 6.8.6). A doc size of 20 is being used here, but the tests will require 100K+ characters/doc. I've already used function AlphaNumericString(int: length) to generate the docs, and it works, but since there's no need for a different doc for each line at this point, CharBufImage() seemed a lighter option, and that would be very helpful.

Here is the test schema:

CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'} AND durable_writes = true;
CREATE TABLE test.docs (
    id uuid,
    doc text,
    PRIMARY KEY (id)
);

Here is the yaml file (I'm using NoSQLBench 4.15.51):

# test-charbuf.yaml
#
description: Test CharBufImage usage.
scenarios:
 default: run driver=cql hosts=srv cycles=1..10
 test: run driver=stdout cycles=1..10
bindings:
 doc: CharBufImage("a-z",20) -> java.nio.CharBuffer
blocks:
 - name: block_1
 tags:
 phase: insert
 params:
 cl: ONE
 statements:
 - cmd1: |
 insert into test.docs (id, doc) values ( uuid(), '{doc}' );

I run the two scenarios this way:

/opt/nb/nb test-charbuf test
/opt/nb/nb test-charbuf

With the test scenario, which uses driver stdout, the output is as expected.

ze@local:~/tst$ /opt/nb/nb test-charbuf test
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );

However, with the default scenario, which uses the CQL driver, I get the following (the host address has been masked off).

ze@local:~/tst$ /opt/nb/nb test-charbuf
[scenarios:001] INFO com.datastax.driver.core - DataStax Java Driver 1.9.0 for DataStax Enterprise (DSE) and Apache Cassandra®
[scenarios:001] INFO com.datastax.driver.core.GuavaCompatibility - Detected Guava >= 19 in the classpath, using modern compatibility layer
[scenarios:001] INFO com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.
[scenarios:001] INFO com.datastax.driver.core.utils.UUIDs - PID obtained through native call to getpid(): 43453
[scenarios:001] INFO com.datastax.driver.core.NettyUtil - Found Netty's native epoll transport in the classpath, using it
[scenarios:001] INFO com.datastax.driver.core.policies.DCAwareRoundRobinPolicy - Using data-center name 'dc1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
[scenarios:001] INFO com.datastax.driver.core.Cluster - New Cassandra host srv/xxxxxxxxxxxxxxx:9042 added
 6092 ERROR [testcharbuf_default_001:001] CoreMotor Error in core motor loop:java.lang.RuntimeException: Binding error:BindingsTemplate:'doc'=>"CharBufImage("a-z",20) -> java.nio.CharBuffer"=>[xqaoxvdxybvcumaiddqu](HeapCharBufferR)
java.lang.RuntimeException: Binding error:BindingsTemplate:'doc'=>"CharBufImage("a-z",20) -> java.nio.CharBuffer"=>[xqaoxvdxybvcumaiddqu](HeapCharBufferR)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:41)
    at io.nosqlbench.activitytype.cql.statements.core.ReadyCQLStatement.bind(ReadyCQLStatement.java:51)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runPhase(CqlAction.java:106)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runCycle(CqlAction.java:85)
    at io.nosqlbench.engine.api.activityimpl.motor.CoreMotor.run(CoreMotor.java:405)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
    at java.base/java.util.Arrays$ArrayList.get(Unknown Source)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:49)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:12)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:39)
 ... 7 more
 6104 WARN [scenarios:001] SCENARIO Error in scenario, shutting down.
 6153 ERROR [main] ERRORHANDLER Error from driver or included library: java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
java.util.concurrent.ExecutionException: java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
    at java.base/java.util.concurrent.FutureTask.report(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.get(Unknown Source)
    at io.nosqlbench.engine.core.script.ScenariosExecutor.getAsyncResultStatus(ScenariosExecutor.java:150)
    at io.nosqlbench.engine.core.script.ScenariosExecutor.awaitAllResults(ScenariosExecutor.java:115)
    at io.nosqlbench.engine.core.script.ScenariosExecutor.awaitAllResults(ScenariosExecutor.java:73)
    at io.nosqlbench.engine.cli.NBCLI.run(NBCLI.java:405)
    at io.nosqlbench.engine.cli.NBCLI.main(NBCLI.java:69)
Caused by: java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
    at io.nosqlbench.engine.core.script.Scenario.runScenario(Scenario.java:307)
    at io.nosqlbench.engine.core.script.Scenario.call(Scenario.java:362)
    at io.nosqlbench.engine.core.script.Scenario.call(Scenario.java:56)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
    at io.nosqlbench.engine.core.lifecycle.ActivityExecutor.notifyException(ActivityExecutor.java:526)
    at io.nosqlbench.engine.core.lifecycle.ActivityExceptionHandler.uncaughtException(ActivityExceptionHandler.java:30)
    at java.base/java.lang.Thread.dispatchUncaughtException(Unknown Source)
Caused by: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
 ... 3 more
Caused by: java.lang.RuntimeException: Binding error:BindingsTemplate:'doc'=>"CharBufImage("a-z",20) -> java.nio.CharBuffer"=>[xqaoxvdxybvcumaiddqu](HeapCharBufferR)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:41)
    at io.nosqlbench.activitytype.cql.statements.core.ReadyCQLStatement.bind(ReadyCQLStatement.java:51)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runPhase(CqlAction.java:106)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runCycle(CqlAction.java:85)
    at io.nosqlbench.engine.api.activityimpl.motor.CoreMotor.run(CoreMotor.java:405)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
    at java.base/java.util.Arrays$ArrayList.get(Unknown Source)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:49)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:12)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:39)
 ... 7 more
 6155 ERROR [main] ERRORHANDLER cause (see stack trace for details):javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
 6156 WARN [Thread-1] SCENARIO Scenario was interrupted by process exit, shutting down
nosqlbench
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

1 Answer

jshook_49093 avatar image
jshook_49093 answered tecles_192911 commented

@tecles_192911,
The first thing you want to do is change

insert into test.docs (id, doc) values ( uuid(), '{doc}' );

to

insert into test.docs (id, doc) values ( uuid(), {doc} );

Since, with the single quotes, the value is parsed as a literal value.

The next error you may see is that there is no codec to map from CharBuffer to varchar, so you can add a ToString() in your binding to look like this:

bindings:
 doc: CharBufImage("a-z",20); ToString();

This will cause the CharBuf to be converted before being presented to the driver's codec mapping layer.

Then you will get results like this:

cqlsh> select * from test.docs ;

 id                                   | doc
--------------------------------------+----------------------
 e4ecf429-7ce4-4705-a031-6ace1de887ce | xqaoxvdxybvcumaiddqu
 52ed6fb9-1d97-44b0-ad38-60721f16a46e | xqaoxvdxybvcumaiddqu
 5e2487c4-79ec-43fb-adee-9249fda1e7a5 | xqaoxvdxybvcumaiddqu
 695bb3ca-d3ff-4d25-a5ca-969217f4dc0b | xqaoxvdxybvcumaiddqu
 ...


As an additional improvement, I would highly recommend you create a bigger image and use the argument form for CharBufImage which extracts a random offset, like this:

bindings:
 doc: CharBufImage("a-z",1000,20); ToString();

Then, you'll see some pseudo-randomness in your data, like this:

cqlsh> select * from test.docs ;

 id                                   | doc
--------------------------------------+----------------------
 35477187-42fc-4305-94b8-e406121ec103 | wmnwdytzlacqoqjmhdbj
 999ee417-d2e2-49ea-ae0d-1c07cbc73d1c | zwfwmnwdytzlacqoqjmh
 94422128-8adb-4e1b-a775-c7e54ac259fa | epxmqxmrbiuzzvzufzcq
 ...

Of course, you'll want to flavor this to your test requirements based on how much variety you need to simulate.

Happy Testing!

1 comment Share
10 |1000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

tecles_192911 avatar image tecles_192911 ·

Thank you, @jshook_49093. Clear and precise.

The test runs perfectly now and I will follow your recommendation.

I really appreciated your help!

1 Like 1 ·