Need help with NoSQLBench's CharBufImage() binding function

Question

This question was closed Nov 03, 2021 at 11:38 PM by Erick Ramirez for the following reason: The question is answered, right answer was accepted

question

tecles_192911 asked Oct 28, '21 tecles_192911 commented Nov 3, '21

Need help with NoSQLBench's CharBufImage() binding function

Hi,

I have used NoSQLBench a bit and it is a great tool. Now, I am trying to use the binding function CharBufImage(), but I must be doing something wrong.

The idea is to insert lines containing long texts in a Cassandra table (I'm using DSE version 6.8.6). A doc size of 20 is being used here, but the tests will require 100K+ characters/doc. I've already used function AlphaNumericString(int: length) to generate the docs, and it works, but since there's no need for a different doc for each line at this point, CharBufImage() seemed a lighter option, and that would be very helpful.

Here is the test schema:

CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '1'} AND durable_writes = true;
CREATE TABLE test.docs (
    id uuid,
    doc text,
    PRIMARY KEY (id)
);

Here is the yaml file (I'm using NoSQLBench 4.15.51):

# test-charbuf.yaml
#
description: Test CharBufImage usage.
scenarios:
 default: run driver=cql hosts=srv cycles=1..10
 test: run driver=stdout cycles=1..10
bindings:
 doc: CharBufImage("a-z",20) -> java.nio.CharBuffer
blocks:
 - name: block_1
 tags:
 phase: insert
 params:
 cl: ONE
 statements:
 - cmd1: |
 insert into test.docs (id, doc) values ( uuid(), '{doc}' );

I run the two scenarios this way:

/opt/nb/nb test-charbuf test
/opt/nb/nb test-charbuf

With the test scenario, which uses driver stdout, the output is as expected.

ze@local:~/tst$ /opt/nb/nb test-charbuf test
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );
insert into test.docs (id, doc) values ( uuid(), 'xqaoxvdxybvcumaiddqu' );

However, with the default scenario, which uses the CQL driver, I get the following (the host address has been masked off).

ze@local:~/tst$ /opt/nb/nb test-charbuf
[scenarios:001] INFO com.datastax.driver.core - DataStax Java Driver 1.9.0 for DataStax Enterprise (DSE) and Apache Cassandra®
[scenarios:001] INFO com.datastax.driver.core.GuavaCompatibility - Detected Guava >= 19 in the classpath, using modern compatibility layer
[scenarios:001] INFO com.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.
[scenarios:001] INFO com.datastax.driver.core.utils.UUIDs - PID obtained through native call to getpid(): 43453
[scenarios:001] INFO com.datastax.driver.core.NettyUtil - Found Netty's native epoll transport in the classpath, using it
[scenarios:001] INFO com.datastax.driver.core.policies.DCAwareRoundRobinPolicy - Using data-center name 'dc1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
[scenarios:001] INFO com.datastax.driver.core.Cluster - New Cassandra host srv/xxxxxxxxxxxxxxx:9042 added
 6092 ERROR [testcharbuf_default_001:001] CoreMotor Error in core motor loop:java.lang.RuntimeException: Binding error:BindingsTemplate:'doc'=>"CharBufImage("a-z",20) -> java.nio.CharBuffer"=>[xqaoxvdxybvcumaiddqu](HeapCharBufferR)
java.lang.RuntimeException: Binding error:BindingsTemplate:'doc'=>"CharBufImage("a-z",20) -> java.nio.CharBuffer"=>[xqaoxvdxybvcumaiddqu](HeapCharBufferR)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:41)
    at io.nosqlbench.activitytype.cql.statements.core.ReadyCQLStatement.bind(ReadyCQLStatement.java:51)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runPhase(CqlAction.java:106)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runCycle(CqlAction.java:85)
    at io.nosqlbench.engine.api.activityimpl.motor.CoreMotor.run(CoreMotor.java:405)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
    at java.base/java.util.Arrays$ArrayList.get(Unknown Source)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:49)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:12)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:39)
 ... 7 more
 6104 WARN [scenarios:001] SCENARIO Error in scenario, shutting down.
 6153 ERROR [main] ERRORHANDLER Error from driver or included library: java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
java.util.concurrent.ExecutionException: java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
    at java.base/java.util.concurrent.FutureTask.report(Unknown Source)
    at java.base/java.util.concurrent.FutureTask.get(Unknown Source)
    at io.nosqlbench.engine.core.script.ScenariosExecutor.getAsyncResultStatus(ScenariosExecutor.java:150)
    at io.nosqlbench.engine.core.script.ScenariosExecutor.awaitAllResults(ScenariosExecutor.java:115)
    at io.nosqlbench.engine.core.script.ScenariosExecutor.awaitAllResults(ScenariosExecutor.java:73)
    at io.nosqlbench.engine.cli.NBCLI.run(NBCLI.java:405)
    at io.nosqlbench.engine.cli.NBCLI.main(NBCLI.java:69)
Caused by: java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
    at io.nosqlbench.engine.core.script.Scenario.runScenario(Scenario.java:307)
    at io.nosqlbench.engine.core.script.Scenario.call(Scenario.java:362)
    at io.nosqlbench.engine.core.script.Scenario.call(Scenario.java:56)
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
    at io.nosqlbench.engine.core.lifecycle.ActivityExecutor.notifyException(ActivityExecutor.java:526)
    at io.nosqlbench.engine.core.lifecycle.ActivityExceptionHandler.uncaughtException(ActivityExceptionHandler.java:30)
    at java.base/java.lang.Thread.dispatchUncaughtException(Unknown Source)
Caused by: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
 ... 3 more
Caused by: java.lang.RuntimeException: Binding error:BindingsTemplate:'doc'=>"CharBufImage("a-z",20) -> java.nio.CharBuffer"=>[xqaoxvdxybvcumaiddqu](HeapCharBufferR)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:41)
    at io.nosqlbench.activitytype.cql.statements.core.ReadyCQLStatement.bind(ReadyCQLStatement.java:51)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runPhase(CqlAction.java:106)
    at io.nosqlbench.activitytype.cql.core.CqlAction.runCycle(CqlAction.java:85)
    at io.nosqlbench.engine.api.activityimpl.motor.CoreMotor.run(CoreMotor.java:405)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
    at java.base/java.util.Arrays$ArrayList.get(Unknown Source)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:49)
    at io.nosqlbench.activitytype.cql.statements.binders.UnsettableValuesBinder.bindValues(UnsettableValuesBinder.java:12)
    at io.nosqlbench.virtdata.core.bindings.ContextualArrayBindings.bind(ContextualArrayBindings.java:39)
 ... 7 more
 6155 ERROR [main] ERRORHANDLER cause (see stack trace for details):javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
java.lang.RuntimeException: javax.script.ScriptException: java.lang.RuntimeException: Error in activity thread testcharbuf_default_001:001
 6156 WARN [Thread-1] SCENARIO Scenario was interrupted by process exit, shutting down

nosqlbench

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

DataStax Enterprise is powered by the best distribution of Apache Cassandra ™

Apache, Apache Cassandra, Cassandra, Apache Tomcat, Tomcat, Apache Lucene, Lucene, Apache Solr, Apache Hadoop, Hadoop, Apache Spark, Spark, Apache TinkerPop, TinkerPop, Apache Kafka and Kafka are either registered trademarks or trademarks of the Apache Software Foundation or its subsidiaries in Canada, the United States and/or other countries.

Privacy Policy Terms of Use

Answer 1 · 2021-10-29T15:41:32Z

jshook_49093 answered Oct 29, '21 tecles_192911 commented Nov 3, '21

@tecles_192911,
The first thing you want to do is change

insert into test.docs (id, doc) values ( uuid(), '{doc}' );

to

insert into test.docs (id, doc) values ( uuid(), {doc} );

Since, with the single quotes, the value is parsed as a literal value.

The next error you may see is that there is no codec to map from CharBuffer to varchar, so you can add a ToString() in your binding to look like this:

bindings:
 doc: CharBufImage("a-z",20); ToString();

This will cause the CharBuf to be converted before being presented to the driver's codec mapping layer.

Then you will get results like this:

cqlsh> select * from test.docs ;

 id                                   | doc
--------------------------------------+----------------------
 e4ecf429-7ce4-4705-a031-6ace1de887ce | xqaoxvdxybvcumaiddqu
 52ed6fb9-1d97-44b0-ad38-60721f16a46e | xqaoxvdxybvcumaiddqu
 5e2487c4-79ec-43fb-adee-9249fda1e7a5 | xqaoxvdxybvcumaiddqu
 695bb3ca-d3ff-4d25-a5ca-969217f4dc0b | xqaoxvdxybvcumaiddqu
 ...

As an additional improvement, I would highly recommend you create a bigger image and use the argument form for CharBufImage which extracts a random offset, like this:

bindings:
 doc: CharBufImage("a-z",1000,20); ToString();

Then, you'll see some pseudo-randomness in your data, like this:

cqlsh> select * from test.docs ;

 id                                   | doc
--------------------------------------+----------------------
 35477187-42fc-4305-94b8-e406121ec103 | wmnwdytzlacqoqjmhdbj
 999ee417-d2e2-49ea-ae0d-1c07cbc73d1c | zwfwmnwdytzlacqoqjmh
 94422128-8adb-4e1b-a775-c7e54ac259fa | epxmqxmrbiuzzvzufzcq
 ...

Of course, you'll want to flavor this to your test requirements based on how much variety you need to simulate.

Happy Testing!

1 Share

10 |1000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 1.0 MiB each and 10.0 MiB total.

tecles_192911 commented · Oct 30, 2021 at 07:39 AM

Thank you, @jshook_49093. Clear and precise.

The test runs perfectly now and I will follow your recommendation.

I really appreciated your help!

1 ·

question

Need help with NoSQLBench's CharBufImage() binding function

1 Answer

question details

OTHER RESOURCES

question

Need help with NoSQLBench's CharBufImage() binding function

1 Answer

question details

Related Questions

OTHER RESOURCES