We used dsbulk to perform unload operation, we could see the time taken was actually more than it shows in its original logs.
Attaching the details here,
[aaa@bbb bin]$ date; ./dsbulk unload -k *** -t *** -h *** -u *** -p *** > /aaa/bbb/xxx.csv;date; Wed Mar 10 06:08:40 CST 2021 Username and password provided but auth provider not specified, inferring PlainTextAuthProvider Operation directory: /aaa/bbb/dsbulk-1.7.0/bin/logs/UNLOAD_20210310-120841-278359 total | failed | rows/s | p50ms | p99ms | p999ms 38,260 | 0 | 14,304 | 8.20 | 78.64 | 78.64 Operation UNLOAD_20210310-120841-278359 completed successfully in 2 seconds. Wed Mar 10 06:08:47 CST 2021
Here we could see that the original time taken is 7 secs but it shows only 2 secs in its logs. Is this because of time taken to establish session? Or something else?
So if session connectivity is the reason, say we have 70 tables, and if there is a delay of 7 secs for each table will the latency be 490 secs?
Also can we use multi threading/parallel processing here? to improve performance? We need to export multiple tables in a given time using a single python script.
Please let us know your suggestions/recommendations here. Thanks