SQLite 3.41.0 and pearson correlation extension function (#106)

* Build SQLite 3.41.0

* Update pivot_vtab

* Add Pearson correlation coefficient function extension, build

* Add an easy way to running test locally without Nodejs

* Use RSS sum to pick top2 processes for the report

* Try previous Ubuntu LTS as a workaround for Firefox worker not starting

.github/workflows/main.yml

@@ -1,4 +1,4 @@
-name: Deploy to GitHub Pages and create release
+name: Create release
 on:
   workflow_dispatch:
   push:
@@ -7,7 +7,7 @@ on:
 
 jobs:
   deploy:
-    name: Deploy to GitHub Pages and create release
+    name: Create release
     runs-on: ubuntu-latest    
     steps:
     - uses: actions/checkout@v2
@@ -24,10 +24,11 @@ jobs:
         npm install
         npm run build
 
-    - name: Create archive
+    - name: Create archives
       run: |
         cd dist
-        zip -9 -r  dist.zip . -x "js/*.map" -x "/*.map"
+        zip -9 -r ../dist.zip . -x "js/*.map" -x "/*.map"
+        zip -9 -r  ../dist_map.zip .
 
     - name: Create Release Notes
       run: |
@@ -39,16 +40,6 @@ jobs:
     - name: Create release
       uses: ncipollo/release-action@v1
       with:
-        artifacts: "dist/dist.zip"
+        artifacts: "dist.zip,dist_map.zip"
         token: ${{ secrets.GITHUB_TOKEN }}
         bodyFile: "CHANGELOG.md"
-
-    - name: Deploy 🚀
-      uses: JamesIves/github-pages-deploy-action@4.1.1
-      with:
-        token: ${{ secrets.GITHUB_TOKEN }}
-        branch: build # The branch the action should deploy to.
-        folder: dist/ # The folder the action should deploy.
-        clean: true # Automatically remove deleted files from the deploy branch
-        clean-exclude: .nojekyll
-

.github/workflows/test.yml

@@ -11,7 +11,7 @@ on:
 jobs:
   test:
     name: Run tests
-    runs-on: ubuntu-latest    
+    runs-on: ubuntu-20.04
     steps:
     - uses: actions/checkout@v2
     - name: Use Node.js

Dockerfile.test

@@ -0,0 +1,24 @@
+# An easy way to run tests locally without Nodejs installed:
+#
+#     docker build -t sqliteviz/test -f Dockerfile.test .
+#
+
+FROM node:12
+
+RUN set -ex; \
+    apt update; \
+    apt install -y chromium firefox-esr; \
+    npm install -g npm@7
+
+WORKDIR /tmp/build
+
+COPY package.json package-lock.json ./
+COPY lib lib
+RUN npm install
+
+COPY . .
+
+RUN set -ex; \
+    sed -i 's/browsers: \[.*\],/browsers: ['"'FirefoxHeadlessTouch'"'],/' karma.conf.js
+
+RUN npm run lint -- --no-fix && npm run test

README.md

@@ -18,10 +18,10 @@ With sqliteviz you can:
 https://user-images.githubusercontent.com/24638357/128249848-f8fab0f5-9add-46e0-a9c1-dd5085a8623e.mp4
 
 ## Quickstart
-The latest release of sqliteviz is deployed on GitHub Pages at [lana-k.github.io/sqliteviz][6].
+The latest release of sqliteviz is deployed on [sqliteviz.com/app][6].
 
 ## Wiki
-For user documentation, check out sqliteviz [Wiki][7].
+For user documentation, check out sqliteviz [documentation][7].
 
 ## Motivation
 It's a kind of middleground between [Plotly Falcon][1] and [Redash][2].
@@ -34,8 +34,8 @@ It is built on top of [react-chart-editor][3], [PivotTable.js][12], [sql.js][4]
 [3]: https://github.com/plotly/react-chart-editor
 [4]: https://github.com/sql-js/sql.js
 [5]: https://github.com/vuejs/vue
-[6]: https://lana-k.github.io/sqliteviz/
-[7]: https://github.com/lana-k/sqliteviz/wiki
+[6]: https://sqliteviz.com/app/
+[7]: https://sqliteviz.com/docs
 [8]: https://github.com/surmon-china/vue-codemirror#readme
 [9]: https://www.papaparse.com/
 [10]: https://github.com/lana-k/sqliteviz/wiki/Predefined-queries

lib/sql-js/Dockerfile

@@ -1,4 +1,4 @@
-FROM emscripten/emsdk:2.0.24
+FROM emscripten/emsdk:3.0.1
 
 WORKDIR /tmp/build
 

lib/sql-js/README.md

@@ -43,6 +43,8 @@ SQLite [miscellaneous extensions][3] included:
 SQLite 3rd party extensions included:
 
 1. [pivot_vtab][5] -- a pivot virtual table
+2. `pearson` correlation coefficient function extension from [sqlean][21]
+   (which is part of [squib][20])
 
 To ease the step to have working clone locally, the build is committed into
 the repository.
@@ -99,3 +101,5 @@ described in [this message from SQLite Forum][12]:
 [17]: https://sqlite.org/contrib/
 [18]: https://sqlite.org/contrib//download/extension-functions.c?get=25
 [19]: https://github.com/lana-k/sqliteviz/blob/master/tests/lib/database/sqliteExtensions.spec.js
+[20]: https://github.com/mrwilson/squib/blob/master/pearson.c
+[21]: https://github.com/nalgeon/sqlean/blob/incubator/src/pearson.c

lib/sql-js/benchmark/Dockerfile

@@ -1,4 +1,4 @@
-FROM node:12-buster
+FROM node:14-bullseye
 
 RUN set -ex; \
     echo 'deb http://deb.debian.org/debian unstable main' \

lib/sql-js/benchmark/README.md

@@ -1,14 +1,25 @@
 # SQLite WebAssembly build micro-benchmark
 
-This directory contains a micro-benchmark for evaluating SQLite
-WebAssembly builds performance on typical SQL queries, run from
-`make.sh` script. It can also serve as a smoke test.
+This directory contains a micro-benchmark for evaluating SQLite WebAssembly
+builds performance on read and write SQL queries, run from `make.sh` script. If
+the script has permission to `nice` processes and [Procpath][1] is installed,
+e.g. it is run with `sudo -E env PATH=$PATH ./make.sh`, it'll `renice` all
+processes running inside the benchmark containers. It can also serve as a smoke
+test (e.g. for memory leaks).
 
-The benchmark operates on a set of SQLite WebAssembly builds expected
-in `lib/build-$NAME` directories each containing `sql-wasm.js` and
-`sql-wasm.wasm`. Then it creates a Docker image for each, and runs
-the benchmark in Firefox and Chromium using Karma in the container.
+The benchmark operates on a set of SQLite WebAssembly builds expected in
+`lib/build-$NAME` directories each containing `sql-wasm.js` and
+`sql-wasm.wasm`. Then it creates a Docker image for each, and runs the
+benchmark in Firefox and Chromium using Karma in the container.
 
-After successful run, the benchmark result of each build is contained
-in `build-$NAME-result.json`. The JSON result files can be analysed
-using `result-analysis.ipynb` Jupyter notebook.
+After successful run, the benchmark produces the following per each build:
+
+- `build-$NAME-result.json`
+- `build-$NAME.sqlite` (if Procpath is installed)
+- `build-$NAME.svg` (if Procpath is installed)
+
+These files can be analysed using `result-analysis.ipynb` Jupyter notebook.
+The SVG is a chart with CPU and RSS usage of each test container (i.e. Chromium
+run, then Firefox run per container).
+
+[1]: https://pypi.org/project/Procpath/

lib/sql-js/benchmark/make.sh

@@ -1,21 +1,47 @@
 #!/bin/bash -e
 
+cleanup () {
+  rm -rf lib/dist "$renice_flag_file"
+  docker rm -f sqljs-benchmark-run 2> /dev/null || true
+}
+trap cleanup EXIT
+
 if [ ! -f sample.csv ]; then
   wget --header="accept-encoding: gzip" -q -O- \
     https://github.com/plotly/datasets/raw/547090bd/wellspublic.csv \
     | gunzip -c > sample.csv
 fi
 
+PLAYBOOK=procpath/karma_docker.procpath
+
+# for renice to work run like "sudo -E env PATH=$PATH ./make.sh"
+test_ni=$(nice -n -5 nice)
+if [ $test_ni == -5 ]; then
+  renice_flag_file=$(mktemp)
+fi
+{
+  while [ -f $renice_flag_file ]; do
+    procpath --logging-level ERROR play -f $PLAYBOOK renice:watch
+  done
+} &
+
+shopt -s nullglob
 for d in lib/build-* ; do
-  rm -r lib/dist || true
+  rm -rf lib/dist
   cp -r $d lib/dist
+  sample_name=$(basename $d)
 
-  name=$(basename $d)
-  docker build -t sqliteviz/sqljs-benchmark:$name .
-  docker rm sqljs-benchmark-$name 2> /dev/null || true
-  docker run -it --name sqljs-benchmark-$name sqliteviz/sqljs-benchmark:$name
-  docker cp sqljs-benchmark-$name:/tmp/build/suite-result.json ${name}-result.json
-  docker rm sqljs-benchmark-$name
+  docker build -t sqliteviz/sqljs-benchmark .
+  docker rm sqljs-benchmark-run 2> /dev/null || true
+  docker run -d -it --cpus 2 --name sqljs-benchmark-run sqliteviz/sqljs-benchmark
+  {
+    rm -f ${sample_name}.sqlite
+    procpath play -f $PLAYBOOK -o database_file=${sample_name}.sqlite track:record
+    procpath play -f $PLAYBOOK -o database_file=${sample_name}.sqlite \
+      -o plot_file=${sample_name}.svg track:plot
+  } &
+
+  docker attach sqljs-benchmark-run
+  docker cp sqljs-benchmark-run:/tmp/build/suite-result.json ${sample_name}-result.json
+  docker rm sqljs-benchmark-run
 done
-
-rm -r lib/dist

lib/sql-js/benchmark/procpath/karma_docker.procpath

@@ -0,0 +1,28 @@
+# This command may run when "sqljs-benchmark-run" does not yet exist or run
+[renice:watch]
+interval: 2
+repeat: 30
+environment:
+  ROOT_PID=docker inspect -f "{{.State.Pid}}" sqljs-benchmark-run 2> /dev/null || true
+query:
+  PIDS=$..children[?(@.stat.pid in [$ROOT_PID])]..pid
+command:
+  echo $PIDS | tr , '\n' | xargs --no-run-if-empty -I{} -- renice -n -5 -p {}
+
+# Expected input arguments: database_file
+[track:record]
+interval: 1
+stop_without_result: 1
+environment:
+  ROOT_PID=docker inspect -f "{{.State.Pid}}" sqljs-benchmark-run
+query:
+  $..children[?(@.stat.pid == $ROOT_PID)]
+pid_list: $ROOT_PID
+
+# Expected input arguments: database_file, plot_file
+[track:plot]
+moving_average_window: 5
+title: Chromium vs Firefox (№1 RSS, №2 CPU)
+custom_query_file:
+  procpath/top2_rss.sql
+  procpath/top2_cpu.sql

lib/sql-js/benchmark/procpath/top2_cpu.sql

@@ -0,0 +1,29 @@
+WITH diff_all AS (
+    SELECT
+        record_id,
+        ts,
+        stat_pid,
+        stat_utime + stat_stime - LAG(stat_utime + stat_stime) OVER (
+            PARTITION BY stat_pid
+            ORDER BY record_id
+        ) tick_diff,
+        ts - LAG(ts) OVER (
+            PARTITION BY stat_pid
+            ORDER BY record_id
+        ) ts_diff
+    FROM record
+), diff AS (
+    SELECT * FROM diff_all WHERE tick_diff IS NOT NULL
+), one_time_pid_condition AS (
+    SELECT stat_pid
+    FROM record
+    GROUP BY 1
+    ORDER BY SUM(stat_utime + stat_stime) DESC
+    LIMIT 2
+)
+SELECT
+    ts,
+    stat_pid pid,
+    100.0 * tick_diff / (SELECT value FROM meta WHERE key = 'clock_ticks') / ts_diff value
+FROM diff
+JOIN one_time_pid_condition USING(stat_pid)

lib/sql-js/benchmark/procpath/top2_rss.sql

@@ -0,0 +1,13 @@
+WITH one_time_pid_condition AS (
+    SELECT stat_pid
+    FROM record
+    GROUP BY 1
+    ORDER BY SUM(stat_rss) DESC
+    LIMIT 2
+)
+SELECT
+    ts,
+    stat_pid pid,
+    stat_rss / 1024.0 / 1024 * (SELECT value FROM meta WHERE key = 'page_size') value
+FROM record
+JOIN one_time_pid_condition USING(stat_pid)

lib/sql-js/build.py

@@ -2,9 +2,11 @@ import logging
 import subprocess
 from pathlib import Path
 
-
+# See the setting descriptions on these pages:
+# - https://emscripten.org/docs/optimizing/Optimizing-Code.html
+# - https://github.com/emscripten-core/emscripten/blob/main/src/settings.js
 cflags = (
-    '-O2',
+    # SQLite configuration
     '-DSQLITE_DEFAULT_CACHE_SIZE=-65536',  # 64 MiB
     '-DSQLITE_DEFAULT_MEMSTATUS=0',
     '-DSQLITE_DEFAULT_SYNCHRONOUS=0',
@@ -13,26 +15,26 @@ cflags = (
     '-DSQLITE_ENABLE_FTS3',
     '-DSQLITE_ENABLE_FTS3_PARENTHESIS',
     '-DSQLITE_ENABLE_FTS5',
-    '-DSQLITE_ENABLE_JSON1',
     '-DSQLITE_ENABLE_NORMALIZE',
     '-DSQLITE_EXTRA_INIT=extra_init',
     '-DSQLITE_OMIT_DEPRECATED',
     '-DSQLITE_OMIT_LOAD_EXTENSION',
     '-DSQLITE_OMIT_SHARED_CACHE',
     '-DSQLITE_THREADSAFE=0',
+    # Compile-time optimisation
+    '-Os',  # reduces the code size about in half comparing to -O2
+    '-flto',
 )
 emflags = (
     # Base
     '--memory-init-file', '0',
-    '-s', 'RESERVED_FUNCTION_POINTERS=64',
     '-s', 'ALLOW_TABLE_GROWTH=1',
-    '-s', 'SINGLE_FILE=0',
     # WASM
     '-s', 'WASM=1',
     '-s', 'ALLOW_MEMORY_GROWTH=1',
-    # Optimisation
-    '-s', 'INLINING_LIMIT=50',
-    '-O3',
+    '-s', 'ENVIRONMENT=web,worker',
+    # Link-time optimisation
+    '-Os',
     '-flto',
     # sql.js
     '-s', 'EXPORTED_FUNCTIONS=@src/sqljs/exported_functions.json',
@@ -50,22 +52,22 @@ def build(src: Path, dst: Path):
         'emcc',
         *cflags,
         '-c', src / 'sqlite3.c',
-        '-o', out / 'sqlite3.bc',
+        '-o', out / 'sqlite3.o',
     ])
     logging.info('Building LLVM bitcode for extension-functions.c')
     subprocess.check_call([
         'emcc',
         *cflags,
         '-c', src / 'extension-functions.c',
-        '-o', out / 'extension-functions.bc',
+        '-o', out / 'extension-functions.o',
     ])
 
     logging.info('Building WASM from bitcode')
     subprocess.check_call([
         'emcc',
         *emflags,
-        out / 'sqlite3.bc',
-        out / 'extension-functions.bc',
+        out / 'sqlite3.o',
+        out / 'extension-functions.o',
         '-o', out / 'sql-wasm.js',
     ])
 

lib/sql-js/configure.py

@@ -8,7 +8,7 @@ from pathlib import Path
 from urllib import request
 
 
-amalgamation_url = 'https://sqlite.org/2021/sqlite-amalgamation-3360000.zip'
+amalgamation_url = 'https://sqlite.org/2023/sqlite-amalgamation-3410000.zip'
 
 # Extension-functions
 # ===================
@@ -20,7 +20,7 @@ contrib_functions_url = 'https://sqlite.org/contrib/download/extension-functions
 extension_urls = (
     # Miscellaneous extensions
     # ========================
-    ('https://sqlite.org/src/raw/c6bd5d24?at=series.c', 'sqlite3_series_init'),
+    ('https://sqlite.org/src/raw/8d79354f?at=series.c', 'sqlite3_series_init'),
     ('https://sqlite.org/src/raw/dbfd8543?at=closure.c', 'sqlite3_closure_init'),
     ('https://sqlite.org/src/raw/5bb2264c?at=uuid.c', 'sqlite3_uuid_init'),
     ('https://sqlite.org/src/raw/5853b0e5?at=regexp.c', 'sqlite3_regexp_init'),
@@ -28,10 +28,11 @@ extension_urls = (
     ('https://sqlite.org/src/raw/09f967dc?at=decimal.c', 'sqlite3_decimal_init'),
     # Third-party extension
     # =====================
-    ('https://github.com/jakethaw/pivot_vtab/raw/08ab0797/pivot_vtab.c', 'sqlite3_pivotvtab_init'),
+    ('https://github.com/jakethaw/pivot_vtab/raw/9323ef93/pivot_vtab.c', 'sqlite3_pivotvtab_init'),
+    ('https://github.com/nalgeon/sqlean/raw/95e8d21a/src/pearson.c', 'sqlite3_pearson_init'),
 )
 
-sqljs_url = 'https://github.com/sql-js/sql.js/archive/refs/tags/v1.5.0.zip'
+sqljs_url = 'https://github.com/sql-js/sql.js/archive/refs/tags/v1.7.0.zip'
 
 
 def _generate_extra_init_c_function(init_function_names):

package.json

@@ -1,6 +1,6 @@
 {
   "name": "sqliteviz",
-  "version": "0.17.0",
+  "version": "0.21.1",
   "license": "Apache-2.0",
   "private": true,
   "scripts": {

public/index.html

@@ -69,6 +69,8 @@
         }
       }
     </style>
+    <!-- head extention slot start -->
+    <!-- head extention slot end -->
   </head>
   <body>
     <noscript>
@@ -93,6 +95,8 @@
         </svg>
       </div>
     </div>
+    <!-- extention slot start -->
+    <!-- extention slot end -->
     <!-- built files will be auto injected -->
   </body>
 </html>

src/assets/images/logo_simple.svg

@@ -0,0 +1,3 @@
+<svg width="48" height="48" viewBox="0 0 48 48" fill="none" xmlns="http://www.w3.org/2000/svg">
+<path d="M26.8311 34.6554C25.4675 33.8178 24.177 32.8655 22.9735 31.8086V14.3616H30.5728V36.753C29.3146 36.0982 28.0673 35.399 26.8311 34.6554ZM41.4669 25.8486H33.8675V38.1514C36.3477 39.3055 38.884 40.3334 41.4669 41.2313V25.8486ZM22.9735 35.3046L22.4768 34.9051C21.7152 34.2725 21.0033 33.6232 20.3245 32.9739L2.2947 30.8763L5.60596 37.3024L28.7848 39.2002C26.7511 38.0537 24.8082 36.7513 22.9735 35.3046ZM41.0695 44.6441C38.4829 43.7946 35.9458 42.7997 33.4702 41.6641L32.543 41.198L17.2616 40.1825L19.8444 45.593L46.5 46.209C44.6788 45.7761 42.8411 45.2434 41.0695 44.6441ZM9.34768 14.3616C12.2649 19.4905 15.735 24.2807 19.6954 28.6455V11.2651L2.99007 2.99115L1.5 22.3859L18.702 31.2592C14.1919 26.5283 10.9703 20.7087 9.34768 14.3616Z" fill="#119DFF"/>
+</svg>

src/components/CsvImport/index.vue

@@ -112,6 +112,7 @@ import SqlTable from '@/components/SqlTable'
 import Logs from '@/components/Logs'
 import time from '@/lib/utils/time'
 import fIo from '@/lib/utils/fileIo'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'CsvImport',
@@ -335,6 +336,7 @@ export default {
       this.$store.commit('setCurrentTabId', tabId)
       this.importCsvCompleted = false
       this.$emit('finish')
+      events.send('inquiry.create', null, { auto: true })
     }
   }
 }

src/components/DbUploader.vue

@@ -58,6 +58,7 @@ import fIo from '@/lib/utils/fileIo'
 import ChangeDbIcon from '@/components/svg/changeDb'
 import database from '@/lib/database'
 import CsvImport from '@/components/CsvImport'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'DbUploader',
@@ -127,6 +128,11 @@ export default {
       if (fIo.isDatabase(file)) {
         this.loadDb(file)
       } else {
+        events.send('database.import', file.size, {
+          from: 'csv',
+          new_db: true
+        })
+
         this.file = file
         await this.$nextTick()
         const csvImport = this.$refs.addCsv

src/components/svg/html.vue

@@ -0,0 +1,49 @@
+<template>
+  <svg
+    width="19"
+    height="18"
+    viewBox="0 0 19 18"
+    fill="none"
+    xmlns="http://www.w3.org/2000/svg"
+  >
+    <path
+      d="M5.1626 10.0745L7.56641 10.8831V12.2322L3.68164 10.6501V9.4812L7.56641
+        7.89917V9.2439L5.1626 10.0745ZM8.99023 13.3H7.93994L10.124 6.35229H11.1787L8.99023
+        13.3ZM14.1099 10.0613L11.7192 9.24829V7.90356L15.582 9.4856V10.6545L11.7192
+        12.2366V10.8918L14.1099 10.0613Z"
+        fill="#A2B1C6"
+     />
+    <path
+      d="M2.17041 0.0637207H16.2185V1.56372H2.17041V9.30354H0.67041V1.56372C0.67041 0.73872
+      1.34541 0.0637207 2.17041 0.0637207Z"
+      fill="#A2B1C6"
+    />
+    <path
+      d="M17.1704 0.0637207H15.3052V1.56372H17.1704V9.84163H18.6704V1.56372C18.6704 0.73872
+      17.9954 0.0637207 17.1704 0.0637207Z"
+      fill="#A2B1C6"
+    />
+    <path
+      d="M2.17041 17.1098H15.8754V15.6098H2.17041V8.78486H0.67041V15.6098C0.67041 16.4348
+      1.34541 17.1098 2.17041 17.1098Z"
+      fill="#A2B1C6"
+    />
+    <path
+      d="M17.1704 17.1098H15.3052V15.6098H17.1704V8.55939H18.6704V15.6098C18.6704 16.4348
+      17.9954 17.1098 17.1704 17.1098Z"
+      fill="#A2B1C6"
+    />
+    <path
+      fill-rule="evenodd"
+      clip-rule="evenodd"
+      d="M18.1197 4.13787H1.76172V3.03787H18.1197V4.13787Z"
+      fill="#A2B1C6"
+    />
+  </svg>
+</template>
+
+<script>
+export default {
+  name: 'HtmlIcon'
+}
+</script>

src/lib/chartHelper.js

@@ -1,5 +1,6 @@
 import dereference from 'react-chart-editor/lib/lib/dereference'
 import plotly from 'plotly.js'
+import { nanoid } from 'nanoid'
 
 export function getOptionsFromDataSources (dataSources) {
   if (!dataSources) {
@@ -33,8 +34,43 @@ export async function getImageDataUrl (element, type) {
   })
 }
 
+export function getChartData (element) {
+  const chartElement = element.querySelector('.js-plotly-plot')
+  return {
+    data: chartElement.data,
+    layout: chartElement.layout
+  }
+}
+
+export function getHtml (options) {
+  const chartId = nanoid()
+  return `
+      <script src="https://cdn.plot.ly/plotly-latest.js" charset="UTF-8"></script>
+      <div id="${chartId}"></div>
+      <script>
+        const el = document.getElementById("${chartId}")
+        
+        let timeout
+        function debounceResize() {
+          clearTimeout(timeout)
+          timeout = setTimeout(() => {
+            var r = el.getBoundingClientRect()
+            Plotly.relayout(el, {width: r.width, height: r.height})
+          }, 200)
+        }
+        
+        const resizeObserver = new ResizeObserver(debounceResize)
+        resizeObserver.observe(el)
+      
+        Plotly.newPlot(el, ${JSON.stringify(options.data)}, ${JSON.stringify(options.layout)})
+      </script>
+  `
+}
+
 export default {
   getOptionsFromDataSources,
   getOptionsForSave,
-  getImageDataUrl
+  getImageDataUrl,
+  getHtml,
+  getChartData
 }

src/lib/database/index.js

@@ -7,6 +7,8 @@ import Worker from './_worker.js'
 // https://github.com/nolanlawson/promise-worker
 import PromiseWorker from 'promise-worker'
 
+import events from '@/lib/utils/events'
+
 function getNewDatabase () {
   const worker = new Worker()
   return new Database(worker)
@@ -76,6 +78,11 @@ class Database {
 
     this.dbName = file ? fu.getFileName(file) : 'database'
     this.refreshSchema()
+
+    events.send('database.import', file ? file.size : 0, {
+      from: file ? 'sqlite' : 'none',
+      new_db: true
+    })
   }
 
   async refreshSchema () {
@@ -114,6 +121,7 @@ class Database {
       throw new Error(data.error)
     }
     fu.exportToFile(data, fileName)
+    events.send('database.export', data.byteLength, { to: 'sqlite' })
   }
 
   async validateTableName (name) {

src/lib/storedInquiries/index.js

@@ -1,5 +1,6 @@
 import { nanoid } from 'nanoid'
 import fu from '@/lib/utils/fileIo'
+import events from '@/lib/utils/events'
 import migration from './_migrations'
 
 const migrate = migration._migrate
@@ -103,9 +104,19 @@ export default {
   importInquiries () {
     return fu.importFile()
       .then(str => {
-        return this.deserialiseInquiries(str)
+        const inquires = this.deserialiseInquiries(str)
+
+        events.send('inquiry.import', inquires.length)
+
+        return inquires
       })
   },
+  export (inquiryList, fileName) {
+    const jsonStr = this.serialiseInquiries(inquiryList)
+    fu.exportToFile(jsonStr, fileName)
+
+    events.send('inquiry.export', inquiryList.length)
+  },
 
   async readPredefinedInquiries () {
     const res = await fu.readFile('./inquiries.json')

src/lib/utils/events.js

@@ -0,0 +1,12 @@
+export default {
+  send (name, value, labels) {
+    const event = new CustomEvent('sqliteviz-app-event', {
+      detail: {
+        name,
+        value,
+        labels: labels || {}
+      }
+    })
+    window.dispatchEvent(event)
+  }
+}

src/registerServiceWorker.js

@@ -1,3 +1,4 @@
+import events from '@/lib/utils/events'
 let refresh = false
 
 function invokeServiceWorkerUpdateFlow (registration) {
@@ -41,4 +42,8 @@ if ('serviceWorker' in navigator) {
       }
     })
   })
+
+  window.addEventListener('appinstalled', () => {
+    events.send('pwa.install')
+  })
 }

src/views/Main/AppDiagnosticInfo.vue

@@ -62,6 +62,7 @@ export default {
 <style scoped>
 #app-info-icon {
   cursor: pointer;
+  width: 24px;
 }
 #app-info-container {
   display: flex;

src/views/Main/Inquiries/index.vue

@@ -159,7 +159,6 @@ import TextField from '@/components/TextField'
 import CheckBox from '@/components/CheckBox'
 import tooltipMixin from '@/tooltipMixin'
 import storedInquiries from '@/lib/storedInquiries'
-import fu from '@/lib/utils/fileIo'
 
 export default {
   name: 'Inquiries',
@@ -385,8 +384,7 @@ export default {
       return this.$store.state.tabs.findIndex(tab => tab.id === id)
     },
     exportToFile (inquiryList, fileName) {
-      const jsonStr = storedInquiries.serialiseInquiries(inquiryList)
-      fu.exportToFile(jsonStr, fileName)
+      storedInquiries.export(inquiryList, fileName)
     },
     exportSelectedInquiries () {
       const inquiryList = this.allInquiries.filter(

src/views/Main/MainMenu.vue

@@ -1,9 +1,12 @@
 <template>
   <nav>
-    <div>
+    <div id="nav-links">
+      <a href="https://sqliteviz.com">
+        <img :src="require('@/assets/images/logo_simple.svg')">
+      </a>
       <router-link to="/workspace">Workspace</router-link>
       <router-link to="/inquiries">Inquiries</router-link>
-      <a href="https://github.com/lana-k/sqliteviz/wiki" target="_blank">Help</a>
+      <a href="https://sqliteviz.com/docs" target="_blank">Help</a>
     </div>
     <div id="nav-buttons">
       <button
@@ -57,6 +60,7 @@ import TextField from '@/components/TextField'
 import CloseIcon from '@/components/svg/close'
 import storedInquiries from '@/lib/storedInquiries'
 import AppDiagnosticInfo from './AppDiagnosticInfo'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'MainMenu',
@@ -110,6 +114,8 @@ export default {
           this.$router.push('/workspace')
         }
       })
+
+      events.send('inquiry.create', null, { auto: false })
     },
     cancelSave () {
       this.$modal.hide('save')
@@ -163,6 +169,7 @@ export default {
 
       // Signal about saving
       this.$root.$emit('inquirySaved')
+      events.send('inquiry.save')
     },
     _keyListener (e) {
       if (this.$route.path === '/workspace') {
@@ -237,4 +244,12 @@ button {
 #nav-buttons {
   display: flex;
 }
+
+#nav-links {
+  display: flex;
+  align-items: center;
+}
+#nav-links img {
+  width: 32px;
+}
 </style>

src/views/Main/Workspace/Schema/index.vue

@@ -33,6 +33,7 @@
 
 <script>
 import fIo from '@/lib/utils/fileIo'
+import events from '@/lib/utils/events'
 import TableDescription from './TableDescription'
 import TextField from '@/components/TextField'
 import TreeChevron from '@/components/svg/treeChevron'
@@ -86,6 +87,11 @@ export default {
       csvImport.reset()
       await csvImport.previewCsv()
       csvImport.open()
+
+      events.send('database.import', this.file.size, {
+        from: 'csv',
+        new_db: false
+      })
     }
   }
 }

src/views/Main/Workspace/Tabs/Tab/DataView/Chart/index.vue

@@ -31,10 +31,15 @@ import PlotlyEditor from 'react-chart-editor'
 import chartHelper from '@/lib/chartHelper'
 import dereference from 'react-chart-editor/lib/lib/dereference'
 import fIo from '@/lib/utils/fileIo'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'Chart',
-  props: ['dataSources', 'initOptions', 'importToPngEnabled', 'importToSvgEnabled'],
+  props: [
+    'dataSources', 'initOptions',
+    'importToPngEnabled', 'importToSvgEnabled',
+    'forPivot'
+  ],
   components: {
     PlotlyEditor
   },
@@ -57,9 +62,20 @@ export default {
   },
   created () {
     // https://github.com/plotly/plotly.js/issues/4555
-    plotly.setPlotConfig ({
+    plotly.setPlotConfig({
       notifyOnLogging: 1
     })
+    this.$watch(
+      () => this.state.data.map(trace => `${trace.type}-${trace.mode}`)
+        .join(','),
+      (value) => {
+        events.send('viz_plotly.render', null, {
+          type: value,
+          pivot: !!this.forPivot
+        })
+      },
+      { deep: true }
+    )
   },
   mounted () {
     this.resizeObserver = new ResizeObserver(this.handleResize)
@@ -105,6 +121,13 @@ export default {
       fIo.downloadFromUrl(url, 'chart')
     },
 
+    saveAsHtml () {
+      fIo.exportToFile(
+        chartHelper.getHtml(this.state),
+        'chart.html',
+        'text/html'
+      )
+    },
     async prepareCopy (type = 'png') {
       return await chartHelper.getImageDataUrl(this.$refs.plotlyEditor.$el, type)
     }

src/views/Main/Workspace/Tabs/Tab/DataView/Pivot/index.vue

@@ -19,10 +19,11 @@ import $ from 'jquery'
 import 'pivottable'
 import 'pivottable/dist/pivot.css'
 import PivotUi from './PivotUi'
-import { getPivotCanvas } from './pivotHelper'
+import pivotHelper from './pivotHelper'
 import Chart from '@/views/Main/Workspace/Tabs/Tab/DataView/Chart'
 import chartHelper from '@/lib/chartHelper'
 import Vue from 'vue'
+import events from '@/lib/utils/events'
 const ChartClass = Vue.extend(Chart)
 
 export default {
@@ -87,8 +88,17 @@ export default {
     'pivotOptions.rendererName': {
       immediate: true,
       handler () {
-        this.$emit('update:importToPngEnabled', this.pivotOptions.rendererName !== 'TSV Export')
-        this.$emit('update:importToSvgEnabled', this.viewStandartChart || this.viewCustomChart)
+        this.$emit(
+          'update:importToPngEnabled',
+          this.pivotOptions.rendererName !== 'TSV Export'
+        )
+        this.$emit(
+          'update:importToSvgEnabled',
+          this.viewStandartChart || this.viewCustomChart
+        )
+        events.send('viz_pivot.render', null, {
+          type: this.pivotOptions.rendererName
+        })
       }
     },
     pivotOptions () {
@@ -169,7 +179,7 @@ export default {
       } else {
         const source = this.viewStandartChart
           ? await chartHelper.getImageDataUrl(this.$refs.pivotOutput, 'png')
-          : (await getPivotCanvas(this.$refs.pivotOutput)).toDataURL('image/png')
+          : (await pivotHelper.getPivotCanvas(this.$refs.pivotOutput)).toDataURL('image/png')
 
         this.$emit('loadingImageCompleted')
         fIo.downloadFromUrl(source, 'pivot')
@@ -179,11 +189,11 @@ export default {
     async prepareCopy () {
       if (this.viewCustomChart) {
         return await this.pivotOptions.rendererOptions.customChartComponent.prepareCopy()
-      } else if (this.viewStandartChart) {
-        return await chartHelper.getImageDataUrl(this.$refs.pivotOutput, 'png')
-      } else {
-        return await getPivotCanvas(this.$refs.pivotOutput)
       }
+      if (this.viewStandartChart) {
+        return await chartHelper.getImageDataUrl(this.$refs.pivotOutput, 'png')
+      }
+      return await pivotHelper.getPivotCanvas(this.$refs.pivotOutput)
     },
 
     async saveAsSvg () {
@@ -193,6 +203,28 @@ export default {
         const url = await chartHelper.getImageDataUrl(this.$refs.pivotOutput, 'svg')
         fIo.downloadFromUrl(url, 'pivot')
       }
+    },
+
+    saveAsHtml () {
+      if (this.viewCustomChart) {
+        this.pivotOptions.rendererOptions.customChartComponent.saveAsHtml()
+        return
+      }
+
+      if (this.viewStandartChart) {
+        const chartState = chartHelper.getChartData(this.$refs.pivotOutput)
+        fIo.exportToFile(
+          chartHelper.getHtml(chartState),
+          'chart.html',
+          'text/html'
+        )
+        return
+      }
+      fIo.exportToFile(
+        pivotHelper.getPivotHtml(this.$refs.pivotOutput),
+        'pivot.html',
+        'text/html'
+      )
     }
   }
 }

src/views/Main/Workspace/Tabs/Tab/DataView/Pivot/pivotHelper.js

@@ -51,6 +51,8 @@ export function _getDataSources (pivotData) {
 
 function customChartRenderer (data, options) {
   options.customChartComponent.dataSources = _getDataSources(data)
+  options.customChartComponent.forPivot = true
+
   options.customChartComponent.$mount()
 
   return $(options.customChartComponent.$el)
@@ -81,3 +83,40 @@ export async function getPivotCanvas (pivotOutput) {
   const tableElement = pivotOutput.querySelector('.pvtTable')
   return await html2canvas(tableElement, { logging: false })
 }
+
+export function getPivotHtml (pivotOutput) {
+  return `
+      <style>
+        table.pvtTable {
+          font-family: Arial, sans-serif;
+          font-size: 12px;
+          text-align: left;
+          border-collapse: collapse;
+          min-width: 100%;
+        }
+        table.pvtTable .pvtColLabel {
+          text-align: center;
+        }
+        table.pvtTable .pvtTotalLabel {
+          text-align: right;
+        }
+        table.pvtTable tbody tr td {
+          color: #506784;
+          border: 1px solid #DFE8F3;
+          text-align: right;
+        }
+        table.pvtTable thead tr th,
+        table.pvtTable tbody tr th {
+          background-color: #506784;
+          color: #fff;
+          border: 1px solid #DFE8F3;
+        }
+      </style>
+      ${pivotOutput.outerHTML}
+  `
+}
+
+export default {
+  getPivotCanvas,
+  getPivotHtml
+}

src/views/Main/Workspace/Tabs/Tab/DataView/index.vue

@@ -51,6 +51,13 @@
         <export-to-svg-icon />
       </icon-button>
 
+      <icon-button
+        tooltip="Save as HTML"
+        tooltip-position="top-left"
+        @click="saveAsHtml"
+      >
+        <HtmlIcon />
+      </icon-button>
       <icon-button
         :loading="copyingImage"
         tooltip="Copy visualisation to clipboard"
@@ -81,12 +88,14 @@ import SideToolBar from '../SideToolBar'
 import IconButton from '@/components/IconButton'
 import ChartIcon from '@/components/svg/chart'
 import PivotIcon from '@/components/svg/pivot'
+import HtmlIcon from '@/components/svg/html'
 import ExportToSvgIcon from '@/components/svg/exportToSvg'
 import PngIcon from '@/components/svg/png'
 import ClipboardIcon from '@/components/svg/clipboard'
 import cIo from '@/lib/utils/clipboardIo'
 import loadingDialog from '@/components/LoadingDialog'
 import time from '@/lib/utils/time'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'DataView',
@@ -100,6 +109,7 @@ export default {
     PivotIcon,
     ExportToSvgIcon,
     PngIcon,
+    HtmlIcon,
     ClipboardIcon,
     loadingDialog
   },
@@ -114,6 +124,11 @@ export default {
       dataToCopy: null
     }
   },
+  computed: {
+    plotlyInPivot () {
+      return this.mode === 'pivot' && this.$refs.viewComponent.viewCustomChart
+    }
+  },
   watch: {
     mode () {
       this.$emit('update')
@@ -139,6 +154,7 @@ export default {
       */
       await time.sleep(0)
       this.$refs.viewComponent.saveAsPng()
+      this.exportSignal('png')
     },
     getOptionsForSave () {
       return this.$refs.viewComponent.getOptionsForSave()
@@ -169,6 +185,7 @@ export default {
     async copyToClipboard () {
       cIo.copyImage(this.dataToCopy)
       this.$modal.hide('prepareCopy')
+      this.exportSignal('clipboard')
     },
     cancelCopy () {
       this.dataToCopy = null
@@ -177,6 +194,26 @@ export default {
 
     saveAsSvg () {
       this.$refs.viewComponent.saveAsSvg()
+      this.exportSignal('svg')
+    },
+    saveAsHtml () {
+      this.$refs.viewComponent.saveAsHtml()
+      this.exportSignal('html')
+    },
+    exportSignal (to) {
+      const eventLabels = { type: to }
+
+      if (this.mode === 'chart' || this.plotlyInPivot) {
+        eventLabels.pivot = this.plotlyInPivot
+      }
+
+      events.send(
+        this.mode === 'chart' || this.plotlyInPivot
+          ? 'viz_plotly.export'
+          : 'viz_pivot.export',
+        null,
+        eventLabels
+      )
     }
   }
 }

src/views/Main/Workspace/Tabs/Tab/RunResult.vue

@@ -72,6 +72,7 @@ import fIo from '@/lib/utils/fileIo'
 import cIo from '@/lib/utils/clipboardIo'
 import time from '@/lib/utils/time'
 import loadingDialog from '@/components/LoadingDialog'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'RunResult',
@@ -117,10 +118,24 @@ export default {
     },
 
     exportToCsv () {
+      if (this.result && this.result.values) {
+        events.send('resultset.export',
+          this.result.values[this.result.columns[0]].length,
+          { to: 'csv' }
+        )
+      }
+
       fIo.exportToFile(csv.serialize(this.result), 'result_set.csv', 'text/csv')
     },
 
     async prepareCopy () {
+      if (this.result && this.result.values) {
+        events.send('resultset.export',
+          this.result.values[this.result.columns[0]].length,
+          { to: 'clipboard' }
+        )
+      }
+
       if ('ClipboardItem' in window) {
         this.preparingCopy = true
         this.$modal.show('prepareCSVCopy')

src/views/Main/Workspace/Tabs/Tab/SqlEditor/hint.js

@@ -3,14 +3,20 @@ import 'codemirror/addon/hint/show-hint.js'
 import 'codemirror/addon/hint/sql-hint.js'
 import store from '@/store'
 
+function _getHintText (hint) {
+  return typeof hint === 'string' ? hint : hint.text
+}
 export function getHints (cm, options) {
-  const token = cm.getTokenAt(cm.getCursor()).string.toUpperCase()
   const result = CM.hint.sql(cm, options)
+
   // Don't show the hint if there is only one option
-  // and the token is already completed with this option
-  if (result.list.length === 1 && result.list[0].text.toUpperCase() === token) {
+  // and the replacingText is already equals to this option
+  const replacedText = cm.getRange(result.from, result.to).toUpperCase()
+  if (result.list.length === 1 &&
+    _getHintText(result.list[0]).toUpperCase() === replacedText) {
     result.list = []
   }
+
   return result
 }
 

src/views/Main/Workspace/Tabs/Tab/index.vue

@@ -56,6 +56,7 @@ import DataView from './DataView'
 import RunResult from './RunResult'
 import time from '@/lib/utils/time'
 import Teleport from 'vue2-teleport'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'Tab',
@@ -108,6 +109,8 @@ export default {
       const fromPosition = this.layout[from]
       this.layout[from] = this.layout[to]
       this.layout[to] = fromPosition
+
+      events.send('inquiry.panel', null, { panel: to })
     },
     onDataViewUpdate () {
       this.$store.commit('updateTab', { index: this.tabIndex, isSaved: false })
@@ -121,11 +124,21 @@ export default {
         const start = new Date()
         this.result = await state.db.execute(this.query + ';')
         this.time = time.getPeriod(start, new Date())
+
+        if (this.result && this.result.values) {
+          events.send('resultset.create',
+            this.result.values[this.result.columns[0]].length
+          )
+        }
+
+        events.send('query.run', parseFloat(this.time), { status: 'success' })
       } catch (err) {
         this.error = {
           type: 'error',
           message: err
         }
+
+        events.send('query.run', 0, { status: 'error' })
       }
       state.db.refreshSchema()
       this.isGettingResults = false

src/views/Main/Workspace/index.vue

@@ -19,6 +19,7 @@
 import Splitpanes from '@/components/Splitpanes'
 import Schema from './Schema'
 import Tabs from './Tabs'
+import events from '@/lib/utils/events'
 
 export default {
   name: 'Workspace',
@@ -49,6 +50,8 @@ export default {
 
       const tabId = await this.$store.dispatch('addTab', { query: stmt })
       this.$store.commit('setCurrentTabId', tabId)
+
+      events.send('inquiry.create', null, { auto: true })
     }
   }
 }

tests/components/Logs.spec.js

@@ -55,6 +55,7 @@ describe('Logs.vue', () => {
     await wrapper.vm.$nextTick()
     messages.push({ type: 'error', message: 'msg 5' })
 
+    await wrapper.vm.$nextTick()
     await wrapper.vm.$nextTick()
     const height = wrapper.find('.logs-container').element.scrollHeight
     expect(wrapper.find('.logs-container').element.scrollTop)

tests/lib/chartHelper.spec.js

@@ -64,7 +64,40 @@ describe('chartHelper.js', () => {
     expect(/^data:image\/png/.test(url)).to.equal(true)
 
     url = await chartHelper.getImageDataUrl(element, 'svg')
-    console.log()
     expect(/^data:image\/svg\+xml/.test(url)).to.equal(true)
   })
+
+  it('getChartData returns plotly data and layout from element', async () => {
+    const element = document.createElement('div')
+    const child = document.createElement('div')
+    element.append(child)
+    child.classList.add('js-plotly-plot')
+    child.data = 'plotly data'
+    child.layout = 'plotly layout'
+
+    const chartData = chartHelper.getChartData(element)
+    expect(chartData).to.eql({
+      data: 'plotly data',
+      layout: 'plotly layout'
+    })
+  })
+
+  it('getHtml returns valid html', async () => {
+    const options = {
+      data: 'plotly data',
+      layout: 'plotly layout'
+    }
+
+    const html = chartHelper.getHtml(options)
+    const doc = document.createElement('div')
+    doc.innerHTML = html
+
+    expect(doc.innerHTML).to.equal(html)
+    expect(doc.children).to.have.lengthOf(3)
+    expect(doc.children[0].src).to.includes('plotly-latest.js')
+    expect(doc.children[1].id).to.have.lengthOf(21)
+    expect(doc.children[2].innerHTML).to.includes(doc.children[1].id)
+    expect(doc.children[2].innerHTML)
+      .to.includes('Plotly.newPlot(el, "plotly data", "plotly layout"')
+  })
 })

tests/lib/database/database.spec.js

@@ -44,7 +44,7 @@ describe('database.js', () => {
     expect(schema[0].columns[0].type).to.equal('N/A')
 
     expect(schema[0].columns[1].name).to.equal('col2')
-    expect(schema[0].columns[1].type).to.equal('integer')
+    expect(schema[0].columns[1].type).to.equal('INTEGER')
   })
 
   it('creates schema with view', async () => {
@@ -68,12 +68,12 @@ describe('database.js', () => {
 
     expect(schema[0].columns[1]).to.eql({
       name: 'col2',
-      type: 'integer'
+      type: 'INTEGER'
     })
 
     expect(schema[1].columns).to.eql([{
       name: 'amount',
-      type: 'integer'
+      type: 'INTEGER'
     }])
   })
 

tests/lib/database/sqliteExtensions.spec.js

@@ -293,7 +293,7 @@ describe('SQLite extensions', function () {
 
   it('supports decimal', async function () {
     const actual = await db.execute(`
-      select
+      SELECT
         decimal_add(decimal('0.1'), decimal('0.2')) "add",
         decimal_sub(0.2, 0.1) sub,
         decimal_mul(power(2, 69), 2) mul,
@@ -430,4 +430,29 @@ describe('SQLite extensions', function () {
       ]
     })
   })
+
+  it('supports pearson', async function () {
+    const actual = await db.execute(`
+      CREATE TABLE dataset(x REAL, y REAL, z REAL);
+      INSERT INTO dataset VALUES
+        (5,3,3.2),  (5,6,4.3),  (5,9,5.4),
+        (10,3,4),   (10,6,3.8), (10,9,3.6),
+        (15,3,4.8), (15,6,4),   (15,9,3.5);
+
+      SELECT
+        pearson(x, x) xx,
+        pearson(x, y) xy,
+        abs(-0.12666 - pearson(x, z)) < 0.00001 xz,
+        pearson(y, x) yx,
+        pearson(y, y) yy,
+        abs(0.10555 - pearson(y, z))  < 0.00001 yz,
+        abs(-0.12666 - pearson(z, x)) < 0.00001 zx,
+        abs(0.10555 - pearson(z, y))  < 0.00001 zy,
+        pearson(z, z) zz
+      FROM dataset;
+    `)
+    expect(actual.values).to.eql({
+      xx: [1], xy: [0], xz: [1], yx: [0], yy: [1], yz: [1], zx: [1], zy: [1], zz: [1]
+    })
+  })
 })

tests/lib/utils/time.spec.js

@@ -29,12 +29,12 @@ describe('time.js', () => {
   })
 
   it('sleep resolves after n ms', async () => {
-    let before = performance.now()
+    let before = Date.now()
     await time.sleep(10)
-    expect(performance.now() - before).to.be.least(10)
+    expect(Date.now() - before).to.be.least(10)
 
-    before = performance.now()
+    before = Date.now()
     await time.sleep(30)
-    expect(performance.now() - before).to.be.least(30)
+    expect(Date.now() - before).to.be.least(30)
   })
 })

tests/views/Main/Workspace/Tabs/Tab/DataView/DataView.spec.js

@@ -59,6 +59,31 @@ describe('DataView.vue', () => {
     expect(pivot.saveAsSvg.calledOnce).to.equal(true)
   })
 
+  it('method saveAsHtml calls the same method of the current view component', async () => {
+    const wrapper = mount(DataView)
+
+    // Find chart and spy the method
+    const chart = wrapper.findComponent({ name: 'Chart' }).vm
+    sinon.spy(chart, 'saveAsHtml')
+
+    // Export to html
+    const htmlBtn = createWrapper(wrapper.findComponent({ name: 'htmlIcon' }).vm.$parent)
+    await htmlBtn.trigger('click')
+    expect(chart.saveAsHtml.calledOnce).to.equal(true)
+
+    // Switch to pivot
+    const pivotBtn = createWrapper(wrapper.findComponent({ name: 'pivotIcon' }).vm.$parent)
+    await pivotBtn.trigger('click')
+
+    // Find pivot and spy the method
+    const pivot = wrapper.findComponent({ name: 'pivot' }).vm
+    sinon.spy(pivot, 'saveAsHtml')
+
+    // Export to svg
+    await htmlBtn.trigger('click')
+    expect(pivot.saveAsHtml.calledOnce).to.equal(true)
+  })
+
   it('shows alert when ClipboardItem is not supported', async () => {
     const ClipboardItem = window.ClipboardItem
     delete window.ClipboardItem

tests/views/Main/Workspace/Tabs/Tab/DataView/Pivot/Pivot.spec.js

@@ -5,6 +5,7 @@ import chartHelper from '@/lib/chartHelper'
 import fIo from '@/lib/utils/fileIo'
 import $ from 'jquery'
 import sinon from 'sinon'
+import pivotHelper from '@/views/Main/Workspace/Tabs/Tab/DataView/Pivot/pivotHelper'
 
 describe('Pivot.vue', () => {
   let container
@@ -271,6 +272,41 @@ describe('Pivot.vue', () => {
     expect(chartComponent.saveAsSvg.called).to.equal(true)
   })
 
+  it('saveAsHtml calls chart method if renderer is Custom Chart', async () => {
+    const wrapper = mount(Pivot, {
+      propsData: {
+        dataSources: {
+          item: ['foo', 'bar', 'bar', 'bar'],
+          year: [2021, 2021, 2020, 2020]
+        },
+        initOptions: {
+          rows: ['item'],
+          cols: ['year'],
+          colOrder: 'key_a_to_z',
+          rowOrder: 'key_a_to_z',
+          aggregatorName: 'Count',
+          vals: [],
+          renderer: $.pivotUtilities.renderers['Custom chart'],
+          rendererName: 'Custom chart',
+          rendererOptions: {
+            customChartOptions: {
+              data: [],
+              layout: {},
+              frames: []
+            }
+          }
+        }
+      },
+      attachTo: container
+    })
+
+    const chartComponent = wrapper.vm.pivotOptions.rendererOptions.customChartComponent
+    sinon.stub(chartComponent, 'saveAsHtml')
+
+    await wrapper.vm.saveAsHtml()
+    expect(chartComponent.saveAsHtml.called).to.equal(true)
+  })
+
   it('saveAsPng calls chart method if renderer is Custom Chart', async () => {
     const wrapper = mount(Pivot, {
       propsData: {
@@ -333,6 +369,66 @@ describe('Pivot.vue', () => {
     expect(chartHelper.getImageDataUrl.calledOnce).to.equal(true)
   })
 
+  it('saveAsHtml - standart chart', async () => {
+    sinon.spy(chartHelper, 'getChartData')
+    sinon.spy(chartHelper, 'getHtml')
+
+    const wrapper = mount(Pivot, {
+      propsData: {
+        dataSources: {
+          item: ['foo', 'bar', 'bar', 'bar'],
+          year: [2021, 2021, 2020, 2020]
+        },
+        initOptions: {
+          rows: ['item'],
+          cols: ['year'],
+          colOrder: 'key_a_to_z',
+          rowOrder: 'key_a_to_z',
+          aggregatorName: 'Count',
+          vals: [],
+          renderer: $.pivotUtilities.renderers['Bar Chart'],
+          rendererName: 'Bar Chart'
+        }
+      },
+      attachTo: container
+    })
+
+    await wrapper.vm.saveAsHtml()
+    expect(chartHelper.getChartData.calledOnce).to.equal(true)
+    const chartData = await chartHelper.getChartData.returnValues[0]
+    expect(chartHelper.getHtml.calledOnceWith(chartData)).to.equal(true)
+  })
+
+  it('saveAsHtml - table', async () => {
+    sinon.stub(pivotHelper, 'getPivotHtml')
+    sinon.stub(fIo, 'exportToFile')
+
+    const wrapper = mount(Pivot, {
+      propsData: {
+        dataSources: {
+          item: ['foo', 'bar', 'bar', 'bar'],
+          year: [2021, 2021, 2020, 2020]
+        },
+        initOptions: {
+          rows: ['item'],
+          cols: ['year'],
+          colOrder: 'key_a_to_z',
+          rowOrder: 'key_a_to_z',
+          aggregatorName: 'Count',
+          vals: [],
+          renderer: $.pivotUtilities.renderers.Table,
+          rendererName: 'Table'
+        }
+      },
+      attachTo: container
+    })
+
+    await wrapper.vm.saveAsHtml()
+    expect(pivotHelper.getPivotHtml.calledOnce).to.equal(true)
+    const html = pivotHelper.getPivotHtml.returnValues[0]
+    expect(fIo.exportToFile.calledOnceWith(html, 'pivot.html', 'text/html')).to.equal(true)
+  })
+
   it('saveAsPng - standart chart', async () => {
     sinon.stub(chartHelper, 'getImageDataUrl').returns('standat chart data url')
     sinon.stub(fIo, 'downloadFromUrl')
@@ -365,6 +461,7 @@ describe('Pivot.vue', () => {
   })
 
   it('saveAsPng - table', async () => {
+    sinon.stub(pivotHelper, 'getPivotCanvas').returns(document.createElement('canvas'))
     sinon.stub(HTMLCanvasElement.prototype, 'toDataURL').returns('canvas data url')
     sinon.stub(fIo, 'downloadFromUrl')
 

tests/views/Main/Workspace/Tabs/Tab/DataView/Pivot/pivotHelper.spec.js

@@ -1,6 +1,6 @@
 import { expect } from 'chai'
-import { _getDataSources, getPivotCanvas }
-  from '@/views/Main/Workspace/Tabs/Tab/DataView/Pivot/pivotHelper'
+import { _getDataSources, getPivotCanvas, getPivotHtml }
+from '@/views/Main/Workspace/Tabs/Tab/DataView/Pivot/pivotHelper'
 
 describe('pivotHelper.js', () => {
   it('_getDataSources returns data sources', () => {
@@ -63,4 +63,19 @@ describe('pivotHelper.js', () => {
 
     expect(await getPivotCanvas(pivotOutput)).to.be.instanceof(HTMLCanvasElement)
   })
+
+  it('getPivotHtml returns html with styles', async () => {
+    const pivotOutput = document.createElement('div')
+    pivotOutput.append('test')
+
+    const html = getPivotHtml(pivotOutput)
+    const doc = document.createElement('div')
+    doc.innerHTML = html
+
+    expect(doc.innerHTML).to.equal(html)
+    expect(doc.children).to.have.lengthOf(2)
+    expect(doc.children[0].tagName).to.equal('STYLE')
+    expect(doc.children[1].tagName).to.equal('DIV')
+    expect(doc.children[1].innerHTML).to.equal('test')
+  })
 })

tests/views/Main/Workspace/Tabs/Tab/SqlEditor/hint.spec.js

@@ -138,15 +138,37 @@ describe('hint.js', () => {
     'getHints returns [ ] if there is only one option and token is completed with this option',
     () => {
       // mock CM.hint.sql and editor
-      sinon.stub(CM.hint, 'sql').returns({ list: [{ text: 'SELECT' }] })
+      sinon.stub(CM.hint, 'sql').returns({
+        list: [{ text: 'SELECT' }],
+        from: null, // from/to doesn't metter because getRange is mocked
+        to: null
+      })
+
       const editor = {
-        getTokenAt () {
-          return {
-            string: 'select',
-            type: 'keyword'
+        getRange () {
+          return 'select'
+        }
+      }
+
+      const hints = getHints(editor, {})
+      expect(hints.list).to.eql([])
+    }
+  )
+
+  it(
+    'getHints returns [ ] if there is only one string option and token ' +
+    'is completed with this option',
+    () => {
+      // mock CM.hint.sql and editor
+      sinon.stub(CM.hint, 'sql').returns({
+        list: ['house.name'],
+        from: null, // from/to doesn't metter because getRange is mocked
+        to: null
+      })
+      const editor = {
+        getRange () {
+          return 'house.name'
         }
-        },
-        getCursor: sinon.stub()
       }
 
       const hints = getHints(editor, {})
@@ -160,15 +182,11 @@ describe('hint.js', () => {
       { text: 'SELECT' },
       { text: 'ST' }
     ]
-    sinon.stub(CM.hint, 'sql').returns({ list })
+    sinon.stub(CM.hint, 'sql').returns({ list, from: null, to: null })
     const editor = {
-      getTokenAt () {
-        return {
-          string: 'se',
-          type: 'keyword'
+      getRange () {
+        return 'se'
       }
-      },
-      getCursor: sinon.stub()
     }
 
     const hints = getHints(editor, {})
@@ -182,15 +200,11 @@ describe('hint.js', () => {
     () => {
       // mock CM.hint.sql and editor
       const list = [{ text: 'SELECT' }]
-      sinon.stub(CM.hint, 'sql').returns({ list })
+      sinon.stub(CM.hint, 'sql').returns({ list, from: null, to: null })
       const editor = {
-        getTokenAt () {
-          return {
-            string: 'sele',
-            type: 'keyword'
+        getRange () {
+          return 'sele'
         }
-        },
-        getCursor: sinon.stub()
       }
 
       const hints = getHints(editor, {})

vue.config.js

@@ -41,5 +41,12 @@ module.exports = {
       .end()
 
     config.module.rule('js').exclude.add(/worker\.js$/)
+
+    config
+      .plugin('html')
+      .tap(args => {
+        args[0].minify = false
+        return args
+      })
   }
 }