3560 add vim 8.2 and nvim 0.4 to ci tests (#3561)

* Add vim82 and neovim04 to CI tests.

* Fix test_sign_column_hightlighting test.

In vim82 with verbose=1 the output of highlight command changes breaking
the ale#sign#SetUpDefaultColumnWithoutErrorsHighlight(). This commit
forces verbose=0 when the method starts and restores the previous value
before exiting.

* No return values in vim82 returns a numeric value instead of a empty string.

* Fix test_reek_handler test

The FuzzyJSONDecode() method catches E474 when it fails to parse the
input as JSON but Vim8.2 throws E491 instead. This commit modifies the
function to catch both E474 or E491.

* Fix perl6 handler test.

Perl6 handler catches json parse errors using the E474 error but in
Vim82 it changed to E491. This commit modifies the handler so both
errors are considered.

* Fix list opening tests.

In Vim 8.2 the call `range(1, bufnr('$'))` always returns quickfix
buffers no matter if they are closed or not. Using `ls` does not show
them but the above range will always include them.

This new behavior breaks the ale#list#IsQuickfixOpen() method that in
turn breaks many other things. This commit fixes this by using the
getqflist() and getloclist() methods instead.

* Fix test updates loclist test.

For some reason in Vim 8.2 the sign offset seems to not reset between
tests causing the sign_id to not match in the Assert. When the test is
run individually it passes but when run as part of the whole suite the
sign_id is off by one.

Forcing the offset in the test setup seems to fix the issue.

* Fix omnifunc completion test.

For unknown reasons the SetCompletionResponse tests fail in Neovim 0.2
and 0.4. Unfortunatelly the only solution I found is to disable them
for neovim.

* Fix linter warnings

* Fix smoker test.

Add vim 8.2 to the list of versions that need some retires due to
randomly failing tests.

* Add docker image build job.

Trying some clever trick to build the docker image if not available
locally or in Docker hub. It uses the Dockerfile md5 checksum as tag so
only when changes on that file occur will the image be downloaded or
build.

* Add labels to Docker image

* Remove tests for middle versions 8.1 and 0.3.5

* Use same vader commit as appveyor

* Implement image push to Docker Hub

Co-authored-by: Horacio Sanson <horacio@allm.inc>

1 files changed, 66 insertions, 26 deletions

diff --git a/run-tests b/run-tests
index 06a05384..ee2b10cd 100755
--- a/run-tests
+++ b/run-tests
@@ -9,8 +9,11 @@ set -u
 # options, or read the output below.
 #
 
-image=w0rp/ale
-current_image_id=1c04720f5d17
+image=denseanalysis/ale
+
+# Create docker image tag based on Dockerfile contents
+image_tag=$(md5sum Dockerfile | cut -d' ' -f1)
+git_version=$(git describe --always --tags)
 
 # Used in all test scripts for running the selected Docker image.
 DOCKER_RUN_IMAGE="$image"
@@ -21,9 +24,9 @@ tests='test/*.vader test/*/*.vader test/*/*/*.vader test/*/*/*.vader'
 verbose_flag=''
 quiet_flag=''
 run_neovim_02_tests=1
-run_neovim_03_tests=1
+run_neovim_04_tests=1
 run_vim_80_tests=1
-run_vim_81_tests=1
+run_vim_82_tests=1
 run_linters=1
 
 while [ $# -ne 0 ]; do
@@ -36,58 +39,66 @@ while [ $# -ne 0 ]; do
         quiet_flag='-q'
         shift
     ;;
+    --build-image)
+        run_vim_80_tests=0
+        run_vim_82_tests=0
+        run_neovim_02_tests=0
+        run_neovim_04_tests=0
+        run_linters=0
+        shift
+    ;;
     --neovim-only)
         run_vim_80_tests=0
-        run_vim_81_tests=0
+        run_vim_82_tests=0
         run_linters=0
         shift
     ;;
     --neovim-02-only)
-        run_neovim_03_tests=0
+        run_neovim_04_tests=0
         run_vim_80_tests=0
-        run_vim_81_tests=0
+        run_vim_82_tests=0
         run_linters=0
         shift
     ;;
-    --neovim-03-only)
+    --neovim-04-only)
         run_neovim_02_tests=0
         run_vim_80_tests=0
-        run_vim_81_tests=0
+        run_vim_82_tests=0
         run_linters=0
         shift
     ;;
     --vim-only)
         run_neovim_02_tests=0
-        run_neovim_03_tests=0
+        run_neovim_04_tests=0
         run_linters=0
         shift
     ;;
     --vim-80-only)
         run_neovim_02_tests=0
-        run_neovim_03_tests=0
-        run_vim_81_tests=0
+        run_neovim_04_tests=0
+        run_vim_82_tests=0
         run_linters=0
         shift
     ;;
-    --vim-81-only)
+    --vim-82-only)
         run_neovim_02_tests=0
-        run_neovim_03_tests=0
+        run_neovim_04_tests=0
         run_vim_80_tests=0
         run_linters=0
         shift
     ;;
     --linters-only)
         run_vim_80_tests=0
-        run_vim_81_tests=0
+        run_vim_82_tests=0
         run_neovim_02_tests=0
-        run_neovim_03_tests=0
+        run_neovim_04_tests=0
         shift
     ;;
     --fast)
         run_vim_80_tests=0
-        run_vim_81_tests=0
+        run_vim_82_tests=0
         run_neovim_02_tests=0
-        run_neovim_03_tests=1
+        run_neovim_04_tests=1
         shift
     ;;
     --help)
@@ -99,12 +110,13 @@ while [ $# -ne 0 ]; do
         echo 'Options:'
         echo '  -v                Enable verbose output'
         echo '  -q                Hide output for successful tests'
-        echo '  --neovim-only     Run tests only for NeoVim 0.2 and 0.3'
+        echo '  --build-image     Run docker image build only.'
+        echo '  --neovim-only     Run tests only for NeoVim'
         echo '  --neovim-02-only  Run tests only for NeoVim 0.2'
-        echo '  --neovim-03-only  Run tests only for NeoVim 0.3'
-        echo '  --vim-only        Run tests only for Vim 8.0 and 8.1'
+        echo '  --neovim-04-only  Run tests only for NeoVim 0.4'
+        echo '  --vim-only        Run tests only for Vim'
         echo '  --vim-80-only     Run tests only for Vim 8.0'
-        echo '  --vim-81-only     Run tests only for Vim 8.1'
+        echo '  --vim-82-only     Run tests only for Vim 8.2'
         echo '  --linters-only    Run only Vint and custom checks'
         echo '  --fast            Run only the fastest Vim and custom checks'
         echo '  --help            Show this help text'
@@ -138,8 +150,36 @@ fi
 # Delete .swp files in the test directory, which cause Vim 8 to hang.
 find test -name '*.swp' -delete
 
-docker images -q w0rp/ale | grep "^$current_image_id" > /dev/null \
-    || docker pull "$image"
+# Check if docker un image is available locally
+has_image=$(docker images ls --filter reference="${image}:${image_tag}" --quiet | wc -l)
+
+if [ "$has_image" -eq 0 ]
+then
+
+  echo "Downloading run image ${image}:${image_tag}"
+  docker pull "${image}:${image_tag}" &> /dev/null
+
+  if [ $? -eq 1 ]
+  then
+    echo "Could not pull image ${image}:${image_tag}"
+    echo "Building run image ${image}:${image_tag}"
+    docker build --build-arg GIT_VERSION="$git_version" -t "${image}:${image_tag}" .
+    docker tag "${image}:${image_tag}" "${image}:latest"
+
+    if [[ -z "${DOCKER_HUB_USER:-}" || -z "${DOCKER_HUB_PASS:-}" ]]
+    then
+      echo "Docker Hub credentials not set, skip push"
+    else
+      echo "Push ${image}:${image_tag} to Docker Hub"
+      echo "$DOCKER_HUB_PASS" | docker login -u "$DOCKER_HUB_USER" --password-stdin
+      docker push "${image}:${image_tag}"
+    fi
+  fi
+else
+  echo "Docker run image ${image}:${image_tag} ready"
+fi
+
+docker tag "${image}:${image_tag}" "${image}:latest"
 
 output_dir=$(mktemp -d 2>/dev/null || mktemp -d -t 'mytmpdir')
 
@@ -174,9 +214,9 @@ trap cancel_tests INT TERM
 
 for vim in $(docker run --rm "$DOCKER_RUN_IMAGE" ls /vim-build/bin | grep '^neovim\|^vim' ); do
     if ( [[ $vim =~ ^vim-v8.0 ]] && ((run_vim_80_tests)) ) \
-    || ( [[ $vim =~ ^vim-v8.1 ]] && ((run_vim_81_tests)) ) \
+    || ( [[ $vim =~ ^vim-v8.2 ]] && ((run_vim_82_tests)) ) \
     || ( [[ $vim =~ ^neovim-v0.2 ]] && ((run_neovim_02_tests)) ) \
-    || ( [[ $vim =~ ^neovim-v0.3 ]] && ((run_neovim_03_tests)) ); then
+    || ( [[ $vim =~ ^neovim-v0.4 ]] && ((run_neovim_04_tests)) ); then
         echo "Starting Vim: $vim..."
         file_number=$((file_number+1))
         test/script/run-vader-tests $quiet_flag $verbose_flag "$vim" "$tests" \