Connection timeout when using --headless with chromedriver (webdriver)

I have been unable to get headless mode working with chromedriver. This is a followup to this twitter conversation: https://twitter.com/bus_kanaka/status/1073990357271101441

The basic problem appears to be that the webdriver POST to create the connection takes 60 seconds to timeout and at that point returns an error message about invalid version (although I suspect that may just be a bogus response). The same process with Chrome works correctly. Also, removing the --headless option allows the script to complete (although --headless mode is needed for the parallel browser render testing system that I am developing). This problem happens with both dev (brave 72.0.60.5, chromedriver 72.0.60.5 / 997b1040b63bac324e815797ba52be0cd8f616ed) and beta (brave v72.0.59.14, chromedriver 72.0.59.14 / 997b1040b63bac324e815797ba52be0cd8f616ed) versions.

I have developed a small script that reproduces the problem. Here is the script followed by the process I use to reproduce this:

#!/bin/bash

if [ "${1}" = "brave" ]; then
    BINARY=${BINARY:-/usr/bin/brave-browser}
    caps='{"desiredCapabilities":{"chromeOptions":{"binary":"'${BINARY}'","args":["--headless","--disable-gpu"]}}}'
else
    caps='{"desiredCapabilities":{"chromeOptions":{"args":["--headless"]}}}'
fi

resp=$(curl -X POST -d "${caps}"  http://localhost:7002/session)
echo "resp: ${resp}"

SESSIONID=$(echo "${resp}" | jq -r ".sessionId")
echo "SESSIONID: ${SESSIONID}"

curl -v -X POST -d '{"url": "http://localhost:9080/test.html"}' http://localhost:7002/session/${SESSIONID}/url
curl -v http://localhost:7002/session/${SESSIONID}/screenshot | jq -r ".value" | base64 -d > test.png

Here is the process I used to reproduce this:

$ time BINARY=/path/to/brave-v0.60.5/brave ./load_test1.sh brave
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   423  100   264  100   159      4      2  0:01:19  0:01:00  0:00:19    27
resp: {"sessionId":"337ba18b26639d0f012f097df1bb3172","status":33,"value":{"message":"session not created: Chrome version must be >= 69.0.3497.0\n  (Driver info: chromedriver=72.0.60.5 (997b1040b63bac324e815797ba52be0cd8f616ed),platform=Linux 4.4.0-43-generic x86_64)"}}
SESSIONID: 337ba18b26639d0f012f097df1bb3172
Note: Unnecessary use of -X or --request, POST is already inferred.
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 7002 (#0)
> POST /session/337ba18b26639d0f012f097df1bb3172/url HTTP/1.1
> Host: localhost:7002
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 42
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 42 out of 42 bytes
< HTTP/1.1 200 OK
< Content-Length:223
< Content-Type:application/json; charset=utf-8
< Connection:close
< 
* Closing connection 0
{"sessionId":"337ba18b26639d0f012f097df1bb3172","status":6,"value":{"message":"invalid session id\n  (Driver info: chromedriver=72.0.60.5 (997b1040b63bac324e815797ba52be0cd8f616ed),platform=Linux 4.4.0-43-generic x86_64)"}}  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 7002 (#0)
> GET /session/337ba18b26639d0f012f097df1bb3172/screenshot HTTP/1.1
> Host: localhost:7002
> User-Agent: curl/7.47.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Length:223
< Content-Type:application/json; charset=utf-8
< Connection:close
< 
{ [223 bytes data]
100   223  100   223    0     0   112k      0 --:--:-- --:--:-- --:--:--  217k
* Closing connection 0
base64: invalid input

real	1m0.102s
user	0m0.016s
sys	0m0.012s

I hadn’t heard and update so I decided to test brave and chromedriver 73.0.62.7 (dev channel) and this appears to be fixed now. Out of curiosity, any idea what caused this and what the fix was?