cache-digest: Truncate hash from MSB

clang + libc++ does not work, it still requires libc++_shared.so
runtime even if -lstdc++ is used, which supposed to link with static
version of libc++.

.travis.yml

@@ -56,6 +56,9 @@ script:
   - make
   - make check
   - cd integration-tests
-  - export GOPATH="$PWD/integration-tests/golang"
-  - make itprep
-  - make it
+  # As of April, 23, 2016, golang http2 build fails, probably because
+  # the default go version is too old.
+
+  # - export GOPATH="$PWD/integration-tests/golang"
+  # - make itprep
+  # - make it

AUTHORS

@@ -23,6 +23,7 @@ Andy Davies
 Ant Bryan
 Bernard Spil
 Brian Card
+Brian Suh
 Daniel Stenberg
 Dave Reisner
 David Beitey
@@ -31,9 +32,11 @@ Etienne Cimon
 Fabian Möller
 Fabian Wiesel
 Gabi Davar
+Jacob Champion
 Jan-E
 Janusz Dziemidowicz
 Jay Satiro
+Jianqing Wang
 Jim Morrison
 José F. Calcerrada
 Kamil Dudka
@@ -44,6 +47,7 @@ Kit Chan
 Kyle Schomp
 Lucas Pardue
 MATSUMOTO Ryosuke
+Mike Conlen
 Mike Frysinger
 Nicholas Hurley
 Nora Shoemaker

CMakeLists.txt

@@ -24,13 +24,13 @@
 
 cmake_minimum_required(VERSION 3.0)
 # XXX using 1.8.90 instead of 1.9.0-DEV
-project(nghttp2 VERSION 1.9.1)
+project(nghttp2 VERSION 1.13.90)
 
 # See versioning rule:
 #  http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
-set(LT_CURRENT  20)
+set(LT_CURRENT  23)
 set(LT_REVISION 0)
-set(LT_AGE      6)
+set(LT_AGE      9)
 
 set(CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/cmake")
 include(Version)

README.rst

@@ -104,7 +104,9 @@ The Python bindings require the following packages:
 * python >= 2.7
 * python-setuptools
 
-If you are using Ubuntu 14.04 LTS (trusty) or Debian 7.0 (wheezy) and above run the following to install the needed packages::
+If you are using Ubuntu 14.04 LTS (trusty) or Debian 7.0 (wheezy) and above run the following to install the needed packages:
+
+.. code-block:: text
 
     sudo apt-get install g++ make binutils autoconf automake autotools-dev libtool pkg-config \
       zlib1g-dev libcunit1-dev libssl-dev libxml2-dev libev-dev libevent-dev libjansson-dev \
@@ -138,7 +140,9 @@ Building from git
 -----------------
 
 Building from git is easy, but please be sure that at least autoconf 2.68 is
-used::
+used:
+
+.. code-block:: text
 
     $ autoreconf -i
     $ automake
@@ -172,6 +176,23 @@ To compile the source code, gcc >= 4.8.3 or clang >= 3.4 is required.
    applications were not built, then using ``--enable-app`` may find
    that cause, such as the missing dependency.
 
+Notes for building on Windows (MSVC)
+------------------------------------
+
+The easiest way to build native Windows nghttp2 dll is use `cmake
+<https://cmake.org/>`_.  The free version of `Visual C++ Build Tools
+<http://landinghub.visualstudio.com/visual-cpp-build-tools>`_ works
+fine.
+
+1. Install cmake for windows
+2. Open "Visual C++ ... Native Build Tool Command Prompt", and inside
+   nghttp2 directly, run ``cmake``.
+3. Then run ``cmake --build`` to build library.
+4. nghttp2.dll, nghttp2.lib, nghttp2.exp are placed under lib directory.
+
+Note that the above steps most likely produce nghttp2 library only.
+No bundled applications are compiled.
+
 Notes for building on Windows (Mingw/Cygwin)
 --------------------------------------------
 
@@ -188,7 +209,9 @@ Secondly, you need to undefine the macro ``__STRICT_ANSI__``, if you
 not, the functions ``fdopen``, ``fileno`` and ``strptime`` will not
 available.
 
-the sample command like this::
+the sample command like this:
+
+.. code-block:: text
 
     $ export CFLAGS="-U__STRICT_ANSI__ -I$libev_PREFIX/include -L$libev_PREFIX/lib"
     $ export CXXFLAGS=$CFLAGS
@@ -206,7 +229,9 @@ Building the documentation
 
    Documentation is still incomplete.
 
-To build the documentation, run::
+To build the documentation, run:
+
+.. code-block:: text
 
     $ make html
 
@@ -235,12 +260,16 @@ its testing framework.  We depend on the following libraries:
 * https://github.com/tatsuhiro-t/spdy
 
 To download the above packages, after settings ``GOPATH``, run the
-following command under ``integration-tests`` directory::
+following command under ``integration-tests`` directory:
+
+.. code-block:: text
 
     $ make itprep
 
 To run the tests, run the following command under
-``integration-tests`` directory::
+``integration-tests`` directory:
+
+.. code-block:: text
 
     $ make it
 
@@ -361,7 +390,9 @@ nghttp - client
 with prior knowledge, HTTP Upgrade and NPN/ALPN TLS extension.
 
 It has verbose output mode for framing information.  Here is sample
-output from ``nghttp`` client::
+output from ``nghttp`` client:
+
+.. code-block:: text
 
     $ nghttp -nv https://nghttp2.org
     [  0.190] Connected
@@ -444,7 +475,9 @@ output from ``nghttp`` client::
     [  0.228] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
 	      (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])
 
-The HTTP Upgrade is performed like so::
+The HTTP Upgrade is performed like so:
+
+.. code-block:: text
 
     $ nghttp -nvu http://nghttp2.org
     [  0.011] Connected
@@ -540,7 +573,9 @@ The HTTP Upgrade is performed like so::
 	      (last_stream_id=2, error_code=NO_ERROR(0x00), opaque_data(0)=[])
 
 Using the ``-s`` option, ``nghttp`` prints out some timing information for
-requests, sorted by completion time::
+requests, sorted by completion time:
+
+.. code-block:: text
 
     $ nghttp -nas https://nghttp2.org/
     ***** Statistics *****
@@ -584,7 +619,9 @@ HTTP/2 connections.  No HTTP Upgrade is supported.
 The ``-p`` option allows users to configure server push.
 
 Just like ``nghttp``, it has a verbose output mode for framing
-information.  Here is sample output from ``nghttpd``::
+information.  Here is sample output from ``nghttpd``:
+
+.. code-block:: text
 
     $ nghttpd --no-tls -v 8080
     IPv4: listen 0.0.0.0:8080
@@ -688,13 +725,17 @@ are not encrypted by default.  To encrypt backend connections, use
 sample configuration file ``nghttpx.conf.sample``.
 
 In the default mode, ``nghttpx`` works as reverse proxy to the backend
-server::
+server:
+
+.. code-block:: text
 
     Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1, HTTP/2) --> Web Server
                                           [reverse proxy]
 
 With the ``--http2-proxy`` option, it works as forward proxy, and it
-is so called secure HTTP/2 proxy (aka SPDY proxy)::
+is so called secure HTTP/2 proxy (aka SPDY proxy):
+
+.. code-block:: text
 
     Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Proxy
                                            [secure proxy]          (e.g., Squid, ATS)
@@ -716,14 +757,18 @@ create a proxy.pac script like this:
 machine nghttpx is running on.  Please note that Chrome requires a valid
 certificate for secure proxy.
 
-Then run Chrome with the following arguments::
+Then run Chrome with the following arguments:
+
+.. code-block:: text
 
     $ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
 
 The backend HTTP/2 connections can be tunneled through an HTTP proxy.
 The proxy is specified using ``--backend-http-proxy-uri``.  The
 following figure illustrates how nghttpx talks to the outside HTTP/2
-proxy through an HTTP proxy::
+proxy through an HTTP proxy:
+
+.. code-block:: text
 
     Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/2) --
 
@@ -737,7 +782,9 @@ The ``h2load`` program is a benchmarking tool for HTTP/2 and SPDY.
 The SPDY support is enabled if the program was built with the spdylay
 library.  The UI of ``h2load`` is heavily inspired by ``weighttp``
 (https://github.com/lighttpd/weighttp).  The typical usage is as
-follows::
+follows:
+
+.. code-block:: text
 
     $ h2load -n100000 -c100 -m100 https://localhost:8443/
     starting benchmark...
@@ -825,7 +872,9 @@ Example:
 With the ``-t`` option, the program can accept more familiar HTTP/1 style
 header field blocks.  Each header set is delimited by an empty line:
 
-Example::
+Example:
+
+.. code-block:: text
 
     :method: GET
     :scheme: https

android-config

@@ -39,9 +39,8 @@ PATH="$TOOLCHAIN"/bin:"$PATH"
     --without-libxml2 \
     --disable-python-bindings \
     --disable-examples \
-    --enable-werror \
-    CC="$TOOLCHAIN"/bin/clang \
-    CXX="$TOOLCHAIN"/bin/clang++ \
+    CC="$TOOLCHAIN"/bin/arm-linux-androideabi-gcc \
+    CXX="$TOOLCHAIN"/bin/arm-linux-androideabi-g++ \
     CPPFLAGS="-fPIE -I$PREFIX/include" \
     PKG_CONFIG_LIBDIR="$PREFIX/lib/pkgconfig" \
     LDFLAGS="-fPIE -pie -L$PREFIX/lib"

configure.ac

@@ -25,7 +25,7 @@ dnl Do not change user variables!
 dnl http://www.gnu.org/software/automake/manual/html_node/Flag-Variables-Ordering.html
 
 AC_PREREQ(2.61)
-AC_INIT([nghttp2], [1.9.1], [t-tujikawa@users.sourceforge.net])
+AC_INIT([nghttp2], [1.14.0-DEV], [t-tujikawa@users.sourceforge.net])
 AC_CONFIG_AUX_DIR([.])
 AC_CONFIG_MACRO_DIR([m4])
 AC_CONFIG_HEADERS([config.h])
@@ -44,9 +44,9 @@ m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
 
 dnl See versioning rule:
 dnl  http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
-AC_SUBST(LT_CURRENT, 20)
+AC_SUBST(LT_CURRENT, 23)
 AC_SUBST(LT_REVISION, 0)
-AC_SUBST(LT_AGE, 6)
+AC_SUBST(LT_AGE, 9)
 
 major=`echo $PACKAGE_VERSION |cut -d. -f1 | sed -e "s/[^0-9]//g"`
 minor=`echo $PACKAGE_VERSION |cut -d. -f2 | sed -e "s/[^0-9]//g"`

doc/CMakeLists.txt

@@ -30,6 +30,7 @@ set(APIDOCS
   nghttp2_nv_compare_name.rst
   nghttp2_option_del.rst
   nghttp2_option_new.rst
+  nghttp2_option_set_builtin_recv_extension_type.rst
   nghttp2_option_set_max_reserved_remote_streams.rst
   nghttp2_option_set_no_auto_ping_ack.rst
   nghttp2_option_set_no_auto_window_update.rst
@@ -117,6 +118,7 @@ set(APIDOCS
   nghttp2_stream_get_sum_dependency_weight.rst
   nghttp2_stream_get_weight.rst
   nghttp2_strerror.rst
+  nghttp2_submit_altsvc.rst
   nghttp2_submit_data.rst
   nghttp2_submit_extension.rst
   nghttp2_submit_goaway.rst

doc/Makefile.am

@@ -47,6 +47,7 @@ APIDOCS= \
 	nghttp2_hd_inflate_get_num_table_entries.rst \
 	nghttp2_hd_inflate_get_table_entry.rst \
 	nghttp2_hd_inflate_hd.rst \
+	nghttp2_hd_inflate_hd2.rst \
 	nghttp2_hd_inflate_new.rst \
 	nghttp2_hd_inflate_new2.rst \
 	nghttp2_http2_strerror.rst \
@@ -54,7 +55,9 @@ APIDOCS= \
 	nghttp2_nv_compare_name.rst \
 	nghttp2_option_del.rst \
 	nghttp2_option_new.rst \
+	nghttp2_option_set_builtin_recv_extension_type.rst \
 	nghttp2_option_set_max_reserved_remote_streams.rst \
+	nghttp2_option_set_max_send_header_block_length.rst \
 	nghttp2_option_set_no_auto_ping_ack.rst \
 	nghttp2_option_set_no_auto_window_update.rst \
 	nghttp2_option_set_no_http_messaging.rst \
@@ -125,6 +128,7 @@ APIDOCS= \
 	nghttp2_session_server_new.rst \
 	nghttp2_session_server_new2.rst \
 	nghttp2_session_server_new3.rst \
+	nghttp2_session_set_local_window_size.rst \
 	nghttp2_session_set_next_stream_id.rst \
 	nghttp2_session_set_stream_user_data.rst \
 	nghttp2_session_terminate_session.rst \
@@ -141,6 +145,7 @@ APIDOCS= \
 	nghttp2_stream_get_sum_dependency_weight.rst \
 	nghttp2_stream_get_weight.rst \
 	nghttp2_strerror.rst \
+	nghttp2_submit_altsvc.rst \
 	nghttp2_submit_data.rst \
 	nghttp2_submit_extension.rst \
 	nghttp2_submit_goaway.rst \

doc/_exts/sphinxcontrib/rubydomain.py

@@ -15,6 +15,7 @@ from docutils import nodes
 from docutils.parsers.rst import directives
 
 from sphinx import addnodes
+from sphinx import version_info
 from sphinx.roles import XRefRole
 from sphinx.locale import l_, _
 from sphinx.domains import Domain, ObjType, Index
@@ -231,8 +232,8 @@ class RubyObject(ObjectDescription):
 
         indextext = self.get_index_text(modname, name_cls)
         if indextext:
-            self.indexnode['entries'].append(('single', indextext,
-                                              fullname, fullname))
+            self.indexnode['entries'].append(
+                _make_index('single', indextext, fullname, fullname))
 
     def before_content(self):
         # needed for automatic qualification of members (reset in subclasses)
@@ -415,11 +416,19 @@ class RubyModule(Directive):
         # modindex currently
         if not noindex:
             indextext = _('%s (module)') % modname
-            inode = addnodes.index(entries=[('single', indextext,
-                                             'module-' + modname, modname)])
+            inode = addnodes.index(entries=[_make_index(
+                'single', indextext, 'module-' + modname, modname)])
             ret.append(inode)
         return ret
 
+def _make_index(entrytype, entryname, target, ignored, key=None):
+    # Sphinx 1.4 introduced backward incompatible changes, it now
+    # requires 5 tuples.  Last one is categorization key.  See
+    # http://www.sphinx-doc.org/en/stable/extdev/nodes.html#sphinx.addnodes.index
+    if version_info >= (1, 4, 0, '', 0):
+        return (entrytype, entryname, target, ignored, key)
+    else:
+        return (entrytype, entryname, target, ignored)
 
 class RubyCurrentModule(Directive):
     """

doc/_themes/sphinx_rtd_theme/layout.html

@@ -81,6 +81,7 @@
 
 <body class="wy-body-for-nav" role="document">
 
+  {% block extrabody %} {% endblock %}
   <div class="wy-grid-for-nav">
 
     {# SIDE NAV, TOGGLES ON MOBILE #}

doc/bash_completion/nghttp

@@ -8,7 +8,7 @@ _nghttp()
     _get_comp_words_by_ref cur prev
     case $cur in
         -*)
-            COMPREPLY=( $( compgen -W '--no-push --verbose --no-dep --get-assets --har --header-table-size --multiply --padding --hexdump --max-concurrent-streams --continuation --connection-window-bits --peer-max-concurrent-streams --timeout --data --no-content-length --version --color --cert --upgrade --remote-name --trailer --weight --help --key --null-out --window-bits --stat --header ' -- "$cur" ) )
+            COMPREPLY=( $( compgen -W '--no-push --verbose --no-dep --get-assets --har --header-table-size --multiply --padding --hexdump --max-concurrent-streams --continuation --connection-window-bits --peer-max-concurrent-streams --timeout --data --no-content-length --version --color --cert --upgrade --remote-name --trailer --weight --help --key --null-out --window-bits --expect-continue --stat --header ' -- "$cur" ) )
             ;;
         *)
             _filedir

doc/bash_completion/nghttpx

@@ -8,7 +8,7 @@ _nghttpx()
     _get_comp_words_by_ref cur prev
     case $cur in
         -*)
-            COMPREPLY=( $( compgen -W '--worker-read-rate --include --frontend-http2-dump-response-header --tls-ticket-key-file --verify-client-cacert --max-response-header-fields --backend-request-buffer --max-request-header-fields --backend-http2-connection-window-bits --conf --backend-http2-max-concurrent-streams --worker-write-burst --npn-list --fetch-ocsp-response-file --no-via --tls-session-cache-memcached-cert-file --no-http2-cipher-black-list --mruby-file --no-server-push --stream-read-timeout --tls-ticket-key-memcached --forwarded-for --accesslog-syslog --frontend-http2-read-timeout --listener-disable-timeout --frontend-http2-connection-window-bits --ciphers --strip-incoming-x-forwarded-for --private-key-passwd-file --backend-keep-alive-timeout --backend-http-proxy-uri --rlimit-nofile --tls-ticket-key-memcached-cert-file --ocsp-update-interval --backend-address-family --tls-session-cache-memcached-private-key-file --error-page --backend-write-timeout --tls-dyn-rec-warmup-threshold --tls-ticket-key-memcached-max-retry --http2-no-cookie-crumbling --worker-read-burst --dh-param-file --accesslog-format --errorlog-syslog --request-header-field-buffer --errorlog-file --frontend-http2-max-concurrent-streams --frontend-write-timeout --tls-ticket-key-cipher --read-burst --backend --insecure --log-level --host-rewrite --tls-proto-list --tls-ticket-key-memcached-interval --worker-frontend-connections --syslog-facility --fastopen --no-location-rewrite --tls-session-cache-memcached --no-ocsp --backend-response-buffer --workers --add-forwarded --frontend-http2-window-bits --worker-write-rate --add-request-header --backend-tls-sni-field --subcert --help --frontend-frame-debug --pid-file --frontend-http2-dump-request-header --daemon --write-rate --altsvc --user --add-x-forwarded-for --frontend-read-timeout --tls-ticket-key-memcached-max-fail --backlog --write-burst --backend-connections-per-host --backend-http2-window-bits --response-header-field-buffer --tls-ticket-key-memcached-address-family --padding --tls-session-cache-memcached-address-family --stream-write-timeout --cacert --tls-ticket-key-memcached-private-key-file --forwarded-by --version --add-response-header --backend-read-timeout --frontend --accesslog-file --http2-proxy --client-private-key-file --client-cert-file --accept-proxy-protocol --tls-dyn-rec-idle-timeout --verify-client --read-rate --backend-connections-per-frontend --strip-incoming-forwarded ' -- "$cur" ) )
+            COMPREPLY=( $( compgen -W '--worker-read-rate --include --frontend-http2-dump-response-header --tls-ticket-key-file --verify-client-cacert --max-response-header-fields --backend-request-buffer --max-request-header-fields --backend-http2-connection-window-bits --conf --backend-http2-max-concurrent-streams --worker-write-burst --npn-list --fetch-ocsp-response-file --no-via --tls-session-cache-memcached-cert-file --no-http2-cipher-black-list --mruby-file --no-server-push --stream-read-timeout --tls-ticket-key-memcached --forwarded-for --accesslog-syslog --frontend-http2-read-timeout --listener-disable-timeout --frontend-http2-connection-window-bits --ciphers --strip-incoming-x-forwarded-for --private-key-passwd-file --backend-keep-alive-timeout --backend-http-proxy-uri --rlimit-nofile --tls-ticket-key-memcached-cert-file --ocsp-update-interval --forwarded-by --tls-session-cache-memcached-private-key-file --error-page --backend-write-timeout --tls-dyn-rec-warmup-threshold --tls-ticket-key-memcached-max-retry --http2-no-cookie-crumbling --worker-read-burst --dh-param-file --accesslog-format --errorlog-syslog --request-header-field-buffer --api-max-request-body --errorlog-file --frontend-http2-max-concurrent-streams --frontend-write-timeout --tls-ticket-key-cipher --read-burst --backend --insecure --backend-max-backoff --log-level --host-rewrite --tls-proto-list --tls-ticket-key-memcached-interval --frontend-http2-setting-timeout --worker-frontend-connections --syslog-facility --fastopen --no-location-rewrite --tls-session-cache-memcached --no-ocsp --backend-response-buffer --workers --add-forwarded --frontend-http2-window-bits --worker-write-rate --add-request-header --backend-http2-settings-timeout --subcert --no-kqueue --help --frontend-frame-debug --pid-file --frontend-http2-dump-request-header --daemon --write-rate --altsvc --user --add-x-forwarded-for --frontend-read-timeout --tls-ticket-key-memcached-max-fail --backlog --write-burst --backend-connections-per-host --backend-http2-window-bits --response-header-field-buffer --tls-ticket-key-memcached-address-family --padding --tls-session-cache-memcached-address-family --stream-write-timeout --cacert --tls-ticket-key-memcached-private-key-file --backend-address-family --version --add-response-header --backend-read-timeout --frontend --accesslog-file --http2-proxy --client-private-key-file --client-cert-file --accept-proxy-protocol --tls-dyn-rec-idle-timeout --verify-client --read-rate --backend-connections-per-frontend --strip-incoming-forwarded ' -- "$cur" ) )
             ;;
         *)
             _filedir

doc/h2load.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "H2LOAD" "1" "March 27, 2016" "1.9.1" "nghttp2"
+.TH "H2LOAD" "1" "Jul 21, 2016" "1.13.0" "nghttp2"
 .SH NAME
 h2load \- HTTP/2 benchmarking tool
 .
@@ -138,7 +138,9 @@ Default: \fBh2c\fP
 .TP
 .B \-d, \-\-data=<PATH>
 Post FILE to  server.  The request method  is changed to
-POST.
+POST.   For  http/1.1 connection,  if  \fI\%\-d\fP  is used,  the
+maximum number of in\-flight pipelined requests is set to
+1.
 .UNINDENT
 .INDENT 0.0
 .TP
@@ -149,7 +151,7 @@ representing the  number of  connections to be  made per
 rate period.   The maximum  number of connections  to be
 made  is  given  in  \fI\%\-c\fP   option.   This  rate  will  be
 distributed among  threads as  evenly as  possible.  For
-example,  with   \fB\-t2\fP  and   \fB\-r4\fP,  each  thread   gets  2
+example,  with   \fI\%\-t\fP2  and   \fI\%\-r\fP4,  each  thread   gets  2
 connections per period.  When the rate is 0, the program
 will run  as it  normally does, creating  connections at
 whatever variable rate it  wants.  The default value for
@@ -412,7 +414,7 @@ performance.  To set smaller flow control window, use \fI\%\-w\fP and
 window size described in HTTP/2 and SPDY protocol specification.
 .SH SEE ALSO
 .sp
-\fInghttp(1)\fP, \fInghttpd(1)\fP, \fInghttpx(1)\fP
+\fBnghttp(1)\fP, \fBnghttpd(1)\fP, \fBnghttpx(1)\fP
 .SH AUTHOR
 Tatsuhiro Tsujikawa
 .SH COPYRIGHT

doc/h2load.1.rst

@@ -108,7 +108,9 @@ OPTIONS
 .. option:: -d, --data=<PATH>
 
     Post FILE to  server.  The request method  is changed to
-    POST.
+    POST.   For  http/1.1 connection,  if  :option:`-d`  is used,  the
+    maximum number of in-flight pipelined requests is set to
+    1.
 
 .. option:: -r, --rate=<N>
 
@@ -118,7 +120,7 @@ OPTIONS
     rate period.   The maximum  number of connections  to be
     made  is  given  in  :option:`-c`   option.   This  rate  will  be
     distributed among  threads as  evenly as  possible.  For
-    example,  with   :option:`-t2`  and   :option:`\-r4`,  each  thread   gets  2
+    example,  with   :option:`-t`\2  and   :option:`-r`\4,  each  thread   gets  2
     connections per period.  When the rate is 0, the program
     will run  as it  normally does, creating  connections at
     whatever variable rate it  wants.  The default value for

doc/nghttp.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTP" "1" "March 27, 2016" "1.9.1" "nghttp2"
+.TH "NGHTTP" "1" "Jul 21, 2016" "1.13.0" "nghttp2"
 .SH NAME
 nghttp \- HTTP/2 client
 .
@@ -217,6 +217,14 @@ accepts.
 .UNINDENT
 .INDENT 0.0
 .TP
+.B \-\-expect\-continue
+Perform an Expect/Continue handshake:  wait to send DATA
+(up to  a short  timeout)  until the server sends  a 100
+Continue interim response. This option is ignored unless
+combined with the \fI\%\-d\fP option.
+.UNINDENT
+.INDENT 0.0
+.TP
 .B \-\-version
 Display version information and exit.
 .UNINDENT
@@ -292,7 +300,7 @@ stream 11 with the weight 12.  The other resources (e.g., icon) depend
 on stream 11 with the weight 2.
 .SH SEE ALSO
 .sp
-\fInghttpd(1)\fP, \fInghttpx(1)\fP, \fIh2load(1)\fP
+\fBnghttpd(1)\fP, \fBnghttpx(1)\fP, \fBh2load(1)\fP
 .SH AUTHOR
 Tatsuhiro Tsujikawa
 .SH COPYRIGHT

doc/nghttp.1.rst

@@ -169,6 +169,13 @@ OPTIONS
     The  number of  concurrent  pushed  streams this  client
     accepts.
 
+.. option:: --expect-continue
+
+    Perform an Expect/Continue handshake:  wait to send DATA
+    (up to  a short  timeout)  until the server sends  a 100
+    Continue interim response. This option is ignored unless
+    combined with the :option:`-d` option.
+
 .. option:: --version
 
     Display version information and exit.
@@ -201,7 +208,9 @@ implementation.
 
 When connection is established, nghttp sends 5 PRIORITY frames to idle
 streams 3, 5, 7, 9 and 11 to create "anchor" nodes in dependency
-tree::
+tree:
+
+.. code-block:: text
 
                       +-----+
                       |id=0 |

doc/nghttp.h2r

@@ -12,7 +12,9 @@ implementation.
 
 When connection is established, nghttp sends 5 PRIORITY frames to idle
 streams 3, 5, 7, 9 and 11 to create "anchor" nodes in dependency
-tree::
+tree:
+
+.. code-block:: text
 
                       +-----+
                       |id=0 |

doc/nghttpd.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTPD" "1" "March 27, 2016" "1.9.1" "nghttp2"
+.TH "NGHTTPD" "1" "Jul 21, 2016" "1.13.0" "nghttp2"
 .SH NAME
 nghttpd \- HTTP/2 server
 .
@@ -209,7 +209,7 @@ The <SIZE> argument is an integer and an optional unit (e.g., 10K is
 10 * 1024).  Units are K, M and G (powers of 1024).
 .SH SEE ALSO
 .sp
-\fInghttp(1)\fP, \fInghttpx(1)\fP, \fIh2load(1)\fP
+\fBnghttp(1)\fP, \fBnghttpx(1)\fP, \fBh2load(1)\fP
 .SH AUTHOR
 Tatsuhiro Tsujikawa
 .SH COPYRIGHT

doc/nghttpx.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTPX" "1" "March 27, 2016" "1.9.1" "nghttp2"
+.TH "NGHTTPX" "1" "Jul 21, 2016" "1.13.0" "nghttp2"
 .SH NAME
 nghttpx \- HTTP/2 proxy
 .
@@ -40,14 +40,14 @@ A reverse proxy for HTTP/2, HTTP/1 and SPDY.
 .TP
 .B <PRIVATE_KEY>
 Set  path  to  server\(aqs private  key.   Required  unless
-"no\-tls" keyword is used in \fI\%\-\-frontend\fP option.
+"no\-tls" parameter is used in \fI\%\-\-frontend\fP option.
 .UNINDENT
 .INDENT 0.0
 .TP
 .B <CERT>
 Set  path  to  server\(aqs  certificate.   Required  unless
-"no\-tls" keyword is used  in \fI\%\-\-frontend\fP option.  To make
-OCSP stapling work, this must be an absolute path.
+"no\-tls"  parameter is  used in  \fI\%\-\-frontend\fP option.   To
+make OCSP stapling work, this must be an absolute path.
 .UNINDENT
 .SH OPTIONS
 .sp
@@ -55,7 +55,7 @@ The options are categorized into several groups.
 .SS Connections
 .INDENT 0.0
 .TP
-.B \-b, \-\-backend=(<HOST>,<PORT>|unix:<PATH>)[;[<PATTERN>[:...]][;proto=<PROTO>][;tls]]
+.B \-b, \-\-backend=(<HOST>,<PORT>|unix:<PATH>)[;[<PATTERN>[:...]][[;PARAM]...]
 Set  backend  host  and   port.   The  multiple  backend
 addresses are  accepted by repeating this  option.  UNIX
 domain socket  can be  specified by prefixing  path name
@@ -117,17 +117,63 @@ and \fI\%\-b\fP\(aq127.0.0.1,8080;www.nghttp2.org\(aq.
 The backend addresses sharing same <PATTERN> are grouped
 together forming  load balancing  group.
 .sp
-Optionally,   backend   application  protocol   can   be
-specified in <PROTO>.  All that share the same <PATTERN>
-must  have  the  same  <PROTO> value  if  it  is  given.
-<PROTO>  should be  one  of the  following list  without
-quotes: "h2", "http/1.1".  The  default value of <PROTO>
-is "http/1.1".  Note that  usually "h2" refers to HTTP/2
-over TLS.  But  in this option, it may  mean HTTP/2 over
-cleartext TCP unless "tls" keyword is used (see below).
+Several parameters <PARAM> are accepted after <PATTERN>.
+The  parameters are  delimited  by  ";".  The  available
+parameters       are:      "proto=<PROTO>",       "tls",
+"sni=<SNI_HOST>",     "fall=<N>",    "rise=<N>",     and
+"affinity=<METHOD>".  The parameter consists of keyword,
+and optionally followed by  "=" and value.  For example,
+the parameter "proto=h2" consists of the keyword "proto"
+and  value "h2".   The parameter  "tls" consists  of the
+keyword   "tls"  without   value.   Each   parameter  is
+described as follows.
 .sp
-Optionally,  TLS  can  be enabled  by  specifying  "tls"
-keyword.  TLS is not enabled by default.
+The backend application protocol  can be specified using
+optional  "proto"   parameter,  and   in  the   form  of
+"proto=<PROTO>".  <PROTO> should be one of the following
+list  without  quotes:  "h2", "http/1.1".   The  default
+value of <PROTO> is  "http/1.1".  Note that usually "h2"
+refers to HTTP/2  over TLS.  But in this  option, it may
+mean HTTP/2  over cleartext TCP unless  "tls" keyword is
+used (see below).
+.sp
+TLS  can   be  enabled  by  specifying   optional  "tls"
+parameter.  TLS is not enabled by default.
+.sp
+With "sni=<SNI_HOST>" parameter, it can override the TLS
+SNI  field  value  with  given  <SNI_HOST>.   This  will
+default to the backend <HOST> name
+.sp
+The  feature  to detect  whether  backend  is online  or
+offline can be enabled  using optional "fall" and "rise"
+parameters.   Using  "fall=<N>"  parameter,  if  nghttpx
+cannot connect  to a  this backend <N>  times in  a row,
+this  backend  is  assumed  to be  offline,  and  it  is
+excluded from load balancing.  If <N> is 0, this backend
+never  be excluded  from load  balancing whatever  times
+nghttpx cannot connect  to it, and this  is the default.
+There is  also "rise=<N>" parameter.  After  backend was
+excluded from load balancing group, nghttpx periodically
+attempts to make a connection to the failed backend, and
+if the  connection is made  successfully <N> times  in a
+row, the backend is assumed to  be online, and it is now
+eligible  for load  balancing target.   If <N>  is 0,  a
+backend  is permanently  offline, once  it goes  in that
+state, and this is the default behaviour.
+.sp
+The     session     affinity    is     enabled     using
+"affinity=<METHOD>"  parameter.   If  "ip" is  given  in
+<METHOD>, client  IP based session affinity  is enabled.
+If  "none" is  given  in <METHOD>,  session affinity  is
+disabled, and this is the default.  The session affinity
+is enabled per  <PATTERN>.  If at least  one backend has
+"affinity" parameter,  and its  <METHOD> is  not "none",
+session  affinity is  enabled  for  all backend  servers
+sharing  the  same  <PATTERN>.   It is  advised  to  set
+"affinity"  parameter  to   all  backend  explicitly  if
+session affinity  is desired.  The session  affinity may
+break if one of the backend gets unreachable, or backend
+settings are reloaded or replaced by API.
 .sp
 Since ";" and ":" are  used as delimiter, <PATTERN> must
 not  contain these  characters.  Since  ";" has  special
@@ -137,7 +183,7 @@ Default: \fB127.0.0.1,80\fP
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-f, \-\-frontend=(<HOST>,<PORT>|unix:<PATH>)[;no\-tls]
+.B \-f, \-\-frontend=(<HOST>,<PORT>|unix:<PATH>)[[;PARAM]...]
 Set  frontend  host and  port.   If  <HOST> is  \(aq*\(aq,  it
 assumes  all addresses  including  both  IPv4 and  IPv6.
 UNIX domain  socket can  be specified by  prefixing path
@@ -145,8 +191,24 @@ name  with  "unix:" (e.g.,  unix:/var/run/nghttpx.sock).
 This  option can  be used  multiple times  to listen  to
 multiple addresses.
 .sp
+This option  can take  0 or  more parameters,  which are
+described  below.   Note   that  "api"  and  "healthmon"
+parameters are mutually exclusive.
+.sp
 Optionally, TLS  can be disabled by  specifying "no\-tls"
-keyword.  TLS is enabled by default.
+parameter.  TLS is enabled by default.
+.sp
+To  make this  frontend as  API endpoint,  specify "api"
+parameter.   This   is  disabled  by  default.    It  is
+important  to  limit the  access  to  the API  frontend.
+Otherwise, someone  may change  the backend  server, and
+break your services,  or expose confidential information
+to the outside the world.
+.sp
+To  make  this  frontend  as  health  monitor  endpoint,
+specify  "healthmon"  parameter.   This is  disabled  by
+default.  Any  requests which come through  this address
+are replied with 200 HTTP status, without no body.
 .sp
 Default: \fB*,3000\fP
 .UNINDENT
@@ -332,6 +394,13 @@ value is 0 then fast open is disabled.
 .sp
 Default: \fB0\fP
 .UNINDENT
+.INDENT 0.0
+.TP
+.B \-\-no\-kqueue
+Don\(aqt use  kqueue.  This  option is only  applicable for
+the platforms  which have kqueue.  For  other platforms,
+this option will be simply ignored.
+.UNINDENT
 .SS Timeout
 .INDENT 0.0
 .TP
@@ -401,6 +470,36 @@ disables this feature.
 .sp
 Default: \fB30s\fP
 .UNINDENT
+.INDENT 0.0
+.TP
+.B \-\-frontend\-http2\-setting\-timeout=<DURATION>
+Specify  timeout before  SETTINGS ACK  is received  from
+client.
+.sp
+Default: \fB10s\fP
+.UNINDENT
+.INDENT 0.0
+.TP
+.B \-\-backend\-http2\-settings\-timeout=<DURATION>
+Specify  timeout before  SETTINGS ACK  is received  from
+backend server.
+.sp
+Default: \fB10s\fP
+.UNINDENT
+.INDENT 0.0
+.TP
+.B \-\-backend\-max\-backoff=<DURATION>
+Specify  maximum backoff  interval.  This  is used  when
+doing health  check against offline backend  (see "fail"
+parameter  in \fI\%\-\-backend\fP  option).   It is  also used  to
+limit  the  maximum   interval  to  temporarily  disable
+backend  when nghttpx  failed to  connect to  it.  These
+intervals are calculated  using exponential backoff, and
+consecutive failed attempts increase the interval.  This
+option caps its maximum value.
+.sp
+Default: \fB2m\fP
+.UNINDENT
 .SS SSL/TLS
 .INDENT 0.0
 .TP
@@ -441,12 +540,6 @@ stapling work, <CERTPATH> must be absolute path.
 .UNINDENT
 .INDENT 0.0
 .TP
-.B \-\-backend\-tls\-sni\-field=<HOST>
-Explicitly  set the  content of  the TLS  SNI extension.
-This will default to the backend HOST name.
-.UNINDENT
-.INDENT 0.0
-.TP
 .B \-\-dh\-param\-file=<PATH>
 Path to file that contains  DH parameters in PEM format.
 Without  this   option,  DHE   cipher  suites   are  not
@@ -542,7 +635,7 @@ ticket  key generator  to rotate  keys frequently.   See
 "TLS SESSION  TICKET RESUMPTION" section in  manual page
 to know the data format in memcached entry.  Optionally,
 memcached  connection  can  be  encrypted  with  TLS  by
-specifying "tls" keyword.
+specifying "tls" parameter.
 .UNINDENT
 .INDENT 0.0
 .TP
@@ -628,7 +721,7 @@ Specify  address of  memcached server  to store  session
 cache.   This  enables   shared  session  cache  between
 multiple   nghttpx  instances.    Optionally,  memcached
 connection can be encrypted with TLS by specifying "tls"
-keyword.
+parameter.
 .UNINDENT
 .INDENT 0.0
 .TP
@@ -768,8 +861,8 @@ does not support server push.
 .TP
 .B (default mode)
 Accept HTTP/2, SPDY and HTTP/1.1 over SSL/TLS.  "no\-tls"
-keyword is used in  \fI\%\-\-frontend\fP option, accept HTTP/2 and
-HTTP/1.1  over  cleartext  TCP.  The  incoming  HTTP/1.1
+parameter is  used in  \fI\%\-\-frontend\fP option,  accept HTTP/2
+and HTTP/1.1 over cleartext  TCP.  The incoming HTTP/1.1
 connection  can  be  upgraded  to  HTTP/2  through  HTTP
 Upgrade.
 .UNINDENT
@@ -1030,6 +1123,14 @@ originally  generates  HTTP  error status  code  <CODE>.
 HTTP  status  code.  If  error  status  code comes  from
 backend server, the custom error pages are not used.
 .UNINDENT
+.SS API
+.INDENT 0.0
+.TP
+.B \-\-api\-max\-request\-body=<SIZE>
+Set the maximum size of request body for API request.
+.sp
+Default: \fB16K\fP
+.UNINDENT
 .SS Debug
 .INDENT 0.0
 .TP
@@ -1402,7 +1503,27 @@ Return the current phase.
 .INDENT 7.0
 .TP
 .B attribute [R] remote_addr
-Return IP address of a remote client.
+Return IP address of a remote client.  If connection is made
+via UNIX domain socket, this returns the string "localhost".
+.UNINDENT
+.INDENT 7.0
+.TP
+.B attribute [R] server_addr
+Return address of server that accepted the connection.  This
+is a string which specified in \fI\%\-\-frontend\fP option,
+excluding port number, and not a resolved IP address.  For
+UNIX domain socket, this is a path to UNIX domain socket.
+.UNINDENT
+.INDENT 7.0
+.TP
+.B attribute [R] server_port
+Return port number of the server frontend which accepted the
+connection from client.
+.UNINDENT
+.INDENT 7.0
+.TP
+.B attribute [R] tls_used
+Return true if TLS is used on the connection.
 .UNINDENT
 .UNINDENT
 .INDENT 0.0
@@ -1445,7 +1566,13 @@ value is assigned.
 .B attribute [R/W] path
 Request path, including query component (i.e., /index.html).
 On assignment, copy of given value is assigned.  The path does
-not include authority component of URI.
+not include authority component of URI.  This may include
+query component.  nghttpx makes certain normalization for
+path.  It decodes percent\-encoding for unreserved characters
+(see \fI\%https://tools.ietf.org/html/rfc3986#section\-2.3\fP), and
+resolves ".." and ".".  But it may leave characters which
+should be percent\-encoded as is. So be careful when comparing
+path against desired string.
 .UNINDENT
 .INDENT 7.0
 .TP
@@ -1478,7 +1605,7 @@ Clear all existing request header fields.
 .UNINDENT
 .INDENT 7.0
 .TP
-.B push uri
+.B push(uri)
 Initiate to push resource identified by \fIuri\fP\&.  Only HTTP/2
 protocol supports this feature.  For the other protocols, this
 method is noop.  \fIuri\fP can be absolute URI, absolute path or
@@ -1608,9 +1735,64 @@ App.new
 .fi
 .UNINDENT
 .UNINDENT
+.SH API ENDPOINTS
+.sp
+nghttpx exposes API endpoints to manipulate it via HTTP based API.  By
+default, API endpoint is disabled.  To enable it, add a dedicated
+frontend for API using \fI\%\-\-frontend\fP option with "api"
+parameter.  All requests which come from this frontend address, will
+be treated as API request.
+.sp
+The response is normally JSON dictionary, and at least includes the
+following keys:
+.INDENT 0.0
+.TP
+.B status
+The status of the request processing.  The following values are
+defined:
+.INDENT 7.0
+.TP
+.B Success
+The request was successful.
+.TP
+.B Failure
+The request was failed.  No change has been made.
+.UNINDENT
+.TP
+.B code
+HTTP status code
+.UNINDENT
+.sp
+We wrote "normally", since nghttpx may return ordinal HTML response in
+some cases where the error has occurred before reaching API endpoint
+(e.g., header field is too large).
+.sp
+The following section describes available API endpoints.
+.SS PUT /api/v1beta1/backendconfig
+.sp
+This API replaces the current backend server settings with the
+requested ones.  The request method should be PUT, but POST is also
+acceptable.  The request body must be nghttpx configuration file
+format.  For configuration file format, see \fI\%FILES\fP section.  The
+line separator inside the request body must be single LF (0x0A).
+Currently, only \fI\%backend\fP option is parsed, the
+others are simply ignored.  The semantics of this API is replace the
+current backend with the backend options in request body.  Describe
+the desired set of backend severs, and nghttpx makes it happen.  If
+there is no \fI\%backend\fP option is found in request
+body, the current set of backend is replaced with the \fI\%backend\fP option\(aqs default value, which is \fB127.0.0.1,80\fP\&.
+.sp
+The replacement is done instantly without breaking existing
+connections or requests.  It also avoids any process creation as is
+the case with hot swapping with signals.
+.sp
+The one limitation is that only numeric IP address is allowd in
+\fI\%backend\fP in request body while non numeric
+hostname is allowed in command\-line or configuration file is read
+using \fI\%\-\-conf\fP\&.
 .SH SEE ALSO
 .sp
-\fInghttp(1)\fP, \fInghttpd(1)\fP, \fIh2load(1)\fP
+\fBnghttp(1)\fP, \fBnghttpd(1)\fP, \fBh2load(1)\fP
 .SH AUTHOR
 Tatsuhiro Tsujikawa
 .SH COPYRIGHT

doc/nghttpx.1.rst

@@ -20,13 +20,13 @@ A reverse proxy for HTTP/2, HTTP/1 and SPDY.
 
     
     Set  path  to  server's private  key.   Required  unless
-    "no-tls" keyword is used in :option:`--frontend` option.
+    "no-tls" parameter is used in :option:`--frontend` option.
 
 .. describe:: <CERT>
 
     Set  path  to  server's  certificate.   Required  unless
-    "no-tls" keyword is used  in :option:`--frontend` option.  To make
-    OCSP stapling work, this must be an absolute path.
+    "no-tls"  parameter is  used in  :option:`--frontend` option.   To
+    make OCSP stapling work, this must be an absolute path.
 
 
 OPTIONS
@@ -37,7 +37,8 @@ The options are categorized into several groups.
 Connections
 ~~~~~~~~~~~
 
-.. option:: -b, --backend=(<HOST>,<PORT>|unix:<PATH>)[;[<PATTERN>[:...]][;proto=<PROTO>][;tls]]
+.. option:: -b, --backend=(<HOST>,<PORT>|unix:<PATH>)[;[<PATTERN>[:...]][[;PARAM]...]
+
 
     Set  backend  host  and   port.   The  multiple  backend
     addresses are  accepted by repeating this  option.  UNIX
@@ -100,17 +101,63 @@ Connections
     The backend addresses sharing same <PATTERN> are grouped
     together forming  load balancing  group.
 
-    Optionally,   backend   application  protocol   can   be
-    specified in <PROTO>.  All that share the same <PATTERN>
-    must  have  the  same  <PROTO> value  if  it  is  given.
-    <PROTO>  should be  one  of the  following list  without
-    quotes: "h2", "http/1.1".  The  default value of <PROTO>
-    is "http/1.1".  Note that  usually "h2" refers to HTTP/2
-    over TLS.  But  in this option, it may  mean HTTP/2 over
-    cleartext TCP unless "tls" keyword is used (see below).
+    Several parameters <PARAM> are accepted after <PATTERN>.
+    The  parameters are  delimited  by  ";".  The  available
+    parameters       are:      "proto=<PROTO>",       "tls",
+    "sni=<SNI_HOST>",     "fall=<N>",    "rise=<N>",     and
+    "affinity=<METHOD>".  The parameter consists of keyword,
+    and optionally followed by  "=" and value.  For example,
+    the parameter "proto=h2" consists of the keyword "proto"
+    and  value "h2".   The parameter  "tls" consists  of the
+    keyword   "tls"  without   value.   Each   parameter  is
+    described as follows.
 
-    Optionally,  TLS  can  be enabled  by  specifying  "tls"
-    keyword.  TLS is not enabled by default.
+    The backend application protocol  can be specified using
+    optional  "proto"   parameter,  and   in  the   form  of
+    "proto=<PROTO>".  <PROTO> should be one of the following
+    list  without  quotes:  "h2", "http/1.1".   The  default
+    value of <PROTO> is  "http/1.1".  Note that usually "h2"
+    refers to HTTP/2  over TLS.  But in this  option, it may
+    mean HTTP/2  over cleartext TCP unless  "tls" keyword is
+    used (see below).
+
+    TLS  can   be  enabled  by  specifying   optional  "tls"
+    parameter.  TLS is not enabled by default.
+
+    With "sni=<SNI_HOST>" parameter, it can override the TLS
+    SNI  field  value  with  given  <SNI_HOST>.   This  will
+    default to the backend <HOST> name
+
+    The  feature  to detect  whether  backend  is online  or
+    offline can be enabled  using optional "fall" and "rise"
+    parameters.   Using  "fall=<N>"  parameter,  if  nghttpx
+    cannot connect  to a  this backend <N>  times in  a row,
+    this  backend  is  assumed  to be  offline,  and  it  is
+    excluded from load balancing.  If <N> is 0, this backend
+    never  be excluded  from load  balancing whatever  times
+    nghttpx cannot connect  to it, and this  is the default.
+    There is  also "rise=<N>" parameter.  After  backend was
+    excluded from load balancing group, nghttpx periodically
+    attempts to make a connection to the failed backend, and
+    if the  connection is made  successfully <N> times  in a
+    row, the backend is assumed to  be online, and it is now
+    eligible  for load  balancing target.   If <N>  is 0,  a
+    backend  is permanently  offline, once  it goes  in that
+    state, and this is the default behaviour.
+
+    The     session     affinity    is     enabled     using
+    "affinity=<METHOD>"  parameter.   If  "ip" is  given  in
+    <METHOD>, client  IP based session affinity  is enabled.
+    If  "none" is  given  in <METHOD>,  session affinity  is
+    disabled, and this is the default.  The session affinity
+    is enabled per  <PATTERN>.  If at least  one backend has
+    "affinity" parameter,  and its  <METHOD> is  not "none",
+    session  affinity is  enabled  for  all backend  servers
+    sharing  the  same  <PATTERN>.   It is  advised  to  set
+    "affinity"  parameter  to   all  backend  explicitly  if
+    session affinity  is desired.  The session  affinity may
+    break if one of the backend gets unreachable, or backend
+    settings are reloaded or replaced by API.
 
     Since ";" and ":" are  used as delimiter, <PATTERN> must
     not  contain these  characters.  Since  ";" has  special
@@ -119,7 +166,7 @@ Connections
 
     Default: ``127.0.0.1,80``
 
-.. option:: -f, --frontend=(<HOST>,<PORT>|unix:<PATH>)[;no-tls]
+.. option:: -f, --frontend=(<HOST>,<PORT>|unix:<PATH>)[[;PARAM]...]
 
     Set  frontend  host and  port.   If  <HOST> is  '\*',  it
     assumes  all addresses  including  both  IPv4 and  IPv6.
@@ -128,8 +175,24 @@ Connections
     This  option can  be used  multiple times  to listen  to
     multiple addresses.
 
+    This option  can take  0 or  more parameters,  which are
+    described  below.   Note   that  "api"  and  "healthmon"
+    parameters are mutually exclusive.
+
     Optionally, TLS  can be disabled by  specifying "no-tls"
-    keyword.  TLS is enabled by default.
+    parameter.  TLS is enabled by default.
+
+    To  make this  frontend as  API endpoint,  specify "api"
+    parameter.   This   is  disabled  by  default.    It  is
+    important  to  limit the  access  to  the API  frontend.
+    Otherwise, someone  may change  the backend  server, and
+    break your services,  or expose confidential information
+    to the outside the world.
+
+    To  make  this  frontend  as  health  monitor  endpoint,
+    specify  "healthmon"  parameter.   This is  disabled  by
+    default.  Any  requests which come through  this address
+    are replied with 200 HTTP status, without no body.
 
 
     Default: ``*,3000``
@@ -299,6 +362,13 @@ Performance
 
     Default: ``0``
 
+.. option:: --no-kqueue
+
+    Don't use  kqueue.  This  option is only  applicable for
+    the platforms  which have kqueue.  For  other platforms,
+    this option will be simply ignored.
+
+
 Timeout
 ~~~~~~~
 
@@ -361,6 +431,33 @@ Timeout
 
     Default: ``30s``
 
+.. option:: --frontend-http2-setting-timeout=<DURATION>
+
+    Specify  timeout before  SETTINGS ACK  is received  from
+    client.
+
+    Default: ``10s``
+
+.. option:: --backend-http2-settings-timeout=<DURATION>
+
+    Specify  timeout before  SETTINGS ACK  is received  from
+    backend server.
+
+    Default: ``10s``
+
+.. option:: --backend-max-backoff=<DURATION>
+
+    Specify  maximum backoff  interval.  This  is used  when
+    doing health  check against offline backend  (see "fail"
+    parameter  in :option:`--backend`  option).   It is  also used  to
+    limit  the  maximum   interval  to  temporarily  disable
+    backend  when nghttpx  failed to  connect to  it.  These
+    intervals are calculated  using exponential backoff, and
+    consecutive failed attempts increase the interval.  This
+    option caps its maximum value.
+
+    Default: ``2m``
+
 
 SSL/TLS
 ~~~~~~~
@@ -397,11 +494,6 @@ SSL/TLS
     option  can  be  used  multiple  times.   To  make  OCSP
     stapling work, <CERTPATH> must be absolute path.
 
-.. option:: --backend-tls-sni-field=<HOST>
-
-    Explicitly  set the  content of  the TLS  SNI extension.
-    This will default to the backend HOST name.
-
 .. option:: --dh-param-file=<PATH>
 
     Path to file that contains  DH parameters in PEM format.
@@ -490,7 +582,7 @@ SSL/TLS
     "TLS SESSION  TICKET RESUMPTION" section in  manual page
     to know the data format in memcached entry.  Optionally,
     memcached  connection  can  be  encrypted  with  TLS  by
-    specifying "tls" keyword.
+    specifying "tls" parameter.
 
 .. option:: --tls-ticket-key-memcached-address-family=(auto|IPv4|IPv6)
 
@@ -565,7 +657,7 @@ SSL/TLS
     cache.   This  enables   shared  session  cache  between
     multiple   nghttpx  instances.    Optionally,  memcached
     connection can be encrypted with TLS by specifying "tls"
-    keyword.
+    parameter.
 
 .. option:: --tls-session-cache-memcached-address-family=(auto|IPv4|IPv6)
 
@@ -696,8 +788,8 @@ Mode
 
     
     Accept HTTP/2, SPDY and HTTP/1.1 over SSL/TLS.  "no-tls"
-    keyword is used in  :option:`--frontend` option, accept HTTP/2 and
-    HTTP/1.1  over  cleartext  TCP.  The  incoming  HTTP/1.1
+    parameter is  used in  :option:`--frontend` option,  accept HTTP/2
+    and HTTP/1.1 over cleartext  TCP.  The incoming HTTP/1.1
     connection  can  be  upgraded  to  HTTP/2  through  HTTP
     Upgrade.
 
@@ -924,6 +1016,16 @@ HTTP
     backend server, the custom error pages are not used.
 
 
+API
+~~~
+
+.. option:: --api-max-request-body=<SIZE>
+
+    Set the maximum size of request body for API request.
+
+    Default: ``16K``
+
+
 Debug
 ~~~~~
 
@@ -1097,7 +1199,7 @@ backend server and extracts URI-reference with parameter
 and pushes those URIs to the frontend client. Here is a sample Link
 header field to initiate server push:
 
-.. code-block:: http
+.. code-block:: text
 
   Link: </fonts/font.woff>; rel=preload
   Link: </css/theme.css>; rel=preload
@@ -1277,7 +1379,24 @@ respectively.
 
     .. rb:attr_reader:: remote_addr
 
-        Return IP address of a remote client.
+        Return IP address of a remote client.  If connection is made
+        via UNIX domain socket, this returns the string "localhost".
+
+    .. rb:attr_reader:: server_addr
+
+        Return address of server that accepted the connection.  This
+	is a string which specified in :option:`--frontend` option,
+	excluding port number, and not a resolved IP address.  For
+	UNIX domain socket, this is a path to UNIX domain socket.
+
+    .. rb:attr_reader:: server_port
+
+        Return port number of the server frontend which accepted the
+        connection from client.
+
+    .. rb:attr_reader:: tls_used
+
+        Return true if TLS is used on the connection.
 
 .. rb:class:: Request
 
@@ -1313,7 +1432,13 @@ respectively.
 
         Request path, including query component (i.e., /index.html).
         On assignment, copy of given value is assigned.  The path does
-        not include authority component of URI.
+        not include authority component of URI.  This may include
+        query component.  nghttpx makes certain normalization for
+        path.  It decodes percent-encoding for unreserved characters
+        (see https://tools.ietf.org/html/rfc3986#section-2.3), and
+        resolves ".." and ".".  But it may leave characters which
+        should be percent-encoded as is. So be careful when comparing
+        path against desired string.
 
     .. rb:attr_reader:: headers
 
@@ -1340,7 +1465,7 @@ respectively.
 
         Clear all existing request header fields.
 
-    .. rb:method:: push uri
+    .. rb:method:: push(uri)
 
         Initiate to push resource identified by *uri*.  Only HTTP/2
         protocol supports this feature.  For the other protocols, this
@@ -1451,6 +1576,62 @@ addresses:
 
     App.new
 
+API ENDPOINTS
+-------------
+
+nghttpx exposes API endpoints to manipulate it via HTTP based API.  By
+default, API endpoint is disabled.  To enable it, add a dedicated
+frontend for API using :option:`--frontend` option with "api"
+parameter.  All requests which come from this frontend address, will
+be treated as API request.
+
+The response is normally JSON dictionary, and at least includes the
+following keys:
+
+status
+  The status of the request processing.  The following values are
+  defined:
+
+  Success
+    The request was successful.
+
+  Failure
+    The request was failed.  No change has been made.
+
+code
+  HTTP status code
+
+We wrote "normally", since nghttpx may return ordinal HTML response in
+some cases where the error has occurred before reaching API endpoint
+(e.g., header field is too large).
+
+The following section describes available API endpoints.
+
+PUT /api/v1beta1/backendconfig
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This API replaces the current backend server settings with the
+requested ones.  The request method should be PUT, but POST is also
+acceptable.  The request body must be nghttpx configuration file
+format.  For configuration file format, see `FILES`_ section.  The
+line separator inside the request body must be single LF (0x0A).
+Currently, only :option:`backend <--backend>` option is parsed, the
+others are simply ignored.  The semantics of this API is replace the
+current backend with the backend options in request body.  Describe
+the desired set of backend severs, and nghttpx makes it happen.  If
+there is no :option:`backend <--backend>` option is found in request
+body, the current set of backend is replaced with the :option:`backend
+<--backend>` option's default value, which is ``127.0.0.1,80``.
+
+The replacement is done instantly without breaking existing
+connections or requests.  It also avoids any process creation as is
+the case with hot swapping with signals.
+
+The one limitation is that only numeric IP address is allowd in
+:option:`backend <--backend>` in request body while non numeric
+hostname is allowed in command-line or configuration file is read
+using :option:`--conf`.
+
 SEE ALSO
 --------
 

doc/nghttpx.h2r

@@ -85,7 +85,7 @@ backend server and extracts URI-reference with parameter
 and pushes those URIs to the frontend client. Here is a sample Link
 header field to initiate server push:
 
-.. code-block:: http
+.. code-block:: text
 
   Link: </fonts/font.woff>; rel=preload
   Link: </css/theme.css>; rel=preload
@@ -265,7 +265,24 @@ respectively.
 
     .. rb:attr_reader:: remote_addr
 
-        Return IP address of a remote client.
+        Return IP address of a remote client.  If connection is made
+        via UNIX domain socket, this returns the string "localhost".
+
+    .. rb:attr_reader:: server_addr
+
+        Return address of server that accepted the connection.  This
+	is a string which specified in :option:`--frontend` option,
+	excluding port number, and not a resolved IP address.  For
+	UNIX domain socket, this is a path to UNIX domain socket.
+
+    .. rb:attr_reader:: server_port
+
+        Return port number of the server frontend which accepted the
+        connection from client.
+
+    .. rb:attr_reader:: tls_used
+
+        Return true if TLS is used on the connection.
 
 .. rb:class:: Request
 
@@ -301,7 +318,13 @@ respectively.
 
         Request path, including query component (i.e., /index.html).
         On assignment, copy of given value is assigned.  The path does
-        not include authority component of URI.
+        not include authority component of URI.  This may include
+        query component.  nghttpx makes certain normalization for
+        path.  It decodes percent-encoding for unreserved characters
+        (see https://tools.ietf.org/html/rfc3986#section-2.3), and
+        resolves ".." and ".".  But it may leave characters which
+        should be percent-encoded as is. So be careful when comparing
+        path against desired string.
 
     .. rb:attr_reader:: headers
 
@@ -328,7 +351,7 @@ respectively.
 
         Clear all existing request header fields.
 
-    .. rb:method:: push uri
+    .. rb:method:: push(uri)
 
         Initiate to push resource identified by *uri*.  Only HTTP/2
         protocol supports this feature.  For the other protocols, this
@@ -439,6 +462,62 @@ addresses:
 
     App.new
 
+API ENDPOINTS
+-------------
+
+nghttpx exposes API endpoints to manipulate it via HTTP based API.  By
+default, API endpoint is disabled.  To enable it, add a dedicated
+frontend for API using :option:`--frontend` option with "api"
+parameter.  All requests which come from this frontend address, will
+be treated as API request.
+
+The response is normally JSON dictionary, and at least includes the
+following keys:
+
+status
+  The status of the request processing.  The following values are
+  defined:
+
+  Success
+    The request was successful.
+
+  Failure
+    The request was failed.  No change has been made.
+
+code
+  HTTP status code
+
+We wrote "normally", since nghttpx may return ordinal HTML response in
+some cases where the error has occurred before reaching API endpoint
+(e.g., header field is too large).
+
+The following section describes available API endpoints.
+
+PUT /api/v1beta1/backendconfig
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This API replaces the current backend server settings with the
+requested ones.  The request method should be PUT, but POST is also
+acceptable.  The request body must be nghttpx configuration file
+format.  For configuration file format, see `FILES`_ section.  The
+line separator inside the request body must be single LF (0x0A).
+Currently, only :option:`backend <--backend>` option is parsed, the
+others are simply ignored.  The semantics of this API is replace the
+current backend with the backend options in request body.  Describe
+the desired set of backend severs, and nghttpx makes it happen.  If
+there is no :option:`backend <--backend>` option is found in request
+body, the current set of backend is replaced with the :option:`backend
+<--backend>` option's default value, which is ``127.0.0.1,80``.
+
+The replacement is done instantly without breaking existing
+connections or requests.  It also avoids any process creation as is
+the case with hot swapping with signals.
+
+The one limitation is that only numeric IP address is allowd in
+:option:`backend <--backend>` in request body while non numeric
+hostname is allowed in command-line or configuration file is read
+using :option:`--conf`.
+
 SEE ALSO
 --------
 

doc/programmers-guide.rst

@@ -177,12 +177,15 @@ Any deviation results in stream error of type PROTOCOL_ERROR.  If
 error is found in PUSH_PROMISE frame, stream error is raised against
 promised stream.
 
-Implement HTTP/2 non-critical extensions
-----------------------------------------
+Implement user defined HTTP/2 non-critical extensions
+-----------------------------------------------------
 
 As of nghttp2 v1.8.0, we have added HTTP/2 non-critical extension
-framework, which lets application send and receive HTTP/2 non-critical
-extension frames.
+framework, which lets application send and receive user defined custom
+HTTP/2 non-critical extension frames.  nghttp2 also offers built-in
+functionality to send and receive official HTTP/2 extension frames
+(e.g., ALTSVC frame).  For these built-in handler, refer to the next
+section.
 
 To send extension frame, use `nghttp2_submit_extension()`, and
 implement :type:`nghttp2_pack_extension_callback`.  The callback
@@ -383,3 +386,41 @@ its creation:
 .. code-block:: c
 
     nghttp2_session_client_new2(&session, callbacks, user_data, option);
+
+How to use built-in HTTP/2 extension frame handlers
+---------------------------------------------------
+
+In the previous section, we talked about the user defined HTTP/2
+extension frames.  In this section, we talk about HTTP/2 extension
+frame support built into nghttp2 library.
+
+As of this writing, nghttp2 supports ALTSVC extension frame.  To send
+ALTSVC frame, use `nghttp2_submit_altsvc()` function.
+
+To receive ALTSVC frame through built-in functionality, application
+has to use `nghttp2_option_set_builtin_recv_extension_type()` to
+indicate the willingness of receiving ALTSVC frame:
+
+.. code-block:: c
+
+    nghttp2_option_set_builtin_recv_extension_type(option, NGHTTP2_ALTSVC);
+
+This is very similar to the case when we used to receive user defined
+frames.
+
+If the same frame type is set using
+`nghttp2_option_set_builtin_recv_extension_type()` and
+`nghttp2_option_set_user_recv_extension_type()`, the latter takes
+precedence.  Application can implement its own frame handler rather
+than using built-in handler.
+
+The :type:`nghttp2_option` must be set to :type:`nghttp2_session` on
+its creation, like so:
+
+.. code-block:: c
+
+    nghttp2_session_client_new2(&session, callbacks, user_data, option);
+
+When ALTSVC is received, :type:`nghttp2_on_frame_recv_callback` will
+be called as usual.
+

doc/sources/building-android-binary.rst

@@ -17,19 +17,16 @@ installed in the following way.  First, let us introduce
 under ``$ANDROID_HOME/toolchain``.  An user can freely choose the path
 for ``ANDROID_HOME``.  For example, to install toolchain under
 ``$ANDROID_HOME/toolchain``, do this in the the directory where NDK is
-unpacked::
+unpacked:
 
-    $ build/tools/make-standalone-toolchain.sh \
-      --install-dir=$ANDROID_HOME/toolchain \
-      --toolchain=arm-linux-androideabi-4.9 \
-      --llvm-version=3.5 \
-      --platform=android-16
+.. code-block:: text
 
-The additional flag ``--system=linux-x86_64`` may be required if you
-are using x86_64 system.
+    $ build/tools/make_standalone_toolchain.py \
+      --arch arm --api 16 --stl gnustl
+      --install-dir $ANDROID_HOME/toolchain
 
-The platform level is not important here because we don't use Android
-specific C/C++ API.
+The API level (``--api``) is not important here because we don't use
+Android specific C/C++ API.
 
 The dependent libraries, such as OpenSSL and libev should be built
 with the toolchain and installed under ``$ANDROID_HOME/usr/local``.
@@ -45,7 +42,9 @@ spdylay as well.
 
 Before running ``android-config`` and ``android-make``,
 ``ANDROID_HOME`` environment variable must be set to point to the
-correct path.  Also add ``$ANDROID_HOME/toolchain/bin`` to ``PATH``::
+correct path.  Also add ``$ANDROID_HOME/toolchain/bin`` to ``PATH``:
+
+.. code-block:: text
 
     $ export PATH=$PATH:$ANDROID_HOME/toolchain/bin
 
@@ -133,24 +132,24 @@ To configure spdylay, use the following script:
     #!/bin/sh -e
 
     if [ -z "$ANDROID_HOME" ]; then
-	echo 'No $ANDROID_HOME specified.'
-	exit 1
+        echo 'No $ANDROID_HOME specified.'
+        exit 1
     fi
     PREFIX=$ANDROID_HOME/usr/local
     TOOLCHAIN=$ANDROID_HOME/toolchain
     PATH=$TOOLCHAIN/bin:$PATH
 
     ./configure \
-	--disable-shared \
-	--host=arm-linux-androideabi \
-	--build=`dpkg-architecture -qDEB_BUILD_GNU_TYPE` \
-	--prefix=$PREFIX \
-	--without-libxml2 \
-	--disable-src \
-	--disable-examples \
-	CPPFLAGS="-I$PREFIX/include" \
-	PKG_CONFIG_LIBDIR="$PREFIX/lib/pkgconfig" \
-	LDFLAGS="-L$PREFIX/lib"
+        --disable-shared \
+        --host=arm-linux-androideabi \
+        --build=`dpkg-architecture -qDEB_BUILD_GNU_TYPE` \
+        --prefix=$PREFIX \
+        --without-libxml2 \
+        --disable-src \
+        --disable-examples \
+        CPPFLAGS="-I$PREFIX/include" \
+        PKG_CONFIG_LIBDIR="$PREFIX/lib/pkgconfig" \
+        LDFLAGS="-L$PREFIX/lib"
 
 And run ``make install`` to build and install.
 
@@ -159,6 +158,8 @@ then ``android-make`` to compile nghttp2 source files.
 
 If all went well, application binaries, such as nghttpx, are created
 under src directory.  Strip debugging information from the binary
-using the following command::
+using the following command:
+
+.. code-block:: text
 
     $ arm-linux-androideabi-strip src/nghttpx

doc/sources/index.rst

@@ -51,3 +51,4 @@ Resources
 
 * HTTP/2 https://tools.ietf.org/html/rfc7540
 * HPACK https://tools.ietf.org/html/rfc7541
+* HTTP Alternative Services https://tools.ietf.org/html/rfc7838

doc/sources/nghttpx-howto.rst

@@ -48,12 +48,16 @@ explicitly.
 The backend is supposed to be Web server.  For example, to make
 nghttpx listen to encrypted HTTP/2 requests at port 8443, and a
 backend Web server is configured to listen to HTTP request at port
-8080 in the same host, run nghttpx command-line like this::
+8080 in the same host, run nghttpx command-line like this:
+
+.. code-block:: text
 
     $ nghttpx -f0.0.0.0,8443 -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
 
 Then HTTP/2 enabled client can access to the nghttpx in HTTP/2.  For
-example, you can send GET request to the server using nghttp::
+example, you can send GET request to the server using nghttp:
+
+.. code-block:: text
 
     $ nghttp -nv https://localhost:8443/
 
@@ -89,7 +93,9 @@ connection, use :option:`--backend` option, and specify ``h2`` in
 For example, to make nghttpx listen to encrypted HTTP/2 requests at
 port 8443, and a backend HTTP proxy server is configured to listen to
 HTTP/1 request at port 8080 in the same host, run nghttpx command-line
-like this::
+like this:
+
+.. code-block:: text
 
     $ nghttpx -s -f'*,8443' -b127.0.0.1,8080 /path/to/server.key /path/to/server.crt
 
@@ -118,13 +124,17 @@ to proxy.pac file, something like this:
 
     file:///path/to/proxy.pac
 
-For Chromium, use following command-line::
+For Chromium, use following command-line:
+
+.. code-block:: text
 
     $ google-chrome --proxy-pac-url=file:///path/to/proxy.pac --use-npn
 
 As HTTP/1 proxy server, Squid may work as out-of-box.  Traffic server
 requires to be configured as forward proxy.  Here is the minimum
-configuration items to edit::
+configuration items to edit:
+
+.. code-block:: text
 
     CONFIG proxy.config.reverse_proxy.enabled INT 0
     CONFIG proxy.config.url_remap.remap_required INT 0
@@ -152,9 +162,9 @@ Enable SSL/TLS on memcached connection
 --------------------------------------
 
 By default, memcached connection is not encrypted.  To enable
-encryption, use :option:`--tls-ticket-key-memcached-tls` for TLS
-ticket key, and use :option:`--tls-session-cache-memcached-tls` for
-TLS session cache.
+encryption, use ``tls`` keyword in
+:option:`--tls-ticket-key-memcached` for TLS ticket key, and
+:option:`--tls-session-cache-memcached` for TLS session cache.
 
 Specifying additional server certificates
 -----------------------------------------
@@ -383,3 +393,12 @@ Use following options instead of ``--client-proxy``:
    http2-proxy=yes
    frontend=<ADDR>,<PORT>;no-tls
    backend=<ADDR>,<PORT>;;proto=h2;tls
+
+We also removed ``--backend-http2-connections-per-worker`` option.  It
+was present because previously the number of backend h2 connection was
+statically configured, and defaulted to 1.  Now the number of backend
+h2 connection is increased on demand.  We know the maximum number of
+concurrent streams per connection.  When we push as many request as
+the maximum concurrency to the one connection, we create another new
+connection so that we can distribute load and avoid delay the request
+processing.  This is done automatically without any configuration.

doc/sources/python-apiref.rst

@@ -140,7 +140,9 @@ HTTP/2 servers
    Python 3.4 or later is required to use these objects.  To
    explicitly configure nghttp2 build to use Python 3.4, specify the
    ``PYTHON`` variable to the path to Python 3.4 executable when
-   invoking configure script like this::
+   invoking configure script like this:
+
+   .. code-block:: text
 
        $ ./configure PYTHON=/usr/bin/python3.4
 

doc/sources/tutorial-client.rst

@@ -7,7 +7,9 @@ the end of this page.  It also resides in the examples directory in
 the archive or repository.
 
 This simple client takes a single HTTPS URI and retrieves the resource
-at the URI. The synopsis is::
+at the URI. The synopsis is:
+
+.. code-block:: text
 
     $ libevent-client HTTPS_URI
 
@@ -31,6 +33,17 @@ protocol the library supports::
       return SSL_TLSEXT_ERR_OK;
     }
 
+If you are following TLS related RFC, you know that NPN is not the
+standardized way to negotiate HTTP/2.  NPN itself is not event
+published as RFC.  The standard way to negotiate HTTP/2 is ALPN,
+Application-Layer Protocol Negotiation Extension, defined in `RFC 7301
+<https://tools.ietf.org/html/rfc7301>`_.  The one caveat of ALPN is
+that OpenSSL >= 1.0.2 is required.  We use macro to enable/disable
+ALPN support depending on OpenSSL version.  OpenSSL's ALPN
+implementation does not require callback function like the above.  But
+we have to instruct OpenSSL SSL_CTX to use ALPN, which we'll talk
+about soon.
+
 The callback is added to the SSL_CTX object using
 ``SSL_CTX_set_next_proto_select_cb()``::
 
@@ -46,9 +59,18 @@ The callback is added to the SSL_CTX object using
                               SSL_OP_NO_COMPRESSION |
                               SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION);
       SSL_CTX_set_next_proto_select_cb(ssl_ctx, select_next_proto_cb, NULL);
+
+    #if OPENSSL_VERSION_NUMBER >= 0x10002000L
+      SSL_CTX_set_alpn_protos(ssl_ctx, (const unsigned char *)"\x02h2", 3);
+    #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
       return ssl_ctx;
     }
 
+Here we see ``SSL_CTX_get_alpn_protos()`` function call.  We instructs
+OpenSSL to notify the server that we support h2, ALPN identifier for
+HTTP/2.
+
 The example client defines a couple of structs:
 
 We define and use a ``http2_session_data`` structure to store data
@@ -124,7 +146,27 @@ underlying network socket::
       if (events & BEV_EVENT_CONNECTED) {
         int fd = bufferevent_getfd(bev);
         int val = 1;
+        const unsigned char *alpn = NULL;
+        unsigned int alpnlen = 0;
+        SSL *ssl;
+
         fprintf(stderr, "Connected\n");
+
+        ssl = bufferevent_openssl_get_ssl(session_data->bev);
+
+        SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen);
+    #if OPENSSL_VERSION_NUMBER >= 0x10002000L
+        if (alpn == NULL) {
+          SSL_get0_alpn_selected(ssl, &alpn, &alpnlen);
+        }
+    #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
+        if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) {
+          fprintf(stderr, "h2 is not negotiated\n");
+          delete_http2_session_data(session_data);
+          return;
+        }
+
         setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (char *)&val, sizeof(val));
         initialize_nghttp2_session(session_data);
         send_client_connection_header(session_data);
@@ -144,6 +186,9 @@ underlying network socket::
       delete_http2_session_data(session_data);
     }
 
+Here we validate that HTTP/2 is negotiated, and if not, drop
+connection.
+
 For ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR``, and ``BEV_EVENT_TIMEOUT``
 events, we just simply tear down the connection.
 

doc/sources/tutorial-server.rst

@@ -10,7 +10,9 @@ archive or repository.
 
 This simple server takes 3 arguments: The port number to listen on,
 the path to your SSL/TLS private key file, and the path to your
-certificate file.  The synopsis is::
+certificate file.  The synopsis is:
+
+.. code-block:: text
 
     $ libevent-server PORT /path/to/server.key /path/to/server.crt
 
@@ -25,7 +27,17 @@ application protocols the server supports to a client.  In this
 example program, when creating the ``SSL_CTX`` object, we store the
 application protocol name in the wire format of NPN in a statically
 allocated buffer. This is safe because we only create one ``SSL_CTX``
-object in the program's entire lifetime::
+object in the program's entire lifetime.
+
+If you are following TLS related RFC, you know that NPN is not the
+standardized way to negotiate HTTP/2.  NPN itself is not even
+published as RFC.  The standard way to negotiate HTTP/2 is ALPN,
+Application-Layer Protocol Negotiation Extension, defined in `RFC 7301
+<https://tools.ietf.org/html/rfc7301>`_.  The one caveat of ALPN is
+that OpenSSL >= 1.0.2 is required.  We use macro to enable/disable
+ALPN support depending on OpenSSL version.  In ALPN, client sends the
+list of supported application protocols, and server selects one of
+them.  We provide the callback for it::
 
     static unsigned char next_proto_list[256];
     static size_t next_proto_list_len;
@@ -37,6 +49,22 @@ object in the program's entire lifetime::
       return SSL_TLSEXT_ERR_OK;
     }
 
+    #if OPENSSL_VERSION_NUMBER >= 0x10002000L
+    static int alpn_select_proto_cb(SSL *ssl _U_, const unsigned char **out,
+                                    unsigned char *outlen, const unsigned char *in,
+                                    unsigned int inlen, void *arg _U_) {
+      int rv;
+
+      rv = nghttp2_select_next_protocol((unsigned char **)out, outlen, in, inlen);
+
+      if (rv != 1) {
+        return SSL_TLSEXT_ERR_NOACK;
+      }
+
+      return SSL_TLSEXT_ERR_OK;
+    }
+    #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
     static SSL_CTX *create_ssl_ctx(const char *key_file, const char *cert_file) {
       SSL_CTX *ssl_ctx;
       EC_KEY *ecdh;
@@ -51,6 +79,11 @@ object in the program's entire lifetime::
       next_proto_list_len = 1 + NGHTTP2_PROTO_VERSION_ID_LEN;
 
       SSL_CTX_set_next_protos_advertised_cb(ssl_ctx, next_proto_cb, NULL);
+
+    #if OPENSSL_VERSION_NUMBER >= 0x10002000L
+      SSL_CTX_set_alpn_select_cb(ssl_ctx, alpn_select_proto_cb, NULL);
+    #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
       return ssl_ctx;
     }
 
@@ -64,6 +97,11 @@ OpenSSL implementation, we just assign the pointer to the NPN buffers
 we filled in earlier. The NPN callback function is set to the
 ``SSL_CTX`` object using ``SSL_CTX_set_next_protos_advertised_cb()``.
 
+In ``alpn_select_proto_cb()``, we use `nghttp2_select_next_protocol()`
+to select application protocol.  The `nghttp2_select_next_protocol()`
+returns 1 only if it selected h2 (ALPN identifier for HTTP/2), and out
+parameters were assigned accordingly.
+
 Next, let's take a look at the main structures used by the example
 application:
 
@@ -167,11 +205,31 @@ underlying network socket::
     static void eventcb(struct bufferevent *bev _U_, short events, void *ptr) {
       http2_session_data *session_data = (http2_session_data *)ptr;
       if (events & BEV_EVENT_CONNECTED) {
+        const unsigned char *alpn = NULL;
+        unsigned int alpnlen = 0;
+        SSL *ssl;
+
         fprintf(stderr, "%s connected\n", session_data->client_addr);
 
+        ssl = bufferevent_openssl_get_ssl(session_data->bev);
+
+        SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen);
+    #if OPENSSL_VERSION_NUMBER >= 0x10002000L
+        if (alpn == NULL) {
+          SSL_get0_alpn_selected(ssl, &alpn, &alpnlen);
+        }
+    #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
+        if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) {
+          fprintf(stderr, "%s h2 is not negotiated\n", session_data->client_addr);
+          delete_http2_session_data(session_data);
+          return;
+        }
+
         initialize_nghttp2_session(session_data);
 
-        if (send_server_connection_header(session_data) != 0) {
+        if (send_server_connection_header(session_data) != 0 ||
+            session_send(session_data) != 0) {
           delete_http2_session_data(session_data);
           return;
         }
@@ -188,6 +246,9 @@ underlying network socket::
       delete_http2_session_data(session_data);
     }
 
+Here we validate that HTTP/2 is negotiated, and if not, drop
+connection.
+
 For the ``BEV_EVENT_EOF``, ``BEV_EVENT_ERROR``, and
 ``BEV_EVENT_TIMEOUT`` events, we just simply tear down the connection.
 The ``delete_http2_session_data()`` function destroys the

examples/client.c

@@ -219,9 +219,9 @@ static int on_frame_send_callback(nghttp2_session *session,
       const nghttp2_nv *nva = frame->headers.nva;
       printf("[INFO] C ----------------------------> S (HEADERS)\n");
       for (i = 0; i < frame->headers.nvlen; ++i) {
-        fwrite(nva[i].name, nva[i].namelen, 1, stdout);
+        fwrite(nva[i].name, 1, nva[i].namelen, stdout);
         printf(": ");
-        fwrite(nva[i].value, nva[i].valuelen, 1, stdout);
+        fwrite(nva[i].value, 1, nva[i].valuelen, stdout);
         printf("\n");
       }
     }
@@ -249,9 +249,9 @@ static int on_frame_recv_callback(nghttp2_session *session,
       if (req) {
         printf("[INFO] C <---------------------------- S (HEADERS)\n");
         for (i = 0; i < frame->headers.nvlen; ++i) {
-          fwrite(nva[i].name, nva[i].namelen, 1, stdout);
+          fwrite(nva[i].name, 1, nva[i].namelen, stdout);
           printf(": ");
-          fwrite(nva[i].value, nva[i].valuelen, 1, stdout);
+          fwrite(nva[i].value, 1, nva[i].valuelen, stdout);
           printf("\n");
         }
       }
@@ -562,7 +562,11 @@ static void fetch_uri(const struct URI *uri) {
     diec("nghttp2_session_client_new", rv);
   }
 
-  nghttp2_submit_settings(connection.session, NGHTTP2_FLAG_NONE, NULL, 0);
+  rv = nghttp2_submit_settings(connection.session, NGHTTP2_FLAG_NONE, NULL, 0);
+
+  if (rv != 0) {
+    diec("nghttp2_submit_settings", rv);
+  }
 
   /* Submit the HTTP request to the outbound queue. */
   submit_request(&connection, &req);
@@ -691,9 +695,6 @@ int main(int argc, char **argv) {
   act.sa_handler = SIG_IGN;
   sigaction(SIGPIPE, &act, 0);
 
-#ifndef OPENSSL_IS_BORINGSSL
-  OPENSSL_config(NULL);
-#endif /* OPENSSL_IS_BORINGSSL */
   SSL_load_error_strings();
   SSL_library_init();
 

examples/deflate.c

@@ -109,9 +109,9 @@ static void deflate(nghttp2_hd_deflater *deflater,
   printf("Input (%zu byte(s)):\n\n", sum);
 
   for (i = 0; i < nvlen; ++i) {
-    fwrite(nva[i].name, nva[i].namelen, 1, stdout);
+    fwrite(nva[i].name, 1, nva[i].namelen, stdout);
     printf(": ");
-    fwrite(nva[i].value, nva[i].valuelen, 1, stdout);
+    fwrite(nva[i].value, 1, nva[i].valuelen, stdout);
     printf("\n");
   }
 
@@ -186,9 +186,9 @@ int inflate_header_block(nghttp2_hd_inflater *inflater, uint8_t *in,
     inlen -= proclen;
 
     if (inflate_flags & NGHTTP2_HD_INFLATE_EMIT) {
-      fwrite(nv.name, nv.namelen, 1, stderr);
+      fwrite(nv.name, 1, nv.namelen, stderr);
       fprintf(stderr, ": ");
-      fwrite(nv.value, nv.valuelen, 1, stderr);
+      fwrite(nv.value, 1, nv.valuelen, stderr);
       fprintf(stderr, "\n");
     }
 

examples/libevent-client.c

@@ -179,9 +179,9 @@ static void delete_http2_session_data(http2_session_data *session_data) {
 
 static void print_header(FILE *f, const uint8_t *name, size_t namelen,
                          const uint8_t *value, size_t valuelen) {
-  fwrite(name, namelen, 1, f);
+  fwrite(name, 1, namelen, f);
   fprintf(f, ": ");
-  fwrite(value, valuelen, 1, f);
+  fwrite(value, 1, valuelen, f);
   fprintf(f, "\n");
 }
 
@@ -272,7 +272,7 @@ static int on_data_chunk_recv_callback(nghttp2_session *session _U_,
                                        void *user_data) {
   http2_session_data *session_data = (http2_session_data *)user_data;
   if (session_data->stream_data->stream_id == stream_id) {
-    fwrite(data, len, 1, stdout);
+    fwrite(data, 1, len, stdout);
   }
   return 0;
 }
@@ -322,6 +322,11 @@ static SSL_CTX *create_ssl_ctx(void) {
                           SSL_OP_NO_COMPRESSION |
                           SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION);
   SSL_CTX_set_next_proto_select_cb(ssl_ctx, select_next_proto_cb, NULL);
+
+#if OPENSSL_VERSION_NUMBER >= 0x10002000L
+  SSL_CTX_set_alpn_protos(ssl_ctx, (const unsigned char *)"\x02h2", 3);
+#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
   return ssl_ctx;
 }
 
@@ -475,7 +480,27 @@ static void eventcb(struct bufferevent *bev, short events, void *ptr) {
   if (events & BEV_EVENT_CONNECTED) {
     int fd = bufferevent_getfd(bev);
     int val = 1;
+    const unsigned char *alpn = NULL;
+    unsigned int alpnlen = 0;
+    SSL *ssl;
+
     fprintf(stderr, "Connected\n");
+
+    ssl = bufferevent_openssl_get_ssl(session_data->bev);
+
+    SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen);
+#if OPENSSL_VERSION_NUMBER >= 0x10002000L
+    if (alpn == NULL) {
+      SSL_get0_alpn_selected(ssl, &alpn, &alpnlen);
+    }
+#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
+    if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) {
+      fprintf(stderr, "h2 is not negotiated\n");
+      delete_http2_session_data(session_data);
+      return;
+    }
+
     setsockopt(fd, IPPROTO_TCP, TCP_NODELAY, (char *)&val, sizeof(val));
     initialize_nghttp2_session(session_data);
     send_client_connection_header(session_data);
@@ -569,9 +594,6 @@ int main(int argc, char **argv) {
   act.sa_handler = SIG_IGN;
   sigaction(SIGPIPE, &act, NULL);
 
-#ifndef OPENSSL_IS_BORINGSSL
-  OPENSSL_config(NULL);
-#endif /* OPENSSL_IS_BORINGSSL */
   SSL_load_error_strings();
   SSL_library_init();
 

examples/libevent-server.c

@@ -116,6 +116,22 @@ static int next_proto_cb(SSL *s _U_, const unsigned char **data,
   return SSL_TLSEXT_ERR_OK;
 }
 
+#if OPENSSL_VERSION_NUMBER >= 0x10002000L
+static int alpn_select_proto_cb(SSL *ssl _U_, const unsigned char **out,
+                                unsigned char *outlen, const unsigned char *in,
+                                unsigned int inlen, void *arg _U_) {
+  int rv;
+
+  rv = nghttp2_select_next_protocol((unsigned char **)out, outlen, in, inlen);
+
+  if (rv != 1) {
+    return SSL_TLSEXT_ERR_NOACK;
+  }
+
+  return SSL_TLSEXT_ERR_OK;
+}
+#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
 /* Create SSL_CTX. */
 static SSL_CTX *create_ssl_ctx(const char *key_file, const char *cert_file) {
   SSL_CTX *ssl_ctx;
@@ -152,6 +168,11 @@ static SSL_CTX *create_ssl_ctx(const char *key_file, const char *cert_file) {
   next_proto_list_len = 1 + NGHTTP2_PROTO_VERSION_ID_LEN;
 
   SSL_CTX_set_next_protos_advertised_cb(ssl_ctx, next_proto_cb, NULL);
+
+#if OPENSSL_VERSION_NUMBER >= 0x10002000L
+  SSL_CTX_set_alpn_select_cb(ssl_ctx, alpn_select_proto_cb, NULL);
+#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
   return ssl_ctx;
 }
 
@@ -640,11 +661,31 @@ static void writecb(struct bufferevent *bev, void *ptr) {
 static void eventcb(struct bufferevent *bev _U_, short events, void *ptr) {
   http2_session_data *session_data = (http2_session_data *)ptr;
   if (events & BEV_EVENT_CONNECTED) {
+    const unsigned char *alpn = NULL;
+    unsigned int alpnlen = 0;
+    SSL *ssl;
+
     fprintf(stderr, "%s connected\n", session_data->client_addr);
 
+    ssl = bufferevent_openssl_get_ssl(session_data->bev);
+
+    SSL_get0_next_proto_negotiated(ssl, &alpn, &alpnlen);
+#if OPENSSL_VERSION_NUMBER >= 0x10002000L
+    if (alpn == NULL) {
+      SSL_get0_alpn_selected(ssl, &alpn, &alpnlen);
+    }
+#endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
+
+    if (alpn == NULL || alpnlen != 2 || memcmp("h2", alpn, 2) != 0) {
+      fprintf(stderr, "%s h2 is not negotiated\n", session_data->client_addr);
+      delete_http2_session_data(session_data);
+      return;
+    }
+
     initialize_nghttp2_session(session_data);
 
-    if (send_server_connection_header(session_data) != 0) {
+    if (send_server_connection_header(session_data) != 0 ||
+        session_send(session_data) != 0) {
       delete_http2_session_data(session_data);
       return;
     }
@@ -740,9 +781,6 @@ int main(int argc, char **argv) {
   act.sa_handler = SIG_IGN;
   sigaction(SIGPIPE, &act, NULL);
 
-#ifndef OPENSSL_IS_BORINGSSL
-  OPENSSL_config(NULL);
-#endif /* OPENSSL_IS_BORINGSSL */
   SSL_load_error_strings();
   SSL_library_init();
 

gennghttpxfun.py

@@ -129,6 +129,11 @@ OPTIONS = [
     "backend-tls",
     "backend-connections-per-host",
     "error-page",
+    "no-kqueue",
+    "frontend-http2-settings-timeout",
+    "backend-http2-settings-timeout",
+    "api-max-request-body",
+    "backend-max-backoff",
 ]
 
 LOGVARS = [

help2rst.py

@@ -166,7 +166,8 @@ def format_text(text):
     else:
         text = re.sub(r'\*', r'\*', text)
     # markup option reference
-    text = re.sub(r'(^|\s)(-[a-zA-Z0-9-]+)', r'\1:option:`\2`', text)
+    text = re.sub(r'(^|\s)(-[a-zA-Z0-9]|--[a-zA-Z0-9-]+)',
+                  r'\1:option:`\2`', text)
     # sphinx does not like markup like ':option:`-f`='.  We need
     # backslash between ` and =.
     text = re.sub(r'(:option:`.*?`)(\S)', r'\1\\\2', text)

integration-tests/nghttpx_http1_test.go

@@ -3,6 +3,7 @@ package nghttp2
 import (
 	"bufio"
 	"bytes"
+	"encoding/json"
 	"fmt"
 	"golang.org/x/net/http2/hpack"
 	"golang.org/x/net/websocket"
@@ -793,3 +794,197 @@ func TestH1H2RespPhaseReturn(t *testing.T) {
 		t.Errorf("body = %v; want %v", got, want)
 	}
 }
+
+// TestH1APIBackendconfig exercise backendconfig API endpoint routine
+// for successful case.
+func TestH1APIBackendconfig(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name:   "TestH1APIBackendconfig",
+		path:   "/api/v1beta1/backendconfig",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH1APIBackendconfigQuery exercise backendconfig API endpoint
+// routine with query.
+func TestH1APIBackendconfigQuery(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name:   "TestH1APIBackendconfigQuery",
+		path:   "/api/v1beta1/backendconfig?foo=bar",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH1APIBackendconfigBadMethod exercise backendconfig API endpoint
+// routine with bad method.
+func TestH1APIBackendconfigBadMethod(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name:   "TestH1APIBackendconfigBadMethod",
+		path:   "/api/v1beta1/backendconfig",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.status, 405; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 405; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH1APINotFound exercise backendconfig API endpoint routine when
+// API endpoint is not found.
+func TestH1APINotFound(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name:   "TestH1APINotFound",
+		path:   "/api/notfound",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.status, 404; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 404; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH1Healthmon tests health monitor endpoint.
+func TestH1Healthmon(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3011;healthmon;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3011)
+	defer st.Close()
+
+	res, err := st.http1(requestParam{
+		name: "TestH1Healthmon",
+		path: "/alpha/bravo",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http1() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+}
+
+// TestH1ResponseBeforeRequestEnd tests the situation where response
+// ends before request body finishes.
+func TestH1ResponseBeforeRequestEnd(t *testing.T) {
+	st := newServerTester([]string{"--mruby-file=" + testDir + "/req-return.rb"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatal("request should not be forwarded")
+	})
+	defer st.Close()
+
+	if _, err := io.WriteString(st.conn, fmt.Sprintf(`POST / HTTP/1.1
+Host: %v
+Test-Case: TestH1ResponseBeforeRequestEnd
+Content-Length: 1000000
+
+`, st.authority)); err != nil {
+		t.Fatalf("Error io.WriteString() = %v", err)
+	}
+
+	resp, err := http.ReadResponse(bufio.NewReader(st.conn), nil)
+	if err != nil {
+		t.Fatalf("Error http.ReadResponse() = %v", err)
+	}
+
+	if got, want := resp.StatusCode, 404; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+}

integration-tests/nghttpx_http2_test.go

@@ -2,6 +2,7 @@ package nghttp2
 
 import (
 	"crypto/tls"
+	"encoding/json"
 	"fmt"
 	"golang.org/x/net/http2"
 	"golang.org/x/net/http2/hpack"
@@ -1843,3 +1844,190 @@ func TestH2H2RespPhaseReturn(t *testing.T) {
 		t.Errorf("body = %v; want %v", got, want)
 	}
 }
+
+// TestH2APIBackendconfig exercise backendconfig API endpoint routine
+// for successful case.
+func TestH2APIBackendconfig(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name:   "TestH2APIBackendconfig",
+		path:   "/api/v1beta1/backendconfig",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH2APIBackendconfigQuery exercise backendconfig API endpoint
+// routine with query.
+func TestH2APIBackendconfigQuery(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name:   "TestH2APIBackendconfigQuery",
+		path:   "/api/v1beta1/backendconfig?foo=bar",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH2APIBackendconfigBadMethod exercise backendconfig API endpoint
+// routine with bad method.
+func TestH2APIBackendconfigBadMethod(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name:   "TestH2APIBackendconfigBadMethod",
+		path:   "/api/v1beta1/backendconfig",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 405; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 405; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH2APINotFound exercise backendconfig API endpoint routine when
+// API endpoint is not found.
+func TestH2APINotFound(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3010;api;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name:   "TestH2APINotFound",
+		path:   "/api/notfound",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 404; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 404; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestH2Healthmon tests health monitor endpoint.
+func TestH2Healthmon(t *testing.T) {
+	st := newServerTesterConnectPort([]string{"-f127.0.0.1,3011;healthmon;no-tls"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3011)
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name: "TestH2Healthmon",
+		path: "/alpha/bravo",
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+}
+
+// TestH2ResponseBeforeRequestEnd tests the situation where response
+// ends before request body finishes.
+func TestH2ResponseBeforeRequestEnd(t *testing.T) {
+	st := newServerTester([]string{"--mruby-file=" + testDir + "/req-return.rb"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatal("request should not be forwarded")
+	})
+	defer st.Close()
+
+	res, err := st.http2(requestParam{
+		name:        "TestH2ResponseBeforeRequestEnd",
+		noEndStream: true,
+	})
+	if err != nil {
+		t.Fatalf("Error st.http2() = %v", err)
+	}
+	if got, want := res.status, 404; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+}

integration-tests/nghttpx_spdy_test.go

@@ -1,6 +1,7 @@
 package nghttp2
 
 import (
+	"encoding/json"
 	"github.com/tatsuhiro-t/spdy"
 	"golang.org/x/net/http2/hpack"
 	"net/http"
@@ -474,3 +475,190 @@ func TestS3H2RespPhaseReturn(t *testing.T) {
 		t.Errorf("body = %v; want %v", got, want)
 	}
 }
+
+// TestS3APIBackendconfig exercise backendconfig API endpoint routine
+// for successful case.
+func TestS3APIBackendconfig(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APIBackendconfig",
+		path:   "/api/v1beta1/backendconfig",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3APIBackendconfigQuery exercise backendconfig API endpoint
+// routine with query.
+func TestS3APIBackendconfigQuery(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APIBackendconfigQuery",
+		path:   "/api/v1beta1/backendconfig?foo=bar",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3APIBackendconfigBadMethod exercise backendconfig API endpoint
+// routine with bad method.
+func TestS3APIBackendconfigBadMethod(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APIBackendconfigBadMethod",
+		path:   "/api/v1beta1/backendconfig",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 405; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 405; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3APINotFound exercise backendconfig API endpoint routine when
+// API endpoint is not found.
+func TestS3APINotFound(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APINotFound",
+		path:   "/api/notfound",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 404; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 404; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3Healthmon tests health monitor endpoint.
+func TestS3Healthmon(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3011;healthmon"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3011)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3Healthmon",
+		path: "/alpha/bravo",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+}
+
+// TestS3ResponseBeforeRequestEnd tests the situation where response
+// ends before request body finishes.
+func TestS3ResponseBeforeRequestEnd(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--mruby-file=" + testDir + "/req-return.rb"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatal("request should not be forwarded")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:        "TestS3ResponseBeforeRequestEnd",
+		noEndStream: true,
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 404; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+}

integration-tests/server_tester.go

@@ -66,27 +66,36 @@ type serverTester struct {
 // newServerTester creates test context for plain TCP frontend
 // connection.
 func newServerTester(args []string, t *testing.T, handler http.HandlerFunc) *serverTester {
-	return newServerTesterInternal(args, t, handler, false, nil)
+	return newServerTesterInternal(args, t, handler, false, serverPort, nil)
+}
+
+func newServerTesterConnectPort(args []string, t *testing.T, handler http.HandlerFunc, port int) *serverTester {
+	return newServerTesterInternal(args, t, handler, false, port, nil)
 }
 
 func newServerTesterHandler(args []string, t *testing.T, handler http.Handler) *serverTester {
-	return newServerTesterInternal(args, t, handler, false, nil)
+	return newServerTesterInternal(args, t, handler, false, serverPort, nil)
 }
 
 // newServerTester creates test context for TLS frontend connection.
 func newServerTesterTLS(args []string, t *testing.T, handler http.HandlerFunc) *serverTester {
-	return newServerTesterInternal(args, t, handler, true, nil)
+	return newServerTesterInternal(args, t, handler, true, serverPort, nil)
+}
+
+func newServerTesterTLSConnectPort(args []string, t *testing.T, handler http.HandlerFunc, port int) *serverTester {
+	return newServerTesterInternal(args, t, handler, true, port, nil)
 }
 
 // newServerTester creates test context for TLS frontend connection
 // with given clientConfig
 func newServerTesterTLSConfig(args []string, t *testing.T, handler http.HandlerFunc, clientConfig *tls.Config) *serverTester {
-	return newServerTesterInternal(args, t, handler, true, clientConfig)
+	return newServerTesterInternal(args, t, handler, true, serverPort, clientConfig)
 }
 
 // newServerTesterInternal creates test context.  If frontendTLS is
-// true, set up TLS frontend connection.
-func newServerTesterInternal(src_args []string, t *testing.T, handler http.Handler, frontendTLS bool, clientConfig *tls.Config) *serverTester {
+// true, set up TLS frontend connection.  connectPort is the server
+// side port where client connection is made.
+func newServerTesterInternal(src_args []string, t *testing.T, handler http.Handler, frontendTLS bool, connectPort int, clientConfig *tls.Config) *serverTester {
 	ts := httptest.NewUnstartedServer(handler)
 
 	args := []string{}
@@ -138,7 +147,7 @@ func newServerTesterInternal(src_args []string, t *testing.T, handler http.Handl
 	args = append(args, fmt.Sprintf("-f127.0.0.1,%v;%v", serverPort, noTLS), b,
 		"--errorlog-file="+logDir+"/log.txt", "-LINFO")
 
-	authority := fmt.Sprintf("127.0.0.1:%v", serverPort)
+	authority := fmt.Sprintf("127.0.0.1:%v", connectPort)
 
 	st := &serverTester{
 		cmd:          exec.Command(serverBin, args...),
@@ -160,6 +169,8 @@ func newServerTesterInternal(src_args []string, t *testing.T, handler http.Handl
 
 	retry := 0
 	for {
+		time.Sleep(50 * time.Millisecond)
+
 		var conn net.Conn
 		var err error
 		if frontendTLS {
@@ -170,7 +181,7 @@ func newServerTesterInternal(src_args []string, t *testing.T, handler http.Handl
 				tlsConfig = clientConfig
 			}
 			tlsConfig.InsecureSkipVerify = true
-			tlsConfig.NextProtos = []string{"h2-14", "spdy/3.1"}
+			tlsConfig.NextProtos = []string{"h2", "spdy/3.1"}
 			conn, err = tls.Dial("tcp", authority, tlsConfig)
 		} else {
 			conn, err = net.Dial("tcp", authority)
@@ -181,7 +192,6 @@ func newServerTesterInternal(src_args []string, t *testing.T, handler http.Handl
 				st.Close()
 				st.t.Fatalf("Error server is not responding too long; server command-line arguments may be invalid")
 			}
-			time.Sleep(150 * time.Millisecond)
 			continue
 		}
 		if frontendTLS {
@@ -287,6 +297,7 @@ type requestParam struct {
 	body        []byte              // request body
 	trailer     []hpack.HeaderField // trailer part
 	httpUpgrade bool                // true if upgraded to HTTP/2 through HTTP Upgrade
+	noEndStream bool                // true if END_STREAM should not be sent
 }
 
 // wrapper for request body to set trailer part
@@ -370,8 +381,9 @@ func (st *serverTester) http1(rp requestParam) (*serverResponse, error) {
 		if err != nil {
 			st.t.Fatalf("Error parsing URL from st.url %v: %v", st.url, err)
 		}
-		u.Path = rp.path
-		reqURL = u.String()
+		u.Path = ""
+		u.RawQuery = ""
+		reqURL = u.String() + rp.path
 	}
 
 	req, err := http.NewRequest(method, reqURL, body)
@@ -460,7 +472,7 @@ func (st *serverTester) spdy(rp requestParam) (*serverResponse, error) {
 	}
 
 	var synStreamFlags spdy.ControlFlags
-	if len(rp.body) == 0 {
+	if len(rp.body) == 0 && !rp.noEndStream {
 		synStreamFlags = spdy.ControlFlagFin
 	}
 	if err := st.spdyFr.WriteFrame(&spdy.SynStreamFrame{
@@ -474,9 +486,13 @@ func (st *serverTester) spdy(rp requestParam) (*serverResponse, error) {
 	}
 
 	if len(rp.body) != 0 {
+		var dataFlags spdy.DataFlags
+		if !rp.noEndStream {
+			dataFlags = spdy.DataFlagFin
+		}
 		if err := st.spdyFr.WriteFrame(&spdy.DataFrame{
 			StreamId: id,
-			Flags:    spdy.DataFlagFin,
+			Flags:    dataFlags,
 			Data:     rp.body,
 		}); err != nil {
 			return nil, err
@@ -589,7 +605,7 @@ func (st *serverTester) http2(rp requestParam) (*serverResponse, error) {
 
 		err := st.fr.WriteHeaders(http2.HeadersFrameParam{
 			StreamID:      id,
-			EndStream:     len(rp.body) == 0 && len(rp.trailer) == 0,
+			EndStream:     len(rp.body) == 0 && len(rp.trailer) == 0 && !rp.noEndStream,
 			EndHeaders:    true,
 			BlockFragment: st.headerBlkBuf.Bytes(),
 		})
@@ -599,7 +615,7 @@ func (st *serverTester) http2(rp requestParam) (*serverResponse, error) {
 
 		if len(rp.body) != 0 {
 			// TODO we assume rp.body fits in 1 frame
-			if err := st.fr.WriteData(id, len(rp.trailer) == 0, rp.body); err != nil {
+			if err := st.fr.WriteData(id, len(rp.trailer) == 0 && !rp.noEndStream, rp.body); err != nil {
 				return nil, err
 			}
 		}
@@ -746,3 +762,8 @@ func cloneHeader(h http.Header) http.Header {
 }
 
 func noopHandler(w http.ResponseWriter, r *http.Request) {}
+
+type APIResponse struct {
+	Status string `json:"status,omitempty"`
+	Code   int    `json:"code,omitempty"`
+}

integration-tests/setenv.in

@@ -8,4 +8,5 @@ fi
 export CGO_CFLAGS="-I@abs_top_srcdir@/lib/includes -I@abs_top_builddir@/lib/includes"
 export CGO_LDFLAGS="-L$libdir"
 export LD_LIBRARY_PATH="$libdir"
+export GODEBUG=cgocheck=0
 "$@"

lib/includes/nghttp2/nghttp2.h

@@ -81,7 +81,7 @@ extern "C" {
 /**
  * @macro
  *
- * The seriazlied form of ALPN protocol identifier this library
+ * The serialized form of ALPN protocol identifier this library
  * supports.  Notice that first byte is the length of following
  * protocol identifier.  This is the same wire format of `TLS ALPN
  * extension <https://tools.ietf.org/html/rfc7301>`_.  This is useful
@@ -422,7 +422,7 @@ typedef enum {
 /**
  * @struct
  *
- * The object representing single contagious buffer.
+ * The object representing single contiguous buffer.
  */
 typedef struct {
   /**
@@ -591,7 +591,12 @@ typedef enum {
    * callbacks because the library processes this frame type and its
    * preceding HEADERS/PUSH_PROMISE as a single frame.
    */
-  NGHTTP2_CONTINUATION = 0x09
+  NGHTTP2_CONTINUATION = 0x09,
+  /**
+   * The ALTSVC frame, which is defined in `RFC 7383
+   * <https://tools.ietf.org/html/rfc7838#section-4>`_.
+   */
+  NGHTTP2_ALTSVC = 0x0a
 } nghttp2_frame_type;
 
 /**
@@ -1449,9 +1454,19 @@ typedef int (*nghttp2_on_data_chunk_recv_callback)(nghttp2_session *session,
  * `nghttp2_session_server_new()`.
  *
  * The implementation of this function must return 0 if it succeeds.
- * If nonzero is returned, it is treated as fatal error and
- * `nghttp2_session_send()` and `nghttp2_session_mem_send()` functions
- * immediately return :enum:`NGHTTP2_ERR_CALLBACK_FAILURE`.
+ * It can also return :enum:`NGHTTP2_ERR_CANCEL` to cancel the
+ * transmission of the given frame.
+ *
+ * If there is a fatal error while executing this callback, the
+ * implementation should return :enum:`NGHTTP2_ERR_CALLBACK_FAILURE`,
+ * which makes `nghttp2_session_send()` and
+ * `nghttp2_session_mem_send()` functions immediately return
+ * :enum:`NGHTTP2_ERR_CALLBACK_FAILURE`.
+ *
+ * If the other value is returned, it is treated as if
+ * :enum:`NGHTTP2_ERR_CALLBACK_FAILURE` is returned.  But the
+ * implementation should not rely on this since the library may define
+ * new return value to extend its capability.
  *
  * To set this callback to :type:`nghttp2_session_callbacks`, use
  * `nghttp2_session_callbacks_set_before_frame_send_callback()`.
@@ -2373,6 +2388,26 @@ NGHTTP2_EXTERN void
 nghttp2_option_set_user_recv_extension_type(nghttp2_option *option,
                                             uint8_t type);
 
+/**
+ * @function
+ *
+ * Sets extension frame type the application is willing to receive
+ * using builtin handler.  The |type| is the extension frame type to
+ * receive, and must be strictly greater than 0x9.  Otherwise, this
+ * function does nothing.  The application can call this function
+ * multiple times to set more than one frame type to receive.  The
+ * application does not have to call this function if it just sends
+ * extension frames.
+ *
+ * If same frame type is passed to both
+ * `nghttp2_option_set_builtin_recv_extension_type()` and
+ * `nghttp2_option_set_user_recv_extension_type()`, the latter takes
+ * precedence.
+ */
+NGHTTP2_EXTERN void
+nghttp2_option_set_builtin_recv_extension_type(nghttp2_option *option,
+                                               uint8_t type);
+
 /**
  * @function
  *
@@ -2387,6 +2422,21 @@ nghttp2_option_set_user_recv_extension_type(nghttp2_option *option,
 NGHTTP2_EXTERN void nghttp2_option_set_no_auto_ping_ack(nghttp2_option *option,
                                                         int val);
 
+/**
+ * @function
+ *
+ * This option sets the maximum length of header block (a set of
+ * header fields per one HEADERS frame) to send.  The length of a
+ * given set of header fields is calculated using
+ * `nghttp2_hd_deflate_bound()`.  The default value is 64KiB.  If
+ * application attempts to send header fields larger than this limit,
+ * the transmission of the frame fails with error code
+ * :enum:`NGHTTP2_ERR_FRAME_SIZE_ERROR`.
+ */
+NGHTTP2_EXTERN void
+nghttp2_option_set_max_send_header_block_length(nghttp2_option *option,
+                                                size_t val);
+
 /**
  * @function
  *
@@ -2579,14 +2629,20 @@ NGHTTP2_EXTERN void nghttp2_session_del(nghttp2_session *session);
  *
  * 6. :type:`nghttp2_before_frame_send_callback` is invoked.
  *
- * 7. :type:`nghttp2_send_callback` is invoked one or more times to
+ * 7. If :enum:`NGHTTP2_ERR_CANCEL` is returned from
+ *    :type:`nghttp2_before_frame_send_callback`, the current frame
+ *    transmission is canceled, and
+ *    :type:`nghttp2_on_frame_not_send_callback` is invoked.  Abort
+ *    the following steps.
+ *
+ * 8. :type:`nghttp2_send_callback` is invoked one or more times to
  *    send the frame.
  *
- * 8. :type:`nghttp2_on_frame_send_callback` is invoked.
+ * 9. :type:`nghttp2_on_frame_send_callback` is invoked.
  *
- * 9. If the transmission of the frame triggers closure of the stream,
- *    the stream is closed and
- *    :type:`nghttp2_on_stream_close_callback` is invoked.
+ * 10. If the transmission of the frame triggers closure of the
+ *     stream, the stream is closed and
+ *     :type:`nghttp2_on_stream_close_callback` is invoked.
  *
  * This function returns 0 if it succeeds, or one of the following
  * negative error codes:
@@ -3558,8 +3614,8 @@ nghttp2_submit_response(nghttp2_session *session, int32_t stream_id,
  *
  * The |nva| is an array of name/value pair :type:`nghttp2_nv` with
  * |nvlen| elements.  The application is responsible not to include
- * required pseudo-header fields (header field whose name starts with
- * ":") in |nva|.
+ * pseudo-header fields (header field whose name starts with ":") in
+ * |nva|.
  *
  * This function creates copies of all name/value pairs in |nva|.  It
  * also lower-cases all names in |nva|.  The order of elements in
@@ -3574,20 +3630,20 @@ nghttp2_submit_response(nghttp2_session *session, int32_t stream_id,
  * :type:`nghttp2_on_frame_not_send_callback` is called.
  *
  * For server, trailer fields must follow response HEADERS or response
- * DATA with END_STREAM flag set.  The library does not enforce this
- * requirement, and applications should do this for themselves.  If
- * `nghttp2_submit_trailer()` is called before any response HEADERS
+ * DATA without END_STREAM flat set.  The library does not enforce
+ * this requirement, and applications should do this for themselves.
+ * If `nghttp2_submit_trailer()` is called before any response HEADERS
  * submission (usually by `nghttp2_submit_response()`), the content of
  * |nva| will be sent as response headers, which will result in error.
  *
  * This function has the same effect with `nghttp2_submit_headers()`,
- * with flags = :enum:`NGHTTP2_FLAG_END_HEADERS` and both pri_spec and
+ * with flags = :enum:`NGHTTP2_FLAG_END_STREAM` and both pri_spec and
  * stream_user_data to NULL.
  *
  * To submit trailer fields after `nghttp2_submit_response()` is
  * called, the application has to specify
- * :type:`nghttp2_data_provider` to `nghttp2_submit_response()`.  In
- * side :type:`nghttp2_data_source_read_callback`, when setting
+ * :type:`nghttp2_data_provider` to `nghttp2_submit_response()`.
+ * Inside of :type:`nghttp2_data_source_read_callback`, when setting
  * :enum:`NGHTTP2_DATA_FLAG_EOF`, also set
  * :enum:`NGHTTP2_DATA_FLAG_NO_END_STREAM`.  After that, the
  * application can send trailer fields using
@@ -4046,14 +4102,17 @@ nghttp2_session_check_server_session(nghttp2_session *session);
  * that value as window_size_increment is queued.  If the
  * |window_size_increment| is larger than the received bytes from the
  * remote endpoint, the local window size is increased by that
- * difference.
+ * difference.  If the sole purpose is to increase the local window
+ * size, consider to use `nghttp2_session_set_local_window_size()`.
  *
  * If the |window_size_increment| is negative, the local window size
  * is decreased by -|window_size_increment|.  If automatic
  * WINDOW_UPDATE is enabled
  * (`nghttp2_option_set_no_auto_window_update()`), and the library
  * decided that the WINDOW_UPDATE should be submitted, then
- * WINDOW_UPDATE is queued with the current received bytes count.
+ * WINDOW_UPDATE is queued with the current received bytes count.  If
+ * the sole purpose is to decrease the local window size, consider to
+ * use `nghttp2_session_set_local_window_size()`.
  *
  * If the |window_size_increment| is 0, the function does nothing and
  * returns 0.
@@ -4071,6 +4130,44 @@ NGHTTP2_EXTERN int nghttp2_submit_window_update(nghttp2_session *session,
                                                 int32_t stream_id,
                                                 int32_t window_size_increment);
 
+/**
+ * @function
+ *
+ * Set local window size (local endpoints's window size) to the given
+ * |window_size| for the given stream denoted by |stream_id|.  To
+ * change connection level window size, specify 0 to |stream_id|.  To
+ * increase window size, this function may submit WINDOW_UPDATE frame
+ * to transmission queue.
+ *
+ * The |flags| is currently ignored and should be
+ * :enum:`NGHTTP2_FLAG_NONE`.
+ *
+ * This sounds similar to `nghttp2_submit_window_update()`, but there
+ * are 2 differences.  The first difference is that this function
+ * takes the absolute value of window size to set, rather than the
+ * delta.  To change the window size, this may be easier to use since
+ * the application just declares the intended window size, rather than
+ * calculating delta.  The second difference is that
+ * `nghttp2_submit_window_update()` affects the received bytes count
+ * which has not acked yet.  By the specification of
+ * `nghttp2_submit_window_update()`, to strictly increase the local
+ * window size, we have to submit delta including all received bytes
+ * count, which might not be desirable in some cases.  On the other
+ * hand, this function does not affect the received bytes count.  It
+ * just sets the local window size to the given value.
+ *
+ * This function returns 0 if it succeeds, or one of the following
+ * negative error codes:
+ *
+ * :enum:`NGHTTP2_ERR_INVALID_ARGUMENT`
+ *     The |stream_id| is negative.
+ * :enum:`NGHTTP2_ERR_NOMEM`
+ *     Out of memory.
+ */
+NGHTTP2_EXTERN int
+nghttp2_session_set_local_window_size(nghttp2_session *session, uint8_t flags,
+                                      int32_t stream_id, int32_t window_size);
+
 /**
  * @function
  *
@@ -4113,6 +4210,80 @@ NGHTTP2_EXTERN int nghttp2_submit_extension(nghttp2_session *session,
                                             uint8_t type, uint8_t flags,
                                             int32_t stream_id, void *payload);
 
+/**
+ * @struct
+ *
+ * The payload of ALTSVC frame.  ALTSVC frame is a non-critical
+ * extension to HTTP/2.  If this frame is received, and
+ * `nghttp2_option_set_user_recv_extension_type()` is not set, and
+ * `nghttp2_option_set_builtin_recv_extension_type()` is set for
+ * :enum:`NGHTTP2_ALTSVC`, ``nghttp2_extension.payload`` will point to
+ * this struct.
+ *
+ * It has the following members:
+ */
+typedef struct {
+  /**
+   * The pointer to origin which this alternative service is
+   * associated with.  This is not necessarily NULL-terminated.
+   */
+  uint8_t *origin;
+  /**
+   * The length of the |origin|.
+   */
+  size_t origin_len;
+  /**
+   * The pointer to Alt-Svc field value contained in ALTSVC frame.
+   * This is not necessarily NULL-terminated.
+   */
+  uint8_t *field_value;
+  /**
+   * The length of the |field_value|.
+   */
+  size_t field_value_len;
+} nghttp2_ext_altsvc;
+
+/**
+ * @function
+ *
+ * Submits ALTSVC frame.
+ *
+ * ALTSVC frame is a non-critical extension to HTTP/2, and defined in
+ * is defined in `RFC 7383
+ * <https://tools.ietf.org/html/rfc7838#section-4>`_.
+ *
+ * The |flags| is currently ignored and should be
+ * :enum:`NGHTTP2_FLAG_NONE`.
+ *
+ * The |origin| points to the origin this alternative service is
+ * associated with.  The |origin_len| is the length of the origin.  If
+ * |stream_id| is 0, the origin must be specified.  If |stream_id| is
+ * not zero, the origin must be empty (in other words, |origin_len|
+ * must be 0).
+ *
+ * The ALTSVC frame is only usable from server side.  If this function
+ * is invoked with client side session, this function returns
+ * :enum:`NGHTTP2_ERR_INVALID_STATE`.
+ *
+ * This function returns 0 if it succeeds, or one of the following
+ * negative error codes:
+ *
+ * :enum:`NGHTTP2_ERR_NOMEM`
+ *     Out of memory
+ * :enum:`NGHTTP2_ERR_INVALID_STATE`
+ *     The function is called from client side session
+ * :enum:`NGHTTP2_ERR_INVALID_ARGUMENT`
+ *     The sum of |origin_len| and |field_value_len| is larger than
+ *     16382; or |origin_len| is 0 while |stream_id| is 0; or
+ *     |origin_len| is not 0 while |stream_id| is not 0.
+ */
+NGHTTP2_EXTERN int nghttp2_submit_altsvc(nghttp2_session *session,
+                                         uint8_t flags, int32_t stream_id,
+                                         const uint8_t *origin,
+                                         size_t origin_len,
+                                         const uint8_t *field_value,
+                                         size_t field_value_len);
+
 /**
  * @function
  *
@@ -4454,7 +4625,7 @@ NGHTTP2_EXTERN void nghttp2_hd_inflate_del(nghttp2_hd_inflater *inflater);
  * This function must not be called while header block is being
  * inflated.  In other words, this function must be called after
  * initialization of |inflater|, but before calling
- * `nghttp2_hd_inflate_hd()`, or after
+ * `nghttp2_hd_inflate_hd2()`, or after
  * `nghttp2_hd_inflate_end_headers()`.  Otherwise,
  * `NGHTTP2_ERR_INVALID_STATE` was returned.
  *
@@ -4495,6 +4666,10 @@ typedef enum {
 /**
  * @function
  *
+ * .. warning::
+ *
+ *   Deprecated.  Use `nghttp2_hd_inflate_hd2()` instead.
+ *
  * Inflates name/value block stored in |in| with length |inlen|.  This
  * function performs decompression.  For each successful emission of
  * header name/value pair, :enum:`NGHTTP2_HD_INFLATE_EMIT` is set in
@@ -4574,6 +4749,88 @@ NGHTTP2_EXTERN ssize_t nghttp2_hd_inflate_hd(nghttp2_hd_inflater *inflater,
                                              int *inflate_flags, uint8_t *in,
                                              size_t inlen, int in_final);
 
+/**
+ * @function
+ *
+ * Inflates name/value block stored in |in| with length |inlen|.  This
+ * function performs decompression.  For each successful emission of
+ * header name/value pair, :enum:`NGHTTP2_HD_INFLATE_EMIT` is set in
+ * |*inflate_flags| and name/value pair is assigned to the |nv_out|
+ * and the function returns.  The caller must not free the members of
+ * |nv_out|.
+ *
+ * The |nv_out| may include pointers to the memory region in the |in|.
+ * The caller must retain the |in| while the |nv_out| is used.
+ *
+ * The application should call this function repeatedly until the
+ * ``(*inflate_flags) & NGHTTP2_HD_INFLATE_FINAL`` is nonzero and
+ * return value is non-negative.  This means the all input values are
+ * processed successfully.  Then the application must call
+ * `nghttp2_hd_inflate_end_headers()` to prepare for the next header
+ * block input.
+ *
+ * The caller can feed complete compressed header block.  It also can
+ * feed it in several chunks.  The caller must set |in_final| to
+ * nonzero if the given input is the last block of the compressed
+ * header.
+ *
+ * This function returns the number of bytes processed if it succeeds,
+ * or one of the following negative error codes:
+ *
+ * :enum:`NGHTTP2_ERR_NOMEM`
+ *     Out of memory.
+ * :enum:`NGHTTP2_ERR_HEADER_COMP`
+ *     Inflation process has failed.
+ * :enum:`NGHTTP2_ERR_BUFFER_ERROR`
+ *     The header field name or value is too large.
+ *
+ * Example follows::
+ *
+ *     int inflate_header_block(nghttp2_hd_inflater *hd_inflater,
+ *                              uint8_t *in, size_t inlen, int final)
+ *     {
+ *         ssize_t rv;
+ *
+ *         for(;;) {
+ *             nghttp2_nv nv;
+ *             int inflate_flags = 0;
+ *
+ *             rv = nghttp2_hd_inflate_hd2(hd_inflater, &nv, &inflate_flags,
+ *                                         in, inlen, final);
+ *
+ *             if(rv < 0) {
+ *                 fprintf(stderr, "inflate failed with error code %zd", rv);
+ *                 return -1;
+ *             }
+ *
+ *             in += rv;
+ *             inlen -= rv;
+ *
+ *             if(inflate_flags & NGHTTP2_HD_INFLATE_EMIT) {
+ *                 fwrite(nv.name, nv.namelen, 1, stderr);
+ *                 fprintf(stderr, ": ");
+ *                 fwrite(nv.value, nv.valuelen, 1, stderr);
+ *                 fprintf(stderr, "\n");
+ *             }
+ *             if(inflate_flags & NGHTTP2_HD_INFLATE_FINAL) {
+ *                 nghttp2_hd_inflate_end_headers(hd_inflater);
+ *                 break;
+ *             }
+ *             if((inflate_flags & NGHTTP2_HD_INFLATE_EMIT) == 0 &&
+ *                inlen == 0) {
+ *                break;
+ *             }
+ *         }
+ *
+ *         return 0;
+ *     }
+ *
+ */
+NGHTTP2_EXTERN ssize_t
+nghttp2_hd_inflate_hd2(nghttp2_hd_inflater *inflater, nghttp2_nv *nv_out,
+                       int *inflate_flags, const uint8_t *in, size_t inlen,
+                       int in_final);
+
 /**
  * @function
  *

lib/nghttp2_buf.h

@@ -312,8 +312,8 @@ int nghttp2_bufs_orb_hold(nghttp2_bufs *bufs, uint8_t b);
   } while (0)
 
 /*
- * Copies all data stored in |bufs| to the contagious buffer.  This
- * function allocates the contagious memory to store all data in
+ * Copies all data stored in |bufs| to the contiguous buffer.  This
+ * function allocates the contiguous memory to store all data in
  * |bufs| and assigns it to |*out|.
  *
  * The contents of |bufs| is left unchanged.

lib/nghttp2_frame.c

@@ -193,6 +193,30 @@ void nghttp2_frame_extension_init(nghttp2_extension *frame, uint8_t type,
 
 void nghttp2_frame_extension_free(nghttp2_extension *frame _U_) {}
 
+void nghttp2_frame_altsvc_init(nghttp2_extension *frame, int32_t stream_id,
+                               uint8_t *origin, size_t origin_len,
+                               uint8_t *field_value, size_t field_value_len) {
+  nghttp2_ext_altsvc *altsvc;
+
+  nghttp2_frame_hd_init(&frame->hd, 2 + origin_len + field_value_len,
+                        NGHTTP2_ALTSVC, NGHTTP2_FLAG_NONE, stream_id);
+
+  altsvc = frame->payload;
+  altsvc->origin = origin;
+  altsvc->origin_len = origin_len;
+  altsvc->field_value = field_value;
+  altsvc->field_value_len = field_value_len;
+}
+
+void nghttp2_frame_altsvc_free(nghttp2_extension *frame, nghttp2_mem *mem) {
+  nghttp2_ext_altsvc *altsvc;
+
+  altsvc = frame->payload;
+  /* We use the same buffer for altsvc->origin and
+     altsvc->field_value. */
+  nghttp2_mem_free(mem, altsvc->origin);
+}
+
 size_t nghttp2_frame_priority_len(uint8_t flags) {
   if (flags & NGHTTP2_FLAG_PRIORITY) {
     return NGHTTP2_PRIORITY_SPECLEN;
@@ -439,25 +463,11 @@ size_t nghttp2_frame_pack_settings_payload(uint8_t *buf,
   return NGHTTP2_FRAME_SETTINGS_ENTRY_LENGTH * niv;
 }
 
-int nghttp2_frame_unpack_settings_payload(nghttp2_settings *frame,
-                                          nghttp2_settings_entry *iv,
-                                          size_t niv, nghttp2_mem *mem) {
-  size_t payloadlen = niv * sizeof(nghttp2_settings_entry);
-
-  if (niv == 0) {
-    frame->iv = NULL;
-  } else {
-    frame->iv = nghttp2_mem_malloc(mem, payloadlen);
-
-    if (frame->iv == NULL) {
-      return NGHTTP2_ERR_NOMEM;
-    }
-
-    memcpy(frame->iv, iv, payloadlen);
-  }
-
+void nghttp2_frame_unpack_settings_payload(nghttp2_settings *frame,
+                                           nghttp2_settings_entry *iv,
+                                           size_t niv) {
+  frame->iv = iv;
   frame->niv = niv;
-  return 0;
 }
 
 void nghttp2_frame_unpack_settings_entry(nghttp2_settings_entry *iv,
@@ -668,6 +678,79 @@ void nghttp2_frame_unpack_window_update_payload(nghttp2_window_update *frame,
       nghttp2_get_uint32(payload) & NGHTTP2_WINDOW_SIZE_INCREMENT_MASK;
 }
 
+int nghttp2_frame_pack_altsvc(nghttp2_bufs *bufs, nghttp2_extension *frame) {
+  int rv;
+  nghttp2_buf *buf;
+  nghttp2_ext_altsvc *altsvc;
+
+  altsvc = frame->payload;
+
+  buf = &bufs->head->buf;
+
+  assert(nghttp2_buf_avail(buf) >=
+         2 + altsvc->origin_len + altsvc->field_value_len);
+
+  buf->pos -= NGHTTP2_FRAME_HDLEN;
+
+  nghttp2_frame_pack_frame_hd(buf->pos, &frame->hd);
+
+  nghttp2_put_uint16be(buf->last, (uint16_t)altsvc->origin_len);
+  buf->last += 2;
+
+  rv = nghttp2_bufs_add(bufs, altsvc->origin, altsvc->origin_len);
+
+  assert(rv == 0);
+
+  rv = nghttp2_bufs_add(bufs, altsvc->field_value, altsvc->field_value_len);
+
+  assert(rv == 0);
+
+  return 0;
+}
+
+void nghttp2_frame_unpack_altsvc_payload(nghttp2_extension *frame,
+                                         size_t origin_len, uint8_t *payload,
+                                         size_t payloadlen) {
+  nghttp2_ext_altsvc *altsvc;
+  uint8_t *p;
+
+  altsvc = frame->payload;
+  p = payload;
+
+  altsvc->origin = p;
+
+  p += origin_len;
+
+  altsvc->origin_len = origin_len;
+
+  altsvc->field_value = p;
+  altsvc->field_value_len = (size_t)(payload + payloadlen - p);
+}
+
+int nghttp2_frame_unpack_altsvc_payload2(nghttp2_extension *frame,
+                                         const uint8_t *payload,
+                                         size_t payloadlen, nghttp2_mem *mem) {
+  uint8_t *buf;
+  size_t origin_len;
+
+  if (payloadlen < 2) {
+    return NGHTTP2_FRAME_SIZE_ERROR;
+  }
+
+  origin_len = nghttp2_get_uint16(payload);
+
+  buf = nghttp2_mem_malloc(mem, payloadlen - 2);
+  if (!buf) {
+    return NGHTTP2_ERR_NOMEM;
+  }
+
+  nghttp2_cpymem(buf, payload + 2, payloadlen - 2);
+
+  nghttp2_frame_unpack_altsvc_payload(frame, origin_len, buf, payloadlen - 2);
+
+  return 0;
+}
+
 nghttp2_settings_entry *nghttp2_frame_iv_copy(const nghttp2_settings_entry *iv,
                                               size_t niv, nghttp2_mem *mem) {
   nghttp2_settings_entry *iv_copy;

lib/nghttp2_frame.h

@@ -52,14 +52,12 @@
 #define NGHTTP2_FRAMEBUF_CHUNKLEN                                              \
   (NGHTTP2_FRAME_HDLEN + 1 + NGHTTP2_MAX_PAYLOADLEN)
 
-/* Number of inbound buffer */
-#define NGHTTP2_FRAMEBUF_MAX_NUM 5
-
 /* The default length of DATA frame payload. */
 #define NGHTTP2_DATA_PAYLOADLEN NGHTTP2_MAX_FRAME_SIZE_MIN
 
-/* Maximum headers payload length, calculated in compressed form.
-   This applies to transmission only. */
+/* Maximum headers block size to send, calculated using
+   nghttp2_hd_deflate_bound().  This is the default value, and can be
+   overridden by nghttp2_option_set_max_send_header_block_size(). */
 #define NGHTTP2_MAX_HEADERSLEN 65536
 
 /* The number of bytes for each SETTINGS entry */
@@ -72,7 +70,7 @@
 #define NGHTTP2_MAX_PADLEN 256
 
 /* Union of extension frame payload */
-typedef union { int dummy; } nghttp2_ext_frame_payload;
+typedef union { nghttp2_ext_altsvc altsvc; } nghttp2_ext_frame_payload;
 
 void nghttp2_frame_pack_frame_hd(uint8_t *buf, const nghttp2_frame_hd *hd);
 
@@ -215,18 +213,12 @@ void nghttp2_frame_unpack_settings_entry(nghttp2_settings_entry *iv,
                                          const uint8_t *payload);
 
 /*
- * Makes a copy of |iv| in frame->settings.iv. The |niv| is assigned
- * to frame->settings.niv.
- *
- * This function returns 0 if it succeeds or one of the following
- * negative error codes:
- *
- * NGHTTP2_ERR_NOMEM
- *     Out of memory.
+ * Initializes payload of frame->settings.  The |frame| takes
+ * ownership of |iv|.
  */
-int nghttp2_frame_unpack_settings_payload(nghttp2_settings *frame,
-                                          nghttp2_settings_entry *iv,
-                                          size_t niv, nghttp2_mem *mem);
+void nghttp2_frame_unpack_settings_payload(nghttp2_settings *frame,
+                                           nghttp2_settings_entry *iv,
+                                           size_t niv);
 
 /*
  * Unpacks SETTINGS payload into |*iv_ptr|. The number of entries are
@@ -367,6 +359,45 @@ void nghttp2_frame_unpack_window_update_payload(nghttp2_window_update *frame,
                                                 const uint8_t *payload,
                                                 size_t payloadlen);
 
+/*
+ * Packs ALTSVC frame |frame| in wire frame format and store it in
+ * |bufs|.
+ *
+ * The caller must make sure that nghttp2_bufs_reset(bufs) is called
+ * before calling this function.
+ *
+ * This function always succeeds and returns 0.
+ */
+int nghttp2_frame_pack_altsvc(nghttp2_bufs *bufs, nghttp2_extension *ext);
+
+/*
+ * Unpacks ALTSVC wire format into |frame|.  The |payload| of
+ * |payloadlen| bytes contains frame payload.  This function assumes
+ * that frame->payload points to the nghttp2_ext_altsvc object.
+ *
+ * This function always succeeds and returns 0.
+ */
+void nghttp2_frame_unpack_altsvc_payload(nghttp2_extension *frame,
+                                         size_t origin_len, uint8_t *payload,
+                                         size_t payloadlen);
+
+/*
+ * Unpacks ALTSVC wire format into |frame|.  This function only exists
+ * for unit test.  After allocating buffer for fields, this function
+ * internally calls nghttp2_frame_unpack_altsvc_payload().
+ *
+ * This function returns 0 if it succeeds, or one of the following
+ * negative error codes:
+ *
+ * NGHTTP2_ERR_NOMEM
+ *     Out of memory.
+ * NGHTTP2_ERR_FRAME_SIZE_ERROR
+ *     The payload is too small.
+ */
+int nghttp2_frame_unpack_altsvc_payload2(nghttp2_extension *frame,
+                                         const uint8_t *payload,
+                                         size_t payloadlen, nghttp2_mem *mem);
+
 /*
  * Initializes HEADERS frame |frame| with given values.  |frame| takes
  * ownership of |nva|, so caller must not free it. If |stream_id| is
@@ -445,6 +476,25 @@ void nghttp2_frame_extension_init(nghttp2_extension *frame, uint8_t type,
 
 void nghttp2_frame_extension_free(nghttp2_extension *frame);
 
+/*
+ * Initializes ALTSVC frame |frame| with given values.  This function
+ * assumes that frame->payload points to nghttp2_ext_altsvc object.
+ * Also |origin| and |field_value| are allocated in single buffer,
+ * starting |origin|.  On success, this function takes ownership of
+ * |origin|, so caller must not free it.
+ */
+void nghttp2_frame_altsvc_init(nghttp2_extension *frame, int32_t stream_id,
+                               uint8_t *origin, size_t origin_len,
+                               uint8_t *field_value, size_t field_value_len);
+
+/*
+ * Frees up resources under |frame|.  This function does not free
+ * nghttp2_ext_altsvc object pointed by frame->payload.  This function
+ * only frees origin pointed by nghttp2_ext_altsvc.origin.  Therefore,
+ * other fields must be allocated in the same buffer with origin.
+ */
+void nghttp2_frame_altsvc_free(nghttp2_extension *frame, nghttp2_mem *mem);
+
 /*
  * Returns the number of padding bytes after payload.  The total
  * padding length is given in the |padlen|.  The returned value does

lib/nghttp2_hd.c

@@ -584,28 +584,19 @@ static nghttp2_hd_entry *hd_map_find(nghttp2_hd_map *map, int *exact_match,
 }
 
 static void hd_map_remove(nghttp2_hd_map *map, nghttp2_hd_entry *ent) {
-  nghttp2_hd_entry **bucket;
-  nghttp2_hd_entry *p;
+  nghttp2_hd_entry **dst;
 
-  bucket = &map->table[ent->hash & (HD_MAP_SIZE - 1)];
+  dst = &map->table[ent->hash & (HD_MAP_SIZE - 1)];
 
-  if (*bucket == NULL) {
-    return;
-  }
+  for (; *dst; dst = &(*dst)->next) {
+    if (*dst != ent) {
+      continue;
+    }
 
-  if (*bucket == ent) {
-    *bucket = ent->next;
+    *dst = ent->next;
     ent->next = NULL;
     return;
   }
-
-  for (p = *bucket; p; p = p->next) {
-    if (p->next == ent) {
-      p->next = ent->next;
-      ent->next = NULL;
-      return;
-    }
-  }
 }
 
 static int hd_ringbuf_init(nghttp2_hd_ringbuf *ringbuf, size_t bufsize,
@@ -871,11 +862,11 @@ static size_t encode_length(uint8_t *buf, size_t n, size_t prefix) {
  * of bytes processed, or returns -1, indicating decoding error.
  */
 static ssize_t decode_length(uint32_t *res, size_t *shift_ptr, int *final,
-                             uint32_t initial, size_t shift, uint8_t *in,
-                             uint8_t *last, size_t prefix) {
+                             uint32_t initial, size_t shift, const uint8_t *in,
+                             const uint8_t *last, size_t prefix) {
   uint32_t k = (uint8_t)((1 << prefix) - 1);
   uint32_t n = initial;
-  uint8_t *start = in;
+  const uint8_t *start = in;
 
   *shift_ptr = 0;
   *final = 0;
@@ -1003,7 +994,7 @@ static int emit_string(nghttp2_bufs *bufs, const uint8_t *str, size_t len) {
   blocklen = count_encoded_length(enclen, 7);
 
   DEBUGF(fprintf(stderr, "deflatehd: emit string str="));
-  DEBUGF(fwrite(str, len, 1, stderr));
+  DEBUGF(fwrite(str, 1, len, stderr));
   DEBUGF(fprintf(stderr, ", length=%zu, huffman=%d, encoded_length=%zu\n", len,
                  huffman, enclen));
 
@@ -1634,8 +1625,8 @@ static void hd_inflate_set_huffman_encoded(nghttp2_hd_inflater *inflater,
  *   Integer decoding failed
  */
 static ssize_t hd_inflate_read_len(nghttp2_hd_inflater *inflater, int *rfin,
-                                   uint8_t *in, uint8_t *last, size_t prefix,
-                                   size_t maxlen) {
+                                   const uint8_t *in, const uint8_t *last,
+                                   size_t prefix, size_t maxlen) {
   ssize_t rv;
   uint32_t out;
 
@@ -1676,8 +1667,8 @@ static ssize_t hd_inflate_read_len(nghttp2_hd_inflater *inflater, int *rfin,
  *   Huffman decoding failed
  */
 static ssize_t hd_inflate_read_huff(nghttp2_hd_inflater *inflater,
-                                    nghttp2_buf *buf, uint8_t *in,
-                                    uint8_t *last) {
+                                    nghttp2_buf *buf, const uint8_t *in,
+                                    const uint8_t *last) {
   ssize_t readlen;
   int final = 0;
   if ((size_t)(last - in) >= inflater->left) {
@@ -1708,7 +1699,7 @@ static ssize_t hd_inflate_read_huff(nghttp2_hd_inflater *inflater,
  *   Header decompression failed
  */
 static ssize_t hd_inflate_read(nghttp2_hd_inflater *inflater, nghttp2_buf *buf,
-                               uint8_t *in, uint8_t *last) {
+                               const uint8_t *in, const uint8_t *last) {
   size_t len = nghttp2_min((size_t)(last - in), inflater->left);
 
   buf->last = nghttp2_cpymem(buf->last, in, len);
@@ -1831,11 +1822,18 @@ static int hd_inflate_commit_indname(nghttp2_hd_inflater *inflater,
 ssize_t nghttp2_hd_inflate_hd(nghttp2_hd_inflater *inflater, nghttp2_nv *nv_out,
                               int *inflate_flags, uint8_t *in, size_t inlen,
                               int in_final) {
+  return nghttp2_hd_inflate_hd2(inflater, nv_out, inflate_flags, in, inlen,
+                                in_final);
+}
+
+ssize_t nghttp2_hd_inflate_hd2(nghttp2_hd_inflater *inflater,
+                               nghttp2_nv *nv_out, int *inflate_flags,
+                               const uint8_t *in, size_t inlen, int in_final) {
   ssize_t rv;
   nghttp2_hd_nv hd_nv;
 
-  rv = nghttp2_hd_inflate_hd2(inflater, &hd_nv, inflate_flags, in, inlen,
-                              in_final);
+  rv = nghttp2_hd_inflate_hd_nv(inflater, &hd_nv, inflate_flags, in, inlen,
+                                in_final);
 
   if (rv < 0) {
     return rv;
@@ -1854,12 +1852,13 @@ ssize_t nghttp2_hd_inflate_hd(nghttp2_hd_inflater *inflater, nghttp2_nv *nv_out,
   return rv;
 }
 
-ssize_t nghttp2_hd_inflate_hd2(nghttp2_hd_inflater *inflater,
-                               nghttp2_hd_nv *nv_out, int *inflate_flags,
-                               uint8_t *in, size_t inlen, int in_final) {
+ssize_t nghttp2_hd_inflate_hd_nv(nghttp2_hd_inflater *inflater,
+                                 nghttp2_hd_nv *nv_out, int *inflate_flags,
+                                 const uint8_t *in, size_t inlen,
+                                 int in_final) {
   ssize_t rv = 0;
-  uint8_t *first = in;
-  uint8_t *last = in + inlen;
+  const uint8_t *first = in;
+  const uint8_t *last = in + inlen;
   int rfin = 0;
   int busy = 0;
   nghttp2_mem *mem;

lib/nghttp2_hd.h

@@ -353,9 +353,9 @@ void nghttp2_hd_inflate_free(nghttp2_hd_inflater *inflater);
  * that return values and semantics are the same as
  * nghttp2_hd_inflate_hd().
  */
-ssize_t nghttp2_hd_inflate_hd2(nghttp2_hd_inflater *inflater,
-                               nghttp2_hd_nv *nv_out, int *inflate_flags,
-                               uint8_t *in, size_t inlen, int in_final);
+ssize_t nghttp2_hd_inflate_hd_nv(nghttp2_hd_inflater *inflater,
+                                 nghttp2_hd_nv *nv_out, int *inflate_flags,
+                                 const uint8_t *in, size_t inlen, int in_final);
 
 /* For unittesting purpose */
 int nghttp2_hd_emit_indname_block(nghttp2_bufs *bufs, size_t index,

lib/nghttp2_helper.c

@@ -213,6 +213,38 @@ int nghttp2_adjust_local_window_size(int32_t *local_window_size_ptr,
   return 0;
 }
 
+int nghttp2_increase_local_window_size(int32_t *local_window_size_ptr,
+                                       int32_t *recv_window_size_ptr,
+                                       int32_t *recv_reduction_ptr,
+                                       int32_t *delta_ptr) {
+  int32_t recv_reduction_delta;
+  int32_t delta;
+
+  delta = *delta_ptr;
+
+  assert(delta >= 0);
+
+  /* The delta size is strictly more than received bytes. Increase
+     local_window_size by that difference |delta|. */
+  if (*local_window_size_ptr > NGHTTP2_MAX_WINDOW_SIZE - delta) {
+    return NGHTTP2_ERR_FLOW_CONTROL;
+  }
+
+  *local_window_size_ptr += delta;
+  /* If there is recv_reduction due to earlier window_size
+     reduction, we have to adjust it too. */
+  recv_reduction_delta = nghttp2_min(*recv_reduction_ptr, delta);
+  *recv_reduction_ptr -= recv_reduction_delta;
+
+  *recv_window_size_ptr += recv_reduction_delta;
+
+  /* recv_reduction_delta must be paied from *delta_ptr, since it was
+     added in window size reduction (see below). */
+  *delta_ptr -= recv_reduction_delta;
+
+  return 0;
+}
+
 int nghttp2_should_send_window_update(int32_t local_window_size,
                                       int32_t recv_window_size) {
   return recv_window_size > 0 && recv_window_size >= local_window_size / 2;

lib/nghttp2_helper.h

@@ -89,6 +89,22 @@ int nghttp2_adjust_local_window_size(int32_t *local_window_size_ptr,
                                      int32_t *recv_reduction_ptr,
                                      int32_t *delta_ptr);
 
+/*
+ * This function works like nghttp2_adjust_local_window_size().  The
+ * difference is that this function assumes *delta_ptr >= 0, and
+ * *recv_window_size_ptr is not decreased by *delta_ptr.
+ *
+ * This function returns 0 if it succeeds, or one of the following
+ * negative error codes:
+ *
+ * NGHTTP2_ERR_FLOW_CONTROL
+ *     local_window_size overflow or gets negative.
+ */
+int nghttp2_increase_local_window_size(int32_t *local_window_size_ptr,
+                                       int32_t *recv_window_size_ptr,
+                                       int32_t *recv_reduction_ptr,
+                                       int32_t *delta_ptr);
+
 /*
  * Returns non-zero if the function decided that WINDOW_UPDATE should
  * be sent.

lib/nghttp2_map.c

@@ -170,20 +170,18 @@ nghttp2_map_entry *nghttp2_map_find(nghttp2_map *map, key_type key) {
 
 int nghttp2_map_remove(nghttp2_map *map, key_type key) {
   uint32_t h;
-  nghttp2_map_entry *entry, *prev;
+  nghttp2_map_entry **dst;
+
   h = hash(key, map->tablelen);
-  prev = NULL;
-  for (entry = map->table[h]; entry; entry = entry->next) {
-    if (entry->key == key) {
-      if (prev == NULL) {
-        map->table[h] = entry->next;
-      } else {
-        prev->next = entry->next;
-      }
-      --map->size;
-      return 0;
+
+  for (dst = &map->table[h]; *dst; dst = &(*dst)->next) {
+    if ((*dst)->key != key) {
+      continue;
     }
-    prev = entry;
+
+    *dst = (*dst)->next;
+    --map->size;
+    return 0;
   }
   return NGHTTP2_ERR_INVALID_ARGUMENT;
 }

lib/nghttp2_mem.c

@@ -52,8 +52,8 @@ void nghttp2_mem_free(nghttp2_mem *mem, void *ptr) {
   mem->free(ptr, mem->mem_user_data);
 }
 
-void nghttp2_mem_free2(nghttp2_free free, void *ptr, void *mem_user_data) {
-  free(ptr, mem_user_data);
+void nghttp2_mem_free2(nghttp2_free free_func, void *ptr, void *mem_user_data) {
+  free_func(ptr, mem_user_data);
 }
 
 void *nghttp2_mem_calloc(nghttp2_mem *mem, size_t nmemb, size_t size) {

lib/nghttp2_mem.h

@@ -38,7 +38,7 @@ nghttp2_mem *nghttp2_mem_default(void);
    |mem|. */
 void *nghttp2_mem_malloc(nghttp2_mem *mem, size_t size);
 void nghttp2_mem_free(nghttp2_mem *mem, void *ptr);
-void nghttp2_mem_free2(nghttp2_free free, void *ptr, void *mem_user_data);
+void nghttp2_mem_free2(nghttp2_free free_func, void *ptr, void *mem_user_data);
 void *nghttp2_mem_calloc(nghttp2_mem *mem, size_t nmemb, size_t size);
 void *nghttp2_mem_realloc(nghttp2_mem *mem, void *ptr, size_t size);
 

lib/nghttp2_option.c

@@ -24,6 +24,8 @@
  */
 #include "nghttp2_option.h"
 
+#include "nghttp2_session.h"
+
 int nghttp2_option_new(nghttp2_option **option_ptr) {
   *option_ptr = calloc(1, sizeof(nghttp2_option));
 
@@ -63,6 +65,10 @@ void nghttp2_option_set_max_reserved_remote_streams(nghttp2_option *option,
   option->max_reserved_remote_streams = val;
 }
 
+static void set_ext_type(uint8_t *ext_types, uint8_t type) {
+  ext_types[type / 8] = (uint8_t)(ext_types[type / 8] | (1 << (type & 0x7)));
+}
+
 void nghttp2_option_set_user_recv_extension_type(nghttp2_option *option,
                                                  uint8_t type) {
   if (type < 10) {
@@ -70,11 +76,28 @@ void nghttp2_option_set_user_recv_extension_type(nghttp2_option *option,
   }
 
   option->opt_set_mask |= NGHTTP2_OPT_USER_RECV_EXT_TYPES;
-  option->user_recv_ext_types[type / 8] =
-      (uint8_t)(option->user_recv_ext_types[type / 8] | (1 << (type & 0x7)));
+  set_ext_type(option->user_recv_ext_types, type);
+}
+
+void nghttp2_option_set_builtin_recv_extension_type(nghttp2_option *option,
+                                                    uint8_t type) {
+  switch (type) {
+  case NGHTTP2_ALTSVC:
+    option->opt_set_mask |= NGHTTP2_OPT_BUILTIN_RECV_EXT_TYPES;
+    option->builtin_recv_ext_types |= NGHTTP2_TYPEMASK_ALTSVC;
+    return;
+  default:
+    return;
+  }
 }
 
 void nghttp2_option_set_no_auto_ping_ack(nghttp2_option *option, int val) {
   option->opt_set_mask |= NGHTTP2_OPT_NO_AUTO_PING_ACK;
   option->no_auto_ping_ack = val;
 }
+
+void nghttp2_option_set_max_send_header_block_length(nghttp2_option *option,
+                                                     size_t val) {
+  option->opt_set_mask |= NGHTTP2_OPT_MAX_SEND_HEADER_BLOCK_LENGTH;
+  option->max_send_header_block_length = val;
+}

lib/nghttp2_option.h

@@ -61,13 +61,19 @@ typedef enum {
   NGHTTP2_OPT_NO_HTTP_MESSAGING = 1 << 3,
   NGHTTP2_OPT_MAX_RESERVED_REMOTE_STREAMS = 1 << 4,
   NGHTTP2_OPT_USER_RECV_EXT_TYPES = 1 << 5,
-  NGHTTP2_OPT_NO_AUTO_PING_ACK = 1 << 6
+  NGHTTP2_OPT_NO_AUTO_PING_ACK = 1 << 6,
+  NGHTTP2_OPT_BUILTIN_RECV_EXT_TYPES = 1 << 7,
+  NGHTTP2_OPT_MAX_SEND_HEADER_BLOCK_LENGTH = 1 << 8,
 } nghttp2_option_flag;
 
 /**
  * Struct to store option values for nghttp2_session.
  */
 struct nghttp2_option {
+  /**
+   * NGHTTP2_OPT_MAX_SEND_HEADER_BLOCK_LENGTH
+   */
+  size_t max_send_header_block_length;
   /**
    * Bitwise OR of nghttp2_option_flag to determine that which fields
    * are specified.
@@ -81,6 +87,10 @@ struct nghttp2_option {
    * NGHTTP2_OPT_MAX_RESERVED_REMOTE_STREAMS
    */
   uint32_t max_reserved_remote_streams;
+  /**
+   * NGHTTP2_OPT_BUILTIN_RECV_EXT_TYPES
+   */
+  uint32_t builtin_recv_ext_types;
   /**
    * NGHTTP2_OPT_NO_AUTO_WINDOW_UPDATE
    */

lib/nghttp2_outbound_item.c

@@ -72,9 +72,25 @@ void nghttp2_outbound_item_free(nghttp2_outbound_item *item, nghttp2_mem *mem) {
   case NGHTTP2_WINDOW_UPDATE:
     nghttp2_frame_window_update_free(&frame->window_update);
     break;
-  default:
-    nghttp2_frame_extension_free(&frame->ext);
-    break;
+  default: {
+    nghttp2_ext_aux_data *aux_data;
+
+    aux_data = &item->aux_data.ext;
+
+    if (aux_data->builtin == 0) {
+      nghttp2_frame_extension_free(&frame->ext);
+      break;
+    }
+
+    switch (frame->hd.type) {
+    case NGHTTP2_ALTSVC:
+      nghttp2_frame_altsvc_free(&frame->ext, mem);
+      break;
+    default:
+      assert(0);
+      break;
+    }
+  }
   }
 }
 

lib/nghttp2_outbound_item.h

@@ -87,11 +87,19 @@ typedef struct {
   uint8_t flags;
 } nghttp2_goaway_aux_data;
 
+/* struct used for extension frame */
+typedef struct {
+  /* nonzero if this extension frame is serialized by library
+     function, instead of user-defined callbacks. */
+  uint8_t builtin;
+} nghttp2_ext_aux_data;
+
 /* Additional data which cannot be stored in nghttp2_frame struct */
 typedef union {
   nghttp2_data_aux_data data;
   nghttp2_headers_aux_data headers;
   nghttp2_goaway_aux_data goaway;
+  nghttp2_ext_aux_data ext;
 } nghttp2_aux_data;
 
 struct nghttp2_outbound_item;
@@ -99,6 +107,9 @@ typedef struct nghttp2_outbound_item nghttp2_outbound_item;
 
 struct nghttp2_outbound_item {
   nghttp2_frame frame;
+  /* Storage for extension frame payload.  frame->ext.payload points
+     to this structure to avoid frequent memory allocation. */
+  nghttp2_ext_frame_payload ext_frame_payload;
   nghttp2_aux_data aux_data;
   /* The priority used in priority comparion.  Smaller is served
      ealier.  For PING, SETTINGS and non-DATA frames (excluding

lib/nghttp2_session.c

@@ -142,6 +142,10 @@ static int session_detect_idle_stream(nghttp2_session *session,
   return 0;
 }
 
+static int check_ext_type_set(const uint8_t *ext_types, uint8_t type) {
+  return (ext_types[type / 8] & (1 << (type & 0x7))) > 0;
+}
+
 static int session_call_error_callback(nghttp2_session *session,
                                        const char *fmt, ...) {
   size_t bufsize;
@@ -301,6 +305,13 @@ static void session_inbound_frame_reset(nghttp2_session *session) {
     break;
   case NGHTTP2_SETTINGS:
     nghttp2_frame_settings_free(&iframe->frame.settings, mem);
+
+    nghttp2_mem_free(mem, iframe->iv);
+
+    iframe->iv = NULL;
+    iframe->niv = 0;
+    iframe->max_niv = 0;
+
     break;
   case NGHTTP2_PUSH_PROMISE:
     nghttp2_frame_push_promise_free(&iframe->frame.push_promise, mem);
@@ -316,7 +327,20 @@ static void session_inbound_frame_reset(nghttp2_session *session) {
     break;
   default:
     /* extension frame */
-    nghttp2_frame_extension_free(&iframe->frame.ext);
+    if (check_ext_type_set(session->user_recv_ext_types,
+                           iframe->frame.hd.type)) {
+      nghttp2_frame_extension_free(&iframe->frame.ext);
+    } else {
+      switch (iframe->frame.hd.type) {
+      case NGHTTP2_ALTSVC:
+        if ((session->builtin_recv_ext_types & NGHTTP2_TYPEMASK_ALTSVC) == 0) {
+          break;
+        }
+        nghttp2_frame_altsvc_free(&iframe->frame.ext, mem);
+        break;
+      }
+    }
+
     break;
   }
 
@@ -332,12 +356,10 @@ static void session_inbound_frame_reset(nghttp2_session *session) {
   nghttp2_buf_free(&iframe->lbuf, mem);
   nghttp2_buf_wrap_init(&iframe->lbuf, NULL, 0);
 
-  iframe->niv = 0;
+  iframe->raw_lbuf = NULL;
+
   iframe->payloadleft = 0;
   iframe->padlen = 0;
-  iframe->iv[NGHTTP2_INBOUND_NUM_IV - 1].settings_id =
-      NGHTTP2_SETTINGS_HEADER_TABLE_SIZE;
-  iframe->iv[NGHTTP2_INBOUND_NUM_IV - 1].value = UINT32_MAX;
 }
 
 static void init_settings(nghttp2_settings_storage *settings) {
@@ -367,6 +389,7 @@ static int session_new(nghttp2_session **session_ptr,
                        void *user_data, int server,
                        const nghttp2_option *option, nghttp2_mem *mem) {
   int rv;
+  size_t nbuffer;
 
   if (mem == NULL) {
     mem = nghttp2_mem_default();
@@ -419,16 +442,6 @@ static int session_new(nghttp2_session **session_ptr,
     (*session_ptr)->server = 1;
   }
 
-  /* 1 for Pad Field. */
-  rv = nghttp2_bufs_init3(&(*session_ptr)->aob.framebufs,
-                          NGHTTP2_FRAMEBUF_CHUNKLEN, NGHTTP2_FRAMEBUF_MAX_NUM,
-                          1, NGHTTP2_FRAME_HDLEN + 1, mem);
-  if (rv != 0) {
-    goto fail_aob_framebuf;
-  }
-
-  active_outbound_item_reset(&(*session_ptr)->aob, mem);
-
   init_settings(&(*session_ptr)->remote_settings);
   init_settings(&(*session_ptr)->local_settings);
 
@@ -438,6 +451,8 @@ static int session_new(nghttp2_session **session_ptr,
   /* Limit max outgoing concurrent streams to sensible value */
   (*session_ptr)->remote_settings.max_concurrent_streams = 100;
 
+  (*session_ptr)->max_send_header_block_length = NGHTTP2_MAX_HEADERSLEN;
+
   if (option) {
     if ((option->opt_set_mask & NGHTTP2_OPT_NO_AUTO_WINDOW_UPDATE) &&
         option->no_auto_window_update) {
@@ -474,12 +489,39 @@ static int session_new(nghttp2_session **session_ptr,
              sizeof((*session_ptr)->user_recv_ext_types));
     }
 
+    if (option->opt_set_mask & NGHTTP2_OPT_BUILTIN_RECV_EXT_TYPES) {
+      (*session_ptr)->builtin_recv_ext_types = option->builtin_recv_ext_types;
+    }
+
     if ((option->opt_set_mask & NGHTTP2_OPT_NO_AUTO_PING_ACK) &&
         option->no_auto_ping_ack) {
       (*session_ptr)->opt_flags |= NGHTTP2_OPTMASK_NO_AUTO_PING_ACK;
     }
+
+    if (option->opt_set_mask & NGHTTP2_OPT_MAX_SEND_HEADER_BLOCK_LENGTH) {
+      (*session_ptr)->max_send_header_block_length =
+          option->max_send_header_block_length;
+    }
   }
 
+  nbuffer = ((*session_ptr)->max_send_header_block_length +
+             NGHTTP2_FRAMEBUF_CHUNKLEN - 1) /
+            NGHTTP2_FRAMEBUF_CHUNKLEN;
+
+  if (nbuffer == 0) {
+    nbuffer = 1;
+  }
+
+  /* 1 for Pad Field. */
+  rv = nghttp2_bufs_init3(&(*session_ptr)->aob.framebufs,
+                          NGHTTP2_FRAMEBUF_CHUNKLEN, nbuffer, 1,
+                          NGHTTP2_FRAME_HDLEN + 1, mem);
+  if (rv != 0) {
+    goto fail_aob_framebuf;
+  }
+
+  active_outbound_item_reset(&(*session_ptr)->aob, mem);
+
   (*session_ptr)->callbacks = *callbacks;
   (*session_ptr)->user_data = user_data;
 
@@ -1642,6 +1684,11 @@ static int session_predicate_push_promise_send(nghttp2_session *session,
 static int session_predicate_window_update_send(nghttp2_session *session,
                                                 int32_t stream_id) {
   nghttp2_stream *stream;
+
+  if (session_is_closing(session)) {
+    return NGHTTP2_ERR_SESSION_CLOSING;
+  }
+
   if (stream_id == 0) {
     /* Connection-level window update */
     return 0;
@@ -1650,9 +1697,6 @@ static int session_predicate_window_update_send(nghttp2_session *session,
   if (stream == NULL) {
     return NGHTTP2_ERR_STREAM_CLOSED;
   }
-  if (session_is_closing(session)) {
-    return NGHTTP2_ERR_SESSION_CLOSING;
-  }
   if (stream->state == NGHTTP2_STREAM_CLOSING) {
     return NGHTTP2_ERR_STREAM_CLOSING;
   }
@@ -1662,6 +1706,29 @@ static int session_predicate_window_update_send(nghttp2_session *session,
   return 0;
 }
 
+static int session_predicate_altsvc_send(nghttp2_session *session,
+                                         int32_t stream_id) {
+  nghttp2_stream *stream;
+
+  if (session_is_closing(session)) {
+    return NGHTTP2_ERR_SESSION_CLOSING;
+  }
+
+  if (stream_id == 0) {
+    return 0;
+  }
+
+  stream = nghttp2_session_get_stream(session, stream_id);
+  if (stream == NULL) {
+    return NGHTTP2_ERR_STREAM_CLOSED;
+  }
+  if (stream->state == NGHTTP2_STREAM_CLOSING) {
+    return NGHTTP2_ERR_STREAM_CLOSING;
+  }
+
+  return 0;
+}
+
 /* Take into account settings max frame size and both connection-level
    flow control here */
 static ssize_t
@@ -1900,7 +1967,7 @@ static int session_prep_frame(nghttp2_session *session,
             session, frame->headers.nva, frame->headers.nvlen,
             NGHTTP2_PRIORITY_SPECLEN);
 
-        if (estimated_payloadlen > NGHTTP2_MAX_HEADERSLEN) {
+        if (estimated_payloadlen > session->max_send_header_block_length) {
           return NGHTTP2_ERR_FRAME_SIZE_ERROR;
         }
 
@@ -1919,7 +1986,7 @@ static int session_prep_frame(nghttp2_session *session,
             session, frame->headers.nva, frame->headers.nvlen,
             NGHTTP2_PRIORITY_SPECLEN);
 
-        if (estimated_payloadlen > NGHTTP2_MAX_HEADERSLEN) {
+        if (estimated_payloadlen > session->max_send_header_block_length) {
           return NGHTTP2_ERR_FRAME_SIZE_ERROR;
         }
 
@@ -2038,7 +2105,7 @@ static int session_prep_frame(nghttp2_session *session,
       estimated_payloadlen = session_estimate_headers_payload(
           session, frame->push_promise.nva, frame->push_promise.nvlen, 0);
 
-      if (estimated_payloadlen > NGHTTP2_MAX_HEADERSLEN) {
+      if (estimated_payloadlen > session->max_send_header_block_length) {
         return NGHTTP2_ERR_FRAME_SIZE_ERROR;
       }
 
@@ -2101,19 +2168,45 @@ static int session_prep_frame(nghttp2_session *session,
       /* We never handle CONTINUATION here. */
       assert(0);
       break;
-    default:
+    default: {
+      nghttp2_ext_aux_data *aux_data;
+
       /* extension frame */
-      if (session_is_closing(session)) {
-        return NGHTTP2_ERR_SESSION_CLOSING;
+
+      aux_data = &item->aux_data.ext;
+
+      if (aux_data->builtin == 0) {
+        if (session_is_closing(session)) {
+          return NGHTTP2_ERR_SESSION_CLOSING;
+        }
+
+        rv = session_pack_extension(session, &session->aob.framebufs, frame);
+        if (rv != 0) {
+          return rv;
+        }
+
+        break;
       }
 
-      rv = session_pack_extension(session, &session->aob.framebufs, frame);
-      if (rv != 0) {
-        return rv;
+      switch (frame->hd.type) {
+      case NGHTTP2_ALTSVC:
+        rv = session_predicate_altsvc_send(session, frame->hd.stream_id);
+        if (rv != 0) {
+          return rv;
+        }
+
+        nghttp2_frame_pack_altsvc(&session->aob.framebufs, &frame->ext);
+
+        break;
+      default:
+        /* Unreachable here */
+        assert(0);
+        break;
       }
 
       break;
     }
+    }
     return 0;
   } else {
     size_t next_readmax;
@@ -2271,6 +2364,10 @@ static int session_call_before_frame_send(nghttp2_session *session,
   if (session->callbacks.before_frame_send_callback) {
     rv = session->callbacks.before_frame_send_callback(session, frame,
                                                        session->user_data);
+    if (rv == NGHTTP2_ERR_CANCEL) {
+      return rv;
+    }
+
     if (rv != 0) {
       return NGHTTP2_ERR_CALLBACK_FAILURE;
     }
@@ -2808,7 +2905,6 @@ static ssize_t nghttp2_session_mem_send_internal(nghttp2_session *session,
     return rv;
   }
 
-  *data_ptr = NULL;
   for (;;) {
     switch (aob->state) {
     case NGHTTP2_OB_POP_ITEM: {
@@ -2912,6 +3008,51 @@ static ssize_t nghttp2_session_mem_send_internal(nghttp2_session *session,
         if (nghttp2_is_fatal(rv)) {
           return rv;
         }
+
+        if (rv == NGHTTP2_ERR_CANCEL) {
+          int32_t opened_stream_id = 0;
+          uint32_t error_code = NGHTTP2_INTERNAL_ERROR;
+
+          if (session->callbacks.on_frame_not_send_callback) {
+            if (session->callbacks.on_frame_not_send_callback(
+                    session, frame, rv, session->user_data) != 0) {
+              return NGHTTP2_ERR_CALLBACK_FAILURE;
+            }
+          }
+
+          /* We have to close stream opened by canceled request
+             HEADERS or PUSH_PROMISE. */
+          switch (item->frame.hd.type) {
+          case NGHTTP2_HEADERS:
+            if (item->frame.headers.cat == NGHTTP2_HCAT_REQUEST) {
+              opened_stream_id = item->frame.hd.stream_id;
+              /* We don't have to check
+                 item->aux_data.headers.canceled since it has already
+                 been checked. */
+              /* Set error_code to REFUSED_STREAM so that application
+                 can send request again. */
+              error_code = NGHTTP2_REFUSED_STREAM;
+            }
+            break;
+          case NGHTTP2_PUSH_PROMISE:
+            opened_stream_id = item->frame.push_promise.promised_stream_id;
+            break;
+          }
+          if (opened_stream_id) {
+            /* careful not to override rv */
+            int rv2;
+            rv2 = nghttp2_session_close_stream(session, opened_stream_id,
+                                               error_code);
+
+            if (nghttp2_is_fatal(rv2)) {
+              return rv2;
+            }
+          }
+
+          active_outbound_item_reset(aob, mem);
+
+          break;
+        }
       } else {
         DEBUGF(fprintf(stderr, "send: next frame: DATA\n"));
 
@@ -3065,6 +3206,8 @@ ssize_t nghttp2_session_mem_send(nghttp2_session *session,
   int rv;
   ssize_t len;
 
+  *data_ptr = NULL;
+
   len = nghttp2_session_mem_send_internal(session, data_ptr, 1);
   if (len <= 0) {
     return len;
@@ -3086,7 +3229,7 @@ ssize_t nghttp2_session_mem_send(nghttp2_session *session,
 }
 
 int nghttp2_session_send(nghttp2_session *session) {
-  const uint8_t *data;
+  const uint8_t *data = NULL;
   ssize_t datalen;
   ssize_t sentlen;
   nghttp2_bufs *framebufs;
@@ -3397,8 +3540,8 @@ static int inflate_header_block(nghttp2_session *session, nghttp2_frame *frame,
   DEBUGF(fprintf(stderr, "recv: decoding header block %zu bytes\n", inlen));
   for (;;) {
     inflate_flags = 0;
-    proclen = nghttp2_hd_inflate_hd2(&session->hd_inflater, &nv, &inflate_flags,
-                                     in, inlen, final);
+    proclen = nghttp2_hd_inflate_hd_nv(&session->hd_inflater, &nv,
+                                       &inflate_flags, in, inlen, final);
     if (nghttp2_is_fatal((int)proclen)) {
       return (int)proclen;
     }
@@ -4294,39 +4437,39 @@ int nghttp2_session_on_settings_received(nghttp2_session *session,
 }
 
 static int session_process_settings_frame(nghttp2_session *session) {
-  int rv;
   nghttp2_inbound_frame *iframe = &session->iframe;
   nghttp2_frame *frame = &iframe->frame;
   size_t i;
   nghttp2_settings_entry min_header_size_entry;
-  nghttp2_mem *mem;
 
-  mem = &session->mem;
-  min_header_size_entry = iframe->iv[NGHTTP2_INBOUND_NUM_IV - 1];
+  if (iframe->max_niv) {
+    min_header_size_entry = iframe->iv[iframe->max_niv - 1];
 
-  if (min_header_size_entry.value < UINT32_MAX) {
-    /* If we have less value, then we must have
-       SETTINGS_HEADER_TABLE_SIZE in i < iframe->niv */
-    for (i = 0; i < iframe->niv; ++i) {
-      if (iframe->iv[i].settings_id == NGHTTP2_SETTINGS_HEADER_TABLE_SIZE) {
-        break;
+    if (min_header_size_entry.value < UINT32_MAX) {
+      /* If we have less value, then we must have
+         SETTINGS_HEADER_TABLE_SIZE in i < iframe->niv */
+      for (i = 0; i < iframe->niv; ++i) {
+        if (iframe->iv[i].settings_id == NGHTTP2_SETTINGS_HEADER_TABLE_SIZE) {
+          break;
+        }
+      }
+
+      assert(i < iframe->niv);
+
+      if (min_header_size_entry.value != iframe->iv[i].value) {
+        iframe->iv[iframe->niv++] = iframe->iv[i];
+        iframe->iv[i] = min_header_size_entry;
       }
     }
-
-    assert(i < iframe->niv);
-
-    if (min_header_size_entry.value != iframe->iv[i].value) {
-      iframe->iv[iframe->niv++] = iframe->iv[i];
-      iframe->iv[i] = min_header_size_entry;
-    }
   }
 
-  rv = nghttp2_frame_unpack_settings_payload(&frame->settings, iframe->iv,
-                                             iframe->niv, mem);
-  if (rv != 0) {
-    assert(nghttp2_is_fatal(rv));
-    return rv;
-  }
+  nghttp2_frame_unpack_settings_payload(&frame->settings, iframe->iv,
+                                        iframe->niv);
+
+  iframe->iv = NULL;
+  iframe->niv = 0;
+  iframe->max_niv = 0;
+
   return nghttp2_session_on_settings_received(session, frame, 0 /* ACK */);
 }
 
@@ -4589,6 +4732,52 @@ static int session_process_window_update_frame(nghttp2_session *session) {
   return nghttp2_session_on_window_update_received(session, frame);
 }
 
+int nghttp2_session_on_altsvc_received(nghttp2_session *session,
+                                       nghttp2_frame *frame) {
+  nghttp2_ext_altsvc *altsvc;
+  nghttp2_stream *stream;
+
+  altsvc = frame->ext.payload;
+
+  /* session->server case has been excluded */
+
+  if (frame->hd.stream_id == 0) {
+    if (altsvc->origin_len == 0) {
+      return 0;
+    }
+  } else {
+    if (altsvc->origin_len > 0) {
+      return 0;
+    }
+
+    stream = nghttp2_session_get_stream(session, frame->hd.stream_id);
+    if (!stream) {
+      return 0;
+    }
+
+    if (stream->state == NGHTTP2_STREAM_CLOSING) {
+      return 0;
+    }
+  }
+
+  return session_call_on_frame_received(session, frame);
+}
+
+static int session_process_altsvc_frame(nghttp2_session *session) {
+  nghttp2_inbound_frame *iframe = &session->iframe;
+  nghttp2_frame *frame = &iframe->frame;
+
+  nghttp2_frame_unpack_altsvc_payload(
+      &frame->ext, nghttp2_get_uint16(iframe->sbuf.pos), iframe->lbuf.pos,
+      nghttp2_buf_len(&iframe->lbuf));
+
+  /* nghttp2_frame_unpack_altsvc_payload steals buffer from
+     iframe->lbuf */
+  nghttp2_buf_wrap_init(&iframe->lbuf, NULL, 0);
+
+  return nghttp2_session_on_altsvc_received(session, frame);
+}
+
 static int session_process_extension_frame(nghttp2_session *session) {
   int rv;
   nghttp2_inbound_frame *iframe = &session->iframe;
@@ -4936,6 +5125,7 @@ static size_t inbound_frame_buf_read(nghttp2_inbound_frame *iframe,
  */
 static void inbound_frame_set_settings_entry(nghttp2_inbound_frame *iframe) {
   nghttp2_settings_entry iv;
+  nghttp2_settings_entry *min_header_table_size_entry;
   size_t i;
 
   nghttp2_frame_unpack_settings_entry(&iv, iframe->sbuf.pos);
@@ -4949,8 +5139,11 @@ static void inbound_frame_set_settings_entry(nghttp2_inbound_frame *iframe) {
   case NGHTTP2_SETTINGS_MAX_HEADER_LIST_SIZE:
     break;
   default:
-    DEBUGF(fprintf(stderr, "recv: ignore unknown settings id=0x%02x\n",
-                   iv.settings_id));
+    DEBUGF(
+        fprintf(stderr, "recv: unknown settings id=0x%02x\n", iv.settings_id));
+
+    iframe->iv[iframe->niv++] = iv;
+
     return;
   }
 
@@ -4965,10 +5158,13 @@ static void inbound_frame_set_settings_entry(nghttp2_inbound_frame *iframe) {
     iframe->iv[iframe->niv++] = iv;
   }
 
-  if (iv.settings_id == NGHTTP2_SETTINGS_HEADER_TABLE_SIZE &&
-      iv.value < iframe->iv[NGHTTP2_INBOUND_NUM_IV - 1].value) {
+  if (iv.settings_id == NGHTTP2_SETTINGS_HEADER_TABLE_SIZE) {
+    /* Keep track of minimum value of SETTINGS_HEADER_TABLE_SIZE */
+    min_header_table_size_entry = &iframe->iv[iframe->max_niv - 1];
 
-    iframe->iv[NGHTTP2_INBOUND_NUM_IV - 1] = iv;
+    if (iv.value < min_header_table_size_entry->value) {
+      min_header_table_size_entry->value = iv.value;
+    }
   }
 }
 
@@ -5097,6 +5293,15 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
 
         iframe->state = NGHTTP2_IB_IGN_ALL;
 
+        rv = session_call_error_callback(
+            session, "Remote peer returned unexpected data while we expected "
+                     "SETTINGS frame.  Perhaps, peer does not support HTTP/2 "
+                     "properly.");
+
+        if (nghttp2_is_fatal(rv)) {
+          return rv;
+        }
+
         rv = nghttp2_session_terminate_session_with_reason(
             session, NGHTTP2_PROTOCOL_ERROR, "SETTINGS expected");
 
@@ -5332,6 +5537,25 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
         iframe->state = NGHTTP2_IB_READ_SETTINGS;
 
         if (iframe->payloadleft) {
+          nghttp2_settings_entry *min_header_table_size_entry;
+
+          /* We allocate iv with addtional one entry, to store the
+             minimum header table size. */
+          iframe->max_niv =
+              iframe->frame.hd.length / NGHTTP2_FRAME_SETTINGS_ENTRY_LENGTH + 1;
+
+          iframe->iv = nghttp2_mem_malloc(mem, sizeof(nghttp2_settings_entry) *
+                                                   iframe->max_niv);
+
+          if (!iframe->iv) {
+            return NGHTTP2_ERR_NOMEM;
+          }
+
+          min_header_table_size_entry = &iframe->iv[iframe->max_niv - 1];
+          min_header_table_size_entry->settings_id =
+              NGHTTP2_SETTINGS_HEADER_TABLE_SIZE;
+          min_header_table_size_entry->value = UINT32_MAX;
+
           inbound_frame_set_mark(iframe, NGHTTP2_FRAME_SETTINGS_ENTRY_LENGTH);
           break;
         }
@@ -5423,25 +5647,65 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
 
         break;
       default:
-        DEBUGF(fprintf(stderr, "recv: unknown frame\n"));
+        DEBUGF(fprintf(stderr, "recv: extension frame\n"));
 
-        if (!session->callbacks.unpack_extension_callback ||
-            (session->user_recv_ext_types[iframe->frame.hd.type / 8] &
-             (1 << (iframe->frame.hd.type & 0x7))) == 0) {
-          /* Silently ignore unknown frame type. */
+        if (check_ext_type_set(session->user_recv_ext_types,
+                               iframe->frame.hd.type)) {
+          if (!session->callbacks.unpack_extension_callback) {
+            /* Silently ignore unknown frame type. */
+            busy = 1;
+
+            iframe->state = NGHTTP2_IB_IGN_PAYLOAD;
+
+            break;
+          }
 
           busy = 1;
 
-          iframe->state = NGHTTP2_IB_IGN_PAYLOAD;
+          iframe->state = NGHTTP2_IB_READ_EXTENSION_PAYLOAD;
 
           break;
+        } else {
+          switch (iframe->frame.hd.type) {
+          case NGHTTP2_ALTSVC:
+            if ((session->builtin_recv_ext_types & NGHTTP2_TYPEMASK_ALTSVC) ==
+                0) {
+              busy = 1;
+              iframe->state = NGHTTP2_IB_IGN_PAYLOAD;
+              break;
+            }
+
+            DEBUGF(fprintf(stderr, "recv: ALTSVC\n"));
+
+            iframe->frame.hd.flags = NGHTTP2_FLAG_NONE;
+            iframe->frame.ext.payload = &iframe->ext_frame_payload.altsvc;
+
+            if (session->server) {
+              busy = 1;
+              iframe->state = NGHTTP2_IB_IGN_PAYLOAD;
+              break;
+            }
+
+            if (iframe->payloadleft < 2) {
+              busy = 1;
+              iframe->state = NGHTTP2_IB_FRAME_SIZE_ERROR;
+              break;
+            }
+
+            busy = 1;
+
+            iframe->state = NGHTTP2_IB_READ_NBYTE;
+            inbound_frame_set_mark(iframe, 2);
+
+            break;
+          default:
+            busy = 1;
+
+            iframe->state = NGHTTP2_IB_IGN_PAYLOAD;
+
+            break;
+          }
         }
-
-        busy = 1;
-
-        iframe->state = NGHTTP2_IB_READ_EXTENSION_PAYLOAD;
-
-        break;
       }
 
       if (!on_begin_frame_called) {
@@ -5651,6 +5915,37 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
         session_inbound_frame_reset(session);
 
         break;
+      case NGHTTP2_ALTSVC: {
+        size_t origin_len;
+
+        origin_len = nghttp2_get_uint16(iframe->sbuf.pos);
+
+        DEBUGF(fprintf(stderr, "recv: origin_len=%zu\n", origin_len));
+
+        if (2 + origin_len > iframe->payloadleft) {
+          busy = 1;
+          iframe->state = NGHTTP2_IB_FRAME_SIZE_ERROR;
+          break;
+        }
+
+        if (iframe->frame.hd.length > 2) {
+          iframe->raw_lbuf =
+              nghttp2_mem_malloc(mem, iframe->frame.hd.length - 2);
+
+          if (iframe->raw_lbuf == NULL) {
+            return NGHTTP2_ERR_NOMEM;
+          }
+
+          nghttp2_buf_wrap_init(&iframe->lbuf, iframe->raw_lbuf,
+                                iframe->frame.hd.length);
+        }
+
+        busy = 1;
+
+        iframe->state = NGHTTP2_IB_READ_ALTSVC_PAYLOAD;
+
+        break;
+      }
       default:
         /* This is unknown frame */
         session_inbound_frame_reset(session);
@@ -6176,6 +6471,36 @@ ssize_t nghttp2_session_mem_recv(nghttp2_session *session, const uint8_t *in,
 
       session_inbound_frame_reset(session);
 
+      break;
+    case NGHTTP2_IB_READ_ALTSVC_PAYLOAD:
+      DEBUGF(fprintf(stderr, "recv: [IB_READ_ALTSVC_PAYLOAD]\n"));
+
+      readlen = inbound_frame_payload_readlen(iframe, in, last);
+
+      if (readlen > 0) {
+        iframe->lbuf.last = nghttp2_cpymem(iframe->lbuf.last, in, readlen);
+
+        iframe->payloadleft -= readlen;
+        in += readlen;
+      }
+
+      DEBUGF(fprintf(stderr, "recv: readlen=%zu, payloadleft=%zu\n", readlen,
+                     iframe->payloadleft));
+
+      if (iframe->payloadleft) {
+        assert(nghttp2_buf_avail(&iframe->lbuf) > 0);
+
+        break;
+      }
+
+      rv = session_process_altsvc_frame(session);
+
+      if (nghttp2_is_fatal(rv)) {
+        return rv;
+      }
+
+      session_inbound_frame_reset(session);
+
       break;
     }
 
@@ -6405,17 +6730,12 @@ int nghttp2_session_add_window_update(nghttp2_session *session, uint8_t flags,
 static void
 session_append_inflight_settings(nghttp2_session *session,
                                  nghttp2_inflight_settings *settings) {
-  nghttp2_inflight_settings *i;
+  nghttp2_inflight_settings **i;
 
-  if (!session->inflight_settings_head) {
-    session->inflight_settings_head = settings;
-    return;
-  }
-
-  for (i = session->inflight_settings_head; i->next; i = i->next)
+  for (i = &session->inflight_settings_head; *i; i = &(*i)->next)
     ;
 
-  i->next = settings;
+  *i = settings;
 }
 
 int nghttp2_session_add_settings(nghttp2_session *session, uint8_t flags,
@@ -6489,10 +6809,10 @@ int nghttp2_session_add_settings(nghttp2_session *session, uint8_t flags,
 
   if (flags & NGHTTP2_FLAG_ACK) {
     ++session->obq_flood_counter_;
+  } else {
+    session_append_inflight_settings(session, inflight_settings);
   }
 
-  session_append_inflight_settings(session, inflight_settings);
-
   /* Extract NGHTTP2_SETTINGS_MAX_CONCURRENT_STREAMS and ENABLE_PUSH
      here.  We use it to refuse the incoming stream and PUSH_PROMISE
      with RST_STREAM. */
@@ -6645,6 +6965,14 @@ int nghttp2_session_pack_data(nghttp2_session *session, nghttp2_bufs *bufs,
 
   reschedule_stream(stream);
 
+  if (frame->hd.length == 0 && (data_flags & NGHTTP2_DATA_FLAG_EOF) &&
+      (data_flags & NGHTTP2_DATA_FLAG_NO_END_STREAM)) {
+    /* DATA payload length is 0, and DATA frame does not bear
+       END_STREAM.  In this case, there is no point to send 0 length
+       DATA frame. */
+    return NGHTTP2_ERR_CANCEL;
+  }
+
   return 0;
 }
 

lib/nghttp2_session.h

@@ -54,6 +54,15 @@ typedef enum {
   NGHTTP2_OPTMASK_NO_AUTO_PING_ACK = 1 << 3
 } nghttp2_optmask;
 
+/*
+ * bitmask for built-in type to enable the default handling for that
+ * type of the frame.
+ */
+typedef enum {
+  NGHTTP2_TYPEMASK_NONE = 0,
+  NGHTTP2_TYPEMASK_ALTSVC = 1 << 0
+} nghttp2_typemask;
+
 typedef enum {
   NGHTTP2_OB_POP_ITEM,
   NGHTTP2_OB_SEND_DATA,
@@ -107,21 +116,20 @@ typedef enum {
   NGHTTP2_IB_READ_DATA,
   NGHTTP2_IB_IGN_DATA,
   NGHTTP2_IB_IGN_ALL,
+  NGHTTP2_IB_READ_ALTSVC_PAYLOAD,
   NGHTTP2_IB_READ_EXTENSION_PAYLOAD
 } nghttp2_inbound_state;
 
-#define NGHTTP2_INBOUND_NUM_IV 7
-
 typedef struct {
   nghttp2_frame frame;
   /* Storage for extension frame payload.  frame->ext.payload points
      to this structure to avoid frequent memory allocation. */
   nghttp2_ext_frame_payload ext_frame_payload;
-  /* The received SETTINGS entry. The protocol says that we only cares
-     about the defined settings ID. If unknown ID is received, it is
-     ignored.  We use last entry to hold minimum header table size if
-     same settings are multiple times. */
-  nghttp2_settings_entry iv[NGHTTP2_INBOUND_NUM_IV];
+  /* The received SETTINGS entry.  For the standard settings entries,
+     we only keep the last seen value.  For
+     SETTINGS_HEADER_TABLE_SIZE, we also keep minimum value in the
+     last index. */
+  nghttp2_settings_entry *iv;
   /* buffer pointers to small buffer, raw_sbuf */
   nghttp2_buf sbuf;
   /* buffer pointers to large buffer, raw_lbuf */
@@ -130,6 +138,8 @@ typedef struct {
   uint8_t *raw_lbuf;
   /* The number of entry filled in |iv| */
   size_t niv;
+  /* The number of entries |iv| can store. */
+  size_t max_niv;
   /* How many bytes we still need to receive for current frame */
   size_t payloadleft;
   /* padding length for the current frame */
@@ -246,6 +256,9 @@ struct nghttp2_session {
   size_t nvbuflen;
   /* Counter for detecting flooding in outbound queue */
   size_t obq_flood_counter_;
+  /* The maximum length of header block to send.  Calculated by the
+     same way as nghttp2_hd_deflate_bound() does. */
+  size_t max_send_header_block_length;
   /* Next Stream ID. Made unsigned int to detect >= (1 << 31). */
   uint32_t next_stream_id;
   /* The last stream ID this session initiated.  For client session,
@@ -294,6 +307,9 @@ struct nghttp2_session {
   /* Unacked local SETTINGS_MAX_CONCURRENT_STREAMS value. We use this
      to refuse the incoming stream if it exceeds this value. */
   uint32_t pending_local_max_concurrent_stream;
+  /* The bitwose OR of zero or more of nghttp2_typemask to indicate
+     that the default handling of extension frame is enabled. */
+  uint32_t builtin_recv_ext_types;
   /* Unacked local ENABLE_PUSH value.  We use this to refuse
      PUSH_PROMISE before SETTINGS ACK is received. */
   uint8_t pending_enable_push;
@@ -716,6 +732,19 @@ int nghttp2_session_on_goaway_received(nghttp2_session *session,
 int nghttp2_session_on_window_update_received(nghttp2_session *session,
                                               nghttp2_frame *frame);
 
+/*
+ * Called when ALTSVC is recieved, assuming |frame| is properly
+ * initialized.
+ *
+ * This function returns 0 if it succeeds, or one of the following
+ * negative error codes:
+ *
+ * NGHTTP2_ERR_CALLBACK_FAILURE
+ *   The callback function failed.
+ */
+int nghttp2_session_on_altsvc_received(nghttp2_session *session,
+                                       nghttp2_frame *frame);
+
 /*
  * Called when DATA is received, assuming |frame| is properly
  * initialized.

lib/nghttp2_submit.c

@@ -214,7 +214,7 @@ int32_t nghttp2_submit_headers(nghttp2_session *session, uint8_t flags,
 int nghttp2_submit_ping(nghttp2_session *session, uint8_t flags,
                         const uint8_t *opaque_data) {
   flags &= NGHTTP2_FLAG_ACK;
-  return nghttp2_session_add_ping(session, NGHTTP2_FLAG_NONE, opaque_data);
+  return nghttp2_session_add_ping(session, flags, opaque_data);
 }
 
 int nghttp2_submit_priority(nghttp2_session *session, uint8_t flags _U_,
@@ -410,6 +410,159 @@ int nghttp2_submit_window_update(nghttp2_session *session, uint8_t flags,
   return 0;
 }
 
+int nghttp2_session_set_local_window_size(nghttp2_session *session,
+                                          uint8_t flags, int32_t stream_id,
+                                          int32_t window_size) {
+  int32_t window_size_increment;
+  nghttp2_stream *stream;
+  int rv;
+
+  if (window_size < 0) {
+    return NGHTTP2_ERR_INVALID_ARGUMENT;
+  }
+
+  flags = 0;
+
+  if (stream_id == 0) {
+    window_size_increment = window_size - session->local_window_size;
+
+    if (window_size_increment == 0) {
+      return 0;
+    }
+
+    if (window_size_increment < 0) {
+      return nghttp2_adjust_local_window_size(
+          &session->local_window_size, &session->recv_window_size,
+          &session->recv_reduction, &window_size_increment);
+    }
+
+    rv = nghttp2_increase_local_window_size(
+        &session->local_window_size, &session->recv_window_size,
+        &session->recv_reduction, &window_size_increment);
+
+    if (rv != 0) {
+      return rv;
+    }
+  } else {
+    stream = nghttp2_session_get_stream(session, stream_id);
+
+    if (stream == NULL) {
+      return 0;
+    }
+
+    window_size_increment = window_size - stream->local_window_size;
+
+    if (window_size_increment == 0) {
+      return 0;
+    }
+
+    if (window_size_increment < 0) {
+      return nghttp2_adjust_local_window_size(
+          &stream->local_window_size, &stream->recv_window_size,
+          &stream->recv_reduction, &window_size_increment);
+    }
+
+    rv = nghttp2_increase_local_window_size(
+        &stream->local_window_size, &stream->recv_window_size,
+        &stream->recv_reduction, &window_size_increment);
+
+    if (rv != 0) {
+      return rv;
+    }
+  }
+
+  if (window_size_increment > 0) {
+    return nghttp2_session_add_window_update(session, flags, stream_id,
+                                             window_size_increment);
+  }
+
+  return 0;
+}
+
+int nghttp2_submit_altsvc(nghttp2_session *session, uint8_t flags _U_,
+                          int32_t stream_id, const uint8_t *origin,
+                          size_t origin_len, const uint8_t *field_value,
+                          size_t field_value_len) {
+  nghttp2_mem *mem;
+  uint8_t *buf, *p;
+  uint8_t *origin_copy;
+  uint8_t *field_value_copy;
+  nghttp2_outbound_item *item;
+  nghttp2_frame *frame;
+  nghttp2_ext_altsvc *altsvc;
+  int rv;
+
+  mem = &session->mem;
+
+  if (!session->server) {
+    return NGHTTP2_ERR_INVALID_STATE;
+  }
+
+  if (2 + origin_len + field_value_len > NGHTTP2_MAX_PAYLOADLEN) {
+    return NGHTTP2_ERR_INVALID_ARGUMENT;
+  }
+
+  if (stream_id == 0) {
+    if (origin_len == 0) {
+      return NGHTTP2_ERR_INVALID_ARGUMENT;
+    }
+  } else if (origin_len != 0) {
+    return NGHTTP2_ERR_INVALID_ARGUMENT;
+  }
+
+  buf = nghttp2_mem_malloc(mem, origin_len + field_value_len + 2);
+  if (buf == NULL) {
+    return NGHTTP2_ERR_NOMEM;
+  }
+
+  p = buf;
+
+  origin_copy = p;
+  if (origin_len) {
+    p = nghttp2_cpymem(p, origin, origin_len);
+  }
+  *p++ = '\0';
+
+  field_value_copy = p;
+  if (field_value_len) {
+    p = nghttp2_cpymem(p, field_value, field_value_len);
+  }
+  *p++ = '\0';
+
+  item = nghttp2_mem_malloc(mem, sizeof(nghttp2_outbound_item));
+  if (item == NULL) {
+    rv = NGHTTP2_ERR_NOMEM;
+    goto fail_item_malloc;
+  }
+
+  nghttp2_outbound_item_init(item);
+
+  item->aux_data.ext.builtin = 1;
+
+  altsvc = &item->ext_frame_payload.altsvc;
+
+  frame = &item->frame;
+  frame->ext.payload = altsvc;
+
+  nghttp2_frame_altsvc_init(&frame->ext, stream_id, origin_copy, origin_len,
+                            field_value_copy, field_value_len);
+
+  rv = nghttp2_session_add_item(session, item);
+  if (rv != 0) {
+    nghttp2_frame_altsvc_free(&frame->ext, mem);
+    nghttp2_mem_free(mem, item);
+
+    return rv;
+  }
+
+  return 0;
+
+fail_item_malloc:
+  free(buf);
+
+  return rv;
+}
+
 static uint8_t set_request_flags(const nghttp2_priority_spec *pri_spec,
                                  const nghttp2_data_provider *data_prd) {
   uint8_t flags = NGHTTP2_FLAG_NONE;

python/cnghttp2.pxd

@@ -314,9 +314,10 @@ cdef extern from 'nghttp2/nghttp2.h':
     int nghttp2_hd_inflate_change_table_size(nghttp2_hd_inflater *inflater,
                                              size_t hd_table_bufsize_max)
 
-    ssize_t nghttp2_hd_inflate_hd(nghttp2_hd_inflater *inflater,
-                                  nghttp2_nv *nv_out, int *inflate_flags,
-                                  uint8_t *input, size_t inlen, int in_final)
+    ssize_t nghttp2_hd_inflate_hd2(nghttp2_hd_inflater *inflater,
+                                   nghttp2_nv *nv_out, int *inflate_flags,
+                                   const uint8_t *input, size_t inlen,
+                                   int in_final)
 
     int nghttp2_hd_inflate_end_headers(nghttp2_hd_inflater *inflater)
 

python/nghttp2.pyx

@@ -191,9 +191,9 @@ cdef class HDInflater:
         res = []
         while True:
             inflate_flags = 0
-            rv = cnghttp2.nghttp2_hd_inflate_hd(self._inflater, &nv,
-                                                &inflate_flags,
-                                                buf, buflen, 1)
+            rv = cnghttp2.nghttp2_hd_inflate_hd2(self._inflater, &nv,
+                                                 &inflate_flags,
+                                                 buf, buflen, 1)
             if rv < 0:
                 raise Exception(_strerror(rv))
             buf += rv

releasechk

@@ -1,5 +1,6 @@
 #!/bin/sh -e
 
+autoreconf -i
 git submodule update --init
 ./configure --with-mruby --with-neverbleed --enable-asio-lib
-make -j3 distcheck DISTCHECK_CONFIGURE_FLAGS="--with-mruby --with-neverbleed --enable-asio-lib --enable-werror"
+make -j8 distcheck DISTCHECK_CONFIGURE_FLAGS="--with-mruby --with-neverbleed --enable-asio-lib --enable-werror"

src/CMakeLists.txt

@@ -39,7 +39,7 @@ link_libraries(
 if(ENABLE_APP)
   set(HELPER_OBJECTS
     util.cc
-    http2.cc timegm.c app_helper.cc nghttp2_gzip.c
+    http2.cc timegm.c app_helper.cc nghttp2_gzip.c cache_digest.cc
   )
 
   # nghttp client
@@ -100,6 +100,7 @@ if(ENABLE_APP)
     shrpx_worker.cc
     shrpx_log_config.cc
     shrpx_connect_blocker.cc
+    shrpx_live_check.cc
     shrpx_downstream_connection_pool.cc
     shrpx_rate_limit.cc
     shrpx_connection.cc
@@ -108,6 +109,9 @@ if(ENABLE_APP)
     shrpx_worker_process.cc
     shrpx_signal.cc
     shrpx_router.cc
+    shrpx_api_downstream_connection.cc
+    shrpx_health_monitor_downstream_connection.cc
+    cache_digest.cc
   )
   if(HAVE_SPDYLAY)
     list(APPEND NGHTTPX_SRCS
@@ -147,6 +151,7 @@ if(ENABLE_APP)
       shrpx_config_test.cc
       shrpx_worker_test.cc
       shrpx_http_test.cc
+      shrpx_router_test.cc
       http2_test.cc
       util_test.cc
       nghttp2_gzip_test.c
@@ -155,6 +160,7 @@ if(ENABLE_APP)
       memchunk_test.cc
       template_test.cc
       base64_test.cc
+      cache_digest_test.cc
     )
     add_executable(nghttpx-unittest EXCLUDE_FROM_ALL
       ${NGHTTPX_UNITTEST_SOURCES}
@@ -162,7 +168,7 @@ if(ENABLE_APP)
     )
     target_include_directories(nghttpx-unittest PRIVATE ${CUNIT_INCLUDE_DIRS})
     target_compile_definitions(nghttpx-unittest
-      PRIVATE "-DNGHTTP2_TESTS_DIR=\"${CMAKE_SOURCE_DIR}/tests\""
+      PRIVATE "-DNGHTTP2_SRC_DIR=\"${CMAKE_SOURCE_DIR}/src\""
     )
     target_link_libraries(nghttpx-unittest nghttpx_static ${CUNIT_LIBRARIES})
     if(HAVE_MRUBY)

src/HttpServer.cc

@@ -62,6 +62,7 @@
 #include "util.h"
 #include "ssl.h"
 #include "template.h"
+#include "cache_digest.h"
 
 #ifndef O_BINARY
 #define O_BINARY (0)
@@ -825,10 +826,22 @@ int Http2Handler::on_write() { return write_(*this); }
 int Http2Handler::connection_made() {
   int r;
 
-  r = nghttp2_session_server_new(&session_, sessions_->get_callbacks(), this);
+  nghttp2_option *opt;
+  r = nghttp2_option_new(&opt);
 
   if (r != 0) {
-    return r;
+    return -1;
+  }
+
+  nghttp2_option_set_user_recv_extension_type(opt, NGHTTP2_DRAFT_CACHE_DIGEST);
+
+  r = nghttp2_session_server_new2(&session_, sessions_->get_callbacks(), this,
+                                  opt);
+
+  nghttp2_option_del(opt);
+
+  if (r != 0) {
+    return -1;
   }
 
   auto config = sessions_->get_config();
@@ -852,16 +865,15 @@ int Http2Handler::connection_made() {
 
   r = nghttp2_submit_settings(session_, NGHTTP2_FLAG_NONE, entry.data(), niv);
   if (r != 0) {
-    return r;
+    return -1;
   }
 
   if (config->connection_window_bits != -1) {
-    r = nghttp2_submit_window_update(
+    r = nghttp2_session_set_local_window_size(
         session_, NGHTTP2_FLAG_NONE, 0,
-        (1 << config->connection_window_bits) - 1 -
-            NGHTTP2_INITIAL_CONNECTION_WINDOW_SIZE);
+        (1 << config->connection_window_bits) - 1);
     if (r != 0) {
-      return r;
+      return -1;
     }
   }
 
@@ -1065,6 +1077,39 @@ void Http2Handler::terminate_session(uint32_t error_code) {
   nghttp2_session_terminate_session(session_, error_code);
 }
 
+std::vector<uint8_t> &Http2Handler::get_extbuf() { return extbuf_; }
+
+void Http2Handler::set_cache_digest(const StringRef &authority,
+                                    std::unique_ptr<CacheDigest> cache_digest) {
+  origin_cache_digest_[authority.str()] = std::move(cache_digest);
+}
+
+bool Http2Handler::cache_digest_includes(const StringRef &authority,
+                                         const StringRef &uri) const {
+  uint64_t key;
+  int rv;
+
+  auto it = origin_cache_digest_.find(authority.str());
+
+  if (it == std::end(origin_cache_digest_)) {
+    return false;
+  }
+
+  auto &cache_digest = (*it).second;
+
+  auto key_nbits = cache_digest->logn + cache_digest->logp;
+
+  rv = cache_digest_hash(key, key_nbits, uri);
+
+  if (rv != 0) {
+    return false;
+  }
+
+  const auto &keys = cache_digest->keys;
+
+  return std::binary_search(std::begin(keys), std::end(keys), key);
+}
+
 ssize_t file_read_callback(nghttp2_session *session, int32_t stream_id,
                            uint8_t *buf, size_t length, uint32_t *data_flags,
                            nghttp2_data_source *source, void *user_data) {
@@ -1121,8 +1166,12 @@ void prepare_status_response(Stream *stream, Http2Handler *hd, int status) {
   data_prd.read_callback = file_read_callback;
 
   HeaderRefs headers;
+  headers.reserve(2);
   headers.emplace_back(StringRef::from_lit("content-type"),
                        StringRef::from_lit("text/html; charset=UTF-8"));
+  headers.emplace_back(
+      StringRef::from_lit("content-length"),
+      util::make_string_ref_uint(stream->balloc, file_ent->length));
   hd->submit_response(StringRef{status_page->status}, stream->stream_id,
                       headers, &data_prd);
 }
@@ -1285,7 +1334,7 @@ void prepare_response(Stream *stream, Http2Handler *hd,
     p = std::copy(std::begin(htdocs), std::end(htdocs), p);
     p = std::copy(std::begin(path), std::end(path), p);
     if (trailing_slash) {
-      p = std::copy(std::begin(DEFAULT_HTML), std::end(DEFAULT_HTML), p);
+      std::copy(std::begin(DEFAULT_HTML), std::end(DEFAULT_HTML), p);
     }
   }
 
@@ -1539,6 +1588,20 @@ int hd_on_frame_recv_callback(nghttp2_session *session,
       hd->remove_settings_timer();
     }
     break;
+  case NGHTTP2_DRAFT_CACHE_DIGEST: {
+    auto stream = hd->get_stream(frame->hd.stream_id);
+    if (!stream) {
+      return 0;
+    }
+
+    auto &header = stream->header;
+
+    hd->set_cache_digest(header.authority,
+                         std::unique_ptr<CacheDigest>(
+                             static_cast<CacheDigest *>(frame->ext.payload)));
+
+    break;
+  }
   default:
     break;
   }
@@ -1546,6 +1609,36 @@ int hd_on_frame_recv_callback(nghttp2_session *session,
 }
 } // namespace
 
+namespace {
+int before_frame_send_callback(nghttp2_session *session,
+                               const nghttp2_frame *frame, void *user_data) {
+  auto hd = static_cast<Http2Handler *>(user_data);
+
+  if (frame->hd.type != NGHTTP2_PUSH_PROMISE) {
+    return 0;
+  }
+
+  auto promised_stream = hd->get_stream(frame->push_promise.promised_stream_id);
+
+  if (promised_stream == nullptr) {
+    return 0;
+  }
+
+  auto &header = promised_stream->header;
+
+  auto uri = header.scheme.str();
+  uri += "://";
+  uri += header.authority;
+  uri += header.path;
+
+  if (hd->cache_digest_includes(header.authority, StringRef{uri})) {
+    return NGHTTP2_ERR_CANCEL;
+  }
+
+  return 0;
+}
+} // namespace
+
 namespace {
 int hd_on_frame_send_callback(nghttp2_session *session,
                               const nghttp2_frame *frame, void *user_data) {
@@ -1718,6 +1811,46 @@ int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
 }
 } // namespace
 
+namespace {
+int on_extension_chunk_recv_callback(nghttp2_session *session,
+                                     const nghttp2_frame_hd *frame_hd,
+                                     const uint8_t *data, size_t len,
+                                     void *user_data) {
+  auto hd = static_cast<Http2Handler *>(user_data);
+
+  auto &buf = hd->get_extbuf();
+  buf.insert(std::end(buf), data, data + len);
+
+  return 0;
+}
+} // namespace
+
+namespace {
+int unpack_extension_callback(nghttp2_session *session, void **payload,
+                              const nghttp2_frame_hd *frame_hd,
+                              void *user_data) {
+  int rv;
+  auto hd = static_cast<Http2Handler *>(user_data);
+
+  auto cache_digest = make_unique<CacheDigest>();
+
+  auto &buf = hd->get_extbuf();
+
+  rv = cache_digest_decode(cache_digest->keys, cache_digest->logn,
+                           cache_digest->logp, buf.data(), buf.size());
+
+  buf.clear();
+
+  if (rv != 0) {
+    return NGHTTP2_ERR_CANCEL;
+  }
+
+  *payload = cache_digest.release();
+
+  return 0;
+}
+} // namespace
+
 namespace {
 void fill_callback(nghttp2_session_callbacks *callbacks, const Config *config) {
   nghttp2_session_callbacks_set_on_stream_close_callback(
@@ -1753,6 +1886,15 @@ void fill_callback(nghttp2_session_callbacks *callbacks, const Config *config) {
     nghttp2_session_callbacks_set_select_padding_callback(
         callbacks, select_padding_callback);
   }
+
+  nghttp2_session_callbacks_set_unpack_extension_callback(
+      callbacks, unpack_extension_callback);
+
+  nghttp2_session_callbacks_set_on_extension_chunk_recv_callback(
+      callbacks, on_extension_chunk_recv_callback);
+
+  nghttp2_session_callbacks_set_before_frame_send_callback(
+      callbacks, before_frame_send_callback);
 }
 } // namespace
 
@@ -1793,6 +1935,16 @@ void run_worker(Worker *worker) {
 }
 } // namespace
 
+namespace {
+int get_ev_loop_flags() {
+  if (ev_supported_backends() & ~ev_recommended_backends() & EVBACKEND_KQUEUE) {
+    return ev_recommended_backends() | EVBACKEND_KQUEUE;
+  }
+
+  return 0;
+}
+} // namespace
+
 class AcceptHandler {
 public:
   AcceptHandler(HttpServer *sv, Sessions *sessions, const Config *config)
@@ -1805,7 +1957,7 @@ public:
         std::cerr << "spawning thread #" << i << std::endl;
       }
       auto worker = make_unique<Worker>();
-      auto loop = ev_loop_new(0);
+      auto loop = ev_loop_new(get_ev_loop_flags());
       worker->sessions =
           make_unique<Sessions>(sv, loop, config_, sessions_->get_ssl_ctx());
       ev_async_init(&worker->w, worker_acceptcb);

src/HttpServer.h

@@ -153,6 +153,12 @@ struct Stream {
 
 class Sessions;
 
+struct CacheDigest {
+  std::vector<uint64_t> keys;
+  uint32_t logn;
+  uint32_t logp;
+};
+
 class Http2Handler {
 public:
   Http2Handler(Sessions *sessions, int fd, SSL *ssl, int64_t session_id);
@@ -206,6 +212,15 @@ public:
 
   WriteBuf *get_wb();
 
+  std::vector<uint8_t> &get_extbuf();
+
+  // Sets given cache digest.  Overwrites existing one if any.
+  void set_cache_digest(const StringRef &origin,
+                        std::unique_ptr<CacheDigest> cache_digest);
+  // Returns true if |uri| is included in cache digest.
+  bool cache_digest_includes(const StringRef &origin,
+                             const StringRef &uri) const;
+
 private:
   ev_io wev_;
   ev_io rev_;
@@ -213,6 +228,10 @@ private:
   std::map<int32_t, std::unique_ptr<Stream>> id2stream_;
   WriteBuf wb_;
   std::function<int(Http2Handler &)> read_, write_;
+  // Received cache digest hash keys per origin
+  std::map<std::string, std::unique_ptr<CacheDigest>> origin_cache_digest_;
+  // Buffer for extension frame payload
+  std::vector<uint8_t> extbuf_;
   int64_t session_id_;
   nghttp2_session *session_;
   Sessions *sessions_;

src/Makefile.am

@@ -22,7 +22,10 @@
 # WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 SUBDIRS = includes
 
-EXTRA_DIST = CMakeLists.txt
+EXTRA_DIST = \
+	CMakeLists.txt \
+	test.example.com.pem \
+	test.nghttp2.org.pem
 
 bin_PROGRAMS =
 check_PROGRAMS =
@@ -61,10 +64,10 @@ if ENABLE_APP
 bin_PROGRAMS += nghttp nghttpd nghttpx
 
 HELPER_OBJECTS = util.cc \
-	http2.cc timegm.c app_helper.cc nghttp2_gzip.c
+	http2.cc timegm.c app_helper.cc nghttp2_gzip.c cache_digest.cc
 HELPER_HFILES = util.h \
 	http2.h timegm.h app_helper.h nghttp2_config.h \
-	nghttp2_gzip.h network.h
+	nghttp2_gzip.h network.h cache_digest.h
 
 HTML_PARSER_OBJECTS =
 HTML_PARSER_HFILES = HtmlParser.h
@@ -120,6 +123,7 @@ NGHTTPX_SRCS = \
 	shrpx_worker.cc shrpx_worker.h \
 	shrpx_log_config.cc shrpx_log_config.h \
 	shrpx_connect_blocker.cc shrpx_connect_blocker.h \
+	shrpx_live_check.cc shrpx_live_check.h \
 	shrpx_downstream_connection_pool.cc shrpx_downstream_connection_pool.h \
 	shrpx_rate_limit.cc shrpx_rate_limit.h \
 	shrpx_connection.cc shrpx_connection.h \
@@ -131,7 +135,11 @@ NGHTTPX_SRCS = \
 	shrpx_process.h \
 	shrpx_signal.cc shrpx_signal.h \
 	shrpx_router.cc shrpx_router.h \
-	buffer.h memchunk.h template.h allocator.h
+	shrpx_api_downstream_connection.cc shrpx_api_downstream_connection.h \
+	shrpx_health_monitor_downstream_connection.cc \
+	shrpx_health_monitor_downstream_connection.h \
+	buffer.h memchunk.h template.h allocator.h \
+	cache_digest.cc cache_digest.h
 
 if HAVE_SPDYLAY
 NGHTTPX_SRCS += shrpx_spdy_upstream.cc shrpx_spdy_upstream.h
@@ -173,6 +181,7 @@ nghttpx_unittest_SOURCES = shrpx-unittest.cc \
 	shrpx_config_test.cc shrpx_config_test.h \
 	shrpx_worker_test.cc shrpx_worker_test.h \
 	shrpx_http_test.cc shrpx_http_test.h \
+	shrpx_router_test.cc shrpx_router_test.h \
 	http2_test.cc http2_test.h \
 	util_test.cc util_test.h \
 	nghttp2_gzip_test.c nghttp2_gzip_test.h \
@@ -180,9 +189,10 @@ nghttpx_unittest_SOURCES = shrpx-unittest.cc \
 	buffer_test.cc buffer_test.h \
 	memchunk_test.cc memchunk_test.h \
 	template_test.cc template_test.h \
-	base64_test.cc base64_test.h
+	base64_test.cc base64_test.h \
+	cache_digest_test.cc cache_digest_test.h
 nghttpx_unittest_CPPFLAGS = ${AM_CPPFLAGS} \
-	-DNGHTTP2_TESTS_DIR=\"$(top_srcdir)/tests\"
+	-DNGHTTP2_SRC_DIR=\"$(top_srcdir)/src\"
 nghttpx_unittest_LDADD = libnghttpx.a ${LDADD} @CUNIT_LIBS@ @TESTLDADD@
 
 if HAVE_MRUBY
@@ -225,6 +235,7 @@ lib_LTLIBRARIES = libnghttp2_asio.la
 libnghttp2_asio_la_SOURCES = \
 	util.cc util.h http2.cc http2.h \
 	ssl.cc ssl.h \
+	ssl_compat.h \
 	timegm.c timegm.h \
 	asio_common.cc asio_common.h \
 	asio_io_service_pool.cc asio_io_service_pool.h \

src/allocator.h

@@ -54,20 +54,45 @@ struct BlockAllocator {
         block_size(block_size),
         isolation_threshold(std::min(block_size, isolation_threshold)) {}
 
-  ~BlockAllocator() {
+  ~BlockAllocator() { reset(); }
+
+  BlockAllocator(BlockAllocator &&other) noexcept
+      : retain(other.retain),
+        head(other.head),
+        block_size(other.block_size),
+        isolation_threshold(other.isolation_threshold) {
+    other.retain = nullptr;
+    other.head = nullptr;
+  }
+
+  BlockAllocator &operator=(BlockAllocator &&other) noexcept {
+    reset();
+
+    retain = other.retain;
+    head = other.head;
+    block_size = other.block_size;
+    isolation_threshold = other.isolation_threshold;
+
+    other.retain = nullptr;
+    other.head = nullptr;
+
+    return *this;
+  }
+
+  BlockAllocator(const BlockAllocator &) = delete;
+  BlockAllocator &operator=(const BlockAllocator &) = delete;
+
+  void reset() {
     for (auto mb = retain; mb;) {
       auto next = mb->next;
       delete[] reinterpret_cast<uint8_t *>(mb);
       mb = next;
     }
+
+    retain = nullptr;
+    head = nullptr;
   }
 
-  BlockAllocator(BlockAllocator &&) = default;
-  BlockAllocator &operator=(BlockAllocator &&) = default;
-
-  BlockAllocator(const BlockAllocator &) = delete;
-  BlockAllocator &operator=(const BlockAllocator &) = delete;
-
   MemBlock *alloc_mem_block(size_t size) {
     auto block = new uint8_t[sizeof(MemBlock) + size];
     auto mb = reinterpret_cast<MemBlock *>(block);

src/app_helper.cc

@@ -104,6 +104,10 @@ std::string strframetype(uint8_t type) {
     return "GOAWAY";
   case NGHTTP2_WINDOW_UPDATE:
     return "WINDOW_UPDATE";
+  case NGHTTP2_ALTSVC:
+    return "ALTSVC";
+  case NGHTTP2_DRAFT_CACHE_DIGEST:
+    return "CACHE_DIGSET";
   }
 
   std::string s = "extension(0x";
@@ -339,6 +343,14 @@ void print_frame(print_type ptype, const nghttp2_frame *frame) {
     fprintf(outfile, "(window_size_increment=%d)\n",
             frame->window_update.window_size_increment);
     break;
+  case NGHTTP2_ALTSVC: {
+    auto altsvc = static_cast<nghttp2_ext_altsvc *>(frame->ext.payload);
+    print_frame_attr_indent();
+    fprintf(outfile, "(origin=[%.*s], altsvc_field_value=[%.*s])\n",
+            static_cast<int>(altsvc->origin_len), altsvc->origin,
+            static_cast<int>(altsvc->field_value_len), altsvc->field_value);
+    break;
+  }
   default:
     break;
   }

src/app_helper.h

@@ -41,6 +41,11 @@
 
 namespace nghttp2 {
 
+enum nghttp2_draft_frame_type {
+  // draft-kazuho-h2-cache-digest-01
+  NGHTTP2_DRAFT_CACHE_DIGEST = 0xf1
+};
+
 int verbose_on_header_callback(nghttp2_session *session,
                                const nghttp2_frame *frame, const uint8_t *name,
                                size_t namelen, const uint8_t *value,

src/asio_client_session_impl.cc

@@ -373,9 +373,8 @@ bool session_impl::setup_session() {
        {NGHTTP2_SETTINGS_INITIAL_WINDOW_SIZE, window_size}}};
   nghttp2_submit_settings(session_, NGHTTP2_FLAG_NONE, iv.data(), iv.size());
   // increase connection window size up to window_size
-  nghttp2_submit_window_update(session_, NGHTTP2_FLAG_NONE, 0,
-                               window_size -
-                                   NGHTTP2_INITIAL_CONNECTION_WINDOW_SIZE);
+  nghttp2_session_set_local_window_size(session_, NGHTTP2_FLAG_NONE, 0,
+                                        window_size);
   return true;
 }
 

src/ca-config.json

@@ -0,0 +1,17 @@
+{
+    "signing": {
+	"default": {
+	    "expiry": "87600h"
+	},
+	"profiles": {
+	    "server": {
+		"expiry": "87600h",
+		"usages": [
+		    "signing",
+		    "key encipherment",
+		    "server auth"
+		]
+	    }
+	}
+    }
+}

src/ca.nghttp2.org-key.pem

@@ -0,0 +1,27 @@
+-----BEGIN RSA PRIVATE KEY-----
+MIIEpQIBAAKCAQEA1kVkF8QSUwW/HV9EFRPSMoiOVmYwB8vqKDtT0d6MFiKAM8/Y
+JFUq2uKlUydgT4IPE7PATvVcIj3GtL9XzPhscqYO/S0Y7scyTE2VAPmtz+StPWf2
+wZ1IQR09HrnDTc44KvYGZpefBZkD9UjbmJ9a1ZmJjJiMr3hTnKE/sxZ2+dMsnMZX
+N822cfaHyTN+T0+Tyw5vBBboCDsZzxmf+9FFIDJNs3NL34cR8EZRhpfaegapH8bt
+OJ+D+RZ2kg7E/YYkGcS6NodvTjSUFCFHpWjHCfTFhn/owBIAooCdWorh6dc8Q72l
+AodwNLXS8uuPgPqM5s4Cz57m7Zgs4OilNmIdawIDAQABAoIBAQCwqLtygLye6KD+
+RXorapEmCsJX5553/x6Klwdvg+25ni5XCWjp47IWj0DBQzi7tL5bfxrxvod8z7QR
+d6SbIMLA77px8Ima7G7CzEAqcrBkM+TFOP8P+G4HCWVH/N5SOtDCUt9KHH4Grna9
+95jdx5yreRAX8/oh/bHp9GRBcicbpwYMVWOnjTE2seEUYQOpdpYdP4bOPUvAju0l
+mwmy2/dDGmbibktN3sdHEhDodKu+Znv7nFZo0jzhlyoXse653WcvaQeZZYuojvSe
+Sr92DvPp7UaYrb4KvT7ujXiPavSV2m/4EmGtyqevUf2dZ6sfMXZjmXsjWz9txhWp
+4BgbHyHRAoGBAPqyuNj2CDD3FE7N3Hxyba8d+ZtsVUNawjq2gwOvT9NLsMstOGyH
+OCc1v4W6Sq4w1wo4nIJyY8kNZwtReaTHOPZlDgBhVvk/x8eLBu+QTMRyocRt1LoD
+8HyKxWSAnYTtCh/GUEQ37amIqvOJ5GNL+25WDzevLa5kMYWG743uxEupAoGBANrN
+c/fVxepvP0GISlLpL3aZCFGAjMrq3xUYcf/w4wPoMq6AdpIPeRVBmJ1/Uqw1FkV8
+NRKJNPE2YcMuv8iMeQlacoPd34KT9ob80EYVlMwAkeC0NK+FfiM/UteR0wB49gmi
+ugX9YlJytOP9aUgPvEGT6l+XtgGC44W1TQWe62zzAoGBAKZenNU+0UjNb6isbToZ
+Jjkkh1Vhm2PLg0I7hM6ZNTxf6r+rDtrXEajTvnocmxrmRo796r+W8immv09/jl6P
+53l8rsIJ1xIqBYai+MNa29cyy6/zw0x++MVtwnlj8SUZubJEhVgAVbRAglKEnBBZ
+iE48xnSJyKMG0uZuGePzJEmhAoGBAIOHJcNBumum3DuklikpC+MbMyjrQbdpYRjp
+TP4x7AWZO34ysxQyQPNKL1feBfCHKRA0DiNKX4zwx+vw2lDQQKIiwNwMMCPqljOn
+HfxDVOMdJJQTP+iTMrQ1iLMVceXC0QQR0glvu/8b/SlgWD19WAmDxUwZgst9xw/F
+YLuUQKmJAoGAREeTugd4hc0U/YV/BQQjSCLhl11EtVry/oQMHj8KZpIJhP7tj8lw
+hSE0+z04oMhiTeq55PYKQkTo5l6V4PW0zfpEwlKEEm0erab1G9Ddh7us47XFcKLl
+Rmk192EVZ0lQuzftsYv7dzRLiAR7yDFXwD1ELIK/uPkwBtu7wtHlq+M=
+-----END RSA PRIVATE KEY-----

src/ca.nghttp2.org.csr

@@ -0,0 +1,17 @@
+-----BEGIN CERTIFICATE REQUEST-----
+MIICwjCCAaoCAQAwXjELMAkGA1UEBhMCQVUxEzARBgNVBAgTClNvbWUtU3RhdGUx
+ITAfBgNVBAoTGEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDEXMBUGA1UEAxMOY2Eu
+bmdodHRwMi5vcmcwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDWRWQX
+xBJTBb8dX0QVE9IyiI5WZjAHy+ooO1PR3owWIoAzz9gkVSra4qVTJ2BPgg8Ts8BO
+9VwiPca0v1fM+Gxypg79LRjuxzJMTZUA+a3P5K09Z/bBnUhBHT0eucNNzjgq9gZm
+l58FmQP1SNuYn1rVmYmMmIyveFOcoT+zFnb50yycxlc3zbZx9ofJM35PT5PLDm8E
+FugIOxnPGZ/70UUgMk2zc0vfhxHwRlGGl9p6Bqkfxu04n4P5FnaSDsT9hiQZxLo2
+h29ONJQUIUelaMcJ9MWGf+jAEgCigJ1aiuHp1zxDvaUCh3A0tdLy64+A+ozmzgLP
+nubtmCzg6KU2Yh1rAgMBAAGgHzAdBgkqhkiG9w0BCQ4xEDAOMAwGA1UdEwQFMAMB
+Af8wDQYJKoZIhvcNAQELBQADggEBACI5v8GbOXKv38h9/tuGEwJ9uxpYEljgGt8h
+QL5lwfEifh/7A8b39b9JEzWk5hnMRCOb8J6Jc3/6nmVgtKkQ+Mceupqpwsp1gT/v
+uUoAkJE03Iuja9zLhHmy74oZ7LWOQrZ1T7Z0eGQ+5u+LBZiPKnKxmkLCQoUPTbc4
+NQ9BbKhr8OaoJ4DDvJnszcL7to6kih7SkdoNZsq4zB0/ai/cPhvoVgkYfbLH2++D
+Tcs7TqU2L7gKzqXUtHeAKM2y81ewL7QTrcYzgiW86s3NmquxZG5pq0mjD+P4BYLc
+MOdnCxKbBuE/1R29pa6+JKgc46jOa2yRgv5+8rXkkpu53Ke3FGc=
+-----END CERTIFICATE REQUEST-----

src/ca.nghttp2.org.csr.json

@@ -0,0 +1,17 @@
+{
+    "CN": "ca.nghttp2.org",
+    "key": {
+	"algo": "rsa",
+	"size": 2048
+    },
+    "ca": {
+	"expiry": "87600h"
+    },
+    "names": [
+	{
+	    "C": "AU",
+	    "ST": "Some-State",
+	    "O": "Internet Widgits Pty Ltd"
+	}
+    ]
+}

src/ca.nghttp2.org.pem

@@ -0,0 +1,22 @@
+-----BEGIN CERTIFICATE-----
+MIIDrTCCApWgAwIBAgIUe4dvx8haIjsT3ZpNCMrl62Xk6E0wDQYJKoZIhvcNAQEL
+BQAwXjELMAkGA1UEBhMCQVUxEzARBgNVBAgTClNvbWUtU3RhdGUxITAfBgNVBAoT
+GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDEXMBUGA1UEAxMOY2EubmdodHRwMi5v
+cmcwHhcNMTYwNjI1MDkzMzAwWhcNMjYwNjIzMDkzMzAwWjBeMQswCQYDVQQGEwJB
+VTETMBEGA1UECBMKU29tZS1TdGF0ZTEhMB8GA1UEChMYSW50ZXJuZXQgV2lkZ2l0
+cyBQdHkgTHRkMRcwFQYDVQQDEw5jYS5uZ2h0dHAyLm9yZzCCASIwDQYJKoZIhvcN
+AQEBBQADggEPADCCAQoCggEBANZFZBfEElMFvx1fRBUT0jKIjlZmMAfL6ig7U9He
+jBYigDPP2CRVKtripVMnYE+CDxOzwE71XCI9xrS/V8z4bHKmDv0tGO7HMkxNlQD5
+rc/krT1n9sGdSEEdPR65w03OOCr2BmaXnwWZA/VI25ifWtWZiYyYjK94U5yhP7MW
+dvnTLJzGVzfNtnH2h8kzfk9Pk8sObwQW6Ag7Gc8Zn/vRRSAyTbNzS9+HEfBGUYaX
+2noGqR/G7Tifg/kWdpIOxP2GJBnEujaHb040lBQhR6Voxwn0xYZ/6MASAKKAnVqK
+4enXPEO9pQKHcDS10vLrj4D6jObOAs+e5u2YLODopTZiHWsCAwEAAaNjMGEwDgYD
+VR0PAQH/BAQDAgEGMA8GA1UdEwEB/wQFMAMBAf8wHQYDVR0OBBYEFNA5xVR1Zcax
+RJL9VC6pzuLmvduGMB8GA1UdIwQYMBaAFNA5xVR1ZcaxRJL9VC6pzuLmvduGMA0G
+CSqGSIb3DQEBCwUAA4IBAQCmdVfn/hUyEdvkKG7svg5d8o6BENOj8695KtWmzJjK
+zxH8J5Vy3mn89XrHQ+BOYXCDPyhs0aDS8aq3Z+HY0n9z1oAicyGzlVwZQQNX3YId
+Y2vcf7qu/2ATm/1S+mebE1/EXMUlWISKKUYXjggCwFgjDhH87Ai+A8MKScVdmqgL
+Hf+fRSzH3ToW7BCXlRl5bPAq2g+v1ALYc8wU9cT1MYm4dqAXh870LGFyUpaSWmFr
+TtX1DXBTgLp62syNlDthAvGigYFDtCa4cDM2vdTD9wpec2V9EKpfVqiRDDuYjUVX
+UXl27MvkNWnEBKCIoNv5abWXpZVG2zQdEMmUOkVuAXUC
+-----END CERTIFICATE-----

src/cache_digest.cc

@@ -0,0 +1,395 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2016 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "cache_digest.h"
+
+#include <cassert>
+#include <array>
+#include <limits>
+
+#include <openssl/evp.h>
+
+namespace nghttp2 {
+
+namespace {
+// Truncates |md| to |nbits| bits counting from MSB.  |nbits| is
+// guaranteed to be less than or equal to 62.
+uint64_t truncate_hash(const uint8_t *md, uint32_t nbits) {
+  uint64_t v;
+
+  v = (static_cast<uint64_t>(md[0]) << 56) +
+      (static_cast<uint64_t>(md[1]) << 48) +
+      (static_cast<uint64_t>(md[2]) << 40) +
+      (static_cast<uint64_t>(md[3]) << 32) +
+      (static_cast<uint64_t>(md[4]) << 24) +
+      (static_cast<uint64_t>(md[5]) << 16) +
+      (static_cast<uint64_t>(md[6]) << 8) + static_cast<uint64_t>(md[31]);
+
+  v >>= 64 - nbits;
+
+  return v;
+}
+} // namespace
+
+namespace {
+int compute_hash_values(std::vector<uint64_t> &hash_values,
+                        const std::vector<std::string> &uris, uint32_t nbits) {
+  int rv;
+
+  if (nbits > 62) {
+    return -1;
+  }
+
+  auto ctx = EVP_MD_CTX_create();
+
+  hash_values.resize(uris.size());
+
+  std::array<uint8_t, 32> md;
+
+  auto p = std::begin(hash_values);
+  for (auto &u : uris) {
+    rv = EVP_DigestInit_ex(ctx, EVP_sha256(), nullptr);
+    if (rv != 1) {
+      return -1;
+    }
+
+    rv = EVP_DigestUpdate(ctx, u.c_str(), u.size());
+    if (rv != 1) {
+      return -1;
+    }
+
+    unsigned int len = md.size();
+
+    rv = EVP_DigestFinal_ex(ctx, md.data(), &len);
+    if (rv != 1) {
+      return -1;
+    }
+
+    assert(len == 32);
+
+    *p++ = truncate_hash(md.data(), nbits);
+  }
+
+  EVP_MD_CTX_destroy(ctx);
+
+  return 0;
+}
+} // namespace
+
+namespace {
+std::pair<uint8_t *, size_t> append_uint32(uint8_t *p, size_t b, uint32_t v,
+                                           size_t nbits) {
+  v &= (1 << nbits) - 1;
+
+  if (8 > b + nbits) {
+    *p |= (v << (8 - b - nbits));
+    return {p, b + nbits};
+  }
+
+  if (8 == b + nbits) {
+    *p++ |= v;
+    return {p, 0};
+  }
+
+  auto h = 8 - b;
+  auto left = nbits - h;
+
+  *p++ |= (v >> left);
+  b = 0;
+
+  for (; left >= 8; left -= 8) {
+    *p++ = (v >> (left - 8)) & 0xff;
+  }
+
+  if (left > 0) {
+    *p = (v & ((1 << left) - 1)) << (8 - left);
+  }
+
+  return {p, left};
+}
+} // namespace
+
+namespace {
+std::pair<uint8_t *, size_t> append_0bit(uint8_t *p, size_t b, size_t nbits) {
+  if (8 > b + nbits) {
+    return {p, b + nbits};
+  }
+
+  if (8 == b + nbits) {
+    return {++p, 0};
+  }
+
+  nbits -= 8 - b;
+  ++p;
+
+  p += nbits / 8;
+
+  return {p, nbits % 8};
+}
+
+std::pair<uint8_t *, size_t> append_single_1bit(uint8_t *p, size_t b) {
+  if (8 > b + 1) {
+    *p |= (1 << (7 - b));
+    return {p, b + 1};
+  }
+
+  *p++ |= 1;
+
+  return {p, 0};
+}
+} // namespace
+
+ssize_t cache_digest_encode(uint8_t *data, size_t datalen,
+                            const std::vector<std::string> &uris,
+                            uint32_t logp) {
+  uint32_t n = 1;
+  uint32_t logn = 0;
+
+  if (logp > 31) {
+    return -1;
+  }
+
+  for (; n < uris.size(); n *= 2, ++logn)
+    ;
+
+  if (n - uris.size() > uris.size() - n / 2) {
+    --logn;
+  }
+
+  auto maxlen = 2 * n + n * logp;
+  if (maxlen > datalen) {
+    return -1;
+  }
+
+  std::vector<uint64_t> hash_values;
+
+  if (compute_hash_values(hash_values, uris, logn + logp) != 0) {
+    return -1;
+  }
+
+  std::sort(std::begin(hash_values), std::end(hash_values));
+
+  auto last = data;
+
+  size_t b = 0;
+
+  std::fill_n(data, maxlen, 0);
+
+  std::tie(last, b) = append_uint32(last, b, logn, 5);
+  std::tie(last, b) = append_uint32(last, b, logp, 5);
+
+  auto c = std::numeric_limits<uint64_t>::max();
+
+  for (auto v : hash_values) {
+    if (v == c) {
+      continue;
+    }
+    auto d = v - c - 1;
+    auto q = d >> logp;
+    auto r = d & ((1u << logp) - 1);
+
+    std::tie(last, b) = append_0bit(last, b, q);
+    std::tie(last, b) = append_single_1bit(last, b);
+    std::tie(last, b) = append_uint32(last, b, r, logp);
+
+    c = v;
+  }
+
+  if (b != 0) {
+    // we already zero-filled.
+    ++last;
+  }
+
+  return last - data;
+}
+
+int cache_digest_hash(uint64_t &key, size_t nbits, const StringRef &s) {
+  int rv;
+
+  std::array<uint8_t, 32> md;
+
+  auto ctx = EVP_MD_CTX_create();
+
+  rv = EVP_DigestInit_ex(ctx, EVP_sha256(), nullptr);
+  if (rv != 1) {
+    return -1;
+  }
+
+  rv = EVP_DigestUpdate(ctx, s.c_str(), s.size());
+  if (rv != 1) {
+    return -1;
+  }
+
+  unsigned int len = md.size();
+
+  rv = EVP_DigestFinal_ex(ctx, md.data(), &len);
+  if (rv != 1) {
+    return -1;
+  }
+
+  assert(len == 32);
+
+  EVP_MD_CTX_destroy(ctx);
+
+  key = truncate_hash(md.data(), nbits);
+
+  return 0;
+}
+
+namespace {
+std::pair<const uint8_t *, size_t> read_uint32(uint32_t &res, size_t nbits,
+                                               const uint8_t *p, size_t b) {
+  if (b + nbits < 8) {
+    res = (*p >> (8 - b - nbits)) & ((1 << nbits) - 1);
+    return {p, b + nbits};
+  }
+
+  if (b + nbits == 8) {
+    res = *p & ((1 << nbits) - 1);
+    return {++p, 0};
+  }
+
+  res = *p & ((1 << (8 - b)) - 1);
+
+  ++p;
+  nbits -= 8 - b;
+
+  for (; nbits >= 8; nbits -= 8) {
+    res <<= 8;
+    res += *p++;
+  }
+
+  if (nbits) {
+    res <<= nbits;
+    res += *p >> (8 - nbits);
+  }
+
+  return {p, nbits};
+}
+} // namespace
+
+namespace {
+size_t leading_zero(uint8_t c) {
+  for (size_t i = 0; i < 8; ++i) {
+    if (c & (1 << (7 - i))) {
+      return i;
+    }
+  }
+
+  return 8;
+}
+} // namespace
+
+namespace {
+std::pair<const uint8_t *, size_t>
+read_until_1bit(uint32_t &res, const uint8_t *p, size_t b, const uint8_t *end) {
+  uint8_t mask = (1 << (8 - b)) - 1;
+
+  if (*p & mask) {
+    res = leading_zero(*p & mask) - b;
+    b += res + 1;
+    if (b == 8) {
+      return {++p, 0};
+    }
+    return {p, b};
+  }
+
+  res = 8 - b;
+
+  ++p;
+
+  for (; p != end; ++p, res += 8) {
+    if (!*p) {
+      continue;
+    }
+
+    auto nlz = leading_zero(*p);
+
+    res += nlz;
+    b = nlz + 1;
+
+    if (b == 8) {
+      return {++p, 0};
+    }
+    return {p, b};
+  }
+
+  return {end, 0};
+}
+} // namespace
+
+int cache_digest_decode(std::vector<uint64_t> &keys, uint32_t &logn,
+                        uint32_t &logp, const uint8_t *data, size_t datalen) {
+  auto last = data;
+  size_t b = 0;
+
+  auto end = data + datalen;
+
+  if ((end - data) * 8 < 10) {
+    return -1;
+  }
+
+  keys.resize(0);
+
+  logn = 0;
+  logp = 0;
+
+  std::tie(last, b) = read_uint32(logn, 5, last, b);
+  std::tie(last, b) = read_uint32(logp, 5, last, b);
+
+  uint64_t c = std::numeric_limits<uint64_t>::max();
+
+  for (;;) {
+    uint32_t q, r;
+
+    auto may_end = end - last == 1 && b > 0;
+    std::tie(last, b) = read_until_1bit(q, last, b, end);
+
+    if (last == end) {
+      if (may_end) {
+        return 0;
+      }
+
+      return -1;
+    }
+
+    if ((end - last) * 8 < static_cast<intptr_t>(b + logp)) {
+      return -1;
+    }
+
+    std::tie(last, b) = read_uint32(r, logp, last, b);
+
+    auto d = (static_cast<uint64_t>(q) << logp) + r;
+
+    c += d + 1;
+
+    keys.push_back(c);
+
+    if (last == end) {
+      return 0;
+    }
+  }
+}
+
+} // namespace nghttp2

src/cache_digest.h

@@ -0,0 +1,48 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2016 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef CACHE_DIGEST_H
+#define CACHE_DIGEST_H
+
+#include "nghttp2_config.h"
+
+#include <vector>
+#include <string>
+
+#include "template.h"
+
+namespace nghttp2 {
+
+ssize_t cache_digest_encode(uint8_t *data, size_t datalen,
+                            const std::vector<std::string> &uris,
+                            uint32_t logp);
+
+int cache_digest_decode(std::vector<uint64_t> &keys, uint32_t &logn,
+                        uint32_t &logp, const uint8_t *data, size_t datalen);
+
+int cache_digest_hash(uint64_t &key, size_t nbits, const StringRef &s);
+
+} // namespace nghttp2
+
+#endif // CACHE_DIGEST_H

src/cache_digest_test.cc

@@ -0,0 +1,76 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2016 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "cache_digest_test.h"
+
+#include <vector>
+#include <algorithm>
+
+#include <CUnit/CUnit.h>
+
+#include "cache_digest.h"
+#include "template.h"
+
+namespace nghttp2 {
+
+void test_cache_digest_encode_decode(void) {
+  int rv;
+
+  auto uris = std::vector<std::string>{"https://nghttp2.org/foo",
+                                       "https://nghttp2.org/bar",
+                                       "https://nghttp2.org/buzz"};
+
+  auto pbits = 31;
+  std::array<uint8_t, 16_k> cdbuf;
+  auto cdlen = cache_digest_encode(cdbuf.data(), cdbuf.size(), uris, pbits);
+
+  std::vector<uint64_t> keys;
+  uint32_t logn, logp;
+
+  rv = cache_digest_decode(keys, logn, logp, cdbuf.data(), cdlen);
+
+  CU_ASSERT(0 == rv);
+
+  auto query_keys = std::vector<uint64_t>(uris.size());
+  for (size_t i = 0; i < uris.size(); ++i) {
+    auto &uri = uris[i];
+
+    uint64_t key;
+
+    rv = cache_digest_hash(key, logn + logp, StringRef{uri});
+
+    CU_ASSERT(0 == rv);
+
+    query_keys[i] = key;
+  }
+
+  CU_ASSERT(
+      std::binary_search(std::begin(keys), std::end(keys), query_keys[0]));
+  CU_ASSERT(
+      std::binary_search(std::begin(keys), std::end(keys), query_keys[1]));
+  CU_ASSERT(
+      std::binary_search(std::begin(keys), std::end(keys), query_keys[2]));
+}
+
+} // namespace nghttp2

src/cache_digest_test.h

@@ -0,0 +1,38 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2016 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef CACHE_DIGEST_TEST_H
+#define CACHE_DIGEST_TEST_H
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif // HAVE_CONFIG_H
+
+namespace nghttp2 {
+
+void test_cache_digest_encode_decode(void);
+
+} // namespace nghttp2
+
+#endif // CACHE_DIGEST_TEST_H

src/h2load.cc

@@ -101,10 +101,12 @@ Config::Config()
       unix_addr{} {}
 
 Config::~Config() {
-  if (base_uri_unix) {
-    delete addrs;
-  } else {
-    freeaddrinfo(addrs);
+  if (addrs) {
+    if (base_uri_unix) {
+      delete addrs;
+    } else {
+      freeaddrinfo(addrs);
+    }
   }
 
   if (data_fd != -1) {
@@ -267,7 +269,7 @@ bool check_stop_client_request_timeout(Client *client, ev_timer *w) {
   auto nreq = client->req_todo - client->req_started;
 
   if (nreq == 0 ||
-      client->streams.size() >= (size_t)config.max_concurrent_streams) {
+      client->streams.size() >= client->session->max_concurrent_streams()) {
     // no more requests to make, stop timer
     ev_timer_stop(client->worker->loop, w);
     return true;
@@ -316,7 +318,8 @@ void client_request_timeout_cb(struct ev_loop *loop, ev_timer *w, int revents) {
 } // namespace
 
 Client::Client(uint32_t id, Worker *worker, size_t req_todo)
-    : cstat{},
+    : wb(&worker->mcpool),
+      cstat{},
       worker(worker),
       ssl(nullptr),
       next_addr(config.addrs),
@@ -328,7 +331,8 @@ Client::Client(uint32_t id, Worker *worker, size_t req_todo)
       req_done(0),
       id(id),
       fd(-1),
-      new_connection_requested(false) {
+      new_connection_requested(false),
+      final(false) {
   ev_io_init(&wev, writecb, 0, EV_WRITE);
   ev_io_init(&rev, readcb, 0, EV_READ);
 
@@ -516,6 +520,8 @@ void Client::disconnect() {
     close(fd);
     fd = -1;
   }
+
+  final = false;
 }
 
 int Client::submit_request() {
@@ -858,7 +864,7 @@ int Client::connection_made() {
 
   if (!config.timing_script) {
     auto nreq =
-        std::min(req_todo - req_started, (size_t)config.max_concurrent_streams);
+        std::min(req_todo - req_started, session->max_concurrent_streams());
     for (; nreq > 0; --nreq) {
       if (submit_request() != 0) {
         process_request_failure();
@@ -905,6 +911,10 @@ int Client::on_read(const uint8_t *data, size_t len) {
 }
 
 int Client::on_write() {
+  if (wb.rleft() >= BACKOFF_WRITE_BUFFER_THRES) {
+    return 0;
+  }
+
   if (session->on_write() != 0) {
     return -1;
   }
@@ -938,28 +948,32 @@ int Client::read_clear() {
 }
 
 int Client::write_clear() {
+  std::array<struct iovec, 2> iov;
+
   for (;;) {
-    if (wb.rleft() > 0) {
-      ssize_t nwrite;
-      while ((nwrite = write(fd, wb.pos, wb.rleft())) == -1 && errno == EINTR)
-        ;
-      if (nwrite == -1) {
-        if (errno == EAGAIN || errno == EWOULDBLOCK) {
-          ev_io_start(worker->loop, &wev);
-          return 0;
-        }
-        return -1;
-      }
-      wb.drain(nwrite);
-      continue;
-    }
-    wb.reset();
     if (on_write() != 0) {
       return -1;
     }
-    if (wb.rleft() == 0) {
+
+    auto iovcnt = wb.riovec(iov.data(), iov.size());
+
+    if (iovcnt == 0) {
       break;
     }
+
+    ssize_t nwrite;
+    while ((nwrite = writev(fd, iov.data(), iovcnt)) == -1 && errno == EINTR)
+      ;
+
+    if (nwrite == -1) {
+      if (errno == EAGAIN || errno == EWOULDBLOCK) {
+        ev_io_start(worker->loop, &wev);
+        return 0;
+      }
+      return -1;
+    }
+
+    wb.drain(nwrite);
   }
 
   ev_io_stop(worker->loop, &wev);
@@ -1052,35 +1066,36 @@ int Client::read_tls() {
 int Client::write_tls() {
   ERR_clear_error();
 
+  struct iovec iov;
+
   for (;;) {
-    if (wb.rleft() > 0) {
-      auto rv = SSL_write(ssl, wb.pos, wb.rleft());
-
-      if (rv <= 0) {
-        auto err = SSL_get_error(ssl, rv);
-        switch (err) {
-        case SSL_ERROR_WANT_READ:
-          // renegotiation started
-          return -1;
-        case SSL_ERROR_WANT_WRITE:
-          ev_io_start(worker->loop, &wev);
-          return 0;
-        default:
-          return -1;
-        }
-      }
-
-      wb.drain(rv);
-
-      continue;
-    }
-    wb.reset();
     if (on_write() != 0) {
       return -1;
     }
-    if (wb.rleft() == 0) {
+
+    auto iovcnt = wb.riovec(&iov, 1);
+
+    if (iovcnt == 0) {
       break;
     }
+
+    auto rv = SSL_write(ssl, iov.iov_base, iov.iov_len);
+
+    if (rv <= 0) {
+      auto err = SSL_get_error(ssl, rv);
+      switch (err) {
+      case SSL_ERROR_WANT_READ:
+        // renegotiation started
+        return -1;
+      case SSL_ERROR_WANT_WRITE:
+        ev_io_start(worker->loop, &wev);
+        return 0;
+      default:
+        return -1;
+      }
+    }
+
+    wb.drain(rv);
   }
 
   ev_io_stop(worker->loop, &wev);
@@ -1134,10 +1149,20 @@ void Client::signal_write() { ev_io_start(worker->loop, &wev); }
 
 void Client::try_new_connection() { new_connection_requested = true; }
 
+namespace {
+int get_ev_loop_flags() {
+  if (ev_supported_backends() & ~ev_recommended_backends() & EVBACKEND_KQUEUE) {
+    return ev_recommended_backends() | EVBACKEND_KQUEUE;
+  }
+
+  return 0;
+}
+} // namespace
+
 Worker::Worker(uint32_t id, SSL_CTX *ssl_ctx, size_t req_todo, size_t nclients,
                size_t rate, size_t max_samples, Config *config)
     : stats(req_todo, nclients),
-      loop(ev_loop_new(0)),
+      loop(ev_loop_new(get_ev_loop_flags())),
       ssl_ctx(ssl_ctx),
       config(config),
       id(id),
@@ -1655,7 +1680,9 @@ Options:
               Default: )" << NGHTTP2_CLEARTEXT_PROTO_VERSION_ID << R"(
   -d, --data=<PATH>
               Post FILE to  server.  The request method  is changed to
-              POST.
+              POST.   For  http/1.1 connection,  if  -d  is used,  the
+              maximum number of in-flight pipelined requests is set to
+              1.
   -r, --rate=<N>
               Specifies  the  fixed  rate  at  which  connections  are
               created.   The   rate  must   be  a   positive  integer,
@@ -2202,6 +2229,11 @@ int main(int argc, char **argv) {
     }
   }
 
+  std::string content_length_str;
+  if (config.data_fd != -1) {
+    content_length_str = util::utos(config.data_length);
+  }
+
   auto method_it =
       std::find_if(std::begin(shared_nva), std::end(shared_nva),
                    [](const Header &nv) { return nv.name == ":method"; });
@@ -2232,14 +2264,20 @@ int main(int argc, char **argv) {
       h1req += nv.value;
       h1req += "\r\n";
     }
+
+    if (!content_length_str.empty()) {
+      h1req += "Content-Length: ";
+      h1req += content_length_str;
+      h1req += "\r\n";
+    }
     h1req += "\r\n";
 
     config.h1reqs.push_back(std::move(h1req));
 
     // For nghttp2
     std::vector<nghttp2_nv> nva;
-    // 1 for :path
-    nva.reserve(1 + shared_nva.size());
+    // 2 for :path, and possible content-length
+    nva.reserve(2 + shared_nva.size());
 
     nva.push_back(http2::make_nv_ls(":path", req));
 
@@ -2247,12 +2285,18 @@ int main(int argc, char **argv) {
       nva.push_back(http2::make_nv(nv.name, nv.value, false));
     }
 
+    if (!content_length_str.empty()) {
+      nva.push_back(http2::make_nv(StringRef::from_lit("content-length"),
+                                   StringRef{content_length_str}));
+    }
+
     config.nva.push_back(std::move(nva));
 
     // For spdylay
     std::vector<const char *> cva;
-    // 2 for :path and :version, 1 for terminal nullptr
-    cva.reserve(2 * (2 + shared_nva.size()) + 1);
+    // 3 for :path, :version, and possible content-length, 1 for
+    // terminal nullptr
+    cva.reserve(2 * (3 + shared_nva.size()) + 1);
 
     cva.push_back(":path");
     cva.push_back(req.c_str());
@@ -2267,6 +2311,12 @@ int main(int argc, char **argv) {
     }
     cva.push_back(":version");
     cva.push_back("HTTP/1.1");
+
+    if (!content_length_str.empty()) {
+      cva.push_back("content-length");
+      cva.push_back(content_length_str.c_str());
+    }
+
     cva.push_back(nullptr);
 
     config.nv.push_back(std::move(cva));

src/h2load.h

@@ -50,13 +50,15 @@
 #include <openssl/ssl.h>
 
 #include "http2.h"
-#include "buffer.h"
+#include "memchunk.h"
 #include "template.h"
 
 using namespace nghttp2;
 
 namespace h2load {
 
+constexpr auto BACKOFF_WRITE_BUFFER_THRES = 16_k;
+
 class Session;
 struct Worker;
 
@@ -225,6 +227,7 @@ struct Sampling {
 };
 
 struct Worker {
+  MemchunkPool mcpool;
   Stats stats;
   Sampling request_times_smp;
   Sampling client_smp;
@@ -267,6 +270,7 @@ struct Stream {
 };
 
 struct Client {
+  DefaultMemchunks wb;
   std::unordered_map<int32_t, Stream> streams;
   ClientStat cstat;
   std::unique_ptr<Session> session;
@@ -293,11 +297,13 @@ struct Client {
   // The client id per worker
   uint32_t id;
   int fd;
-  Buffer<64_k> wb;
   ev_timer conn_active_watcher;
   ev_timer conn_inactivity_watcher;
   std::string selected_proto;
   bool new_connection_requested;
+  // true if the current connection will be closed, and no more new
+  // request cannot be processed.
+  bool final;
 
   enum { ERR_CONNECT_FAIL = -100 };
 

src/h2load_http1_session.cc

@@ -57,7 +57,7 @@ namespace {
 int htp_msg_begincb(http_parser *htp) {
   auto session = static_cast<Http1Session *>(htp->data);
 
-  if (session->stream_resp_counter_ >= session->stream_req_counter_) {
+  if (session->stream_resp_counter_ > session->stream_req_counter_) {
     return -1;
   }
 
@@ -82,16 +82,21 @@ int htp_msg_completecb(http_parser *htp) {
   auto session = static_cast<Http1Session *>(htp->data);
   auto client = session->get_client();
 
-  auto final = http_should_keep_alive(htp) == 0;
+  client->final = http_should_keep_alive(htp) == 0;
   auto req_stat = client->get_req_stat(session->stream_resp_counter_);
 
   assert(req_stat);
 
-  client->on_stream_close(session->stream_resp_counter_, true, final);
+  auto config = client->worker->config;
+  if (req_stat->data_offset >= config->data_length) {
+    client->on_stream_close(session->stream_resp_counter_, true, client->final);
+  }
 
   session->stream_resp_counter_ += 2;
 
-  if (final) {
+  if (client->final) {
+    session->stream_req_counter_ = session->stream_resp_counter_;
+
     http_parser_pause(htp, 1);
     // Connection is going down.  If we have still request to do,
     // create new connection and keep on doing the job.
@@ -169,12 +174,16 @@ int Http1Session::submit_request() {
   auto req_stat = client_->get_req_stat(stream_req_counter_);
 
   client_->record_request_time(req_stat);
-  client_->wb.write(req.c_str(), req.size());
+  client_->wb.append(req);
 
-  // increment for next request
-  stream_req_counter_ += 2;
+  if (config->data_fd == -1 || config->data_length == 0) {
+    // increment for next request
+    stream_req_counter_ += 2;
 
-  return 0;
+    return 0;
+  }
+
+  return on_write();
 }
 
 int Http1Session::on_read(const uint8_t *data, size_t len) {
@@ -206,6 +215,51 @@ int Http1Session::on_write() {
   if (complete_) {
     return -1;
   }
+
+  auto config = client_->worker->config;
+  auto req_stat = client_->get_req_stat(stream_req_counter_);
+  if (!req_stat) {
+    return 0;
+  }
+
+  if (req_stat->data_offset < config->data_length) {
+    auto req_stat = client_->get_req_stat(stream_req_counter_);
+    auto &wb = client_->wb;
+
+    // TODO unfortunately, wb has no interface to use with read(2)
+    // family functions.
+    std::array<uint8_t, 16_k> buf;
+
+    ssize_t nread;
+    while ((nread = pread(config->data_fd, buf.data(), buf.size(),
+                          req_stat->data_offset)) == -1 &&
+           errno == EINTR)
+      ;
+
+    if (nread == -1) {
+      return -1;
+    }
+
+    req_stat->data_offset += nread;
+
+    wb.append(buf.data(), nread);
+
+    if (client_->worker->config->verbose) {
+      std::cout << "[send " << nread << " byte(s)]" << std::endl;
+    }
+
+    if (req_stat->data_offset == config->data_length) {
+      // increment for next request
+      stream_req_counter_ += 2;
+
+      if (stream_resp_counter_ == stream_req_counter_) {
+        // Response has already been received
+        client_->on_stream_close(stream_resp_counter_ - 2, true,
+                                 client_->final);
+      }
+    }
+  }
+
   return 0;
 }
 
@@ -213,4 +267,10 @@ void Http1Session::terminate() { complete_ = true; }
 
 Client *Http1Session::get_client() { return client_; }
 
+size_t Http1Session::max_concurrent_streams() {
+  auto config = client_->worker->config;
+
+  return config->data_fd == -1 ? config->max_concurrent_streams : 1;
+}
+
 } // namespace h2load

src/h2load_http1_session.h

@@ -42,6 +42,7 @@ public:
   virtual int on_read(const uint8_t *data, size_t len);
   virtual int on_write();
   virtual void terminate();
+  virtual size_t max_concurrent_streams();
   Client *get_client();
   int32_t stream_req_counter_;
   int32_t stream_resp_counter_;

src/h2load_http2_session.cc

@@ -149,8 +149,13 @@ ssize_t file_read_callback(nghttp2_session *session, int32_t stream_id,
 
   req_stat->data_offset += nread;
 
-  if (nread == 0 || req_stat->data_offset == config->data_length) {
+  if (req_stat->data_offset == config->data_length) {
     *data_flags |= NGHTTP2_DATA_FLAG_EOF;
+    return nread;
+  }
+
+  if (req_stat->data_offset > config->data_length || nread == 0) {
+    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
   }
 
   return nread;
@@ -164,11 +169,11 @@ ssize_t send_callback(nghttp2_session *session, const uint8_t *data,
   auto client = static_cast<Client *>(user_data);
   auto &wb = client->wb;
 
-  if (wb.wleft() == 0) {
+  if (wb.rleft() >= BACKOFF_WRITE_BUFFER_THRES) {
     return NGHTTP2_ERR_WOULDBLOCK;
   }
 
-  return wb.write(data, length);
+  return wb.append(data, length);
 }
 } // namespace
 
@@ -214,13 +219,10 @@ void Http2Session::on_connect() {
 
   assert(rv == 0);
 
-  auto extra_connection_window =
-      (1 << client_->worker->config->connection_window_bits) - 1 -
-      NGHTTP2_INITIAL_CONNECTION_WINDOW_SIZE;
-  if (extra_connection_window != 0) {
-    nghttp2_submit_window_update(session_, NGHTTP2_FLAG_NONE, 0,
-                                 extra_connection_window);
-  }
+  auto connection_window =
+      (1 << client_->worker->config->connection_window_bits) - 1;
+  nghttp2_session_set_local_window_size(session_, NGHTTP2_FLAG_NONE, 0,
+                                        connection_window);
 
   client_->signal_write();
 }
@@ -287,4 +289,8 @@ void Http2Session::terminate() {
   nghttp2_session_terminate_session(session_, NGHTTP2_NO_ERROR);
 }
 
+size_t Http2Session::max_concurrent_streams() {
+  return (size_t)client_->worker->config->max_concurrent_streams;
+}
+
 } // namespace h2load

src/h2load_http2_session.h

@@ -42,6 +42,7 @@ public:
   virtual int on_read(const uint8_t *data, size_t len);
   virtual int on_write();
   virtual void terminate();
+  virtual size_t max_concurrent_streams();
 
 private:
   Client *client_;

src/h2load_session.h

@@ -50,6 +50,8 @@ public:
   virtual int on_write() = 0;
   // Called when the underlying session must be terminated.
   virtual void terminate() = 0;
+  // Return the maximum concurrency per connection
+  virtual size_t max_concurrent_streams() = 0;
 };
 
 } // namespace h2load

src/h2load_spdy_session.cc

@@ -112,11 +112,11 @@ ssize_t send_callback(spdylay_session *session, const uint8_t *data,
   auto client = static_cast<Client *>(user_data);
   auto &wb = client->wb;
 
-  if (wb.wleft() == 0) {
+  if (wb.rleft() >= BACKOFF_WRITE_BUFFER_THRES) {
     return SPDYLAY_ERR_DEFERRED;
   }
 
-  return wb.write(data, length);
+  return wb.append(data, length);
 }
 } // namespace
 
@@ -282,4 +282,8 @@ void SpdySession::handle_window_update(int32_t stream_id, size_t recvlen) {
   }
 }
 
+size_t SpdySession::max_concurrent_streams() {
+  return (size_t)client_->worker->config->max_concurrent_streams;
+}
+
 } // namespace h2load

src/h2load_spdy_session.h

@@ -44,6 +44,7 @@ public:
   virtual int on_read(const uint8_t *data, size_t len);
   virtual int on_write();
   virtual void terminate();
+  virtual size_t max_concurrent_streams();
   void handle_window_update(int32_t stream_id, size_t recvlen);
 
 private:

src/nghttp.cc

@@ -59,6 +59,7 @@
 #include "base64.h"
 #include "ssl.h"
 #include "template.h"
+#include "cache_digest.h"
 
 #ifndef O_BINARY
 #define O_BINARY (0)
@@ -98,6 +99,7 @@ Config::Config()
       padding(0),
       max_concurrent_streams(100),
       peer_max_concurrent_streams(100),
+      cache_digest_bits(7),
       weight(NGHTTP2_DEFAULT_WEIGHT),
       multiply(1),
       timeout(0.),
@@ -113,10 +115,12 @@ Config::Config()
       no_content_length(false),
       no_dep(false),
       hexdump(false),
-      no_push(false) {
+      no_push(false),
+      expect_continue(false) {
   nghttp2_option_new(&http2_option);
   nghttp2_option_set_peer_max_concurrent_streams(http2_option,
                                                  peer_max_concurrent_streams);
+  nghttp2_option_set_builtin_recv_extension_type(http2_option, NGHTTP2_ALTSVC);
 }
 
 Config::~Config() { nghttp2_option_del(http2_option); }
@@ -303,6 +307,44 @@ void Request::record_response_end_time() {
   timing.response_end_time = get_time();
 }
 
+namespace {
+void continue_timeout_cb(struct ev_loop *loop, ev_timer *w, int revents) {
+  auto client = static_cast<HttpClient *>(ev_userdata(loop));
+  auto req = static_cast<Request *>(w->data);
+  int error;
+
+  error = nghttp2_submit_data(client->session, NGHTTP2_FLAG_END_STREAM,
+                              req->stream_id, req->data_prd);
+
+  if (error) {
+    std::cerr << "[ERROR] nghttp2_submit_data() returned error: "
+              << nghttp2_strerror(error) << std::endl;
+    nghttp2_submit_rst_stream(client->session, NGHTTP2_FLAG_NONE,
+                              req->stream_id, NGHTTP2_INTERNAL_ERROR);
+  }
+
+  client->signal_write();
+}
+} // namespace
+
+ContinueTimer::ContinueTimer(struct ev_loop *loop, Request *req) : loop(loop) {
+  ev_timer_init(&timer, continue_timeout_cb, 1., 0.);
+  timer.data = req;
+}
+
+ContinueTimer::~ContinueTimer() { stop(); }
+
+void ContinueTimer::start() { ev_timer_start(loop, &timer); }
+
+void ContinueTimer::stop() { ev_timer_stop(loop, &timer); }
+
+void ContinueTimer::dispatch_continue() {
+  // Only dispatch the timeout callback if it hasn't already been called.
+  if (ev_is_active(&timer)) {
+    ev_feed_event(loop, &timer, 0);
+  }
+}
+
 namespace {
 int htp_msg_begincb(http_parser *htp) {
   if (config.verbose) {
@@ -353,16 +395,28 @@ int submit_request(HttpClient *client, const Headers &headers, Request *req) {
                                {"accept", "*/*"},
                                {"accept-encoding", "gzip, deflate"},
                                {"user-agent", "nghttp2/" NGHTTP2_VERSION}};
+  bool expect_continue = false;
+
   if (config.continuation) {
     for (size_t i = 0; i < 6; ++i) {
       build_headers.emplace_back("continuation-test-" + util::utos(i + 1),
                                  std::string(4_k, '-'));
     }
   }
+
   auto num_initial_headers = build_headers.size();
-  if (!config.no_content_length && req->data_prd) {
-    build_headers.emplace_back("content-length", util::utos(req->data_length));
+
+  if (req->data_prd) {
+    if (!config.no_content_length) {
+      build_headers.emplace_back("content-length",
+                                 util::utos(req->data_length));
+    }
+    if (config.expect_continue) {
+      expect_continue = true;
+      build_headers.emplace_back("expect", "100-continue");
+    }
   }
+
   for (auto &kv : headers) {
     size_t i;
     for (i = 0; i < num_initial_headers; ++i) {
@@ -400,12 +454,22 @@ int submit_request(HttpClient *client, const Headers &headers, Request *req) {
     nva.push_back(http2::make_nv_ls("trailer", trailer_names));
   }
 
-  auto stream_id =
-      nghttp2_submit_request(client->session, &req->pri_spec, nva.data(),
-                             nva.size(), req->data_prd, req);
+  int32_t stream_id;
+
+  if (expect_continue) {
+    stream_id = nghttp2_submit_headers(client->session, 0, -1, &req->pri_spec,
+                                       nva.data(), nva.size(), req);
+  } else {
+    stream_id =
+        nghttp2_submit_request(client->session, &req->pri_spec, nva.data(),
+                               nva.size(), req->data_prd, req);
+  }
+
   if (stream_id < 0) {
-    std::cerr << "[ERROR] nghttp2_submit_request() returned error: "
-              << nghttp2_strerror(stream_id) << std::endl;
+    std::cerr << "[ERROR] nghttp2_submit_"
+              << (expect_continue ? "headers" : "request")
+              << "() returned error: " << nghttp2_strerror(stream_id)
+              << std::endl;
     return -1;
   }
 
@@ -414,6 +478,11 @@ int submit_request(HttpClient *client, const Headers &headers, Request *req) {
 
   req->req_nva = std::move(build_headers);
 
+  if (expect_continue) {
+    auto timer = make_unique<ContinueTimer>(client->loop, req);
+    req->continue_timer = std::move(timer);
+  }
+
   return 0;
 }
 } // namespace
@@ -462,7 +531,8 @@ void settings_timeout_cb(struct ev_loop *loop, ev_timer *w, int revents) {
 
 HttpClient::HttpClient(const nghttp2_session_callbacks *callbacks,
                        struct ev_loop *loop, SSL_CTX *ssl_ctx)
-    : session(nullptr),
+    : wb(&mcpool),
+      session(nullptr),
       callbacks(callbacks),
       loop(loop),
       ssl_ctx(ssl_ctx),
@@ -476,7 +546,8 @@ HttpClient::HttpClient(const nghttp2_session_callbacks *callbacks,
       state(ClientState::IDLE),
       upgrade_response_status_code(0),
       fd(-1),
-      upgrade_response_complete(false) {
+      upgrade_response_complete(false),
+      cache_digest_sent(false) {
   ev_io_init(&wev, writecb, 0, EV_WRITE);
   ev_io_init(&rev, readcb, 0, EV_READ);
 
@@ -614,6 +685,12 @@ int HttpClient::initiate_connection() {
 void HttpClient::disconnect() {
   state = ClientState::IDLE;
 
+  for (auto req = std::begin(reqvec); req != std::end(reqvec); ++req) {
+    if ((*req)->continue_timer) {
+      (*req)->continue_timer->stop();
+    }
+  }
+
   ev_timer_stop(loop, &settings_timer);
 
   ev_timer_stop(loop, &rt);
@@ -671,29 +748,32 @@ int HttpClient::read_clear() {
 int HttpClient::write_clear() {
   ev_timer_again(loop, &rt);
 
+  std::array<struct iovec, 2> iov;
+
   for (;;) {
-    if (wb.rleft() > 0) {
-      ssize_t nwrite;
-      while ((nwrite = write(fd, wb.pos, wb.rleft())) == -1 && errno == EINTR)
-        ;
-      if (nwrite == -1) {
-        if (errno == EAGAIN || errno == EWOULDBLOCK) {
-          ev_io_start(loop, &wev);
-          ev_timer_again(loop, &wt);
-          return 0;
-        }
-        return -1;
-      }
-      wb.drain(nwrite);
-      continue;
-    }
-    wb.reset();
     if (on_writefn(*this) != 0) {
       return -1;
     }
-    if (wb.rleft() == 0) {
+
+    auto iovcnt = wb.riovec(iov.data(), iov.size());
+
+    if (iovcnt == 0) {
       break;
     }
+
+    ssize_t nwrite;
+    while ((nwrite = writev(fd, iov.data(), iovcnt)) == -1 && errno == EINTR)
+      ;
+    if (nwrite == -1) {
+      if (errno == EAGAIN || errno == EWOULDBLOCK) {
+        ev_io_start(loop, &wev);
+        ev_timer_again(loop, &wt);
+        return 0;
+      }
+      return -1;
+    }
+
+    wb.drain(nwrite);
   }
 
   ev_io_stop(loop, &wev);
@@ -873,7 +953,7 @@ int HttpClient::on_upgrade_connect() {
   }
   req += "\r\n";
 
-  wb.write(req.c_str(), req.size());
+  wb.append(req);
 
   if (config.verbose) {
     print_timer();
@@ -1074,9 +1154,9 @@ int HttpClient::connection_made() {
   ev_timer_again(loop, &settings_timer);
 
   if (config.connection_window_bits != -1) {
-    int32_t wininc = (1 << config.connection_window_bits) - 1 -
-                     NGHTTP2_INITIAL_CONNECTION_WINDOW_SIZE;
-    rv = nghttp2_submit_window_update(session, NGHTTP2_FLAG_NONE, 0, wininc);
+    int32_t window_size = (1 << config.connection_window_bits) - 1;
+    rv = nghttp2_session_set_local_window_size(session, NGHTTP2_FLAG_NONE, 0,
+                                               window_size);
     if (rv != 0) {
       return -1;
     }
@@ -1120,11 +1200,24 @@ int HttpClient::on_read(const uint8_t *data, size_t len) {
 }
 
 int HttpClient::on_write() {
-  auto rv = nghttp2_session_send(session);
-  if (rv != 0) {
-    std::cerr << "[ERROR] nghttp2_session_send() returned error: "
-              << nghttp2_strerror(rv) << std::endl;
-    return -1;
+  for (;;) {
+    if (wb.rleft() >= 16384) {
+      return 0;
+    }
+
+    const uint8_t *data;
+    auto len = nghttp2_session_mem_send(session, &data);
+    if (len < 0) {
+      std::cerr << "[ERROR] nghttp2_session_send() returned error: "
+                << nghttp2_strerror(len) << std::endl;
+      return -1;
+    }
+
+    if (len == 0) {
+      break;
+    }
+
+    wb.append(data, len);
   }
 
   if (nghttp2_session_want_read(session) == 0 &&
@@ -1204,36 +1297,37 @@ int HttpClient::write_tls() {
 
   ERR_clear_error();
 
+  struct iovec iov;
+
   for (;;) {
-    if (wb.rleft() > 0) {
-      auto rv = SSL_write(ssl, wb.pos, wb.rleft());
-
-      if (rv <= 0) {
-        auto err = SSL_get_error(ssl, rv);
-        switch (err) {
-        case SSL_ERROR_WANT_READ:
-          // renegotiation started
-          return -1;
-        case SSL_ERROR_WANT_WRITE:
-          ev_io_start(loop, &wev);
-          ev_timer_again(loop, &wt);
-          return 0;
-        default:
-          return -1;
-        }
-      }
-
-      wb.drain(rv);
-
-      continue;
-    }
-    wb.reset();
     if (on_writefn(*this) != 0) {
       return -1;
     }
-    if (wb.rleft() == 0) {
+
+    auto iovcnt = wb.riovec(&iov, 1);
+
+    if (iovcnt == 0) {
       break;
     }
+
+    auto rv = SSL_write(ssl, iov.iov_base, iov.iov_len);
+
+    if (rv <= 0) {
+      auto err = SSL_get_error(ssl, rv);
+      switch (err) {
+      case SSL_ERROR_WANT_READ:
+        // renegotiation started
+        return -1;
+      case SSL_ERROR_WANT_WRITE:
+        ev_io_start(loop, &wev);
+        ev_timer_again(loop, &wt);
+        return 0;
+      default:
+        return -1;
+      }
+    }
+
+    wb.drain(rv);
   }
 
   ev_io_stop(loop, &wev);
@@ -1616,11 +1710,19 @@ void check_response_header(nghttp2_session *session, Request *req) {
   }
 
   if (req->status / 100 == 1) {
+    if (req->continue_timer && (req->status == 100)) {
+      // If the request is waiting for a 100 Continue, complete the handshake.
+      req->continue_timer->dispatch_continue();
+    }
+
     req->expect_final_response = true;
     req->status = 0;
     req->res_nva.clear();
     http2::init_hdidx(req->res_hdidx);
     return;
+  } else if (req->continue_timer) {
+    // A final response stops any pending Expect/Continue handshake.
+    req->continue_timer->stop();
   }
 
   if (gzip) {
@@ -1895,6 +1997,48 @@ int before_frame_send_callback(nghttp2_session *session,
 
 } // namespace
 
+namespace {
+int on_frame_send_callback(nghttp2_session *session, const nghttp2_frame *frame,
+                           void *user_data) {
+  int rv;
+
+  if (config.verbose) {
+    verbose_on_frame_send_callback(session, frame, user_data);
+  }
+
+  if (frame->hd.type != NGHTTP2_HEADERS ||
+      frame->headers.cat != NGHTTP2_HCAT_REQUEST) {
+    return 0;
+  }
+
+  auto req = static_cast<Request *>(
+      nghttp2_session_get_stream_user_data(session, frame->hd.stream_id));
+  if (!req) {
+    return 0;
+  }
+
+  // If this request is using Expect/Continue, start its ContinueTimer.
+  if (req->continue_timer) {
+    req->continue_timer->start();
+  }
+
+  auto client = get_client(user_data);
+
+  if (!client->cache_digest_sent && !config.cache_digest_uris.empty()) {
+    client->cache_digest_sent = true;
+
+    rv = nghttp2_submit_extension(session, NGHTTP2_DRAFT_CACHE_DIGEST,
+                                  NGHTTP2_FLAG_NONE, frame->hd.stream_id, NULL);
+
+    if (nghttp2_is_fatal(rv)) {
+      return NGHTTP2_ERR_CALLBACK_FAILURE;
+    }
+  }
+
+  return 0;
+}
+} // namespace
+
 namespace {
 int on_frame_not_send_callback(nghttp2_session *session,
                                const nghttp2_frame *frame, int lib_error_code,
@@ -1928,6 +2072,11 @@ int on_stream_close_callback(nghttp2_session *session, int32_t stream_id,
     return 0;
   }
 
+  // If this request is using Expect/Continue, stop its ContinueTimer.
+  if (req->continue_timer) {
+    req->continue_timer->stop();
+  }
+
   update_html_parser(client, req, nullptr, 0, 1);
   ++client->complete;
 
@@ -2138,7 +2287,10 @@ int communicate(
                 << std::endl;
       goto fin;
     }
+
+    ev_set_userdata(loop, &client);
     ev_run(loop, 0);
+    ev_set_userdata(loop, nullptr);
 
 #ifdef HAVE_JANSSON
     if (!config.harfile.empty()) {
@@ -2198,7 +2350,9 @@ ssize_t file_read_callback(nghttp2_session *session, int32_t stream_id,
     return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
   }
 
-  if (nread == 0) {
+  req->data_offset += nread;
+
+  if (req->data_offset == req->data_length) {
     *data_flags |= NGHTTP2_DATA_FLAG_EOF;
     if (!config.trailer.empty()) {
       std::vector<nghttp2_nv> nva;
@@ -2215,8 +2369,12 @@ ssize_t file_read_callback(nghttp2_session *session, int32_t stream_id,
         *data_flags |= NGHTTP2_DATA_FLAG_NO_END_STREAM;
       }
     }
-  } else {
-    req->data_offset += nread;
+
+    return nread;
+  }
+
+  if (req->data_offset > req->data_length || nread == 0) {
+    return NGHTTP2_ERR_TEMPORAL_CALLBACK_FAILURE;
   }
 
   return nread;
@@ -2224,16 +2382,31 @@ ssize_t file_read_callback(nghttp2_session *session, int32_t stream_id,
 } // namespace
 
 namespace {
-ssize_t send_callback(nghttp2_session *session, const uint8_t *data,
-                      size_t length, int flags, void *user_data) {
-  auto client = static_cast<HttpClient *>(user_data);
-  auto &wb = client->wb;
+ssize_t pack_cache_digest_frame(nghttp2_session *session, uint8_t *buf,
+                                size_t len, const nghttp2_frame *frame,
+                                void *user_data) {
+  ssize_t encodedlen;
 
-  if (wb.wleft() == 0) {
-    return NGHTTP2_ERR_WOULDBLOCK;
+  encodedlen = cache_digest_encode(buf, len, config.cache_digest_uris,
+                                   config.cache_digest_bits);
+
+  if (encodedlen == -1) {
+    return NGHTTP2_ERR_CANCEL;
   }
 
-  return wb.write(data, length);
+  return encodedlen;
+}
+} // namespace
+
+namespace {
+ssize_t pack_extension_callback(nghttp2_session *session, uint8_t *buf,
+                                size_t len, const nghttp2_frame *frame,
+                                void *user_data) {
+  if (frame->hd.type != NGHTTP2_DRAFT_CACHE_DIGEST) {
+    return NGHTTP2_ERR_CANCEL;
+  }
+
+  return pack_cache_digest_frame(session, buf, len, frame, user_data);
 }
 } // namespace
 
@@ -2251,9 +2424,6 @@ int run(char **uris, int n) {
                                                        on_frame_recv_callback2);
 
   if (config.verbose) {
-    nghttp2_session_callbacks_set_on_frame_send_callback(
-        callbacks, verbose_on_frame_send_callback);
-
     nghttp2_session_callbacks_set_on_invalid_frame_recv_callback(
         callbacks, verbose_on_invalid_frame_recv_callback);
 
@@ -2273,16 +2443,20 @@ int run(char **uris, int n) {
   nghttp2_session_callbacks_set_before_frame_send_callback(
       callbacks, before_frame_send_callback);
 
+  nghttp2_session_callbacks_set_on_frame_send_callback(callbacks,
+                                                       on_frame_send_callback);
+
   nghttp2_session_callbacks_set_on_frame_not_send_callback(
       callbacks, on_frame_not_send_callback);
 
-  nghttp2_session_callbacks_set_send_callback(callbacks, send_callback);
-
   if (config.padding) {
     nghttp2_session_callbacks_set_select_padding_callback(
         callbacks, select_padding_callback);
   }
 
+  nghttp2_session_callbacks_set_pack_extension_callback(
+      callbacks, pack_extension_callback);
+
   std::string prev_scheme;
   std::string prev_host;
   uint16_t prev_port = 0;
@@ -2503,6 +2677,18 @@ Options:
   --max-concurrent-streams=<N>
               The  number of  concurrent  pushed  streams this  client
               accepts.
+  --expect-continue
+              Perform an Expect/Continue handshake:  wait to send DATA
+              (up to  a short  timeout)  until the server sends  a 100
+              Continue interim response. This option is ignored unless
+              combined with the -d option.
+  -C, --cache-digest-uri=<URI>
+              Add  <URI> to  cache digest.   Use this  option multiple
+              times to add more than 1 URI to cache digest.
+  --cache-digest-bits=<N>
+              Set the number  of bits to specify the  probability of a
+              false positive that is acceptable, expressed as "1/<N>".
+              Default: 7
   --version   Display version information and exit.
   -h, --help  Display this help and exit.
 
@@ -2543,6 +2729,7 @@ int main(int argc, char **argv) {
         {"header-table-size", required_argument, nullptr, 'c'},
         {"padding", required_argument, nullptr, 'b'},
         {"har", required_argument, nullptr, 'r'},
+        {"cache-digest-uri", required_argument, nullptr, 'C'},
         {"cert", required_argument, &flag, 1},
         {"key", required_argument, &flag, 2},
         {"color", no_argument, &flag, 3},
@@ -2554,14 +2741,19 @@ int main(int argc, char **argv) {
         {"hexdump", no_argument, &flag, 10},
         {"no-push", no_argument, &flag, 11},
         {"max-concurrent-streams", required_argument, &flag, 12},
+        {"expect-continue", no_argument, &flag, 13},
+        {"cache-digest-bits", required_argument, &flag, 14},
         {nullptr, 0, nullptr, 0}};
     int option_index = 0;
-    int c = getopt_long(argc, argv, "M:Oab:c:d:gm:np:r:hH:vst:uw:W:",
+    int c = getopt_long(argc, argv, "C:M:Oab:c:d:gm:np:r:hH:vst:uw:W:",
                         long_options, &option_index);
     if (c == -1) {
       break;
     }
     switch (c) {
+    case 'C':
+      config.cache_digest_uris.push_back(optarg);
+      break;
     case 'M':
       // peer-max-concurrent-streams option
       config.peer_max_concurrent_streams = strtoul(optarg, nullptr, 10);
@@ -2751,6 +2943,22 @@ int main(int argc, char **argv) {
         // max-concurrent-streams option
         config.max_concurrent_streams = strtoul(optarg, nullptr, 10);
         break;
+      case 13:
+        // expect-continue option
+        config.expect_continue = true;
+        break;
+      case 14: {
+        // cache-digest-bits
+        auto n = strtoul(optarg, nullptr, 10);
+        if (n <= 0 || n >= 32) {
+          std::cerr
+              << "--cache-digest-bits: specify in the range [1, 31], inclusive"
+              << std::endl;
+          exit(EXIT_FAILURE);
+        }
+        config.cache_digest_bits = n;
+        break;
+      }
       }
       break;
     default:

src/nghttp.h

@@ -49,7 +49,7 @@
 
 #include "http-parser/http_parser.h"
 
-#include "buffer.h"
+#include "memchunk.h"
 #include "http2.h"
 #include "nghttp2_gzip.h"
 #include "template.h"
@@ -64,6 +64,7 @@ struct Config {
 
   Headers headers;
   Headers trailer;
+  std::vector<std::string> cache_digest_uris;
   std::string certfile;
   std::string keyfile;
   std::string datafile;
@@ -74,6 +75,7 @@ struct Config {
   size_t padding;
   size_t max_concurrent_streams;
   ssize_t peer_max_concurrent_streams;
+  uint32_t cache_digest_bits;
   int32_t weight;
   int multiply;
   // milliseconds
@@ -91,6 +93,7 @@ struct Config {
   bool no_dep;
   bool hexdump;
   bool no_push;
+  bool expect_continue;
 };
 
 enum class RequestState { INITIAL, ON_REQUEST, ON_RESPONSE, ON_COMPLETE };
@@ -109,6 +112,23 @@ struct RequestTiming {
   RequestTiming() : state(RequestState::INITIAL) {}
 };
 
+struct Request; // forward declaration for ContinueTimer
+
+struct ContinueTimer {
+  ContinueTimer(struct ev_loop *loop, Request *req);
+  ~ContinueTimer();
+
+  void start();
+  void stop();
+
+  // Schedules an immediate run of the continue callback on the loop, if the
+  // callback has not already been run
+  void dispatch_continue();
+
+  struct ev_loop *loop;
+  ev_timer timer;
+};
+
 struct Request {
   // For pushed request, |uri| is empty and |u| is zero-cleared.
   Request(const std::string &uri, const http_parser_url &u,
@@ -156,6 +176,8 @@ struct Request {
   // used for incoming PUSH_PROMISE
   http2::HeaderIndex req_hdidx;
   bool expect_final_response;
+  // only assigned if this request is using Expect/Continue
+  std::unique_ptr<ContinueTimer> continue_timer;
 };
 
 struct SessionTiming {
@@ -221,6 +243,8 @@ struct HttpClient {
   void output_har(FILE *outfile);
 #endif // HAVE_JANSSON
 
+  MemchunkPool mcpool;
+  DefaultMemchunks wb;
   std::vector<std::unique_ptr<Request>> reqvec;
   // Insert path already added in reqvec to prevent multiple request
   // for 1 resource.
@@ -261,7 +285,8 @@ struct HttpClient {
   // true if the response message of HTTP Upgrade request is fully
   // received. It is not relevant the upgrade succeeds, or not.
   bool upgrade_response_complete;
-  Buffer<64_k> wb;
+  // true if cache digest was sent or there is no need to send it.
+  bool cache_digest_sent;
   // SETTINGS payload sent as token68 in HTTP Upgrade
   std::array<uint8_t, 128> settings_payload;
 

src/shrpx-unittest.cc

@@ -44,6 +44,8 @@
 #include "base64_test.h"
 #include "shrpx_config.h"
 #include "ssl.h"
+#include "shrpx_router_test.h"
+#include "cache_digest_test.h"
 
 static int init_suite1(void) { return 0; }
 
@@ -71,8 +73,8 @@ int main(int argc, char *argv[]) {
   // add the tests to the suite
   if (!CU_add_test(pSuite, "ssl_create_lookup_tree",
                    shrpx::test_shrpx_ssl_create_lookup_tree) ||
-      !CU_add_test(pSuite, "ssl_cert_lookup_tree_add_cert_from_file",
-                   shrpx::test_shrpx_ssl_cert_lookup_tree_add_cert_from_file) ||
+      !CU_add_test(pSuite, "ssl_cert_lookup_tree_add_cert_from_x509",
+                   shrpx::test_shrpx_ssl_cert_lookup_tree_add_cert_from_x509) ||
       !CU_add_test(pSuite, "ssl_tls_hostname_match",
                    shrpx::test_shrpx_ssl_tls_hostname_match) ||
       !CU_add_test(pSuite, "http2_add_header", shrpx::test_http2_add_header) ||
@@ -125,6 +127,9 @@ int main(int argc, char *argv[]) {
                    shrpx::test_shrpx_http_create_forwarded) ||
       !CU_add_test(pSuite, "http_create_via_header_value",
                    shrpx::test_shrpx_http_create_via_header_value) ||
+      !CU_add_test(pSuite, "router_match", shrpx::test_shrpx_router_match) ||
+      !CU_add_test(pSuite, "router_match_prefix",
+                   shrpx::test_shrpx_router_match_prefix) ||
       !CU_add_test(pSuite, "util_streq", shrpx::test_util_streq) ||
       !CU_add_test(pSuite, "util_strieq", shrpx::test_util_strieq) ||
       !CU_add_test(pSuite, "util_inp_strlower",
@@ -194,7 +199,9 @@ int main(int argc, char *argv[]) {
       !CU_add_test(pSuite, "template_string_ref",
                    nghttp2::test_template_string_ref) ||
       !CU_add_test(pSuite, "base64_encode", nghttp2::test_base64_encode) ||
-      !CU_add_test(pSuite, "base64_decode", nghttp2::test_base64_decode)) {
+      !CU_add_test(pSuite, "base64_decode", nghttp2::test_base64_decode) ||
+      !CU_add_test(pSuite, "cache_digest_encode_decode",
+                   nghttp2::test_cache_digest_encode_decode)) {
     CU_cleanup_registry();
     return CU_get_error();
   }

src/shrpx.cc

@@ -81,6 +81,7 @@
 #include "shrpx_worker_process.h"
 #include "shrpx_process.h"
 #include "shrpx_signal.h"
+#include "shrpx_connection.h"
 #include "util.h"
 #include "app_helper.h"
 #include "ssl.h"
@@ -145,52 +146,6 @@ struct SignalServer {
   pid_t worker_process_pid;
 };
 
-namespace {
-int resolve_hostname(Address *addr, const char *hostname, uint16_t port,
-                     int family) {
-  int rv;
-
-  auto service = util::utos(port);
-
-  addrinfo hints{};
-  hints.ai_family = family;
-  hints.ai_socktype = SOCK_STREAM;
-#ifdef AI_ADDRCONFIG
-  hints.ai_flags |= AI_ADDRCONFIG;
-#endif // AI_ADDRCONFIG
-  addrinfo *res;
-
-  rv = getaddrinfo(hostname, service.c_str(), &hints, &res);
-  if (rv != 0) {
-    LOG(FATAL) << "Unable to resolve address for " << hostname << ": "
-               << gai_strerror(rv);
-    return -1;
-  }
-
-  auto res_d = defer(freeaddrinfo, res);
-
-  char host[NI_MAXHOST];
-  rv = getnameinfo(res->ai_addr, res->ai_addrlen, host, sizeof(host), nullptr,
-                   0, NI_NUMERICHOST);
-  if (rv != 0) {
-    LOG(FATAL) << "Address resolution for " << hostname
-               << " failed: " << gai_strerror(rv);
-
-    return -1;
-  }
-
-  if (LOG_ENABLED(INFO)) {
-    LOG(INFO) << "Address resolution for " << hostname
-              << " succeeded: " << host;
-  }
-
-  memcpy(&addr->su, res->ai_addr, res->ai_addrlen);
-  addr->len = res->ai_addrlen;
-
-  return 0;
-}
-} // namespace
-
 namespace {
 int chown_to_running_user(const char *path) {
   return chown(path, get_config()->uid, get_config()->gid);
@@ -199,18 +154,59 @@ int chown_to_running_user(const char *path) {
 
 namespace {
 void save_pid() {
-  std::ofstream out(get_config()->pid_file.c_str(), std::ios::binary);
-  out << get_config()->pid << "\n";
-  out.close();
-  if (!out) {
-    LOG(ERROR) << "Could not save PID to file " << get_config()->pid_file;
+  constexpr auto SUFFIX = StringRef::from_lit(".XXXXXX");
+  auto &pid_file = get_config()->pid_file;
+
+  auto len = get_config()->pid_file.size() + SUFFIX.size();
+  auto buf = make_unique<char[]>(len + 1);
+  auto p = buf.get();
+
+  p = std::copy(std::begin(pid_file), std::end(pid_file), p);
+  p = std::copy(std::begin(SUFFIX), std::end(SUFFIX), p);
+  *p = '\0';
+
+  auto temp_path = buf.get();
+
+  auto fd = mkstemp(temp_path);
+  if (fd == -1) {
+    auto error = errno;
+    LOG(ERROR) << "Could not save PID to file " << pid_file << ": "
+               << strerror(error);
+    exit(EXIT_FAILURE);
+  }
+
+  auto content = util::utos(get_config()->pid) + '\n';
+
+  if (write(fd, content.c_str(), content.size()) == -1) {
+    auto error = errno;
+    LOG(ERROR) << "Could not save PID to file " << pid_file << ": "
+               << strerror(error);
+    exit(EXIT_FAILURE);
+  }
+
+  if (fsync(fd) == -1) {
+    auto error = errno;
+    LOG(ERROR) << "Could not save PID to file " << pid_file << ": "
+               << strerror(error);
+    exit(EXIT_FAILURE);
+  }
+
+  close(fd);
+
+  if (rename(temp_path, pid_file.c_str()) == -1) {
+    auto error = errno;
+    LOG(ERROR) << "Could not save PID to file " << pid_file << ": "
+               << strerror(error);
+
+    unlink(temp_path);
+
     exit(EXIT_FAILURE);
   }
 
   if (get_config()->uid != 0) {
-    if (chown_to_running_user(get_config()->pid_file.c_str()) == -1) {
+    if (chown_to_running_user(pid_file.c_str()) == -1) {
       auto error = errno;
-      LOG(WARN) << "Changing owner of pid file " << get_config()->pid_file
+      LOG(WARN) << "Changing owner of pid file " << pid_file
                 << " failed: " << strerror(error);
     }
   }
@@ -945,10 +941,6 @@ int event_loop() {
     redirect_stderr_to_errorlog();
   }
 
-  if (!get_config()->pid_file.empty()) {
-    save_pid();
-  }
-
   SignalServer ssv;
 
   rv = pipe(ssv.ipc_fd.data());
@@ -969,7 +961,7 @@ int event_loop() {
     return -1;
   }
 
-  auto loop = EV_DEFAULT;
+  auto loop = ev_default_loop(get_config()->ev_loop_flags);
 
   auto pid = fork_worker_process(&ssv);
 
@@ -995,6 +987,14 @@ int event_loop() {
   worker_process_childev.data = nullptr;
   ev_child_start(loop, &worker_process_childev);
 
+  // Write PID file when we are ready to accept connection from peer.
+  // This makes easier to write restart script for nghttpx.  Because
+  // when we know that PID file is recreated, it means we can send
+  // QUIT signal to the old process to make it shutdown gracefully.
+  if (!get_config()->pid_file.empty()) {
+    save_pid();
+  }
+
   ev_run(loop, 0);
 
   return 0;
@@ -1029,11 +1029,6 @@ constexpr auto DEFAULT_ACCESSLOG_FORMAT = StringRef::from_lit(
     R"("$http_referer" "$http_user_agent")");
 } // namespace
 
-namespace {
-constexpr char DEFAULT_DOWNSTREAM_HOST[] = "127.0.0.1";
-constexpr int16_t DEFAULT_DOWNSTREAM_PORT = 80;
-} // namespace;
-
 namespace {
 void fill_default_config() {
   *mod_config() = {};
@@ -1042,6 +1037,10 @@ void fill_default_config() {
   mod_config()->conf_path = "/etc/nghttpx/nghttpx.conf";
   mod_config()->pid = getpid();
 
+  if (ev_supported_backends() & ~ev_recommended_backends() & EVBACKEND_KQUEUE) {
+    mod_config()->ev_loop_flags = ev_recommended_backends() | EVBACKEND_KQUEUE;
+  }
+
   auto &tlsconf = mod_config()->tls;
   {
     auto &ticketconf = tlsconf.ticket;
@@ -1089,6 +1088,12 @@ void fill_default_config() {
   auto &http2conf = mod_config()->http2;
   {
     auto &upstreamconf = http2conf.upstream;
+
+    {
+      auto &timeoutconf = upstreamconf.timeout;
+      timeoutconf.settings = 10_s;
+    }
+
     // window bits for HTTP/2 and SPDY upstream connection per
     // stream. 2**16-1 = 64KiB-1, which is HTTP/2 default. Please note
     // that SPDY/3 default is 64KiB.
@@ -1101,10 +1106,21 @@ void fill_default_config() {
     nghttp2_option_new(&upstreamconf.option);
     nghttp2_option_set_no_auto_window_update(upstreamconf.option, 1);
     nghttp2_option_set_no_recv_client_magic(upstreamconf.option, 1);
+
+    // For API endpoint, we enable automatic window update.  This is
+    // because we are a sink.
+    nghttp2_option_new(&upstreamconf.alt_mode_option);
+    nghttp2_option_set_no_recv_client_magic(upstreamconf.alt_mode_option, 1);
   }
 
   {
     auto &downstreamconf = http2conf.downstream;
+
+    {
+      auto &timeoutconf = downstreamconf.timeout;
+      timeoutconf.settings = 10_s;
+    }
+
     downstreamconf.window_bits = 16;
     downstreamconf.connection_window_bits = 30;
     downstreamconf.max_concurrent_streams = 100;
@@ -1151,7 +1167,8 @@ void fill_default_config() {
   }
 
   {
-    auto &downstreamconf = connconf.downstream;
+    connconf.downstream = std::make_shared<DownstreamConfig>();
+    auto &downstreamconf = *connconf.downstream;
     {
       auto &timeoutconf = downstreamconf.timeout;
       // Read/Write timeouts for downstream connection
@@ -1159,6 +1176,7 @@ void fill_default_config() {
       timeoutconf.write = 30_s;
       // Timeout for pooled (idle) connections
       timeoutconf.idle_read = 2_s;
+      timeoutconf.max_backoff = 120_s;
     }
 
     downstreamconf.connections_per_host = 8;
@@ -1166,6 +1184,9 @@ void fill_default_config() {
     downstreamconf.response_buffer_size = 128_k;
     downstreamconf.family = AF_UNSPEC;
   }
+
+  auto &apiconf = mod_config()->api;
+  apiconf.max_request_body = 16_k;
 }
 
 } // namespace
@@ -1189,16 +1210,17 @@ void print_help(std::ostream &out) {
   out << R"(
   <PRIVATE_KEY>
               Set  path  to  server's private  key.   Required  unless
-              "no-tls" keyword is used in --frontend option.
+              "no-tls" parameter is used in --frontend option.
   <CERT>      Set  path  to  server's  certificate.   Required  unless
-              "no-tls" keyword is used  in --frontend option.  To make
-              OCSP stapling work, this must be an absolute path.
+              "no-tls"  parameter is  used in  --frontend option.   To
+              make OCSP stapling work, this must be an absolute path.
 
 Options:
   The options are categorized into several groups.
 
 Connections:
-  -b, --backend=(<HOST>,<PORT>|unix:<PATH>)[;[<PATTERN>[:...]][;proto=<PROTO>][;tls]]
+  -b, --backend=(<HOST>,<PORT>|unix:<PATH>)[;[<PATTERN>[:...]][[;PARAM]...]
+
               Set  backend  host  and   port.   The  multiple  backend
               addresses are  accepted by repeating this  option.  UNIX
               domain socket  can be  specified by prefixing  path name
@@ -1260,17 +1282,63 @@ Connections:
               The backend addresses sharing same <PATTERN> are grouped
               together forming  load balancing  group.
 
-              Optionally,   backend   application  protocol   can   be
-              specified in <PROTO>.  All that share the same <PATTERN>
-              must  have  the  same  <PROTO> value  if  it  is  given.
-              <PROTO>  should be  one  of the  following list  without
-              quotes: "h2", "http/1.1".  The  default value of <PROTO>
-              is "http/1.1".  Note that  usually "h2" refers to HTTP/2
-              over TLS.  But  in this option, it may  mean HTTP/2 over
-              cleartext TCP unless "tls" keyword is used (see below).
+              Several parameters <PARAM> are accepted after <PATTERN>.
+              The  parameters are  delimited  by  ";".  The  available
+              parameters       are:      "proto=<PROTO>",       "tls",
+              "sni=<SNI_HOST>",     "fall=<N>",    "rise=<N>",     and
+              "affinity=<METHOD>".  The parameter consists of keyword,
+              and optionally followed by  "=" and value.  For example,
+              the parameter "proto=h2" consists of the keyword "proto"
+              and  value "h2".   The parameter  "tls" consists  of the
+              keyword   "tls"  without   value.   Each   parameter  is
+              described as follows.
 
-              Optionally,  TLS  can  be enabled  by  specifying  "tls"
-              keyword.  TLS is not enabled by default.
+              The backend application protocol  can be specified using
+              optional  "proto"   parameter,  and   in  the   form  of
+              "proto=<PROTO>".  <PROTO> should be one of the following
+              list  without  quotes:  "h2", "http/1.1".   The  default
+              value of <PROTO> is  "http/1.1".  Note that usually "h2"
+              refers to HTTP/2  over TLS.  But in this  option, it may
+              mean HTTP/2  over cleartext TCP unless  "tls" keyword is
+              used (see below).
+
+              TLS  can   be  enabled  by  specifying   optional  "tls"
+              parameter.  TLS is not enabled by default.
+
+              With "sni=<SNI_HOST>" parameter, it can override the TLS
+              SNI  field  value  with  given  <SNI_HOST>.   This  will
+              default to the backend <HOST> name
+
+              The  feature  to detect  whether  backend  is online  or
+              offline can be enabled  using optional "fall" and "rise"
+              parameters.   Using  "fall=<N>"  parameter,  if  nghttpx
+              cannot connect  to a  this backend <N>  times in  a row,
+              this  backend  is  assumed  to be  offline,  and  it  is
+              excluded from load balancing.  If <N> is 0, this backend
+              never  be excluded  from load  balancing whatever  times
+              nghttpx cannot connect  to it, and this  is the default.
+              There is  also "rise=<N>" parameter.  After  backend was
+              excluded from load balancing group, nghttpx periodically
+              attempts to make a connection to the failed backend, and
+              if the  connection is made  successfully <N> times  in a
+              row, the backend is assumed to  be online, and it is now
+              eligible  for load  balancing target.   If <N>  is 0,  a
+              backend  is permanently  offline, once  it goes  in that
+              state, and this is the default behaviour.
+
+              The     session     affinity    is     enabled     using
+              "affinity=<METHOD>"  parameter.   If  "ip" is  given  in
+              <METHOD>, client  IP based session affinity  is enabled.
+              If  "none" is  given  in <METHOD>,  session affinity  is
+              disabled, and this is the default.  The session affinity
+              is enabled per  <PATTERN>.  If at least  one backend has
+              "affinity" parameter,  and its  <METHOD> is  not "none",
+              session  affinity is  enabled  for  all backend  servers
+              sharing  the  same  <PATTERN>.   It is  advised  to  set
+              "affinity"  parameter  to   all  backend  explicitly  if
+              session affinity  is desired.  The session  affinity may
+              break if one of the backend gets unreachable, or backend
+              settings are reloaded or replaced by API.
 
               Since ";" and ":" are  used as delimiter, <PATTERN> must
               not  contain these  characters.  Since  ";" has  special
@@ -1278,7 +1346,7 @@ Connections:
 
               Default: )" << DEFAULT_DOWNSTREAM_HOST << ","
       << DEFAULT_DOWNSTREAM_PORT << R"(
-  -f, --frontend=(<HOST>,<PORT>|unix:<PATH>)[;no-tls]
+  -f, --frontend=(<HOST>,<PORT>|unix:<PATH>)[[;PARAM]...]
               Set  frontend  host and  port.   If  <HOST> is  '*',  it
               assumes  all addresses  including  both  IPv4 and  IPv6.
               UNIX domain  socket can  be specified by  prefixing path
@@ -1286,8 +1354,24 @@ Connections:
               This  option can  be used  multiple times  to listen  to
               multiple addresses.
 
+              This option  can take  0 or  more parameters,  which are
+              described  below.   Note   that  "api"  and  "healthmon"
+              parameters are mutually exclusive.
+
               Optionally, TLS  can be disabled by  specifying "no-tls"
-              keyword.  TLS is enabled by default.
+              parameter.  TLS is enabled by default.
+
+              To  make this  frontend as  API endpoint,  specify "api"
+              parameter.   This   is  disabled  by  default.    It  is
+              important  to  limit the  access  to  the API  frontend.
+              Otherwise, someone  may change  the backend  server, and
+              break your services,  or expose confidential information
+              to the outside the world.
+
+              To  make  this  frontend  as  health  monitor  endpoint,
+              specify  "healthmon"  parameter.   This is  disabled  by
+              default.  Any  requests which come through  this address
+              are replied with 200 HTTP status, without no body.
 
               Default: *,3000
   --backlog=<N>
@@ -1375,7 +1459,7 @@ Performance:
               HTTP/2).   To  limit  the   number  of  connections  per
               frontend        for       default        mode,       use
               --backend-connections-per-frontend.
-              Default: )" << get_config()->conn.downstream.connections_per_host
+              Default: )" << get_config()->conn.downstream->connections_per_host
       << R"(
   --backend-connections-per-frontend=<N>
               Set  maximum number  of  backend concurrent  connections
@@ -1385,7 +1469,7 @@ Performance:
               with          --http2-proxy         option,          use
               --backend-connections-per-host.
               Default: )"
-      << get_config()->conn.downstream.connections_per_frontend << R"(
+      << get_config()->conn.downstream->connections_per_frontend << R"(
   --rlimit-nofile=<N>
               Set maximum number of open files (RLIMIT_NOFILE) to <N>.
               If 0 is given, nghttpx does not set the limit.
@@ -1393,12 +1477,12 @@ Performance:
   --backend-request-buffer=<SIZE>
               Set buffer size used to store backend request.
               Default: )"
-      << util::utos_unit(get_config()->conn.downstream.request_buffer_size)
+      << util::utos_unit(get_config()->conn.downstream->request_buffer_size)
       << R"(
   --backend-response-buffer=<SIZE>
               Set buffer size used to store backend response.
               Default: )"
-      << util::utos_unit(get_config()->conn.downstream.response_buffer_size)
+      << util::utos_unit(get_config()->conn.downstream->response_buffer_size)
       << R"(
   --fastopen=<N>
               Enables  "TCP Fast  Open" for  the listening  socket and
@@ -1406,6 +1490,10 @@ Performance:
               that have not yet completed the three-way handshake.  If
               value is 0 then fast open is disabled.
               Default: )" << get_config()->conn.listener.fastopen << R"(
+  --no-kqueue Don't use  kqueue.  This  option is only  applicable for
+              the platforms  which have kqueue.  For  other platforms,
+              this option will be simply ignored.
+
 Timeout:
   --frontend-http2-read-timeout=<DURATION>
               Specify  read  timeout  for  HTTP/2  and  SPDY  frontend
@@ -1434,15 +1522,15 @@ Timeout:
   --backend-read-timeout=<DURATION>
               Specify read timeout for backend connection.
               Default: )"
-      << util::duration_str(get_config()->conn.downstream.timeout.read) << R"(
+      << util::duration_str(get_config()->conn.downstream->timeout.read) << R"(
   --backend-write-timeout=<DURATION>
               Specify write timeout for backend connection.
               Default: )"
-      << util::duration_str(get_config()->conn.downstream.timeout.write) << R"(
+      << util::duration_str(get_config()->conn.downstream->timeout.write) << R"(
   --backend-keep-alive-timeout=<DURATION>
               Specify keep-alive timeout for backend connection.
               Default: )"
-      << util::duration_str(get_config()->conn.downstream.timeout.idle_read)
+      << util::duration_str(get_config()->conn.downstream->timeout.idle_read)
       << R"(
   --listener-disable-timeout=<DURATION>
               After accepting  connection failed,  connection listener
@@ -1450,6 +1538,30 @@ Timeout:
               disables this feature.
               Default: )"
       << util::duration_str(get_config()->conn.listener.timeout.sleep) << R"(
+  --frontend-http2-setting-timeout=<DURATION>
+              Specify  timeout before  SETTINGS ACK  is received  from
+              client.
+              Default: )"
+      << util::duration_str(get_config()->http2.upstream.timeout.settings)
+      << R"(
+  --backend-http2-settings-timeout=<DURATION>
+              Specify  timeout before  SETTINGS ACK  is received  from
+              backend server.
+              Default: )"
+      << util::duration_str(get_config()->http2.downstream.timeout.settings)
+      << R"(
+  --backend-max-backoff=<DURATION>
+              Specify  maximum backoff  interval.  This  is used  when
+              doing health  check against offline backend  (see "fail"
+              parameter  in --backend  option).   It is  also used  to
+              limit  the  maximum   interval  to  temporarily  disable
+              backend  when nghttpx  failed to  connect to  it.  These
+              intervals are calculated  using exponential backoff, and
+              consecutive failed attempts increase the interval.  This
+              option caps its maximum value.
+              Default: )"
+      << util::duration_str(get_config()->conn.downstream->timeout.max_backoff)
+      << R"(
 
 SSL/TLS:
   --ciphers=<SUITE>
@@ -1474,9 +1586,6 @@ SSL/TLS:
               indicated  by  client  using TLS  SNI  extension.   This
               option  can  be  used  multiple  times.   To  make  OCSP
               stapling work, <CERTPATH> must be absolute path.
-  --backend-tls-sni-field=<HOST>
-              Explicitly  set the  content of  the TLS  SNI extension.
-              This will default to the backend HOST name.
   --dh-param-file=<PATH>
               Path to file that contains  DH parameters in PEM format.
               Without  this   option,  DHE   cipher  suites   are  not
@@ -1546,7 +1655,7 @@ SSL/TLS:
               "TLS SESSION  TICKET RESUMPTION" section in  manual page
               to know the data format in memcached entry.  Optionally,
               memcached  connection  can  be  encrypted  with  TLS  by
-              specifying "tls" keyword.
+              specifying "tls" parameter.
   --tls-ticket-key-memcached-address-family=(auto|IPv4|IPv6)
               Specify address  family of memcached connections  to get
               TLS ticket keys.  If "auto" is given, both IPv4 and IPv6
@@ -1595,7 +1704,7 @@ SSL/TLS:
               cache.   This  enables   shared  session  cache  between
               multiple   nghttpx  instances.    Optionally,  memcached
               connection can be encrypted with TLS by specifying "tls"
-              keyword.
+              parameter.
   --tls-session-cache-memcached-address-family=(auto|IPv4|IPv6)
               Specify address family of memcached connections to store
               session cache.  If  "auto" is given, both  IPv4 and IPv6
@@ -1686,8 +1795,8 @@ HTTP/2 and SPDY:
 Mode:
   (default mode)
               Accept HTTP/2, SPDY and HTTP/1.1 over SSL/TLS.  "no-tls"
-              keyword is used in  --frontend option, accept HTTP/2 and
-              HTTP/1.1  over  cleartext  TCP.  The  incoming  HTTP/1.1
+              parameter is  used in  --frontend option,  accept HTTP/2
+              and HTTP/1.1 over cleartext  TCP.  The incoming HTTP/1.1
               connection  can  be  upgraded  to  HTTP/2  through  HTTP
               Upgrade.
   -s, --http2-proxy
@@ -1852,6 +1961,12 @@ HTTP:
               HTTP  status  code.  If  error  status  code comes  from
               backend server, the custom error pages are not used.
 
+API:
+  --api-max-request-body=<SIZE>
+              Set the maximum size of request body for API request.
+              Default: )" << util::utos_unit(get_config()->api.max_request_body)
+      << R"(
+
 Debug:
   --frontend-http2-dump-request-header=<PATH>
               Dumps request headers received by HTTP/2 frontend to the
@@ -1934,7 +2049,7 @@ void process_options(int argc, char **argv,
     std::set<StringRef> include_set;
 
     for (auto &p : cmdcfgs) {
-      if (parse_config(p.first, p.second, include_set) == -1) {
+      if (parse_config(mod_config(), p.first, p.second, include_set) == -1) {
         LOG(FATAL) << "Failed to parse command-line argument.";
         exit(EXIT_FAILURE);
       }
@@ -2035,9 +2150,10 @@ void process_options(int argc, char **argv,
 
   tlsconf.alpn_prefs = ssl::set_alpn_prefs(tlsconf.npn_list);
 
+  tlsconf.bio_method = create_bio_method();
+
   auto &listenerconf = mod_config()->conn.listener;
   auto &upstreamconf = mod_config()->conn.upstream;
-  auto &downstreamconf = mod_config()->conn.downstream;
 
   if (listenerconf.addrs.empty()) {
     UpstreamAddr addr{};
@@ -2071,151 +2187,11 @@ void process_options(int argc, char **argv,
     }
   }
 
-  auto &addr_groups = downstreamconf.addr_groups;
-
-  if (addr_groups.empty()) {
-    DownstreamAddrConfig addr{};
-    addr.host = ImmutableString::from_lit(DEFAULT_DOWNSTREAM_HOST);
-    addr.port = DEFAULT_DOWNSTREAM_PORT;
-
-    DownstreamAddrGroupConfig g(StringRef::from_lit("/"));
-    g.proto = PROTO_HTTP1;
-    g.addrs.push_back(std::move(addr));
-    mod_config()->router.add_route(StringRef{g.pattern}, addr_groups.size());
-    addr_groups.push_back(std::move(g));
-  } else if (get_config()->http2_proxy) {
-    // We don't support host mapping in these cases.  Move all
-    // non-catch-all patterns to catch-all pattern.
-    DownstreamAddrGroupConfig catch_all(StringRef::from_lit("/"));
-    auto proto = PROTO_NONE;
-    auto tls = false;
-    auto tls_seen = false;
-    for (auto &g : addr_groups) {
-      if (proto == PROTO_NONE) {
-        proto = g.proto;
-      } else if (proto != g.proto) {
-        LOG(ERROR) << SHRPX_OPT_BACKEND << ": <PATTERN> was ignored with "
-                                           "--http2-proxy, and protocol must "
-                                           "be the same for all backends.";
-        exit(EXIT_FAILURE);
-      }
-
-      if (!tls_seen) {
-        tls = g.tls;
-        tls_seen = true;
-      } else if (tls != g.tls) {
-        LOG(ERROR) << SHRPX_OPT_BACKEND
-                   << ": <PATTERN> was ignored with --http2-proxy, and tls "
-                      "must be enabled or disabled for all backends.";
-        exit(EXIT_FAILURE);
-      }
-
-      std::move(std::begin(g.addrs), std::end(g.addrs),
-                std::back_inserter(catch_all.addrs));
-    }
-    catch_all.proto = proto;
-    catch_all.tls = tls;
-    std::vector<DownstreamAddrGroupConfig>().swap(addr_groups);
-    std::vector<WildcardPattern>().swap(mod_config()->wildcard_patterns);
-    // maybe not necessary?
-    mod_config()->router = Router();
-    mod_config()->router.add_route(StringRef{catch_all.pattern},
-                                   addr_groups.size());
-    addr_groups.push_back(std::move(catch_all));
-  } else {
-    auto &wildcard_patterns = mod_config()->wildcard_patterns;
-    std::sort(std::begin(wildcard_patterns), std::end(wildcard_patterns),
-              [](const WildcardPattern &lhs, const WildcardPattern &rhs) {
-                return std::lexicographical_compare(
-                    rhs.host.rbegin(), rhs.host.rend(), lhs.host.rbegin(),
-                    lhs.host.rend());
-              });
-    if (LOG_ENABLED(INFO)) {
-      LOG(INFO) << "Reverse sorted wildcard hosts (compared from tail to head, "
-                   "and sorted in reverse order):";
-      for (auto &wp : mod_config()->wildcard_patterns) {
-        LOG(INFO) << wp.host;
-      }
-    }
-  }
-
-  if (LOG_ENABLED(INFO)) {
-    LOG(INFO) << "Resolving backend address";
-  }
-
-  ssize_t catch_all_group = -1;
-  for (size_t i = 0; i < addr_groups.size(); ++i) {
-    auto &g = addr_groups[i];
-    if (g.pattern == StringRef::from_lit("/")) {
-      catch_all_group = i;
-    }
-    if (LOG_ENABLED(INFO)) {
-      LOG(INFO) << "Host-path pattern: group " << i << ": '" << g.pattern
-                << "', proto=" << strproto(g.proto) << (g.tls ? ", tls" : "");
-      for (auto &addr : g.addrs) {
-        LOG(INFO) << "group " << i << " -> " << addr.host.c_str()
-                  << (addr.host_unix ? "" : ":" + util::utos(addr.port));
-      }
-    }
-  }
-
-  if (catch_all_group == -1) {
-    LOG(FATAL) << "backend: No catch-all backend address is configured";
+  if (configure_downstream_group(mod_config(), get_config()->http2_proxy, false,
+                                 tlsconf) != 0) {
     exit(EXIT_FAILURE);
   }
 
-  downstreamconf.addr_group_catch_all = catch_all_group;
-
-  if (LOG_ENABLED(INFO)) {
-    LOG(INFO) << "Catch-all pattern is group " << catch_all_group;
-  }
-
-  for (auto &g : addr_groups) {
-    for (auto &addr : g.addrs) {
-
-      if (addr.host_unix) {
-        // for AF_UNIX socket, we use "localhost" as host for backend
-        // hostport.  This is used as Host header field to backend and
-        // not going to be passed to any syscalls.
-        addr.hostport = "localhost";
-
-        auto path = addr.host.c_str();
-        auto pathlen = addr.host.size();
-
-        if (pathlen + 1 > sizeof(addr.addr.su.un.sun_path)) {
-          LOG(FATAL) << "UNIX domain socket path " << path << " is too long > "
-                     << sizeof(addr.addr.su.un.sun_path);
-          exit(EXIT_FAILURE);
-        }
-
-        if (LOG_ENABLED(INFO)) {
-          LOG(INFO) << "Use UNIX domain socket path " << path
-                    << " for backend connection";
-        }
-
-        addr.addr.su.un.sun_family = AF_UNIX;
-        // copy path including terminal NULL
-        std::copy_n(path, pathlen + 1, addr.addr.su.un.sun_path);
-        addr.addr.len = sizeof(addr.addr.su.un);
-
-        continue;
-      }
-
-      addr.hostport = ImmutableString(
-          util::make_http_hostport(StringRef(addr.host), addr.port));
-
-      auto hostport = util::make_hostport(StringRef{addr.host}, addr.port);
-
-      if (resolve_hostname(&addr.addr, addr.host.c_str(), addr.port,
-                           downstreamconf.family) == -1) {
-        LOG(FATAL) << "Resolving backend address failed: " << hostport;
-        exit(EXIT_FAILURE);
-      }
-      LOG(NOTICE) << "Resolved backend address: " << hostport << " -> "
-                  << util::to_numeric_addr(&addr.addr);
-    }
-  }
-
   auto &proxy = mod_config()->downstream_http_proxy;
   if (!proxy.host.empty()) {
     auto hostport = util::make_hostport(StringRef{proxy.host}, proxy.port);
@@ -2519,6 +2495,13 @@ int main(int argc, char **argv) {
         {SHRPX_OPT_BACKEND_CONNECTIONS_PER_HOST.c_str(), required_argument,
          &flag, 121},
         {SHRPX_OPT_ERROR_PAGE.c_str(), required_argument, &flag, 122},
+        {SHRPX_OPT_NO_KQUEUE.c_str(), no_argument, &flag, 123},
+        {SHRPX_OPT_FRONTEND_HTTP2_SETTINGS_TIMEOUT.c_str(), required_argument,
+         &flag, 124},
+        {SHRPX_OPT_BACKEND_HTTP2_SETTINGS_TIMEOUT.c_str(), required_argument,
+         &flag, 125},
+        {SHRPX_OPT_API_MAX_REQUEST_BODY.c_str(), required_argument, &flag, 126},
+        {SHRPX_OPT_BACKEND_MAX_BACKOFF.c_str(), required_argument, &flag, 127},
         {nullptr, 0, nullptr, 0}};
 
     int option_index = 0;
@@ -3094,6 +3077,28 @@ int main(int argc, char **argv) {
         // --error-page
         cmdcfgs.emplace_back(SHRPX_OPT_ERROR_PAGE, StringRef{optarg});
         break;
+      case 123:
+        // --no-kqueue
+        cmdcfgs.emplace_back(SHRPX_OPT_NO_KQUEUE, StringRef::from_lit("yes"));
+        break;
+      case 124:
+        // --frontend-http2-settings-timeout
+        cmdcfgs.emplace_back(SHRPX_OPT_FRONTEND_HTTP2_SETTINGS_TIMEOUT,
+                             StringRef{optarg});
+        break;
+      case 125:
+        // --backend-http2-settings-timeout
+        cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_HTTP2_SETTINGS_TIMEOUT,
+                             StringRef{optarg});
+        break;
+      case 126:
+        // --api-max-request-body
+        cmdcfgs.emplace_back(SHRPX_OPT_API_MAX_REQUEST_BODY, StringRef{optarg});
+        break;
+      case 127:
+        // --backend-max-backoff
+        cmdcfgs.emplace_back(SHRPX_OPT_BACKEND_MAX_BACKOFF, StringRef{optarg});
+        break;
       default:
         break;
       }

src/shrpx_accept_handler.cc

@@ -58,51 +58,49 @@ AcceptHandler::~AcceptHandler() {
 }
 
 void AcceptHandler::accept_connection() {
-  for (;;) {
-    sockaddr_union sockaddr;
-    socklen_t addrlen = sizeof(sockaddr);
+  sockaddr_union sockaddr;
+  socklen_t addrlen = sizeof(sockaddr);
 
 #ifdef HAVE_ACCEPT4
-    auto cfd = accept4(faddr_->fd, &sockaddr.sa, &addrlen,
-                       SOCK_NONBLOCK | SOCK_CLOEXEC);
-#else // !HAVE_ACCEPT4
-    auto cfd = accept(faddr_->fd, &sockaddr.sa, &addrlen);
+  auto cfd =
+      accept4(faddr_->fd, &sockaddr.sa, &addrlen, SOCK_NONBLOCK | SOCK_CLOEXEC);
+#else  // !HAVE_ACCEPT4
+  auto cfd = accept(faddr_->fd, &sockaddr.sa, &addrlen);
 #endif // !HAVE_ACCEPT4
 
-    if (cfd == -1) {
-      switch (errno) {
-      case EINTR:
-      case ENETDOWN:
-      case EPROTO:
-      case ENOPROTOOPT:
-      case EHOSTDOWN:
+  if (cfd == -1) {
+    switch (errno) {
+    case EINTR:
+    case ENETDOWN:
+    case EPROTO:
+    case ENOPROTOOPT:
+    case EHOSTDOWN:
 #ifdef ENONET
-      case ENONET:
+    case ENONET:
 #endif // ENONET
-      case EHOSTUNREACH:
-      case EOPNOTSUPP:
-      case ENETUNREACH:
-        continue;
-      case EMFILE:
-      case ENFILE:
-        LOG(WARN) << "acceptor: running out file descriptor; disable acceptor "
-                     "temporarily";
-        conn_hnr_->sleep_acceptor(get_config()->conn.listener.timeout.sleep);
-        break;
-      }
-
-      break;
+    case EHOSTUNREACH:
+    case EOPNOTSUPP:
+    case ENETUNREACH:
+      return;
+    case EMFILE:
+    case ENFILE:
+      LOG(WARN) << "acceptor: running out file descriptor; disable acceptor "
+                   "temporarily";
+      conn_hnr_->sleep_acceptor(get_config()->conn.listener.timeout.sleep);
+      return;
+    default:
+      return;
     }
+  }
 
 #ifndef HAVE_ACCEPT4
-    util::make_socket_nonblocking(cfd);
-    util::make_socket_closeonexec(cfd);
+  util::make_socket_nonblocking(cfd);
+  util::make_socket_closeonexec(cfd);
 #endif // !HAVE_ACCEPT4
 
-    util::make_socket_nodelay(cfd);
+  util::make_socket_nodelay(cfd);
 
-    conn_hnr_->handle_connection(cfd, &sockaddr.sa, addrlen, faddr_);
-  }
+  conn_hnr_->handle_connection(cfd, &sockaddr.sa, addrlen, faddr_);
 }
 
 void AcceptHandler::enable() { ev_io_start(conn_hnr_->get_loop(), &wev_); }

src/shrpx_api_downstream_connection.cc

@@ -0,0 +1,311 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2016 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "shrpx_api_downstream_connection.h"
+
+#include "shrpx_client_handler.h"
+#include "shrpx_upstream.h"
+#include "shrpx_downstream.h"
+#include "shrpx_worker.h"
+#include "shrpx_connection_handler.h"
+
+namespace shrpx {
+
+APIDownstreamConnection::APIDownstreamConnection(Worker *worker)
+    : worker_(worker), abandoned_(false) {}
+
+APIDownstreamConnection::~APIDownstreamConnection() {}
+
+int APIDownstreamConnection::attach_downstream(Downstream *downstream) {
+  if (LOG_ENABLED(INFO)) {
+    DCLOG(INFO, this) << "Attaching to DOWNSTREAM:" << downstream;
+  }
+
+  downstream_ = downstream;
+
+  return 0;
+}
+
+void APIDownstreamConnection::detach_downstream(Downstream *downstream) {
+  if (LOG_ENABLED(INFO)) {
+    DCLOG(INFO, this) << "Detaching from DOWNSTREAM:" << downstream;
+  }
+  downstream_ = nullptr;
+}
+
+// API status, which is independent from HTTP status code.  But
+// generally, 2xx code for API_SUCCESS, and otherwise API_FAILURE.
+enum {
+  API_SUCCESS,
+  API_FAILURE,
+};
+
+int APIDownstreamConnection::send_reply(unsigned int http_status,
+                                        int api_status) {
+  abandoned_ = true;
+
+  auto upstream = downstream_->get_upstream();
+
+  auto &resp = downstream_->response();
+
+  resp.http_status = http_status;
+
+  auto &balloc = downstream_->get_block_allocator();
+
+  StringRef api_status_str;
+
+  switch (api_status) {
+  case API_SUCCESS:
+    api_status_str = StringRef::from_lit("Success");
+    break;
+  case API_FAILURE:
+    api_status_str = StringRef::from_lit("Failure");
+    break;
+  default:
+    assert(0);
+  }
+
+  constexpr auto M1 = StringRef::from_lit("{\"status\":\"");
+  constexpr auto M2 = StringRef::from_lit("\",\"code\":");
+  constexpr auto M3 = StringRef::from_lit("}");
+
+  // 3 is the number of digits in http_status, assuming it is 3 digits
+  // number.
+  auto buflen = M1.size() + M2.size() + M3.size() + api_status_str.size() + 3;
+
+  auto buf = make_byte_ref(balloc, buflen);
+  auto p = buf.base;
+
+  p = std::copy(std::begin(M1), std::end(M1), p);
+  p = std::copy(std::begin(api_status_str), std::end(api_status_str), p);
+  p = std::copy(std::begin(M2), std::end(M2), p);
+  p = util::utos(p, http_status);
+  p = std::copy(std::begin(M3), std::end(M3), p);
+
+  buf.len = p - buf.base;
+
+  auto content_length = util::make_string_ref_uint(balloc, buf.len);
+
+  resp.fs.add_header_token(StringRef::from_lit("content-length"),
+                           content_length, false, http2::HD_CONTENT_LENGTH);
+
+  switch (http_status) {
+  case 400:
+  case 405:
+  case 413:
+    resp.fs.add_header_token(StringRef::from_lit("connection"),
+                             StringRef::from_lit("close"), false,
+                             http2::HD_CONNECTION);
+    break;
+  }
+
+  if (upstream->send_reply(downstream_, buf.base, buf.len) != 0) {
+    return -1;
+  }
+
+  return 0;
+}
+
+int APIDownstreamConnection::push_request_headers() {
+  auto &req = downstream_->request();
+  auto &resp = downstream_->response();
+
+  auto path =
+      StringRef{std::begin(req.path),
+                std::find(std::begin(req.path), std::end(req.path), '?')};
+
+  if (path != StringRef::from_lit("/api/v1beta1/backendconfig")) {
+    send_reply(404, API_FAILURE);
+
+    return 0;
+  }
+
+  if (req.method != HTTP_POST && req.method != HTTP_PUT) {
+    resp.fs.add_header_token(StringRef::from_lit("allow"),
+                             StringRef::from_lit("POST, PUT"), false, -1);
+    send_reply(405, API_FAILURE);
+
+    return 0;
+  }
+
+  // This works with req.fs.content_length == -1
+  if (req.fs.content_length >
+      static_cast<int64_t>(get_config()->api.max_request_body)) {
+    send_reply(413, API_FAILURE);
+
+    return 0;
+  }
+
+  return 0;
+}
+
+int APIDownstreamConnection::push_upload_data_chunk(const uint8_t *data,
+                                                    size_t datalen) {
+  if (abandoned_) {
+    return 0;
+  }
+
+  auto output = downstream_->get_request_buf();
+
+  auto &apiconf = get_config()->api;
+
+  if (output->rleft() + datalen > apiconf.max_request_body) {
+    send_reply(413, API_FAILURE);
+
+    return 0;
+  }
+
+  output->append(data, datalen);
+
+  // We don't have to call Upstream::resume_read() here, because
+  // request buffer is effectively unlimited.  Actually, we cannot
+  // call it here since it could recursively call this function again.
+
+  return 0;
+}
+
+int APIDownstreamConnection::end_upload_data() {
+  if (abandoned_) {
+    return 0;
+  }
+
+  auto output = downstream_->get_request_buf();
+
+  std::array<struct iovec, 2> iov;
+  auto iovcnt = output->riovec(iov.data(), 2);
+
+  if (iovcnt == 0) {
+    send_reply(200, API_SUCCESS);
+
+    return 0;
+  }
+
+  std::unique_ptr<uint8_t[]> large_buf;
+
+  // If data spans in multiple chunks, pull them up into one
+  // contiguous buffer.
+  if (iovcnt > 1) {
+    large_buf = make_unique<uint8_t[]>(output->rleft());
+    auto len = output->rleft();
+    output->remove(large_buf.get(), len);
+
+    iov[0].iov_base = large_buf.get();
+    iov[0].iov_len = len;
+  }
+
+  Config config{};
+  config.conn.downstream = std::make_shared<DownstreamConfig>();
+  const auto &downstreamconf = config.conn.downstream;
+
+  auto &src = get_config()->conn.downstream;
+
+  downstreamconf->timeout = src->timeout;
+  downstreamconf->connections_per_host = src->connections_per_host;
+  downstreamconf->connections_per_frontend = src->connections_per_frontend;
+  downstreamconf->request_buffer_size = src->request_buffer_size;
+  downstreamconf->response_buffer_size = src->response_buffer_size;
+  downstreamconf->family = src->family;
+
+  std::set<StringRef> include_set;
+
+  for (auto first = reinterpret_cast<const uint8_t *>(iov[0].iov_base),
+            last = first + iov[0].iov_len;
+       first != last;) {
+    auto eol = std::find(first, last, '\n');
+    if (eol == last) {
+      break;
+    }
+
+    if (first == eol || *first == '#') {
+      first = ++eol;
+      continue;
+    }
+
+    auto eq = std::find(first, eol, '=');
+    if (eq == eol) {
+      send_reply(400, API_FAILURE);
+      return 0;
+    }
+
+    auto opt = StringRef{first, eq};
+    auto optval = StringRef{eq + 1, eol};
+
+    auto optid = option_lookup_token(opt.c_str(), opt.size());
+
+    switch (optid) {
+    case SHRPX_OPTID_BACKEND:
+      break;
+    default:
+      first = ++eol;
+      continue;
+    }
+
+    if (parse_config(&config, optid, opt, optval, include_set) != 0) {
+      send_reply(400, API_FAILURE);
+      return 0;
+    }
+
+    first = ++eol;
+  }
+
+  auto &tlsconf = get_config()->tls;
+  if (configure_downstream_group(&config, get_config()->http2_proxy, true,
+                                 tlsconf) != 0) {
+    send_reply(400, API_FAILURE);
+    return 0;
+  }
+
+  auto conn_handler = worker_->get_connection_handler();
+
+  conn_handler->send_replace_downstream(downstreamconf);
+
+  send_reply(200, API_SUCCESS);
+
+  return 0;
+}
+
+void APIDownstreamConnection::pause_read(IOCtrlReason reason) {}
+
+int APIDownstreamConnection::resume_read(IOCtrlReason reason, size_t consumed) {
+  return 0;
+}
+
+void APIDownstreamConnection::force_resume_read() {}
+
+int APIDownstreamConnection::on_read() { return 0; }
+
+int APIDownstreamConnection::on_write() { return 0; }
+
+void APIDownstreamConnection::on_upstream_change(Upstream *uptream) {}
+
+bool APIDownstreamConnection::poolable() const { return false; }
+
+DownstreamAddrGroup *
+APIDownstreamConnection::get_downstream_addr_group() const {
+  return nullptr;
+}
+
+DownstreamAddr *APIDownstreamConnection::get_addr() const { return nullptr; }
+
+} // namespace shrpx

src/shrpx_api_downstream_connection.h

@@ -0,0 +1,69 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2016 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef SHRPX_API_DOWNSTREAM_CONNECTION_H
+#define SHRPX_API_DOWNSTREAM_CONNECTION_H
+
+#include "shrpx_downstream_connection.h"
+
+namespace shrpx {
+
+class Worker;
+
+class APIDownstreamConnection : public DownstreamConnection {
+public:
+  APIDownstreamConnection(Worker *worker);
+  virtual ~APIDownstreamConnection();
+  virtual int attach_downstream(Downstream *downstream);
+  virtual void detach_downstream(Downstream *downstream);
+
+  virtual int push_request_headers();
+  virtual int push_upload_data_chunk(const uint8_t *data, size_t datalen);
+  virtual int end_upload_data();
+
+  virtual void pause_read(IOCtrlReason reason);
+  virtual int resume_read(IOCtrlReason reason, size_t consumed);
+  virtual void force_resume_read();
+
+  virtual int on_read();
+  virtual int on_write();
+
+  virtual void on_upstream_change(Upstream *uptream);
+
+  // true if this object is poolable.
+  virtual bool poolable() const;
+
+  virtual DownstreamAddrGroup *get_downstream_addr_group() const;
+  virtual DownstreamAddr *get_addr() const;
+
+  int send_reply(unsigned int http_status, int api_status);
+
+private:
+  Worker *worker_;
+  bool abandoned_;
+};
+
+} // namespace shrpx
+
+#endif // SHRPX_API_DOWNSTREAM_CONNECTION_H

src/shrpx_client_handler.cc

@@ -47,6 +47,9 @@
 #include "shrpx_downstream_connection_pool.h"
 #include "shrpx_downstream.h"
 #include "shrpx_http2_session.h"
+#include "shrpx_connect_blocker.h"
+#include "shrpx_api_downstream_connection.h"
+#include "shrpx_health_monitor_downstream_connection.h"
 #ifdef HAVE_SPDYLAY
 #include "shrpx_spdy_upstream.h"
 #endif // HAVE_SPDYLAY
@@ -122,10 +125,6 @@ int ClientHandler::read_clear() {
       rb_.reset();
     } else if (rb_.wleft() == 0) {
       conn_.rlimit.stopw();
-      if (reset_conn_rtimer_required_) {
-        reset_conn_rtimer_required_ = false;
-        ev_timer_again(conn_.loop, &conn_.rt);
-      }
       return 0;
     }
 
@@ -136,10 +135,6 @@ int ClientHandler::read_clear() {
     auto nread = conn_.read_clear(rb_.last, rb_.wleft());
 
     if (nread == 0) {
-      if (reset_conn_rtimer_required_) {
-        reset_conn_rtimer_required_ = false;
-        ev_timer_again(conn_.loop, &conn_.rt);
-      }
       return 0;
     }
 
@@ -154,8 +149,6 @@ int ClientHandler::read_clear() {
 int ClientHandler::write_clear() {
   std::array<iovec, 2> iov;
 
-  ev_timer_again(conn_.loop, &conn_.rt);
-
   for (;;) {
     if (on_write() != 0) {
       return -1;
@@ -225,11 +218,6 @@ int ClientHandler::read_tls() {
       rb_.reset();
     } else if (rb_.wleft() == 0) {
       conn_.rlimit.stopw();
-      if (reset_conn_rtimer_required_) {
-        reset_conn_rtimer_required_ = false;
-        ev_timer_again(conn_.loop, &conn_.rt);
-      }
-
       return 0;
     }
 
@@ -240,11 +228,6 @@ int ClientHandler::read_tls() {
     auto nread = conn_.read_tls(rb_.last, rb_.wleft());
 
     if (nread == 0) {
-      if (reset_conn_rtimer_required_) {
-        reset_conn_rtimer_required_ = false;
-        ev_timer_again(conn_.loop, &conn_.rt);
-      }
-
       return 0;
     }
 
@@ -259,8 +242,6 @@ int ClientHandler::read_tls() {
 int ClientHandler::write_tls() {
   struct iovec iov;
 
-  ev_timer_again(conn_.loop, &conn_.rt);
-
   ERR_clear_error();
 
   for (;;) {
@@ -405,8 +386,9 @@ ClientHandler::ClientHandler(Worker *worker, int fd, SSL *ssl,
       faddr_(faddr),
       worker_(worker),
       left_connhd_len_(NGHTTP2_CLIENT_MAGIC_LEN),
+      affinity_hash_(0),
       should_close_after_write_(false),
-      reset_conn_rtimer_required_(false) {
+      affinity_hash_computed_(false) {
 
   ++worker_->get_worker_stat()->num_connections;
 
@@ -512,13 +494,15 @@ void ClientHandler::reset_upstream_write_timeout(ev_tstamp t) {
   }
 }
 
-void ClientHandler::signal_reset_upstream_conn_rtimer() {
-  reset_conn_rtimer_required_ = true;
+void ClientHandler::repeat_read_timer() {
+  ev_timer_again(conn_.loop, &conn_.rt);
 }
 
+void ClientHandler::stop_read_timer() { ev_timer_stop(conn_.loop, &conn_.rt); }
+
 int ClientHandler::validate_next_proto() {
   const unsigned char *next_proto = nullptr;
-  unsigned int next_proto_len;
+  unsigned int next_proto_len = 0;
 
   // First set callback for catch all cases
   on_read_ = &ClientHandler::upstream_read;
@@ -666,8 +650,18 @@ void ClientHandler::pool_downstream_connection(
                      << " in group " << group;
   }
 
-  auto &dconn_pool = group->shared_addr->dconn_pool;
-  dconn_pool.add_downstream_connection(std::move(dconn));
+  auto &shared_addr = group->shared_addr;
+
+  if (shared_addr->affinity == AFFINITY_NONE) {
+    auto &dconn_pool = group->shared_addr->dconn_pool;
+    dconn_pool.add_downstream_connection(std::move(dconn));
+
+    return;
+  }
+
+  auto addr = dconn->get_addr();
+  auto &dconn_pool = addr->dconn_pool;
+  dconn_pool->add_downstream_connection(std::move(dconn));
 }
 
 void ClientHandler::remove_downstream_connection(DownstreamConnection *dconn) {
@@ -680,15 +674,214 @@ void ClientHandler::remove_downstream_connection(DownstreamConnection *dconn) {
   dconn_pool.remove_downstream_connection(dconn);
 }
 
+namespace {
+// Computes 32bits hash for session affinity for IP address |ip|.
+uint32_t compute_affinity_from_ip(const StringRef &ip) {
+  int rv;
+  std::array<uint8_t, 32> buf;
+
+  rv = util::sha256(buf.data(), ip);
+  if (rv != 0) {
+    // Not sure when sha256 failed.  Just fall back to another
+    // function.
+    return util::hash32(ip);
+  }
+
+  return (static_cast<uint32_t>(buf[0]) << 24) |
+         (static_cast<uint32_t>(buf[1]) << 16) |
+         (static_cast<uint32_t>(buf[2]) << 8) | static_cast<uint32_t>(buf[3]);
+}
+} // namespace
+
+Http2Session *ClientHandler::select_http2_session_with_affinity(
+    const std::shared_ptr<DownstreamAddrGroup> &group, DownstreamAddr *addr) {
+  auto &shared_addr = group->shared_addr;
+
+  if (LOG_ENABLED(INFO)) {
+    CLOG(INFO, this) << "Selected DownstreamAddr=" << addr
+                     << ", index=" << (addr - shared_addr->addrs.data());
+  }
+
+  if (addr->http2_extra_freelist.size()) {
+    auto session = addr->http2_extra_freelist.head;
+
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "Use Http2Session " << session
+                       << " from http2_extra_freelist";
+    }
+
+    if (session->max_concurrency_reached(1)) {
+      if (LOG_ENABLED(INFO)) {
+        CLOG(INFO, this) << "Maximum streams are reached for Http2Session("
+                         << session << ").";
+      }
+
+      session->remove_from_freelist();
+    }
+    return session;
+  }
+
+  auto session = new Http2Session(conn_.loop, worker_->get_cl_ssl_ctx(),
+                                  worker_, group, addr);
+
+  if (LOG_ENABLED(INFO)) {
+    CLOG(INFO, this) << "Create new Http2Session " << session;
+  }
+
+  session->add_to_extra_freelist();
+
+  return session;
+}
+
+namespace {
+// Returns true if load of |lhs| is lighter than that of |rhs|.
+// Currently, we assume that lesser streams means lesser load.
+bool load_lighter(const DownstreamAddr *lhs, const DownstreamAddr *rhs) {
+  return lhs->num_dconn < rhs->num_dconn;
+}
+} // namespace
+
+Http2Session *ClientHandler::select_http2_session(
+    const std::shared_ptr<DownstreamAddrGroup> &group) {
+  auto &shared_addr = group->shared_addr;
+
+  // First count the working backend addresses.
+  size_t min = 0;
+  for (const auto &addr : shared_addr->addrs) {
+    if (addr.proto != PROTO_HTTP2 || addr.connect_blocker->blocked()) {
+      continue;
+    }
+
+    ++min;
+  }
+
+  if (min == 0) {
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "No working backend address found";
+    }
+
+    return nullptr;
+  }
+
+  auto &http2_avail_freelist = shared_addr->http2_avail_freelist;
+
+  if (http2_avail_freelist.size() >= min) {
+    auto session = http2_avail_freelist.head;
+    session->remove_from_freelist();
+
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "Use Http2Session " << session
+                       << " from http2_avail_freelist";
+    }
+
+    if (session->max_concurrency_reached(1)) {
+      if (LOG_ENABLED(INFO)) {
+        CLOG(INFO, this) << "Maximum streams are reached for Http2Session("
+                         << session << ").";
+      }
+    } else {
+      session->add_to_avail_freelist();
+    }
+    return session;
+  }
+
+  DownstreamAddr *selected_addr = nullptr;
+
+  for (auto &addr : shared_addr->addrs) {
+    if (addr.proto != PROTO_HTTP2 || (addr.http2_extra_freelist.size() == 0 &&
+                                      addr.connect_blocker->blocked())) {
+      continue;
+    }
+
+    if (addr.in_avail) {
+      continue;
+    }
+
+    if (selected_addr == nullptr || load_lighter(&addr, selected_addr)) {
+      selected_addr = &addr;
+    }
+  }
+
+  assert(selected_addr);
+
+  if (LOG_ENABLED(INFO)) {
+    CLOG(INFO, this) << "Selected DownstreamAddr=" << selected_addr
+                     << ", index="
+                     << (selected_addr - shared_addr->addrs.data());
+  }
+
+  if (selected_addr->http2_extra_freelist.size()) {
+    auto session = selected_addr->http2_extra_freelist.head;
+    session->remove_from_freelist();
+
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "Use Http2Session " << session
+                       << " from http2_extra_freelist";
+    }
+
+    if (session->max_concurrency_reached(1)) {
+      if (LOG_ENABLED(INFO)) {
+        CLOG(INFO, this) << "Maximum streams are reached for Http2Session("
+                         << session << ").";
+      }
+    } else {
+      session->add_to_avail_freelist();
+    }
+    return session;
+  }
+
+  auto session = new Http2Session(conn_.loop, worker_->get_cl_ssl_ctx(),
+                                  worker_, group, selected_addr);
+
+  if (LOG_ENABLED(INFO)) {
+    CLOG(INFO, this) << "Create new Http2Session " << session;
+  }
+
+  session->add_to_avail_freelist();
+
+  return session;
+}
+
+namespace {
+// The chosen value is small enough for uint32_t, and large enough for
+// the number of backend.
+constexpr uint32_t WEIGHT_MAX = 65536;
+} // namespace
+
+namespace {
+bool pri_less(const WeightedPri &lhs, const WeightedPri &rhs) {
+  if (lhs.cycle < rhs.cycle) {
+    return rhs.cycle - lhs.cycle <= WEIGHT_MAX;
+  }
+
+  return lhs.cycle - rhs.cycle > WEIGHT_MAX;
+}
+} // namespace
+
+namespace {
+uint32_t next_cycle(const WeightedPri &pri) {
+  return pri.cycle + WEIGHT_MAX / std::min(WEIGHT_MAX, pri.weight);
+}
+} // namespace
+
 std::unique_ptr<DownstreamConnection>
 ClientHandler::get_downstream_connection(Downstream *downstream) {
   size_t group_idx;
-  auto &downstreamconf = get_config()->conn.downstream;
+  auto &downstreamconf = *worker_->get_downstream_config();
+  auto &routerconf = downstreamconf.router;
+
   auto catch_all = downstreamconf.addr_group_catch_all;
   auto &groups = worker_->get_downstream_addr_groups();
 
   const auto &req = downstream->request();
 
+  switch (faddr_->alt_mode) {
+  case ALTMODE_API:
+    return make_unique<APIDownstreamConnection>(worker_);
+  case ALTMODE_HEALTHMON:
+    return make_unique<HealthMonitorDownstreamConnection>();
+  }
+
   // Fast path.  If we have one group, it must be catch-all group.
   // proxy mode falls in this case.
   if (groups.size() == 1) {
@@ -699,21 +892,19 @@ ClientHandler::get_downstream_connection(Downstream *downstream) {
     //  have dealt with proxy case already, just use catch-all group.
     group_idx = catch_all;
   } else {
-    auto &router = get_config()->router;
-    auto &wildcard_patterns = get_config()->wildcard_patterns;
+    auto &balloc = downstream->get_block_allocator();
+
     if (!req.authority.empty()) {
-      group_idx =
-          match_downstream_addr_group(router, wildcard_patterns, req.authority,
-                                      req.path, groups, catch_all);
+      group_idx = match_downstream_addr_group(
+          routerconf, req.authority, req.path, groups, catch_all, balloc);
     } else {
       auto h = req.fs.header(http2::HD_HOST);
       if (h) {
-        group_idx = match_downstream_addr_group(
-            router, wildcard_patterns, h->value, req.path, groups, catch_all);
+        group_idx = match_downstream_addr_group(routerconf, h->value, req.path,
+                                                groups, catch_all, balloc);
       } else {
-        group_idx =
-            match_downstream_addr_group(router, wildcard_patterns, StringRef{},
-                                        req.path, groups, catch_all);
+        group_idx = match_downstream_addr_group(
+            routerconf, StringRef{}, req.path, groups, catch_all, balloc);
       }
     }
   }
@@ -722,66 +913,121 @@ ClientHandler::get_downstream_connection(Downstream *downstream) {
     CLOG(INFO, this) << "Downstream address group_idx: " << group_idx;
   }
 
-  auto &group = worker_->get_downstream_addr_groups()[group_idx];
-  auto &shared_addr = group.shared_addr;
-  auto &dconn_pool = shared_addr->dconn_pool;
+  auto &group = groups[group_idx];
+  auto &shared_addr = group->shared_addr;
 
-  auto dconn = dconn_pool.pop_downstream_connection();
+  if (shared_addr->affinity == AFFINITY_IP) {
+    if (!affinity_hash_computed_) {
+      affinity_hash_ = compute_affinity_from_ip(StringRef{ipaddr_});
+      affinity_hash_computed_ = true;
+    }
 
-  if (!dconn) {
+    const auto &affinity_hash = shared_addr->affinity_hash;
+
+    auto it = std::lower_bound(
+        std::begin(affinity_hash), std::end(affinity_hash), affinity_hash_,
+        [](const AffinityHash &lhs, uint32_t rhs) { return lhs.hash < rhs; });
+
+    if (it == std::end(affinity_hash)) {
+      it = std::begin(affinity_hash);
+    }
+
+    auto idx = (*it).idx;
+
+    auto &addr = shared_addr->addrs[idx];
+    if (addr.proto == PROTO_HTTP2) {
+      auto http2session = select_http2_session_with_affinity(group, &addr);
+
+      auto dconn = make_unique<Http2DownstreamConnection>(http2session);
+
+      dconn->set_client_handler(this);
+
+      return std::move(dconn);
+    }
+
+    auto &dconn_pool = addr.dconn_pool;
+    auto dconn = dconn_pool->pop_downstream_connection();
+
+    if (!dconn) {
+      dconn = make_unique<HttpDownstreamConnection>(group, idx, conn_.loop,
+                                                    worker_);
+    }
+
+    dconn->set_client_handler(this);
+
+    return dconn;
+  }
+
+  auto http1_weight = shared_addr->http1_pri.weight;
+  auto http2_weight = shared_addr->http2_pri.weight;
+
+  auto proto = PROTO_NONE;
+
+  if (http1_weight > 0 && http2_weight > 0) {
+    // We only advance cycle if both weight has nonzero to keep its
+    // distance under WEIGHT_MAX.
+    if (pri_less(shared_addr->http1_pri, shared_addr->http2_pri)) {
+      proto = PROTO_HTTP1;
+      shared_addr->http1_pri.cycle = next_cycle(shared_addr->http1_pri);
+    } else {
+      proto = PROTO_HTTP2;
+      shared_addr->http2_pri.cycle = next_cycle(shared_addr->http2_pri);
+    }
+  } else if (http1_weight > 0) {
+    proto = PROTO_HTTP1;
+  } else if (http2_weight > 0) {
+    proto = PROTO_HTTP2;
+  }
+
+  if (proto == PROTO_NONE) {
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "No working downstream address found";
+    }
+
+    return nullptr;
+  }
+
+  if (proto == PROTO_HTTP2) {
     if (LOG_ENABLED(INFO)) {
       CLOG(INFO, this) << "Downstream connection pool is empty."
                        << " Create new one";
     }
 
-    if (shared_addr->proto == PROTO_HTTP2) {
-      auto &http2_freelist = shared_addr->http2_freelist;
+    auto http2session = select_http2_session(group);
 
-      if (http2_freelist.empty() ||
-          http2_freelist.size() < shared_addr->addrs.size()) {
-        if (LOG_ENABLED(INFO)) {
-          if (http2_freelist.empty()) {
-            CLOG(INFO, this)
-                << "http2_freelist is empty; create new Http2Session";
-          } else {
-            CLOG(INFO, this) << "Create new Http2Session; current "
-                             << http2_freelist.size() << ", min "
-                             << shared_addr->addrs.size();
-          }
-        }
-        auto session = make_unique<Http2Session>(
-            conn_.loop, shared_addr->tls ? worker_->get_cl_ssl_ctx() : nullptr,
-            worker_, &group);
-        http2_freelist.append(session.release());
-      }
-
-      auto http2session = http2_freelist.head;
-
-      if (http2session->max_concurrency_reached(1)) {
-        if (LOG_ENABLED(INFO)) {
-          CLOG(INFO, this) << "Maximum streams are reached for Http2Session("
-                           << http2session
-                           << "). Remove Http2Session from http2_freelist";
-        }
-        http2_freelist.remove(http2session);
-      }
-
-      dconn = make_unique<Http2DownstreamConnection>(http2session);
-    } else {
-      dconn =
-          make_unique<HttpDownstreamConnection>(&group, conn_.loop, worker_);
+    if (http2session == nullptr) {
+      return nullptr;
     }
+
+    auto dconn = make_unique<Http2DownstreamConnection>(http2session);
+
     dconn->set_client_handler(this);
-    return dconn;
+
+    return std::move(dconn);
+  }
+
+  auto &dconn_pool = shared_addr->dconn_pool;
+
+  // pool connection must be HTTP/1.1 connection
+  auto dconn = dconn_pool.pop_downstream_connection();
+
+  if (dconn) {
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "Reuse downstream connection DCONN:" << dconn.get()
+                       << " from pool";
+    }
+  } else {
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "Downstream connection pool is empty."
+                       << " Create new one";
+    }
+
+    dconn =
+        make_unique<HttpDownstreamConnection>(group, -1, conn_.loop, worker_);
   }
 
   dconn->set_client_handler(this);
 
-  if (LOG_ENABLED(INFO)) {
-    CLOG(INFO, this) << "Reuse downstream connection DCONN:" << dconn.get()
-                     << " from pool";
-  }
-
   return dconn;
 }
 
@@ -1196,4 +1442,6 @@ StringRef ClientHandler::get_forwarded_for() const {
   return StringRef{forwarded_for_};
 }
 
+const UpstreamAddr *ClientHandler::get_upstream_addr() const { return faddr_; }
+
 } // namespace shrpx

src/shrpx_client_handler.h

@@ -49,6 +49,8 @@ class ConnectBlocker;
 class DownstreamConnectionPool;
 class Worker;
 struct WorkerStat;
+struct DownstreamAddrGroup;
+struct DownstreamAddr;
 
 class ClientHandler {
 public:
@@ -86,7 +88,7 @@ public:
   struct ev_loop *get_loop() const;
   void reset_upstream_read_timeout(ev_tstamp t);
   void reset_upstream_write_timeout(ev_tstamp t);
-  void signal_reset_upstream_conn_rtimer();
+
   int validate_next_proto();
   const std::string &get_ipaddr() const;
   const std::string &get_port() const;
@@ -122,7 +124,7 @@ public:
                        int64_t body_bytes_sent);
   Worker *get_worker() const;
 
-  using ReadBuf = Buffer<8_k>;
+  using ReadBuf = Buffer<16_k>;
 
   ReadBuf *get_rb();
 
@@ -141,6 +143,17 @@ public:
   // header field.
   StringRef get_forwarded_for() const;
 
+  Http2Session *
+  select_http2_session(const std::shared_ptr<DownstreamAddrGroup> &group);
+
+  Http2Session *select_http2_session_with_affinity(
+      const std::shared_ptr<DownstreamAddrGroup> &group, DownstreamAddr *addr);
+
+  const UpstreamAddr *get_upstream_addr() const;
+
+  void repeat_read_timer();
+  void stop_read_timer();
+
 private:
   Connection conn_;
   ev_timer reneg_shutdown_timer_;
@@ -161,8 +174,11 @@ private:
   Worker *worker_;
   // The number of bytes of HTTP/2 client connection header to read
   size_t left_connhd_len_;
+  // hash for session affinity using client IP
+  uint32_t affinity_hash_;
   bool should_close_after_write_;
-  bool reset_conn_rtimer_required_;
+  // true if affinity_hash_ is computed
+  bool affinity_hash_computed_;
   ReadBuf rb_;
 };
 

src/shrpx_config.h

@@ -275,11 +275,30 @@ constexpr auto SHRPX_OPT_BACKEND_TLS = StringRef::from_lit("backend-tls");
 constexpr auto SHRPX_OPT_BACKEND_CONNECTIONS_PER_HOST =
     StringRef::from_lit("backend-connections-per-host");
 constexpr auto SHRPX_OPT_ERROR_PAGE = StringRef::from_lit("error-page");
+constexpr auto SHRPX_OPT_NO_KQUEUE = StringRef::from_lit("no-kqueue");
+constexpr auto SHRPX_OPT_FRONTEND_HTTP2_SETTINGS_TIMEOUT =
+    StringRef::from_lit("frontend-http2-settings-timeout");
+constexpr auto SHRPX_OPT_BACKEND_HTTP2_SETTINGS_TIMEOUT =
+    StringRef::from_lit("backend-http2-settings-timeout");
+constexpr auto SHRPX_OPT_API_MAX_REQUEST_BODY =
+    StringRef::from_lit("api-max-request-body");
+constexpr auto SHRPX_OPT_BACKEND_MAX_BACKOFF =
+    StringRef::from_lit("backend-max-backoff");
 
 constexpr size_t SHRPX_OBFUSCATED_NODE_LENGTH = 8;
 
+constexpr char DEFAULT_DOWNSTREAM_HOST[] = "127.0.0.1";
+constexpr int16_t DEFAULT_DOWNSTREAM_PORT = 80;
+
 enum shrpx_proto { PROTO_NONE, PROTO_HTTP1, PROTO_HTTP2, PROTO_MEMCACHED };
 
+enum shrpx_session_affinity {
+  // No session affinity
+  AFFINITY_NONE,
+  // Client IP affinity
+  AFFINITY_IP,
+};
+
 enum shrpx_forwarded_param {
   FORWARDED_NONE = 0,
   FORWARDED_BY = 0x1,
@@ -299,6 +318,15 @@ struct AltSvc {
   uint16_t port;
 };
 
+enum UpstreamAltMode {
+  // No alternative mode
+  ALTMODE_NONE,
+  // API processing mode
+  ALTMODE_API,
+  // Health monitor mode
+  ALTMODE_HEALTHMON,
+};
+
 struct UpstreamAddr {
   // The frontend address (e.g., FQDN, hostname, IP address).  If
   // |host_unix| is true, this is UNIX domain socket path.
@@ -312,6 +340,8 @@ struct UpstreamAddr {
   // For TCP socket, this is either AF_INET or AF_INET6.  For UNIX
   // domain socket, this is 0.
   int family;
+  // Alternate mode
+  int alt_mode;
   // true if |host| contains UNIX domain socket path.
   bool host_unix;
   // true if TLS is enabled.
@@ -319,14 +349,6 @@ struct UpstreamAddr {
   int fd;
 };
 
-struct TLSSessionCache {
-  // ASN1 representation of SSL_SESSION object.  See
-  // i2d_SSL_SESSION(3SSL).
-  std::vector<uint8_t> session_data;
-  // The last time stamp when this cache entry is created or updated.
-  ev_tstamp last_updated;
-};
-
 struct DownstreamAddrConfig {
   Address addr;
   // backend address.  If |host_unix| is true, this is UNIX domain
@@ -335,23 +357,39 @@ struct DownstreamAddrConfig {
   // <HOST>:<PORT>.  This does not treat 80 and 443 specially.  If
   // |host_unix| is true, this is "localhost".
   ImmutableString hostport;
+  // hostname sent as SNI field
+  ImmutableString sni;
+  size_t fall;
+  size_t rise;
+  // Application protocol used in this group
+  shrpx_proto proto;
   // backend port.  0 if |host_unix| is true.
   uint16_t port;
   // true if |host| contains UNIX domain socket path.
   bool host_unix;
+  bool tls;
+};
+
+// Mapping hash to idx which is an index into
+// DownstreamAddrGroupConfig::addrs.
+struct AffinityHash {
+  AffinityHash(size_t idx, uint32_t hash) : idx(idx), hash(hash) {}
+
+  size_t idx;
+  uint32_t hash;
 };
 
 struct DownstreamAddrGroupConfig {
   DownstreamAddrGroupConfig(const StringRef &pattern)
-      : pattern(pattern.c_str(), pattern.size()),
-        proto(PROTO_HTTP1),
-        tls(false) {}
+      : pattern(pattern.c_str(), pattern.size()), affinity(AFFINITY_NONE) {}
 
   ImmutableString pattern;
   std::vector<DownstreamAddrConfig> addrs;
-  // Application protocol used in this group
-  shrpx_proto proto;
-  bool tls;
+  // Bunch of session affinity hash.  Only used if affinity ==
+  // AFFINITY_IP.
+  std::vector<AffinityHash> affinity_hash;
+  // Session affinity
+  shrpx_session_affinity affinity;
 };
 
 struct TicketKey {
@@ -466,6 +504,7 @@ struct TLSConfig {
   std::vector<std::string> npn_list;
   // list of supported SSL/TLS protocol strings.
   std::vector<std::string> tls_proto_list;
+  BIO_METHOD *bio_method;
   // Bit mask to disable SSL/TLS protocol versions.  This will be
   // passed to SSL_CTX_set_options().
   long int tls_proto_mask;
@@ -534,13 +573,20 @@ struct Http2Config {
       } dump;
       bool frame_debug;
     } debug;
+    struct {
+      ev_tstamp settings;
+    } timeout;
     nghttp2_option *option;
+    nghttp2_option *alt_mode_option;
     nghttp2_session_callbacks *callbacks;
     size_t window_bits;
     size_t connection_window_bits;
     size_t max_concurrent_streams;
   } upstream;
   struct {
+    struct {
+      ev_tstamp settings;
+    } timeout;
     nghttp2_option *option;
     nghttp2_session_callbacks *callbacks;
     size_t window_bits;
@@ -575,6 +621,53 @@ struct RateLimitConfig {
   size_t burst;
 };
 
+// Wildcard host pattern routing.  We strips left most '*' from host
+// field.  router includes all path patterns sharing the same wildcard
+// host.
+struct WildcardPattern {
+  WildcardPattern(const StringRef &host)
+      : host(std::begin(host), std::end(host)) {}
+
+  ImmutableString host;
+  Router router;
+};
+
+// Configuration to select backend to forward request
+struct RouterConfig {
+  Router router;
+  // Router for reversed wildcard hosts.  Since this router has
+  // wildcard hosts reversed without '*', one should call match()
+  // function with reversed host stripping last character.  This is
+  // because we require at least one character must match for '*'.
+  // The index stored in this router is index of wildcard_patterns.
+  Router rev_wildcard_router;
+  std::vector<WildcardPattern> wildcard_patterns;
+};
+
+struct DownstreamConfig {
+  struct {
+    ev_tstamp read;
+    ev_tstamp write;
+    ev_tstamp idle_read;
+    // The maximum backoff while checking health check for offline
+    // backend or while detaching failed backend from load balancing
+    // group temporarily.
+    ev_tstamp max_backoff;
+  } timeout;
+  RouterConfig router;
+  std::vector<DownstreamAddrGroupConfig> addr_groups;
+  // The index of catch-all group in downstream_addr_groups.
+  size_t addr_group_catch_all;
+  size_t connections_per_host;
+  size_t connections_per_frontend;
+  size_t request_buffer_size;
+  size_t response_buffer_size;
+  // Address family of backend connection.  One of either AF_INET,
+  // AF_INET6 or AF_UNSPEC.  This is ignored if backend connection
+  // is made via Unix domain socket.
+  int family;
+};
+
 struct ConnectionConfig {
   struct {
     struct {
@@ -602,43 +695,24 @@ struct ConnectionConfig {
     bool accept_proxy_protocol;
   } upstream;
 
-  struct {
-    struct {
-      ev_tstamp read;
-      ev_tstamp write;
-      ev_tstamp idle_read;
-    } timeout;
-    std::vector<DownstreamAddrGroupConfig> addr_groups;
-    // The index of catch-all group in downstream_addr_groups.
-    size_t addr_group_catch_all;
-    size_t connections_per_host;
-    size_t connections_per_frontend;
-    size_t request_buffer_size;
-    size_t response_buffer_size;
-    // Address family of backend connection.  One of either AF_INET,
-    // AF_INET6 or AF_UNSPEC.  This is ignored if backend connection
-    // is made via Unix domain socket.
-    int family;
-  } downstream;
+  std::shared_ptr<DownstreamConfig> downstream;
 };
 
-// Wildcard host pattern routing.  We strips left most '*' from host
-// field.  router includes all path pattern sharing same wildcard
-// host.
-struct WildcardPattern {
-  ImmutableString host;
-  Router router;
+struct APIConfig {
+  // Maximum request body size for one API request
+  size_t max_request_body;
+  // true if at least one of UpstreamAddr has api enabled
+  bool enabled;
 };
 
 struct Config {
-  Router router;
-  std::vector<WildcardPattern> wildcard_patterns;
   HttpProxy downstream_http_proxy;
   HttpConfig http;
   Http2Config http2;
   TLSConfig tls;
   LoggingConfig logging;
   ConnectionConfig conn;
+  APIConfig api;
   ImmutableString pid_file;
   ImmutableString conf_path;
   ImmutableString user;
@@ -656,20 +730,166 @@ struct Config {
   bool verbose;
   bool daemon;
   bool http2_proxy;
+  // flags passed to ev_default_loop() and ev_loop_new()
+  int ev_loop_flags;
 };
 
 const Config *get_config();
 Config *mod_config();
 void create_config();
 
+// generated by gennghttpxfun.py
+enum {
+  SHRPX_OPTID_ACCEPT_PROXY_PROTOCOL,
+  SHRPX_OPTID_ACCESSLOG_FILE,
+  SHRPX_OPTID_ACCESSLOG_FORMAT,
+  SHRPX_OPTID_ACCESSLOG_SYSLOG,
+  SHRPX_OPTID_ADD_FORWARDED,
+  SHRPX_OPTID_ADD_REQUEST_HEADER,
+  SHRPX_OPTID_ADD_RESPONSE_HEADER,
+  SHRPX_OPTID_ADD_X_FORWARDED_FOR,
+  SHRPX_OPTID_ALTSVC,
+  SHRPX_OPTID_API_MAX_REQUEST_BODY,
+  SHRPX_OPTID_BACKEND,
+  SHRPX_OPTID_BACKEND_ADDRESS_FAMILY,
+  SHRPX_OPTID_BACKEND_CONNECTIONS_PER_FRONTEND,
+  SHRPX_OPTID_BACKEND_CONNECTIONS_PER_HOST,
+  SHRPX_OPTID_BACKEND_HTTP_PROXY_URI,
+  SHRPX_OPTID_BACKEND_HTTP1_CONNECTIONS_PER_FRONTEND,
+  SHRPX_OPTID_BACKEND_HTTP1_CONNECTIONS_PER_HOST,
+  SHRPX_OPTID_BACKEND_HTTP1_TLS,
+  SHRPX_OPTID_BACKEND_HTTP2_CONNECTION_WINDOW_BITS,
+  SHRPX_OPTID_BACKEND_HTTP2_CONNECTIONS_PER_WORKER,
+  SHRPX_OPTID_BACKEND_HTTP2_MAX_CONCURRENT_STREAMS,
+  SHRPX_OPTID_BACKEND_HTTP2_SETTINGS_TIMEOUT,
+  SHRPX_OPTID_BACKEND_HTTP2_WINDOW_BITS,
+  SHRPX_OPTID_BACKEND_IPV4,
+  SHRPX_OPTID_BACKEND_IPV6,
+  SHRPX_OPTID_BACKEND_KEEP_ALIVE_TIMEOUT,
+  SHRPX_OPTID_BACKEND_MAX_BACKOFF,
+  SHRPX_OPTID_BACKEND_NO_TLS,
+  SHRPX_OPTID_BACKEND_READ_TIMEOUT,
+  SHRPX_OPTID_BACKEND_REQUEST_BUFFER,
+  SHRPX_OPTID_BACKEND_RESPONSE_BUFFER,
+  SHRPX_OPTID_BACKEND_TLS,
+  SHRPX_OPTID_BACKEND_TLS_SNI_FIELD,
+  SHRPX_OPTID_BACKEND_WRITE_TIMEOUT,
+  SHRPX_OPTID_BACKLOG,
+  SHRPX_OPTID_CACERT,
+  SHRPX_OPTID_CERTIFICATE_FILE,
+  SHRPX_OPTID_CIPHERS,
+  SHRPX_OPTID_CLIENT,
+  SHRPX_OPTID_CLIENT_CERT_FILE,
+  SHRPX_OPTID_CLIENT_PRIVATE_KEY_FILE,
+  SHRPX_OPTID_CLIENT_PROXY,
+  SHRPX_OPTID_CONF,
+  SHRPX_OPTID_DAEMON,
+  SHRPX_OPTID_DH_PARAM_FILE,
+  SHRPX_OPTID_ERROR_PAGE,
+  SHRPX_OPTID_ERRORLOG_FILE,
+  SHRPX_OPTID_ERRORLOG_SYSLOG,
+  SHRPX_OPTID_FASTOPEN,
+  SHRPX_OPTID_FETCH_OCSP_RESPONSE_FILE,
+  SHRPX_OPTID_FORWARDED_BY,
+  SHRPX_OPTID_FORWARDED_FOR,
+  SHRPX_OPTID_FRONTEND,
+  SHRPX_OPTID_FRONTEND_FRAME_DEBUG,
+  SHRPX_OPTID_FRONTEND_HTTP2_CONNECTION_WINDOW_BITS,
+  SHRPX_OPTID_FRONTEND_HTTP2_DUMP_REQUEST_HEADER,
+  SHRPX_OPTID_FRONTEND_HTTP2_DUMP_RESPONSE_HEADER,
+  SHRPX_OPTID_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS,
+  SHRPX_OPTID_FRONTEND_HTTP2_READ_TIMEOUT,
+  SHRPX_OPTID_FRONTEND_HTTP2_SETTINGS_TIMEOUT,
+  SHRPX_OPTID_FRONTEND_HTTP2_WINDOW_BITS,
+  SHRPX_OPTID_FRONTEND_NO_TLS,
+  SHRPX_OPTID_FRONTEND_READ_TIMEOUT,
+  SHRPX_OPTID_FRONTEND_WRITE_TIMEOUT,
+  SHRPX_OPTID_HEADER_FIELD_BUFFER,
+  SHRPX_OPTID_HOST_REWRITE,
+  SHRPX_OPTID_HTTP2_BRIDGE,
+  SHRPX_OPTID_HTTP2_MAX_CONCURRENT_STREAMS,
+  SHRPX_OPTID_HTTP2_NO_COOKIE_CRUMBLING,
+  SHRPX_OPTID_HTTP2_PROXY,
+  SHRPX_OPTID_INCLUDE,
+  SHRPX_OPTID_INSECURE,
+  SHRPX_OPTID_LISTENER_DISABLE_TIMEOUT,
+  SHRPX_OPTID_LOG_LEVEL,
+  SHRPX_OPTID_MAX_HEADER_FIELDS,
+  SHRPX_OPTID_MAX_REQUEST_HEADER_FIELDS,
+  SHRPX_OPTID_MAX_RESPONSE_HEADER_FIELDS,
+  SHRPX_OPTID_MRUBY_FILE,
+  SHRPX_OPTID_NO_HOST_REWRITE,
+  SHRPX_OPTID_NO_HTTP2_CIPHER_BLACK_LIST,
+  SHRPX_OPTID_NO_KQUEUE,
+  SHRPX_OPTID_NO_LOCATION_REWRITE,
+  SHRPX_OPTID_NO_OCSP,
+  SHRPX_OPTID_NO_SERVER_PUSH,
+  SHRPX_OPTID_NO_VIA,
+  SHRPX_OPTID_NPN_LIST,
+  SHRPX_OPTID_OCSP_UPDATE_INTERVAL,
+  SHRPX_OPTID_PADDING,
+  SHRPX_OPTID_PID_FILE,
+  SHRPX_OPTID_PRIVATE_KEY_FILE,
+  SHRPX_OPTID_PRIVATE_KEY_PASSWD_FILE,
+  SHRPX_OPTID_READ_BURST,
+  SHRPX_OPTID_READ_RATE,
+  SHRPX_OPTID_REQUEST_HEADER_FIELD_BUFFER,
+  SHRPX_OPTID_RESPONSE_HEADER_FIELD_BUFFER,
+  SHRPX_OPTID_RLIMIT_NOFILE,
+  SHRPX_OPTID_STREAM_READ_TIMEOUT,
+  SHRPX_OPTID_STREAM_WRITE_TIMEOUT,
+  SHRPX_OPTID_STRIP_INCOMING_FORWARDED,
+  SHRPX_OPTID_STRIP_INCOMING_X_FORWARDED_FOR,
+  SHRPX_OPTID_SUBCERT,
+  SHRPX_OPTID_SYSLOG_FACILITY,
+  SHRPX_OPTID_TLS_DYN_REC_IDLE_TIMEOUT,
+  SHRPX_OPTID_TLS_DYN_REC_WARMUP_THRESHOLD,
+  SHRPX_OPTID_TLS_PROTO_LIST,
+  SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED,
+  SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED_ADDRESS_FAMILY,
+  SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED_CERT_FILE,
+  SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED_PRIVATE_KEY_FILE,
+  SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED_TLS,
+  SHRPX_OPTID_TLS_TICKET_KEY_CIPHER,
+  SHRPX_OPTID_TLS_TICKET_KEY_FILE,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_ADDRESS_FAMILY,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_CERT_FILE,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_INTERVAL,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_MAX_FAIL,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_MAX_RETRY,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_PRIVATE_KEY_FILE,
+  SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_TLS,
+  SHRPX_OPTID_USER,
+  SHRPX_OPTID_VERIFY_CLIENT,
+  SHRPX_OPTID_VERIFY_CLIENT_CACERT,
+  SHRPX_OPTID_WORKER_FRONTEND_CONNECTIONS,
+  SHRPX_OPTID_WORKER_READ_BURST,
+  SHRPX_OPTID_WORKER_READ_RATE,
+  SHRPX_OPTID_WORKER_WRITE_BURST,
+  SHRPX_OPTID_WORKER_WRITE_RATE,
+  SHRPX_OPTID_WORKERS,
+  SHRPX_OPTID_WRITE_BURST,
+  SHRPX_OPTID_WRITE_RATE,
+  SHRPX_OPTID_MAXIDX,
+};
+
+// Looks up token for given option name |name| of length |namelen|.
+int option_lookup_token(const char *name, size_t namelen);
+
 // Parses option name |opt| and value |optarg|.  The results are
-// stored into statically allocated Config object. This function
-// returns 0 if it succeeds, or -1.  The |included_set| contains the
-// all paths already included while processing this configuration, to
-// avoid loop in --include option.
-int parse_config(const StringRef &opt, const StringRef &optarg,
+// stored into the object pointed by |config|. This function returns 0
+// if it succeeds, or -1.  The |included_set| contains the all paths
+// already included while processing this configuration, to avoid loop
+// in --include option.
+int parse_config(Config *config, const StringRef &opt, const StringRef &optarg,
                  std::set<StringRef> &included_set);
 
+// Similar to parse_config() above, but additional |optid| which
+// should be the return value of option_lookup_token(opt).
+int parse_config(Config *config, int optid, const StringRef &opt,
+                 const StringRef &optarg, std::set<StringRef> &included_set);
+
 // Loads configurations from |filename| and stores them in statically
 // allocated Config object. This function returns 0 if it succeeds, or
 // -1.  See parse_config() for |include_set|.
@@ -702,6 +922,13 @@ read_tls_ticket_key_file(const std::vector<std::string> &files,
 // Returns string representation of |proto|.
 StringRef strproto(shrpx_proto proto);
 
+int configure_downstream_group(Config *config, bool http2_proxy,
+                               bool numeric_addr_only,
+                               const TLSConfig &tlsconf);
+
+int resolve_hostname(Address *addr, const char *hostname, uint16_t port,
+                     int family, int additional_flags = 0);
+
 } // namespace shrpx
 
 #endif // SHRPX_CONFIG_H

src/shrpx_connect_blocker.cc

@@ -23,30 +23,52 @@
  * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
  */
 #include "shrpx_connect_blocker.h"
+#include "shrpx_config.h"
 
 namespace shrpx {
 
 namespace {
 void connect_blocker_cb(struct ev_loop *loop, ev_timer *w, int revents) {
+  auto connect_blocker = static_cast<ConnectBlocker *>(w->data);
   if (LOG_ENABLED(INFO)) {
     LOG(INFO) << "Unblock";
   }
+
+  connect_blocker->call_unblock_func();
 }
 } // namespace
 
-ConnectBlocker::ConnectBlocker(std::mt19937 &gen, struct ev_loop *loop)
-    : gen_(gen), loop_(loop), fail_count_(0) {
+ConnectBlocker::ConnectBlocker(std::mt19937 &gen, struct ev_loop *loop,
+                               std::function<void()> block_func,
+                               std::function<void()> unblock_func)
+    : gen_(gen),
+      block_func_(block_func),
+      unblock_func_(unblock_func),
+      loop_(loop),
+      fail_count_(0),
+      offline_(false) {
   ev_timer_init(&timer_, connect_blocker_cb, 0., 0.);
+  timer_.data = this;
 }
 
 ConnectBlocker::~ConnectBlocker() { ev_timer_stop(loop_, &timer_); }
 
 bool ConnectBlocker::blocked() const { return ev_is_active(&timer_); }
 
-void ConnectBlocker::on_success() { fail_count_ = 0; }
+void ConnectBlocker::on_success() {
+  if (ev_is_active(&timer_)) {
+    return;
+  }
 
+  fail_count_ = 0;
+}
+
+// Use the similar backoff algorithm described in
+// https://github.com/grpc/grpc/blob/master/doc/connection-backoff.md
 namespace {
 constexpr size_t MAX_BACKOFF_EXP = 10;
+constexpr auto MULTIPLIER = 1.6;
+constexpr auto JITTER = 0.2;
 } // namespace
 
 void ConnectBlocker::on_failure() {
@@ -54,11 +76,19 @@ void ConnectBlocker::on_failure() {
     return;
   }
 
+  call_block_func();
+
   ++fail_count_;
 
-  auto max_backoff = (1 << std::min(MAX_BACKOFF_EXP, fail_count_)) - 1;
-  auto dist = std::uniform_int_distribution<>(0, max_backoff);
-  auto backoff = dist(gen_);
+  auto base_backoff =
+      util::int_pow(MULTIPLIER, std::min(MAX_BACKOFF_EXP, fail_count_));
+  auto dist = std::uniform_real_distribution<>(-JITTER * base_backoff,
+                                               JITTER * base_backoff);
+
+  auto &downstreamconf = *get_config()->conn.downstream;
+
+  auto backoff =
+      std::min(downstreamconf.timeout.max_backoff, base_backoff + dist(gen_));
 
   LOG(WARN) << "Could not connect " << fail_count_
             << " times in a row; sleep for " << backoff << " seconds";
@@ -67,4 +97,38 @@ void ConnectBlocker::on_failure() {
   ev_timer_start(loop_, &timer_);
 }
 
+size_t ConnectBlocker::get_fail_count() const { return fail_count_; }
+
+void ConnectBlocker::offline() {
+  if (offline_) {
+    return;
+  }
+
+  if (!ev_is_active(&timer_)) {
+    call_block_func();
+  }
+
+  offline_ = true;
+
+  ev_timer_stop(loop_, &timer_);
+  ev_timer_set(&timer_, std::numeric_limits<double>::max(), 0.);
+  ev_timer_start(loop_, &timer_);
+}
+
+void ConnectBlocker::online() {
+  ev_timer_stop(loop_, &timer_);
+
+  call_unblock_func();
+
+  fail_count_ = 0;
+
+  offline_ = false;
+}
+
+bool ConnectBlocker::in_offline() const { return offline_; }
+
+void ConnectBlocker::call_block_func() { block_func_(); }
+
+void ConnectBlocker::call_unblock_func() { unblock_func_(); }
+
 } // namespace shrpx

src/shrpx_connect_blocker.h

@@ -35,7 +35,9 @@ namespace shrpx {
 
 class ConnectBlocker {
 public:
-  ConnectBlocker(std::mt19937 &gen, struct ev_loop *loop);
+  ConnectBlocker(std::mt19937 &gen, struct ev_loop *loop,
+                 std::function<void()> block_func,
+                 std::function<void()> unblock_func);
   ~ConnectBlocker();
 
   // Returns true if making connection is not allowed.
@@ -48,13 +50,34 @@ public:
   // backoff.
   void on_failure();
 
+  size_t get_fail_count() const;
+
+  // Peer is now considered offline.  This effectively means that the
+  // connection is blocked until online() is called.
+  void offline();
+
+  // Peer is now considered online
+  void online();
+
+  // Returns true if peer is considered offline.
+  bool in_offline() const;
+
+  void call_block_func();
+  void call_unblock_func();
+
 private:
-  std::mt19937 gen_;
+  std::mt19937 &gen_;
+  // Called when blocking is started
+  std::function<void()> block_func_;
+  // Called when unblocked
+  std::function<void()> unblock_func_;
   ev_timer timer_;
   struct ev_loop *loop_;
   // The number of consecutive connection failure.  Reset to 0 on
   // success.
   size_t fail_count_;
+  // true if peer is considered offline.
+  bool offline_;
 };
 
 } // namespace