fixup! nghttpx: Send nghttpx-0rtt-uniq header if request is replayable

The incoming nghttpx-0rtt-uniq header fields from inbound client are
stripped by default.  Use --no-strip-incoming-nghttpx-0rtt-uniq in
order not to strip them.

AUTHORS

@@ -35,7 +35,6 @@ Dave Reisner
 David Beitey
 David Weekly
 Dmitriy Vetutnev
-Dylan Plecki
 Etienne Cimon
 Fabian Möller
 Fabian Wiesel

CMakeLists.txt

@@ -24,15 +24,15 @@
 
 cmake_minimum_required(VERSION 3.0)
 # XXX using 1.8.90 instead of 1.9.0-DEV
-project(nghttp2 VERSION 1.30.0)
+project(nghttp2 VERSION 1.28.90)
 
 # See versioning rule:
 #  http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
 set(LT_CURRENT  29)
-set(LT_REVISION 2)
+set(LT_REVISION 0)
 set(LT_AGE      15)
 
-set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake" ${CMAKE_MODULE_PATH})
+set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/cmake")
 include(Version)
 
 math(EXPR LT_SOVERSION "${LT_CURRENT} - ${LT_AGE}")
@@ -302,7 +302,6 @@ check_type_size("time_t"  SIZEOF_TIME_T)
 include(CheckFunctionExists)
 check_function_exists(_Exit     HAVE__EXIT)
 check_function_exists(accept4   HAVE_ACCEPT4)
-check_function_exists(mkostemp  HAVE_MKOSTEMP)
 
 include(CheckSymbolExists)
 # XXX does this correctly detect initgroups (un)availability on cygwin?

README.rst

@@ -4,10 +4,10 @@ nghttp2 - HTTP/2 C Library
 This is an implementation of the Hypertext Transfer Protocol version 2
 in C.
 
-The framing layer of HTTP/2 is implemented as a reusable C library.
-On top of that, we have implemented an HTTP/2 client, server and
-proxy.  We have also developed load test and benchmarking tools for
-HTTP/2.
+The framing layer of HTTP/2 is implemented as a reusable C
+library.  On top of that, we have implemented an HTTP/2 client, server
+and proxy.  We have also developed load test and benchmarking tools for
+HTTP/2 and SPDY.
 
 An HPACK encoder and decoder are available as a public API.
 
@@ -34,8 +34,8 @@ implementation.
 
 * https://nghttp2.org/ (TLS + ALPN/NPN)
 
-  This endpoint supports ``h2``, ``h2-16``, ``h2-14``, and
-  ``http/1.1`` via ALPN/NPN and requires TLSv1.2 for HTTP/2
+  This endpoint supports ``h2``, ``h2-16``, ``h2-14``, ``spdy/3.1``
+  and ``http/1.1`` via ALPN/NPN and requires TLSv1.2 for HTTP/2
   connection.
 
 * http://nghttp2.org/ (HTTP Upgrade and HTTP/2 Direct)
@@ -76,6 +76,14 @@ ALPN support requires OpenSSL >= 1.0.2 (released 22 January 2015).
 LibreSSL >= 2.2.0 can be used instead of OpenSSL, but OpenSSL has more
 features than LibreSSL at the time of this writing.
 
+To enable the SPDY protocol in the application program ``nghttpx`` and
+``h2load``, the following package is required:
+
+* spdylay >= 1.3.2
+
+We no longer recommend to build nghttp2 with SPDY protocol support
+enabled.  SPDY support will be removed soon.
+
 To enable ``-a`` option (getting linked assets from the downloaded
 resource) in ``nghttp``, the following package is required:
 
@@ -122,9 +130,13 @@ and above, run the following to install the required packages:
 
     sudo apt-get install g++ make binutils autoconf automake autotools-dev libtool pkg-config \
       zlib1g-dev libcunit1-dev libssl-dev libxml2-dev libev-dev libevent-dev libjansson-dev \
-      libc-ares-dev libjemalloc-dev libsystemd-dev \
+      libc-ares-dev libjemalloc-dev libsystemd-dev libspdylay-dev \
       cython python3-dev python-setuptools
 
+Since Ubuntu 15.10, spdylay has been available as a package named
+`libspdylay-dev`.  For the earlier Ubuntu release, you need to build
+it yourself: http://tatsuhiro-t.github.io/spdylay/
+
 To enable mruby support for nghttpx, `mruby
 <https://github.com/mruby/mruby>`_ is required.  We need to build
 mruby with C++ ABI explicitly turned on, and probably need other
@@ -320,6 +332,7 @@ its testing framework.  We depend on the following libraries:
 * golang.org/x/net/http2
 * golang.org/x/net/websocket
 * https://github.com/tatsuhiro-t/go-nghttp2
+* https://github.com/tatsuhiro-t/spdy
 
 To download the above packages, after settings ``GOPATH``, run the
 following command under ``integration-tests`` directory:
@@ -337,6 +350,11 @@ To run the tests, run the following command under
 
 Inside the tests, we use port 3009 to run the test subject server.
 
+.. note::
+
+   github.com/tatsuhiro-t/spdy is a copy used to be available at
+   golang.org/x/net/spdy, but it is now gone.
+
 Migration from v0.7.15 or earlier
 ---------------------------------
 
@@ -737,7 +755,7 @@ information.  Here is sample output from ``nghttpd``:
 nghttpx - proxy
 +++++++++++++++
 
-``nghttpx`` is a multi-threaded reverse proxy for HTTP/2, and
+``nghttpx`` is a multi-threaded reverse proxy for HTTP/2, SPDY and
 HTTP/1.1, and powers http://nghttp2.org and supports HTTP/2 server
 push.
 
@@ -752,30 +770,31 @@ to know how to migrate from earlier releases.
 ``nghttpx`` implements `important performance-oriented features
 <https://istlsfastyet.com/#server-performance>`_ in TLS, such as
 session IDs, session tickets (with automatic key rotation), OCSP
-stapling, dynamic record sizing, ALPN/NPN, forward secrecy and HTTP/2.
-``nghttpx`` also offers the functionality to share session cache and
-ticket keys among multiple ``nghttpx`` instances via memcached.
+stapling, dynamic record sizing, ALPN/NPN, forward secrecy and SPDY &
+HTTP/2.  ``nghttpx`` also offers the functionality to share session
+cache and ticket keys among multiple ``nghttpx`` instances via
+memcached.
 
 ``nghttpx`` has 2 operation modes:
 
-================== ================ ================ =============
-Mode option        Frontend         Backend          Note
-================== ================ ================ =============
-default mode       HTTP/2, HTTP/1.1 HTTP/1.1, HTTP/2 Reverse proxy
-``--http2-proxy``  HTTP/2, HTTP/1.1 HTTP/1.1, HTTP/2 Forward proxy
-================== ================ ================ =============
+================== ====================== ================ =============
+Mode option        Frontend               Backend          Note
+================== ====================== ================ =============
+default mode       HTTP/2, SPDY, HTTP/1.1 HTTP/1.1, HTTP/2 Reverse proxy
+``--http2-proxy``  HTTP/2, SPDY, HTTP/1.1 HTTP/1.1, HTTP/2 Forward proxy
+================== ====================== ================ =============
 
 The interesting mode at the moment is the default mode.  It works like
-a reverse proxy and listens for HTTP/2, and HTTP/1.1 and can be
+a reverse proxy and listens for HTTP/2, SPDY and HTTP/1.1 and can be
 deployed as a SSL/TLS terminator for existing web server.
 
 In all modes, the frontend connections are encrypted by SSL/TLS by
 default.  To disable encryption, use the ``no-tls`` keyword in
-``--frontend`` option.  If encryption is disabled, incoming HTTP/1.1
-connections can be upgraded to HTTP/2 through HTTP Upgrade.  On the
-other hard, backend connections are not encrypted by default.  To
-encrypt backend connections, use ``tls`` keyword in ``--backend``
-option.
+``--frontend`` option.  If encryption is disabled, SPDY is disabled in
+the frontend and incoming HTTP/1.1 connections can be upgraded to
+HTTP/2 through HTTP Upgrade.  On the other hard, backend connections
+are not encrypted by default.  To encrypt backend connections, use
+``tls`` keyword in ``--backend`` option.
 
 ``nghttpx`` supports a configuration file.  See the ``--conf`` option and
 sample configuration file ``nghttpx.conf.sample``.
@@ -785,16 +804,16 @@ server:
 
 .. code-block:: text
 
-    Client <-- (HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/1.1, HTTP/2) --> Web Server
-                                    [reverse proxy]
+    Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1, HTTP/2) --> Web Server
+                                          [reverse proxy]
 
 With the ``--http2-proxy`` option, it works as forward proxy, and it
-is so called secure HTTP/2 proxy:
+is so called secure HTTP/2 proxy (aka SPDY proxy):
 
 .. code-block:: text
 
-    Client <-- (HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Proxy
-                                     [secure proxy]          (e.g., Squid, ATS)
+    Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/1.1) --> Proxy
+                                           [secure proxy]          (e.g., Squid, ATS)
 
 The ``Client`` in the above example needs to be configured to use
 ``nghttpx`` as secure proxy.
@@ -826,7 +845,7 @@ proxy through an HTTP proxy:
 
 .. code-block:: text
 
-    Client <-- (HTTP/2, HTTP/1.1) --> nghttpx <-- (HTTP/2) --
+    Client <-- (HTTP/2, SPDY, HTTP/1.1) --> nghttpx <-- (HTTP/2) --
 
             --===================---> HTTP/2 Proxy
               (HTTP proxy tunnel)     (e.g., nghttpx -s)
@@ -834,8 +853,9 @@ proxy through an HTTP proxy:
 Benchmarking tool
 -----------------
 
-The ``h2load`` program is a benchmarking tool for HTTP/2.  The UI of
-``h2load`` is heavily inspired by ``weighttp``
+The ``h2load`` program is a benchmarking tool for HTTP/2 and SPDY.
+The SPDY support is enabled if the program was built with the spdylay
+library.  The UI of ``h2load`` is heavily inspired by ``weighttp``
 (https://github.com/lighttpd/weighttp).  The typical usage is as
 follows:
 

cmakeconfig.h.in

@@ -34,9 +34,6 @@
 /* Define to 1 if you have the `accept4` function. */
 #cmakedefine HAVE_ACCEPT4 1
 
-/* Define to 1 if you have the `mkostemp` function. */
-#cmakedefine HAVE_MKOSTEMP 1
-
 /* Define to 1 if you have the `initgroups` function. */
 #cmakedefine01 HAVE_DECL_INITGROUPS
 

configure.ac

@@ -25,7 +25,7 @@ dnl Do not change user variables!
 dnl http://www.gnu.org/software/automake/manual/html_node/Flag-Variables-Ordering.html
 
 AC_PREREQ(2.61)
-AC_INIT([nghttp2], [1.30.0], [t-tujikawa@users.sourceforge.net])
+AC_INIT([nghttp2], [1.29.0-DEV], [t-tujikawa@users.sourceforge.net])
 AC_CONFIG_AUX_DIR([.])
 AC_CONFIG_MACRO_DIR([m4])
 AC_CONFIG_HEADERS([config.h])
@@ -45,7 +45,7 @@ m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])])
 dnl See versioning rule:
 dnl  http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
 AC_SUBST(LT_CURRENT, 29)
-AC_SUBST(LT_REVISION, 2)
+AC_SUBST(LT_REVISION, 0)
 AC_SUBST(LT_AGE, 15)
 
 major=`echo $PACKAGE_VERSION |cut -d. -f1 | sed -e "s/[^0-9]//g"`
@@ -117,6 +117,11 @@ AC_ARG_WITH([jemalloc],
                     [Use jemalloc [default=check]])],
     [request_jemalloc=$withval], [request_jemalloc=check])
 
+AC_ARG_WITH([spdylay],
+    [AS_HELP_STRING([--with-spdylay],
+                    [(Deprecated) Use spdylay [default=no]])],
+    [request_spdylay=$withval], [request_spdylay=no])
+
 AC_ARG_WITH([systemd],
     [AS_HELP_STRING([--with-systemd],
                     [Enable systemd support in nghttpx [default=check]])],
@@ -453,6 +458,26 @@ if test "x${request_jemalloc}" = "xyes" &&
   AC_MSG_ERROR([jemalloc was requested (--with-jemalloc) but not found])
 fi
 
+# spdylay (for src/nghttpx and src/h2load)
+have_spdylay=no
+if test "x${request_spdylay}" != "xno"; then
+  PKG_CHECK_MODULES([LIBSPDYLAY], [libspdylay >= 1.3.2],
+                    [have_spdylay=yes], [have_spdylay=no])
+  if test "x${have_spdylay}" = "xyes"; then
+    AC_DEFINE([HAVE_SPDYLAY], [1], [Define to 1 if you have `spdylay` library.])
+  else
+    AC_MSG_NOTICE($LIBSPDYLAY_PKG_ERRORS)
+    AC_MSG_NOTICE([The SPDY support in nghttpx and h2load will be disabled.])
+  fi
+fi
+
+if test "x${request_spdylay}" = "xyes" &&
+   test "x${have_spdylay}" != "xyes"; then
+  AC_MSG_ERROR([spdylay was requested (--with-spdylay) but not found])
+fi
+
+AM_CONDITIONAL([HAVE_SPDYLAY], [ test "x${have_spdylay}" = "xyes" ])
+
 # Check Boost Asio library
 have_asio_lib=no
 
@@ -688,7 +713,6 @@ AC_CHECK_FUNCS([ \
   memchr \
   memmove \
   memset \
-  mkostemp \
   socket \
   sqrt \
   strchr \
@@ -903,6 +927,7 @@ AC_MSG_NOTICE([summary of build options:
       Libev:          ${have_libev} (CFLAGS='${LIBEV_CFLAGS}' LIBS='${LIBEV_LIBS}')
       Libc-ares       ${have_libcares} (CFLAGS='${LIBCARES_CFLAGS}' LIBS='${LIBCARES_LIBS}')
       Libevent(SSL):  ${have_libevent_openssl} (CFLAGS='${LIBEVENT_OPENSSL_CFLAGS}' LIBS='${LIBEVENT_OPENSSL_LIBS}')
+      Spdylay:        ${have_spdylay} (CFLAGS='${LIBSPDYLAY_CFLAGS}' LIBS='${LIBSPDYLAY_LIBS}')
       Jansson:        ${have_jansson} (CFLAGS='${JANSSON_CFLAGS}' LIBS='${JANSSON_LIBS}')
       Jemalloc:       ${have_jemalloc} (LIBS='${JEMALLOC_LIBS}')
       Zlib:           ${have_zlib} (CFLAGS='${ZLIB_CFLAGS}' LIBS='${ZLIB_LIBS}')
@@ -924,3 +949,7 @@ AC_MSG_NOTICE([summary of build options:
       Python bindings:${enable_python_bindings}
       Threading:      ${enable_threads}
 ])
+
+if test "x${have_spdylay}" == "xyes"; then
+  AC_MSG_WARN([spdylay support was deprecated, and will be removed in v1.29.0.])
+fi

doc/h2load.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "H2LOAD" "1" "Feb 02, 2018" "1.30.0" "nghttp2"
+.TH "H2LOAD" "1" "Nov 25, 2017" "1.28.0" "nghttp2"
 .SH NAME
 h2load \- HTTP/2 benchmarking tool
 .
@@ -35,7 +35,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 \fBh2load\fP [OPTIONS]... [URI]...
 .SH DESCRIPTION
 .sp
-benchmarking tool for HTTP/2 server
+benchmarking tool for HTTP/2 and SPDY server
 .INDENT 0.0
 .TP
 .B <URI>
@@ -101,6 +101,7 @@ Default: \fB1\fP
 .TP
 .B \-w, \-\-window\-bits=<N>
 Sets the stream level initial window size to (2**<N>)\-1.
+For SPDY, 2**<N> is used instead.
 .sp
 Default: \fB30\fP
 .UNINDENT
@@ -108,7 +109,9 @@ Default: \fB30\fP
 .TP
 .B \-W, \-\-connection\-window\-bits=<N>
 Sets  the  connection  level   initial  window  size  to
-(2**<N>)\-1.
+(2**<N>)\-1.  For SPDY, if <N>  is strictly less than 16,
+this option  is ignored.   Otherwise 2**<N> is  used for
+SPDY.
 .sp
 Default: \fB30\fP
 .UNINDENT
@@ -130,7 +133,8 @@ Default: \fBECDHE\-ECDSA\-AES256\-GCM\-SHA384:ECDHE\-RSA\-AES256\-GCM\-SHA384:EC
 .B \-p, \-\-no\-tls\-proto=<PROTOID>
 Specify ALPN identifier of the  protocol to be used when
 accessing http URI without SSL/TLS.
-Available protocols: h2c and http/1.1
+Available protocols: h2c and
+http/1.1
 .sp
 Default: \fBh2c\fP
 .UNINDENT
@@ -346,7 +350,8 @@ compression.  Let \fBdecompressed(headers)\fP to the number of bytes
 used for header fields after decompression.  The \fBspace savings\fP
 is calculated  by (1 \- \fBheaders\fP  / \fBdecompressed(headers)\fP) *
 100.  For HTTP/1.1, this is usually  0.00%, since it does not have
-header compression.  For HTTP/2, it shows some insightful numbers.
+header compression.  For HTTP/2 and SPDY, it shows some insightful
+numbers.
 .TP
 .B data
 The number of response body bytes received from the server.
@@ -443,7 +448,7 @@ h2load sets large flow control window by default, and effectively
 disables flow control to avoid under utilization of server
 performance.  To set smaller flow control window, use \fI\%\-w\fP and
 \fI\%\-W\fP options.  For example, use \fB\-w16 \-W16\fP to set default
-window size described in HTTP/2 protocol specification.
+window size described in HTTP/2 and SPDY protocol specification.
 .SH SEE ALSO
 .sp
 \fBnghttp(1)\fP, \fBnghttpd(1)\fP, \fBnghttpx(1)\fP

doc/h2load.1.rst

@@ -14,7 +14,7 @@ SYNOPSIS
 DESCRIPTION
 -----------
 
-benchmarking tool for HTTP/2 server
+benchmarking tool for HTTP/2 and SPDY server
 
 .. describe:: <URI>
 
@@ -76,13 +76,16 @@ OPTIONS
 .. option:: -w, --window-bits=<N>
 
     Sets the stream level initial window size to (2\*\*<N>)-1.
+    For SPDY, 2\*\*<N> is used instead.
 
     Default: ``30``
 
 .. option:: -W, --connection-window-bits=<N>
 
     Sets  the  connection  level   initial  window  size  to
-    (2\*\*<N>)-1.
+    (2\*\*<N>)-1.  For SPDY, if <N>  is strictly less than 16,
+    this option  is ignored.   Otherwise 2\*\*<N> is  used for
+    SPDY.
 
     Default: ``30``
 
@@ -101,7 +104,8 @@ OPTIONS
 
     Specify ALPN identifier of the  protocol to be used when
     accessing http URI without SSL/TLS.
-    Available protocols: h2c and http/1.1
+    Available protocols: h2c and
+    http/1.1
 
     Default: ``h2c``
 
@@ -293,7 +297,8 @@ traffic
     used for header fields after decompression.  The ``space savings``
     is calculated  by (1 - ``headers``  / ``decompressed(headers)``) *
     100.  For HTTP/1.1, this is usually  0.00%, since it does not have
-    header compression.  For HTTP/2, it shows some insightful numbers.
+    header compression.  For HTTP/2 and SPDY, it shows some insightful
+    numbers.
   data
     The number of response body bytes received from the server.
 
@@ -361,7 +366,7 @@ h2load sets large flow control window by default, and effectively
 disables flow control to avoid under utilization of server
 performance.  To set smaller flow control window, use :option:`-w` and
 :option:`-W` options.  For example, use ``-w16 -W16`` to set default
-window size described in HTTP/2 protocol specification.
+window size described in HTTP/2 and SPDY protocol specification.
 
 SEE ALSO
 --------

doc/h2load.h2r

@@ -41,7 +41,8 @@ traffic
     used for header fields after decompression.  The ``space savings``
     is calculated  by (1 - ``headers``  / ``decompressed(headers)``) *
     100.  For HTTP/1.1, this is usually  0.00%, since it does not have
-    header compression.  For HTTP/2, it shows some insightful numbers.
+    header compression.  For HTTP/2 and SPDY, it shows some insightful
+    numbers.
   data
     The number of response body bytes received from the server.
 
@@ -109,7 +110,7 @@ h2load sets large flow control window by default, and effectively
 disables flow control to avoid under utilization of server
 performance.  To set smaller flow control window, use :option:`-w` and
 :option:`-W` options.  For example, use ``-w16 -W16`` to set default
-window size described in HTTP/2 protocol specification.
+window size described in HTTP/2 and SPDY protocol specification.
 
 SEE ALSO
 --------

doc/nghttp.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTP" "1" "Feb 02, 2018" "1.30.0" "nghttp2"
+.TH "NGHTTP" "1" "Nov 25, 2017" "1.28.0" "nghttp2"
 .SH NAME
 nghttp \- HTTP/2 client
 .

doc/nghttpd.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTPD" "1" "Feb 02, 2018" "1.30.0" "nghttp2"
+.TH "NGHTTPD" "1" "Nov 25, 2017" "1.28.0" "nghttp2"
 .SH NAME
 nghttpd \- HTTP/2 server
 .

doc/nghttpx.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH "NGHTTPX" "1" "Feb 02, 2018" "1.30.0" "nghttp2"
+.TH "NGHTTPX" "1" "Nov 25, 2017" "1.28.0" "nghttp2"
 .SH NAME
 nghttpx \- HTTP/2 proxy
 .
@@ -35,7 +35,7 @@ level margin: \\n[rst2man-indent\\n[rst2man-indent-level]]
 \fBnghttpx\fP [OPTIONS]... [<PRIVATE_KEY> <CERT>]
 .SH DESCRIPTION
 .sp
-A reverse proxy for HTTP/2, and HTTP/1.
+A reverse proxy for HTTP/2, HTTP/1 and SPDY.
 .INDENT 0.0
 .TP
 .B <PRIVATE_KEY>
@@ -228,13 +228,6 @@ the   same   <PATTERN>.    It    is   advised   to   set
 "redirect\-if\-no\-tls"    parameter   to    all   backends
 explicitly if this feature is desired.
 .sp
-If "upgrade\-scheme"  parameter is used along  with "tls"
-parameter, HTTP/2 :scheme pseudo header field is changed
-to "https" from "http" when forwarding a request to this
-particular backend.  This is  a workaround for a backend
-server  which  requires  "https" :scheme  pseudo  header
-field on TLS encrypted connection.
-.sp
 Since ";" and ":" are  used as delimiter, <PATTERN> must
 not  contain these  characters.  Since  ";" has  special
 meaning in shell, the option value must be quoted.
@@ -478,7 +471,8 @@ this option will be simply ignored.
 .INDENT 0.0
 .TP
 .B \-\-frontend\-http2\-read\-timeout=<DURATION>
-Specify read timeout for HTTP/2 frontend connection.
+Specify  read  timeout  for  HTTP/2  and  SPDY  frontend
+connection.
 .sp
 Default: \fB3m\fP
 .UNINDENT
@@ -507,16 +501,16 @@ Default: \fB1m\fP
 .INDENT 0.0
 .TP
 .B \-\-stream\-read\-timeout=<DURATION>
-Specify  read timeout  for HTTP/2  streams.  0  means no
-timeout.
+Specify  read timeout  for HTTP/2  and SPDY  streams.  0
+means no timeout.
 .sp
 Default: \fB0\fP
 .UNINDENT
 .INDENT 0.0
 .TP
 .B \-\-stream\-write\-timeout=<DURATION>
-Specify write  timeout for  HTTP/2 streams.  0  means no
-timeout.
+Specify write  timeout for  HTTP/2 and SPDY  streams.  0
+means no timeout.
 .sp
 Default: \fB1m\fP
 .UNINDENT
@@ -989,12 +983,12 @@ HTTP/2.   To  use  those   cipher  suites  with  HTTP/2,
 consider   to  use   \fI\%\-\-client\-no\-http2\-cipher\-black\-list\fP
 option.  But be aware its implications.
 .UNINDENT
-.SS HTTP/2
+.SS HTTP/2 and SPDY
 .INDENT 0.0
 .TP
 .B \-c, \-\-frontend\-http2\-max\-concurrent\-streams=<N>
 Set the maximum number of  the concurrent streams in one
-frontend HTTP/2 session.
+frontend HTTP/2 and SPDY session.
 .sp
 Default: \(ga\(ga 100\(ga\(ga
 .UNINDENT
@@ -1011,16 +1005,17 @@ Default: \fB100\fP
 .INDENT 0.0
 .TP
 .B \-\-frontend\-http2\-window\-size=<SIZE>
-Sets  the  per\-stream  initial  window  size  of  HTTP/2
-frontend connection.
+Sets the  per\-stream initial  window size of  HTTP/2 and
+SPDY frontend connection.
 .sp
 Default: \fB65535\fP
 .UNINDENT
 .INDENT 0.0
 .TP
 .B \-\-frontend\-http2\-connection\-window\-size=<SIZE>
-Sets the  per\-connection window size of  HTTP/2 frontend
-connection.
+Sets the  per\-connection window size of  HTTP/2 and SPDY
+frontend  connection.  For  SPDY  connection, the  value
+less than 64KiB is rounded up to 64KiB.
 .sp
 Default: \fB65535\fP
 .UNINDENT
@@ -1061,7 +1056,8 @@ default mode and HTTP/2  frontend via Link header field.
 It is  also supported if  both frontend and  backend are
 HTTP/2 in default mode.  In  this case, server push from
 backend session is relayed  to frontend, and server push
-via Link header field is also supported.
+via Link header field  is also supported.  SPDY frontend
+does not support server push.
 .UNINDENT
 .INDENT 0.0
 .TP
@@ -1132,7 +1128,7 @@ Default: \fB4K\fP
 .INDENT 0.0
 .TP
 .B (default mode)
-Accept  HTTP/2,  and  HTTP/1.1 over  SSL/TLS.   "no\-tls"
+Accept HTTP/2, SPDY and HTTP/1.1 over SSL/TLS.  "no\-tls"
 parameter is  used in  \fI\%\-\-frontend\fP option,  accept HTTP/2
 and HTTP/1.1 over cleartext  TCP.  The incoming HTTP/1.1
 connection  can  be  upgraded  to  HTTP/2  through  HTTP
@@ -1469,7 +1465,7 @@ Default: \fB443\fP
 .B \-\-api\-max\-request\-body=<SIZE>
 Set the maximum size of request body for API request.
 .sp
-Default: \fB32M\fP
+Default: \fB16K\fP
 .UNINDENT
 .SS DNS
 .INDENT 0.0

doc/nghttpx.1.rst

@@ -14,7 +14,7 @@ SYNOPSIS
 DESCRIPTION
 -----------
 
-A reverse proxy for HTTP/2, and HTTP/1.
+A reverse proxy for HTTP/2, HTTP/1 and SPDY.
 
 .. describe:: <PRIVATE_KEY>
 
@@ -212,13 +212,6 @@ Connections
     "redirect-if-no-tls"    parameter   to    all   backends
     explicitly if this feature is desired.
 
-    If "upgrade-scheme"  parameter is used along  with "tls"
-    parameter, HTTP/2 :scheme pseudo header field is changed
-    to "https" from "http" when forwarding a request to this
-    particular backend.  This is  a workaround for a backend
-    server  which  requires  "https" :scheme  pseudo  header
-    field on TLS encrypted connection.
-
     Since ";" and ":" are  used as delimiter, <PATTERN> must
     not  contain these  characters.  Since  ";" has  special
     meaning in shell, the option value must be quoted.
@@ -447,7 +440,8 @@ Timeout
 
 .. option:: --frontend-http2-read-timeout=<DURATION>
 
-    Specify read timeout for HTTP/2 frontend connection.
+    Specify  read  timeout  for  HTTP/2  and  SPDY  frontend
+    connection.
 
     Default: ``3m``
 
@@ -472,15 +466,15 @@ Timeout
 
 .. option:: --stream-read-timeout=<DURATION>
 
-    Specify  read timeout  for HTTP/2  streams.  0  means no
-    timeout.
+    Specify  read timeout  for HTTP/2  and SPDY  streams.  0
+    means no timeout.
 
     Default: ``0``
 
 .. option:: --stream-write-timeout=<DURATION>
 
-    Specify write  timeout for  HTTP/2 streams.  0  means no
-    timeout.
+    Specify write  timeout for  HTTP/2 and SPDY  streams.  0
+    means no timeout.
 
     Default: ``1m``
 
@@ -909,13 +903,13 @@ SSL/TLS
     option.  But be aware its implications.
 
 
-HTTP/2
-~~~~~~
+HTTP/2 and SPDY
+~~~~~~~~~~~~~~~
 
 .. option:: -c, --frontend-http2-max-concurrent-streams=<N>
 
     Set the maximum number of  the concurrent streams in one
-    frontend HTTP/2 session.
+    frontend HTTP/2 and SPDY session.
 
     Default: `` 100``
 
@@ -930,15 +924,16 @@ HTTP/2
 
 .. option:: --frontend-http2-window-size=<SIZE>
 
-    Sets  the  per-stream  initial  window  size  of  HTTP/2
-    frontend connection.
+    Sets the  per-stream initial  window size of  HTTP/2 and
+    SPDY frontend connection.
 
     Default: ``65535``
 
 .. option:: --frontend-http2-connection-window-size=<SIZE>
 
-    Sets the  per-connection window size of  HTTP/2 frontend
-    connection.
+    Sets the  per-connection window size of  HTTP/2 and SPDY
+    frontend  connection.  For  SPDY  connection, the  value
+    less than 64KiB is rounded up to 64KiB.
 
     Default: ``65535``
 
@@ -974,7 +969,8 @@ HTTP/2
     It is  also supported if  both frontend and  backend are
     HTTP/2 in default mode.  In  this case, server push from
     backend session is relayed  to frontend, and server push
-    via Link header field is also supported.
+    via Link header field  is also supported.  SPDY frontend
+    does not support server push.
 
 .. option:: --frontend-http2-optimize-write-buffer-size
 
@@ -1042,7 +1038,7 @@ Mode
 .. describe:: (default mode)
 
     
-    Accept  HTTP/2,  and  HTTP/1.1 over  SSL/TLS.   "no-tls"
+    Accept HTTP/2, SPDY and HTTP/1.1 over SSL/TLS.  "no-tls"
     parameter is  used in  :option:`--frontend` option,  accept HTTP/2
     and HTTP/1.1 over cleartext  TCP.  The incoming HTTP/1.1
     connection  can  be  upgraded  to  HTTP/2  through  HTTP
@@ -1332,7 +1328,7 @@ API
 
     Set the maximum size of request body for API request.
 
-    Default: ``32M``
+    Default: ``16K``
 
 
 DNS

doc/sources/h2load-howto.rst

@@ -3,8 +3,10 @@
 h2load - HTTP/2 benchmarking tool - HOW-TO
 ==========================================
 
-:doc:`h2load.1` is benchmarking tool for HTTP/2 and HTTP/1.1.  It
-supports SSL/TLS and clear text for all supported protocols.
+:doc:`h2load.1` is benchmarking tool for HTTP/2 and HTTP/1.1.  If
+built with spdylay (http://tatsuhiro-t.github.io/spdylay/) library, it
+also supports SPDY protocol.  It supports SSL/TLS and clear text for
+all supported protocols.
 
 Compiling from source
 ---------------------
@@ -84,18 +86,20 @@ seconds warming up period:
 Flow Control
 ------------
 
-HTTP/2 has flow control and it may affect benchmarking results.  By
-default, h2load uses large enough flow control window, which
-effectively disables flow control.  To adjust receiver flow control
-window size, there are following options:
+HTTP/2 and SPDY/3 or later employ flow control and it may affect
+benchmarking results.  By default, h2load uses large enough flow
+control window, which effectively disables flow control.  To adjust
+receiver flow control window size, there are following options:
 
 :option:`-w`
    Sets  the stream  level  initial  window size  to
-   (2**<N>)-1.
+   (2**<N>)-1.  For SPDY, 2**<N> is used instead.
 
 :option:`-W`
    Sets the connection level  initial window size to
-   (2**<N>)-1.
+   (2**<N>)-1.  For  SPDY, if  <N> is  strictly less
+   than  16,  this  option  is  ignored.   Otherwise
+   2**<N> is used for SPDY.
 
 Multi-Threading
 ---------------

doc/sources/nghttpx-howto.rst

@@ -4,10 +4,10 @@ nghttpx - HTTP/2 proxy - HOW-TO
 ===============================
 
 :doc:`nghttpx.1` is a proxy translating protocols between HTTP/2 and
-other protocols (e.g., HTTP/1).  It operates in several modes and each
-mode may require additional programs to work with.  This article
-describes each operation mode and explains the intended use-cases.  It
-also covers some useful options later.
+other protocols (e.g., HTTP/1, SPDY).  It operates in several modes
+and each mode may require additional programs to work with.  This
+article describes each operation mode and explains the intended
+use-cases.  It also covers some useful options later.
 
 Default mode
 ------------
@@ -15,7 +15,9 @@ Default mode
 If nghttpx is invoked without :option:`--http2-proxy`, it operates in
 default mode.  In this mode, it works as reverse proxy (gateway) for
 both HTTP/2 and HTTP/1 clients to backend servers.  This is also known
-as "HTTP/2 router".
+as "HTTP/2 router".  If nghttpx is linked with spdylay library and
+frontend connection is SSL/TLS, the frontend also supports SPDY
+protocol.
 
 By default, frontend connection is encrypted using SSL/TLS.  So
 server's private key and certificate must be supplied to the command
@@ -23,10 +25,11 @@ line (or through configuration file).  In this case, the frontend
 protocol selection will be done via ALPN or NPN.
 
 To turn off encryption on frontend connection, use ``no-tls`` keyword
-in :option:`--frontend` option.  HTTP/2 and HTTP/1 are available on
-the frontend, and an HTTP/1 connection can be upgraded to HTTP/2 using
-HTTP Upgrade.  Starting HTTP/2 connection by sending HTTP/2 connection
-preface is also supported.
+in :option:`--frontend` option.  In this case, SPDY protocol is not
+available even if spdylay library is linked to nghttpx.  HTTP/2 and
+HTTP/1 are available on the frontend, and an HTTP/1 connection can be
+upgraded to HTTP/2 using HTTP Upgrade.  Starting HTTP/2 connection by
+sending HTTP/2 connection preface is also supported.
 
 nghttpx can listen on multiple frontend addresses.  This is achieved
 by using multiple :option:`--frontend` options.  For each frontend
@@ -68,8 +71,9 @@ mode`_.  The difference is that this mode acts like a forward proxy and
 assumes the backend is an HTTP proxy server (e.g., Squid, Apache Traffic
 Server).  HTTP/1 requests must include an absolute URI in request line.
 
-By default, the frontend connection is encrypted.  So this mode is
-also called secure proxy.
+By default, the frontend connection is encrypted.  So this mode is also
+called secure proxy.  If nghttpx is linked with spdylay, it supports
+SPDY protocols and it works as so called SPDY proxy.
 
 To turn off encryption on the frontend connection, use ``no-tls`` keyword
 in :option:`--frontend` option.
@@ -98,8 +102,8 @@ like this:
 At the time of this writing, Firefox 41 and Chromium v46 can use
 nghttpx as HTTP/2 proxy.
 
-To make Firefox or Chromium use nghttpx as HTTP/2 proxy, user has to
-create proxy.pac script file like this:
+To make Firefox or Chromium use nghttpx as HTTP/2 or SPDY proxy, user
+has to create proxy.pac script file like this:
 
 .. code-block:: javascript
 

genheaderfunc.py

@@ -31,6 +31,7 @@ HEADERS = [
     "user-agent",
     "date",
     "content-type",
+    "nghttpx-0rtt-uniq",
     # disallowed h1 headers
     'connection',
     'keep-alive',

gennghttpxfun.py

@@ -168,6 +168,11 @@ OPTIONS = [
     "no-strip-incoming-x-forwarded-proto",
     "ocsp-startup",
     "no-verify-ocsp",
+    "tls-anti-replay-memcached",
+    "tls-anti-replay-memcached-cert-file",
+    "tls-anti-replay-memcached-private-key-file",
+    "tls-anti-replay-memcached-address-family",
+    "no-strip-incoming-nghttpx-0rtt-uniq",
 ]
 
 LOGVARS = [

integration-tests/Makefile.am

@@ -24,6 +24,7 @@
 GO_FILES = \
 	nghttpx_http1_test.go \
 	nghttpx_http2_test.go \
+	nghttpx_spdy_test.go \
 	server_tester.go
 
 EXTRA_DIST = \
@@ -42,6 +43,7 @@ EXTRA_DIST = \
 itprep:
 	go get -d -v golang.org/x/net/http2
 	go get -d -v github.com/tatsuhiro-t/go-nghttp2
+	go get -d -v github.com/tatsuhiro-t/spdy
 	go get -d -v golang.org/x/net/websocket
 
 it:

integration-tests/nghttpx_spdy_test.go

@@ -0,0 +1,664 @@
+package nghttp2
+
+import (
+	"encoding/json"
+	"github.com/tatsuhiro-t/spdy"
+	"golang.org/x/net/http2/hpack"
+	"net/http"
+	"testing"
+)
+
+// TestS3H1PlainGET tests whether simple SPDY GET request works.
+func TestS3H1PlainGET(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, noopHandler)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1PlainGET",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	want := 200
+	if got := res.status; got != want {
+		t.Errorf("status = %v; want %v", got, want)
+	}
+}
+
+// TestS3H1BadRequestCL tests that server rejects request whose
+// content-length header field value does not match its request body
+// size.
+func TestS3H1BadRequestCL(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, noopHandler)
+	defer st.Close()
+
+	// we set content-length: 1024, but the actual request body is
+	// 3 bytes.
+	res, err := st.spdy(requestParam{
+		name:   "TestS3H1BadRequestCL",
+		method: "POST",
+		header: []hpack.HeaderField{
+			pair("content-length", "1024"),
+		},
+		body: []byte("foo"),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	want := spdy.ProtocolError
+	if got := res.spdyRstErrCode; got != want {
+		t.Errorf("res.spdyRstErrCode = %v; want %v", got, want)
+	}
+}
+
+// TestS3H1MultipleRequestCL tests that server rejects request with
+// multiple Content-Length request header fields.
+func TestS3H1MultipleRequestCL(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Errorf("server should not forward bad request")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1MultipleRequestCL",
+		header: []hpack.HeaderField{
+			pair("content-length", "1"),
+			pair("content-length", "1"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	want := 400
+	if got := res.status; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1InvalidRequestCL tests that server rejects request with
+// Content-Length which cannot be parsed as a number.
+func TestS3H1InvalidRequestCL(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Errorf("server should not forward bad request")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1InvalidRequestCL",
+		header: []hpack.HeaderField{
+			pair("content-length", ""),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	want := 400
+	if got := res.status; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1GenerateVia tests that server generates Via header field to and
+// from backend server.
+func TestS3H1GenerateVia(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "1.1 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1GenerateVia",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "1.1 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1AppendVia tests that server adds value to existing Via
+// header field to and from backend server.
+func TestS3H1AppendVia(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo, 1.1 nghttpx"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1AppendVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar, 1.1 nghttpx"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1NoVia tests that server does not add value to existing Via
+// header field to and from backend server.
+func TestS3H1NoVia(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--no-via"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("Via"), "foo"; got != want {
+			t.Errorf("Via: %v; want %v", got, want)
+		}
+		w.Header().Add("Via", "bar")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1NoVia",
+		header: []hpack.HeaderField{
+			pair("via", "foo"),
+		},
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.header.Get("Via"), "bar"; got != want {
+		t.Errorf("Via: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1HeaderFieldBuffer tests that request with header fields
+// larger than configured buffer size is rejected.
+func TestS3H1HeaderFieldBuffer(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--request-header-field-buffer=10"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatal("execution path should not be here")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1HeaderFieldBuffer",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.spdyRstErrCode, spdy.InternalError; got != want {
+		t.Errorf("res.spdyRstErrCode: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1HeaderFields tests that request with header fields more
+// than configured number is rejected.
+func TestS3H1HeaderFields(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--max-request-header-fields=1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatal("execution path should not be here")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1HeaderFields",
+		// we have at least 5 pseudo-header fields sent, and
+		// that ensures that buffer limit exceeds.
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.spdyRstErrCode, spdy.InternalError; got != want {
+		t.Errorf("res.spdyRstErrCode: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1InvalidMethod tests that server rejects invalid method with
+// 501.
+func TestS3H1InvalidMethod(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Errorf("server should not forward this request")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3H1InvalidMethod",
+		method: "get",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 501; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1BadHost tests that server rejects request including bad
+// character in :host header field.
+func TestS3H1BadHost(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Errorf("server should not forward this request")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:      "TestS3H1BadHost",
+		authority: `foo\bar`,
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 400; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1BadScheme tests that server rejects request including bad
+// character in :scheme header field.
+func TestS3H1BadScheme(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Errorf("server should not forward this request")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3H1BadScheme",
+		scheme: `http*`,
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 400; got != want {
+		t.Errorf("status: %v; want %v", got, want)
+	}
+}
+
+// TestS3H1ReqPhaseSetHeader tests mruby request phase hook
+// modifies request header fields.
+func TestS3H1ReqPhaseSetHeader(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--mruby-file=" + testDir + "/req-set-header.rb"}, t, func(w http.ResponseWriter, r *http.Request) {
+		if got, want := r.Header.Get("User-Agent"), "mruby"; got != want {
+			t.Errorf("User-Agent = %v; want %v", got, want)
+		}
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1ReqPhaseSetHeader",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	if got, want := res.status, 200; got != want {
+		t.Errorf("status = %v; want %v", got, want)
+	}
+}
+
+// TestS3H1ReqPhaseReturn tests mruby request phase hook returns
+// custom response.
+func TestS3H1ReqPhaseReturn(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--mruby-file=" + testDir + "/req-return.rb"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1ReqPhaseReturn",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	if got, want := res.status, 404; got != want {
+		t.Errorf("status = %v; want %v", got, want)
+	}
+
+	hdtests := []struct {
+		k, v string
+	}{
+		{"content-length", "20"},
+		{"from", "mruby"},
+	}
+	for _, tt := range hdtests {
+		if got, want := res.header.Get(tt.k), tt.v; got != want {
+			t.Errorf("%v = %v; want %v", tt.k, got, want)
+		}
+	}
+
+	if got, want := string(res.body), "Hello World from req"; got != want {
+		t.Errorf("body = %v; want %v", got, want)
+	}
+}
+
+// TestS3H1RespPhaseSetHeader tests mruby response phase hook modifies
+// response header fields.
+func TestS3H1RespPhaseSetHeader(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--mruby-file=" + testDir + "/resp-set-header.rb"}, t, noopHandler)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1RespPhaseSetHeader",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	if got, want := res.status, 200; got != want {
+		t.Errorf("status = %v; want %v", got, want)
+	}
+
+	if got, want := res.header.Get("alpha"), "bravo"; got != want {
+		t.Errorf("alpha = %v; want %v", got, want)
+	}
+}
+
+// TestS3H1RespPhaseReturn tests mruby response phase hook returns
+// custom response.
+func TestS3H1RespPhaseReturn(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--mruby-file=" + testDir + "/resp-return.rb"}, t, noopHandler)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H1RespPhaseReturn",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	if got, want := res.status, 404; got != want {
+		t.Errorf("status = %v; want %v", got, want)
+	}
+
+	hdtests := []struct {
+		k, v string
+	}{
+		{"content-length", "21"},
+		{"from", "mruby"},
+	}
+	for _, tt := range hdtests {
+		if got, want := res.header.Get(tt.k), tt.v; got != want {
+			t.Errorf("%v = %v; want %v", tt.k, got, want)
+		}
+	}
+
+	if got, want := string(res.body), "Hello World from resp"; got != want {
+		t.Errorf("body = %v; want %v", got, want)
+	}
+}
+
+// // TestS3H2ConnectFailure tests that server handles the situation that
+// // connection attempt to HTTP/2 backend failed.
+// func TestS3H2ConnectFailure(t *testing.T) {
+// 	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--http2-bridge"}, t, noopHandler)
+// 	defer st.Close()
+
+// 	// simulate backend connect attempt failure
+// 	st.ts.Close()
+
+// 	res, err := st.spdy(requestParam{
+// 		name: "TestS3H2ConnectFailure",
+// 	})
+// 	if err != nil {
+// 		t.Fatalf("Error st.spdy() = %v", err)
+// 	}
+// 	want := 503
+// 	if got := res.status; got != want {
+// 		t.Errorf("status: %v; want %v", got, want)
+// 	}
+// }
+
+// TestS3H2ReqPhaseReturn tests mruby request phase hook returns
+// custom response.
+func TestS3H2ReqPhaseReturn(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--http2-bridge", "--mruby-file=" + testDir + "/req-return.rb"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H2ReqPhaseReturn",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	if got, want := res.status, 404; got != want {
+		t.Errorf("status = %v; want %v", got, want)
+	}
+
+	hdtests := []struct {
+		k, v string
+	}{
+		{"content-length", "20"},
+		{"from", "mruby"},
+	}
+	for _, tt := range hdtests {
+		if got, want := res.header.Get(tt.k), tt.v; got != want {
+			t.Errorf("%v = %v; want %v", tt.k, got, want)
+		}
+	}
+
+	if got, want := string(res.body), "Hello World from req"; got != want {
+		t.Errorf("body = %v; want %v", got, want)
+	}
+}
+
+// TestS3H2RespPhaseReturn tests mruby response phase hook returns
+// custom response.
+func TestS3H2RespPhaseReturn(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--http2-bridge", "--mruby-file=" + testDir + "/resp-return.rb"}, t, noopHandler)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3H2RespPhaseReturn",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+
+	if got, want := res.status, 404; got != want {
+		t.Errorf("status = %v; want %v", got, want)
+	}
+
+	hdtests := []struct {
+		k, v string
+	}{
+		{"content-length", "21"},
+		{"from", "mruby"},
+	}
+	for _, tt := range hdtests {
+		if got, want := res.header.Get(tt.k), tt.v; got != want {
+			t.Errorf("%v = %v; want %v", tt.k, got, want)
+		}
+	}
+
+	if got, want := string(res.body), "Hello World from resp"; got != want {
+		t.Errorf("body = %v; want %v", got, want)
+	}
+}
+
+// TestS3APIBackendconfig exercise backendconfig API endpoint routine
+// for successful case.
+func TestS3APIBackendconfig(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APIBackendconfig",
+		path:   "/api/v1beta1/backendconfig",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3APIBackendconfigQuery exercise backendconfig API endpoint
+// routine with query.
+func TestS3APIBackendconfigQuery(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APIBackendconfigQuery",
+		path:   "/api/v1beta1/backendconfig?foo=bar",
+		method: "PUT",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Success"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 200; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3APIBackendconfigBadMethod exercise backendconfig API endpoint
+// routine with bad method.
+func TestS3APIBackendconfigBadMethod(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APIBackendconfigBadMethod",
+		path:   "/api/v1beta1/backendconfig",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 405; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 405; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3APINotFound exercise backendconfig API endpoint routine when
+// API endpoint is not found.
+func TestS3APINotFound(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3010;api"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3010)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:   "TestS3APINotFound",
+		path:   "/api/notfound",
+		method: "GET",
+		body: []byte(`# comment
+backend=127.0.0.1,3011
+
+`),
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 404; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+
+	var apiResp APIResponse
+	err = json.Unmarshal(res.body, &apiResp)
+	if err != nil {
+		t.Fatalf("Error unmarshaling API response: %v", err)
+	}
+	if got, want := apiResp.Status, "Failure"; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+	if got, want := apiResp.Code, 404; got != want {
+		t.Errorf("apiResp.Status: %v; want %v", got, want)
+	}
+}
+
+// TestS3Healthmon tests health monitor endpoint.
+func TestS3Healthmon(t *testing.T) {
+	st := newServerTesterTLSConnectPort([]string{"--npn-list=spdy/3.1", "-f127.0.0.1,3011;healthmon"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatalf("request should not be forwarded")
+	}, 3011)
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name: "TestS3Healthmon",
+		path: "/alpha/bravo",
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 200; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+}
+
+// TestS3ResponseBeforeRequestEnd tests the situation where response
+// ends before request body finishes.
+func TestS3ResponseBeforeRequestEnd(t *testing.T) {
+	st := newServerTesterTLS([]string{"--npn-list=spdy/3.1", "--mruby-file=" + testDir + "/req-return.rb"}, t, func(w http.ResponseWriter, r *http.Request) {
+		t.Fatal("request should not be forwarded")
+	})
+	defer st.Close()
+
+	res, err := st.spdy(requestParam{
+		name:        "TestS3ResponseBeforeRequestEnd",
+		noEndStream: true,
+	})
+	if err != nil {
+		t.Fatalf("Error st.spdy() = %v", err)
+	}
+	if got, want := res.status, 404; got != want {
+		t.Errorf("res.status: %v; want %v", got, want)
+	}
+}

lib/nghttp2_session.c

@@ -2225,9 +2225,8 @@ static int session_prep_frame(nghttp2_session *session,
       assert(session->obq_flood_counter_ > 0);
       --session->obq_flood_counter_;
     }
-    /* PING frame is allowed to be sent unless termination GOAWAY is
-       sent */
-    if (session->goaway_flags & NGHTTP2_GOAWAY_TERM_ON_SEND) {
+
+    if (session_is_closing(session)) {
       return NGHTTP2_ERR_SESSION_CLOSING;
     }
     nghttp2_frame_pack_ping(&session->aob.framebufs, &frame->ping);
@@ -2418,16 +2417,19 @@ static int session_close_stream_on_goaway(nghttp2_session *session,
   nghttp2_stream *stream, *next_stream;
   nghttp2_close_stream_on_goaway_arg arg = {session, NULL, last_stream_id,
                                             incoming};
+  uint32_t error_code;
 
   rv = nghttp2_map_each(&session->streams, find_stream_on_goaway_func, &arg);
   assert(rv == 0);
 
+  error_code =
+      session->server && incoming ? NGHTTP2_REFUSED_STREAM : NGHTTP2_CANCEL;
+
   stream = arg.head;
   while (stream) {
     next_stream = stream->closed_next;
     stream->closed_next = NULL;
-    rv = nghttp2_session_close_stream(session, stream->stream_id,
-                                      NGHTTP2_REFUSED_STREAM);
+    rv = nghttp2_session_close_stream(session, stream->stream_id, error_code);
 
     /* stream may be deleted here */
 

src/Makefile.am

@@ -40,6 +40,7 @@ AM_CPPFLAGS = \
 	-I$(top_srcdir)/lib \
 	-I$(top_srcdir)/src/includes \
 	-I$(top_srcdir)/third-party \
+	@LIBSPDYLAY_CFLAGS@ \
 	@LIBXML2_CFLAGS@ \
 	@LIBEV_CFLAGS@ \
 	@OPENSSL_CFLAGS@ \
@@ -51,6 +52,7 @@ AM_CPPFLAGS = \
 LDADD = $(top_builddir)/lib/libnghttp2.la \
 	$(top_builddir)/third-party/libhttp-parser.la \
 	@JEMALLOC_LIBS@ \
+	@LIBSPDYLAY_LIBS@ \
 	@LIBXML2_LIBS@ \
 	@LIBEV_LIBS@ \
 	@OPENSSL_LIBS@ \
@@ -95,6 +97,10 @@ h2load_SOURCES = util.cc util.h \
 	h2load_http2_session.cc h2load_http2_session.h \
 	h2load_http1_session.cc h2load_http1_session.h
 
+if HAVE_SPDYLAY
+h2load_SOURCES += h2load_spdy_session.cc h2load_spdy_session.h
+endif # HAVE_SPDYLAY
+
 NGHTTPX_SRCS = \
 	util.cc util.h http2.cc http2.h timegm.c timegm.h base64.h \
 	app_helper.cc app_helper.h \
@@ -142,6 +148,10 @@ NGHTTPX_SRCS = \
 	buffer.h memchunk.h template.h allocator.h \
 	xsi_strerror.c xsi_strerror.h
 
+if HAVE_SPDYLAY
+NGHTTPX_SRCS += shrpx_spdy_upstream.cc shrpx_spdy_upstream.h
+endif # HAVE_SPDYLAY
+
 if HAVE_MRUBY
 NGHTTPX_SRCS += \
 	shrpx_mruby.cc shrpx_mruby.h \

src/h2load.cc

@@ -46,12 +46,19 @@
 #include <future>
 #include <random>
 
+#ifdef HAVE_SPDYLAY
+#include <spdylay/spdylay.h>
+#endif // HAVE_SPDYLAY
+
 #include <openssl/err.h>
 
 #include "http-parser/http_parser.h"
 
 #include "h2load_http1_session.h"
 #include "h2load_http2_session.h"
+#ifdef HAVE_SPDYLAY
+#include "h2load_spdy_session.h"
+#endif // HAVE_SPDYLAY
 #include "tls.h"
 #include "http2.h"
 #include "util.h"
@@ -871,6 +878,14 @@ int Client::connection_made() {
       } else if (util::streq(NGHTTP2_H1_1, proto)) {
         session = make_unique<Http1Session>(this);
       }
+#ifdef HAVE_SPDYLAY
+      else {
+        auto spdy_version = spdylay_npn_get_version(next_proto, next_proto_len);
+        if (spdy_version) {
+          session = make_unique<SpdySession>(this, spdy_version);
+        }
+      }
+#endif // HAVE_SPDYLAY
 
       // Just assign next_proto to selected_proto anyway to show the
       // negotiation result.
@@ -915,6 +930,20 @@ int Client::connection_made() {
       session = make_unique<Http1Session>(this);
       selected_proto = NGHTTP2_H1_1.str();
       break;
+#ifdef HAVE_SPDYLAY
+    case Config::PROTO_SPDY2:
+      session = make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY2);
+      selected_proto = "spdy/2";
+      break;
+    case Config::PROTO_SPDY3:
+      session = make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY3);
+      selected_proto = "spdy/3";
+      break;
+    case Config::PROTO_SPDY3_1:
+      session = make_unique<SpdySession>(this, SPDYLAY_PROTO_SPDY3_1);
+      selected_proto = "spdy/3.1";
+      break;
+#endif // HAVE_SPDYLAY
     default:
       // unreachable
       assert(0);
@@ -1759,13 +1788,17 @@ void print_version(std::ostream &out) {
 namespace {
 void print_usage(std::ostream &out) {
   out << R"(Usage: h2load [OPTIONS]... [URI]...
-benchmarking tool for HTTP/2 server)"
+benchmarking tool for HTTP/2 and SPDY server)"
       << std::endl;
 }
 } // namespace
 
 namespace {
-constexpr char DEFAULT_NPN_LIST[] = "h2,h2-16,h2-14,http/1.1";
+constexpr char DEFAULT_NPN_LIST[] = "h2,h2-16,h2-14"
+#ifdef HAVE_SPDYLAY
+                                    ",spdy/3.1,spdy/3,spdy/2"
+#endif // HAVE_SPDYLAY
+                                    ",http/1.1";
 } // namespace
 
 namespace {
@@ -1816,11 +1849,14 @@ Options:
               Default: 1
   -w, --window-bits=<N>
               Sets the stream level initial window size to (2**<N>)-1.
+              For SPDY, 2**<N> is used instead.
               Default: )"
       << config.window_bits << R"(
   -W, --connection-window-bits=<N>
               Sets  the  connection  level   initial  window  size  to
-              (2**<N>)-1.
+              (2**<N>)-1.  For SPDY, if <N>  is strictly less than 16,
+              this option  is ignored.   Otherwise 2**<N> is  used for
+              SPDY.
               Default: )" << config.connection_window_bits << R"(
   -H, --header=<HEADER>
               Add/Override a header to the requests.
@@ -1831,9 +1867,17 @@ Options:
       << config.ciphers << R"(
   -p, --no-tls-proto=<PROTOID>
               Specify ALPN identifier of the  protocol to be used when
-              accessing http URI without SSL/TLS.
-              Available protocols: )" << NGHTTP2_CLEARTEXT_PROTO_VERSION_ID
-      << R"( and )" << NGHTTP2_H1_1 << R"(
+              accessing http URI without SSL/TLS.)";
+
+#ifdef HAVE_SPDYLAY
+  out << R"(
+              Available protocols: spdy/2, spdy/3, spdy/3.1, )";
+#else  // !HAVE_SPDYLAY
+  out << R"(
+              Available protocols: )";
+#endif // !HAVE_SPDYLAY
+  out << NGHTTP2_CLEARTEXT_PROTO_VERSION_ID << R"( and
+              )" << NGHTTP2_H1_1 << R"(
               Default: )"
       << NGHTTP2_CLEARTEXT_PROTO_VERSION_ID << R"(
   -d, --data=<PATH>
@@ -2071,6 +2115,14 @@ int main(int argc, char **argv) {
         config.no_tls_proto = Config::PROTO_HTTP2;
       } else if (util::strieq(NGHTTP2_H1_1, proto)) {
         config.no_tls_proto = Config::PROTO_HTTP1_1;
+#ifdef HAVE_SPDYLAY
+      } else if (util::strieq_l("spdy/2", proto)) {
+        config.no_tls_proto = Config::PROTO_SPDY2;
+      } else if (util::strieq_l("spdy/3", proto)) {
+        config.no_tls_proto = Config::PROTO_SPDY3;
+      } else if (util::strieq_l("spdy/3.1", proto)) {
+        config.no_tls_proto = Config::PROTO_SPDY3_1;
+#endif // HAVE_SPDYLAY
       } else {
         std::cerr << "-p: unsupported protocol " << proto << std::endl;
         exit(EXIT_FAILURE);
@@ -2455,6 +2507,7 @@ int main(int argc, char **argv) {
 
   config.h1reqs.reserve(reqlines.size());
   config.nva.reserve(reqlines.size());
+  config.nv.reserve(reqlines.size());
 
   for (auto &req : reqlines) {
     // For HTTP/1.1
@@ -2504,6 +2557,35 @@ int main(int argc, char **argv) {
     }
 
     config.nva.push_back(std::move(nva));
+
+    // For spdylay
+    std::vector<const char *> cva;
+    // 3 for :path, :version, and possible content-length, 1 for
+    // terminal nullptr
+    cva.reserve(2 * (3 + shared_nva.size()) + 1);
+
+    cva.push_back(":path");
+    cva.push_back(req.c_str());
+
+    for (auto &nv : shared_nva) {
+      if (nv.name == ":authority") {
+        cva.push_back(":host");
+      } else {
+        cva.push_back(nv.name.c_str());
+      }
+      cva.push_back(nv.value.c_str());
+    }
+    cva.push_back(":version");
+    cva.push_back("HTTP/1.1");
+
+    if (!content_length_str.empty()) {
+      cva.push_back("content-length");
+      cva.push_back(content_length_str.c_str());
+    }
+
+    cva.push_back(nullptr);
+
+    config.nv.push_back(std::move(cva));
   }
 
   // Don't DOS our server!

src/h2load.h

@@ -64,6 +64,7 @@ struct Worker;
 
 struct Config {
   std::vector<std::vector<nghttp2_nv>> nva;
+  std::vector<std::vector<const char *>> nv;
   std::vector<std::string> h1reqs;
   std::vector<ev_tstamp> timings;
   nghttp2::Headers custom_headers;
@@ -92,7 +93,13 @@ struct Config {
   ev_tstamp conn_active_timeout;
   // amount of time to wait after the last request is made on a connection
   ev_tstamp conn_inactivity_timeout;
-  enum { PROTO_HTTP2, PROTO_HTTP1_1 } no_tls_proto;
+  enum {
+    PROTO_HTTP2,
+    PROTO_SPDY2,
+    PROTO_SPDY3,
+    PROTO_SPDY3_1,
+    PROTO_HTTP1_1
+  } no_tls_proto;
   uint32_t header_table_size;
   uint32_t encoder_header_table_size;
   // file descriptor for upload data

src/h2load_spdy_session.cc

@@ -0,0 +1,289 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2014 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "h2load_spdy_session.h"
+
+#include <cassert>
+#include <cerrno>
+
+#include "h2load.h"
+#include "util.h"
+
+using namespace nghttp2;
+
+namespace h2load {
+
+SpdySession::SpdySession(Client *client, uint16_t spdy_version)
+    : client_(client), session_(nullptr), spdy_version_(spdy_version) {}
+
+SpdySession::~SpdySession() { spdylay_session_del(session_); }
+
+namespace {
+void before_ctrl_send_callback(spdylay_session *session,
+                               spdylay_frame_type type, spdylay_frame *frame,
+                               void *user_data) {
+  auto client = static_cast<Client *>(user_data);
+  if (type != SPDYLAY_SYN_STREAM) {
+    return;
+  }
+  client->on_request(frame->syn_stream.stream_id);
+  auto req_stat = client->get_req_stat(frame->syn_stream.stream_id);
+  client->record_request_time(req_stat);
+}
+} // namespace
+
+namespace {
+void on_ctrl_recv_callback(spdylay_session *session, spdylay_frame_type type,
+                           spdylay_frame *frame, void *user_data) {
+  auto client = static_cast<Client *>(user_data);
+  if (type != SPDYLAY_SYN_REPLY) {
+    return;
+  }
+  for (auto p = frame->syn_reply.nv; *p; p += 2) {
+    auto name = *p;
+    auto value = *(p + 1);
+    auto namelen = strlen(name);
+    auto valuelen = strlen(value);
+    client->on_header(frame->syn_reply.stream_id,
+                      reinterpret_cast<const uint8_t *>(name), namelen,
+                      reinterpret_cast<const uint8_t *>(value), valuelen);
+    client->worker->stats.bytes_head_decomp += namelen + valuelen;
+  }
+
+  // Strictly speaking, we have to subtract 2 (unused field) if SPDY
+  // version is 2.  But it is already deprecated, and we don't do
+  // extra work for it.
+  client->worker->stats.bytes_head += frame->syn_reply.hd.length - 4;
+
+  if (frame->syn_stream.hd.flags & SPDYLAY_CTRL_FLAG_FIN) {
+    client->record_ttfb();
+  }
+}
+} // namespace
+
+namespace {
+void on_data_chunk_recv_callback(spdylay_session *session, uint8_t flags,
+                                 int32_t stream_id, const uint8_t *data,
+                                 size_t len, void *user_data) {
+  auto client = static_cast<Client *>(user_data);
+
+  client->record_ttfb();
+  client->worker->stats.bytes_body += len;
+
+  auto spdy_session = static_cast<SpdySession *>(client->session.get());
+
+  spdy_session->handle_window_update(stream_id, len);
+}
+} // namespace
+
+namespace {
+void on_stream_close_callback(spdylay_session *session, int32_t stream_id,
+                              spdylay_status_code status_code,
+                              void *user_data) {
+  auto client = static_cast<Client *>(user_data);
+  client->on_stream_close(stream_id, status_code == SPDYLAY_OK);
+}
+} // namespace
+
+namespace {
+ssize_t send_callback(spdylay_session *session, const uint8_t *data,
+                      size_t length, int flags, void *user_data) {
+  auto client = static_cast<Client *>(user_data);
+  auto &wb = client->wb;
+
+  if (wb.rleft() >= BACKOFF_WRITE_BUFFER_THRES) {
+    return SPDYLAY_ERR_WOULDBLOCK;
+  }
+
+  return wb.append(data, length);
+}
+} // namespace
+
+namespace {
+ssize_t file_read_callback(spdylay_session *session, int32_t stream_id,
+                           uint8_t *buf, size_t length, int *eof,
+                           spdylay_data_source *source, void *user_data) {
+  auto client = static_cast<Client *>(user_data);
+  auto config = client->worker->config;
+  auto req_stat = client->get_req_stat(stream_id);
+
+  ssize_t nread;
+  while ((nread = pread(config->data_fd, buf, length, req_stat->data_offset)) ==
+             -1 &&
+         errno == EINTR)
+    ;
+
+  if (nread == -1) {
+    return SPDYLAY_ERR_TEMPORAL_CALLBACK_FAILURE;
+  }
+
+  req_stat->data_offset += nread;
+
+  if (nread == 0 || req_stat->data_offset == config->data_length) {
+    *eof = 1;
+  }
+
+  return nread;
+}
+} // namespace
+
+void SpdySession::on_connect() {
+  spdylay_session_callbacks callbacks = {0};
+  callbacks.send_callback = send_callback;
+  callbacks.before_ctrl_send_callback = before_ctrl_send_callback;
+  callbacks.on_data_chunk_recv_callback = on_data_chunk_recv_callback;
+  callbacks.on_stream_close_callback = on_stream_close_callback;
+  callbacks.on_ctrl_recv_callback = on_ctrl_recv_callback;
+
+  spdylay_session_client_new(&session_, spdy_version_, &callbacks, client_);
+
+  int val = 1;
+  spdylay_session_set_option(session_, SPDYLAY_OPT_NO_AUTO_WINDOW_UPDATE, &val,
+                             sizeof(val));
+
+  spdylay_settings_entry iv;
+  iv.settings_id = SPDYLAY_SETTINGS_INITIAL_WINDOW_SIZE;
+  iv.flags = SPDYLAY_ID_FLAG_SETTINGS_NONE;
+  iv.value = (1 << client_->worker->config->window_bits);
+  spdylay_submit_settings(session_, SPDYLAY_FLAG_SETTINGS_NONE, &iv, 1);
+
+  auto config = client_->worker->config;
+
+  if (spdy_version_ >= SPDYLAY_PROTO_SPDY3_1 &&
+      config->connection_window_bits > 16) {
+    auto delta =
+        (1 << config->connection_window_bits) - SPDYLAY_INITIAL_WINDOW_SIZE;
+    spdylay_submit_window_update(session_, 0, delta);
+  }
+
+  client_->signal_write();
+}
+
+int SpdySession::submit_request() {
+  int rv;
+  auto config = client_->worker->config;
+  auto &nv = config->nv[client_->reqidx++];
+
+  if (client_->reqidx == config->nv.size()) {
+    client_->reqidx = 0;
+  }
+
+  spdylay_data_provider prd{{0}, file_read_callback};
+
+  rv = spdylay_submit_request(session_, 0, nv.data(),
+                              config->data_fd == -1 ? nullptr : &prd, nullptr);
+
+  if (rv != 0) {
+    return -1;
+  }
+
+  return 0;
+}
+
+int SpdySession::on_read(const uint8_t *data, size_t len) {
+  auto rv = spdylay_session_mem_recv(session_, data, len);
+  if (rv < 0) {
+    return -1;
+  }
+
+  assert(static_cast<size_t>(rv) == len);
+
+  if (spdylay_session_want_read(session_) == 0 &&
+      spdylay_session_want_write(session_) == 0 && client_->wb.rleft() == 0) {
+    return -1;
+  }
+
+  client_->signal_write();
+
+  return 0;
+}
+
+int SpdySession::on_write() {
+  auto rv = spdylay_session_send(session_);
+  if (rv != 0) {
+    return -1;
+  }
+
+  if (spdylay_session_want_read(session_) == 0 &&
+      spdylay_session_want_write(session_) == 0 && client_->wb.rleft() == 0) {
+    return -1;
+  }
+  return 0;
+}
+
+void SpdySession::terminate() {
+  spdylay_session_fail_session(session_, SPDYLAY_OK);
+}
+
+namespace {
+int32_t determine_window_update_transmission(spdylay_session *session,
+                                             int32_t stream_id,
+                                             size_t window_bits) {
+  int32_t recv_length;
+
+  if (stream_id == 0) {
+    recv_length = spdylay_session_get_recv_data_length(session);
+  } else {
+    recv_length =
+        spdylay_session_get_stream_recv_data_length(session, stream_id);
+  }
+
+  auto window_size = 1 << window_bits;
+
+  if (recv_length != -1 && recv_length >= window_size / 2) {
+    return recv_length;
+  }
+
+  return -1;
+}
+} // namespace
+
+void SpdySession::handle_window_update(int32_t stream_id, size_t recvlen) {
+  auto config = client_->worker->config;
+  size_t connection_window_bits;
+
+  if (config->connection_window_bits > 16) {
+    connection_window_bits = config->connection_window_bits;
+  } else {
+    connection_window_bits = 16;
+  }
+
+  auto delta =
+      determine_window_update_transmission(session_, 0, connection_window_bits);
+  if (delta > 0) {
+    spdylay_submit_window_update(session_, 0, delta);
+  }
+
+  delta = determine_window_update_transmission(session_, stream_id,
+                                               config->window_bits);
+  if (delta > 0) {
+    spdylay_submit_window_update(session_, stream_id, delta);
+  }
+}
+
+size_t SpdySession::max_concurrent_streams() {
+  return (size_t)client_->worker->config->max_concurrent_streams;
+}
+
+} // namespace h2load

src/h2load_spdy_session.h

@@ -0,0 +1,58 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2014 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef H2LOAD_SPDY_SESSION_H
+#define H2LOAD_SPDY_SESSION_H
+
+#include "h2load_session.h"
+
+#include <spdylay/spdylay.h>
+
+#include "util.h"
+
+namespace h2load {
+
+struct Client;
+
+class SpdySession : public Session {
+public:
+  SpdySession(Client *client, uint16_t spdy_version);
+  virtual ~SpdySession();
+  virtual void on_connect();
+  virtual int submit_request();
+  virtual int on_read(const uint8_t *data, size_t len);
+  virtual int on_write();
+  virtual void terminate();
+  virtual size_t max_concurrent_streams();
+  void handle_window_update(int32_t stream_id, size_t recvlen);
+
+private:
+  Client *client_;
+  spdylay_session *session_;
+  uint16_t spdy_version_;
+};
+
+} // namespace h2load
+
+#endif // H2LOAD_SPDY_SESSION_H

src/http2.cc

@@ -36,8 +36,6 @@ StringRef get_reason_phrase(unsigned int status_code) {
     return StringRef::from_lit("Continue");
   case 101:
     return StringRef::from_lit("Switching Protocols");
-  case 103:
-    return StringRef::from_lit("Early Hints");
   case 200:
     return StringRef::from_lit("OK");
   case 201:
@@ -142,8 +140,6 @@ StringRef stringify_status(BlockAllocator &balloc, unsigned int status_code) {
     return StringRef::from_lit("100");
   case 101:
     return StringRef::from_lit("101");
-  case 103:
-    return StringRef::from_lit("103");
   case 200:
     return StringRef::from_lit("200");
   case 201:
@@ -438,6 +434,11 @@ void copy_headers_to_nva_internal(std::vector<nghttp2_nv> &nva,
       kv = &(*it_via);
       it_via = it;
       break;
+    case HD_NGHTTPX_0RTT_UNIQ:
+      if (flags & HDOP_STRIP_NGHTTPX_ZERO_RTT_UNIQ) {
+        continue;
+      }
+      break;
     }
     nva.push_back(
         make_nv_internal(kv->name, kv->value, kv->no_index, nv_flags));
@@ -924,6 +925,11 @@ int lookup_token(const uint8_t *name, size_t namelen) {
         return HD_X_FORWARDED_PROTO;
       }
       break;
+    case 'q':
+      if (util::streq_l("nghttpx-0rtt-uni", name, 16)) {
+        return HD_NGHTTPX_0RTT_UNIQ;
+      }
+      break;
     }
     break;
   }

src/http2.h

@@ -203,9 +203,13 @@ enum HeaderBuildOp {
   // Via header fields must be stripped.  If this flag is not set, all
   // Via header fields other than last one are added.
   HDOP_STRIP_VIA = 1 << 3,
+  // nghttpx-0rtt-uniq header fields must be stripped.  If this flag
+  // is not set, all nghttpx-0rtt-uniq header fields are added.
+  HDOP_STRIP_NGHTTPX_ZERO_RTT_UNIQ = 1 << 4,
   // Strip above all header fields.
   HDOP_STRIP_ALL = HDOP_STRIP_FORWARDED | HDOP_STRIP_X_FORWARDED_FOR |
-                   HDOP_STRIP_X_FORWARDED_PROTO | HDOP_STRIP_VIA,
+                   HDOP_STRIP_X_FORWARDED_PROTO | HDOP_STRIP_VIA |
+                   HDOP_STRIP_NGHTTPX_ZERO_RTT_UNIQ,
 };
 
 // Appends headers in |headers| to |nv|.  |headers| must be indexed
@@ -312,6 +316,7 @@ enum {
   HD_KEEP_ALIVE,
   HD_LINK,
   HD_LOCATION,
+  HD_NGHTTPX_0RTT_UNIQ,
   HD_PROXY_CONNECTION,
   HD_SERVER,
   HD_TE,

src/shrpx.cc

@@ -1384,8 +1384,11 @@ bool conf_exists(const char *path) {
 } // namespace
 
 namespace {
-constexpr auto DEFAULT_NPN_LIST =
-    StringRef::from_lit("h2,h2-16,h2-14,http/1.1");
+constexpr auto DEFAULT_NPN_LIST = StringRef::from_lit("h2,h2-16,h2-14,"
+#ifdef HAVE_SPDYLAY
+                                                      "spdy/3.1,"
+#endif // HAVE_SPDYLAY
+                                                      "http/1.1");
 } // namespace
 
 namespace {
@@ -1435,6 +1438,12 @@ void fill_default_config(Config *config) {
       memcachedconf.family = AF_UNSPEC;
     }
 
+    auto &anti_replayconf = tlsconf.anti_replay;
+    {
+      auto &memcachedconf = anti_replayconf.memcached;
+      memcachedconf.family = AF_UNSPEC;
+    }
+
     ticketconf.cipher = EVP_aes_128_cbc();
   }
 
@@ -1477,6 +1486,7 @@ void fill_default_config(Config *config) {
   httpconf.max_requests = std::numeric_limits<size_t>::max();
   httpconf.xfp.add = true;
   httpconf.xfp.strip_incoming = true;
+  httpconf.zero_rtt_uniq.strip_incoming = true;
 
   auto &http2conf = config->http2;
   {
@@ -1487,11 +1497,13 @@ void fill_default_config(Config *config) {
       timeoutconf.settings = 10_s;
     }
 
-    // window size for HTTP/2 upstream connection per stream.  2**16-1
-    // = 64KiB-1, which is HTTP/2 default.
+    // window size for HTTP/2 and SPDY upstream connection per stream.
+    // 2**16-1 = 64KiB-1, which is HTTP/2 default.  Please note that
+    // SPDY/3 default is 64KiB.
     upstreamconf.window_size = 64_k - 1;
-    // HTTP/2 has connection-level flow control. The default window
-    // size for HTTP/2 is 64KiB - 1.
+    // HTTP/2 and SPDY/3.1 has connection-level flow control. The
+    // default window size for HTTP/2 is 64KiB - 1.  SPDY/3's default
+    // is 64KiB
     upstreamconf.connection_window_size = 64_k - 1;
     upstreamconf.max_concurrent_streams = 100;
 
@@ -1597,7 +1609,7 @@ void fill_default_config(Config *config) {
   }
 
   auto &apiconf = config->api;
-  apiconf.max_request_body = 32_m;
+  apiconf.max_request_body = 16_k;
 
   auto &dnsconf = config->dns;
   {
@@ -1619,7 +1631,7 @@ void print_version(std::ostream &out) {
 namespace {
 void print_usage(std::ostream &out) {
   out << R"(Usage: nghttpx [OPTIONS]... [<PRIVATE_KEY> <CERT>]
-A reverse proxy for HTTP/2, and HTTP/1.)"
+A reverse proxy for HTTP/2, HTTP/1 and SPDY.)"
       << std::endl;
 }
 } // namespace
@@ -1815,13 +1827,6 @@ Connections:
               "redirect-if-no-tls"    parameter   to    all   backends
               explicitly if this feature is desired.
 
-              If "upgrade-scheme"  parameter is used along  with "tls"
-              parameter, HTTP/2 :scheme pseudo header field is changed
-              to "https" from "http" when forwarding a request to this
-              particular backend.  This is  a workaround for a backend
-              server  which  requires  "https" :scheme  pseudo  header
-              field on TLS encrypted connection.
-
               Since ";" and ":" are  used as delimiter, <PATTERN> must
               not  contain these  characters.  Since  ";" has  special
               meaning in shell, the option value must be quoted.
@@ -1992,7 +1997,8 @@ Performance:
 
 Timeout:
   --frontend-http2-read-timeout=<DURATION>
-              Specify read timeout for HTTP/2 frontend connection.
+              Specify  read  timeout  for  HTTP/2  and  SPDY  frontend
+              connection.
               Default: )"
       << util::duration_str(config->conn.upstream.timeout.http2_read) << R"(
   --frontend-read-timeout=<DURATION>
@@ -2009,13 +2015,13 @@ Timeout:
               Default: )"
       << util::duration_str(config->conn.upstream.timeout.idle_read) << R"(
   --stream-read-timeout=<DURATION>
-              Specify  read timeout  for HTTP/2  streams.  0  means no
-              timeout.
+              Specify  read timeout  for HTTP/2  and SPDY  streams.  0
+              means no timeout.
               Default: )"
       << util::duration_str(config->http2.timeout.stream_read) << R"(
   --stream-write-timeout=<DURATION>
-              Specify write  timeout for  HTTP/2 streams.  0  means no
-              timeout.
+              Specify write  timeout for  HTTP/2 and SPDY  streams.  0
+              means no timeout.
               Default: )"
       << util::duration_str(config->http2.timeout.stream_write) << R"(
   --backend-read-timeout=<DURATION>
@@ -2285,6 +2291,25 @@ SSL/TLS:
   --tls-session-cache-memcached-private-key-file=<PATH>
               Path to client private  key for memcached connections to
               store session cache.
+  --tls-anti-replay-memcached=<HOST>,<PORT>[;tls]
+              Specify address of memcached server to store ClientHello
+              to avoid  0-RTT early data replay.   This enables shared
+              storage between multiple nghttpx instances.  Optionally,
+              memcached  connection  can  be  encrypted  with  TLS  by
+              specifying "tls" parameter.
+  --tls-anti-replay-memcached-address-family=(auto|IPv4|IPv6)
+              Specify address family of memcached connections to store
+              ClientHello to avoid 0-RTT early data replay.  If "auto"
+              is given, both IPv4 and  IPv6 are considered.  If "IPv4"
+              is given, only IPv4 address is considered.  If "IPv6" is
+              given, only IPv6 address is considered.
+              Default: auto
+  --tls-anti-replay-memcached-cert-file=<PATH>
+              Path to client certificate  for memcached connections to
+              store ClientHello to avoid 0-RTT early data replay.
+  --tls-anti-replay-memcached-private-key-file=<PATH>
+              Path to client private  key for memcached connections to
+              store ClientHello to avoid 0-RTT early data replay.
   --tls-dyn-rec-warmup-threshold=<SIZE>
               Specify the  threshold size for TLS  dynamic record size
               behaviour.  During  a TLS  session, after  the threshold
@@ -2352,10 +2377,10 @@ SSL/TLS:
               consider   to  use   --client-no-http2-cipher-black-list
               option.  But be aware its implications.
 
-HTTP/2:
+HTTP/2 and SPDY:
   -c, --frontend-http2-max-concurrent-streams=<N>
               Set the maximum number of  the concurrent streams in one
-              frontend HTTP/2 session.
+              frontend HTTP/2 and SPDY session.
               Default:  )"
       << config->http2.upstream.max_concurrent_streams << R"(
   --backend-http2-max-concurrent-streams=<N>
@@ -2366,13 +2391,14 @@ HTTP/2:
               Default: )"
       << config->http2.downstream.max_concurrent_streams << R"(
   --frontend-http2-window-size=<SIZE>
-              Sets  the  per-stream  initial  window  size  of  HTTP/2
-              frontend connection.
+              Sets the  per-stream initial  window size of  HTTP/2 and
+              SPDY frontend connection.
               Default: )"
       << config->http2.upstream.window_size << R"(
   --frontend-http2-connection-window-size=<SIZE>
-              Sets the  per-connection window size of  HTTP/2 frontend
-              connection.
+              Sets the  per-connection window size of  HTTP/2 and SPDY
+              frontend  connection.  For  SPDY  connection, the  value
+              less than 64KiB is rounded up to 64KiB.
               Default: )"
       << config->http2.upstream.connection_window_size << R"(
   --backend-http2-window-size=<SIZE>
@@ -2398,7 +2424,8 @@ HTTP/2:
               It is  also supported if  both frontend and  backend are
               HTTP/2 in default mode.  In  this case, server push from
               backend session is relayed  to frontend, and server push
-              via Link header field is also supported.
+              via Link header field  is also supported.  SPDY frontend
+              does not support server push.
   --frontend-http2-optimize-write-buffer-size
               (Experimental) Enable write  buffer size optimization in
               frontend HTTP/2 TLS  connection.  This optimization aims
@@ -2453,7 +2480,7 @@ HTTP/2:
 
 Mode:
   (default mode)
-              Accept  HTTP/2,  and  HTTP/1.1 over  SSL/TLS.   "no-tls"
+              Accept HTTP/2, SPDY and HTTP/1.1 over SSL/TLS.  "no-tls"
               parameter is  used in  --frontend option,  accept HTTP/2
               and HTTP/1.1 over cleartext  TCP.  The incoming HTTP/1.1
               connection  can  be  upgraded  to  HTTP/2  through  HTTP
@@ -2589,6 +2616,9 @@ HTTP:
               Default: obfuscated
   --no-via    Don't append to  Via header field.  If  Via header field
               is received, it is left unaltered.
+  --no-strip-incoming-nghttpx-0rtt-uniq
+              Don't strip nghttpx-0rtt-uniq  header field from inbound
+              client requests.
   --no-location-rewrite
               Don't  rewrite location  header field  in default  mode.
               When --http2-proxy  is used, location header  field will
@@ -2767,12 +2797,10 @@ namespace {
 int process_options(Config *config,
                     std::vector<std::pair<StringRef, StringRef>> &cmdcfgs) {
   std::array<char, STRERROR_BUFSIZE> errbuf;
-  std::map<StringRef, size_t> pattern_addr_indexer;
   if (conf_exists(config->conf_path.c_str())) {
     LOG(NOTICE) << "Loading configuration from " << config->conf_path;
     std::set<StringRef> include_set;
-    if (load_config(config, config->conf_path.c_str(), include_set,
-                    pattern_addr_indexer) == -1) {
+    if (load_config(config, config->conf_path.c_str(), include_set) == -1) {
       LOG(FATAL) << "Failed to load configuration from " << config->conf_path;
       return -1;
     }
@@ -2786,8 +2814,7 @@ int process_options(Config *config,
     std::set<StringRef> include_set;
 
     for (auto &p : cmdcfgs) {
-      if (parse_config(config, p.first, p.second, include_set,
-                       pattern_addr_indexer) == -1) {
+      if (parse_config(config, p.first, p.second, include_set) == -1) {
         LOG(FATAL) << "Failed to parse command-line argument.";
         return -1;
       }
@@ -2997,6 +3024,26 @@ int process_options(Config *config,
     }
   }
 
+  {
+    auto &memcachedconf = tlsconf.anti_replay.memcached;
+    if (!memcachedconf.host.empty()) {
+      auto hostport = util::make_hostport(StringRef{memcachedconf.host},
+                                          memcachedconf.port);
+      if (resolve_hostname(&memcachedconf.addr, memcachedconf.host.c_str(),
+                           memcachedconf.port, memcachedconf.family) == -1) {
+        LOG(FATAL) << "Resolving memcached address for TLS anti-replay failed: "
+                   << hostport;
+        return -1;
+      }
+      LOG(NOTICE) << "Memcached address for TLS anti-replay: " << hostport
+                  << " -> " << util::to_numeric_addr(&memcachedconf.addr);
+      if (memcachedconf.tls) {
+        LOG(NOTICE) << "Connection to memcached for TLS anti-replay will be "
+                       "encrypted by TLS";
+      }
+    }
+  }
+
   if (config->rlimit_nofile) {
     struct rlimit lim = {static_cast<rlim_t>(config->rlimit_nofile),
                          static_cast<rlim_t>(config->rlimit_nofile)};
@@ -3406,6 +3453,16 @@ int main(int argc, char **argv) {
         {SHRPX_OPT_NO_STRIP_INCOMING_X_FORWARDED_PROTO.c_str(), no_argument,
          &flag, 158},
         {SHRPX_OPT_SINGLE_PROCESS.c_str(), no_argument, &flag, 159},
+        {SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED.c_str(), required_argument, &flag,
+         160},
+        {SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_ADDRESS_FAMILY.c_str(),
+         required_argument, &flag, 161},
+        {SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_CERT_FILE.c_str(),
+         required_argument, &flag, 162},
+        {SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_PRIVATE_KEY_FILE.c_str(),
+         required_argument, &flag, 163},
+        {SHRPX_OPT_NO_STRIP_INCOMING_NGHTTPX_0RTT_UNIQ.c_str(), no_argument,
+         &flag, 164},
         {nullptr, 0, nullptr, 0}};
 
     int option_index = 0;
@@ -4167,6 +4224,32 @@ int main(int argc, char **argv) {
         cmdcfgs.emplace_back(SHRPX_OPT_SINGLE_PROCESS,
                              StringRef::from_lit("yes"));
         break;
+      case 160:
+        // --tls-anti-replay-memcached
+        cmdcfgs.emplace_back(SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED,
+                             StringRef{optarg});
+        break;
+      case 161:
+        // --tls-anti-replay-memcached-address-family
+        cmdcfgs.emplace_back(SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_ADDRESS_FAMILY,
+                             StringRef{optarg});
+        break;
+      case 162:
+        // --tls-anti-replay-memcached-cert-file
+        cmdcfgs.emplace_back(SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_CERT_FILE,
+                             StringRef{optarg});
+        break;
+      case 163:
+        // --tls-anti-replay-memcached-private-key-file
+        cmdcfgs.emplace_back(
+            SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_PRIVATE_KEY_FILE,
+            StringRef{optarg});
+        break;
+      case 164:
+        // --no-strip-incoming-nghttpx-0rtt-uniq
+        cmdcfgs.emplace_back(SHRPX_OPT_NO_STRIP_INCOMING_NGHTTPX_0RTT_UNIQ,
+                             StringRef::from_lit("yes"));
+        break;
       default:
         break;
       }

src/shrpx_api_downstream_connection.cc

@@ -24,11 +24,6 @@
  */
 #include "shrpx_api_downstream_connection.h"
 
-#include <sys/mman.h>
-#include <fcntl.h>
-#include <unistd.h>
-#include <cstdlib>
-
 #include "shrpx_client_handler.h"
 #include "shrpx_upstream.h"
 #include "shrpx_downstream.h"
@@ -70,13 +65,9 @@ constexpr StringRef API_METHOD_STRING[] = {
 } // namespace
 
 APIDownstreamConnection::APIDownstreamConnection(Worker *worker)
-    : worker_(worker), api_(nullptr), fd_(-1), shutdown_read_(false) {}
+    : worker_(worker), api_(nullptr), shutdown_read_(false) {}
 
-APIDownstreamConnection::~APIDownstreamConnection() {
-  if (fd_ != -1) {
-    close(fd_);
-  }
-}
+APIDownstreamConnection::~APIDownstreamConnection() {}
 
 int APIDownstreamConnection::attach_downstream(Downstream *downstream) {
   if (LOG_ENABLED(INFO)) {
@@ -243,28 +234,6 @@ int APIDownstreamConnection::push_request_headers() {
     return 0;
   }
 
-  switch (req.method) {
-  case HTTP_POST:
-  case HTTP_PUT: {
-    char tempname[] = "/tmp/nghttpx-api.XXXXXX";
-#ifdef HAVE_MKOSTEMP
-    fd_ = mkostemp(tempname, O_CLOEXEC);
-#else  // !HAVE_MKOSTEMP
-    fd_ = mkstemp(tempname);
-#endif // !HAVE_MKOSTEMP
-    if (fd_ == -1) {
-      send_reply(500, API_FAILURE);
-
-      return 0;
-    }
-#ifndef HAVE_MKOSTEMP
-    util::make_socket_closeonexec(fd_);
-#endif // HAVE_MKOSTEMP
-    unlink(tempname);
-    break;
-  }
-  }
-
   return 0;
 }
 
@@ -307,25 +276,17 @@ int APIDownstreamConnection::push_upload_data_chunk(const uint8_t *data,
     return 0;
   }
 
-  auto &req = downstream_->request();
+  auto output = downstream_->get_request_buf();
+
   auto &apiconf = get_config()->api;
 
-  if (static_cast<size_t>(req.recv_body_length) > apiconf.max_request_body) {
+  if (output->rleft() + datalen > apiconf.max_request_body) {
     send_reply(413, API_FAILURE);
 
     return 0;
   }
 
-  ssize_t nwrite;
-  while ((nwrite = write(fd_, data, datalen)) == -1 && errno == EINTR)
-    ;
-  if (nwrite == -1) {
-    auto error = errno;
-    LOG(ERROR) << "Could not write API request body: errno=" << error;
-    send_reply(500, API_FAILURE);
-
-    return 0;
-  }
+  output->append(data, datalen);
 
   // We don't have to call Upstream::resume_read() here, because
   // request buffer is effectively unlimited.  Actually, we cannot
@@ -343,20 +304,29 @@ int APIDownstreamConnection::end_upload_data() {
 }
 
 int APIDownstreamConnection::handle_backendconfig() {
-  auto &req = downstream_->request();
+  auto output = downstream_->get_request_buf();
 
-  if (req.recv_body_length == 0) {
+  std::array<struct iovec, 2> iov;
+  auto iovcnt = output->riovec(iov.data(), 2);
+
+  if (iovcnt == 0) {
     send_reply(200, API_SUCCESS);
 
     return 0;
   }
 
-  auto rp = mmap(nullptr, req.recv_body_length, PROT_READ, MAP_SHARED, fd_, 0);
-  if (rp == reinterpret_cast<void *>(-1)) {
-    send_reply(500, API_FAILURE);
-  }
+  std::unique_ptr<uint8_t[]> large_buf;
 
-  auto unmapper = defer(munmap, rp, req.recv_body_length);
+  // If data spans in multiple chunks, pull them up into one
+  // contiguous buffer.
+  if (iovcnt > 1) {
+    large_buf = make_unique<uint8_t[]>(output->rleft());
+    auto len = output->rleft();
+    output->remove(large_buf.get(), len);
+
+    iov[0].iov_base = large_buf.get();
+    iov[0].iov_len = len;
+  }
 
   Config new_config{};
   new_config.conn.downstream = std::make_shared<DownstreamConfig>();
@@ -373,10 +343,9 @@ int APIDownstreamConnection::handle_backendconfig() {
   downstreamconf->family = src->family;
 
   std::set<StringRef> include_set;
-  std::map<StringRef, size_t> pattern_addr_indexer;
 
-  for (auto first = reinterpret_cast<const uint8_t *>(rp),
-            last = first + req.recv_body_length;
+  for (auto first = reinterpret_cast<const uint8_t *>(iov[0].iov_base),
+            last = first + iov[0].iov_len;
        first != last;) {
     auto eol = std::find(first, last, '\n');
     if (eol == last) {
@@ -407,8 +376,7 @@ int APIDownstreamConnection::handle_backendconfig() {
       continue;
     }
 
-    if (parse_config(&new_config, optid, opt, optval, include_set,
-                     pattern_addr_indexer) != 0) {
+    if (parse_config(&new_config, optid, opt, optval, include_set) != 0) {
       send_reply(400, API_FAILURE);
       return 0;
     }

src/shrpx_api_downstream_connection.h

@@ -96,8 +96,6 @@ private:
   Worker *worker_;
   // This points to the requested APIEndpoint struct.
   const APIEndpoint *api_;
-  // The file descriptor for temporary file to store request body.
-  int fd_;
   // true if we stop reading request body.
   bool shutdown_read_;
 };

src/shrpx_client_handler.cc

@@ -51,6 +51,9 @@
 #include "shrpx_api_downstream_connection.h"
 #include "shrpx_health_monitor_downstream_connection.h"
 #include "shrpx_log.h"
+#ifdef HAVE_SPDYLAY
+#include "shrpx_spdy_upstream.h"
+#endif // HAVE_SPDYLAY
 #include "util.h"
 #include "template.h"
 #include "tls.h"
@@ -556,20 +559,28 @@ int ClientHandler::validate_next_proto() {
   }
 #endif // OPENSSL_VERSION_NUMBER >= 0x10002000L
 
-  StringRef proto;
-
-  if (next_proto) {
-    proto = StringRef{next_proto, next_proto_len};
-
-    if (LOG_ENABLED(INFO)) {
-      CLOG(INFO, this) << "The negotiated next protocol: " << proto;
-    }
-  } else {
+  if (next_proto == nullptr) {
     if (LOG_ENABLED(INFO)) {
       CLOG(INFO, this) << "No protocol negotiated. Fallback to HTTP/1.1";
     }
 
-    proto = StringRef::from_lit("http/1.1");
+    upstream_ = make_unique<HttpsUpstream>(this);
+    alpn_ = StringRef::from_lit("http/1.1");
+
+    // At this point, input buffer is already filled with some bytes.
+    // The read callback is not called until new data come. So consume
+    // input buffer here.
+    if (on_read() != 0) {
+      return -1;
+    }
+
+    return 0;
+  }
+
+  auto proto = StringRef{next_proto, next_proto_len};
+
+  if (LOG_ENABLED(INFO)) {
+    CLOG(INFO, this) << "The negotiated next protocol: " << proto;
   }
 
   if (!tls::in_proto_list(get_config()->tls.npn_list, proto)) {
@@ -597,6 +608,36 @@ int ClientHandler::validate_next_proto() {
     return 0;
   }
 
+#ifdef HAVE_SPDYLAY
+  auto spdy_version = spdylay_npn_get_version(proto.byte(), proto.size());
+  if (spdy_version) {
+    upstream_ = make_unique<SpdyUpstream>(spdy_version, this);
+
+    switch (spdy_version) {
+    case SPDYLAY_PROTO_SPDY2:
+      alpn_ = StringRef::from_lit("spdy/2");
+      break;
+    case SPDYLAY_PROTO_SPDY3:
+      alpn_ = StringRef::from_lit("spdy/3");
+      break;
+    case SPDYLAY_PROTO_SPDY3_1:
+      alpn_ = StringRef::from_lit("spdy/3.1");
+      break;
+    default:
+      alpn_ = StringRef::from_lit("spdy/unknown");
+    }
+
+    // At this point, input buffer is already filled with some bytes.
+    // The read callback is not called until new data come. So consume
+    // input buffer here.
+    if (on_read() != 0) {
+      return -1;
+    }
+
+    return 0;
+  }
+#endif // HAVE_SPDYLAY
+
   if (proto == StringRef::from_lit("http/1.1")) {
     upstream_ = make_unique<HttpsUpstream>(this);
     alpn_ = StringRef::from_lit("http/1.1");
@@ -763,61 +804,40 @@ bool load_lighter(const DownstreamAddr *lhs, const DownstreamAddr *rhs) {
 Http2Session *ClientHandler::select_http2_session(
     const std::shared_ptr<DownstreamAddrGroup> &group) {
   auto &shared_addr = group->shared_addr;
+  auto &http2_avail_freelist = shared_addr->http2_avail_freelist;
+
+  for (auto session = http2_avail_freelist.head; session;) {
+    auto next = session->dlnext;
+
+    session->remove_from_freelist();
+
+    // session may be in graceful shutdown period now.
+    if (session->max_concurrency_reached(0)) {
+      if (LOG_ENABLED(INFO)) {
+        CLOG(INFO, this)
+            << "Maximum streams have been reached for Http2Session(" << session
+            << ").  Skip it";
+      }
+
+      session = next;
 
-  // First count the working backend addresses.
-  size_t min = 0;
-  for (const auto &addr : shared_addr->addrs) {
-    if (addr.proto != PROTO_HTTP2 || addr.connect_blocker->blocked()) {
       continue;
     }
 
-    ++min;
-  }
-
-  if (min == 0) {
     if (LOG_ENABLED(INFO)) {
-      CLOG(INFO, this) << "No working backend address found";
+      CLOG(INFO, this) << "Use Http2Session " << session
+                       << " from http2_avail_freelist";
     }
 
-    return nullptr;
-  }
-
-  auto &http2_avail_freelist = shared_addr->http2_avail_freelist;
-
-  if (http2_avail_freelist.size() >= min) {
-    for (auto session = http2_avail_freelist.head; session;) {
-      auto next = session->dlnext;
-
-      session->remove_from_freelist();
-
-      // session may be in graceful shutdown period now.
-      if (session->max_concurrency_reached(0)) {
-        if (LOG_ENABLED(INFO)) {
-          CLOG(INFO, this)
-              << "Maximum streams have been reached for Http2Session("
-              << session << ").  Skip it";
-        }
-
-        session = next;
-
-        continue;
-      }
-
+    if (session->max_concurrency_reached(1)) {
       if (LOG_ENABLED(INFO)) {
-        CLOG(INFO, this) << "Use Http2Session " << session
-                         << " from http2_avail_freelist";
+        CLOG(INFO, this) << "Maximum streams are reached for Http2Session("
+                         << session << ").";
       }
-
-      if (session->max_concurrency_reached(1)) {
-        if (LOG_ENABLED(INFO)) {
-          CLOG(INFO, this) << "Maximum streams are reached for Http2Session("
-                           << session << ").";
-        }
-      } else {
-        session->add_to_avail_freelist();
-      }
-      return session;
+    } else {
+      session->add_to_avail_freelist();
     }
+    return session;
   }
 
   DownstreamAddr *selected_addr = nullptr;
@@ -850,12 +870,22 @@ Http2Session *ClientHandler::select_http2_session(
       break;
     }
 
+    if (addr.http2_extra_freelist.size() == 0 &&
+        addr.connect_blocker->blocked()) {
+      continue;
+    }
+
     if (selected_addr == nullptr || load_lighter(&addr, selected_addr)) {
       selected_addr = &addr;
     }
   }
 
-  assert(selected_addr);
+  if (selected_addr == nullptr) {
+    if (LOG_ENABLED(INFO)) {
+      CLOG(INFO, this) << "No working backend address found";
+    }
+    return nullptr;
+  }
 
   if (LOG_ENABLED(INFO)) {
     CLOG(INFO, this) << "Selected DownstreamAddr=" << selected_addr

src/shrpx_config.cc

@@ -814,7 +814,6 @@ struct DownstreamParams {
   bool tls;
   bool dns;
   bool redirect_if_not_tls;
-  bool upgrade_scheme;
 };
 
 namespace {
@@ -919,8 +918,6 @@ int parse_downstream_params(DownstreamParams &out,
       out.dns = true;
     } else if (util::strieq_l("redirect-if-not-tls", param)) {
       out.redirect_if_not_tls = true;
-    } else if (util::strieq_l("upgrade-scheme", param)) {
-      out.upgrade_scheme = true;
     } else if (!param.empty()) {
       LOG(ERROR) << "backend: " << param << ": unknown keyword";
       return -1;
@@ -946,7 +943,6 @@ namespace {
 //
 // This function returns 0 if it succeeds, or -1.
 int parse_mapping(Config *config, DownstreamAddrConfig &addr,
-                  std::map<StringRef, size_t> &pattern_addr_indexer,
                   const StringRef &src_pattern, const StringRef &src_params) {
   // This returns at least 1 element (it could be empty string).  We
   // will append '/' to all patterns, so it becomes catch-all pattern.
@@ -980,7 +976,6 @@ int parse_mapping(Config *config, DownstreamAddrConfig &addr,
   addr.tls = params.tls;
   addr.sni = make_string_ref(downstreamconf.balloc, params.sni);
   addr.dns = params.dns;
-  addr.upgrade_scheme = params.upgrade_scheme;
 
   auto &routerconf = downstreamconf.router;
   auto &router = routerconf.router;
@@ -988,6 +983,7 @@ int parse_mapping(Config *config, DownstreamAddrConfig &addr,
   auto &wildcard_patterns = routerconf.wildcard_patterns;
 
   for (const auto &raw_pattern : mapping) {
+    auto done = false;
     StringRef pattern;
     auto slash = std::find(std::begin(raw_pattern), std::end(raw_pattern), '/');
     if (slash == std::end(raw_pattern)) {
@@ -1014,43 +1010,47 @@ int parse_mapping(Config *config, DownstreamAddrConfig &addr,
       *p = '\0';
       pattern = StringRef{iov.base, p};
     }
-    auto it = pattern_addr_indexer.find(pattern);
-    if (it != std::end(pattern_addr_indexer)) {
-      auto &g = addr_groups[(*it).second];
-      // Last value wins if we have multiple different affinity
-      // value under one group.
-      if (params.affinity.type != AFFINITY_NONE) {
-        if (g.affinity.type == AFFINITY_NONE) {
-          g.affinity.type = params.affinity.type;
-          if (params.affinity.type == AFFINITY_COOKIE) {
-            g.affinity.cookie.name = make_string_ref(
-                downstreamconf.balloc, params.affinity.cookie.name);
-            if (!params.affinity.cookie.path.empty()) {
-              g.affinity.cookie.path = make_string_ref(
-                  downstreamconf.balloc, params.affinity.cookie.path);
+    for (auto &g : addr_groups) {
+      if (g.pattern == pattern) {
+        // Last value wins if we have multiple different affinity
+        // value under one group.
+        if (params.affinity.type != AFFINITY_NONE) {
+          if (g.affinity.type == AFFINITY_NONE) {
+            g.affinity.type = params.affinity.type;
+            if (params.affinity.type == AFFINITY_COOKIE) {
+              g.affinity.cookie.name = make_string_ref(
+                  downstreamconf.balloc, params.affinity.cookie.name);
+              if (!params.affinity.cookie.path.empty()) {
+                g.affinity.cookie.path = make_string_ref(
+                    downstreamconf.balloc, params.affinity.cookie.path);
+              }
+              g.affinity.cookie.secure = params.affinity.cookie.secure;
             }
-            g.affinity.cookie.secure = params.affinity.cookie.secure;
+          } else if (g.affinity.type != params.affinity.type ||
+                     g.affinity.cookie.name != params.affinity.cookie.name ||
+                     g.affinity.cookie.path != params.affinity.cookie.path ||
+                     g.affinity.cookie.secure !=
+                         params.affinity.cookie.secure) {
+            LOG(ERROR) << "backend: affinity: multiple different affinity "
+                          "configurations found in a single group";
+            return -1;
           }
-        } else if (g.affinity.type != params.affinity.type ||
-                   g.affinity.cookie.name != params.affinity.cookie.name ||
-                   g.affinity.cookie.path != params.affinity.cookie.path ||
-                   g.affinity.cookie.secure != params.affinity.cookie.secure) {
-          LOG(ERROR) << "backend: affinity: multiple different affinity "
-                        "configurations found in a single group";
-          return -1;
         }
+        // If at least one backend requires frontend TLS connection,
+        // enable it for all backends sharing the same pattern.
+        if (params.redirect_if_not_tls) {
+          g.redirect_if_not_tls = true;
+        }
+        g.addrs.push_back(addr);
+        done = true;
+        break;
       }
-      // If at least one backend requires frontend TLS connection,
-      // enable it for all backends sharing the same pattern.
-      if (params.redirect_if_not_tls) {
-        g.redirect_if_not_tls = true;
-      }
-      g.addrs.push_back(addr);
+    }
+    if (done) {
       continue;
     }
 
     auto idx = addr_groups.size();
-    pattern_addr_indexer.emplace(pattern, idx);
     addr_groups.emplace_back(pattern);
     auto &g = addr_groups.back();
     g.addrs.push_back(addr);
@@ -2072,6 +2072,11 @@ int option_lookup_token(const char *name, size_t namelen) {
     break;
   case 25:
     switch (name[24]) {
+    case 'd':
+      if (util::strieq_l("tls-anti-replay-memcache", name, 24)) {
+        return SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED;
+      }
+      break;
     case 'e':
       if (util::strieq_l("backend-http2-window-siz", name, 24)) {
         return SHRPX_OPTID_BACKEND_HTTP2_WINDOW_SIZE;
@@ -2255,12 +2260,20 @@ int option_lookup_token(const char *name, size_t namelen) {
       if (util::strieq_l("frontend-http2-optimize-window-siz", name, 34)) {
         return SHRPX_OPTID_FRONTEND_HTTP2_OPTIMIZE_WINDOW_SIZE;
       }
+      if (util::strieq_l("tls-anti-replay-memcached-cert-fil", name, 34)) {
+        return SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_CERT_FILE;
+      }
       break;
     case 'o':
       if (util::strieq_l("no-strip-incoming-x-forwarded-prot", name, 34)) {
         return SHRPX_OPTID_NO_STRIP_INCOMING_X_FORWARDED_PROTO;
       }
       break;
+    case 'q':
+      if (util::strieq_l("no-strip-incoming-nghttpx-0rtt-uni", name, 34)) {
+        return SHRPX_OPTID_NO_STRIP_INCOMING_NGHTTPX_0RTT_UNIQ;
+      }
+      break;
     case 'r':
       if (util::strieq_l("frontend-http2-dump-response-heade", name, 34)) {
         return SHRPX_OPTID_FRONTEND_HTTP2_DUMP_RESPONSE_HEADER;
@@ -2338,6 +2351,11 @@ int option_lookup_token(const char *name, size_t namelen) {
         return SHRPX_OPTID_BACKEND_HTTP2_ENCODER_DYNAMIC_TABLE_SIZE;
       }
       break;
+    case 'y':
+      if (util::strieq_l("tls-anti-replay-memcached-address-famil", name, 39)) {
+        return SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_ADDRESS_FAMILY;
+      }
+      break;
     }
     break;
   case 41:
@@ -2364,6 +2382,12 @@ int option_lookup_token(const char *name, size_t namelen) {
     break;
   case 42:
     switch (name[41]) {
+    case 'e':
+      if (util::strieq_l("tls-anti-replay-memcached-private-key-fil", name,
+                         41)) {
+        return SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_PRIVATE_KEY_FILE;
+      }
+      break;
     case 'y':
       if (util::strieq_l("tls-session-cache-memcached-address-famil", name,
                          41)) {
@@ -2387,16 +2411,13 @@ int option_lookup_token(const char *name, size_t namelen) {
 }
 
 int parse_config(Config *config, const StringRef &opt, const StringRef &optarg,
-                 std::set<StringRef> &included_set,
-                 std::map<StringRef, size_t> &pattern_addr_indexer) {
+                 std::set<StringRef> &included_set) {
   auto optid = option_lookup_token(opt.c_str(), opt.size());
-  return parse_config(config, optid, opt, optarg, included_set,
-                      pattern_addr_indexer);
+  return parse_config(config, optid, opt, optarg, included_set);
 }
 
 int parse_config(Config *config, int optid, const StringRef &opt,
-                 const StringRef &optarg, std::set<StringRef> &included_set,
-                 std::map<StringRef, size_t> &pattern_addr_indexer) {
+                 const StringRef &optarg, std::set<StringRef> &included_set) {
   std::array<char, STRERROR_BUFSIZE> errbuf;
   char host[NI_MAXHOST];
   uint16_t port;
@@ -2428,8 +2449,7 @@ int parse_config(Config *config, int optid, const StringRef &opt,
     auto params =
         mapping_end == std::end(optarg) ? mapping_end : mapping_end + 1;
 
-    if (parse_mapping(config, addr, pattern_addr_indexer,
-                      StringRef{mapping, mapping_end},
+    if (parse_mapping(config, addr, StringRef{mapping, mapping_end},
                       StringRef{params, std::end(optarg)}) != 0) {
       return -1;
     }
@@ -2634,7 +2654,8 @@ int parse_config(Config *config, int optid, const StringRef &opt,
     }
 
     // Make 16 bits to the HTTP/2 default 64KiB - 1.  This is the same
-    // behaviour of previous code.
+    // behaviour of previous code.  For SPDY, we adjust this value in
+    // SpdyUpstream to look like the SPDY default.
     *resp = (1 << n) - 1;
 
     return 0;
@@ -3129,8 +3150,7 @@ int parse_config(Config *config, int optid, const StringRef &opt,
     }
 
     included_set.insert(optarg);
-    auto rv =
-        load_config(config, optarg.c_str(), included_set, pattern_addr_indexer);
+    auto rv = load_config(config, optarg.c_str(), included_set);
     included_set.erase(optarg);
 
     if (rv != 0) {
@@ -3157,7 +3177,8 @@ int parse_config(Config *config, int optid, const StringRef &opt,
 
     return 0;
   case SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED:
-  case SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED: {
+  case SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED:
+  case SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED: {
     auto addr_end = std::find(std::begin(optarg), std::end(optarg), ';');
     auto src_params = StringRef{addr_end, std::end(optarg)};
 
@@ -3187,6 +3208,13 @@ int parse_config(Config *config, int optid, const StringRef &opt,
       memcachedconf.tls = params.tls;
       break;
     }
+    case SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED: {
+      auto &memcachedconf = config->tls.anti_replay.memcached;
+      memcachedconf.host = make_string_ref(config->balloc, StringRef{host});
+      memcachedconf.port = port;
+      memcachedconf.tls = params.tls;
+      break;
+    }
     };
 
     return 0;
@@ -3334,6 +3362,16 @@ int parse_config(Config *config, int optid, const StringRef &opt,
     config->tls.ticket.memcached.private_key_file =
         make_string_ref(config->balloc, optarg);
 
+    return 0;
+  case SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_CERT_FILE:
+    config->tls.anti_replay.memcached.cert_file =
+        make_string_ref(config->balloc, optarg);
+
+    return 0;
+  case SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_PRIVATE_KEY_FILE:
+    config->tls.anti_replay.memcached.private_key_file =
+        make_string_ref(config->balloc, optarg);
+
     return 0;
   case SHRPX_OPTID_TLS_TICKET_KEY_MEMCACHED_ADDRESS_FAMILY:
     return parse_address_family(&config->tls.ticket.memcached.family, opt,
@@ -3341,6 +3379,9 @@ int parse_config(Config *config, int optid, const StringRef &opt,
   case SHRPX_OPTID_TLS_SESSION_CACHE_MEMCACHED_ADDRESS_FAMILY:
     return parse_address_family(&config->tls.session_cache.memcached.family,
                                 opt, optarg);
+  case SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_ADDRESS_FAMILY:
+    return parse_address_family(&config->tls.anti_replay.memcached.family, opt,
+                                optarg);
   case SHRPX_OPTID_BACKEND_ADDRESS_FAMILY:
     return parse_address_family(&config->conn.downstream->family, opt, optarg);
   case SHRPX_OPTID_FRONTEND_HTTP2_MAX_CONCURRENT_STREAMS:
@@ -3554,6 +3595,10 @@ int parse_config(Config *config, int optid, const StringRef &opt,
   case SHRPX_OPTID_NO_VERIFY_OCSP:
     config->tls.ocsp.no_verify = util::strieq_l("yes", optarg);
 
+    return 0;
+  case SHRPX_OPTID_NO_STRIP_INCOMING_NGHTTPX_0RTT_UNIQ:
+    config->http.zero_rtt_uniq.strip_incoming = !util::strieq_l("yes", optarg);
+
     return 0;
   case SHRPX_OPTID_CONF:
     LOG(WARN) << "conf: ignored";
@@ -3567,8 +3612,7 @@ int parse_config(Config *config, int optid, const StringRef &opt,
 }
 
 int load_config(Config *config, const char *filename,
-                std::set<StringRef> &include_set,
-                std::map<StringRef, size_t> &pattern_addr_indexer) {
+                std::set<StringRef> &include_set) {
   std::ifstream in(filename, std::ios::binary);
   if (!in) {
     LOG(ERROR) << "Could not open config file " << filename;
@@ -3590,8 +3634,7 @@ int load_config(Config *config, const char *filename,
     *eq = '\0';
 
     if (parse_config(config, StringRef{std::begin(line), eq},
-                     StringRef{eq + 1, std::end(line)}, include_set,
-                     pattern_addr_indexer) != 0) {
+                     StringRef{eq + 1, std::end(line)}, include_set) != 0) {
       return -1;
     }
   }

src/shrpx_config.h

@@ -343,6 +343,16 @@ constexpr auto SHRPX_OPT_NO_STRIP_INCOMING_X_FORWARDED_PROTO =
     StringRef::from_lit("no-strip-incoming-x-forwarded-proto");
 constexpr auto SHRPX_OPT_OCSP_STARTUP = StringRef::from_lit("ocsp-startup");
 constexpr auto SHRPX_OPT_NO_VERIFY_OCSP = StringRef::from_lit("no-verify-ocsp");
+constexpr auto SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED =
+    StringRef::from_lit("tls-anti-replay-memcached");
+constexpr auto SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_CERT_FILE =
+    StringRef::from_lit("tls-anti-replay-memcached-cert-file");
+constexpr auto SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_PRIVATE_KEY_FILE =
+    StringRef::from_lit("tls-anti-replay-memcached-private-key-file");
+constexpr auto SHRPX_OPT_TLS_ANTI_REPLAY_MEMCACHED_ADDRESS_FAMILY =
+    StringRef::from_lit("tls-anti-replay-memcached-address-family");
+constexpr auto SHRPX_OPT_NO_STRIP_INCOMING_NGHTTPX_0RTT_UNIQ =
+    StringRef::from_lit("no-strip-incoming-nghttpx-0rtt-uniq");
 
 constexpr size_t SHRPX_OBFUSCATED_NODE_LENGTH = 8;
 
@@ -461,10 +471,6 @@ struct DownstreamAddrConfig {
   bool tls;
   // true if dynamic DNS is enabled
   bool dns;
-  // true if :scheme pseudo header field should be upgraded to secure
-  // variant (e.g., "https") when forwarding request to a backend
-  // connected by TLS connection.
-  bool upgrade_scheme;
 };
 
 // Mapping hash to idx which is an index into
@@ -581,6 +587,23 @@ struct TLSConfig {
     } memcached;
   } session_cache;
 
+  struct {
+    struct {
+      Address addr;
+      uint16_t port;
+      // Hostname of memcached server.  This is also used as SNI field
+      // if TLS is enabled.
+      StringRef host;
+      // Client private key and certificate for authentication
+      StringRef private_key_file;
+      StringRef cert_file;
+      // Address family of memcached connection.  One of either
+      // AF_INET, AF_INET6 or AF_UNSPEC.
+      int family;
+      bool tls;
+    } memcached;
+  } anti_replay;
+
   // Dynamic record sizing configurations
   struct {
     size_t warmup_threshold;
@@ -683,6 +706,9 @@ struct HttpConfig {
     bool add;
     bool strip_incoming;
   } xfp;
+  struct {
+    bool strip_incoming;
+  } zero_rtt_uniq;
   std::vector<AltSvc> altsvcs;
   std::vector<ErrorPage> error_pages;
   HeaderRefs add_request_headers;
@@ -1075,6 +1101,7 @@ enum {
   SHRPX_OPTID_NO_OCSP,
   SHRPX_OPTID_NO_SERVER_PUSH,
   SHRPX_OPTID_NO_SERVER_REWRITE,
+  SHRPX_OPTID_NO_STRIP_INCOMING_NGHTTPX_0RTT_UNIQ,
   SHRPX_OPTID_NO_STRIP_INCOMING_X_FORWARDED_PROTO,
   SHRPX_OPTID_NO_VERIFY_OCSP,
   SHRPX_OPTID_NO_VIA,
@@ -1101,6 +1128,10 @@ enum {
   SHRPX_OPTID_STRIP_INCOMING_X_FORWARDED_FOR,
   SHRPX_OPTID_SUBCERT,
   SHRPX_OPTID_SYSLOG_FACILITY,
+  SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED,
+  SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_ADDRESS_FAMILY,
+  SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_CERT_FILE,
+  SHRPX_OPTID_TLS_ANTI_REPLAY_MEMCACHED_PRIVATE_KEY_FILE,
   SHRPX_OPTID_TLS_DYN_REC_IDLE_TIMEOUT,
   SHRPX_OPTID_TLS_DYN_REC_WARMUP_THRESHOLD,
   SHRPX_OPTID_TLS_MAX_PROTO_VERSION,
@@ -1143,26 +1174,20 @@ int option_lookup_token(const char *name, size_t namelen);
 // stored into the object pointed by |config|. This function returns 0
 // if it succeeds, or -1.  The |included_set| contains the all paths
 // already included while processing this configuration, to avoid loop
-// in --include option.  The |pattern_addr_indexer| contains a pair of
-// pattern of backend, and its index in DownstreamConfig::addr_groups.
-// It is introduced to speed up loading configuration file with lots
-// of backends.
+// in --include option.
 int parse_config(Config *config, const StringRef &opt, const StringRef &optarg,
-                 std::set<StringRef> &included_set,
-                 std::map<StringRef, size_t> &pattern_addr_indexer);
+                 std::set<StringRef> &included_set);
 
 // Similar to parse_config() above, but additional |optid| which
 // should be the return value of option_lookup_token(opt).
 int parse_config(Config *config, int optid, const StringRef &opt,
-                 const StringRef &optarg, std::set<StringRef> &included_set,
-                 std::map<StringRef, size_t> &pattern_addr_indexer);
+                 const StringRef &optarg, std::set<StringRef> &included_set);
 
 // Loads configurations from |filename| and stores them in |config|.
 // This function returns 0 if it succeeds, or -1.  See parse_config()
 // for |include_set|.
 int load_config(Config *config, const char *filename,
-                std::set<StringRef> &include_set,
-                std::map<StringRef, size_t> &pattern_addr_indexer);
+                std::set<StringRef> &include_set);
 
 // Parses header field in |optarg|.  We expect header field is formed
 // like "NAME: VALUE".  We require that NAME is non empty string.  ":"

src/shrpx_connection.cc

@@ -38,7 +38,6 @@
 #include "shrpx_log.h"
 #include "memchunk.h"
 #include "util.h"
-#include "ssl_compat.h"
 
 using namespace nghttp2;
 
@@ -60,7 +59,8 @@ Connection::Connection(struct ev_loop *loop, int fd, SSL *ssl,
                        IOCb readcb, TimerCb timeoutcb, void *data,
                        size_t tls_dyn_rec_warmup_threshold,
                        ev_tstamp tls_dyn_rec_idle_timeout, shrpx_proto proto)
-    : tls{DefaultMemchunks(mcpool), DefaultPeekMemchunks(mcpool)},
+    : tls{DefaultMemchunks(mcpool), DefaultPeekMemchunks(mcpool),
+          DefaultMemchunks(mcpool)},
       wlimit(loop, &wev, write_limit.rate, write_limit.burst),
       rlimit(loop, &rev, read_limit.rate, read_limit.burst, this),
       loop(loop),
@@ -92,7 +92,15 @@ Connection::Connection(struct ev_loop *loop, int fd, SSL *ssl,
   }
 }
 
-Connection::~Connection() { disconnect(); }
+Connection::~Connection() {
+  disconnect();
+
+#if OPENSSL_1_1_1_API
+  if (tls.ch_md_ctx) {
+    EVP_MD_CTX_free(tls.ch_md_ctx);
+  }
+#endif // OPENSSL_1_1_1_API
+}
 
 void Connection::disconnect() {
   if (tls.ssl) {
@@ -110,20 +118,34 @@ void Connection::disconnect() {
       tls.cached_session_lookup_req = nullptr;
     }
 
+    if (tls.anti_replay_req) {
+      tls.anti_replay_req->canceled = true;
+      tls.anti_replay_req = nullptr;
+    }
+
     SSL_shutdown(tls.ssl);
     SSL_free(tls.ssl);
     tls.ssl = nullptr;
 
+#if OPENSSL_1_1_1_API
+    if (tls.ch_md_ctx) {
+      EVP_MD_CTX_reset(tls.ch_md_ctx);
+    }
+#endif // OPENSSL_1_1_1_API
+
     tls.wbuf.reset();
     tls.rbuf.reset();
     tls.last_write_idle = 0.;
     tls.warmup_writelen = 0;
     tls.last_writelen = 0;
     tls.last_readlen = 0;
-    tls.handshake_state = 0;
+    tls.handshake_state = TLS_CONN_NORMAL;
     tls.initial_handshake_done = false;
     tls.reneg_started = false;
     tls.sct_requested = false;
+    tls.early_data_finish = false;
+    tls.early_cb_called = false;
+    tls.postpone_early_data = false;
   }
 
   if (fd != -1) {
@@ -141,11 +163,23 @@ void Connection::disconnect() {
   wlimit.stopw();
 }
 
-void Connection::prepare_client_handshake() { SSL_set_connect_state(tls.ssl); }
+void Connection::prepare_client_handshake() {
+  SSL_set_connect_state(tls.ssl);
+  // This prevents SSL_read_early_data from being called.
+  tls.early_data_finish = true;
+}
 
 void Connection::prepare_server_handshake() {
   SSL_set_accept_state(tls.ssl);
   tls.server_handshake = true;
+
+#if OPENSSL_1_1_1_API
+  if (!tls.ch_md_ctx) {
+    tls.ch_md_ctx = EVP_MD_CTX_new();
+  }
+
+  EVP_DigestInit_ex(tls.ch_md_ctx, EVP_sha256(), nullptr);
+#endif // OPENSSL_1_1_1_API
 }
 
 // BIO implementation is inspired by openldap implementation:
@@ -219,7 +253,19 @@ int shrpx_bio_read(BIO *b, char *buf, int len) {
     return -1;
   }
 
-  return rbuf.remove(buf, len);
+  len = rbuf.remove(buf, len);
+
+  if (conn->tls.early_cb_called) {
+    return len;
+  }
+
+#if OPENSSL_1_1_1_API
+  if (EVP_DigestUpdate(conn->tls.ch_md_ctx, buf, len) == 0) {
+    return -1;
+  }
+#endif // OPENSSL_1_1_1_API
+
+  return len;
 }
 } // namespace
 
@@ -327,8 +373,9 @@ int Connection::tls_handshake() {
   wlimit.stopw();
   ev_timer_stop(loop, &wt);
 
+  std::array<uint8_t, 16_k> buf;
+
   if (ev_is_active(&rev)) {
-    std::array<uint8_t, 8_k> buf;
     auto nread = read_clear(buf.data(), buf.size());
     if (nread < 0) {
       if (LOG_ENABLED(INFO)) {
@@ -348,6 +395,7 @@ int Connection::tls_handshake() {
 
   switch (tls.handshake_state) {
   case TLS_CONN_WAIT_FOR_SESSION_CACHE:
+  case TLS_CONN_WAIT_FOR_ANTI_REPLAY:
     return SHRPX_ERR_INPROGRESS;
   case TLS_CONN_GOT_SESSION_CACHE: {
     // Use the same trick invented by @kazuho in h2o project.
@@ -381,9 +429,73 @@ int Connection::tls_handshake() {
     break;
   }
 
+  int rv;
+
   ERR_clear_error();
 
-  auto rv = SSL_do_handshake(tls.ssl);
+#if OPENSSL_1_1_1_API
+  if (!tls.server_handshake || tls.early_data_finish) {
+    rv = SSL_do_handshake(tls.ssl);
+  } else {
+    for (;;) {
+      size_t nread;
+
+      rv = SSL_read_early_data(tls.ssl, buf.data(), buf.size(), &nread);
+      if (rv == SSL_READ_EARLY_DATA_ERROR) {
+        if (SSL_get_error(tls.ssl, rv) == SSL_ERROR_WANT_CLIENT_HELLO_CB) {
+          if (LOG_ENABLED(INFO)) {
+            LOG(INFO)
+                << "tls: early_cb returns negative return value; handshake "
+                   "interrupted";
+          }
+          break;
+        }
+
+        // If we have early data, and server sends ServerHello, assume
+        // that handshake is completed in server side, and start
+        // processing request.  If we don't exit handshake code here,
+        // server waits for EndOfEarlyData and Finished message from
+        // client, which voids the purpose of 0-RTT data.  The left
+        // over of handshake is done through write_tls or read_tls.
+        if (!tls.postpone_early_data &&
+            (tls.handshake_state == TLS_CONN_WRITE_STARTED ||
+             tls.wbuf.rleft()) &&
+            tls.earlybuf.rleft()) {
+          rv = 1;
+        }
+
+        break;
+      }
+
+      if (LOG_ENABLED(INFO)) {
+        LOG(INFO) << "tls: read early data " << nread << " bytes";
+      }
+
+      tls.earlybuf.append(buf.data(), nread);
+
+      if (rv == SSL_READ_EARLY_DATA_FINISH) {
+        if (LOG_ENABLED(INFO)) {
+          LOG(INFO) << "tls: read all early data; total "
+                    << tls.earlybuf.rleft() << " bytes";
+        }
+        tls.early_data_finish = true;
+        // The same reason stated above.
+        if (!tls.postpone_early_data &&
+            (tls.handshake_state == TLS_CONN_WRITE_STARTED ||
+             tls.wbuf.rleft()) &&
+            tls.earlybuf.rleft()) {
+          rv = 1;
+        } else {
+          ERR_clear_error();
+          rv = SSL_do_handshake(tls.ssl);
+        }
+        break;
+      }
+    }
+  }
+#else  // !OPENSSL_1_1_1_API
+  rv = SSL_do_handshake(tls.ssl);
+#endif // !OPENSSL_1_1_1_API
 
   if (rv <= 0) {
     auto err = SSL_get_error(tls.ssl, rv);
@@ -397,6 +509,9 @@ int Connection::tls_handshake() {
       }
       break;
     case SSL_ERROR_WANT_WRITE:
+#if OPENSSL_1_1_1_API
+    case SSL_ERROR_WANT_CLIENT_HELLO_CB:
+#endif // OPENSSL_1_1_1_API
       break;
     case SSL_ERROR_SSL:
       if (LOG_ENABLED(INFO)) {
@@ -412,7 +527,8 @@ int Connection::tls_handshake() {
     }
   }
 
-  if (tls.handshake_state == TLS_CONN_WAIT_FOR_SESSION_CACHE) {
+  if (tls.handshake_state == TLS_CONN_WAIT_FOR_SESSION_CACHE ||
+      tls.handshake_state == TLS_CONN_WAIT_FOR_ANTI_REPLAY) {
     if (LOG_ENABLED(INFO)) {
       LOG(INFO) << "tls: handshake is still in progress";
     }
@@ -619,7 +735,21 @@ ssize_t Connection::write_tls(const void *data, size_t len) {
 
   ERR_clear_error();
 
+#if OPENSSL_1_1_1_API
+  int rv;
+  if (SSL_is_init_finished(tls.ssl)) {
+    rv = SSL_write(tls.ssl, data, len);
+  } else {
+    size_t nwrite;
+    rv = SSL_write_early_data(tls.ssl, data, len, &nwrite);
+    // Use the same semantics with SSL_write.
+    if (rv == 1) {
+      rv = nwrite;
+    }
+  }
+#else  // !OPENSSL_1_1_1_API
   auto rv = SSL_write(tls.ssl, data, len);
+#endif // !OPENSSL_1_1_1_API
 
   if (rv <= 0) {
     auto err = SSL_get_error(tls.ssl, rv);
@@ -654,6 +784,14 @@ ssize_t Connection::write_tls(const void *data, size_t len) {
 }
 
 ssize_t Connection::read_tls(void *data, size_t len) {
+  ERR_clear_error();
+
+#if OPENSSL_1_1_1_API
+  if (tls.earlybuf.rleft()) {
+    return tls.earlybuf.remove(data, len);
+  }
+#endif // OPENSSL_1_1_1_API
+
   // SSL_read requires the same arguments (buf pointer and its
   // length) on SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE.
   // rlimit_.avail() or rlimit_.avail() may return different length
@@ -671,7 +809,46 @@ ssize_t Connection::read_tls(void *data, size_t len) {
     tls.last_readlen = 0;
   }
 
-  ERR_clear_error();
+#if OPENSSL_1_1_1_API
+  if (!tls.early_data_finish) {
+    // TLSv1.3 handshake is still going on.
+    size_t nread;
+    auto rv = SSL_read_early_data(tls.ssl, data, len, &nread);
+    if (rv == SSL_READ_EARLY_DATA_ERROR) {
+      auto err = SSL_get_error(tls.ssl, rv);
+      switch (err) {
+      case SSL_ERROR_WANT_READ:
+        tls.last_readlen = len;
+        return 0;
+      case SSL_ERROR_SSL:
+        if (LOG_ENABLED(INFO)) {
+          LOG(INFO) << "SSL_read: "
+                    << ERR_error_string(ERR_get_error(), nullptr);
+        }
+        return SHRPX_ERR_NETWORK;
+      default:
+        if (LOG_ENABLED(INFO)) {
+          LOG(INFO) << "SSL_read: SSL_get_error returned " << err;
+        }
+        return SHRPX_ERR_NETWORK;
+      }
+    }
+
+    if (LOG_ENABLED(INFO)) {
+      LOG(INFO) << "tls: read early data " << nread << " bytes";
+    }
+
+    if (rv == SSL_READ_EARLY_DATA_FINISH) {
+      if (LOG_ENABLED(INFO)) {
+        LOG(INFO) << "tls: read all early data";
+      }
+      tls.early_data_finish = true;
+      // We may have stopped write watcher in write_tls.
+      wlimit.startw();
+    }
+    return nread;
+  }
+#endif // OPENSSL_1_1_1_API
 
   auto rv = SSL_read(tls.ssl, data, len);
 

src/shrpx_connection.h

@@ -32,10 +32,12 @@
 #include <ev.h>
 
 #include <openssl/ssl.h>
+#include <openssl/evp.h>
 
 #include "shrpx_rate_limit.h"
 #include "shrpx_error.h"
 #include "memchunk.h"
+#include "ssl_compat.h"
 
 namespace shrpx {
 
@@ -50,16 +52,26 @@ enum {
   TLS_CONN_WAIT_FOR_SESSION_CACHE,
   TLS_CONN_GOT_SESSION_CACHE,
   TLS_CONN_CANCEL_SESSION_CACHE,
+  TLS_CONN_WAIT_FOR_ANTI_REPLAY,
   TLS_CONN_WRITE_STARTED,
 };
 
 struct TLSConnection {
   DefaultMemchunks wbuf;
   DefaultPeekMemchunks rbuf;
+  // Stores TLSv1.3 early data.
+  DefaultMemchunks earlybuf;
+  // Message digest of ClientHello in hex string.
+  StringRef ch_hex_md;
   SSL *ssl;
   SSL_SESSION *cached_session;
   MemcachedRequest *cached_session_lookup_req;
   tls::TLSSessionCache *client_session_cache;
+#if OPENSSL_1_1_1_API
+  // Message digest context to calculate ClientHello for anti-replay.
+  EVP_MD_CTX *ch_md_ctx;
+#endif // !OPENSSL_1_1_1_API
+  MemcachedRequest *anti_replay_req;
   ev_tstamp last_write_idle;
   size_t warmup_writelen;
   // length passed to SSL_write and SSL_read last time.  This is
@@ -74,6 +86,17 @@ struct TLSConnection {
   // true if ssl is initialized as server, and client requested
   // signed_certificate_timestamp extension.
   bool sct_requested;
+  // true if TLSv1.3 early data has been completely received.  Since
+  // SSL_read_early_data acts like SSL_do_handshake, this field may be
+  // true even if the negotiated TLS version is TLSv1.2 or earlier.
+  // This value is also true if this is client side connection for
+  // convenience.
+  bool early_data_finish;
+  // true if early_cb gets called.
+  bool early_cb_called;
+  // true if processing early data should be postponed until handshake
+  // finishes.
+  bool postpone_early_data;
 };
 
 struct TCPHint {

src/shrpx_connection_handler.cc

@@ -235,9 +235,24 @@ int ConnectionHandler::create_single_worker() {
     }
   }
 
+  SSL_CTX *anti_replay_ssl_ctx = nullptr;
+  {
+    auto &memcachedconf = config->tls.anti_replay.memcached;
+
+    if (memcachedconf.tls) {
+      anti_replay_ssl_ctx = tls::create_ssl_client_context(
+#ifdef HAVE_NEVERBLEED
+          nb_.get(),
+#endif // HAVE_NEVERBLEED
+          tlsconf.cacert, memcachedconf.cert_file,
+          memcachedconf.private_key_file, nullptr);
+      all_ssl_ctx_.push_back(anti_replay_ssl_ctx);
+    }
+  }
+
   single_worker_ = make_unique<Worker>(
-      loop_, sv_ssl_ctx, cl_ssl_ctx, session_cache_ssl_ctx, cert_tree_.get(),
-      ticket_keys_, this, config->conn.downstream);
+      loop_, sv_ssl_ctx, cl_ssl_ctx, session_cache_ssl_ctx, anti_replay_ssl_ctx,
+      cert_tree_.get(), ticket_keys_, this, config->conn.downstream);
 #ifdef HAVE_MRUBY
   if (single_worker_->create_mruby_context() != 0) {
     return -1;
@@ -293,12 +308,28 @@ int ConnectionHandler::create_worker_thread(size_t num) {
     }
   }
 
+  SSL_CTX *anti_replay_ssl_ctx = nullptr;
+  {
+    auto &memcachedconf = config->tls.anti_replay.memcached;
+
+    if (memcachedconf.tls) {
+      anti_replay_ssl_ctx = tls::create_ssl_client_context(
+#ifdef HAVE_NEVERBLEED
+          nb_.get(),
+#endif // HAVE_NEVERBLEED
+          tlsconf.cacert, memcachedconf.cert_file,
+          memcachedconf.private_key_file, nullptr);
+      all_ssl_ctx_.push_back(anti_replay_ssl_ctx);
+    }
+  }
+
   for (size_t i = 0; i < num; ++i) {
     auto loop = ev_loop_new(config->ev_loop_flags);
 
-    auto worker = make_unique<Worker>(
-        loop, sv_ssl_ctx, cl_ssl_ctx, session_cache_ssl_ctx, cert_tree_.get(),
-        ticket_keys_, this, config->conn.downstream);
+    auto worker =
+        make_unique<Worker>(loop, sv_ssl_ctx, cl_ssl_ctx, session_cache_ssl_ctx,
+                            anti_replay_ssl_ctx, cert_tree_.get(), ticket_keys_,
+                            this, config->conn.downstream);
 #ifdef HAVE_MRUBY
     if (worker->create_mruby_context() != 0) {
       return -1;

src/shrpx_downstream.h

@@ -32,7 +32,6 @@
 #include <string>
 #include <memory>
 #include <chrono>
-#include <algorithm>
 
 #include <ev.h>
 
@@ -208,40 +207,7 @@ struct Response {
     unconsumed_body_length -= len;
   }
 
-  // returns true if a resource denoted by scheme, authority, and path
-  // has already been pushed.
-  bool is_resource_pushed(const StringRef &scheme, const StringRef &authority,
-                          const StringRef &path) const {
-    if (!pushed_resources) {
-      return false;
-    }
-    return std::find(std::begin(*pushed_resources), std::end(*pushed_resources),
-                     std::make_tuple(scheme, authority, path)) !=
-           std::end(*pushed_resources);
-  }
-
-  // remember that a resource denoted by scheme, authority, and path
-  // is pushed.
-  void resource_pushed(const StringRef &scheme, const StringRef &authority,
-                       const StringRef &path) {
-    if (!pushed_resources) {
-      pushed_resources = make_unique<
-          std::vector<std::tuple<StringRef, StringRef, StringRef>>>();
-    }
-    pushed_resources->emplace_back(scheme, authority, path);
-  }
-
   FieldStore fs;
-  // array of the tuple of scheme, authority, and path of pushed
-  // resource.  This is required because RFC 8297 says that server
-  // typically includes header fields appeared in non-final response
-  // header fields in final response header fields.  Without checking
-  // that a particular resource has already been pushed, or not, we
-  // end up pushing the same resource at least twice.  It is unknown
-  // that we should use more complex data structure (e.g., std::set)
-  // to find the resources faster.
-  std::unique_ptr<std::vector<std::tuple<StringRef, StringRef, StringRef>>>
-      pushed_resources;
   // the length of response body received so far
   int64_t recv_body_length;
   // The number of bytes not consumed by the application yet.  This is
@@ -503,7 +469,7 @@ private:
   Upstream *upstream_;
   std::unique_ptr<DownstreamConnection> dconn_;
 
-  // only used by HTTP/2 upstream
+  // only used by HTTP/2 or SPDY upstream
   BlockedLink *blocked_link_;
   // The backend address used to fulfill this request.  These are for
   // logging purpose.
@@ -526,7 +492,7 @@ private:
   int request_state_;
   // response state
   int response_state_;
-  // only used by HTTP/2 upstream
+  // only used by HTTP/2 or SPDY upstream
   int dispatch_state_;
   // true if the connection is upgraded (HTTP Upgrade or CONNECT),
   // excluding upgrade to HTTP/2.

src/shrpx_http2_downstream_connection.cc

@@ -271,7 +271,7 @@ int Http2DownstreamConnection::push_request_headers() {
     num_cookies = downstream_->count_crumble_request_cookie();
   }
 
-  // 9 means:
+  // 10 means:
   // 1. :method
   // 2. :scheme
   // 3. :path
@@ -281,8 +281,9 @@ int Http2DownstreamConnection::push_request_headers() {
   // 7. x-forwarded-proto (optional)
   // 8. te (optional)
   // 9. forwarded (optional)
+  // 10. nghttpx-0rtt-uniq (optional)
   auto nva = std::vector<nghttp2_nv>();
-  nva.reserve(req.fs.headers().size() + 9 + num_cookies +
+  nva.reserve(req.fs.headers().size() + 10 + num_cookies +
               httpconf.add_request_headers.size());
 
   nva.push_back(
@@ -291,14 +292,7 @@ int Http2DownstreamConnection::push_request_headers() {
   if (req.method != HTTP_CONNECT) {
     assert(!req.scheme.empty());
 
-    auto addr = http2session_->get_addr();
-    assert(addr);
-    // We will handle more protocol scheme upgrade in the future.
-    if (addr->tls && addr->upgrade_scheme && req.scheme == "http") {
-      nva.push_back(http2::make_nv_ll(":scheme", "https"));
-    } else {
-      nva.push_back(http2::make_nv_ls_nocopy(":scheme", req.scheme));
-    }
+    nva.push_back(http2::make_nv_ls_nocopy(":scheme", req.scheme));
 
     if (req.method == HTTP_OPTIONS && req.path.empty()) {
       nva.push_back(http2::make_nv_ll(":path", "*"));
@@ -318,11 +312,15 @@ int Http2DownstreamConnection::push_request_headers() {
   auto &fwdconf = httpconf.forwarded;
   auto &xffconf = httpconf.xff;
   auto &xfpconf = httpconf.xfp;
+  auto &zero_rtt_uniqconf = httpconf.zero_rtt_uniq;
 
   uint32_t build_flags =
       (fwdconf.strip_incoming ? http2::HDOP_STRIP_FORWARDED : 0) |
       (xffconf.strip_incoming ? http2::HDOP_STRIP_X_FORWARDED_FOR : 0) |
-      (xfpconf.strip_incoming ? http2::HDOP_STRIP_X_FORWARDED_PROTO : 0);
+      (xfpconf.strip_incoming ? http2::HDOP_STRIP_X_FORWARDED_PROTO : 0) |
+      (zero_rtt_uniqconf.strip_incoming
+           ? http2::HDOP_STRIP_NGHTTPX_ZERO_RTT_UNIQ
+           : 0);
 
   http2::copy_headers_to_nva_nocopy(nva, req.fs.headers(), build_flags);
 
@@ -333,6 +331,15 @@ int Http2DownstreamConnection::push_request_headers() {
   auto upstream = downstream_->get_upstream();
   auto handler = upstream->get_client_handler();
 
+#if OPENSSL_1_1_1_API
+  auto conn = handler->get_connection();
+
+  if (!SSL_is_init_finished(conn->tls.ssl)) {
+    nva.push_back(
+        http2::make_nv_ls_nocopy("nghttpx-0rtt-uniq", conn->tls.ch_hex_md));
+  }
+#endif // OPENSSL_1_1_1_API
+
   auto fwd =
       fwdconf.strip_incoming ? nullptr : req.fs.header(http2::HD_FORWARDED);
 

src/shrpx_http2_session.cc

@@ -270,8 +270,8 @@ int Http2Session::disconnect(bool hard) {
   // When deleting Http2DownstreamConnection, it calls this object's
   // remove_downstream_connection().  The multiple
   // Http2DownstreamConnection objects belong to the same
-  // ClientHandler object if upstream is h2.  So be careful when you
-  // delete ClientHandler here.
+  // ClientHandler object if upstream is h2 or SPDY.  So be careful
+  // when you delete ClientHandler here.
   //
   // We allow creating new pending Http2DownstreamConnection with this
   // object.  Upstream::on_downstream_reset() may add
@@ -577,11 +577,11 @@ int Http2Session::initiate_connection() {
       }
     }
 
+    on_write_ = &Http2Session::downstream_write;
+    on_read_ = &Http2Session::downstream_read;
+
     // We have been already connected when no TLS and proxy is used.
     if (state_ == PROXY_CONNECTED) {
-      on_read_ = &Http2Session::read_noop;
-      on_write_ = &Http2Session::write_noop;
-
       return connected();
     }
 
@@ -1520,7 +1520,7 @@ int on_frame_not_send_callback(nghttp2_session *session,
 
     if (upstream->on_downstream_reset(downstream, false)) {
       // This should be done for h1 upstream only.  Deleting
-      // ClientHandler for h2 upstream may lead to crash.
+      // ClientHandler for h2 or SPDY upstream may lead to crash.
       delete upstream->get_client_handler();
     }
 
@@ -1642,9 +1642,6 @@ int Http2Session::connection_made() {
 
   state_ = Http2Session::CONNECTED;
 
-  on_write_ = &Http2Session::downstream_write;
-  on_read_ = &Http2Session::downstream_read;
-
   if (addr_->tls) {
     const unsigned char *next_proto = nullptr;
     unsigned int next_proto_len = 0;

src/shrpx_http2_upstream.cc

@@ -2039,7 +2039,7 @@ int Http2Upstream::prepare_push_promise(Downstream *downstream) {
   int rv;
 
   const auto &req = downstream->request();
-  auto &resp = downstream->response();
+  const auto &resp = downstream->response();
 
   auto base = http2::get_pure_path_component(req.path);
   if (base.empty()) {
@@ -2069,16 +2069,10 @@ int Http2Upstream::prepare_push_promise(Downstream *downstream) {
         authority = req.authority;
       }
 
-      if (resp.is_resource_pushed(scheme, authority, path)) {
-        continue;
-      }
-
       rv = submit_push_promise(scheme, authority, path, downstream);
       if (rv != 0) {
         return -1;
       }
-
-      resp.resource_pushed(scheme, authority, path);
     }
   }
   return 0;
@@ -2188,20 +2182,12 @@ int Http2Upstream::initiate_push(Downstream *downstream, const StringRef &uri) {
     authority = req.authority;
   }
 
-  auto &resp = downstream->response();
-
-  if (resp.is_resource_pushed(scheme, authority, path)) {
-    return 0;
-  }
-
   rv = submit_push_promise(scheme, authority, path, downstream);
 
   if (rv != 0) {
     return -1;
   }
 
-  resp.resource_pushed(scheme, authority, path);
-
   return 0;
 }
 

src/shrpx_http_downstream_connection.cc

@@ -535,11 +535,15 @@ int HttpDownstreamConnection::push_request_headers() {
   auto &fwdconf = httpconf.forwarded;
   auto &xffconf = httpconf.xff;
   auto &xfpconf = httpconf.xfp;
+  auto &zero_rtt_uniqconf = httpconf.zero_rtt_uniq;
 
   uint32_t build_flags =
       (fwdconf.strip_incoming ? http2::HDOP_STRIP_FORWARDED : 0) |
       (xffconf.strip_incoming ? http2::HDOP_STRIP_X_FORWARDED_FOR : 0) |
-      (xfpconf.strip_incoming ? http2::HDOP_STRIP_X_FORWARDED_PROTO : 0);
+      (xfpconf.strip_incoming ? http2::HDOP_STRIP_X_FORWARDED_PROTO : 0) |
+      (zero_rtt_uniqconf.strip_incoming
+           ? http2::HDOP_STRIP_NGHTTPX_ZERO_RTT_UNIQ
+           : 0);
 
   http2::build_http1_headers_from_headers(buf, req.fs.headers(), build_flags);
 
@@ -580,6 +584,16 @@ int HttpDownstreamConnection::push_request_headers() {
   auto upstream = downstream_->get_upstream();
   auto handler = upstream->get_client_handler();
 
+#if OPENSSL_1_1_1_API
+  auto conn = handler->get_connection();
+
+  if (!SSL_is_init_finished(conn->tls.ssl)) {
+    buf->append("Nghttpx-0rtt-uniq: ");
+    buf->append(conn->tls.ch_hex_md);
+    buf->append("\r\n");
+  }
+#endif // OPENSSL_1_1_1_API
+
   auto fwd =
       fwdconf.strip_incoming ? nullptr : req.fs.header(http2::HD_FORWARDED);
 
@@ -782,10 +796,8 @@ void idle_timeoutcb(struct ev_loop *loop, ev_timer *w, int revents) {
   auto conn = static_cast<Connection *>(w->data);
   auto dconn = static_cast<HttpDownstreamConnection *>(conn->data);
 
-  if (w == &conn->rt && !conn->expired_rt()) {
-    return;
-  }
-
+  // We don't have to check conn->expired_rt() since we restart timer
+  // when connection gets idle.
   if (LOG_ENABLED(INFO)) {
     DCLOG(INFO, dconn) << "Idle connection timeout";
   }

src/shrpx_mruby_module_request.cc

@@ -246,9 +246,8 @@ mrb_value request_mod_header(mrb_state *mrb, mrb_value self, bool repl) {
         continue;
       }
       if (i != p) {
-        headers[p] = std::move(kv);
+        headers[p++] = std::move(kv);
       }
-      ++p;
     }
     headers.resize(p);
   }

src/shrpx_mruby_module_response.cc

@@ -138,9 +138,8 @@ mrb_value response_mod_header(mrb_state *mrb, mrb_value self, bool repl) {
         continue;
       }
       if (i != p) {
-        headers[p] = std::move(kv);
+        headers[p++] = std::move(kv);
       }
-      ++p;
     }
     headers.resize(p);
   }

src/shrpx_rate_limit.cc

@@ -108,8 +108,9 @@ void RateLimit::stopw() {
 }
 
 void RateLimit::handle_tls_pending_read() {
-  if (!conn_ || !conn_->tls.ssl ||
-      (SSL_pending(conn_->tls.ssl) == 0 && conn_->tls.rbuf.rleft() == 0)) {
+  if (!conn_ || !conn_->tls.ssl || !conn_->tls.initial_handshake_done ||
+      (SSL_pending(conn_->tls.ssl) == 0 && conn_->tls.rbuf.rleft() == 0 &&
+       conn_->tls.earlybuf.rleft() == 0)) {
     return;
   }
 

src/shrpx_spdy_upstream.h

@@ -0,0 +1,113 @@
+/*
+ * nghttp2 - HTTP/2 C Library
+ *
+ * Copyright (c) 2012 Tatsuhiro Tsujikawa
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining
+ * a copy of this software and associated documentation files (the
+ * "Software"), to deal in the Software without restriction, including
+ * without limitation the rights to use, copy, modify, merge, publish,
+ * distribute, sublicense, and/or sell copies of the Software, and to
+ * permit persons to whom the Software is furnished to do so, subject to
+ * the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be
+ * included in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+ * LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+ * OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+ * WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef SHRPX_SPDY_UPSTREAM_H
+#define SHRPX_SPDY_UPSTREAM_H
+
+#include "shrpx.h"
+
+#include <memory>
+
+#include <ev.h>
+
+#include <spdylay/spdylay.h>
+
+#include "shrpx_upstream.h"
+#include "shrpx_downstream_queue.h"
+#include "memchunk.h"
+#include "buffer.h"
+
+namespace shrpx {
+
+class ClientHandler;
+
+class SpdyUpstream : public Upstream {
+public:
+  SpdyUpstream(uint16_t version, ClientHandler *handler);
+  virtual ~SpdyUpstream();
+  virtual int on_read();
+  virtual int on_write();
+  virtual int on_timeout(Downstream *downstream);
+  virtual int on_downstream_abort_request(Downstream *downstream,
+                                          unsigned int status_code);
+  virtual int
+  on_downstream_abort_request_with_https_redirect(Downstream *downstream);
+  virtual ClientHandler *get_client_handler() const;
+  virtual int downstream_read(DownstreamConnection *dconn);
+  virtual int downstream_write(DownstreamConnection *dconn);
+  virtual int downstream_eof(DownstreamConnection *dconn);
+  virtual int downstream_error(DownstreamConnection *dconn, int events);
+  Downstream *add_pending_downstream(int32_t stream_id);
+  void remove_downstream(Downstream *downstream);
+
+  int rst_stream(Downstream *downstream, int status_code);
+  int error_reply(Downstream *downstream, unsigned int status_code);
+
+  virtual void pause_read(IOCtrlReason reason);
+  virtual int resume_read(IOCtrlReason reason, Downstream *downstream,
+                          size_t consumed);
+
+  virtual int on_downstream_header_complete(Downstream *downstream);
+  virtual int on_downstream_body(Downstream *downstream, const uint8_t *data,
+                                 size_t len, bool flush);
+  virtual int on_downstream_body_complete(Downstream *downstream);
+
+  virtual void on_handler_delete();
+  virtual int on_downstream_reset(Downstream *downstream, bool no_retry);
+
+  virtual int send_reply(Downstream *downstream, const uint8_t *body,
+                         size_t bodylen);
+  virtual int initiate_push(Downstream *downstream, const StringRef &uri);
+  virtual int response_riovec(struct iovec *iov, int iovcnt) const;
+  virtual void response_drain(size_t n);
+  virtual bool response_empty() const;
+
+  virtual Downstream *on_downstream_push_promise(Downstream *downstream,
+                                                 int32_t promised_stream_id);
+  virtual int
+  on_downstream_push_promise_complete(Downstream *downstream,
+                                      Downstream *promised_downstream);
+  virtual bool push_enabled() const;
+  virtual void cancel_premature_downstream(Downstream *promised_downstream);
+
+  bool get_flow_control() const;
+
+  int consume(int32_t stream_id, size_t len);
+
+  void start_downstream(Downstream *downstream);
+  void initiate_downstream(Downstream *downstream);
+
+  DefaultMemchunks *get_response_buf();
+
+private:
+  DefaultMemchunks wb_;
+  DownstreamQueue downstream_queue_;
+  ClientHandler *handler_;
+  spdylay_session *session_;
+  bool flow_control_;
+};
+
+} // namespace shrpx
+
+#endif // SHRPX_SPDY_UPSTREAM_H

src/shrpx_tls.cc

@@ -51,6 +51,10 @@
 
 #include <nghttp2/nghttp2.h>
 
+#ifdef HAVE_SPDYLAY
+#include <spdylay/spdylay.h>
+#endif // HAVE_SPDYLAY
+
 #include "shrpx_log.h"
 #include "shrpx_client_handler.h"
 #include "shrpx_config.h"
@@ -145,8 +149,6 @@ int ssl_pem_passwd_cb(char *buf, int size, int rwflag, void *user_data) {
 } // namespace
 
 namespace {
-// *al is set to SSL_AD_UNRECOGNIZED_NAME by openssl, so we don't have
-// to set it explicitly.
 int servername_callback(SSL *ssl, int *al, void *arg) {
   auto conn = static_cast<Connection *>(SSL_get_app_data(ssl));
   auto handler = static_cast<ClientHandler *>(conn->data);
@@ -508,6 +510,13 @@ int ticket_key_cb(SSL *ssl, unsigned char *key_name, unsigned char *iv,
 
 namespace {
 void info_callback(const SSL *ssl, int where, int ret) {
+#ifdef TLS1_3_VERSION
+  // TLSv1.3 has no renegotiation.
+  if (SSL_version(ssl) == TLS1_3_VERSION) {
+    return;
+  }
+#endif // TLS1_3_VERSION
+
   // To mitigate possible DOS attack using lots of renegotiations, we
   // disable renegotiation. Since OpenSSL does not provide an easy way
   // to disable it, we check that renegotiation is started in this
@@ -525,6 +534,105 @@ void info_callback(const SSL *ssl, int where, int ret) {
 }
 } // namespace
 
+#if OPENSSL_1_1_1_API
+constexpr auto MEMCACHED_ANTI_REPLY_KEY_PREFIX =
+    StringRef::from_lit("nghttpx:anti-reply:");
+
+namespace {
+int early_cb(SSL *ssl, int *al, void *arg) {
+  auto conn = static_cast<Connection *>(SSL_get_app_data(ssl));
+  if (conn->tls.early_cb_called) {
+    return 1;
+  }
+
+  conn->tls.early_cb_called = true;
+
+  const unsigned char *ext;
+  size_t extlen;
+
+  if (!SSL_client_hello_get0_ext(conn->tls.ssl, TLSEXT_TYPE_early_data, &ext,
+                                 &extlen)) {
+    if (LOG_ENABLED(INFO)) {
+      LOG(INFO) << "early_data extension does not exist";
+    }
+    return 1;
+  }
+
+  if (!SSL_client_hello_get0_ext(conn->tls.ssl, TLSEXT_TYPE_psk, &ext,
+                                 &extlen)) {
+    if (LOG_ENABLED(INFO)) {
+      LOG(INFO) << "pre_shared_key extension does not exist";
+    }
+    return 1;
+  }
+
+  std::array<uint8_t, 32> md;
+  unsigned int mdlen;
+  if (EVP_DigestFinal_ex(conn->tls.ch_md_ctx, md.data(), &mdlen) == 0) {
+    LOG(ERROR) << "EVP_DigestFinal_ex failed";
+    return 0;
+  }
+  assert(md.size() == mdlen);
+
+  auto handler = static_cast<ClientHandler *>(conn->data);
+  auto worker = handler->get_worker();
+  auto dispatcher = worker->get_anti_replay_memcached_dispatcher();
+  auto &balloc = handler->get_block_allocator();
+
+  auto &tlsconf = get_config()->tls;
+
+  conn->tls.ch_hex_md =
+      util::format_hex(balloc, StringRef{std::begin(md), std::end(md)});
+
+  if (tlsconf.anti_replay.memcached.host.empty()) {
+    return 1;
+  }
+
+  auto req = make_unique<MemcachedRequest>();
+  req->op = MEMCACHED_OP_ADD;
+  req->key = MEMCACHED_ANTI_REPLY_KEY_PREFIX.str();
+  req->key += conn->tls.ch_hex_md;
+
+  // TODO No value at the moment
+
+  // Set the same timeout value for session with the hope that
+  // OpenSSL library invalidates the outdated ticket.
+  req->expiry = tlsconf.session_timeout.count();
+  req->cb = [conn](MemcachedRequest *req, MemcachedResult res) {
+    if (LOG_ENABLED(INFO)) {
+      LOG(INFO) << "Memcached: ClientHello anti-replay registration done.  key="
+                << req->key << ", status_code=" << res.status_code;
+    }
+
+    // We might stop reading, so start it again
+    conn->rlimit.startw();
+    ev_timer_again(conn->loop, &conn->rt);
+
+    conn->wlimit.startw();
+    ev_timer_again(conn->loop, &conn->wt);
+
+    conn->tls.anti_replay_req = nullptr;
+
+    if (res.status_code != 0) {
+      // If we cannot add key/value, just postpone processing 0-RTT
+      // early data until handshake finishes.  Note that memcached
+      // atomically adds key/value.
+      conn->tls.postpone_early_data = true;
+    }
+
+    conn->tls.handshake_state = TLS_CONN_NORMAL;
+  };
+
+  conn->tls.handshake_state = TLS_CONN_WAIT_FOR_ANTI_REPLAY;
+  conn->tls.anti_replay_req = req.get();
+
+  dispatcher->add_request(std::move(req));
+
+  return -1;
+}
+} // namespace
+#endif // OPENSSL_1_1_1_API
+
 #if OPENSSL_VERSION_NUMBER >= 0x10002000L
 namespace {
 int alpn_select_proto_cb(SSL *ssl, const unsigned char **out,
@@ -911,6 +1019,10 @@ SSL_CTX *create_ssl_context(const char *private_key_file, const char *cert_file,
 #endif // OPENSSL_IS_BORINGSSL
   SSL_CTX_set_info_callback(ssl_ctx, info_callback);
 
+#if OPENSSL_1_1_1_API
+  SSL_CTX_set_client_hello_cb(ssl_ctx, early_cb, nullptr);
+#endif // OPENSSL_1_1_1_API
+
 #ifdef OPENSSL_IS_BORINGSSL
   SSL_CTX_set_early_data_enabled(ssl_ctx, 1);
 #endif // OPENSSL_IS_BORINGSSL

src/shrpx_worker.cc

@@ -68,50 +68,57 @@ void proc_wev_cb(struct ev_loop *loop, ev_timer *w, int revents) {
 }
 } // namespace
 
-// DownstreamKey is used to index SharedDownstreamAddr in order to
-// find the same configuration.
-using DownstreamKey = std::tuple<
-    std::vector<std::tuple<StringRef, StringRef, size_t, size_t, shrpx_proto,
-                           uint16_t, bool, bool, bool, bool>>,
-    bool, int, StringRef, StringRef, int>;
-
 namespace {
-DownstreamKey create_downstream_key(
-    const std::shared_ptr<SharedDownstreamAddr> &shared_addr) {
-  DownstreamKey dkey;
-
-  auto &addrs = std::get<0>(dkey);
-  addrs.resize(shared_addr->addrs.size());
-  auto p = std::begin(addrs);
-  for (auto &a : shared_addr->addrs) {
-    std::get<0>(*p) = a.host;
-    std::get<1>(*p) = a.sni;
-    std::get<2>(*p) = a.fall;
-    std::get<3>(*p) = a.rise;
-    std::get<4>(*p) = a.proto;
-    std::get<5>(*p) = a.port;
-    std::get<6>(*p) = a.host_unix;
-    std::get<7>(*p) = a.tls;
-    std::get<8>(*p) = a.dns;
-    std::get<9>(*p) = a.upgrade_scheme;
-    ++p;
+bool match_shared_downstream_addr(
+    const std::shared_ptr<SharedDownstreamAddr> &lhs,
+    const std::shared_ptr<SharedDownstreamAddr> &rhs) {
+  if (lhs->addrs.size() != rhs->addrs.size()) {
+    return false;
   }
-  std::sort(std::begin(addrs), std::end(addrs));
 
-  std::get<1>(dkey) = shared_addr->redirect_if_not_tls;
+  if (lhs->affinity.type != rhs->affinity.type ||
+      lhs->redirect_if_not_tls != rhs->redirect_if_not_tls) {
+    return false;
+  }
 
-  auto &affinity = shared_addr->affinity;
-  std::get<2>(dkey) = affinity.type;
-  std::get<3>(dkey) = affinity.cookie.name;
-  std::get<4>(dkey) = affinity.cookie.path;
-  std::get<5>(dkey) = affinity.cookie.secure;
+  if (lhs->affinity.type == AFFINITY_COOKIE &&
+      (lhs->affinity.cookie.name != rhs->affinity.cookie.name ||
+       lhs->affinity.cookie.path != rhs->affinity.cookie.path ||
+       lhs->affinity.cookie.secure != rhs->affinity.cookie.secure)) {
+    return false;
+  }
 
-  return dkey;
+  auto used = std::vector<bool>(lhs->addrs.size());
+
+  for (auto &a : lhs->addrs) {
+    size_t i;
+    for (i = 0; i < rhs->addrs.size(); ++i) {
+      if (used[i]) {
+        continue;
+      }
+
+      auto &b = rhs->addrs[i];
+      if (a.host == b.host && a.port == b.port && a.host_unix == b.host_unix &&
+          a.proto == b.proto && a.tls == b.tls && a.sni == b.sni &&
+          a.fall == b.fall && a.rise == b.rise && a.dns == b.dns) {
+        break;
+      }
+    }
+
+    if (i == rhs->addrs.size()) {
+      return false;
+    }
+
+    used[i] = true;
+  }
+
+  return true;
 }
 } // namespace
 
 Worker::Worker(struct ev_loop *loop, SSL_CTX *sv_ssl_ctx, SSL_CTX *cl_ssl_ctx,
                SSL_CTX *tls_session_cache_memcached_ssl_ctx,
+               SSL_CTX *tls_anti_replay_memcached_ssl_ctx,
                tls::CertLookupTree *cert_tree,
                const std::shared_ptr<TicketKeys> &ticket_keys,
                ConnectionHandler *conn_handler,
@@ -147,6 +154,15 @@ Worker::Worker(struct ev_loop *loop, SSL_CTX *sv_ssl_ctx, SSL_CTX *cl_ssl_ctx,
         StringRef{session_cacheconf.memcached.host}, &mcpool_, randgen_);
   }
 
+  auto &anti_replayconf = get_config()->tls.anti_replay;
+
+  if (!anti_replayconf.memcached.host.empty()) {
+    anti_replay_memcached_dispatcher_ = make_unique<MemcachedDispatcher>(
+        &anti_replayconf.memcached.addr, loop,
+        tls_anti_replay_memcached_ssl_ctx, anti_replayconf.memcached.host,
+        &mcpool_, randgen_);
+  }
+
   replace_downstream_config(std::move(downstreamconf));
 }
 
@@ -176,8 +192,6 @@ void Worker::replace_downstream_config(
   downstream_addr_groups_ =
       std::vector<std::shared_ptr<DownstreamAddrGroup>>(groups.size());
 
-  std::map<DownstreamKey, size_t> addr_groups_indexer;
-
   for (size_t i = 0; i < groups.size(); ++i) {
     auto &src = groups[i];
     auto &dst = downstream_addr_groups_[i];
@@ -221,7 +235,6 @@ void Worker::replace_downstream_config(
       dst_addr.fall = src_addr.fall;
       dst_addr.rise = src_addr.rise;
       dst_addr.dns = src_addr.dns;
-      dst_addr.upgrade_scheme = src_addr.upgrade_scheme;
 
       auto shared_addr_ptr = shared_addr.get();
 
@@ -265,11 +278,14 @@ void Worker::replace_downstream_config(
 
     // share the connection if patterns have the same set of backend
     // addresses.
+    auto end = std::begin(downstream_addr_groups_) + i;
+    auto it = std::find_if(
+        std::begin(downstream_addr_groups_), end,
+        [&shared_addr](const std::shared_ptr<DownstreamAddrGroup> &group) {
+          return match_shared_downstream_addr(group->shared_addr, shared_addr);
+        });
 
-    auto dkey = create_downstream_key(shared_addr);
-    auto it = addr_groups_indexer.find(dkey);
-
-    if (it == std::end(addr_groups_indexer)) {
+    if (it == end) {
       if (LOG_ENABLED(INFO)) {
         LOG(INFO) << "number of http/1.1 backend: " << num_http1
                   << ", number of h2 backend: " << num_http2;
@@ -285,15 +301,12 @@ void Worker::replace_downstream_config(
       }
 
       dst->shared_addr = shared_addr;
-
-      addr_groups_indexer.emplace(std::move(dkey), i);
     } else {
-      auto &g = *(std::begin(downstream_addr_groups_) + (*it).second);
       if (LOG_ENABLED(INFO)) {
         LOG(INFO) << dst->pattern << " shares the same backend group with "
-                  << g->pattern;
+                  << (*it)->pattern;
       }
-      dst->shared_addr = g->shared_addr;
+      dst->shared_addr = (*it)->shared_addr;
     }
   }
 }
@@ -471,6 +484,10 @@ MemcachedDispatcher *Worker::get_session_cache_memcached_dispatcher() {
   return session_cache_memcached_dispatcher_.get();
 }
 
+MemcachedDispatcher *Worker::get_anti_replay_memcached_dispatcher() const {
+  return anti_replay_memcached_dispatcher_.get();
+}
+
 std::mt19937 &Worker::get_randgen() { return randgen_; }
 
 #ifdef HAVE_MRUBY

src/shrpx_worker.h

@@ -115,10 +115,6 @@ struct DownstreamAddr {
   bool tls;
   // true if dynamic DNS is enabled
   bool dns;
-  // true if :scheme pseudo header field should be upgraded to secure
-  // variant (e.g., "https") when forwarding request to a backend
-  // connected by TLS connection.
-  bool upgrade_scheme;
 };
 
 // Simplified weighted fair queuing.  Actually we don't use queue here
@@ -225,6 +221,7 @@ class Worker {
 public:
   Worker(struct ev_loop *loop, SSL_CTX *sv_ssl_ctx, SSL_CTX *cl_ssl_ctx,
          SSL_CTX *tls_session_cache_memcached_ssl_ctx,
+         SSL_CTX *tls_anti_replay_memcached_ssl_ctx,
          tls::CertLookupTree *cert_tree,
          const std::shared_ptr<TicketKeys> &ticket_keys,
          ConnectionHandler *conn_handler,
@@ -254,6 +251,7 @@ public:
   void schedule_clear_mcpool();
 
   MemcachedDispatcher *get_session_cache_memcached_dispatcher();
+  MemcachedDispatcher *get_anti_replay_memcached_dispatcher() const;
 
   std::mt19937 &get_randgen();
 
@@ -293,6 +291,7 @@ private:
 
   std::shared_ptr<DownstreamConfig> downstreamconf_;
   std::unique_ptr<MemcachedDispatcher> session_cache_memcached_dispatcher_;
+  std::unique_ptr<MemcachedDispatcher> anti_replay_memcached_dispatcher_;
 #ifdef HAVE_MRUBY
   std::unique_ptr<mruby::MRubyContext> mruby_ctx_;
 #endif // HAVE_MRUBY